The concept of data lakes has been around for a long time, but being able to set up one of these systems, which store vast amounts of raw data in its native formats, was never easy. AWS wants to change this with the launch of AWS Lake Formation. At its core, this new service, which is available today, allows developers to create a secure data lake within a few days.
While “a few days” may still sound like a long time in this age of instant gratification, it’s nothing in the world of enterprise software.
“Everybody is excited about data lakes,” said AWS CEO Andy Jassy in today’s AWS re:Invent keynote. “People realize that there is significant value in moving all that disparate data that lives in your company in different silos and make it much easier by consolidating it in a data lake.”
Setting up a data lake today means you have to, among other things, configure your storage and (on AWS) S3 buckets, move your data, add metadata and add that to a catalog. And then you have to clean up that data and set up the right security policies for the data lake. “This is a lot of work and for most companies, it takes them several months to set up a data lake. It’s frustrating,” said Jassy.
Lake Formation is meant to handle all of these complications with just a few clicks. It sets up the right tags and cleans up and dedupes the data automatically. And it provides admins with a list of security policies to help secure that data.
“This is a step-level change for how easy it is to set up data lakes,” said Jassy.
