AWS Data Lake Design

This module introduces the fundamental concepts of data lakes, covering their key characteristics, components, and benefits. Students will learn why organizations implement data lakes and how they differ from traditional data warehouses. The module provides an overview of common data lake architectures and use cases, establishing a foundation for designing secure and scalable data lake solutions. By the end, students will understand the value proposition of data lakes and their role in modern data infrastructure.

This module explores the AWS services specifically designed for building and managing data lakes. Students will learn about core storage solutions like Amazon S3, cataloging tools such as AWS Glue Data Catalog, and various data movement services including Kinesis and API Gateway. The module also covers data processing services, analytics capabilities, and machine learning tools that integrate with AWS data lakes. Special attention is given to AWS Lake Formation, a service that simplifies data lake setup and management. Practical lab exercises will help students create their first simple data lake.

This module focuses on data ingestion strategies and techniques for AWS data lakes. Students will learn how to select appropriate tools for different data ingestion scenarios, understand when to process data based on its structure, and implement both streaming ingestion using Amazon Kinesis and batch ingestion with AWS Transfer Family. The module emphasizes the importance of data cataloging and demonstrates the use of Glue Crawlers to automatically discover and catalog data assets. Through practical labs, students will gain hands-on experience ingesting web logs into a data lake environment.

This module covers the essential aspects of processing, analyzing, and visualizing data within an AWS data lake. Students will learn about data preparation using AWS Glue jobs, file optimization techniques for improved performance, and querying data with Amazon Athena. The module also addresses critical data lake security considerations and introduces data visualization concepts with Amazon QuickSight. Through demonstrations and hands-on labs, students will create an end-to-end data lake solution that integrates various AWS services, gaining practical experience with real-world data sets like NYC Taxi data.