This project demonstrates an event-driven architecture using AWS services to build a real-time data processing pipeline. The architecture is designed to process events generated in an S3 bucket and uses services like SNS, SQS, and Lambda for reliable...
DataFlow Insights is an automated data pipeline project designed to streamline data ingestion, processing, and visualization using AWS services. This project pushes daily data to Amazon S3, automatically crawls and catalogs it using AWS Glue, queries...
This project demonstrates a serverless data lake architecture using various AWS services. The architecture is designed to ingest, process, and analyze CSV files stored in an Amazon S3 bucket. AWS Lambda functions, Glue Crawlers, Glue Jobs, CloudWatch...
This project demonstrates an ETL (Extract, Transform, Load) pipeline that fetches Spotify data using their API and processes it using AWS services like Lambda, S3, Glue, and Athena. The pipeline is designed to extract raw data from Spotify, transform...
End-To-End ETL Solution with AWS Glue, Athena and Redshift
