Search for a command to run...
Articles tagged with #data-engineering-projects
Business Overview Snowflake's Data Cloud is based on a cutting-edge data platform delivered as a service(SaaS). Snowflake provides data storage, processing, and analytic solutions that arequicker, easier to use, and more versatile than traditional op...
This project implements a scalable data pipeline to extract, transform, and load real estate data from Redfin into Snowflake using AWS services. The data is later visualized in Power BI to provide insights into real estate trends. Overview The pipeli...
Project: Slowly Changing Dimensions in Snowflake Using Streams and Tasks Introduction This project implements a real-time data pipeline for continuous data ingestion and transformation into a Snowflake data warehouse. It leverages various cloud techn...
Project Overview This project demonstrates a real-time data pipeline for stock market data. It integrates Apache Kafka for data streaming and multiple AWS services for data storage and querying. Key Features: Real-time data ingestion using Kafka Pro...
This project demonstrates an ETL pipeline built with Apache Airflow on an AWS EC2 instance. The pipeline pulls data from the OpenWeather API and Amazon S3, performs transformations, loads the data into an RDS PostgreSQL database, joins the datasets, ...
This project demonstrates an event-driven architecture using AWS services to build a real-time data processing pipeline. The architecture is designed to process events generated in an S3 bucket and uses services like SNS, SQS, and Lambda for reliable...
