#airflow

Articles tagged with #airflow

Building an Incremental Zoho Desk to BigQuery Pipeline: Lessons from the Trenches
What I thought would take a few days ended up taking weeks so here's everything I learned building a production Zoho Desk to BigQuery pipeline from scratch. When the Fintech company I worked with dec
Feb 28, 20269 min read
Building an End-to-End Batch Pipeline for NYC Citi Bike 2024 (DE Zoomcamp Final Project)
Over 1.1 million bike trips. Twelve months of data. One pipeline to process all of it. This post documents how I built an end-to-end batch data pipeline for NYC's Citi Bike 2024 dataset as my final pr
Apr 12, 20256 min read5
How I Built an End-to-End ELT Pipeline with Airflow, dbt, and BigQuery (And What Broke Along the Way)
There is a gap between understanding what a data pipeline is and actually building one that works end to end. Tutorials close part of that gap. Real projects close the rest, but they also open up a se
Mar 27, 20257 min read1