DataEng

TL;DR This post will deep dive in the buidling of an end to end data engineering project ⚙️ . The idea will be to retrieve a price list of gas stations in France ⛽, create a job to extract it every day 📅 and craft a dashboard to expose those price to logged user 📊 After reading this blog post, you’ll have fundamentals on how to build data dashboard and scrap your own data sources 🚀...

TL;DR This post will expose you how to convert in a very convenient and fast way 🚀 some Apache Parquet files to CSV, and vice-versa, using either DuckDB 🦆 or Pandas 🐍 for a baseline comparison As a quick bonus, we will embedded this tool in a small convient CLI script, easily triggered from your favorite shell 👨‍💻 Let’s go ! Intro Recently, I’ve been working a little bit more on Data Engineering tasks (setup a Datalake, convert data, design pipelines, make cleanup of some data)....