Posts

TL;DR This post will show some tips on how to work efficiently as a Data Engineer 🚀, either navigating throught documentation or using a local LLM to ease your development experience (having a Mac chip will be mandatory for this one). 👨‍💻 Let’s go ! Intro Nowadays, working in limited internet connection can occur and there is a huge gap compared to our workstation setup 🦾 In those situations, connection speed might be very slow, with a very broken bandwith. This makes it very difficult to work in those environments, but with a few preparation you might be as effective than before ! 💥 ...

TL;DR This post will deep dive in the buidling of an end to end data engineering project ⚙️ . The idea will be to retrieve a price list of gas stations in France ⛽, create a job to extract it every day 📅 and craft a dashboard to expose those price to logged user 📊 After reading this blog post, you’ll have fundamentals on how to build data dashboard and scrap your own data sources 🚀 ...

TL;DR This post will leverage and deep dive in the behavior of genetic algorithm, in order to solve an hypothetical fun problem, using solely Python. We will review high level concepts, and apply them to solve an optimization problem. This hypothetical problem takes shape in the narrative of George Orwell and his dystopian novel 1984 Later you will be able to apply Genetic Algorithms to your own optimization problems. Let’s go ! ...

TL;DR This post will expose you how to convert in a very convenient and fast way 🚀 some Apache Parquet files to CSV, and vice-versa, using either DuckDB 🦆 or Pandas 🐍 for a baseline comparison As a quick bonus, we will embedded this tool in a small convient CLI script, easily triggered from your favorite shell 👨‍💻 Let’s go ! Intro Recently, I’ve been working a little bit more on Data Engineering tasks (setup a Datalake, convert data, design pipelines, make cleanup of some data). 📊 ...

TL;DR This blog post will cover how to setup a monitored 📊 & secure personal cloud storage ⏏️, behind a proxy ⚙️, to let you expose other nice services on your home server 🏠 Runaway from Google Drive and Google Photos 📷, host your cherry picked bytes at home 🚀 ! Intro My girlfriend recently ran into a stressfull issue, let me explain briefly the root of the problem 😬 She enjoy, as I do, taking pictures of our trips and daily life. Fortunately, Google has exposed the incredible feature of saving and sharing Photos from mobile in a glimpse, and also make use of shared albums. Let’s say it, it is incredible ! ...