- Data Engineering Project - Personal Finances with Airflow, Docker, Great Expectations and Metabase December 6, 2022
TLDR: A dockerised Airflow repository to automate API calls to your bank account (only Monzo currently supported) to store transactions in a Postgres database, check data quality with Great Expectations and visualise your data on Metabase. DAGs to pull transactions and store to a database, move money between Monzo Pots for automated budgetting and to create custom push notifications to your Monzo app. About Finding data to use in a project can be difficult and with all the tools to choose from it can become overwhelming.
Read more »- Automating Budget Tracking on Notion - Monzo's API with Airflow May 20, 2022
About Adding expenses to my budgetting spreadsheet at the end of each week was too time consuming so I created a data pipeline to populate my cloud database with transactions made on my Monzo card and push them to a Notion dashboard. I wrote an Airflow pipeline that fetches transactions and Pot balances using Monzo’s API, updates my Postgres database tables on AWS and uses Hightouch to sync the tables with my Notion databases.
Read more »