Analyzing spendings with Python & Pandas,
categorizing banking transactions with Machine Learning
A finance report is generated as a Jupyter notebook:
https://github.com/gerritnowald/budget_book/blob/main/analysis.ipynb
See also this blog post:
https://gerritnowald.wordpress.com/2023/02/23/managing-spending-with-python-pandas/
I started this personal project as the German comdirect bank abolished their online budget planer Finzanzmanager. It aims to reproduce this functionality while being as simple and understandable as possible. In principle this project is applicable to every bank which allows to export transactions as csv files. For the comdirect bank, also an API access is available, which I adopted from Philipp Panhey (see below). I decided to publish this project, since I thought that it might be useful to others. However, this is not a professional and easy to use budget planer and requires some programming knowledge and adaptation to cater to your individual needs.
Initially, transactions are exported from online banking as a csv file.
These form the database, stored locally on your hard drive.
The minimal required columns are
- date
- amount
- description
General settings such as file names & column names have to be set in config.ini.
The account balance over time has to calculated once using calculate_balance.py
(the final balance has to be given in config.ini).
The transactions have to be categorized for detailed analysis.
Two layers of categories can be used, separated with / (e.g. living expenses/groceries).
This has to be done manually in the beginning, e.g. using Excel.
Later, Machine Learning is used to automatically categorize new transactions.
As an example for transactions, see
https://github.com/gerritnowald/budget_book/blob/main/transactions.csv
(without description text)
For the API import for the German comdirect bank, the user has to register and insert the credentials into config_comdirectAPI.json.
New transactions are appended to the database using import_transactions.ipynb.
They can be imported from a csv file.
Alternatively, an API import is available for the German comdirect bank.
The transactions are categorized based on their description text using Machine Learning, see also this blog post:
https://gerritnowald.wordpress.com/2023/04/05/categorize-banking-transactions-with-machine-learning/
A list of all currently used categories is automatically saved as categories.csv.
Also the balance over time is updated.
Since the accuracy of the categoriziation is not perfect, wrong categories should be corrected.
For this, a command line interface is available, CLI.py, see also this blog post:
https://gerritnowald.wordpress.com/2024/02/26/creating-a-command-line-interface-with-python/
On Windows, it can be started using start_CLI.bat.
The command line interface can also be used to split transactions, e.g. when withdrawing cash at the supermarket.
Alternatively, a text editor or Excel can be used to modify the transaction database.
Then, the finance report can be updated by running analysis.ipynb.
Reports for different time frames can be generated by filtering the database.
It is recommended to export the notebook to e.g. save a yearly report for later reference.
Git can be used as backup.
If an online repo is used, it is strongly recommended to make it private and to exclude config_comdirectAPI.json.
- Pandas
- plotly (for sunburst diagram)
- curses (for command line interface)
Contributions are welcome, especially regarding APIs for additional banks.
thanks to Philipp Panhey for the comdirect API access:
https://github.com/phpanhey/comdirect_financialreport