Code and results for assesing energy efficiency of machine learning tasks on popular benchmark data sets. The associated research paper was originally presented at the AI for Energy Innovation Workshop at AAAI '23 and later extended for Meta-learning from Model Quality and Resource Consumption, as published at ECML-PKDD '24.
All code was executed with Python 3.8, please refer to requirements for all dependencies. Depending on how you intend to use this software, only some packages are required.
To investigate the results you can use our publicly available Energy Label Exploration tool, so no code needs to be run on your machine.
To start ELEx locally, simply call python elex.py and open the given URL in any webbrowser.
Call python -m mlee.label_generator to generate an energy label either for a given data set / task / method / environment, or any of the merged logs (provided via command line).
The results (plots and tables) in the paper were generated with the corresponding script.
New experiments can also be executed, simply run the evaluation script.
You can pass the chosen method, software backend and more configuration options via command line.
For each experiment a folder is created, which can be merged into more compact .json format.
Hyperparameters for all (benchmark X method) combinations are can be found in this repo and were identified by a random search.
Note that due to monitoring of power draw, we mainly tested on limited hardware architectures and systems (Linux systems with Intel CPUs).
We already investigated the efficiency of ImageNet models.
We intend to extend and improve our software framework:
- polish the ELEx tool, allow to execute expeirments locally from GUI
- support more implementations, monitoring options, models, metrics, and tasks
- move beyond sustainability and incorporate other important aspects of trustworthiness
- more improvements based on reviewer feedback
If you appreciate our work and code, please cite our papers:
Fischer, R., Jakobs, M., Morik, K. (2023). Energy Efficiency Considerations for Popular AI Benchmarks. In: AI for Energy Innovation Workshop at the 37th AAAI Conference on Artificial Intelligence. https://doi.org/10.48550/arXiv.2304.08359.
Fischer, R., Wever, M., Buschjäger, S., Liebig, T. (2024). MetaQuRe: Meta-learning from Model Quality and Resource Consumption. In: Bifet, A., Davis, J., Krilavičius, T., Kull, M., Ntoutsi, E., Žliobaitė, I. (eds) Machine Learning and Knowledge Discovery in Databases. Research Track. ECML PKDD 2024. Lecture Notes in Computer Science(), vol 14947. Springer, Cham. https://doi.org/10.1007/978-3-031-70368-3_13
or using the bibkeys below:
@inproceedings{fischer_energy_2023, address = {Washington DC}, title = {Energy {Efficiency} {Considerations} for {Popular} {AI} {Benchmarks}}, copyright = {All rights reserved}, url = {https://arxiv.org/abs/2304.08359}, doi = {10.48550/arXiv.2304.08359}, booktitle = {{AI} for {Energy} {Innovation} {Workshop} at the 37th {AAAI} {Conference} on {Artificial} {Intelligence}}, author = {Fischer, Raphael and Jakobs, Matthias and Morik, Katharina}, year = {2023}, }
@InProceedings{10.1007/978-3-031-70368-3_13, author="Fischer, Raphael and Wever, Marcel and Buschj{"a}ger, Sebastian and Liebig, Thomas", editor="Bifet, Albert and Davis, Jesse and Krilavi{\v{c}}ius, Tomas and Kull, Meelis and Ntoutsi, Eirini and {\v{Z}}liobait{.{e}}, Indr{.{e}}", title="{MetaQuRe}: Meta-learning from Model Quality and Resource Consumption", booktitle="Machine Learning and Knowledge Discovery in Databases. Research Track", year="2024", publisher="Springer Nature Switzerland", address="Cham", pages="209--226", isbn="978-3-031-70368-3" }
Copyright (c) 2026 Raphael Fischer