[core] Feature/env vars settings

[asaiacai]

adds env vars for configuration of mlop at runtime

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)

[gemini-code-assist]

Summary of Changes

Hello @asaiacai, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the configurability and debuggability of mlop by enabling runtime adjustments via environment variables and implementing a robust failure logging system. The introduction of MLOP_DEBUG_LEVEL allows users to easily control the application's logging verbosity, while the new failed request logging mechanism provides detailed insights into network communication issues, which will be invaluable for troubleshooting and monitoring.

Highlights

• Environment Variable Configuration: Introduced the ability to configure mlop at runtime using environment variables, specifically MLOP_DEBUG_LEVEL for controlling logging verbosity.
• Failed Request Logging: Added a new feature to log detailed information about failed requests to a file (failed_requests.log) when the system is operating in DEBUG mode.
• Error Information Capture: Enhanced the request retry mechanism to capture and store specific error details (HTTP status codes, response text, or exception messages) for inclusion in the failure logs.
• Comprehensive Testing: Included new unit tests to thoroughly validate the parsing, precedence, and default behavior of the MLOP_DEBUG_LEVEL environment variable.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new failure logging mechanism in mlop/iface.py to record details of failed requests to a file, conditional on the debug log level. This includes adding _last_error_info and _lock_failure_log instance variables, a _log_failed_request method, and modifications to the _try method to capture and log error information upon retry exhaustion or unsuccessful responses. Concurrently, mlop/sets.py is updated to allow configuring the debug log level via the MLOP_DEBUG_LEVEL environment variable, with new tests in tests/test_env_vars.py validating this environment variable parsing. Review comments primarily raise concerns about a race condition in mlop/iface.py where the shared _last_error_info attribute could be overwritten by concurrent threads, leading to incorrect error logging, and suggest moving json and datetime imports to the top of the file.

I am having trouble creating individual review comments. Click here to see my feedback.

mlop/iface.py (100-101)

The _last_error_info attribute is a class member, which means it's shared across all instances of ServerInterface and, more importantly, across all threads that might be calling _try concurrently. This can lead to a race condition where one thread's error information overwrites another's before it can be logged, resulting in incorrect error reporting. Consider making _last_error_info local to the _try method or passing it explicitly through the call stack, or using a thread-local storage mechanism if the error needs to persist across retries within a single logical operation.

mlop/iface.py (409)

This line uses self._last_error_info which is a class member. As noted previously, this introduces a race condition. If multiple threads are making requests, the _last_error_info could be overwritten by another thread's error before this _log_failed_request call is made, leading to incorrect error information being logged. The error information should be local to the current request attempt or passed explicitly.

mlop/iface.py (421)

Storing r.text[:100] in _last_error_info is subject to the same race condition as mentioned earlier. If another thread encounters an error and updates _last_error_info before this thread's _log_failed_request is called, the wrong error message will be logged. Each _try invocation needs its own mechanism to store and retrieve error details.

mlop/iface.py (440)

Similar to the HTTP error case, assigning to self._last_error_info here creates a race condition. The exception e is specific to the current _try call, but storing it in a shared class member means it can be overwritten by concurrent operations. This could lead to _log_failed_request logging an error that didn't pertain to the specific failed request it's trying to log.

mlop/iface.py (370-371)

It's generally a best practice to place all import statements at the top of the file, outside of functions. Importing json and datetime inside _log_failed_request means they are re-imported every time the function is called, which can introduce a minor performance overhead and is less conventional. Moving these to the top of mlop/iface.py would improve maintainability and consistency.

import json
from datetime import datetime

[asaiacai]

Summary

This PR adds comprehensive environment variable support for configuring mlop at runtime, including logging levels, failed request tracking, and custom server URLs.

Features

1. 🔍 MLOP_DEBUG_LEVEL Environment Variable

Configure logging verbosity via environment variable without code changes.

Supported values (case-insensitive):

• String: DEBUG, INFO, WARNING, ERROR, CRITICAL
• Numeric: 10, 20, 30, 40, 50

Usage:
export MLOP_DEBUG_LEVEL=DEBUG
python train.py

Configuration precedence: Function params > env var > default (16)

2. 📝 Failed Request Tracking

Automatically logs failed HTTP requests for debugging connectivity issues.

Features:

• Logs to .mlop/{project}/{run}/failed_requests.log
• NDJSON format for easy parsing
• Only active when x_log_level <= DEBUG (zero overhead otherwise)
• Thread-safe implementation
• Captures: timestamp, request type, URL, payload info, error details, retry count

Example log entry:
{"timestamp": "2026-01-06T22:30:45.123456+00:00", "request_type": "num", "url": "https://trakkur-ingest.trainy.ai/ingest/metrics", "payload_info": "42 items", "error_info": "HTTP 500: Internal Server Error", "retries_attempted": 4}

3. 🌐 URL Configuration Environment Variables

Point mlop to custom or self-hosted server endpoints.

Environment variables:

• MLOP_URL_APP - Web application URL (default: https://trakkur.trainy.ai)
• MLOP_URL_API - API server URL (default: https://trakkur-api.trainy.ai)
• MLOP_URL_INGEST - Metrics ingestion URL (default: https://trakkur-ingest.trainy.ai)
• MLOP_URL_PY - Python API URL (default: https://trakkur-py.trainy.ai)

Usage:
export MLOP_URL_APP=http://localhost:3000
export MLOP_URL_API=http://localhost:3001
export MLOP_URL_INGEST=http://localhost:3003
export MLOP_URL_PY=http://localhost:3004
python train.py

Configuration precedence: Function params > env vars > defaults

[asaiacai]

TODO: add documentation around settings and environment variable configuration

[asaiacai]

^ @ryanhayame