Skip to content

[AUTO-MERGE] Updating the MLFlow backend from file to sqlite-based.#781

Merged
drewoldag merged 6 commits intomainfrom
issue/780/mlflow-sqlite-backend
Mar 16, 2026
Merged

[AUTO-MERGE] Updating the MLFlow backend from file to sqlite-based.#781
drewoldag merged 6 commits intomainfrom
issue/780/mlflow-sqlite-backend

Conversation

@drewoldag
Copy link
Copy Markdown
Collaborator

Change Description

Closes #780

Solution Description

Updating the default location for MLflow metrics to be tracked. Using SQLite database now instead of file-backed storage.

See this migration guide: https://mlflow.org/docs/latest/self-hosting/migrate-from-file-store

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates Hyrax’s MLflow tracking configuration to avoid the deprecated filesystem backend by switching the tracking URI to a SQLite database under the configured results root directory.

Changes:

  • Switch MLflow tracking URI in train to sqlite://.../mlflow/mlflow.db.
  • Switch MLflow tracking URI in test to sqlite://.../mlflow/mlflow.db.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/hyrax/verbs/train.py Points MLflow tracking to a SQLite DB under the results root.
src/hyrax/verbs/test.py Points MLflow tracking to the same SQLite DB under the results root.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/hyrax/verbs/train.py Outdated
Comment thread src/hyrax/verbs/test.py Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.73%. Comparing base (6b45631) to head (2e72f5a).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #781      +/-   ##
==========================================
+ Coverage   64.71%   64.73%   +0.01%     
==========================================
  Files          61       61              
  Lines        5887     5890       +3     
==========================================
+ Hits         3810     3813       +3     
  Misses       2077     2077              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@drewoldag
Copy link
Copy Markdown
Collaborator Author

@copilot make the changes that are mentioned in the two comments, and be sure that the change is consistent in both train.py and test.py. Eventually we would like to abstract that logic out, but not in this PR.

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 13, 2026

@drewoldag I've opened a new pull request, #782, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan

* Create mlflow directory before setting SQLite tracking URI in train and test

Co-authored-by: drewoldag <47493171+drewoldag@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: drewoldag <47493171+drewoldag@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 13, 2026

Before [6b45631] After [b9dc52e] Ratio Benchmark (Parameter)
failed failed n/a data_cache_benchmarks.DataCacheBenchmarks.time_preload_cache_hsc1k
failed failed n/a data_cache_benchmarks.DataCacheBenchmarks.track_cache_hsc1k_hyrax_size_undercount
failed failed n/a data_request_benchmarks.DatasetRequestBenchmarks.time_request_all_data
38.9±0.5ms 40.0±0.4ms 1.03 benchmarks.time_nb_obj_dir
1.91±0.02s 1.94±0.01s 1.01 benchmarks.time_database_connection_help
1.92±0.01s 1.93±0.01s 1.01 benchmarks.time_infer_help
1.92±0s 1.94±0.01s 1.01 benchmarks.time_prepare_help
1.93±0.02s 1.94±0.02s 1.01 benchmarks.time_rebuild_manifest_help
1.93±0.01s 1.94±0.01s 1.01 benchmarks.time_visualize_help
5.84±0s 5.90±0s 1.01 vector_db_benchmarks.VectorDBInsertBenchmarks.time_load_vector_db(2048, 'chromadb')

Click here to view all benchmarks.

Copy link
Copy Markdown
Collaborator

@mtauraso mtauraso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@drewoldag drewoldag enabled auto-merge (squash) March 16, 2026 23:07
@drewoldag drewoldag changed the title Updating the MLFlow backend from file to sqlite-based. [AUTO-MERGE] Updating the MLFlow backend from file to sqlite-based. Mar 16, 2026
@drewoldag drewoldag merged commit cf9737c into main Mar 16, 2026
6 checks passed
@drewoldag drewoldag deleted the issue/780/mlflow-sqlite-backend branch March 16, 2026 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Address the MLFlow warning "filesystem backend is deprecated"

4 participants