1 Release published by 1 person

• v0.1.2-alpha

  published Mar 7, 2025

26 Pull requests merged by 8 people

• Small fixes for jobs - reinstated

  #1094 merged Mar 10, 2025

• Only run integration tests on lumigator code changes

  #1144 merged Mar 10, 2025

• Update DatasetViewer.vue

  #1151 merged Mar 7, 2025

• feat: filter models by selected usecase on create experiment

  #1141 merged Mar 7, 2025

• Settings: Supply secret key name via workflows and jobs

  #1128 merged Mar 7, 2025

• Refactor HuggingFaceModelClient to modular components

  #1134 merged Mar 6, 2025

• Settings: Support settings via UI

  #1116 merged Mar 6, 2025

• Remove accidental log statement

  #1139 merged Mar 6, 2025

• [BUG] Mlflow data was not being persisted on container recreation

  #1137 merged Mar 6, 2025

• Show translation models in models endpoint

  #1113 merged Mar 6, 2025

• feat: support translation experiments from the ui

  #1035 merged Mar 6, 2025

• Evaluating different flavors of DeepSeek R1.

  #1091 merged Mar 5, 2025

• Use key from the secrets table in the env var list in Ray jobs

  #1090 merged Mar 5, 2025

• Fix mistralAPIKey in Helm chart

  #1126 merged Mar 5, 2025

• Fix/adding mlflow to helm chart

  #1111 merged Mar 5, 2025

• Fix generated dataset filenames

  #1119 merged Mar 5, 2025

• Update docs to use task_definition instead of task

  #1095 merged Mar 5, 2025

• Fixing existingOpenaiAPISecret

  #1114 merged Mar 4, 2025

• Settings: Fix redaction bug in parsing model name/path

  #1115 merged Mar 4, 2025

• Update to v0.1.2-alpha

  #1103 merged Mar 4, 2025

• Organize sample_data by task

  #1102 merged Mar 4, 2025

• [BUG] Not catching a missing tracking_uri

  #1100 merged Mar 4, 2025

• WorkflowResponse to have system_prompt as field

  #1098 merged Mar 4, 2025

• Settings: Allow deletion of a user created secret

  #1108 merged Mar 4, 2025

• Fix DB default location

  #1107 merged Mar 4, 2025

• Use model cache profile for local-down and stop-lumigator

  #1106 merged Mar 3, 2025

18 Pull requests opened by 8 people

• Generate openapi offline

  #1112 opened Mar 4, 2025

• Deploy a Model with vLLM and Ray Serve

  #1120 opened Mar 5, 2025

• Notebook to Demo Translation Usecase via SDK

  #1121 opened Mar 5, 2025

• feat: toggle columns in DatasetViewer table

  #1132 opened Mar 6, 2025

• Move logging back into workflows service

  #1138 opened Mar 6, 2025

• Expandable rows

  #1140 opened Mar 6, 2025

• Explicitly use api keys in inference and evaluation

  #1145 opened Mar 7, 2025

• Support batch inference

  #1146 opened Mar 7, 2025

• Support for HF seq2seq multilingual models for translation: mt5-based model architectures with prefix

  #1147 opened Mar 7, 2025

• Jobs: Inference - refactoring and increasing test coverage

  #1153 opened Mar 8, 2025

• Job: Redaction - Add case for redaction where the value is an object

  #1154 opened Mar 10, 2025

• Update `Makefile`: Add targets and test schemas

  #1155 opened Mar 10, 2025

• Config: Revert `DATABASE_URL` name to `SQLALCHEMY_DATABASE_URL`

  #1157 opened Mar 10, 2025

• Bump axios from 1.7.9 to 1.8.2 in /lumigator/frontend

  #1158 opened Mar 10, 2025

• GitHub GPU runners integration tests

  #1159 opened Mar 10, 2025

• chore: setup frontend pre commit hook

  #1161 opened Mar 10, 2025

• Testing: Update `Makefile` (backend) tests to allow a custom `user.conf` to be provided

  #1162 opened Mar 10, 2025

• Try changing helm chart

  #1164 opened Mar 10, 2025

21 Issues closed by 8 people

• [FEATURE]: Filter available models by selected usecase on create experiment

  #1142 closed Mar 7, 2025

• Refactor HuggingFaceModelClient to support future use cases

  #1130 closed Mar 6, 2025

• [FEATURE]: UI: supports settings endpoint

  #929 closed Mar 6, 2025

• [BUG]: Running `make stop-lumigator` deletes all your mlflow data

  #1135 closed Mar 6, 2025

• List suggested translation models in models/ endpoint

  #967 closed Mar 6, 2025

• [FEATURE] Support translations usecase in FE

  #999 closed Mar 6, 2025

• [BUG]: Datasets and annotation jobs are not persisted on restart

  #928 closed Mar 6, 2025

• Pin point the boto version we use

  #1003 closed Mar 6, 2025

• [FEATURE]: Report the output length of prediction when running inference job

  #1080 closed Mar 5, 2025

• [FEATURE]: Dynamic list of mean metric scores

  #1083 closed Mar 5, 2025

• [FEATURE]: Workflows accept a list of eval metrics as a parameter

  #1082 closed Mar 5, 2025

• Use Lumigator to run comparison of DeepSeek R-1 models with ACI-Bench Doctor-Patient summarization

  #811 closed Mar 5, 2025

• [FEATURE]: Jobs: update creation to include API keys when required

  #930 closed Mar 5, 2025

• [BUG]: mistralAPIKey value doesn't work in Helm Chart

  #1117 closed Mar 5, 2025

• [BUG]: A dataset with human GT can result as "AI generated" after predictions are run on it

  #897 closed Mar 5, 2025

• Vibe-check evaluation using the current g-eval code

  #1011 closed Mar 5, 2025

• Update UI designs to accomodate Translation Use-case

  #979 closed Mar 4, 2025

• Organize sample_data by task: translation/ and summarization/

  #965 closed Mar 4, 2025

• WorkflowResponse to have system_prompt as field

  #1097 closed Mar 4, 2025

• [FEATURE]: API: Settings - allow deletion of a configured secret

  #1101 closed Mar 4, 2025

• [BUG]: Persistence lost for SQLite

  #1077 closed Mar 4, 2025

17 Issues opened by 6 people

• Add crash report tracking to the BE

  #1163 opened Mar 10, 2025

• [BUG]: Completed Ground Truth Jobs not shown on UI

  #1160 opened Mar 10, 2025

• Bulk delete datasets and experiments from the UI

  #1156 opened Mar 10, 2025

• Add crash report tracking to the UI

  #1150 opened Mar 7, 2025

• Add a cache layer for our CI

  #1149 opened Mar 7, 2025

• [BUG]: Run integration tests over containers

  #1148 opened Mar 7, 2025

• [FEATURE]: Filter models by the selected usecase when creating experiments

  #1143 opened Mar 6, 2025

• Warning in Ray Inference Logs: Expected `AverageInferenceMetrics` but got `dict` with value `{}` - serialized value may not be as expected

  #1136 opened Mar 6, 2025

• [BUG]: Migration scripts when adding content to mlflow schemas

  #1131 opened Mar 6, 2025

• [BUG]: Ray takes more than two minutes to start after `make local-up`

  #1129 opened Mar 5, 2025

• Change behaviour of models/ endpoint to support Multilingual HF models for translation usecase

  #1127 opened Mar 5, 2025

• [BUG]: User can accidentally "double-trigger" an experiment

  #1125 opened Mar 5, 2025

• Configuration of timeouts in jobs

  #1124 opened Mar 5, 2025

• Document default generation config parameters in our documentation

  #1123 opened Mar 5, 2025

• Move inference_time into inference job metrics.

  #1122 opened Mar 5, 2025

• [BUG]: Silently failing MlFlow Connection

  #1110 opened Mar 4, 2025

• Include max_samples in WorkflowResponse

  #1109 opened Mar 4, 2025

15 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• ci: setup github actions for testing, code formatting & linting the frontend

  #941 commented on Mar 10, 2025 • 5 new comments

• Generate openapi specs without a running server

  #1016 commented on Mar 4, 2025 • 0 new comments

• [FEATURE]: Docker Compose: Remove API key env vars for AI services

  #932 commented on Mar 4, 2025 • 0 new comments

• User guide: creating a Dataset for Lumigator evaluations (summarization)

  #834 commented on Mar 5, 2025 • 0 new comments

• Support Batch Inference in Job

  #955 commented on Mar 5, 2025 • 0 new comments

• [BUG]: Should not be able to specify max_samples or task_definition via workflow

  #1104 commented on Mar 5, 2025 • 0 new comments

• [BUG]: sqlite database is not persisted (in dev environment at least)

  #515 commented on Mar 5, 2025 • 0 new comments

• [FEATURE]: SDK supports settings endpoint

  #927 commented on Mar 5, 2025 • 0 new comments

• [FEATURE]: Jobs: Move from implicit to explicit API key usage in clients

  #931 commented on Mar 5, 2025 • 0 new comments

• [BUG]: Logs in the UI do not appear anymore when job fails

  #908 commented on Mar 6, 2025 • 0 new comments

• Launch github actions in a custom runner with GPU

  #864 commented on Mar 7, 2025 • 0 new comments

• Support for HF seq2seq models for translation

  #976 commented on Mar 10, 2025 • 0 new comments

• [BUG]: Error downloading experiment results

  #1073 commented on Mar 10, 2025 • 0 new comments

• Test self hosted runners

  #914 commented on Mar 7, 2025 • 0 new comments

• Added the script for OSX and Linux

  #1084 commented on Mar 6, 2025 • 0 new comments