Skip to content

Refactor EhrSqlAnnotator for improved error handling and query management#7

Merged
chakravarthik27 merged 2 commits into
mainfrom
improve-the-ehrsql-scenario-in-the-med-helm
Apr 20, 2026
Merged

Refactor EhrSqlAnnotator for improved error handling and query management#7
chakravarthik27 merged 2 commits into
mainfrom
improve-the-ehrsql-scenario-in-the-med-helm

Conversation

@chakravarthik27
Copy link
Copy Markdown

This pull request updates the EHR SQL annotation and scenario components to improve SQL evaluation consistency, clarify instructions for SQL generation, and update data sources. The most significant changes ensure that SQL queries using current time are handled deterministically, update dataset and database URLs, and refine the logic for processing input data.

SQL Evaluation Improvements:

  • Modified the annotation logic in ehr_sql_annotator.py to replace any usage of current_time in SQL queries with the fixed timestamp '2105-12-31 23:59:00', ensuring deterministic evaluation results. Also updated the expected result types from List[str] to List[Tuple[Any, ...]] for more accurate result handling. [1] [2]

Prompt and Adapter Specification Updates:

  • Revised the SQL generation instructions in medhelm_run_specs.py to require the use of the fixed timestamp for current time, enforce stricter output formatting, and clarify that only a single SQL statement should be returned.

Dataset and Database Source Updates:

  • Updated URLs in ehr_sql_scenario.py to point to the latest dataset, schema, and database sources, ensuring the scenario uses current and accessible resources.

Data Processing Logic:

  • Improved input processing in ehr_sql_scenario.py to skip entries where the query is "null", preventing invalid data from being included in the scenario.

Minor Metadata Correction:

  • Fixed a formatting issue in the model_metadata.yaml file for the upstage/solar-pro-241126 model's display name.

References:

Copy link
Copy Markdown

@blidiselalin blidiselalin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@chakravarthik27 chakravarthik27 merged commit 152d6b1 into main Apr 20, 2026
6 checks passed
@chakravarthik27 chakravarthik27 deleted the improve-the-ehrsql-scenario-in-the-med-helm branch April 20, 2026 07:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants