Skip to content

Data annotator and pvmap generator#1264

Merged
ajaits merged 3 commits intodatacommonsorg:masterfrom
ajaits:sv-autoschema
Apr 16, 2025
Merged

Data annotator and pvmap generator#1264
ajaits merged 3 commits intodatacommonsorg:masterfrom
ajaits:sv-autoschema

Conversation

@ajaits
Copy link
Copy Markdown
Contributor

@ajaits ajaits commented Mar 7, 2025

Utilities for generating pv-map

  • data_sampler.py: utility to down sample large csv files
  • data_annotator.py: Annotate columns as place, date, or schema
  • llm_pvmap_generator.py: Generate PV map from data strings.
  • llm_name_generator.py: Generate statvar names

Copy link
Copy Markdown
Contributor

@keyurva keyurva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I skimmed the PR and am approving it but would be good to do smaller PRs moving forward. One overall comment is that each script here is a runnable script. If it makes sense, consider exposing only one runner script and invoke others as modules as needed. This can simplify the individual modules and make them more testable.

@ajaits ajaits requested a review from rohitkumarbhagat April 15, 2025 11:51
@ajaits ajaits enabled auto-merge (squash) April 16, 2025 12:07
@ajaits
Copy link
Copy Markdown
Contributor Author

ajaits commented Apr 16, 2025

I skimmed the PR and am approving it but would be good to do smaller PRs moving forward. One overall comment is that each script here is a runnable script. If it makes sense, consider exposing only one runner script and invoke others as modules as needed. This can simplify the individual modules and make them more testable.

Thanks Keyur.

While each script is can be run separately, all these modules are invoked by the stat_var_processor.py and don't need to be invoked individually.

@ajaits ajaits merged commit 49bdfce into datacommonsorg:master Apr 16, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants