Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch hitran update #505

Merged
merged 12 commits into from Aug 17, 2022
Merged

Conversation

Supriya1702
Copy link
Collaborator

This PR addresses-

  • Replacing hit2df with LOCAL_TABLE_CACHE
  • added a parameter extra_params in fetch_hitran() and dbmanager.py
  • downloading all columns when extra_params = 'all'
  • Adding conditions to raise AccuracyWarning when All columns are not downloaded currently.
  • cleaning of NaN value columns after downloading all columns in hdf5.py

Copy link
Member

@erwanp erwanp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many suggestions above.

Also tests are failing in all nonequilibrium calculations. You can test locally by computing a Spectrum with Tvib different from Trot

radis/io/hitran.py Outdated Show resolved Hide resolved
radis/io/hitran.py Show resolved Hide resolved
radis/io/hitran.py Show resolved Hide resolved
radis/io/hitran.py Outdated Show resolved Hide resolved
radis/io/hitran.py Outdated Show resolved Hide resolved
@erwanp erwanp mentioned this pull request Aug 15, 2022
84 tasks
radis/io/hitran.py Outdated Show resolved Hide resolved
Copy link
Member

@erwanp erwanp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comments

radis/io/hitran.py Outdated Show resolved Hide resolved
radis/io/hitran.py Outdated Show resolved Hide resolved
radis/io/hitran.py Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Aug 16, 2022

Codecov Report

Merging #505 (fca008e) into develop (b853a26) will increase coverage by 0.19%.
The diff coverage is 69.13%.

@@             Coverage Diff             @@
##           develop     #505      +/-   ##
===========================================
+ Coverage    73.10%   73.30%   +0.19%     
===========================================
  Files          137      138       +1     
  Lines        18870    19015     +145     
===========================================
+ Hits         13795    13939     +144     
- Misses        5075     5076       +1     

@erwanp erwanp added this to To Do in GSOC 2022 : Non Air Diluent via automation Aug 16, 2022
@erwanp erwanp added interface not related to the physics of the code db:hitran related to HITRAN database databases related to line databases labels Aug 16, 2022
@erwanp erwanp added this to the 0.13.1 milestone Aug 16, 2022
@Supriya1702
Copy link
Collaborator Author

@erwanp I was thinking of adding test cases to check if we are able to download the new parameters or not. But am not sure how exactly to implement them, because all the current tests that are being run must be downloading the original database, we can add a test by regenerating the database with extra_params = 'all' or how would you like to proceed with it?

GSOC 2022 : Non Air Diluent automation moved this from To Do to In Progress Aug 16, 2022
radis/io/dbmanager.py Show resolved Hide resolved
radis/io/hdf5.py Show resolved Hide resolved
radis/io/hitran.py Outdated Show resolved Hide resolved
@erwanp
Copy link
Member

erwanp commented Aug 16, 2022

@erwanp I was thinking of adding test cases to check if we are able to download the new parameters or not. But am not sure how exactly to implement them, because all the current tests that are being run must be downloading the original database, we can add a test by regenerating the database with extra_params = 'all' or how would you like to proceed with it?

Talking about test cases, i suggested #505 (comment)

Good idea to add the new test cases. We'll need them anyway. We shouldn't load a large database, because it will be downloaded on each CI run. But you can try with CO. Does CO have broadneing parameters by other species ? (CO2, etc. ? )

I'd add a fetch_hitran() test in test_hitran_cdsd similarly to how we already test fetch_hitemp() (using "regen") in test_hitemp.py

radis/io/hitran.py Outdated Show resolved Hide resolved
@anandxkumar
Copy link
Collaborator

Great work @Supriya1702, tested it locally, works fine on my end. The only thing missing will be adding a test to check if the extra_params are loading the extra columns or not. I think CO will be a good example of it as it contains gamma_* and n_* for co2, h2o, he (can be used in future for calculating non-air broadening coeff). Maybe we can set extra_params=True and db_use_cached='regen' to check the same in fetch_databank and then test whether columns are loading for load_columns = equilibrium, non-eq and all?

radis/io/dbmanager.py Show resolved Hide resolved
radis/io/dbmanager.py Outdated Show resolved Hide resolved
radis/io/hitran.py Outdated Show resolved Hide resolved
…columns DataFileManager function in DataBaseManager
Copy link
Member

@erwanp erwanp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

radis/test/io/test_hdf5.py Outdated Show resolved Hide resolved
@erwanp
Copy link
Member

erwanp commented Aug 17, 2022

Well done ; I let you merge when the pipeline succeeds!

@Supriya1702 Supriya1702 merged commit e85cb96 into radis:develop Aug 17, 2022
GSOC 2022 : Non Air Diluent automation moved this from In Progress to Done Aug 17, 2022
@erwanp erwanp mentioned this pull request Aug 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
databases related to line databases db:hitran related to HITRAN database interface not related to the physics of the code
Development

Successfully merging this pull request may close these issues.

None yet

4 participants