Speed up estimators reading and plotting #119

lukeshingles · 2023-11-10T12:06:13Z

Convert batches of estimator text files to parquet format and use polars.scan_parquet to extract only relevant columns.

Example plot of nne, TR, Sr average ionisation, Sr isotopic abundances, and Sr ion stage populations on a 25x50 model went from 32 seconds to 2 seconds.

…ctions

…ment_

lukeshingles added 30 commits November 8, 2023 10:35

Update inputmodel_misc.py

91fb18c

Update modelfromhydro.py

cdfc4ad

Update estimators.py

48d6262

Update estimators.py

1a88598

Update estimators.py

b757d3d

Flatten estimator dictionary (not nested dicts)

0920c4d

Refactor estimator reading func params

0de4d2c

Fix estimator key errors

0a9ebbc

Update estimators.py

ac83ee0

Remove get_heatingcooling option to parse_estimfile() and calling fun…

0b66719

…ctions

Update radfield.py

7d316fc

Update estimators.py

8d4f715

Update plotestimators.py

c8aca8a

Update estimators.py

0ccdb31

Update estimators.py

7fce659

Update updatepackages.yml

d40372e

Update updatepackages.yml

ab49ea3

Remove astropy from spectra.py

40fa39e

Update plotnonthermal.py

1aff644

Replace estimator keys prefix populations_ with nnion_, nniso_, nnele…

c85588c

…ment_

Update .pre-commit-config.yaml

e3edc4b

Update requirements.txt

59d3057

Update plotestimators.py

2a5a4e0

Remove lz4 support

29fb649

Store estimators in parquet files

e47f0ba

Restructure estimator file reading to lower mem usage

f301db3

Update estimators.py

10dd7e0

Update test_artistools.py

c2e6c8f

Update estimators.py

688db76

Update plotestimators.py

77b0ebe

lukeshingles added 24 commits November 12, 2023 22:18

Add formatted names for cell velocities

b1bcf2b

Update plotestimators.py

4987df1

Add --makegif option to plotestimators

501b611

Add png multiplot

36b9870

Update plotestimators.py

7a9c2bb

Fix estimator output file names

97e1a36

Update plotestimators.py

159eab0

Update plotestimators.py

c74b6f9

Update

b875aaa

Update requirements.txt

027b464

Update plotspherical.py

0ddd273

Update plotspherical.py

fbcd24c

Update plotspherical.py

d43c090

Update plotspherical.py

483bb04

Update plotinitialcomposition.py

0bf7975

Update plotestimators.py

087515f

Update plotestimators.py

53ff452

Update plotestimators.py

91766f2

Fix test failure

2d5e6dd

Update estimators.py

3f5e0b6

Update extensions.json

dd8ab91

Update plotinitialcomposition.py

ce0cd4e

Update plotestimators.py

2730eeb

Merge branch 'main' into speedupestimators

96d5566

lukeshingles enabled auto-merge November 14, 2023 22:47

Update test_spectra.py

79b638b

lukeshingles added this pull request to the merge queue Nov 14, 2023

Merged via the queue into main with commit a34bd50 Nov 14, 2023
18 checks passed

lukeshingles deleted the speedupestimators branch November 14, 2023 23:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Speed up estimators reading and plotting #119

Speed up estimators reading and plotting #119

lukeshingles commented Nov 10, 2023 •

edited

Loading

Speed up estimators reading and plotting #119

Speed up estimators reading and plotting #119

Conversation

lukeshingles commented Nov 10, 2023 • edited Loading

lukeshingles commented Nov 10, 2023 •

edited

Loading