[TEP007] CMFGEN density parser #772

vg3095 · 2017-07-25T10:23:33Z

A parser to read velocity , density, electron_density from a CMFGEN file.
Example -
Notebook
CMFGEN file
Converted CMFGEN file

To resolve conflict of same attribute names in IonNumberDensity

vg3095 · 2017-07-25T10:25:18Z

I have overwrited electron_densities in one case for now, if my approach is correct, then I will expand it to others.

wkerzendorf · 2017-07-25T12:54:19Z

tardis/plasma/properties/ion_population.py

-        n_electron_iterations = 0
-
-        while True:
+        if self._electron_densities is None:


maybe make ElectronDensities an adiitional module.

@wkerzendorf Okay, If I make a ElectronDensities module like this under plasma_input.py

class ElectronDensities(ArrayInput): outputs = ('electron_densities',)

First of all, If we are not supplying electron_densities from CMFGEN file (normal cases), then how will I set this value ?
The Logic I saw in IonNumberDensity , calculates both electron_densities and ion_number_density. And if I make a separate module
for it, then I will have to remove electron_densities from the output list of IonNumberDensity, because then it will giver error while
resolving variables(_resolve_update_list method)

And then I think , it will resulting in changing some tests/fixtures which are dependent on electron_density, if above behaviour is changed.

If you could be more descriptive, it will be nice for me 😅

maybe what you did is fine. I was hoping to make it even nicer - but maybe for now it's fine.

Read a file, where first line is for time_explosion, second line is header row containing velocity,density,electron_densities, and temperaturer

If electron_densities is read from a file, then it will directly use these values, without calculating it.

vg3095 · 2017-07-26T14:24:43Z

tardis/io/model_reader.py

+def read_cmfgen_density(fname):
+
+    df = pd.read_csv(fname, comment='#', delimiter='\s+', skiprows=1)
+    velocity = (df['Velocity'].values[::-1] * u.km / u.s).to('cm/s')


@wkerzendorf For CMFGEN files , I think the convention for velocity is different . For ex-

In CMFGEN files velocity values are in decreasing order. CMFGEN file
6.9999926E+04 6.9999184E+04 ........ 8.7744269E+02 8.7166905E+02

In TARDIS files, velocity values are in ascending order. TARDIS file
9000.0000 10500.000 ......................... 21000.000 22500.000

So, when I did not reverse velocity array here(for CMFGEN case), it was not parsing due to this part of code( because I think the convention for velocity is different for CMFGEN case)

v_inner = velocity[:-1] v_outer = velocity[1:] invalid_volume_mask = (v_outer - v_inner) <= 0

So, I have reversed it here, so that CMFGEN velocity values can be in ascending order as well

I don't know , if I am right, so can you check this ?

@vg3095 if you reverse the velocity - you have to reverse all the other columns as well. They all belong together.

For CMFGEN files , the convention for velocity is different than tardis format. In CMFGEN files velocity values are in decreasing order, while in In TARDIS files, velocity values are in ascending order. So, reversing other columns as well.

These tests ensure that if electron_densities and temperature are read from (tardis_model/cmfgen) density file, they are used directly , rather than being calculated.

vg3095 · 2017-07-27T10:41:05Z

I have now added unit tests.

1. While saving cmfgen files to tardis_model file format using cmfgen script, all columns will be stored in reverse order, so that there is no need to reverse it in read_cmfgen_density func. 2. Changes in values of unit tests to reflect for the point above.

vg3095 · 2017-07-28T09:34:03Z

While saving cmfgen files to tardis_model file format using cmfgen script, all columns will be stored in reverse order, so that there is no need to reverse it in read_cmfgen_density function .

wkerzendorf · 2017-07-31T08:08:50Z

tardis/io/model_reader.py

@@ -235,6 +244,20 @@ def read_artis_density(fname):
    return time_of_model, velocity, mean_density


+def read_cmfgen_density(fname):
+


documentation please

wkerzendorf · 2017-07-31T08:11:26Z

tardis/io/tests/data/tardis_model_density.dat

@@ -0,0 +1,13 @@
+0.976 day
+Index Velocity Densities ElectronDensities Temperature


please save without index

and everything lower case and singular - and with underscore.

1. Now first 4 columns are related to velocity,density,electron_densities,temperature. Abundances begin from 5th column onwards. 2. First line of abundances is ignored, same as in rest of cases for other abundance formats.

1. A common CSV file, containing quantities related to both abundances and density parser. 2. Changes in unit tests config files to reflect the same.

Since we are now using a common file for both

vg3095 · 2017-07-31T12:53:02Z

I have now made a single CSV file for abundances, density, etc. Also added docstrings wherever required.

wkerzendorf · 2017-07-31T13:27:36Z

tardis/io/tests/data/tardis_model_format.csv

@@ -0,0 +1,12 @@
+0.976 day


maybe add a t_0: before that. but other than that - very nice!!

wkerzendorf · 2017-07-31T13:33:02Z

tardis/io/tests/data/tardis_model_format.csv

@@ -0,0 +1,12 @@
+0.976 day
+velocity densities electron_densities temperature C O Mg Si Ni56 Ni58
+871.66905 4.2537191E-09 259538070000000 7.6395577 0 0 0 0.6 0.4 0


electron_densities - should be float.
and add one more line with units for the different quantities. with 1 for dimensionless units (such as abundance fractions)>

the temperatures here are 10^4 k

wkerzendorf · 2017-07-31T13:34:22Z

tardis/io/tests/data/tardis_model_format.csv

@@ -0,0 +1,12 @@
+0.976 day


please write as t_0: ...

Now there is a line after header row, which contain quantites. It reads that line and convert attributes according to that quantity

wkerzendorf · 2017-08-01T13:47:09Z

tardis/io/tests/data/tardis_model_format.csv

@@ -0,0 +1,13 @@
+t0: 0.976 day       
+velocity temperature densities electron_densities C O Mg Si Ni56 Ni58
+km/s K gm/cm^3 /cm^3 1 1 1 1 1 1


gm/cm^3 can become g/cm^3

@wkerzendorf I have updated it now

wkerzendorf · 2017-08-02T11:47:31Z

@tardis-sn/tardis-core I'm signing off - can someone else look over this please!

vg3095 added 5 commits July 25, 2017 15:10

Add a cmfgen density file parser

061183a

Add tardis_model in density schema

8a0ad25

Overwrite electron_densities if already present

26399a0

Make electron_densities default None

337c16a

Use _electron_densities instead of electron_densities

6bf3476

To resolve conflict of same attribute names in IonNumberDensity

wkerzendorf reviewed Jul 25, 2017

View reviewed changes

vg3095 added 3 commits July 26, 2017 16:10

Change tardis_model parser structure

1690158

Read a file, where first line is for time_explosion, second line is header row containing velocity,density,electron_densities, and temperaturer

Overwrite electron_densities in IonNumberDensityHeNLTe

ac24617

If electron_densities is read from a file, then it will directly use these values, without calculating it.

Correction while parsing temp.

85b037a

vg3095 commented Jul 26, 2017

View reviewed changes

vg3095 added 3 commits July 27, 2017 15:56

Reversing attr. in read_cmfgen_density func.

3dc0e68

For CMFGEN files , the convention for velocity is different than tardis format. In CMFGEN files velocity values are in decreasing order, while in In TARDIS files, velocity values are in ascending order. So, reversing other columns as well.

Added tests for cmfgen density parser

0016858

Added tests for new tardis_model density config

1cdd63d

These tests ensure that if electron_densities and temperature are read from (tardis_model/cmfgen) density file, they are used directly , rather than being calculated.

vg3095 mentioned this pull request Jul 27, 2017

[TEP007] Add cmfgen2tardis script #771

Merged

vg3095 changed the title ~~[TEP007][WIP] CMFGEN density parser~~ [TEP007] CMFGEN density parser Jul 28, 2017

wkerzendorf reviewed Jul 31, 2017

View reviewed changes

vg3095 added 4 commits July 31, 2017 14:40

Add docstring for read_cmfgen_density method

32629f3

Change tardis_model abund. parser

ba5eb8e

1. Now first 4 columns are related to velocity,density,electron_densities,temperature. Abundances begin from 5th column onwards. 2. First line of abundances is ignored, same as in rest of cases for other abundance formats.

Common CSV file for tardis_model related parser

b5159ee

1. A common CSV file, containing quantities related to both abundances and density parser. 2. Changes in unit tests config files to reflect the same.

Delete old tardis_model config and abund,density files

a31ad72

Since we are now using a common file for both

wkerzendorf reviewed Jul 31, 2017

View reviewed changes

tardis/io/tests/data/tardis_model_format.csv Outdated

@@ -0,0 +1,12 @@

0.976 day

Copy link

Member

wkerzendorf Jul 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please write as t_0: ...

vg3095 added 2 commits August 1, 2017 18:38

Read quantities from CSV file

503e7d6

Now there is a line after header row, which contain quantites. It reads that line and convert attributes according to that quantity

Update CSV file for tests

b2566bc

Update docstring for cmfgen parser

ad7c784

wkerzendorf reviewed Aug 1, 2017

View reviewed changes

Change gm/cm^3 to g/cm^3

5313ddd

vg3095 closed this Aug 2, 2017

vg3095 reopened this Aug 2, 2017

chvogl approved these changes Aug 4, 2017

View reviewed changes

wkerzendorf merged commit 287e25e into tardis-sn:decay Aug 7, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TEP007] CMFGEN density parser #772

[TEP007] CMFGEN density parser #772

vg3095 commented Jul 25, 2017 •

edited

Loading

vg3095 commented Jul 25, 2017

wkerzendorf Jul 25, 2017

vg3095 Jul 25, 2017 •

edited

Loading

wkerzendorf Jul 26, 2017

vg3095 Jul 26, 2017

wkerzendorf Jul 27, 2017

vg3095 commented Jul 27, 2017

vg3095 commented Jul 28, 2017

wkerzendorf Jul 31, 2017

wkerzendorf Jul 31, 2017

wkerzendorf Jul 31, 2017

vg3095 commented Jul 31, 2017

wkerzendorf Jul 31, 2017

wkerzendorf Jul 31, 2017

wkerzendorf Jul 31, 2017

wkerzendorf Jul 31, 2017

wkerzendorf Aug 1, 2017

vg3095 Aug 1, 2017

wkerzendorf commented Aug 2, 2017

		@@ -235,6 +244,20 @@ def read_artis_density(fname):
		return time_of_model, velocity, mean_density


		def read_cmfgen_density(fname):

		@@ -0,0 +1,13 @@
		0.976 day
		Index Velocity Densities ElectronDensities Temperature

[TEP007] CMFGEN density parser #772

[TEP007] CMFGEN density parser #772

Conversation

vg3095 commented Jul 25, 2017 • edited Loading

vg3095 commented Jul 25, 2017

Choose a reason for hiding this comment

vg3095 Jul 25, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vg3095 commented Jul 27, 2017

vg3095 commented Jul 28, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vg3095 commented Jul 31, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wkerzendorf commented Aug 2, 2017

vg3095 commented Jul 25, 2017 •

edited

Loading

vg3095 Jul 25, 2017 •

edited

Loading