Time dimension missing in diurnal time series

[AugustinMortier]

The time dimension (year/period) is missing in the diurnal time series files. For now, there is just one entry per model.

For now, it looks like this:

{
    "model_id": {
        "time": [...],
        "seasonal": {
            "obs": {...},
            "mod": {...}
        },
        "yearly": {
            "obs": {...},
            "mod": {...}
        },
        "meta1": "meta1",
        "meta2": "meta2"
    }
}

it should look something like this:

{
    "model_id": {
        "time": {
            "year1": [...],
            "year2": [...]
        },
        "seasonal": {
            "obs": {
                "year1": {...},
                "year2": {...}
            },
            "mod": {
                "year1": {...},
                "year2": {...}
            }
        },
        "yearly": {
            "obs": {
                "year1": {...},
                "year2": {...}
            },
            "mod": {
                "year1": {...},
                "year2": {...}
            }
        },
        "meta1": "meta1",
        "meta2": "meta2"
    }
}

The time dimension could also be just after the "model_id", but this would repeat the meta data for each time key.

e.g: of current file: emep/2021-reporting-debug/ts/diurnal/WORLD_EBAS-h-vmrox_Surface.json

[jgliss]

@hansbrenna, since you wrote the code for the diurnal processing, perhaps you could have a look at that?

I guess the code below here needs to be modified to comply with the new setup.

Probably relevant here are the parameters "periods" (and perhaps "seasons") to be added to the code through a loop in which the colocated data is "cropped" in time for the corresponding period and season.

See e.g. here for an example where this is implemented in the heatmap processing.

[hansbrenna]

Yes, I can have a look at this.

[hansbrenna]

@AugustinMortier "time" is just the float representation of the day of the week. Shouldn't this be kept unchanged and the time series for the different years be shown stacked?

i.e. there only needs to be 1 entry under time or the multiple entries will be the same.

[AugustinMortier]

True, I forgot that time is the same. We can have just one entry for this key then!

[hansbrenna]

I have made a pull request for this now in #502.

@dulte do you have a test case to run through the modified branch?

[dulte]

@hansbrenna, I've been working exclusively with EMEP models uptill now, so I don't have anything with multiple years (I working on implementing a multiyear feature for EMEP, but that is work in progress). But I can create a test case? I busy updating eEMEP to the latest version for an upcoming VOLCEX exercise, so I won't be able to look at this issue before next week

[hansbrenna]

OK. I can try to make a test case myself.

[hansbrenna]

or maybe @AugustinMortier has a test case I could use?

[AugustinMortier]

No I don't. I just realized that issue when implementing the diurnal plots in AeroVal (it was not implemented before last week)

…

On Fri, Oct 22, 2021, 19:23 Hans Brenna ***@***.***> wrote: or maybe @AugustinMortier <https://github.com/AugustinMortier> has a test case I could use? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#488 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVTXGSQCZHBITIUIZZNIC3UIGMXJANCNFSM5F46VEMQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[hansbrenna]

@AugustinMortier I have run a test case and generated some json files. Are these Ok for you to use?

https://github.com/hansbrenna/pyaerocom_dev_scripts/tree/master/2021/test/dw

[AugustinMortier]

I think it is! I'll try it as soon as possible!