Modify YPD CPMIP metrics

[mcastril]

I think no issue was created yet for this. Forgive me if I am wrong @LuiggiTenorioK

As discussed in other channels and meetings, the idea would be:

1. Copy ASYPD to QSYPD and remove the POST from the formula
2. Rename ASYPD to ¿¿?? and leave the POST in the formula.
3. Rename RSYPD to ASYPD. It would report the rate from SIM to TRANSFER/CLEAN until we have a way to tag jobs.

Then, we'll need a JOB parameter to declare what's a SIM, what's a POST and what's a TRANSFER

In the past, we intended to use TYPE but this was apparently used by some workflows for something different

Sergi, I think we didn't decide a name for 2). Can you propose one?

@agarci3 @eferre1 do you know if TYPE is used in any of the workflows you know?

CC @MissterP

[eferre1]

We use it in the platforms config, to specify one of these parameters (that I can think of): slurm, ps, ecaccess. Then, on the jobs config file, we have it to specify that some templates are in python (instead of the usual ones in bash). So I guess that it could cause conflicts with that behavior.

[sergipalomas]

Hi,

Copy ASYPD to QSYPD and remove the POST from the formula

Yes, the idea is to define a new metric called Queued SYPD (QSYPD), which accounts for simulation time plus the queue. This is essentially the current ASYPD but without considering POST jobs.

Rename RSYPD to ASYPD. It would report the rate from SIM to TRANSFER/CLEAN until we have a way to tag jobs.

Correct. According to the literature, ASYPD should capture the time from the first job submission to the completion of the last job. However, if I recall correctly, implementing this across all workflows is challenging—some workflows are not executed sequentially from start to finish, and running jobs on different platforms can lead to timing inconsistencies if not properly synchronized. (am I right? Could you clarify this please @mcastril).
Just an idea: @MissterP and I are considering treating any job before a SIM as a pre/ini job and any job after SIM a post job, without relying on existing job tags.

Finally, I wanted to mention that I personally dislike the wording for the Actual SYPD. I prefer metric names that clearly define what they measure, and "Actual" means nothing to me. I think Workflow SYPD (WSPYD) would be much clearer and intuitive.

[mcastril]

We use it in the platforms config, to specify one of these parameters (that I can think of): slurm, ps, ecaccess. Then, on the jobs config file, we have it to specify that some templates are in python (instead of the usual ones in bash). So I guess that it could cause conflicts with that behavior.

Eric, maybe you refer to PLATFORMS.TYPE. I wanted to say JOBS.TYPE

[mcastril]

However, if I recall correctly, implementing this across all workflows is challenging—some workflows are not executed sequentially from start to finish, and running jobs on different platforms can lead to timing inconsistencies if not properly synchronized.

It's more the former than the latter, that in some workflows like DestinE the data can be post-processed online (in different steps) but also sometime after, and there are different post-processing steps, not just one.

[mcastril]

Just an idea: @MissterP and I are considering treating any job before a SIM as a pre/ini job and any job after SIM a post job, without relying on existing job tags.

That's okish but you must decide which is the last job of the workflow... and I think this has to be decided first...

For example, if we decide that TRANSFER is the "last job" in EC-Earth workflows we could compare experiments fairly

If we just rely in the last one, one experiment not running TRANSFER could have advantage on the others

But I don't really know if we want to get to this level of detail, maybe you have a more clear idea than I do here

[eferre1]

Eric, maybe you refer to PLATFORMS.TYPE. I wanted to say JOBS.TYPE

Yes, I wasn't sure if there was a distinction internally. So, we use PLATFORMS.TYPE mainly, but JOBS.TYPE is also used in some cases, to specify if some template runs in a different language than the main one used in the project (in our case, main ones are bash, then some python templates use it with TYPE = python).

[mcastril]

Thanks Eric. I think there's a distinction, at least in AS4, but I think also in AS3... @dbeltrankyl can answer better than me

[mcastril]

Summary of the meeting today, thank you all for attending

@dbeltrankyl @sergipalomas @LuiggiTenorioK @ainagaya @orioltinto @eferre1 @gmontanepinto

We discussed this issue, together with the implementation of new metrics #75 . This is another related issue #171

These are the actions to follow-up:

• @sergipalomas has to provide a suggestion for the new name of the current ASYPD (SIM+POST)
• We need to define a parameter to identify the role of the jobs in the workflow. I suggest to vote among ourselves, since we have a good representation.

The next actions once these points are clarified are written in the description. We can be more specific once these points are clarified.

For the parameter, I propose these options:

1. JOBS.[SECTION].STEP
2. JOBS.[SECTION].STEP_TYPE
3. JOBS.[SECTION].CATEGORY
4. JOBS.[SECTION].ROLE

What do you think?

[eferre1]

I think the ROLE or CATEGORY are more clear (at least to me) to what the parameter is related for.

a suggestion for the new name of the current ASYPD (SIM+POST)

For this, I would say that something indicating that it refers to the jobs running in the HPC? We can run POST (and CMOR) on secondary machines, but usually it is on the same HPC until the CLEAN, so I feel like the ASYPD represents the time that we spend on the HPC (maybe it is not the intention on it, so feel free to disregard this opinion if it doesn't make sense).

[ainagaya]

IMO I would try to avoid STEP and STEP_TYPE because they are "too similar" to SPLIT in meaning (which is also a new concept and I think it can lead to confusion), and also because someone might "think" that it's related to, for example, the timestep of the simulation. CATEGORY sounds good to me!

One thing that we should consider is using a variable that is not used in proj configurations. As we will define it under JOBS it can be accessed with CURRENT, and it could cause confusion in edge cases.

What about STAGE?

[gmontanepinto]

I was also thinking on STAGE or PHASE. Maybe CATEGORY is a good option too. I think the other ones might be confusing.

[sergipalomas]

I like STAGE and CATEGORY as well.

For the new naming of the SIM+POST metric, @MissterP and I were thinking of literally Post SYPD (PSYPD). This keeps it easy, short, and straightforward to understand. We, of course, are open to any suggestions you may have.
See the PR #189 API 189 and GUI 285