Observables in SBML should not start with observable_

[matthiaskoenig]

Id overloading is a very bad idea in general and will break many things. In this case you require ids which should be compared to experimental data to start with observable_.
As a consequence one has to write the model for the parameter fitting task which will not work.

• with every new experiment new observables could be used so the model has to be changed as soon as new data becomes available (?!)
• existing models don't have their ids starting with observable_, so PETab does not work with existing models (?!)
  Only solution is to let every id start with observable_ when creating SBML models, but then you can just leave it out right away.

Solution: just specify the ids which are observables for a parameter fitting experiment, no need for an obervable_ prefix anywhere.

Similar parameter overloading occurred in the fbc (flux balance community) and it basically broke all the tools and made models not exchangable between tools. I see the same here with PETab, because I cannot run a simulation experiment on an existing SBML model. Never enforce a model to have tool specific ids!

[JanHasenauer]

We discussed a lot about this topic and I see your points.

I agree that "observable_" could be skipped, however I don't think that this addresses your points.

1. Even if you skip "observable_" we still have "sigma_". We have to be able to assign a specific noise level to an observable. Accordingly, models still need to be adapted.

2. Removing "observable_" will not address the problem that models need to be extended if new data become available. The only alternative I see it to put the observable specification (e.g. the mathML expressions and corresponding IDs) in the data files. This is possible, but it would mean that the data has to be changed as soon as we exchange the model. This is also nothing we would like to do.
   We asked ourselves what happens more frequently: changing of the model or changing of the dataset. In our experience the former is more likely.

I'm I missing something?

[dweindl]

Thanks for your feedback @matthiaskoenig.

Id overloading is a very bad idea in general and will break many things.

Fully agreed. We discussed at some point specifying observables in a separate table.

Something like:

observableID	observableMath	noiseMath	observableTransformation
relativeTotalProtein1	observableParameter1 * (protein1 + phospho_protein1 )	noiseParameter1	lin
...	...	...	...

Would something like that address your concerns?

This would have the advantage of providing a clear separation of the dynamic model and the observation and error model.

Disadvantage could be that when the model changes frequently, it may be tedious to keep things in sync. It depends a bit on the use case as Jan mentioned. If we want to parameterize some existing model, it would be preferable to not modify the SBML file at all. During model development it may be more convenient to have everything inside the SBML. (On the other hand, it would be straightforward to convert a model of the current type to this more general format automatically).

Another disadvantage with just using an existing dynamic model and building the observation model around it outside the SBML file would be that we don't have annotations or units of additional output parameters (unless we make the schema significantly more complex, which I would very much like to avoid).

[dweindl]

After some further discussions, here an updated suggestion with pros and cons and request for comments:

Change: Get rid of the imposed AssignmentRule naming by providing a separate observation table like:

observableId	observableFormula	observableTransformation	noiseFormula	noiseDistribution
relativeTotalProtein1	observableParameter1 * (protein1 + phospho_protein1 )	lin	noiseParameter1	normal
...	...	...	...	...

Description:

• observableId

  Any identifier which would be valid in SBML

• observableFormula

  Observation function as plain text (parsable by sympy -- is there anything close to standard definition for that?). May contain any symbol defined in the SBML model or parameter table. In the simplest case just a species or an AssignmentRule target. May introduce new parameters of the form observableParameter${n}, which are overridden by observableParameter in the measurement table.

• observableTransformation

  As previously specified in the measurement table. Reduces redundancy there.

• noiseFormula

  Noise model parameters as plain text formula. (What was previously an SBML AssignmentRule)

• noiseDistribution

  ['laplace'|'normal']. Replaces noiseDistribution column in the measurement table (reducing redundancy).

---

Pros/Cons compared to previous definition as AssignmentRules:

Cons

• Change of format (more work)
• Formulas in plain text are less flexible than MathML and no nice display as e.g. in Copasi
• Extra file to keep synced. Changes in models usually require changes here.
• No unit definitions for observable parameters

Pros

• Not forcing a naming schema upon users (which may be incompatible with existing models)
• Enables re-use of existing models, e.g. from biomodels without modifications
• Separation of dynamics and observations
• Can still define observables as SBML AssignmentRules and just use that parameter in observableFormula --> easy to convert existing models
• Removes redundant entries in measurement table (observableTransformation and noiseDistribution, which had to be repeated for every single data point)

@matthiaskoenig @fbergmann @refisch What do you think?

[fbergmann]

since we still have some reserved names that are being used. I'd be in favor of leaving things as they are right now. The argument, that a new model can be used 'as is' will be only true for some tools, others will have additional work of transforming the model and introducing the observable transformations. This will be harder to do than renaming a couple of elements in the model currently. Since both observableTransformation and noiseDistribution are optional parameters in the measurement table, also the reduced redundancy would hold not for all.

[dweindl]

Thanks for your feedback @fbergmann.

we still have some reserved names that are being used

You mean {observable,noise}Parameter${n}_${observableId}? Right. Not too happy about them either. Not because of potential name collisions, but mostly because they make it pretty hard to read the measurement table in case of multiple parameters. Literally every time, I am wondering which parameter was parameter1 and which one was parameter2. This could be addressed by explicitly stating the observable parameter to override in the {observable|noise}Parameters columns (e.g. offset:400;scaling:2.5). This way, it would be more intuitive and we could get rid of the remaining reserved / imposed names. Downside is, that it doesn't increase readability in case there is only one parameter (although then the prefix could be made optional).

The argument, that a new model can be used 'as is' will be only true for some tools, others will have additional work of transforming the model and introducing the observable transformations.

That's right. However, above, the "used as is" aspect was more directed at the modeller than at the simulation tool. From a user perspective I don't care too much about the latter. For the tool it needs to be implemented once. The user would have to change every/many model(s).

Since both observableTransformation and noiseDistribution are optional parameters in the measurement table, also the reduced redundancy would hold not for all.

True.

Any other opinions?

[refisch]

Thanks for the necessary and fruitful discussion. We agree that Id overload should be omitted. In general we see the following advantages when using the approach of @dweindl:

1. The conceptual benefit to keep the data/experiment and the model separate

   From our perspective the decision to keep data and experiment separate will enable a clear understanding of the model and deconvolute the format.

2. SBML is a model format that does not aim to describe data

   The main reason for developing peTab as a format was the fact, that SBML does not aim to describe data but is aimed at describing models and pathways. From our perspective, it thus seems pertinent to leave information about observations and the data out of the SBML file. More concretely, assignment rules are in general not meant for error and observable functions. Therefore, we would prefer to externalize the observation and error function.

   Additionally, this makes the peTab format more robust with respect to changes made in new SBML releases, which makes for less maintenance work.

Furthermore, some additional comments:

Changes in models usually require changes here.

Most frequently, if you change something in the model, no need to have changes in observation function. Furthermore, if new data sets for the same observableIDs are added, this file does not change.

Can still define observables as SBML AssignmentRules and just use that parameter in observableFormula --> easy to convert existing models

We would prefer to not give this option to the user. From our experience, multiple options in where to encode the same information can lead to confusion, for both users and developers. Instead, we'd prefer to use the new format-file only.

(after discussions with @adrianhauber, @fgwieland, @JanineEgert)

[yannikschaelte]

Thanks for your feedback @refisch. We would then tentatively accept the introduction of the observable file, since it should solve all of the mentioned problems, and modularly separate the model and the experiment.

We would prefer to not give this option to the user. From our experience, multiple options in where to encode the same information can lead to confusion, for both users and developers. Instead, we'd prefer to use the new format-file only.

In practice, the user cannot really be prevented from specifying arbitrary ids as observable targets, including assignment rules. This may not be the nicest way in terms of separation of model and experiment, but on the other hand e.g. enables using visual editors for defining formulas inside SBML. Thus, I suggest encouraging the clear separation, and change all benchmark models accordingly, but there is no way of explicitly forbidding the other way.

(after discussion with @dweindl @LeonardSchmiester)

[yannikschaelte]

Remains to decide on the additional naming conventions on observable and noise parameters. Suggestion of @dweindl :

You mean {observable,noise}Parameter${n}_${observableId}? Right. Not too happy about them either. Not because of potential name collisions, but mostly because they make it pretty hard to read the measurement table in case of multiple parameters. Literally every time, I am wondering which parameter was parameter1 and which one was parameter2. This could be addressed by explicitly stating the observable parameter to override in the {observable|noise}Parameters columns (e.g. offset:400;scaling:2.5). This way, it would be more intuitive and we could get rid of the remaining reserved / imposed names. Downside is, that it doesn't increase readability in case there is only one parameter (although then the prefix could be made optional).

As an extension, one could also add additional columns observableParameters and noiseParameters to the observable file, having as content e.g. scaling;offset. This would allow to in the measurement file not have to write scaling:2.5;offset:400, but still allow the previous short form 2.5;400, with the order specified by the column in the observable file. This is similar to named arguments in python. In that approach, one would have thus the possibility to write both forms in the measurement file, at the cost of one additional parsing step.

[yannikschaelte]

discussion on the more flexible handling of obervable and noise parameters has moved to #242