Variability (confidence interval) in optimized parameter value

[hsiebinga]

Dear community,

I’ve developed several PBPK tumor models in MoBi regarding radionuclides used in clinical practice and managed to validate these models based on scan data (n=41). The main process is that the drug binds to a receptor on cell membranes (interstitial) and gets internalized into the intracellular compartment.

Exact receptor expression is unknown, while all other parameters regarding the internalization process are based on literature values. Therefore we assume that we can optimize receptor concentrations based on our observed organ (scan) data. Since spleen is known to have highest receptor expression (and thus uptake) the receptor fraction is 1 for spleen. The receptor reference concentration was then optimized based on spleen data using the built-in parameter identification Monte Carlo algorithm.

Results show a reference concentration of 100.11 nmol/L, which is a reasonable estimate. However, I’d like to add variability to this estimated value, to say something about how certain we are about this specific estimation. (So, just to make things clear, I do not mean an evaluation of the model as a whole, which can be done looking at residuals and the total error.) I did check the covariance matrix tab, but didn’t manage to fully understand what is calculated there, although I read threads on the forum and the online manual. So my specific questions are:

• How is the confidence interval on the identified parameter calculated (in the covariance matrix tab)?
• I get a confidence interval of NaN on my reference concentration estimate, how can that be explained?
• Is there another way to include a certainty to this optimized parameter?

Many thanks for your help and I’ll be happy to answer any questions that arise!

Hinke

[StephanSchaller]

Dera Hinke,

I hope I can help with these perliminary answers (in a rush):

• confidence intervals are calculated based on the fisher information
• NaN values pop up if the information on the parameter in the data is "zero" (zero entries in the fisher information) as the inverted fisher information matrix is used to calculate these . This might happen if you have parameters fittet which are non-identifiable (e.g. highly correlated)
• Not sure I get the last question, you can add certainty with additional data or increase certainty per parameter by removing parameters to be fitted (reduce parameter dimensionality)

Hope this helps?

An interesting topic (radionuclide PK). Do you use these models for dosimetry? We would be interested to exchange ideas as we are also working on this topic. Would love to have a chat if you do have time: stephan.schaller@esqlabs.com

hope my answers were helpful,
Stephan

[Yuri05]

@hsiebinga Small addition to the explanation by @StephanSchaller: zero sensitivities (which result then in NaN confidence intervals) are in this particular case ("Reference concentration") just a bug (sensitivity of parameters which are used to calculate the initial amount of a molecule - in this case of a protein - are currently always returned as 0)

[hsiebinga]

Thanks for your explanations, really helpful! To clarify - I only have one fitted parameter, but then with the explanation of the bug it makes sense that I get NaN values.

However, I would still like to add variability (probably a confidence interval) to the fitted parameter. I’ve already tried to fit the start amount instead of the reference concentration, but that didn’t solve the problem. Is there a solution to still obtain a confidence interval to the optimized parameter (e.g. is it possible to calculate it manually or in R?)?

@StephanSchaller , I’m not working on absorbed radiation doses (yet), but just on the peptide part to get insights in the distributions and (tumor) parameters that affect this distribution most. However, it would be very interesting to also include activity and absorbed doses if this would be possible. I’ll keep in touch!

Hinke

[hsiebinga]

@Yuri05 or @StephanSchaller, any thoughts on this?

Many thanks,
Hinke

[StephanSchaller]

@Yuri05 or @StephanSchaller, any thoughts on this?

Nothing off-the-shelve that I can think of from the top of my head. Yes, you would have to set this up in Matlab/R with a optimization package. In R, the FME package (https://fme.r-forge.r-project.org/) or rstan (https://mc-stan.org/docs/2_23/stan-users-guide/error-statistics-from-the-bootstrap.html) might work. We are currently developing a parameter identification framework for the OSP in R which will be able to do this, but this might take some time before it is released. Sorry.

[hsiebinga]

Okay, I'll see what works best for me. Thanks for your reply, really appreciated!

[StephanSchaller]

@hsiebinga, follow-up: how did this work out for you?

[hsiebinga]

Unfortunately, we didn't use one of the R packages you proposed. We chose to use a more simple solution to show the effect of the fitted parameter (reference concentration SSTR) on the total model.

We performed an additional likelihood profiling analysis on the total model: examining changes in the total error of the model (sum of squared residuals) while changing the fitted parameter ‘reference concentration SSTR2’. Since this was only 1 parameter that was optimized, this total error directly described uncertainty of this fitted parameter specifically. Therefore, we adjusted the parameter with -50% and +50% and compared the total errors. We concluded, that the effect of this specific parameter on the total model was rather small, since the total error only differed slightly while adjusting the parameter value.