Save of a design object

[LittleBeannie]

Hi @yihui, I am not an expert in this area, so I would appreciate it if you could provide more details regarding this issue. Thank you!

[yihui]

The motivation came from the Shiny app, in which we allow users to save/load design objects to/from disk.

The PR #509 has removed the major blocker that made it difficult to save/load/use design objects robustly. However, I have been wondering if it's actually worth saving the design objects at all. I mean it would be much easier to save the input parameters and reconstruct the design objects at run time (by passing the parameters to functions like gs_design_ahr()). If we all agree on this approach, there isn't much left to do (since it's exactly want I did for the Shiny app).

One minor remaining technical issue is what file format we should use for saving. Currently we are using the .rds format, for which some people have concerns on security, but R core disagreed with the security implications. Personally I'm not too worried about the security concerns to .rds, especially with R version >= 4.4.0. It's probably the only file format that allows us to save/load R objects reliably. Other file formats such as YAML or JSON are unlikely to be vulnerable to attacks, but the conversion to/from these formats can be lossy for numeric values. If we stick with .rds, one possibility to eliminate the potential security issues is shinylive (i.e., run the Shiny app on the client side without a server so there's nothing to attack).

[nanxstats]

1. I agree we can save design parameters only. Although you should think through the sufficient conditions for reproducibility. Without saving the design object, to reproduce the exact design (bitwise) using a previous parameter-only save, one will need to at least use:

   • The exact gsDesign2 package version
   • The exact gsDesign2 app version

   So these version numbers also need to be recorded in the save. Anything else I'm missing?

2. JSON would be the right plain text format for the save. We don't need "human-friendly" editing here. Google Cloud uses JSON to save access credentials.

3. I feel it's important to have roundtrip guarantee for float-string conversions, but I can be wrong. One approach is to write an R (package) wrapper for the Rust port of dragonbox and put it on CRAN. Simpler ideas welcomed.

[yihui]

For 1, I agree that saving parameters only might leave a hole in reproducibility. Saving the version numbers can help (and I'll do that). A stronger measure is to save an MD5 or SHA256 digest of the design object, and verify by this checksum when the design is loaded. If the checksum remains identical, I think reproducibility is almost surely guaranteed. If not, it will be a little tricky since we'll have to figure out what has changed and the consequences of the changes. For example:

• If a previous version of gsDesign2 had a bug that was fixed in a newer version, then the new design object computed from the old parameters would be the correct one to use (the old design object should be discarded if saved previously). On the other hand, if the new version of gsDesign2 is buggy, users should revert to an old version.

• If the change was due to the rewrite of certain spending or bound functions (which was included in the design object---actually I have never thought this is a good idea), it may not matter at all (bitwise reproducibility would be unnecessary).

For 2 and 3, if we do use JSON, I think the roundtrip guarantee for float-string conversions is definitely necessary.

[nanxstats]

From the productive discussions on this topic today, we reached consensus:

• A more major source of getting discrepant results from a previous save is from the package (and the app) workflow & underlying implementation changes. Keaven prefers to get the package and app into a stable state before the end of 2025 (like how gsDesign has been) so this is less of a concern.
• Save parameters only, and into a JSON file. The saving design object in the gsDesign app was an arbitrary decision and the object is not actively used.
• Just do it and see if round-trip guarantee is actually needed for the numbers we save. If no, then there is no need to implement the round-trip guarantee algorithms. If yes, then implement it.
• Keep deploying only one version of the app to avoid confusion and reduce developer burden.

[yihui]

We decided to keep saving .rds instead of JSON. The security concern is only relevant to R < 4.4.0, and we are currently deploying the app with R > 4.4.0. On the other hand, the app runs in sandboxed environments, which makes the concern even less relevant.

The benefit of using .rds is that it's native to R so we are not limited to the types of objects to save, or subject to potential float-point issues. Using JSON would bring these constraints.