D6.9: Shared persistent Memoisation Library for Python / Sage

[minrk]

• WP6: Data/Knowledge/Software-Bases
• Lead Institution: University of St Andrews
• Due: 2019-02-28 (month 42)
• Nature: Other
• Task: T6.9 (T6.9: Memoisation and production of new data #131)
• Proposal: p. 54
• Final report (sources)

Persistent memoisation is the computational process of storing a program's outputs and retrieving them later, instead of re-running programs that are guaranteed to return the same answer. This approach may be useful in a wide range of fields but is particularly relevant to mathematicians: indeed, it's common for them to face compute intensive exact problems with small inputs and small outputs.

Results can be stored for future calls to a function in the present program session, in later sessions, or even for other users in different parts of the world. Beside the advantages of recalling old results instead of repeating work, a human-readable cache could also be used by researchers to create a reproducible record of computed results that can be used outside the context of the memoisation system itself.

The aim of D6.9 is to establish a persistent memoisation framework to cache results in Python and GAP across sessions, in a way that is easy to deploy and configure, and allows for results to be shared reliably between different researchers. D6.9 requires a "shared persistent memoisation library for Python/Sage", which is fulfilled by pypersist, a new Python library written by the current author. A corresponding GAP package, Memoisation, was also written, featuring similar features and with a certain level of interoperability between the two systems. This is a step in the general effort of WP6 toward bridging together Data, Knowledge, and Software notably for Virtual Research Environments.

In this report, we present a review of some existing tools for memoisation, describe the two new packages, show an example of their use together, and give an overview of the future direction of this project.

[mtorpey]

I have just edited the top comment in this thread with an abstract for the deliverable.

[nthiery]

I have just edited the top comment in this thread with an abstract for the deliverable.

Thanks! I imported it in the repo (by running `make WP6/D6.9/github-issue-description.tex` from the main directory), so it will appear in the report next time you compile it. Thanks for the other improvements to the repo. Did you get a chance to add "support for methods" in the comparison table? Other than this, that sounds ready to submit, right?

[mtorpey]

There are actually a few things you suggested that I haven't added yet – I was away ill on Thursday and Friday. However, I should finish those today, and I'll comment here when I have!

[mtorpey]

Okay, I believe this is now ready to submit, and includes all the feedback I got from @nthiery. Thanks for all the help! :)

[nthiery]

Thanks for all the good work @mtorpey! The report is now submitted after some tiny touches of polishing.

[nthiery]

After discussion in Cernay: since this report has not yet been evaluated, @mtorpey will make soon some minor updates to report on the GAP implementation and the improvements to the Python implementation.
We will then resubmit before August 31st.

[mtorpey]

@nthiery I've made the updates, and I've modified the abstract at the top of this issue. I'm happy to resubmit now.

[nthiery]

Great, thanks a lot! I proofread the changes and they are very informative.

I'll ask or PO to "reject" the previous one so that I can resubmit this one.

[nthiery]

Final version submitted!