# D6.2: Initial DKS base Design (including base survey and Requirements Workshop Report) #136

Closed
opened this Issue Sep 8, 2015 · 78 comments

Projects
None yet
8 participants
Contributor

### minrk commented Sep 8, 2015 • edited by nthiery Edited 1 time nthiery edited Sep 19, 2016 (most recent)

 WP6: Data/Knowledge/Software-Bases Lead Institution: Jacobs University Bremen Due: 2016-08-31 (month 12) Delivered: 2016-09-12 Nature: Report Tasks: T6.1 (#123), T6.3 (#125), T6.5 (#127) Proposal: p.55 Final report The OpenDreamKit proposal had envisioned WP6: Data/Knowledge/Software bases as a foundational enterprise that would develop a knowledge-based architecture over the course of the project and would allow to re-engineer ad-hoc interfaces between systems (e.g. from T3.2 (#51)) on a more semantic basis -- the knowledge aspect (K). Consequently, the proposal envisioned concentrating the data (D) aspect on the mathematical knowledge bases (specifically LMFDB, OEIS, and FindStat) and proposed a host of foundational investigations of mathematical for the software (S) aspect with applications e.g. in the verification of algorithms. Already the kickoff meeting in Paris in September 2015 revealed that the D/K/S aspects are much more tightly coupled in systems than anticipated. This was confirmed by the DKS survey conducted subsequently. In particular, the participants of WP6 identified the interoperability of OpenDreamKit systems to be one of the most critical steps in creating a VRE toolkit. Thus we prioritized tasks T6.1 (#123), T6.2 (#124), T6.3 (#125) and organized a series of workshops and code-maratons to develop a semantic foundation for system interoperability and simultaneously test it in implementations. As a consequence, we have completed the initial design of D/K/S-bases (for this deliverable) in parallel with the initial implementation of a DKS base format based on OMDoc/MMT and the implementation of a DKS base system itself based on the MMT system (both for D6.3 (#137)), all activities fuelling each other. D6.3 (#137) was thus completed about three months ahead of schedule. Note that the RNC schema envisioned in the title proved un-necessary since, with the refined Math-in-the-Middle (MitM) design, the normal OMDoc/MMT schema is sufficient. Due to the resulting tight coupling between this deliverable and D6.3 (#137), and for the convenience of the reader, we have decided to report on both deliverables together. In this report we therefore present the design process towards DKS-theories including the overall architecture (for this deliverable), a survey of the systems involved (for D6.3 (#137)), our current implementation (for D6.3 (#137)) as well as our plans for the future. Each part is labeled by the deliverable they contribute to mainly.

Closed

Open

Open

Member

### kohlhase commented Jun 30, 2016

 this is a sister deliverable to #137 and will be largely done together with that. We are making progress on this in the WP6 workshops. I think that @pdehaye should be also involved in writing this up.
Member

### kohlhase commented Aug 21, 2016

 @pdehaye you made a survey in Paris and St. Andrews I recall, could you please write this up as '(including base survey and Requirements Workshop Report)'. I would be very grateful. cc: @tkw1536
Contributor

### tkw1536 commented Aug 24, 2016 • edited Edited 1 time tkw1536 edited Aug 27, 2016 (most recent)

 I have started writing this up properly. You can find my progress so far at https://github.com/KWARC/OpenDreamKit/tree/master/WP6/D6.2 @pdehaye So far I have not referenced the survey, I will probably get some work done on that this afternoon. Feel free to insert it already. @kohlhase I had to switch from bibtex to biber, as for some reason the bst file was broken. If you want me to switch back, let me know. @florian-rabe: It would be great if I could get some help from you regarding the overall structure and framework of the report. Edit: Updated link because we changed branches
Contributor

### pdehaye commented Aug 25, 2016

 To give a summary of the current situation: the survey is scattered in #123, other issues referenced from #123, and here. Far from ideal, but a good starting point. I am starting to work on this now, first by just dumping the scattered data into the file you are working on (in parallel to installing Latex on my new computer, which could become a blocker...)
Contributor

### tkw1536 commented Aug 27, 2016

 @pdehaye I have given you push access to the KWARC fork. I will do a bunch more writing today, I will probably pull in your changes after that.
Contributor

### pdehaye commented Aug 27, 2016

 I already pushed to KWARC's repo On Sat, Aug 27, 2016 at 10:16 AM, Tom Wiesing notifications@github.com wrote: @pdehaye https://github.com/pdehaye I have given you push access to the KWARC fork. I will do a bunch more writing today, I will probably pull in your changes after that. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub #136 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/ADH2X4SgkQS6u8eZUwqN5NCZ0PR134Iqks5qj_J5gaJpZM4F5zNI .
Contributor

### tkw1536 commented Aug 27, 2016

 I was working on the D6.2 branch, so I didn't even see that. I will just move to master.
Member

### kohlhase commented Aug 27, 2016

 I already pushed to KWARC's repo @pdehaye I think we should organize this so that the survey itself is given as an appendix to the report and instead of the current section there is a short description of what we learn from this survey. That gives us a self-contained report (by the appendix) and the high-level requirements we need to justify the DK(S) theories.
Contributor

### pdehaye commented Aug 27, 2016

 That's a good idea. On 27 Aug 2016, at 14:38, Michael Kohlhase notifications@github.com wrote: I already pushed to KWARC's repo @pdehaye I think we should organize this so that the survey itself is given as an appendix to the report and instead of the current section there is a short description of what we learn from this survey. That gives us a self-contained report (by the appendix) and the high-level requirements we need to justify the DK(S) theories. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
Member

### kohlhase commented Aug 27, 2016

 That's a good idea. if you pull, then @tkw1536 has already put your part into a new file in the appendix.
Member

### kohlhase commented Aug 27, 2016 • edited by slel Edited 1 time slel edited Aug 27, 2016 (most recent)

 @florian-rabe should also subscribe to this.
Member

### kohlhase commented Aug 27, 2016 • edited Edited 1 time kohlhase edited Aug 29, 2016 (most recent)

 Problem I am a bit worried about the fact that we have promised a design of DKS theories for this report and we are mostly reporting only on DK theories. Background Even though we have studied biform theories in OMDoc, we have not worked on this aspect much in the first 12 months, since we have concentrated on the MitM approach to interoperability. Solution? I have thought about this in the last days, and I think that we can (and should) think of the MitM information architecture (think of the clouds picture with the interface theory graphs that are generated from the systems like GAP or Sage and the hand-curated MitM theory in the middle) can be seen as a integration of the S (for Software) component into DKS theories -- actually we do not get "DKS theories", as forseen in the proposal, but "DKS theory graphs", where we have K-Theories (regular MMT theories) for the specifications and math knowledge, DK-theories (as we introduce them in D6.2) for models and concrete mathematical structures (e.g. OEIS, LMFDB, FindStat), and the alignments and interviews (they are really interpretations and implementation mappings) as the integration of the S component. If we then move our focus from verification of algorithms (only mentioned in one sentence at the end of T6.8) towards interoperability, then do not need to build on biform theories, but can build on the MitM architecture instead. I think that this is legitimate, and actually this shift in perception is a major result of our work in ODK. I would like to have some feedback on this, and if that is positive, I would try to extend the report to base itself on this "initial design".
Contributor

### tkw1536 commented Aug 28, 2016

 @kohlhase I have rewritten the introduction towards what you suggested. Apart from adding a figure to illustrate the MiTM architecture it is completed. It would be great to get some feedback.
Contributor

### pdehaye commented Aug 29, 2016

 Does anybody know how to make "paragraph" behave like "subsubsubsection" ? \setcounter{secnumdepth}{4} seems to go part of the way, but it needs a newline at the end, and maybe a change of font. I can Google it, but the answers I find are for the general "article", and don't really work for "deliverablereport". It's necessary for the survey as it has several layers of nesting: appendix / system / {data, knowledge, software} / question section / subsection / subsubsection / paragraph Thanks Paul On Mon, Aug 29, 2016 at 12:19 AM, Tom Wiesing notifications@github.com wrote: @kohlhase https://github.com/kohlhase I have rewritten the introduction towards what you suggested. Apart from adding a figure to illustrate the MiTM architecture it is completed. It would be great to get some feedback. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub #136 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/ADH2X7BrOjFkDMXDNxSgLrMWQMtiWqXIks5qkglvgaJpZM4F5zNI .
Member

### kohlhase commented Aug 29, 2016

 actually, deliverablereport is based on amsart.cls, maybe you should try that in your query.
Member

### kohlhase commented Aug 29, 2016

 actually, I would like to use article.cls in in any case. Maybe we can change deliverablereport.cls?
Member

### kohlhase commented Aug 29, 2016

 actually, I would like to use article.cls in in any case. that is not trivial we need to handle the abstract (amsart does that well)
Member

### kohlhase commented Aug 29, 2016

 I have tweaked the figures a bit and snippetized the sections, please pull, if you are working on this.
Contributor

### pdehaye commented Aug 29, 2016 • edited Edited 1 time pdehaye edited Aug 29, 2016 (most recent)

 Thanks, basing myself on amsart seems like a good start. Needs more tweaking from me though. My macros for the packages introduced too many spaces with \,, but those were needed in some cases. Switched to using xspace package.
Contributor

### pdehaye commented Aug 29, 2016

 I have finished a first pass at the formatting and proofreading of the text of the survey. It is still very coarse. I still need to do a second pass and add a summary in the text. The survey for GAP is still empty. As much as I remember, it was never done formally. I see a few options: ask a GAP developer to fill it now; I read the recent papers describing GAP architecture and fill the survey; I simply refer to the recent papers Honestly I think option 3 is best. It won't exactly fit the format of the others, but will have the same content. What do you think?
Member

### kohlhase commented Aug 29, 2016

 I would try for option 1. In the informal review, the reviewers made it quite clear that they did not like references to the outside too much.

Closed

Closed

Open

Contributor

### tkw1536 commented Aug 30, 2016

 I have written an abstract and a conclusion. Apart from one ednote I have to solve together with @florian-rabe the report is finished from my side.
Member

### kohlhase commented Aug 30, 2016

 I have reworked the intro, and Florian will be working on this today. I expect to finish my part tomorrow. We are still waiting for the contribution from @pdehaye (what have learnt from the survey).
Contributor

### florian-rabe commented Aug 30, 2016

 I've significantly revised the overall structure as well as the (new) sections 1, 3, 5.1, and 5.3. I will still do section 5.2 but not necessarily today.
Contributor

### florian-rabe commented Aug 30, 2016

 @Jazzpirate has added subsections 6.1 and 6.2.
Contributor

### florian-rabe commented Aug 30, 2016

 Finished 5.2 after all. Please check if what I wrote makes sense. @Jazzpirate I left some ednotes for you in this section
Contributor

### florian-rabe commented Aug 30, 2016

 By the way, we should turn this deliverable into a journal paper. Is anybody volunteering to take the lead on that?
Member

### kohlhase commented Sep 5, 2016 • edited Edited 1 time kohlhase edited Sep 5, 2016 (most recent)

 Software for sure. I agree. And I have corrected this.
Contributor

### florian-rabe commented Sep 5, 2016

 I've revised sections 1-6. @pdehaye Section 2 is still missing. @tkw1536 I added one ednote for you.
Member

### kohlhase commented Sep 5, 2016

 @pdehaye Section 2 is still missing. what is your plan there?
Contributor

### tkw1536 commented Sep 5, 2016

 @florian-rabe Added the relevant fragment of the schema theory. The text after that made sense with it in place.
Member

### kohlhase commented Sep 5, 2016

 I synchronized (pull/push) with the ODK upstream.
Contributor

### pdehaye commented Sep 7, 2016

 I will integrate the remainder tomorrow into the report.
Member

### kohlhase commented Sep 9, 2016

 tomorrow into the report. actually, this might take until sunday, ... (semester is starting) ...
Contributor

### pdehaye commented Sep 10, 2016

 I think I am good for my part. Sorry it took so long. I also re-read the whole thing. The most significant change I made is in adding a footnote concerning the progression of workshops. The intent there was to highlight that while many of the OpenDreamKit participants had attempted to do similar stuff in the past, what is new now is the MMT approach. I felt that was very relevant to this deliverable. I have NOT looked at the bibliography, as I don't have it. It seems to be on a folder internal to KWARC. After delivering, this should be fixed so we can make sure to be able to recompile the deliverable just from the repo. Let me know if there is anything else I should do.
Member

### kohlhase commented Sep 10, 2016

 @pdehaye I looked over the changes, well done. actually, I am a bit surprised that you do not have the bibliography. We made a git subrepo at ../../lib/kbibs and that should be pulled automatically for you. Could you have a look?
Contributor

### pdehaye commented Sep 10, 2016

 Actually, yes, I do have the bibliography. Either I missed it or I looked before you included it? Anyways, I have made the necessary changes now (i.e. added FindStat reference). On Sat, Sep 10, 2016 at 9:26 AM, Michael Kohlhase notifications@github.com wrote: @pdehaye https://github.com/pdehaye I looked over the changes, well done. actually, I am a bit surprised that you do not have the bibliography. We made a git subrepo at ../../lib/kbibs and that should be pulled automatically for you. Could you have a look? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub #136 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/ADH2XxqaFB7fou002Cy8F7E2-knz3maDks5qolujgaJpZM4F5zNI .
Member

### kohlhase commented Sep 10, 2016

 Anyways, I have made the necessary changes now excellent. I made a first pass over explaining the "two-deliverables-in-one-report" design. I will make a first D6.3 deliverable stub now.
Contributor

### pdehaye commented Sep 10, 2016

 There is also the pesky problem of author ordering. I absolutely don't care except I want to make sure it is intentionally non alphabetical. On 10 Sep 2016, at 11:18, Michael Kohlhase notifications@github.com wrote: Anyways, I have made the necessary changes now excellent. I made a first pass over explaining the "two-deliverables-in-one-report" design. I will make a first D6.3 deliverable stub now. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
Contributor

### nthiery commented Sep 10, 2016

 On Sat, Sep 10, 2016 at 01:24:49AM -0700, pdehaye wrote: There is also the pesky problem of author ordering. I absolutely don't care except I want to make sure it is intentionally non alphabetical. Just being curious: why not alphabetical? It's the customary way to resolve author ordering in the math community, isn't it?
Member

### kohlhase commented Sep 10, 2016

 I have also not thought about it, but I will make it alphabetical. I am working on these things ATM.
Member

### kohlhase commented Sep 10, 2016 • edited by nthiery Edited 1 time nthiery edited Sep 10, 2016 (most recent)

 I think that the order was just that everyone just added themselves at the end, no malice or so here. The converse would have surprised me :-) Just being curious about @pdehaye's comment.
Contributor

### nthiery commented Sep 10, 2016

 Speaking of authors: for our reports, should the authors be the people having written the report itself, or should it include those that did major work on the content? (brainstorm and implementations here, implementation elsewhere, ...). I am wondering because of other reports like D4.4 (#93), where I'll probably write the bulk of the report, while most of the implementation was done by others.
Member

### kohlhase commented Sep 10, 2016

 I have no clear opinion on this, and I think we should have a general "rule" so I guess you (as the coordinator) should say something public about this. I guess my personal preference to have as authors the people who wrote the report, and make the contributors clear in the report (acknowledgements?)
Member

### kohlhase commented Sep 10, 2016

 OK, I have taken care of all the ednotes. From my point of view, this deliverable is done.
Contributor

### fangohr commented Sep 10, 2016

 Re authors, I'd include everybody who has contributed to the task/deliverable. The report is meant to represent the work done, so it should include the people involved. (If the categories of 'author' and 'contributor' are suggested, then @kohlhase 's suggestion seems reasonable, otherwise I'd just use one category [i.e. authors or contributors], and put everybody into that.)
Contributor

### pdehaye commented Sep 10, 2016

 re: authors, I didn't mean to imply there was malice. Apologies that it came out that way. This is starting to look finished. There is still two ednotes in, one concerning a latex presentation bug for the listings, and one on the handling of constructors/assessors. There is also one typo in the github issue descriptions. It should be "issue", not "issue's". I tried changing it but there seems to be two layers of Makefile's and I didn't fully understand the system used. In the end I left it as is, for @kohlhase to fix.
Contributor

### nthiery commented Sep 10, 2016

On Sat, Sep 10, 2016 at 12:44:30PM -0700, pdehaye wrote:

There is also one typo in the github issue descriptions. It should be
"issue", not "issue's". I tried changing it but there seems to be two
layers of Makefile's and I didn't fully understand the system used. In
the end I left it as is, for [1]@kohlhase to fix.

You just need to fix the issue description on github; I'll upload the

Cheers,

## Nicolas

Nicolas M. Thiéry "Isil" nthiery@users.sf.net
http://Nicolas.Thiery.name/

Contributor

### pdehaye commented Sep 10, 2016

The typo is in the template that processes the GitHub issue description and
introduces it in the latex. It seems the typo originates in the Makefile
itself, but then was propagated. My sed skills are not up to par to do a
global fix: hard to the escaping correctly on s/issue's/issue/g !

On Sat, Sep 10, 2016 at 10:10 PM, Nicolas M. Thiéry <

On Sat, Sep 10, 2016 at 12:44:30PM -0700, pdehaye wrote:

There is also one typo in the github issue descriptions. It should be
"issue", not "issue's". I tried changing it but there seems to be two
layers of Makefile's and I didn't fully understand the system used. In
the end I left it as is, for [1]@kohlhase to fix.

You just need to fix the issue description on github; I'll upload the

Cheers,

## Nicolas

Nicolas M. Thiéry "Isil" nthiery@users.sf.net
http://Nicolas.Thiery.name/

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#136 (comment),
.

Contributor

### nthiery commented Sep 11, 2016

 On Sat, Sep 10, 2016 at 01:25:31PM -0700, pdehaye wrote: The typo is in the template that processes the GitHub issue description and introduces it in the latex. It seems the typo originates in the Makefile itself, but then was propagated. My sed skills are not up to par to do a global fix: hard to the escaping correctly on s/issue's/issue/g ! Oh, I see. I fixed the Makefile. This will propagate automatically next time I'll refetch the issue description and rebuild the reports. Thanks for spotting this!
Member

### kohlhase commented Sep 11, 2016

 There is still two ednotes in, one concerning a latex presentation bug for the listings, and one on the handling of constructors/assessors. these two are mostly for the journal version, we want to eventually make from this report. I have hidden them.
Contributor

### nthiery commented Sep 11, 2016

 Thanks for the feedback on authorship. I added the following to our README: Defining authorship is tricky, as most deliverable involve close collaboration with the community, and the report is often written by a subset of the contributors. Let's use the following simple rule of thumb: the authors of the report should include all persons funded by ODK that contributed non trivially. Not including outsiders in the author list is reasonable, as the report is about the contribution of ODK.''
Contributor

### nthiery commented Sep 11, 2016

 We probably want something stronger than "non trivially".
Contributor

### nthiery commented Sep 19, 2016

 I did some little finalization on the report: moving the abstract to the github description, reusing the same story as for D6.3, adding a link to the report, ... There just remains to possibly update the author list according to what we discussed above (who is considered as having worked on this deliverable). Then it's ready to submit.
Member

### kohlhase commented Sep 20, 2016

 I have updated the authors.
Contributor

### nthiery commented Sep 20, 2016

 On Mon, Sep 19, 2016 at 10:55:20PM -0700, Michael Kohlhase wrote: I have updated the authors. Thanks! pdf updated for D6.2 and D6.3.
Contributor

# Deliverables update

@kohlhase As you know , during the interim review in Bremen, reviewers gave advice to improve deliverables. UPSud tried to follow their advice by updating D5.1 #107
The main critic is that deliverables should be static document. Therefore we went through the #107 report to check if all links were still working and also if the information contained on the links was necessary for the understanding of the report.
As a result when it was necessary annexes were added to the #107 report.

Can you please do the same for this deliverable and add annexes when need be?

Closed

Contributor

### nthiery commented Jan 5, 2017

 Report finally submitted to the EU portal. I fixed the SageMath figures (there was a bug last june which caused duplicate categories to be exported). I also fixed undefined WP links.
Contributor

### nthiery commented Jan 5, 2017

 @kohlhase: I made a small edit to eudeliverablereport.cls to reimplement WPref and add longWPref to link to the relevant githup page (rather than attempting an internal link in the PDF as in the proposal file). Btw: the current implementations of taskref, longref, WPref in eudeliverablereport.cls are ODK specific; maybe we should move them to our own deliverablereport.cls, or find a way to abstract away the web addresses.
Member

### kohlhase commented Jan 5, 2017

 Btw: the current implementations of taskref, longref, WPref in eudeliverablereport.cls are ODK specific; maybe we should move them to our own deliverablereport.cls, or find a way to abstract away the web addresses. I think we (I actually) should make the git issues generation a regular feature of the proposal class. Then we can have a macro \prop@gitissues (or so; maybe make this a key in the \begin{proposal}) and use your definition if that exists. Otherwise we could just reference the proposal generating something like WP 6.2 in \cite{proposal}. What do you think? Then we could commit this back into LaTeX-proposal.
Member

### kohlhase commented Jan 5, 2017

 I made a small edit to eudeliverablereport.cls to reimplement @bpilorget do we need to resubmit to the EU here?
Contributor

### nthiery commented Jan 5, 2017

 The version I submitted today to the EU portal uses the latest style file; so we are good.

Contributor

### nthiery commented Jan 5, 2017

 I think we (I actually) should make the git issues generation a regular feature of the proposal class. Sounds good indeed! Then we can have a macro \prop@gitissues (or so; maybe make this a key in the \begin{proposal}) and use your definition if that exists. Otherwise we could just reference the proposal generating something like WP 6.2 in \cite{proposal}. What do you think? Then we could commit this back into LaTeX-proposal. I imagine that the typical reader will be more interested in accessing the actual WP description rather than tracking provenance information. So I'd say a direct link sounds better than a citation. Also web pages tend to be easier to navigate than a pdf file, which promotes linking to the page rather than the pdf. Just 2 cents though ...

Closed