Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

D6.2: Initial DKS base Design (including base survey and Requirements Workshop Report) #136

Closed
minrk opened this issue Sep 8, 2015 · 78 comments

Comments

@minrk
Copy link
Contributor

@minrk minrk commented Sep 8, 2015

The OpenDreamKit proposal had envisioned WP6: Data/Knowledge/Software bases as a foundational enterprise that would develop a knowledge-based architecture over the course of the project and would allow to re-engineer ad-hoc interfaces between systems (e.g. from T3.2 (#51)) on a more semantic basis -- the knowledge aspect (K). Consequently, the proposal envisioned concentrating the data (D) aspect on the mathematical knowledge bases (specifically LMFDB, OEIS, and FindStat) and proposed a host of foundational investigations of mathematical for the software (S) aspect with applications e.g. in the verification of algorithms.

Already the kickoff meeting in Paris in September 2015 revealed that the D/K/S aspects are much more tightly coupled in systems than anticipated. This was confirmed by the DKS survey conducted subsequently. In particular, the participants of WP6 identified the interoperability of OpenDreamKit systems to be one of the most critical steps in creating a VRE toolkit. Thus we prioritized tasks T6.1 (#123), T6.2 (#124), T6.3 (#125) and organized a series of workshops and code-maratons to develop a semantic foundation for system interoperability and simultaneously test it in implementations.

As a consequence, we have completed the initial design of D/K/S-bases (for this deliverable) in parallel with the initial implementation of a DKS base format based on OMDoc/MMT and the implementation of a DKS base system itself based on the MMT system (both for D6.3 (#137)), all activities fuelling each other. D6.3 (#137) was thus completed about three months ahead of schedule. Note that the RNC schema envisioned in the title proved un-necessary since, with the refined Math-in-the-Middle (MitM) design, the normal OMDoc/MMT schema is sufficient. Due to the resulting tight coupling between this deliverable and D6.3 (#137), and for the convenience of the reader, we have decided to report on both deliverables together.

In this report we therefore present the design process towards DKS-theories including the overall architecture (for this deliverable), a survey of the systems involved (for D6.3 (#137)), our current implementation (for D6.3 (#137)) as well as our plans for the future. Each part is labeled by the deliverable they contribute to mainly.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Jun 30, 2016

this is a sister deliverable to #137 and will be largely done together with that. We are making progress on this in the WP6 workshops. I think that @pdehaye should be also involved in writing this up.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Aug 21, 2016

@pdehaye you made a survey in Paris and St. Andrews I recall, could you please write this up as '(including base survey and Requirements Workshop Report)'. I would be very grateful.
cc: @tkw1536

@tkw1536
Copy link
Contributor

@tkw1536 tkw1536 commented Aug 24, 2016

I have started writing this up properly. You can find my progress so far at

https://github.com/KWARC/OpenDreamKit/tree/master/WP6/D6.2

@pdehaye So far I have not referenced the survey, I will probably get some work done on that this afternoon. Feel free to insert it already.
@kohlhase I had to switch from bibtex to biber, as for some reason the bst file was broken. If you want me to switch back, let me know.
@florian-rabe: It would be great if I could get some help from you regarding the overall structure and framework of the report.

Edit: Updated link because we changed branches

@pdehaye
Copy link
Contributor

@pdehaye pdehaye commented Aug 25, 2016

To give a summary of the current situation: the survey is scattered in #123, other issues referenced from #123, and here. Far from ideal, but a good starting point.
I am starting to work on this now, first by just dumping the scattered data into the file you are working on (in parallel to installing Latex on my new computer, which could become a blocker...)

@tkw1536
Copy link
Contributor

@tkw1536 tkw1536 commented Aug 27, 2016

@pdehaye I have given you push access to the KWARC fork. I will do a bunch more writing today, I will probably pull in your changes after that.

@pdehaye
Copy link
Contributor

@pdehaye pdehaye commented Aug 27, 2016

I already pushed to KWARC's repo

On Sat, Aug 27, 2016 at 10:16 AM, Tom Wiesing notifications@github.com
wrote:

@pdehaye https://github.com/pdehaye I have given you push access to the
KWARC fork. I will do a bunch more writing today, I will probably pull in
your changes after that.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#136 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADH2X4SgkQS6u8eZUwqN5NCZ0PR134Iqks5qj_J5gaJpZM4F5zNI
.

@tkw1536
Copy link
Contributor

@tkw1536 tkw1536 commented Aug 27, 2016

I was working on the D6.2 branch, so I didn't even see that. I will just move to master.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Aug 27, 2016

I already pushed to KWARC's repo

@pdehaye I think we should organize this so that the survey itself is given as an appendix to the report and instead of the current section there is a short description of what we learn from this survey. That gives us a self-contained report (by the appendix) and the high-level requirements we need to justify the DK(S) theories.

@pdehaye
Copy link
Contributor

@pdehaye pdehaye commented Aug 27, 2016

That's a good idea.

On 27 Aug 2016, at 14:38, Michael Kohlhase notifications@github.com wrote:

I already pushed to KWARC's repo

@pdehaye I think we should organize this so that the survey itself is given as an appendix to the report and instead of the current section there is a short description of what we learn from this survey. That gives us a self-contained report (by the appendix) and the high-level requirements we need to justify the DK(S) theories.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Aug 27, 2016

That's a good idea.

if you pull, then @tkw1536 has already put your part into a new file in the appendix.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Aug 27, 2016

@florian-rabe should also subscribe to this.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Aug 27, 2016

Problem I am a bit worried about the fact that we have promised a design of DKS theories for this report and we are mostly reporting only on DK theories.
Background Even though we have studied biform theories in OMDoc, we have not worked on this aspect much in the first 12 months, since we have concentrated on the MitM approach to interoperability.
Solution? I have thought about this in the last days, and I think that we can (and should) think of the MitM information architecture (think of the clouds picture with the interface theory graphs that are generated from the systems like GAP or Sage and the hand-curated MitM theory in the middle) can be seen as a integration of the S (for Software) component into DKS theories -- actually we do not get "DKS theories", as forseen in the proposal, but "DKS theory graphs", where we have

  • K-Theories (regular MMT theories) for the specifications and math knowledge,
  • DK-theories (as we introduce them in D6.2) for models and concrete mathematical structures (e.g. OEIS, LMFDB, FindStat), and
  • the alignments and interviews (they are really interpretations and implementation mappings) as the integration of the S component.
    If we then move our focus from verification of algorithms (only mentioned in one sentence at the end of T6.8) towards interoperability, then do not need to build on biform theories, but can build on the MitM architecture instead. I think that this is legitimate, and actually this shift in perception is a major result of our work in ODK.
    I would like to have some feedback on this, and if that is positive, I would try to extend the report to base itself on this "initial design".
@tkw1536
Copy link
Contributor

@tkw1536 tkw1536 commented Aug 28, 2016

@kohlhase I have rewritten the introduction towards what you suggested. Apart from adding a figure to illustrate the MiTM architecture it is completed. It would be great to get some feedback.

@pdehaye
Copy link
Contributor

@pdehaye pdehaye commented Aug 29, 2016

Does anybody know how to make "paragraph" behave like "subsubsubsection" ?

\setcounter{secnumdepth}{4}
seems to go part of the way, but it needs a newline at the end, and maybe a
change of font.

I can Google it, but the answers I find are for the general "article", and
don't really work for "deliverablereport".

It's necessary for the survey as it has several layers of nesting:

appendix / system / {data, knowledge, software} / question
section / subsection / subsubsection / paragraph

Thanks

Paul

On Mon, Aug 29, 2016 at 12:19 AM, Tom Wiesing notifications@github.com
wrote:

@kohlhase https://github.com/kohlhase I have rewritten the introduction
towards what you suggested. Apart from adding a figure to illustrate the
MiTM architecture it is completed. It would be great to get some feedback.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#136 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADH2X7BrOjFkDMXDNxSgLrMWQMtiWqXIks5qkglvgaJpZM4F5zNI
.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Aug 29, 2016

actually, deliverablereport is based on amsart.cls, maybe you should try that in your query.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Aug 29, 2016

actually, I would like to use article.cls in in any case. Maybe we can change deliverablereport.cls?

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Aug 29, 2016

actually, I would like to use article.cls in in any case.

that is not trivial we need to handle the abstract (amsart does that well)

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Aug 29, 2016

I have tweaked the figures a bit and snippetized the sections, please pull, if you are working on this.

@pdehaye
Copy link
Contributor

@pdehaye pdehaye commented Aug 29, 2016

Thanks, basing myself on amsart seems like a good start. Needs more tweaking from me though. My macros for the packages introduced too many spaces with \,, but those were needed in some cases. Switched to using xspace package.

@pdehaye
Copy link
Contributor

@pdehaye pdehaye commented Aug 29, 2016

I have finished a first pass at the formatting and proofreading of the text of the survey. It is still very coarse. I still need to do a second pass and add a summary in the text.

The survey for GAP is still empty. As much as I remember, it was never done formally. I see a few options:

  • ask a GAP developer to fill it now;
  • I read the recent papers describing GAP architecture and fill the survey;
  • I simply refer to the recent papers

Honestly I think option 3 is best. It won't exactly fit the format of the others, but will have the same content. What do you think?

@pdehaye
Copy link
Contributor

@pdehaye pdehaye commented Sep 10, 2016

There is also the pesky problem of author ordering. I absolutely don't care except I want to make sure it is intentionally non alphabetical.

On 10 Sep 2016, at 11:18, Michael Kohlhase notifications@github.com wrote:

Anyways, I have made the necessary changes now

excellent. I made a first pass over explaining the "two-deliverables-in-one-report" design. I will make a first D6.3 deliverable stub now.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 10, 2016

On Sat, Sep 10, 2016 at 01:24:49AM -0700, pdehaye wrote:

There is also the pesky problem of author ordering. I absolutely don't
care except I want to make sure it is intentionally non alphabetical.

Just being curious: why not alphabetical? It's the customary way to
resolve author ordering in the math community, isn't it?

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Sep 10, 2016

I have also not thought about it, but I will make it alphabetical. I am working on these things ATM.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Sep 10, 2016

I think that the order was just that everyone just added themselves at the end, no malice or so here.
The converse would have surprised me :-) Just being curious about @pdehaye's comment.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 10, 2016

Speaking of authors: for our reports, should the authors be the people having written the report itself, or should it include those that did major work on the content? (brainstorm and implementations here, implementation elsewhere, ...). I am wondering because of other reports like D4.4 (#93), where I'll probably write the bulk of the report, while most of the implementation was done by others.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Sep 10, 2016

I have no clear opinion on this, and I think we should have a general "rule" so I guess you (as the coordinator) should say something public about this.
I guess my personal preference to have as authors the people who wrote the report, and make the contributors clear in the report (acknowledgements?)

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Sep 10, 2016

OK, I have taken care of all the ednotes. From my point of view, this deliverable is done.

@fangohr
Copy link
Contributor

@fangohr fangohr commented Sep 10, 2016

Re authors, I'd include everybody who has contributed to the task/deliverable. The report is meant to represent the work done, so it should include the people involved. (If the categories of 'author' and 'contributor' are suggested, then @kohlhase 's suggestion seems reasonable, otherwise I'd just use one category [i.e. authors or contributors], and put everybody into that.)

@pdehaye
Copy link
Contributor

@pdehaye pdehaye commented Sep 10, 2016

re: authors, I didn't mean to imply there was malice. Apologies that it came out that way.

This is starting to look finished. There is still two ednotes in, one concerning a latex presentation bug for the listings, and one on the handling of constructors/assessors.

There is also one typo in the github issue descriptions. It should be "issue", not "issue's". I tried changing it but there seems to be two layers of Makefile's and I didn't fully understand the system used. In the end I left it as is, for @kohlhase to fix.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 10, 2016

On Sat, Sep 10, 2016 at 12:44:30PM -0700, pdehaye wrote:

There is also one typo in the github issue descriptions. It should be
"issue", not "issue's". I tried changing it but there seems to be two
layers of Makefile's and I didn't fully understand the system used. In
the end I left it as is, for [1]@kohlhase to fix.

You just need to fix the issue description on github; I'll upload the
latest version anyway upon submitting the deliverable.

Cheers,

Nicolas

Nicolas M. Thiéry "Isil" nthiery@users.sf.net
http://Nicolas.Thiery.name/

@pdehaye
Copy link
Contributor

@pdehaye pdehaye commented Sep 10, 2016

The typo is in the template that processes the GitHub issue description and
introduces it in the latex. It seems the typo originates in the Makefile
itself, but then was propagated. My sed skills are not up to par to do a
global fix: hard to the escaping correctly on s/issue's/issue/g !

On Sat, Sep 10, 2016 at 10:10 PM, Nicolas M. Thiéry <
notifications@github.com> wrote:

On Sat, Sep 10, 2016 at 12:44:30PM -0700, pdehaye wrote:

There is also one typo in the github issue descriptions. It should be
"issue", not "issue's". I tried changing it but there seems to be two
layers of Makefile's and I didn't fully understand the system used. In
the end I left it as is, for [1]@kohlhase to fix.

You just need to fix the issue description on github; I'll upload the
latest version anyway upon submitting the deliverable.

Cheers,

Nicolas

Nicolas M. Thiéry "Isil" nthiery@users.sf.net
http://Nicolas.Thiery.name/


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#136 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADH2XwONlMw68WlWdoB6kX0AqYJKcJwvks5qow6sgaJpZM4F5zNI
.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 11, 2016

On Sat, Sep 10, 2016 at 01:25:31PM -0700, pdehaye wrote:

The typo is in the template that processes the GitHub issue description
and
introduces it in the latex. It seems the typo originates in the
Makefile
itself, but then was propagated. My sed skills are not up to par to do
a
global fix: hard to the escaping correctly on s/issue's/issue/g !

Oh, I see. I fixed the Makefile. This will propagate automatically
next time I'll refetch the issue description and rebuild the reports.

Thanks for spotting this!

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Sep 11, 2016

There is still two ednotes in, one concerning a latex presentation bug for the listings, and one on the handling of constructors/assessors.

these two are mostly for the journal version, we want to eventually make from this report. I have hidden them.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 11, 2016

Thanks for the feedback on authorship. I added the following to our README:
``Defining authorship is tricky, as most deliverable involve close collaboration with the community, and the report is often written by a subset of the contributors. Let's use the following simple rule of thumb: the authors of the report should include all persons funded by ODK that contributed non trivially. Not including outsiders in the author list is reasonable, as the report is about the contribution of ODK.''

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 11, 2016

We probably want something stronger than "non trivially".

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 19, 2016

I did some little finalization on the report: moving the abstract to the github description, reusing the same story as for D6.3, adding a link to the report, ...
There just remains to possibly update the author list according to what we discussed above (who is considered as having worked on this deliverable). Then it's ready to submit.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Sep 20, 2016

I have updated the authors.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 20, 2016

On Mon, Sep 19, 2016 at 10:55:20PM -0700, Michael Kohlhase wrote:

I have updated the authors.

Thanks! pdf updated for D6.2 and D6.3.

@bpilorget
Copy link
Contributor

@bpilorget bpilorget commented Oct 24, 2016

Deliverables update

@kohlhase As you know , during the interim review in Bremen, reviewers gave advice to improve deliverables. UPSud tried to follow their advice by updating D5.1 #107
The main critic is that deliverables should be static document. Therefore we went through the #107 report to check if all links were still working and also if the information contained on the links was necessary for the understanding of the report.
As a result when it was necessary annexes were added to the #107 report.

Can you please do the same for this deliverable and add annexes when need be?

@nthiery
Copy link
Contributor

@nthiery nthiery commented Jan 5, 2017

Report finally submitted to the EU portal. I fixed the SageMath figures (there was a bug last june which caused duplicate categories to be exported). I also fixed undefined WP links.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Jan 5, 2017

@kohlhase: I made a small edit to eudeliverablereport.cls to reimplement WPref and add longWPref to link to the relevant githup page (rather than attempting an internal link in the PDF as in the proposal file).

Btw: the current implementations of taskref, longref, WPref in eudeliverablereport.cls are ODK specific; maybe we should move them to our own deliverablereport.cls, or find a way to abstract away the web addresses.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Jan 5, 2017

Btw: the current implementations of taskref, longref, WPref in eudeliverablereport.cls are ODK specific; maybe we should move them to our own deliverablereport.cls, or find a way to abstract away the web addresses.

I think we (I actually) should make the git issues generation a regular feature of the proposal class. Then we can have a macro \prop@gitissues (or so; maybe make this a key in the \begin{proposal}) and use your definition if that exists. Otherwise we could just reference the proposal generating something like WP 6.2 in \cite{proposal}. What do you think? Then we could commit this back into LaTeX-proposal.

@kohlhase
Copy link
Member

@kohlhase kohlhase commented Jan 5, 2017

I made a small edit to eudeliverablereport.cls to reimplement

@bpilorget do we need to resubmit to the EU here?

@nthiery
Copy link
Contributor

@nthiery nthiery commented Jan 5, 2017

The version I submitted today to the EU portal uses the latest style file; so we are good.

@nthiery nthiery closed this Jan 5, 2017
@nthiery
Copy link
Contributor

@nthiery nthiery commented Jan 5, 2017

I think we (I actually) should make the git issues generation a regular feature of the proposal class.

Sounds good indeed!

Then we can have a macro \prop@gitissues (or so; maybe make this a key in the \begin{proposal}) and use your definition if that exists. Otherwise we could just reference the proposal generating something like WP 6.2 in \cite{proposal}. What do you think? Then we could commit this back into LaTeX-proposal.

I imagine that the typical reader will be more interested in accessing the actual WP description rather than tracking provenance information. So I'd say a direct link sounds better than a citation. Also web pages tend to be easier to navigate than a pdf file, which promotes linking to the page rather than the pdf.

Just 2 cents though ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants
You can’t perform that action at this time.