Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

D2.3: Review on emerging technologies #43

Closed
minrk opened this issue Sep 8, 2015 · 67 comments
Closed

D2.3: Review on emerging technologies #43

minrk opened this issue Sep 8, 2015 · 67 comments

Comments

@minrk
Copy link
Contributor

@minrk minrk commented Sep 8, 2015

This deliverable reviews emerging technologies that did not exist, or were not sufficiently visible at the time the OpenDreamKit proposal was written. Its goal is to inform Work Packages on technologies that have a potential impact on the achievement of their tasks, and to suggest corrective actions to be undertaken when needed. A secondary goal for this review is to inform the general public about technologies related to OpenDreamKit.

Parts of this deliverable appeared, or will appear, on OpenDreamKit's weblog, and we will keep posting reviews there as new technologies emerge.

@minrk minrk added this to the D2.3 milestone Sep 8, 2015
@nthiery nthiery modified the milestones: Month 12: 2016-08-31, D2.3 Mar 22, 2016
@pdehaye
Copy link
Contributor

@pdehaye pdehaye commented Jun 30, 2016

@tkw1536
Copy link
Contributor

@tkw1536 tkw1536 commented Jul 4, 2016

  • How long should the blog posts be?
  • How / Where should we publish the blog post? I could not find any useful instructions in the README.
@defeo
Copy link
Contributor

@defeo defeo commented Jul 4, 2016

  • It depends how much you have to say. Try to write something interesting and engaging: you want people to come to the website to read it. Then, if it's too long, we'll cut it for the deliverable. If it's too short, we'll dilute it.
  • Blog posts go in the _post folder of the website: https://github.com/OpenDreamKit/OpenDreamKit.github.io, following Jekyll conventions. I will improve the README over the next weeks.
@VivianePons
Copy link
Contributor

@VivianePons VivianePons commented Aug 18, 2016

Reminder: it is good to have blog posts but we when say "collate" we should
really copy and paste the post content as we were told links are not
enough...

Best

Viviane

2016-08-18 18:26 GMT+02:00 Min RK notifications@github.com:

Review technologies and trends that evolved since the beginning of the
project and may influence our

Plan: write blog posts about such, and collate them in the report.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#43, or mute the
thread
https://github.com/notifications/unsubscribe-auth/ADcG4E6p_NFU1RT4Bn5WyNBS2lW8S9Lrks5qhIeegaJpZM4F5ylv
.

@bpilorget
Copy link
Contributor

@bpilorget bpilorget commented Aug 30, 2016

@defeo Have you begun compiling the different articles? FYI, Viviane is writing hers today

@bpilorget bpilorget closed this Aug 30, 2016
@bpilorget bpilorget reopened this Aug 30, 2016
@bpilorget
Copy link
Contributor

@bpilorget bpilorget commented Aug 30, 2016

The README gives info on how to write deliverables (scroll down): https://github.com/OpenDreamKit/OpenDreamKit

@defeo
Copy link
Contributor

@defeo defeo commented Aug 30, 2016

I only got feedback from @tkw1536, for the moment. I'm waiting for everything else. Ping to @embray, @minrk, @markuspf and @nthiery.

@markuspf
Copy link
Member

@markuspf markuspf commented Aug 30, 2016

There's basically nothing useful I can say about JupyterLab at the moment, other than "yeah, would be interesting as an HPC-GAP frontend."

@VivianePons
Copy link
Contributor

@VivianePons VivianePons commented Aug 30, 2016

I'm writting my piece, should be finished tomorrow.

2016-08-30 17:00 GMT+02:00 Luca De Feo notifications@github.com:

I only got feedback from @tkw1536 https://github.com/tkw1536, for the
moment. I'm waiting for everything else. Ping to @embray
https://github.com/embray, @minrk https://github.com/minrk, @markuspf
https://github.com/markuspf and @nthiery https://github.com/nthiery.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#43 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADcG4ObMOv7o_NHqtzQ3RwyLkbi96qVpks5qlEWBgaJpZM4F5ylv
.

@embray
Copy link
Collaborator

@embray embray commented Aug 31, 2016

Sorry, I'm going to need more time on this. I haven't even tried out the Ubuntu-on-Windows yet, though Bill Hart has written up some useful notes on his experience with it (as it pertains to Sage) so far, so maybe I can just draw from that.

@minrk
Copy link
Contributor Author

@minrk minrk commented Aug 31, 2016

We have had issues in Jupyter with Ubuntu on Windows, stemming from zeromq. It appears that the zeromq socket library when retrieving binaries from distributions such as conda, do not work on Ubuntu-on-Windows. However, people who are careful to use zeromq installed via apt-get, then things seem to work. That tells me that it's currently a very preliminary, unstable environment on which to work, where docker-on-Windows is more stable and reliable for users.

As for JupyterLab, there is not much to say about the thing itself - it's a nicer redesign of the existing notebook webapp with mainly the same capabilities. However, the way it's being built enables building different applications, such as making other non-notebook webpages (e.g. documentation) interactive.

@VivianePons
Copy link
Contributor

@VivianePons VivianePons commented Aug 31, 2016

Hi everyone,

@bpilorget I think it would be more reasonable to ask a delay on this one. October 1st seems ok. I don't have a better excuse than "it is the end of the summer and everyone is busy", we should have foreseen this and obviously, we didn't. Also, to be honest, I don't know who's ideas it was to include this deliverable but it seems that nobody had a clear idea what to do with it.

A delay won't have any impact on other tasks as this is just a survey. On the other hand, having something good will be valuable for the project, much better than if we quickly wrap up a quick report this week.

What do you think? (and @nthiery as well)

Best

Viviane

PS: my piece is started but not finished yet, as I said "it is the end of the summer and everyone is busy"

@defeo
Copy link
Contributor

@defeo defeo commented Aug 31, 2016

Do we need a one month delay? From what I gather, things are not going to be much more mature on the Windows/Ubuntu side in one month. As for JupyterLab, I guess we (@minrk and I?) can just extract from the official post on the Jupyer blog.
If we are late delivering by one or two weeks, I guess we don't need an official extension. Is this enough time for you @VivianePons ?

@VivianePons
Copy link
Contributor

@VivianePons VivianePons commented Aug 31, 2016

One or two weeks is definitely enough for me, I want to be finished with this as soon as possible. What do we do for Windows/Ubuntu? At this point, what are we going to include in the report?

@defeo
Copy link
Contributor

@defeo defeo commented Aug 31, 2016

@VivianePons
Copy link
Contributor

@VivianePons VivianePons commented Aug 31, 2016

So what are our emerging technologies so far?

@defeo
Copy link
Contributor

@defeo defeo commented Aug 31, 2016

So what are our emerging technologies so far?

  • SageMathCloud
  • Docker on Windows
  • JupyterLab
  • Cloud infrastructure ???

If people can think of something obvious I'm forgetting, let me know.

@VivianePons
Copy link
Contributor

@VivianePons VivianePons commented Aug 31, 2016

In anycase, we should notify the PO that we are going to be a bit late.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 19, 2016

I have just proofread the whole document, and made minor edits. It's in fact a very interesting read; thanks for the hard work! Maybe there could have been a few more reviews, but as long as we keep this watch for further emerging technologies, and writing posts about relevant ones, I believe we are all good.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 19, 2016

@defeo: About the author list: should e.g. @minrk be there? (I don't know who wrote the JupyterLab part). If you feel like it you may add me for the binder section + proofreading, but both were minor actions.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 19, 2016

@VivianePons: About SageMathCloud:

On top of the usual interface, SMC's Jupyter offers real time synchronization among multiusers
This does not include real time collaborative editions, right? Maybe this should be clarified.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 19, 2016

Once those two items are taken care off, this will be ready for submission.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 19, 2016

Just to keep this info around for history:

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 19, 2016

@bpilorget: could you please browse through my diff, and implement the changes I made in the corresponding blog posts (not all them are up yet)? Thanks.

@defeo
Copy link
Contributor

@defeo defeo commented Sep 19, 2016

About the author list: should @minrk be there? (I don't know who wrote the JupyterLab part).

I wrote it, taking excerpts from

http://blog.jupyter.org/2016/07/14/jupyter-lab-alpha/

(I think the excerpts are within the limits allowed by reasonable
citation practice)

If you feel like it you may add me for the binder section + proofreading, but both were minor actions.

Sure.

About SageMathCloud:

On top of the usual interface, SMC's Jupyter offers real time synchronization among multiusers
This does not include real time collaborative editions, right? Maybe this should be clarified.

Dunno. @VivianePons?

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 19, 2016

We just checked with @VivianePons on one of her test SMC project. SMC uses a patched version of Jupyter that indeed allows real time collaborative edits. It just has a slightly lower granularity then for SMC notebooks: it's more at the level of the cell than the character.
This is documented here: https://github.com/sagemathinc/smc/wiki/sagejupyter#ipython-nb
We don't know if the patch was submitted or discussed with upstream.
@minrk: if not already done, please double check on this with @williamstein.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 19, 2016

Thanks Luca for the feedback!

@VivianePons
Copy link
Contributor

@VivianePons VivianePons commented Sep 19, 2016

Following the discussion with Nicolas, I added an extra paragraph on two interesting aspects:

  • real time synchro on Jupyter notebook on SMC
  • Jupyer kernels used through the SMC worksheet
    I added it both on the latex and on the md file for the blog posts.

I also moved the post from _activites to _posts (knowing that anyway, I intend to merge those two in the near future).

At the moment, the publishing date is still the same. We should change it when we decide to actually post it.

@bpilorget
Copy link
Contributor

@bpilorget bpilorget commented Sep 19, 2016

could you please browse through my diff, and implement the changes I made in the corresponding blog posts (not all them are up yet)? Thanks.

@nthiery I could only find the Docker for Mac and Windows post. Do you know where the other documents are?

@defeo
Copy link
Contributor

@defeo defeo commented Sep 19, 2016

@williamstein
Copy link
Contributor

@williamstein williamstein commented Sep 19, 2016

Heh, I'm here and can answer any questions about this. @haraldschilly can also.

SMC uses a patched version of Jupyter that indeed allows real time collaborative edits.

More details: the "patching" is monkey patching that happens entirely client side at the Javascript level. No original source code of Jupyter is changed. Instead, after the Jupyter iframe loads, some javascript is executed to add some hooks.

It just has a slightly lower granularity then for SMC notebooks: it's more at the level of the cell than the character.

Sync should work at the exact same level of granularity as SMC worksheets.

You might have been confused by the source, since there are two levels of diffing in the source code, one involving the cells and another involving their content. The cell-level diffing is used entirely to optimize updating the Jupyter HTML/DOM. If you know about React.js (or other "virtual DOM" libraries like vue.js), this is the same idea (except I first implemented it long ago before react was released, so it isn't nearly as good -- it's just a standard trick these days)...

This is documented here: https://github.com/sagemathinc/smc/wiki/sagejupyter#ipython-nb
We don't know if the patch was submitted or discussed with upstream.

At present there's no patch to their code to submit. Realtime sync is a little more complicated than just what goes on inside https://github.com/sagemathinc/smc/blob/master/src/smc-webapp/editor_jupyter.coffee; there's a whole "environment" around that, involving backend servers, etc., which has to be implemented somehow too...

There is a hackpad all about sync that Kyle Kelly and other Jupyter devs have been working on for a while, which I've contributed a lot to: https://nteract.hackpad.com/Realtime-collaboration-Ltj13iXQnQa
I don't know if this is public (?) -- login is definitely required at least.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 19, 2016

On Mon, Sep 19, 2016 at 09:43:38AM -0700, William Stein wrote:

Heh, I'm here and can answer any questions about this.

Great, thanks a lot for the details, that's helpful; I wasn't sure
you'd have time to jump in. Glad to hear as well that their are
already plenty of discussions going on on that topic. Then we can
focus our energy elsewhere :-)

Sync should work at the exact same level of granularity as SMC
worksheets.

You might have been confused by the source, since there are two levels
of diffing in the source code, one involving the cells and another
involving their content. The cell-level diffing is used entirely to
optimize updating the Jupyter HTML/DOM. If you know about React.js (or
other "virtual DOM" libraries like vue.js), this is the same idea
(except I first implemented it long ago before react was released, so
it isn't nearly as good -- it's just a standard trick these days)...

This impression came just from using it. SMC felt more reactive; it
may well have just been some unrelated side effect of latency. It's
good to know that there is no intrinsic difference.

Cheers,
Nicolas

@williamstein
Copy link
Contributor

@williamstein williamstein commented Sep 20, 2016

This impression came just from using it. SMC felt more reactive; it
may well have just been some unrelated side effect of latency. It's
good to know that there is no intrinsic difference.

I may have set a throttling or debouncing parameter differently to reduce
potential or perceived slow-down during usage. But it still operates at
the character level, not cell level.

William

@defeo
Copy link
Contributor

@defeo defeo commented Sep 21, 2016

Corrected date and authors and submitted to the EU portal. Thanks to everyone.

@defeo defeo closed this Sep 21, 2016
@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 22, 2016

Good to know that you can upload on the EU portal (apparently Benoît could not nor Michael). One less thing I have to manage myself. Thanks!
The reason I had not uploaded earlier is that I was waiting for feedback from our PO about the style of our latest deliverables w.r.t. the reviewer's request. But that's allright.

@defeo
Copy link
Contributor

@defeo defeo commented Sep 22, 2016

@nthiery
Copy link
Contributor

@nthiery nthiery commented Sep 22, 2016

On Thu, Sep 22, 2016 at 02:05:55AM -0700, Luca De Feo wrote:

It still says "draft" on the portal, I guess you have to validate it
anyway.

Ah perfect.
Next time, let's close it once it's completely submitted.

Cheers

@defeo
Copy link
Contributor

@defeo defeo commented Sep 22, 2016

Nothing forbids re-opening the issue.

@defeo defeo reopened this Sep 22, 2016
embray added a commit to embray/OpenDreamKit.github.io that referenced this issue Oct 4, 2016
@bpilorget
Copy link
Contributor

@bpilorget bpilorget commented Oct 24, 2016

Deliverables update

@defeo As you know , during the interim review in Bremen, reviewers gave advice to improve deliverables. UPSud tried to follow their advice by updating D5.1 #107
The main critic is that deliverables should be static document. Therefore we went through the #107 report to check if all links were still working and also if the information contained on the links was necessary for the understanding of the report.
As a result when it was necessary annexes were added to the #107 report.

Can you please do the same for this deliverable and add annexes when need be?

@defeo
Copy link
Contributor

@defeo defeo commented Nov 2, 2016

Can you please do the same for this deliverable and add annexes when need be?

@bpilorget, I confirm that this deliverable is ready and complete, and needs no appendices.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Jan 4, 2017

Report submitted on the EU portal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.