D3.5: Integration between SageMathCloud and Sage 's TRAC server #64
Comments
|
@defeo and I are discussing what to do for this deliverable, and it's hard to be exactly sure at this point. As Luca tells me, part of the original background to this goes back to part of the reason William developed SMC in the first place--that he wanted to be able to develop Sage on his Chromebook. The idea was that one might do development of Sage itself in SMC, in which case it would be occasionally useful to have integration directly to Sage's Trac. However, this interpretation is very Sage-specific. In fact Sage is one of the few projects involved in OpenDreamKit that even uses Trac; a more useful interpretation might involve issue trackers in general, not just Trac, and not just Sage's Trac instance. Further, the scope of SageMathCloud/CoCalc has long since surpassed just working on Sage. While it's certainly possible to do development of Sage itself within a CoCalc project, it's not clear that many if any of Sage's core developers are actually doing that. Even for the use case of doing Sage development on a low-power machine, I for example have a more powerful machine I can just SSH into. That's essentially what CoCalc is providing, with a nice web UI on top, but the web UI is not needed for this use case either. Furthermore, even for users who might want to development mathematical software that uses Sage, we want to push them more in the direction of developing third-party modules first, before considering direct integration into Sage (where possible). So it doesn't necessarily make sense to make this specific to Sage, or specific to SageMathCloud--rather, this task should be thought about in a slightly bigger picture. To quote the proposal, "we will experiment with integrating bug reporting and contributing features right in the VRE". That's already a broader statement--it then sites this deliverable as an experiment in that direction. But even as an experiment I don't think it makes total sense to focus specifically on SMC+Sage+Trac. One issue with "integrating bug reporting and contributing features" is, to whom are we reporting within a given VRE? Say, for example, we have a VRE consisting primarily of some notebooks built on Sage. The primary users might be researchers in some specific field, or even students. They might not know how the guts of the VRE work, or even how to produce a sensible bug report for Sage itself. If the VRE involves multiple software systems, as they often will, it's even less clear where to send reports to. However, as VREs may be often shared and used in a collaborative fashion, I do think some manner of bug/enhancement reporting for VRE's themselves might be useful. Although there's progress to be made, the open research community is increasingly comfortable with sharing their work online through source repositories, such as GitHub. So let's take for example GitHub, and a VRE built out of one or more Jupyter notebooks. This is increasingly common, especially now that GitHub can render Jupyter notebooks in its source browser. It would be useful, given a Jupyter notebook, for it to have some baked in metadata concerning where it came from--particularly a link to a main repository for that notebook. If this is stored as metadata in the notebook, then all kinds of things could be done with it. For example, a menu item can be added linking directly to the GitHub issue tracker for that notebook. Another idea we had is that when a traceback occurs in the notebook, it could be displayed with a link to submit that traceback as a bug report to the issue tracker for that notebook (it is possible with most issue trackers, including GitHub, to pre-fill an issue, such as with a title, description, and possibly label(s) indicating that it was an automated submission--it could also automatically include some (anonymous) system information from the submitter). Then it can be up to the original author(s)/community of users for that notebook what to do about that particular crash--whether it's something to be fixed in the VRE itself, or an issue to be submitted to an upstream project. An enhancement on top of that would be to include some level of intelligence to automated issue submission, such as detection of duplicate issues. This might require some involvement on the level of individual kernels. For example, detecting duplicate Python tracebacks might require some level of understanding of the Python traceback format. Luca had the idea that individual Jupyter kernels could plug into an issue-tracker interface. For example if a Sage notebook does not specify its own issue tracker, it could provide a link to Sage's issue tracker as a default fallback. (Here I should be clear that nothing about this feature needs to be GitHub-specific, though GitHub support should be made easy as a common use-case). Anyways, we're just sort of spit-balling ideas here. How does all this sound to you? @nthiery, @fcayre |
|
Hi Erik! Typical scenario: Alice is working in the VRE, reads some (Sage/...) documentation, spots a typo, has a suggestion to fix it. We would want a very simple workflow to submit and review the fix. Of the kind "Edit on github". |
|
I mean, for Sage specifically, I don't really see that as a "VRE" issue at all, or at least not as much. All we really want is an "Edit on GitHub" link for Sage's HTML docs. Hypothetically something similar in the interactive Python docs as well, though I'm not exactly sure what that would look like. |
|
For anything other than documentation this becomes, I think, non-trivial. Though I suppose for functions too it could be possible to have a way to go directly to their source code. |
|
On Fri, Jun 30, 2017 at 07:57:56AM +0000, Erik Bray wrote:
I mean, for Sage specifically, I don't really see that as a "VRE" issue
at all, or at least not as much. All we really want is an "Edit on
GitHub" link for Sage's HTML docs. Hypothetically something similar in
the interactive Python docs as well, though I'm not exactly sure what
that would look like.
I agree that it's not tightly coupled to the VRE.
Small things that a VRE integration could bring:
- "Edit on GitHub" (or trac/...) requires to login on github, and
therefore have an account there. Since the user is already
authenticated on the VRE, that could be used as credentials in the
workflow.
- The edition could take place within the VRE, letting the user test
himself and immediately benefit from his own changes
Within jupyter(lab), one could imagine something like:
%%edit sage.foo.bar
...
%%submit sage.foo.bar
Just throwing ideas in the air ...
|
|
That seems like a possible lot of work with many potential pitfalls for not much benefit. How many people are really going to edit source code or even documentation on jupyter lab, as opposed to the editors they already have working locally? I could see this as potentially useful on a small scales for users wanting to submit issues or small changes to a notebook, but not serious editing of source code (especially for anything across multiple files). |
|
@defeo This deliverable is officially due for 31/08/2017. Can you please report on the progress you have achieved? |
|
Last week @nthiery and I discussed a possible different direction for this deliverable, involving developing better integration between GitHub and Sage's Trac--so doing would potentially enable easier editing of Sage's documentation, among other things. This new direction wouldn't necessarily relate to SageMathCloud, but in retrospect integration with SMC is not necessarily the best way to achieve the original goals of this task. |
|
@embray, what's the status of this deliverable? Do you need help to finalize it? |
|
Regarding better integration between GitHub and Sage's Trac, See a 2016-07 discussion on the sage-devel mailing list: |
|
Regarding using CoCalc for Sage development, especially at Sage days, |
|
Hi, I just saw this deliverable/issue for the first time. I don't understand what the original goal of this is, and I'm also not really sure what the outcome of this discussion is. In particular, I don't think it makes any sense to help editing files "directly". However, what I personally think would help in general, is a streamlined way to report issues -- in a much more informal high-level way. I.e. when a user of a software tool noticies a problem (might be a bug, might be a usage error, etc.), they have a single point to contact someone. That report has these basic properties and tasks:
|
|
I believe this deliverable remains plagued by ill definition. While some of us (e.g. I and @saraedum) have been (slowly) making progress on trying to make it easier to make new contributions to Sage, that work is very specific to Sage and making its development process more accessible. It doesn't really have anything to do with mathematical software in general or have much applicability outside Sage (though perhaps some other projects might be able to learn from our experience, albeit how much I'm not sure). That work also has nothing directly to do with "VRE"s as such (though moving Sage's development to a more GitHub/GitLab-like model as we're trying to work towards will make hypothetical VRE<=>development integration easier for Sage in the future). @haraldschilly What you're talking about seems to fall more under the rubric of a "help desk" (Zendesk; that sort of thing) albeit with some sort of notebook integration. But how would that work? Who would provide first level support for such a thing? Would it depend on the project(s) a user is using? If a user is making a notebook that uses, say, TensorFlow and Sage, and they need help, whose help desk does it go to? I don't think this ticket is really about general user support, though I agree that for a typical user who thinks they might have found a bug, simply throwing them at the development issue tracker is not often the right answer either. The description of task T3.7 which encapsulates this deliverable also mentions "contributing features right in the VRE". But features of what?
|
It is.
Worse case, the deliverable will be about your experience, and how it can benefit other projects.
I say, let's not try to stick too close to something that was written 3 years ago, based on a half-formed idea. The spirit of the task should guide our work, let's do something useful, but let's not waste resources by trying to interpret the wording of the proposal. |
I may already have talked about this with some of you. I have been thinking for a while of having a field in Jupyter's Then a Jupyter add-on (although this is not really useful if it doesn't get installed by default) could help forming a bug report and sending it to that url. The add-on could:
There could even be some other deeper hooks that trigger a notification. For example, any exception trace could be accompanied by a small message "Think this is a bug? Report it". Jupyter makes it easy to do this in a uniform way for many systems. Writing a prototype could be done rather quickly. However not much of this makes sense outside of Jupyter, unless we want to write a standard describing how the HTTP request should be formed. But this seems too simple to deserve writing a standard. |
Of course, that's a tricky question. My proposal says that the kernel is responsible for receiving bug reports. This makes some sense for Sage, but not at all for Python. One can imagine that the kernel may have an internal mechanism to dynamically decide what the reporting URL should be. For example ipython may allow packages to set the URL at import time. It could form a list of packages that have registered a reporting URL, and give the user the choice to which component to send the report. |
While there's some potential value in tying something like this to the kernel itself (as in the case of the Sage kernel), I think in general that's not going to be useful for a number of reasons which I've mentioned here before. Namely: Who do the bug reports go to? Surely, we do not want the standard Python kernel to send bugs to bugs.python.org, as the majority of even suspected bugs that someone might encounter in a notebook is not going to be an upstream Python bug. Within a given notebook a whole slew of technologies might be used, and a given bug report might be applicable to any one of them, or none of them. That's why I proposed maybe having some kind of "homepage" metadata in the metadata of individual notebooks. This would be a homepage for a notebook itself, indicating its upstream origin. If this is a GitHub/GitLab/Bitbucket/etc. repository that could also maybe include a quick link for submitting issues against the notebook's repository (I would be hesitant to go overboard with this though--I don't think we want to end up re-implementing Mylyn in Jupyter (or maybe we do, but that's a big project). |
While I like the idea of, say, individual packages providing metadata like this (some do in fact; e.g. if you look up the distribution metadata for Python packages, albeit not in a very consistent manner), this is likely to be overwhelming to users. Most researchers who encounter a problem, unless they are already experts, have no idea what lower-level library (if any) their problem pertains to. Maybe the issue is entirely user error (which is what I think @haraldschilly was getting at). It would almost be more useful to give them a form to create a well-formed StackOverflow question, but that's not always the most appropriate forum either (in fact, although SO is working on cleaning up their culture, I wouldn't want to toss unwary VRE users to the sharks...) But certainly when it comes to integrating issue reporting in notebooks or other VRE-like environments, I think we need to think at a higher level than is our impulse as software development "experts". |
|
On Thu, Jun 28, 2018 at 06:15:01AM -0700, Luca De Feo wrote:
One can imagine that the kernel may have an internal mechanism to dynamically decide what the reporting URL should be. For example ipython may allow packages to set the URL at import time. It could form a list of packages that have registered a reporting URL, and give the user the choice to which component to send the report.
We use http://getsentry.com/ when we want the application to report
its bugs without having to ask the user to do something.
Maybe it could be a source of inspiration in this case?
…--
Nicolas Chauvat
logilab.fr - services en informatique scientifique et gestion de connaissances
|
|
@nchauvat Any chance you could expand on that? Who is "we" and how are you using it? |
|
On Fri, Jun 29, 2018 at 05:08:36AM -0700, E. M. Bray wrote:
Something like sentry is pretty explicitly for applications things
though, so the question becomes what kinds of errors to report. I
suppose in the case of sage we would at least want to report
start-up errors, and also crashes of the kernel itself. Even those
would not all be of interest (some of them may be perfectly
"expected").
If you take a look at https://docs.sentry.io/clients/ and
https://docs.sentry.io/integrations/ you will see that sentry has a
wide range of communication skills.
One could imagine to plug it into jupyter/binder both server-side and
js/client-site, to plug it into various kernels and to plug it into
notebooks with a special %directive.
On the other end, it integrates with several free-software
forges. Trac is not one of them, but I suppose the plug-in could be
written if needed.
I assume there's some way to filter out/automatically
close reports of certain errors...?
Yes.
…--
Nicolas Chauvat
logilab.fr - services en informatique scientifique et gestion de connaissances
|
|
I'm struggling with how to begin a report for this. The deliverable has morphed away from "bug reporting and contributing features right in the VRE", for which we can find no clear vision, and more towards simply making contributing to Sage more accessible, as well as making Sage's source code more accessible through the documentation. It doesn't even have anything to do with SageMathCloud. I know we are allowed some flexibility on this--that we can sort of "replace" a deliverable with another deliverable of similar work scope if the requirements change over time. But I don't know if that was approved in this case, or at least if the rationale for the change in scope needs to be discussed in the report. Please advise. In the meantime I am doing what I can to write up when I have done and how it fits into things. |
|
One thing for this deliverable that would be very useful to have, though I have not made any moves towards implementing yet, is a simplified version of the magics @nthiery described here: #64 (comment) One thing I have implemented for Sage is that the documentation will include an "edit on gitlab" link, which takes the user to the gitlab page for the documentation of some object, where the user can then make edits online and submit their edits as a merge request. All this really requires is the ability to produce a URL to view the given object in an online source tracker. This URL could just as easily be returned by a magic like In Python, obviously, the source link depends on the package/module that the object belongs to. There could be a magic method or attribute on Python objects like this is already essentially how the How this would be implemented for other kernels is left as an exercise. I believe something like this could be very useful, and the scheme I describe here is something of a more concrete proposal that I believe captures some of what we've been casting about for in this discussion. However, it would take some time to implement a prototype. Perhaps for starters I could get this working just for Sage which is easier. But I don't know if it would be possible to reach some agreement on a general solution in the time before this deliverable is due. I think it's still worth doing anyways, unless I'm overlooking an obvious problem (in the concept that is--the details are obviously still half-baked). |
|
Thanks for bringing this up.
Yes, for now, focus on the report as if the title was "Ease
contribution to SageMath" or something similar that matches the actual
content. Then we will add a section on the rationale for the change of
scope, either in the report itself or in the general report (or
presumably both, with a more concise version in the general).
If you have some preliminary language elements for that rationale,
feel free to throw them in.
Thanks!
|
I do have some text to that effect and had planed to include it. |
|
Almost finished with a draft report of what we have done so far: https://github.com/OpenDreamKit/OpenDreamKit/blob/master/WP3/D3.5/status-report.tex A few more sections to fill in. Also I'm not sure what to write as a "conclusion", especially since none of this work has gone "live" yet. There are no results to report really, except for what work was done and why. |
|
Hi @embray, |
|
@nthiery You wrote
Originally I used those always, every time I mentioned any of those technologies. But what is the recommendation for that? I found that when reading the rendered page, if I used those always, it was very distracting. My approach was to use those macros only when mentioning a specific technology by itself as the subject or object of a sentence, but not in contexts like "Sage community" or "GitHub workflow". |
|
You also changed
to
but in my original wording "development" was deliberate. But what do we mean by "Sage community"? I do not think it should necessarily include people who do regular, or even infrequent development on Sage. I don't want to give the impression that the "Sage community" is just the developers, because the "workflow and culture" e.g. of sage-devel and Trac is not representative, I think, of the whole Sage community, and I want to make Sage contributions more accessible beyond that. Perhaps I should clarify that as something like "the existing community of Sage developers..." and later clarify "which we would like to grow, by introducing more familiar tools", etc. |
|
but in my original wording "development" was deliberate. But what do we
mean by "Sage community"? I do not think it should necessarily include
people who do regular, or even infrequent development on Sage. I don't
want to give the impression that the "Sage community" is just the
developers, because the "workflow and culture" e.g. of sage-devel and
Trac is not representative, I think, of the whole Sage community, and I
want to make Sage contributions more accessible beyond that.
Makes sense indeed. I am too much tainted by Sage's origins where
blurring the line between the user and dev community was a vital
priority to reach the critical mass:-)
|
|
My approach was to use those macros only when mentioning a
specific technology by itself as the subject or object of a
sentence, but not in contexts like "Sage community" or "GitHub
workflow".
I like that rule of thumb! Let's just make sure we use it consistently.
|
I agree. Any recommendations? What is one of the more exciting/important API doc pages you can think of? Particularly one that is interesting in the first paragraph since most of the rest would be cropped. |
I'm not sure how I feel about all this. This seems to be conflating development of Sage itself with development of VREs. I'm having trouble picturing a workflow whereby I would make modifications to Sage sources from within a notebook, much less then rebuild and run those changes (which would require at a minimum restarting the kernel). I just don't see the advantage, and notebooks do not necessarily make good IDEs. But maybe I'm being myopic. I just don't think I could explain a workflow like this when it isn't clear to me how that would be useful.
This seems like a technical detail to me, and also distracts from the overall point of enabling GitLab contributions. |
|
I'm not sure how I feel about all this. This seems to be conflating
development of Sage itself with development of VREs. I'm having trouble
picturing a workflow whereby I would make modifications to Sage sources
from within a notebook, much less then rebuild and run those changes
(which would require at a minimum restarting the kernel). I just don't
see the advantage, and notebooks do not necessarily make good IDEs. But
maybe I'm being myopic. I just don't think I could explain a workflow
like this when it isn't clear to me how that would be useful.
The main point is not so much about developing from the notebook: most
of the items above are about simplifying the workflow in general.
Doing development in a JupyterHub/cocalc/... service (aka "in a VRE")
mostly means that you do not need to install and compile Sage locally;
it would still involve opening a shell, etc.
* inclusion of git-trac as sagemath package
This seems like a technical detail to me, and also distracts from the
overall point of enabling GitLab contributions.
Right; I was just throwing in some random ideas about what could ease
development. I agree that the conclusion can/should stay at a higher
level.
|
|
@nthiery |
|
Thanks @embray. Now proofreading!
|
|
Done with the review. Great report Erik; thanks a lot! Very good choice of documentation screenshot in Figure 1 btw. |
|
Added the missing files. FWIW the subject used in the other screenshots is |
|
I am apparently still missing screenshots/docs-console. Good choice too for laplace. |
|
Strange. I have right in my bash history |
|
Go figure ... Got it now as well as the final polishing. The report is submitted now. Thanks for all the hard work: one more very useful contribution to Sage and a nice report! |
In a deliverable which is about "integration", certainly! |
Task T3.7 (#56) aims to improve the development workflow in mathematical software, to ease contributions by all actors, from experts developers on those projects or coming from other projects, to students, novice users, or non-programmers.
For this deliverable we focused on engaging newcomers to become contributors by reducing the barrier of entry to Sage development. In particular, trivial contributions ought to be trivial: for example, a user or technical writer that notices a typo or a potential phrasing improvement in the documentation should be encouraged and empowered to fix it on the spot and submit it for review and integration into Sage.
The Sage community has a history of blurring the line between developer and user, with even new users often encouraged to make code contributions. However, the core group of frequent development contributors remains relatively small, and has a well-established workflow and culture surrounding its existing development tools--in particular the source code hosting and issue tracking service Trac. This works well for experienced Sage developers, but also creates certain barriers to entry for new contributors, especially those who are new to open source development.
To tackle those barriers, we aimed for a better integration between Sage's Trac-based workflow with platforms that are more modern and familiar to open source novices, such as GitHub and GitLab, without immediately breaking the Sage project's existing workflow [1].
Beyond tackling the immediate goal of this deliverable, we believe that this approach also opens the doors to long-term improvements in Sage's development workflow, as the advantages of using these new tools become more apparent. Part of this work also serves as a prerequisite to improvements in Sage's continuous integration practices, which are being explored in D3.8 (#67).
An aspect that Sage (and indeed any open source mathematics software) prides itself on over closed-source alternatives is the ability to peer inside the software to see how it works. It is not a black box and one does not have to take its results for granted--it is possible to look at the source code in order to understand how a particular algorithm works, or even modify and improve it. To this end we sought ways to more tightly couple Sage's API documentation to its source code and, through access to the source code, encourage contribution of improvements.
To these ends, this deliverable includes the following improvements to Sage's development platform and documentation:
[1]: Upon writing the proposal we envisioned using SageMathCloud (now CoCalc) as a major development platform for Sage, hence the title of this deliverable. However, while CoCalc remains quite viable for this purpose, it became evident that there was a need for solutions not explicitly tied to CoCalc. See the report for a discussion.
The text was updated successfully, but these errors were encountered: