Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JEP 0029] JEP Pre-proposal: New JEP process #27

Open
captainsafia opened this issue Feb 9, 2019 · 21 comments
Open

[JEP 0029] JEP Pre-proposal: New JEP process #27

captainsafia opened this issue Feb 9, 2019 · 21 comments

Comments

@captainsafia
Copy link
Member

@captainsafia captainsafia commented Feb 9, 2019

Summary

During the NES summit, a session was held on how to revamp the JEP process. During this session, we reviewed existing ehnancement proposals from the Django and Rust communities, discussed the workflow for the new JEP, and created a working document to outline different aspects of the new workflow.

Why is this is a JEP?

This new process will be written as a meta-JEP to document and dogfood the new process. This issue serves as the pre-proposal for this meta-JEP.

@mckev-amazon
Copy link

@mckev-amazon mckev-amazon commented Feb 12, 2019

Very excited to see this, and I agree that this should be a JEP. Who would be a suitable Shepherd, and what would be an appropriate review team? Given the importance of this change, I would suggest that the entire steering council participate in some capacity.

For those reading the doc for the first time, the bulk of the proposed improvements to the JEP process are in the section "Discussion Notes" in the linked doc.

There are several possible Jupyter enhancements that we discussed last week that I think would benefit going through this process, in order to get discussion and refinement from the entire Jupyter community. They would also help us refine the JEP process a bit by using a few of them as a beta test or case study of sorts. Those were:

  • Cell Forms / Templates
  • Multi-Language Kernel
  • Kernel "Resolver", "Finder" or Runtime Environment Spec
  • Embedding dependency metadata in the notebook (a variation of the above idea)
  • Adding unique cell IDs to nbformat

I'm sure there are more that I missed, too.

@captainsafia
Copy link
Member Author

@captainsafia captainsafia commented Feb 13, 2019

@mckev-amazon I agree that the review team should be compromised of members of the steering council but we should also make an effort to include key contributor across the ecosystem so we don't miss out on a diverse perspective. Perhaps @rgbkrk or @willingc can initiate bringing this up with the steering council.

As for Shepherd, would you be interested in this role?

@rgbkrk
Copy link
Member

@rgbkrk rgbkrk commented Feb 13, 2019

I think the working document should be turned into a PR, then we can summon folks into that to review it.

@mckev-amazon
Copy link

@mckev-amazon mckev-amazon commented Feb 13, 2019

@captainsafia Sure, happy to do so!

As Shepherd, per the process we outlined, "It's a JEP! Please create a new PR with the JEP contents using template X. On that basis, you can resolve this issue unless you have further questions."

@rgbkrk agreed, that's the "next step" - this is the Pre-Proposal where the general idea is screened and the Shepherd and initial review team is selected, then we move onto the PR containing the actual contents of the proposal.

I guess from the process standpoint, I think it's clear to me that selecting a review team at this point seems a little premature, beyond a simple gut check of "do we know who could potentially review this?".
I also agree that final review team is something the contributor, Shepherd, and watchers on this repo can negotiate after the doc PR is posted. I think the Shepherd should have final say on the required review team members (though I'll obviously be looking for help here as I'm relatively new to the community).

@willingc
Copy link
Member

@willingc willingc commented Feb 27, 2019

It would be nice to see the JEPs evolve in a manner similar to a Python PEP.

@mckev-amazon
Copy link

@mckev-amazon mckev-amazon commented Mar 4, 2019

Nice to see that the WIP PR has been merged in. Two things came to mind from a process perspective:

  1. It seems like in this state there really isn't a place for general, ongoing conversation for a given JEP since there isn't a single PR to use for discussion. This GH issue (#27) is the "pre-proposal" GH Issue, which I think according to our workflow should probably be closed at this point.
  2. WDYT about, as part of the workflow, having a second GH issue created for the duration of the JEP until it reaches a terminal state? Speaking from my perspective as the shepherd, there otherwise isn't a single place to communicate on the issue except offline (email, etc.). In the interest of being as open as possible I'd probably prefer we discuss JEPs somewhere on GH instead of other channels. We can use labels or similar to distinguish Pre-Proposal issues from In-Progress issues.

@willingc willingc changed the title JEP Pre-proposal: New JEP process [29] JEP Pre-proposal: New JEP process Mar 4, 2019
@willingc willingc changed the title [29] JEP Pre-proposal: New JEP process [JEP 0029] JEP Pre-proposal: New JEP process Mar 4, 2019
@choldgraf
Copy link
Contributor

@choldgraf choldgraf commented Mar 4, 2019

A quick question (I see @mckev-amazon suggested we open another issue, that sounds good too, but asking here until this other issue is created):

when the meta-JEP mentions "orgs", does it mean "github organizations" or "organizations in general" (e.g. multiple companies, multiple research groups, etc)? It might be helpful to clarify terminology there

@willingc
Copy link
Member

@willingc willingc commented Mar 5, 2019

@mckev-amazon While it could be one GitHub issue for discussion, I think it's more effective to have issues tagged in the title with the PEP number. Otherwise, we may wind up with the huge scroll of comments. (See existing open issues for examples.)

@choldgraf Yes, we should clarify GH-orgs or orgs in general.

@willingc
Copy link
Member

@willingc willingc commented Mar 5, 2019

@mckev-amazon Maybe a general process issue for status/checklists and then multiple issues for technical points of discussion.

@mckev-amazon
Copy link

@mckev-amazon mckev-amazon commented Mar 5, 2019

@willingc Fair point, makes sense to include a "how to discuss a JEP" section in the JEP doc given that I saw this come us elsewhere (the PR?).

For at least for the submitter <-> shepherd communication track, I'd prefer a single thread (GH issue) for administrative business. A long scroll might be fine (preferred?) IMO, since it captures the linear progression of the issue from a workflow standpoint. But, you're definitely right, we should definitely provide a mechanism for spinning off additional threads and I think tagged GH issues works just fine for me.

@captainsafia
Copy link
Member Author

@captainsafia captainsafia commented Mar 5, 2019

Alright! So it seems like the next set of updates to this draft are as follows.

  • Put in minor edits (@parente)
  • Add section on distributing JEP updates to community (@choldgraf)
  • Clarify what an "org" is in the context of the document (@captainsafia)
  • Add a draft of the "how to discuss a JEP" section (@mckev-amazon)

I'm liking the approach of iteratively creating this draft with multiple people, at least for this JEP. It prevents the work from being blocked on one person and helps us add/clarify things on an as-needed basis.

@mckev-amazon
Copy link

@mckev-amazon mckev-amazon commented Mar 5, 2019

I'm happy to take the "how to discuss a JEP" section! I'll send out a PR on that soon.

@parente
Copy link
Member

@parente parente commented Mar 5, 2019

I'll submit the minor edits I had as a PR this evening.

I did have one comment about:

  • Creating a new GitHub repo in one of the official Jupyter orgs

which might require some discussion before I can submit an edit to the doc. Namely:

Would we deprecate https://github.com/jupyter/governance/blob/master/newsubprojects.md in favor of the JEP process of including new projects under the Jupyter umbrella?

I'm not strongly in favor of one over the other, but rather seeking how to clarify the "right way" for would-be project proposers to proceed.

@lheagy
Copy link

@lheagy lheagy commented Mar 7, 2019

As I read through the JEP Pre-Read, I am seeing a lot of potential overlaps in process and practices with JOSS. The end goals are a bit different, but I think there are potentially a few process items that might be useful here:

  • public list of Shepards / reviewers that can be searched by area of expertise
  • effective use of labels: review process (pre-review, review, accepted, ...), language (this could be Jupyter sub-projects) on the pre-review and review issues
  • separation of pre-review and review issues - each has its own checklist, so at each stage, the conversation is reasonably well-scoped
  • in some senses, the editor role is similar to the Shepard - they are responsible for finding suitable reviewers, helping sort through feedback, and helping keep the conversation moving. They also have the power to accept once the review process is complete -- this does enable JOSS to move relatively quickly because there is a large group of trusted editors, but may or may not be the right model here.

@willingc
Copy link
Member

@willingc willingc commented Mar 8, 2019

@jupyter/steeringcouncil FYI. Six of the 16 steering council members were at an event at Netflix last month with industry folks to discuss Jupyter, nteract, data workflows with papermill, and other topics. Those of you who follow nteract's activities likely have already seen the repo that was used during the event. Here's the link to the repo for reference.

One topic that was discussed at the meeting was rebooting the JEP process to make it more effective and collaborative across the Jupyter orgs, projects, and wider community. It's wonderful to see this renewed focus and moving toward greater visibility and input which has been very successful for the Python PEPs which was an initial model for the existing JEP process.

This issue links to a "Draft" JEP which has been iterated over the past few weeks. There's also iterative work being done on another "Draft" JEP for the Jupyter Server.

Looking forward to seeing many of you next week. ☀️

@choldgraf
Copy link
Contributor

@choldgraf choldgraf commented Mar 29, 2019

Hey all - is there anything that I can do to help help move this process forward? Are we still in a phase of inviting general feedback and commentary from the community? Or is there an action list that we still need to complete?

I know there are more general governance conversations at play, but I really like the structure that y'all have put together and think it's an improvement on the current JEP process!

@Zsailer
Copy link
Member

@Zsailer Zsailer commented Oct 14, 2019

I agree with @choldgraf. I think this proposal is great and would love to help push it forward (thanks to everyone who contributed to its conception).

I'd even like to propose using this process in JEP #28 (Jupyter Server). I don't think I can be the Shepherd since I'm the JEP Contributor, so I'll have to find someone to fulfill that role. That said, let's count JEP #28 as another test case for this process.

@choldgraf
Copy link
Contributor

@choldgraf choldgraf commented Jun 5, 2020

Hey all - to take some first steps towards adopting the proposal, I made a PR to update this repository's JEP guidelines here:

#52

@choldgraf
Copy link
Contributor

@choldgraf choldgraf commented Jun 15, 2020

Another thought as I have been thinking through #29 - could we use the original issue that was created for the JEP as a place for ongoing conversation about the JEP? AKA something like:

  • GitHub issue is created to discuss the JEP. Conversation starts here.
  • Once the JEP PR is opened, conversation around accepting the JEP should happen there.
  • After the JEP PR is merged, subsequent conversations about the JEP itself should happen either in PRs related to implementation, or in the original JEP issue itself. (sort-of like what I am doing with this comment)
    • A different option could be to say "subsequent conversation about the JEP should be in new issues in this repository
  • Once the JEP is considered "implemented", then that issue is closed.

I think much of this process is kind-of implicit, but maybe worth making it explicit?

@meeseeksmachine
Copy link

@meeseeksmachine meeseeksmachine commented Dec 3, 2020

This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/2019-in-person-team-meeting-updates-and-notes/458/1

@choldgraf
Copy link
Contributor

@choldgraf choldgraf commented Dec 3, 2020

(note I think this is a false ping, we re-organized the forum a little bit and this re-ping was a secondary effect of that!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants