Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JEP 0029] JEP Pre-proposal: New JEP process #27

Open
captainsafia opened this Issue Feb 9, 2019 · 16 comments

Comments

Projects
None yet
7 participants
@captainsafia
Copy link
Member

captainsafia commented Feb 9, 2019

Summary

During the NES summit, a session was held on how to revamp the JEP process. During this session, we reviewed existing ehnancement proposals from the Django and Rust communities, discussed the workflow for the new JEP, and created a working document to outline different aspects of the new workflow.

Why is this is a JEP?

This new process will be written as a meta-JEP to document and dogfood the new process. This issue serves as the pre-proposal for this meta-JEP.

@mckev-amazon

This comment has been minimized.

Copy link

mckev-amazon commented Feb 12, 2019

Very excited to see this, and I agree that this should be a JEP. Who would be a suitable Shepherd, and what would be an appropriate review team? Given the importance of this change, I would suggest that the entire steering council participate in some capacity.

For those reading the doc for the first time, the bulk of the proposed improvements to the JEP process are in the section "Discussion Notes" in the linked doc.

There are several possible Jupyter enhancements that we discussed last week that I think would benefit going through this process, in order to get discussion and refinement from the entire Jupyter community. They would also help us refine the JEP process a bit by using a few of them as a beta test or case study of sorts. Those were:

  • Cell Forms / Templates
  • Multi-Language Kernel
  • Kernel "Resolver", "Finder" or Runtime Environment Spec
  • Embedding dependency metadata in the notebook (a variation of the above idea)
  • Adding unique cell IDs to nbformat

I'm sure there are more that I missed, too.

@captainsafia

This comment has been minimized.

Copy link
Member Author

captainsafia commented Feb 13, 2019

@mckev-amazon I agree that the review team should be compromised of members of the steering council but we should also make an effort to include key contributor across the ecosystem so we don't miss out on a diverse perspective. Perhaps @rgbkrk or @willingc can initiate bringing this up with the steering council.

As for Shepherd, would you be interested in this role?

@rgbkrk

This comment has been minimized.

Copy link
Member

rgbkrk commented Feb 13, 2019

I think the working document should be turned into a PR, then we can summon folks into that to review it.

@mckev-amazon

This comment has been minimized.

Copy link

mckev-amazon commented Feb 13, 2019

@captainsafia Sure, happy to do so!

As Shepherd, per the process we outlined, "It's a JEP! Please create a new PR with the JEP contents using template X. On that basis, you can resolve this issue unless you have further questions."

@rgbkrk agreed, that's the "next step" - this is the Pre-Proposal where the general idea is screened and the Shepherd and initial review team is selected, then we move onto the PR containing the actual contents of the proposal.

I guess from the process standpoint, I think it's clear to me that selecting a review team at this point seems a little premature, beyond a simple gut check of "do we know who could potentially review this?".
I also agree that final review team is something the contributor, Shepherd, and watchers on this repo can negotiate after the doc PR is posted. I think the Shepherd should have final say on the required review team members (though I'll obviously be looking for help here as I'm relatively new to the community).

@willingc

This comment has been minimized.

Copy link
Member

willingc commented Feb 27, 2019

It would be nice to see the JEPs evolve in a manner similar to a Python PEP.

@mckev-amazon

This comment has been minimized.

Copy link

mckev-amazon commented Mar 4, 2019

Nice to see that the WIP PR has been merged in. Two things came to mind from a process perspective:

  1. It seems like in this state there really isn't a place for general, ongoing conversation for a given JEP since there isn't a single PR to use for discussion. This GH issue (#27) is the "pre-proposal" GH Issue, which I think according to our workflow should probably be closed at this point.
  2. WDYT about, as part of the workflow, having a second GH issue created for the duration of the JEP until it reaches a terminal state? Speaking from my perspective as the shepherd, there otherwise isn't a single place to communicate on the issue except offline (email, etc.). In the interest of being as open as possible I'd probably prefer we discuss JEPs somewhere on GH instead of other channels. We can use labels or similar to distinguish Pre-Proposal issues from In-Progress issues.

@willingc willingc changed the title JEP Pre-proposal: New JEP process [29] JEP Pre-proposal: New JEP process Mar 4, 2019

@willingc willingc changed the title [29] JEP Pre-proposal: New JEP process [JEP 0029] JEP Pre-proposal: New JEP process Mar 4, 2019

@choldgraf

This comment has been minimized.

Copy link
Contributor

choldgraf commented Mar 4, 2019

A quick question (I see @mckev-amazon suggested we open another issue, that sounds good too, but asking here until this other issue is created):

when the meta-JEP mentions "orgs", does it mean "github organizations" or "organizations in general" (e.g. multiple companies, multiple research groups, etc)? It might be helpful to clarify terminology there

@willingc

This comment has been minimized.

Copy link
Member

willingc commented Mar 5, 2019

@mckev-amazon While it could be one GitHub issue for discussion, I think it's more effective to have issues tagged in the title with the PEP number. Otherwise, we may wind up with the huge scroll of comments. (See existing open issues for examples.)

@choldgraf Yes, we should clarify GH-orgs or orgs in general.

@willingc

This comment has been minimized.

Copy link
Member

willingc commented Mar 5, 2019

@mckev-amazon Maybe a general process issue for status/checklists and then multiple issues for technical points of discussion.

@mckev-amazon

This comment has been minimized.

Copy link

mckev-amazon commented Mar 5, 2019

@willingc Fair point, makes sense to include a "how to discuss a JEP" section in the JEP doc given that I saw this come us elsewhere (the PR?).

For at least for the submitter <-> shepherd communication track, I'd prefer a single thread (GH issue) for administrative business. A long scroll might be fine (preferred?) IMO, since it captures the linear progression of the issue from a workflow standpoint. But, you're definitely right, we should definitely provide a mechanism for spinning off additional threads and I think tagged GH issues works just fine for me.

@captainsafia

This comment has been minimized.

Copy link
Member Author

captainsafia commented Mar 5, 2019

Alright! So it seems like the next set of updates to this draft are as follows.

  • Put in minor edits (@parente)
  • Add section on distributing JEP updates to community (@choldgraf)
  • Clarify what an "org" is in the context of the document (@captainsafia)
  • Add a draft of the "how to discuss a JEP" section (@mckev-amazon)

I'm liking the approach of iteratively creating this draft with multiple people, at least for this JEP. It prevents the work from being blocked on one person and helps us add/clarify things on an as-needed basis.

@mckev-amazon

This comment has been minimized.

Copy link

mckev-amazon commented Mar 5, 2019

I'm happy to take the "how to discuss a JEP" section! I'll send out a PR on that soon.

@parente

This comment has been minimized.

Copy link
Member

parente commented Mar 5, 2019

I'll submit the minor edits I had as a PR this evening.

I did have one comment about:

  • Creating a new GitHub repo in one of the official Jupyter orgs

which might require some discussion before I can submit an edit to the doc. Namely:

Would we deprecate https://github.com/jupyter/governance/blob/master/newsubprojects.md in favor of the JEP process of including new projects under the Jupyter umbrella?

I'm not strongly in favor of one over the other, but rather seeking how to clarify the "right way" for would-be project proposers to proceed.

@lheagy

This comment has been minimized.

Copy link

lheagy commented Mar 7, 2019

As I read through the JEP Pre-Read, I am seeing a lot of potential overlaps in process and practices with JOSS. The end goals are a bit different, but I think there are potentially a few process items that might be useful here:

  • public list of Shepards / reviewers that can be searched by area of expertise
  • effective use of labels: review process (pre-review, review, accepted, ...), language (this could be Jupyter sub-projects) on the pre-review and review issues
  • separation of pre-review and review issues - each has its own checklist, so at each stage, the conversation is reasonably well-scoped
  • in some senses, the editor role is similar to the Shepard - they are responsible for finding suitable reviewers, helping sort through feedback, and helping keep the conversation moving. They also have the power to accept once the review process is complete -- this does enable JOSS to move relatively quickly because there is a large group of trusted editors, but may or may not be the right model here.
@willingc

This comment has been minimized.

Copy link
Member

willingc commented Mar 8, 2019

@jupyter/steeringcouncil FYI. Six of the 16 steering council members were at an event at Netflix last month with industry folks to discuss Jupyter, nteract, data workflows with papermill, and other topics. Those of you who follow nteract's activities likely have already seen the repo that was used during the event. Here's the link to the repo for reference.

One topic that was discussed at the meeting was rebooting the JEP process to make it more effective and collaborative across the Jupyter orgs, projects, and wider community. It's wonderful to see this renewed focus and moving toward greater visibility and input which has been very successful for the Python PEPs which was an initial model for the existing JEP process.

This issue links to a "Draft" JEP which has been iterated over the past few weeks. There's also iterative work being done on another "Draft" JEP for the Jupyter Server.

Looking forward to seeing many of you next week. ☀️

@choldgraf

This comment has been minimized.

Copy link
Contributor

choldgraf commented Mar 29, 2019

Hey all - is there anything that I can do to help help move this process forward? Are we still in a phase of inviting general feedback and commentary from the community? Or is there an action list that we still need to complete?

I know there are more general governance conversations at play, but I really like the structure that y'all have put together and think it's an improvement on the current JEP process!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.