Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SunPy roadmap #323

Closed
wants to merge 8 commits into from
Closed

SunPy roadmap #323

wants to merge 8 commits into from

Conversation

ehsteve
Copy link
Member

@ehsteve ehsteve commented Aug 24, 2022

PR Description

This provides a roadmap to the website as was discussed in the 2022 SunPy coordination meeting. The first version is my interpretation of the discussion that took place and so significant feedback is requested to ensure that it captures what was discussed and what the community wants to achieve!

Copy link
Contributor

@nabobalis nabobalis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is broad and very shallow changes.

project/roadmap.rst Outdated Show resolved Hide resolved
project/roadmap.rst Outdated Show resolved Hide resolved
project/roadmap.rst Outdated Show resolved Hide resolved
project/roadmap.rst Outdated Show resolved Hide resolved
project/roadmap.rst Outdated Show resolved Hide resolved
@nabobalis
Copy link
Contributor

Once we are more happy with the text, we add the config needed to render the page.

This roadmap is discussed and reviewed at the yearly SunPy coordination meetings.
It provides a broad outline of what areas we will be investing our time and energy in.

Spectra
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section kinda misses a key point, which is that for a lot of solar data the spectral axis is one of many axes and we need to handle everything from 1D spectra through spectrograms to full 5D cubes where only one is a spectral axis.

Also I would reword this a little bit to make it more focused on ndcube (which Spectrum1D inherits from).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first I had this all under one section heading with NDCube but this is a roadmap so I didn't think it was helpful to go into those kinds of details. All they want to know what will they be able to do with their data. Mentioning that we are talking about 1d spectrum is important, I agree. I will add that to this section.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with Steve, "Supporting spectral data analysis" is a well scoped, large use case for the solar community. Whether solving this involves a NDCube or some other object is an implementation detail, and the end product should be a tutorial (see Diataxis for my definition of a tutorial here) that steps through doing spectral data analysis in SunPy.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this is explicitly calling out specutils and not ndcube. Why one and not the other?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pointed to specutils because their docs show the kind of workflows that are made possible. I also care very little that ndcube is used in the background. Their docs don't even mention it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, language was bad in my comment above. I think it's fine to mention ndcube in the body, but not in the title.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also sorry for me. I do think it is really awesome that ndcube sits behind specutils. From a workflow point of view though, most users won't interact with it so it is not really relevant to them.

ehsteve and others added 6 commits August 24, 2022 14:41
Co-authored-by: Nabil Freij <nabil.freij@gmail.com>
Co-authored-by: Nabil Freij <nabil.freij@gmail.com>
Co-authored-by: Nabil Freij <nabil.freij@gmail.com>
Co-authored-by: Nabil Freij <nabil.freij@gmail.com>
Co-authored-by: Nabil Freij <nabil.freij@gmail.com>
@ehsteve
Copy link
Member Author

ehsteve commented Aug 24, 2022

@nabobalis or others, could you advertise this PR during the meeting to get people to comment please? Thanks.

@nabobalis
Copy link
Contributor

@nabobalis or others, could you advertise this PR during the meeting to get people to comment please? Thanks.

Will do

Copy link
Member

@dstansby dstansby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've started reviewing this, but as a general comment I think that at a high level we should focus on solar physics workflows that are not supported within the sunpy ecosystem (e.g. "Support for spectral data analysis") instead of implementation details ("Create a spectral data object"). The implementation can then flow in in the text from the workflows we want to support, but shouldn't be what is driving each item in the roadmap.

project/roadmap.rst Outdated Show resolved Hide resolved
This roadmap is discussed and reviewed at the yearly SunPy coordination meetings.
It provides a broad outline of what areas we will be investing our time and energy in.

Spectra
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with Steve, "Supporting spectral data analysis" is a well scoped, large use case for the solar community. Whether solving this involves a NDCube or some other object is an implementation detail, and the end product should be a tutorial (see Diataxis for my definition of a tutorial here) that steps through doing spectral data analysis in SunPy.

@dstansby
Copy link
Member

Other things that are currently missing from this but in the meeting notes are:

  • Making timeseries analysis easier
  • Creating a way to structured way to store and search data locally (ie. database 2.0)
  • Improving project messaging
  • Improving support for radio data
  • Supporting fast visualisation of large datasets
  • Making it easier to do sunpy outreach
  • Improving the doucmentation
  • Improving metadata handling

@ehsteve do you want to write these up? If not I'd be happy to take a shot?

@ehsteve
Copy link
Member Author

ehsteve commented Aug 25, 2022

Those all seem like issues to me and not really roadmap items. Too much in the weeds or things that we will be doing anyway. They don't really affect workflow as mentioned above. Maybe a few could be incorporated into a broader subject.

@dstansby
Copy link
Member

That makes sense - I think I got the wrong end of the stick. I'd still suggest adding "Radio data" as an item, since it seems similar in scope to "spectral data"? Maybe "supporting 'big' data" is worth adding too? It came up yesterday in terms of visualisation, and I think a lot of the work planned for OSTFL fits in with that as a roadmap item.

Co-authored-by: David Stansby <dstansby@gmail.com>
@ehsteve
Copy link
Member Author

ehsteve commented Aug 25, 2022

@dstansby I wasn't exactly sure what we were planning to do with radio data. Are we thinking of supporting radio spectrograms with ndcube?

Supporting big data and optimization could be another topic to add if we can specific about what we intend to do.

Merge requests frequently take several rounds of back and forth with reviewers asking for changes which is frustrating and demoralizing especially for new contributors.
We aim to improve our support for contributors by improving our developer documentation generally but especially by better documenting expectations.
In addition, we will offer to actively help contributors during development and work to minimize the amount of work needed to contribute.
#. Better support for affiliated packages.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be under it's own heading. Maybe call it "Enabling the SunPy Ecosystem" or "Expanding the SunPy Ecosystem"?

@ehsteve
Copy link
Member Author

ehsteve commented Sep 7, 2022

Work on this pull request has moved to this google document to make it easier and faster to iterate. I made some changes based on the discussion at the community meeting which took place 7-Sept-2022.

@nabobalis nabobalis marked this pull request as draft November 23, 2023 14:44
@nabobalis nabobalis closed this Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants