Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Computational models #850

Closed
wants to merge 5 commits into from
Closed

[ENH] Computational models #850

wants to merge 5 commits into from

Conversation

MichaelSchirner
Copy link

This PR incorporates the BEP Computational Models.

It evolved from the extension proposal Computational Models leaving out some of the proposed features (specifically: checksum filename ids for data integrity verification) for full backwards compatibility with BIDS 1.x. Many thanks to everyone who contributed.

@tsalo tsalo added the BEP label Aug 18, 2021
@sappelhoff
Copy link
Member

sappelhoff commented Aug 24, 2021

Hi @MichaelSchirner, thanks for your interest in extending BIDS! Below I'll first try to give you some basic information, and I'll close with my personal opinion. After that, we'll need more people to chime in with their opinion 🙂


This PR is a bit surprising to me. Usually BEP leads start with opening a GitHub issue, and getting in touch with the BIDS maintainers, followed by several (virtual) meetings on how to best steer the BEP to be finally merged into the spec.

I just checked our BEP Lead guidelines and I realize that this crucial information was not entirely obvious from the guidelines. I now added those in the form of the following two points:

  • Familiarize yourself with the BIDS community by browsing current issues, discussions, and proposed changes on GitHub.
  • Open an initial “issue” on the GitHub issue tracker to gauge interest in your potential extension, and to collect feedback by core community members and BIDS maintainers. This is an important step before proceeding in order to make sure that everybody is on the same page.

To proceed with what we have here, we need to hear BIDS maintainers opinions, and we need a lot more community awareness and involvement.

I have seen that you posted your BEP on the BIDS mailing list, that's a good start 🙂 Often, BEPs are also announced on neurostars and on Twitter, where we can retweet the posts with the BIDS handle.

Furthermore, I see that you gave yourself a BEP number (BEP032). Usually these numbers are "handed out" by the BIDS maintainers after the discussions and meetings as outlined above have happened. Then the BEP is also "made official" by adding it to the list on our website. In that list, you can also see that BEP032 is "already taken" by Animal Electrophysiology, so if the community agrees that we want to make your proposed BEP an official BEP, we'll have to use a new number.


Finally, here is my personal opinion:

In your bids-discussion mailing list post you write:

We are aware that there are efforts towards a generic model description format (https://github.com/ModECI/MDF) in the wider BIDS community.

Indeed, I feel like your situation lends itself very well to propel the MDF efforts forward. Computational models for BIDS have been discussed for a long time, and the recent agreement among contributors, leads, and maintainers was to have a computational models specification within the BIDS ecosystem, but not as part of the BIDS specification (which is what a BEP would aim for).

You also write (emphasis by me):

Since we need an immediate solution we will go for the proposed format for now and are happy to collaborate on the MDF in the future.

I appreciate your situation: Sometimes you need something that's finished instead of something that's a working draft (MDF). However, all BEPs that were steered into the BIDS specification have taken a longish time of at least one year. That's because we need to involve many people, go through lots of iterations, prepare the BIDS-validator and BIDS examples, make sure that the proposal tightly integrates with the existing specification, without breaking backwards compatibility and without unnecessarily restricting potential future extensions. In short: It's a lot of work with many hands, and not an "immediate solution".

That's why I would recommend to join efforts with the MDF team. I am curious to hear more opinions and more about the thoughts behind this proposal.

I would also be grateful to hear your feedback on which documentation we need to improve so that we can be more transparent and avoid misunderstandings in the future!


EDIT: tagging some people:

@bids-standard/maintainers @bids-standard/bep_leads @tyarkoni

@PeerHerholz
Copy link
Member

Ahoi hoi folks,

just to add a potentially interesting/useful link: we also have a computational models team set up within the models team. So far nothing really happened there (IIRC), but given the recent work on BEP002 (stats-models) and the efforts present here, there could/should be enough momentum to start with the BIDS BEP route outline by @sappelhoff!

Cheers, Peer

@BrainModes
Copy link

Please note the related tweet posted on Feb 12, 2021 - and please kindly retweet. Thank you.

@Remi-Gau
Copy link
Collaborator

Computational models for BIDS have been discussed for a long time, and the recent agreement among contributors, leads, and maintainers was to have a computational models specification within the BIDS ecosystem, but not as part of the BIDS specification (which is what a BEP would aim for).

I think this point is important to highlight. If even the "basic" BIDS-model stats is supposed to be part of the ecosystem and not part of the spec, isn't it going to be much harder to have "computational models" as part of the spec?

As I see it the scope and complexity of BIDS stats is lower than those of computational model, so that would seem like a lower hanging fruit for integration into the spec and yet this is not the route taken. So I have a hard time imagining this happening for computational model.

This does not mean it cannot take the same path as the bids-stats model (and coordinate with it).

@BrainModes
Copy link

@sappelhoff: "That's why I would recommend to join efforts with the MDF team." Yea we do indeed already. This joint article might be of interest in that context "The importance of standards for sharing of computational models and data". We look forward to the comments on our proposal by the BIDS community.

@MichaelSchirner
Copy link
Author

Dear @sappelhoff, @PeerHerholz, @Remi-Gau: many thanks for your helpful comments! We would be happy to participate in the computational models team, can we (@BrainModes, @MichaelSchirner) kindly be added to this team/repository? My understanding now is that in order to start with the BIDS BEP routine as outlined by @sappelhoff and @PeerHerholz, a next step would be to review the PR to make sure that it integrates with the existing specification and is backwards compatible. Are there people from the BIDS community who would be willing to discuss and review the PR? We would be happy to learn what we can do to help the process. Many thanks again!

@BrainModes
Copy link

Dear all, dear @sappelhoff @PeerHerholz @Remi-Gau, please join Webinar BIDS Computational Models Extension Proposal with demo on Jan 11 at 17 CET. Please retweet. We look forward to the discussion and feedback.

@MichaelSchirner
Copy link
Author

closed and replaced by #967

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants