(Version 1.5, 2024-02-27)
BOSC is run by volunteers--we couldn't do it without your help! To understand the process and so you can more effectively conduct your work, please review these guidelines before signing up as a reviewer.
BOSC 2024 will be a track (also called a Community of Special Interest, or COSI) of ISMB 2024, which will be held in person in Montréal, Canada, as well as virtually. Each COSI has its own review committee and process. This document applies only to BOSC.
BOSC reviewers participate in this process by invitation only. Reviewers are invited by the BOSC Organizing Committee. If you’d like to volunteer as a reviewer, please contact us at bosc@open-bio.org.
Since BOSC 2024 will be part of ISMB, all members of the BOSC review committee members must read and agree to follow the ISMB Code of Conduct, as well as the review process outlined here. This is to ensure that the review process is fair and reliable, that all review requirements are applied consistently, and that reviewers understand abstract acceptance criteria and thus can constructively critique the submissions.
The timeline for BOSC 2024 reviewing is as follows (see https://www.open-bio.org/events/bosc/ for the latest info):
- Abstract submission opens: January 2024
- Invitations to reviewers sent: April 3, 2024
- (Early Poster Acceptance submission closes: March 21. These will be reviewed by members of the BOSC Organizing Committee.)
- Regular abstract submission closes: April 19 (Anywhere on Earth)
- Reviewers can bid on abstracts (this is optional): April 20-21 (bidding will close at 12pm ET on April 21)
- Abstracts assigned to reviewers: April 21, 1pm ET
- Abstract reviews due: May 1 (end of the day anywhere in the world)
- Authors notified: May 13
- Late poster abstract submission closes: May 20
- Late poster notifications: May 28
- ISMB2024: July 12-16, 2024 (Montréal, Canada, and online)
- BOSC 2024: July 15-16, 2024 (last two days of ISMB)
- Post-BOSC CollaborationFest: July 17-18 (Montréal, Canada, and online)
Abstracts are considered as unpublished works from submission of the abstract until the conference and should be treated as confidential. Therefore, reviewers should not share or discuss abstracts (or specific details about the abstracts) with anyone except members of the review committee without express permission of the abstract authors and the BOSC chair(s).
Shortly after abstract submission closes, reviewers are given a chance to ‘bid’ on abstracts (i.e., indicate whether they do or do not want to review specific abstracts). A reviewer can say “Yes, I’d like to review this” to abstracts that may be, but aren’t limited to, work in one of their areas of expertise. During the bidding process, reviewers can also indicate when they don’t have the right background or have potential conflicts of interest, such as prior work with the authors or competing interests. It is not always possible for us to assign abstracts to reviewers who bid “yes”. However, you will never be assigned to review an abstract for which you indicated a conflict. (In fact, if you mark an abstract as “conflict”, you will never be able to see the abstract again in EasyChair, so be careful!)
Each submitted abstract will be assigned to three reviewers. Sometimes, if an abstract is controversial (for example, reviewers disagree on its relevance to BOSC), it will go through an additional review before authors are notified. All submitted abstracts must pass through this review process.
Each reviewer will be assigned several abstracts to review (typically, 5-7 abstracts per round). Please check your assigned abstracts right away and let us know if there are any you can't review (for example, because of a conflict of interest with an author, or because you don’t feel you know enough about the topic to review the abstract fairly), so we can reassign those.
(Late-breaking abstracts - we are not doing this now, but keeping this section for historical accuracy)
An additional late-breaking abstract submission round is normally run shortly (1-2 months) prior to the conference. Abstracts accepted in the late round are normally limited to poster presentation and sometimes for "late-breaking lightning talks"; they are not eligible for long talks.
Submissions to this round can be new submissions or abstracts that were rejected in the first round and have been improved based on reviewers’ comments. These late-round abstracts undergo a similar process for review.
To be accepted (even for poster presentation), abstracts submitted to BOSC must be:
- Relevant: Your abstract must describe how your work relates to some aspect of open source software, open science or open data applied to biological or biomedical science.
- Available: Your code / data / materials (e.g., documentation) must be available from the time of review onwards at the URL you list.
- Open Content: Work discussed in BOSC presentations must be open source / open content, with a recognized license (which must be included in the repository) and a URL for accessing your code or other content. For abstracts about non-software products or projects (for example, educational materials), these should also be made freely available with an appropriate open license. See BOSC submission instructions for more information.
- Updated: If you presented this project at a previous BOSC in any form (talk, demo, and/or poster), your abstract must describe progress since the last BOSC presentation.
If a reviewer determines that an abstract does not meet one or more of the requirements, they will note that on the review form. In that case, the reviewer is not required to supply detailed comments on other aspects of the abstract.
In addition to the abstract requirements listed above, reviewers in the past have also considered the following criteria to evaluate abstracts for a talk. Note that these are not requirements for abstract acceptance, but are additional criteria on which abstracts may be judged. Abstracts that don’t meet these additional criteria are still likely to be offered posters, as long as they meet the requirements described earlier.
- Community impact: Please describe how your work fits into the open source bioinformatics / open science ecosystem. A summary of the current and/or projected community aspect of your project is very helpful.
- Novelty: Innovative approaches are interesting to hear about, but it’s fine to build on existing technology; you don’t need to be novel to get a talk at BOSC. If you can compare your approach with existing approaches, that’s a plus.
- Examples: We like abstracts that include an example of how your approach works (e.g. a figure showing the output of a visualization tool, an example of software input/output, or benchmarks on relevant data).
- Runnability: Although we are happy to consider abstracts that describe early-phase projects, our reviewers are likely to look at your code and try to run it. Reviewers like projects that they are able to download and run, and that are well-documented and easy to use.
- Correctly formatted: Long abstracts must be in PDF format and are limited to two 8.5x11” or A4 pages. We want abstract PDFs to include the title, author name(s), open source license, and code or project URL; however, abstracts are not rejected because of missing metadata.
Starting in 2020, we added a checklist of abstract requirements to the review form. The first step for reviewers is to check whether these criteria are satisfied by the abstract. Abstracts that fail to meet one or more of these requirements will be rejected, but the authors will be encouraged to revise and resubmit their abstracts in the next round.
All comments entered by reviewers are shared with the abstract authors except for those entered in the "Confidential remarks for the chair" field. These confidential remarks are a way to communicate privately to the BOSC Organizers any concerns the reviewer has about the abstract, source, license, or other issues.
The organizers use the scores from the reviewers to help decide (1) whether an abstract should be accepted or rejected, and (2) if accepted, whether it merits a long talk, a short (lightning) talk, or only a poster. In the past, we asked reviewers to assign a single score to each abstract to indicate how high-quality they deemed the abstract, and whether they felt it was suitable for a long talk. We have realized that trying to make a single score serve dual purposes was causing confusion.
We are now asking reviewers to assign two separate scores to each abstract:
- A quality score (ranging from “reject” through “strong accept”)
- A score that indicates how suitable the abstract would be for a long talk (ranging from “only enough content for a lightning talk or poster” to “definitely merits a long talk”).
Some abstracts may be of high quality, but may cover only a small topic (for example, a tool to convert between two file formats) and thus may not earn a high “suitability for long talk” score.
We encourage reviewers to sign reviews when possible; however, we also acknowledge that choosing whether to sign a review may be challenging for some, particularly early-career reviewers who may worry about possible repercussions as a result of a negative review. We therefore leave the decision about whether or not to sign their reviews to the discretion of each reviewer.
After reviewers have submitted their reviews, a sub-committee will evaluate all reviews to make sure they are appropriate, and to ensure all reviewer comments are in line with the conference Code of Conduct. Please see the Review of Reviews SOP to learn more about the process and how the subcommittee is organized. If a decision is made that a review is non-compliant, we will ensure that the review is not included as part of the scoring process and is edited by the sub-committee before passing along any useful comments to the authors. Such decisions will be made on a case-by-case basis.
If an author raises a complaint about a review or any communication during the review process, the review committee will evaluate the complaint to determine the proper course of action, after first notifying the author that the complaint has been received and is being looked into. At least two committee members will then be assigned to evaluate the complaint and whether it potentially violates the Code of Conduct. (Note that the OBF Code of Conduct applies to the review process; for BOSC itself, the ISMB/ECCB Code of Conduct will apply.)
The reviewers’ ratings and detailed comments will be used by the BOSC chairs to decide which abstracts to accept for long or short talks and/or posters. Your constructive comments are also very helpful to authors.
We try to be liberal with poster acceptances for any abstract that shows some connection to open source (or open science), as long as it satisfies the five requirements mentioned above. Typically, talk spots are much more competitive, and we are usually not able to offer talks to all of the good abstracts we receive.
New ISMB rule this year: a presenting author can only present ONE talk or poster. If authors want to present their work as both a talk and a poster, they have to do a separate poster submission with a different author. We will not need to review the same abstract twice (this may require some manual adjustments in the abstract review assignments).