New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[REVIEW]: DeBIR: A Python Package for Dense Bi-Encoder Information Retrieval #5017
Comments
Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks. For a list of things I can do to help you, just type:
For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:
|
|
Wordcount for |
@KonradHoeffner – This is the review thread for the paper. All of our communications will happen here from now on. Please read the "Reviewer instructions & questions" in the first comment above. Please create your checklist typing:
As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines. The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention We aim for the review process to be completed within about 4-6 weeks but please make a start well ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule. |
@amitkumarj441 @JacksonMaxfield @shruti-singh – would any of you be willing to review this submission for JOSS? The submission under consideration is DeBIR: A Python Package for Dense Bi-Encoder Information Retrieval, a Python package for information retrieval and natural language processing. The review process at JOSS is unique: it takes place in a GitHub issue, is open, and author-reviewer-editor conversations are encouraged. You can learn more about the process in these guidelines: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html Based on your experience, we think you might be able to provide a great review of this submission. Please let me know if you think you can help us out! Many thanks |
|
Review checklist for @KonradHoeffnerConflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
|
@Ayuei: The paper has three authors but you are the only contributor of the repository. Can you clarify the contribution of your coauthors? |
@Ayuei: Is it “DeBIR” or “DeBEIR”? The paper is titled DeBIR but the repository title and README contains “DeBEIR”. The code in the paper uses both. Please use a consistent name. Or are they actually two different things? |
@Ayuei: A paper from 2019 is cited next, which talks about how the breakthrough came in 2018 or 2019. Please confirm that 2021 is indeed the correct year. |
@Ayuei: I didn’t know the word “hackability” in this context but I’m not a native speaker and had to look it up to find out that it has two different meanings. If possible I would use a less ambiguous term to not confuse other non-native speakers. |
Please describe author affiliation in more detail. I had to Google “CSIRO’s Data61” to understand what it is. “Australian National University” is also very general. Which department and research group? You could move part of that from the acknowledgements. |
Summary is singular so it should be “is”, not “are”. Also please refer to figure 1 here, which illustrates the stages.
Add “such as”, cite publication
Please explain in more detail which stages of the pipeline are not covered enough by all existing libraries and what the benefits of DeBIR over those existing libraries are. This is important to clarify, how exactly it "either enables some new research challenges to be addressed or makes addressing research challenges significantly better (e.g., faster, easier, simpler)." as part of the "substantial scholarly effort" criterion and for the statement of need criterion. |
@Ayuei – please shorten your BibTeX file to include only those entries cited by this paper. |
@arfon Hi, I've shortened the BibTex and recompiled the paper. @KonradHoeffner Thank you for taking time to review the work and for the detailed comments, I'm still working on addressing concerns and issues. |
👋 @kuutsav would you be willing to review this submission for JOSS? The submission under consideration is DeBIR: A Python Package for Dense Bi-Encoder Information Retrieval, a Python package for information retrieval and natural language processing. The review process at JOSS is unique: it takes place in a GitHub issue, is open, and author-reviewer-editor conversations are encouraged. You can learn more about the process in these guidelines: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html Based on your experience, we think you might be able to provide a great review of this submission. Please let me know if you think you can help us out! Many thanks |
@arfon Thanks for reaching out. I have a very busy schedule this month. If you are still looking for a reviewer at the end of the month, please reach out to me again. All the best! |
I am back to reviewing the codebase and was able to run the Python tests without a hitch, given this library has support enabled for both ES and Solr which works in a standalone manner. However, the automated tests pipeline via Docker runs on MacOs which is not platform-agnostic, though, you may add a bit about instructions on how to run this dockerized image for indexing on Linux. The main pros of this library for reproducing a similar case in IR is the support of an extended list of metrics (NDCG@k, p@k, r@k, etc.). Apart from this, the documentation should clarify on the timeout for ES such as L16. |
Thank you for your review @amitkumarj441, I added more documentation about the ES config in commit b25dd9 I think the testing should be platform agnostic on the main branch (https://github.com/Ayuei/DeBEIR/blob/main/tests/build_test_env.sh), which has seen major refactoring since the paper submission. |
@editorialbot check references |
|
@Ayuei That's helpful to run tests, however, under the main branch in the central instructions file (README.md), you should fix the typo of the shell script file as below
Also, fix the missing and erroneous references as listed above. |
@editorialbot check references |
|
@amitkumarj441 Thanks for catching this! I just fixed this in the main and paper branches. I also fixed the missing and erroneous DOIs (Commit Ayuei/DeBEIR@8a9714a). |
@editorialbot check references |
|
@arfon I think it's much improved, and I'm happy to sign-off the full review. |
@Ayuei – At this point could you make a new release of this software that includes the changes that have resulted from this review. Then, please make an archive of the software in Zenodo/figshare/other service and update this thread with the DOI of the archive? For the Zenodo/figshare archive, please make sure that:
I can then move forward with accepting the submission. |
Hi @arfon, Thank you for the detailed instructions
If there is anything else I need to do, please let me know. |
@editorialbot set 10.5281/zenodo.8103783 as archive |
Done! archive is now 10.5281/zenodo.8103783 |
@editorialbot recommend-accept |
|
|
👋 @openjournals/dsais-eics, this paper is ready to be accepted and published. Check final proof 👉📄 Download article If the paper PDF and the deposit XML files look good in openjournals/joss-papers#4374, then you can now move forward with accepting the submission by compiling again with the command |
@editorialbot accept |
|
Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository. If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file. You can copy the contents for your CITATION.cff file here: CITATION.cff
If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation. |
🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘 |
🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨 Here's what you must now do:
Any issues? Notify your editorial technical team... |
@KonradHoeffner, @amitkumarj441 – many thanks for your reviews here! JOSS relies upon the volunteer effort of people like you and we simply wouldn't be able to do this without you ✨ @Ayuei – your paper is now accepted and published in JOSS ⚡🚀💥 |
🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉 If you would like to include a link to your paper from your README use the following code snippets:
This is how it will look in your documentation: We need your help! The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:
|
Submitting author: @Ayuei (Vincent Nguyen)
Repository: https://github.com/Ayuei/DeBEIR
Branch with paper.md (empty if default branch): paper
Version: v0.0.1
Editor: @arfon
Reviewers: @KonradHoeffner, @amitkumarj441
Archive: 10.5281/zenodo.8103783
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@KonradHoeffner, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @arfon know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Checklists
📝 Checklist for @KonradHoeffner
📝 Checklist for @amitkumarj441
The text was updated successfully, but these errors were encountered: