[REVIEW]: MiNAA: Microbiome Network Alignment Algorithm #5448
Comments
editorialbot
added
C++
Makefile
review
TeX
Track: 2 (BCM)
|
Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks. For a list of things I can do to help you, just type: For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type: |
|
|
|
Wordcount for |
Review checklist for @BechelerConflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
|
|
Hi @waynebhayes! Let me know if you're having trouble generating the checklist or adding your comments to this thread. =D @crsl4 just so you know, @Becheler has asked for an extension to complete their review for personal reasons and I've granted it. @Becheler please keep us up to date about your new intended timeline. [= |
|
No problem! Thanks for keeping me in the loop! Thanks again for agreeing to review this work :) |
|
Hi @graciellehigino @crsl4 |
|
Hi @graciellehigino @crsl4 ! |
|
Hi, sorry I've been struggling a bit with Long COVID. I'll take a look at
the paper now.
…On Tue, May 30, 2023 at 5:52 PM Gracielle Higino ***@***.***> wrote:
Hi @waynebhayes
<https://urldefense.com/v3/__https://github.com/waynebhayes__;!!CzAuKJ42GuquVTTmVmPViYEvSg!OuSVd4GU4O7B5fukkqrcXB7GSGYUucDw9ZTykwbRXk6rgZ-Fp3cYNy89ERgwHaR4ZKZXLcOOfKcNsZKHgYQZqpU$>!
Let me know if you're having trouble generating the checklist or adding
your comments to this thread. =D
@crsl4
<https://urldefense.com/v3/__https://github.com/crsl4__;!!CzAuKJ42GuquVTTmVmPViYEvSg!OuSVd4GU4O7B5fukkqrcXB7GSGYUucDw9ZTykwbRXk6rgZ-Fp3cYNy89ERgwHaR4ZKZXLcOOfKcNsZKHUBAqd3Y$>
just so you know, @Becheler
<https://urldefense.com/v3/__https://github.com/Becheler__;!!CzAuKJ42GuquVTTmVmPViYEvSg!OuSVd4GU4O7B5fukkqrcXB7GSGYUucDw9ZTykwbRXk6rgZ-Fp3cYNy89ERgwHaR4ZKZXLcOOfKcNsZKHIpnWmh8$>
has asked for an extension to complete their review for personal reasons
and I've granted it. @Becheler
<https://urldefense.com/v3/__https://github.com/Becheler__;!!CzAuKJ42GuquVTTmVmPViYEvSg!OuSVd4GU4O7B5fukkqrcXB7GSGYUucDw9ZTykwbRXk6rgZ-Fp3cYNy89ERgwHaR4ZKZXLcOOfKcNsZKHIpnWmh8$>
please keep us up to date about your new intended timeline. [=
—
Reply to this email directly, view it on GitHub
<https://urldefense.com/v3/__https://github.com/openjournals/joss-reviews/issues/5448*issuecomment-1569333987__;Iw!!CzAuKJ42GuquVTTmVmPViYEvSg!OuSVd4GU4O7B5fukkqrcXB7GSGYUucDw9ZTykwbRXk6rgZ-Fp3cYNy89ERgwHaR4ZKZXLcOOfKcNsZKHdzyjMAA$>,
or unsubscribe
<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AE2SJDAQAN5K32DTXKS2RJLXI2I3XANCNFSM6AAAAAAXXY7IGQ__;!!CzAuKJ42GuquVTTmVmPViYEvSg!OuSVd4GU4O7B5fukkqrcXB7GSGYUucDw9ZTykwbRXk6rgZ-Fp3cYNy89ERgwHaR4ZKZXLcOOfKcNsZKHeW-IMxw$>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
*Welcome to the new US Supreme Court*
*Five of the six current right-wing supreme court justices were appointed
by presidents who lost the popular vote. One of them sits in a seat
effectively stolen by Mitch McConnell, who blocked Obama’s late-term
nomination; another sits because McConnell reversed that principle to
fast-track Trump’s late-term nomination. All in all, we have a wildly
unrepresentative group of elitist fringe lunatics, installed in power by
the exact opposite of democratic will, who scrap basic gun control laws in
the wake of horrific mass shootings of children, while calling themselves
“pro-life”.* - Hamilton Nolan, *The Guardian*, June 24, 2022
--
Wayne B. Hayes, Ph.D. Associate Professor in (but not speaking for) The
Department of Computer Science, UC Irvine
|
|
Um... it says "type @editorialbot generate my checklist"... where am I
supposed to type that?
I've read the paper and, from a scientific standpoint, it's woefully
lacking. It's leveraging ancient software for network alignment that's no
longer competitive; their algorithm offers nothing new over and above
existing ones; and they don't describe any *real* data their algorithm can
use. So I'm reluctant to spend much time on this. Furthermore... there
already exist many proven algorithms for network alignment, including my
own, github.com/waynebhayes/SANA, which even has a web interface.
|
|
Hi @waynebhayes Thank you for having taken a look at the software. You should use the That being said, would you be willing to finish your review in the next 3 weeks? Thank you! |
|
Yes I could do it in 3 weeks.
…On Fri, Jul 7, 2023 at 10:32 AM Gracielle Higino ***@***.***> wrote:
Hi @waynebhayes
<https://urldefense.com/v3/__https://github.com/waynebhayes__;!!CzAuKJ42GuquVTTmVmPViYEvSg!IJE5F1OuKJQcdzCSjYzab2PTE4O044vFPptk8yIwQZT5P4VTkBmNQq_cIX8vN587CMMZ5jzghYlOZfqavz1ukUA$>
Thank you for having taken a look at the software. You should use the
@editorialbot commands on this thread on GitHub, always as the first line
of your comment. The bot will then generate a checklist just like the one
that was generated for @Becheler
<https://urldefense.com/v3/__https://github.com/Becheler__;!!CzAuKJ42GuquVTTmVmPViYEvSg!IJE5F1OuKJQcdzCSjYzab2PTE4O044vFPptk8yIwQZT5P4VTkBmNQq_cIX8vN587CMMZ5jzghYlOZfqafURFfKw$>
to guide you on the things that matter to us in a review.
That being said, would you be willing to finish your review in the next 3
weeks?
Thank you!
—
Reply to this email directly, view it on GitHub
<https://urldefense.com/v3/__https://github.com/openjournals/joss-reviews/issues/5448*issuecomment-1625725731__;Iw!!CzAuKJ42GuquVTTmVmPViYEvSg!IJE5F1OuKJQcdzCSjYzab2PTE4O044vFPptk8yIwQZT5P4VTkBmNQq_cIX8vN587CMMZ5jzghYlOZfqa_SLybvY$>,
or unsubscribe
<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AE2SJDBND5R6ZGPL3TX2L6LXPBB2NANCNFSM6AAAAAAXXY7IGQ__;!!CzAuKJ42GuquVTTmVmPViYEvSg!IJE5F1OuKJQcdzCSjYzab2PTE4O044vFPptk8yIwQZT5P4VTkBmNQq_cIX8vN587CMMZ5jzghYlOZfqax2AKxRo$>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
*Welcome to the new US Supreme Court*
*Five of the six current right-wing supreme court justices were appointed
by presidents who lost the popular vote. One of them sits in a seat
effectively stolen by Mitch McConnell, who blocked Obama’s late-term
nomination; another sits because McConnell reversed that principle to
fast-track Trump’s late-term nomination. All in all, we have a wildly
unrepresentative group of elitist fringe lunatics, installed in power by
the exact opposite of democratic will, who scrap basic gun control laws in
the wake of horrific mass shootings of children, while calling themselves
“pro-life”.* - Hamilton Nolan, *The Guardian*, June 24, 2022
--
Wayne B. Hayes, Ph.D. Associate Professor in (but not speaking for) The
Department of Computer Science, UC Irvine
|
|
Also, if I could also point out, most existing network alignment algorithms do not allow a user-specified biological similarity matrix. Is this an available option in SANA? We are also working on a web app, but it's taken a bit longer than expected, but in any case, we think that the inclusion of this matrix will provide flexibility to the algorithm. |
|
@waynebhayes perfect! When you're ready, just visit the GitHub issue thread and copy-paste the command below: When you're done going through the checklist, please comment again saying that you're finished and adding any other comments you might have. Also, please address @crsl4's comment above. Thank you! |
|
Any algorithm that uses sequence similarity must allow some way to input
that similarity; it wouldn't be too hard to substitute any other sort of
similarity you want; about the only possible issue is normalizing, since
sequence similarity can be measured in many ways (identity, p-value, bit
score, etc) and it wouldn't necessarily be clear how to map the similarity
you WANT to use to the similarity that's expected by the algorithm. Still,
I think it would be possible with most algorithms. And yes, SANA is one
such algorithm.
Also, be careful using the word "matrix". You probably don't REALLY want
to use a matrix format, since most protein pairs will have a similarity of
zero, ie., the matrix would be very sparse and there's no point in storing
it that way or requiring users to upload it.
…On Sat, Jul 8, 2023 at 9:26 AM Claudia Solis-Lemus ***@***.***> wrote:
Also, if I could also point out, most existing network alignment
algorithms do not allow a user-specified biological similarity matrix. Is
this an available option in SANA? We are also working on a web app, but
it's taken a bit longer than expected, but in any case, we think that the
inclusion of this matrix will provide flexibility to the algorithm.
—
Reply to this email directly, view it on GitHub
<https://urldefense.com/v3/__https://github.com/openjournals/joss-reviews/issues/5448*issuecomment-1627394263__;Iw!!CzAuKJ42GuquVTTmVmPViYEvSg!N-6tCjRB6B5Dt4sOezCYUk5qB8iS_HxOn9L3-D-RdDfXaBRRzMVxvbyjo4LP6mbz00KqdMVCv0WXD2HW97YbZAE$>,
or unsubscribe
<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AE2SJDBVMDPQRJMA5NNRQ4TXPGC2PANCNFSM6AAAAAAXXY7IGQ__;!!CzAuKJ42GuquVTTmVmPViYEvSg!N-6tCjRB6B5Dt4sOezCYUk5qB8iS_HxOn9L3-D-RdDfXaBRRzMVxvbyjo4LP6mbz00KqdMVCv0WXD2HWawLy1g4$>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
*Welcome to the new US Supreme Court*
*Five of the six current right-wing supreme court justices were appointed
by presidents who lost the popular vote. One of them sits in a seat
effectively stolen by Mitch McConnell, who blocked Obama’s late-term
nomination; another sits because McConnell reversed that principle to
fast-track Trump’s late-term nomination. All in all, we have a wildly
unrepresentative group of elitist fringe lunatics, installed in power by
the exact opposite of democratic will, who scrap basic gun control laws in
the wake of horrific mass shootings of children, while calling themselves
“pro-life”.* - Hamilton Nolan, *The Guardian*, June 24, 2022
--
Wayne B. Hayes, Ph.D. Associate Professor in (but not speaking for) The
Department of Computer Science, UC Irvine
|
In that case, it just means that the details of the similarity computation are not the responsibility of the algorithm, and can/must be abstracted out, for example with a callback defined by the user and invoked by the algorithm.
I think it's fair use of the term as it helps biologists understanding the general idea of the object at hand, and software engineers will be able to generalize the concept easily. Again, if the details and efficiency of the matrix implementation (sparse, symetric etc) are not the responsibility of the algorithm, they can/should be abstracted out of said algorithm, for example by defining a callback that the algorithm can invoke e.g. it is typically done by all STL algorithms to abstract the concept of order: users can give the algorithm the comparison operation of their choice. |
|
I could also add that by "biological similarity", we don't only mean "sequence similarity". Note that the networks here are microbiome networks. The input to create the networks (again, not responsibility of the current algorithm as the networks are taken as given) are usually relative abundances. I also agree with @Becheler. There are sparse representations of matrices (e.g. SparseArrays) which indeed denoted "matrices". |
|
On Fri, Jul 14, 2023 at 2:06 AM Arnaud Becheler ***@***.***> wrote:
and it wouldn't necessarily be clear how to map the similarity you WANT to
> use to the similarity that's expected by the algorithm
In that case, it just means that the details of the similarity computation
are not the responsibility of the algorithm,
Correct.
and can/must be abstracted out, for example with a callback defined by the
user and invoked by the algorithm
Not at all. You just need to describe what the algorithm expects, and the
user can provide their similarity scaled appropriately.
be careful using the word "matrix". You probably don't REALLY want to use
> a matrix format, since most protein pairs will have a similarity of zero
I think it's fair use of the term as it helps biologists understanding the
general idea of the object at hand, and software engineers will be able to
general the concept easily.
Fine.
Again, if the details and efficiency of the matrix implementation (sparse,
symetric etc) are not the responsibility of the algorithm, they can/should
be abstracted out of said algorithm, for example by defining a callback
that the algorithm can invoke e.g. similarity(i,j) without worrying of
how this similiarity is actually computed.
I'm confused. "callback" doesn't solve the problem of how to scale the
similarity, it's just a different method for providing the data to the
algorithm. Using a "matrix" is fine as you first suggested; the "callback"
suggestion is completely a red herring unless I'm missing something.
|
|
On Fri, Jul 14, 2023 at 6:54 AM Claudia Solis-Lemus < ***@***.***> wrote:
I could also add that by "biological similarity", we don't only mean
"sequence similarity".
We've already established that. Many network alignment algorithms can
accept any sort of similarity, and as pointed out in another message, it's
the user's responsibility to provide the similarity scores. My point is
that the paper---unless there is a more recent version?---claims, "yet these
methods are restricted to one specific type of biological similarity (e.g.
sequence similarity in L-GRAAL)." *This is simply not true*. Not even
remotely. The algorithms take any type of similarity you want to provide.
You can provide sequence-based similarity, or Resnik scores based on Gene
Ontology annotations, or any of the several types of graphlet similarity,
or relative abundances as in your case. The similarity is just a number.
For alignment scoring purposes it doesn't matter what it represents.
The paper also says "In addition, unlike most existing network alignment
methods that are tailored for protein or gene interaction networks"... *this
statement is false**.* Network alignment algorithms align *networks*.
*PERIOD.* They can be social networks, PPI networks, gene networks,
microbiome networks, random networks.... it just so happens that the ones
you've looked at were *applied* to protein networks. They could just have
easily been applied to any other types of networks... *including* microbiome
ones.
My primary concern with this work is that the authors don't seem to be at
all familiar with the existing literature on network alignment, or how to
use any of the existing algorithms. They don't define in their paper what
the similarity scores *mean*, and how they are scaled--for example, are
they logarithmic, or linear? Should they be in [0,1], or can they be
arbitrary? Are negative numbers allowed?
And there are no comparisons to existing algorithms at all--and as I said
above, you *cannot* claim that other algorithms "only do sequence". That's
false. So I'm still not convinced the algorithm actually does anything
novel that's not already done by existing algorithms.
Note that the networks here are microbiome networks. The input to create
the networks (again, not responsibility of the current algorithm as the
networks are taken as given) are usually relative abundances.
Fine. So try aligning them with an existing algorithm. Try SANA. It also
has a web interface at http://sana.ics.uci.edu. Or just send me the
networks you'd like to align. I'd be happy to try SANA on them on your
behalf.
I also agree with @Becheler
<https://urldefense.com/v3/__https://github.com/Becheler__;!!CzAuKJ42GuquVTTmVmPViYEvSg!J0BG36PasVe5bAlJ38wl-OYZHtXESGJ5bLm6FD3928AklMzRO6W8Ssh_7iCN8orIlrIPO_5MoepnZDzHsjfJguA$>.
There are sparse representations of matrices (e.g. SparseArrays
<https://urldefense.com/v3/__https://docs.julialang.org/en/v1/stdlib/SparseArrays/__;!!CzAuKJ42GuquVTTmVmPViYEvSg!J0BG36PasVe5bAlJ38wl-OYZHtXESGJ5bLm6FD3928AklMzRO6W8Ssh_7iCN8orIlrIPO_5MoepnZDzHXyScyJA$>)
which indeed denoted "matrices".
Agreed.
|
Our two perspectives are neither exclusive nor contradictory. I think @waynebhayes is missing here several fundamentals of C++ software design, specifically concepts related to the decoupling of data structures and algorithms. When orthogonal aspects of an algorithm begin to be user context dependent rather than developer context dependent, then a popular idiom is to encapsulate these behaviors into a policy-class that implements the required interface. By writing a custom implementation of a given policy, the user of a policy-based library can inject custom behaviors unforeseen by the library implementor. From my perspective, all aspects mentioned by @waynebhayes (scaling, matrix implementation details such as sparse vs dense, meaning and assumptions of the similarity measure etc) are all orthogonal to the alignment algorithm (this specific point @waynebhayes seems to understand). If the policy interface can be reduced to a simple callable operator (what I believe it does: taking a and b points and returning the similarity measured between them - that could be of any return type), then it boils down to requiring the algorithm to take an arbitrary callable (possibly fulfilling some requirements that could be checked with C++20 concepts). I recommend @waynebhayes getting familiar with the Open/Closed principle and reading Modern C++ Design: Generic Programming and Design Patterns Applied by Andrei Alexandrescu, specifically the chapter on Policy-Based Design.
I respectfully disagree. The paper the authors are submitting to is JOSS: unlike many other journals, JOSS focuses on the actual implementation of a software resource. As a Software Engineer in Genetics, I am more than happy to see a diverse crew of small code repositories, with a carefully tested implementation of small components, dedicated to a specific purpose. In the case of software design and code reuse, less is more, and I would personally rather work with the MINAA code base than the more imposing SANA that tightly entangles orthogonal concepts such as Graph, Alignment and SparseMatrix. On another note, I am quite shocked by the insistance of @waynebhayes to compete and refer to their own resource. This is quite unethical reviewing and added to the unpleasant tone of the review, clearly lacks of humility. Contrary to what they say, it is actually surprisingly difficult in Software Engineering to find implementations that "do nothing new" and it is quite naive to believe a competing resource "does it all": most of the time, their idiosyncrasies would complement each other. I would encourage trying to be curious and understand what are the specificities of MINAA's implementation and hypothesis, why using SANA would not have been helpful to the authors in this context (as I said, SANA's tightly coupled class design may be the beginning of an answer), and how a better state of the art could be reached through cooperation and complementation. Respectfully, |
|
I'm sorry, I was under the impression that I was asked to review this
article as an expert in *network alignment* from the standpoint of whether
it constitutes a novel contribution to the *science* of network alignment.
It is my firm opinion that, within that context, it most definitely is *not* a
novel contribution in that it does nothing new that existing algorithms do
not do, does not compare itself to existing algorithms, does not
demonstrate that it can be applied to *real* networks of any
significant size, does not describe how a user interested in the *scientific
outcome* should use the algorithm... among probably other reasons.
In the above context, it is not at all unethical to suggest that a proposed
algorithm compare itself to existing algorithms, in order to demonstrate
novelty. This is a cornerstone of science. I did not *exclusively* suggest
SANA as an algorithm to compare against; I suggested it in this context
because, so far as I am aware, it's the only one (other than the one in the
current paper) that offers an easy-to-use Web interface. I was also
interested in the *scientific* outcome of aligning microbiome networks...
because I am acutely interested in *any* application of network alignment
to new data. My offer of applying SANA to the networks in the paper was
offered out of a genuine feeling of scientific curiosity, not out of any
intent to compete. (And my offer stands.)
If, as you seem to suggest, the goal of this paper and this algorithm is to
demonstrate software engineering concepts in a toy implementation applied
to a toy problem in the context, for example, of a software engineering
class, then I'm 100% OK with that.... but I don't *care* about that
application. If the intent of this paper and this algorithm is to claim
that it does something new that other alignment algorithms do not.... then
my answer, as an expert in the *scientific* field of *network alignment*...
is no. It does not.
I'm done. Thanks for reading.
|
|
Congratulations, everyone! 🎉 |
IDREFS attribute rid references an unknown ID "ref-kurtz2015sparse" |
@crsl4 @reednel oops, I let it slide this reference! Could you fix it? As soon as you do, @Kevin-Mattheus-Moerman will be able to proceed with acceptance [= |
|
@graciellehigino @Kevin-Mattheus-Moerman the reference has been added to |
|
Thanks Reed! We just wait for @Kevin-Mattheus-Moerman to accept then. Thanks! |
|
Hi @Kevin-Mattheus-Moerman any updates? Thanks! |
|
@crsl4 apologies for the delay, because the |
|
@editorialbot recommend-accept |
|
|
|
👋 @openjournals/bcm-eics, this paper is ready to be accepted and published. Check final proof 👉📄 Download article If the paper PDF and the deposit XML files look good in openjournals/joss-papers#5218, then you can now move forward with accepting the submission by compiling again with the command |
editorialbot
added
the
recommend-accept
|
@Kevin-Mattheus-Moerman I don't know if you need our confirmation, but the proof looks good. |
|
@editorialbot set V1.1.0 as version |
|
Done! version is now V1.1.0 |
|
@crsl4 please edit the Zendo archive such that the version tag there is |
|
@Kevin-Mattheus-Moerman I created the Zenodo and Figshare archives, and did set the version to 1.1.0. See Zenodo here: https://zenodo.org/records/10625654. Or if you're referring to something else, can you point me to that? |
|
@reednel I meant that you should include the Hope this helps clarify things. |
|
Understood, thanks for clarifying. Zenodo now reads |
|
@editorialbot accept |
|
|
Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository. If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file. You can copy the contents for your CITATION.cff file here: CITATION.cff
If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation. |
|
🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘 |
|
🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨 Here's what you must now do:
Any issues? Notify your editorial technical team... |
|
@Kevin-Mattheus-Moerman @graciellehigino @Becheler @salix-d Thank you all for your effort toward this publication! |
|
congratulations @crsl4 @reednel on this JOSS publication! 🎉 At this point I would like to thank @graciellehigino for editing this submission. A special thank you also to the reviewers: @Becheler, @salix-d !!! |
|
🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉 If you would like to include a link to your paper from your README use the following code snippets: This is how it will look in your documentation: We need your help! The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:
|
Submitting author: @crsl4 (Claudia Solis-Lemus)
Repository: https://github.com/solislemuslab/minaa
Branch with paper.md (empty if default branch):
Version: V1.1.0
Editor: @graciellehigino
Reviewers: @Becheler, @salix-d
Archive: 10.5281/zenodo.10625654
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@Becheler & @waynebhayes, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @graciellehigino know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Checklists
📝 Checklist for @Becheler
📝 Checklist for @salix-d
The text was updated successfully, but these errors were encountered: