-
-
Notifications
You must be signed in to change notification settings - Fork 111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add custom review tags #1527
Add custom review tags #1527
Conversation
The functionality can also be used to filter records you want to discuss in a later stage (see this discussion) |
@Rensvandeschoot Thank you! Let me know if you have any questions or concerns. |
@voismager Thank you for the PR. This looks promising! To my understanding, tags are used to categorize records. Currently, the relevant/irrelevant labels are the only
I would like to hear feedback from others on these matters. Thanks again for your contribution! |
Thank you again for the PR. This is one of the most promising PRs we have seen. I appreciate the insights shared by everyone, including the potential overlap with reference management software functionalities. I tested the functionality, and everything seems to work correctly. Based on my experiences and discussion with @J535D165 I would like to share the following ideas/suggestions regarding the user experience and interface with you:
Again, thank you all for your efforts and contributions, it is highly appreciated!!! I foresee this work paves the way for numerous promising features in the near future. I am happy to test a new version, and do let us know if you want to discuss the implementation. Also, we have some junior programmers whom I can ask to help with the documentation if you'd like. |
Hi @voismager! Nice pull request, this will open up many possibilities and is a useful feature for many people. I took a look at the part directly related to the state file, since that is where I did the most work myself. Looks good! Two things that I was wondering:
|
This feature is amazing thank you for contributing. It's the only feature missing that holds me back from switching to ASReview. |
We do plan to support adding tags on the fly as a fast follow to this PR. Do you suppose its a blocker for this work? |
Thanks for the amazing, detailed feedback on our PR! We are blown away by the support from this community. Just to let you know we're discussing some of the comments internally (and some team members have been out due to illness and travel) but we hope to have more responses coming up very soon. To respond to the comments from @terrymyc
I see how it seems like we are duplicating the feature of a reference manager, but our main reason for adding tags at this stage is so that it doesn't have to be added in the reference manager, since that would require duplicating work compared to capturing this data at the time of initial review. Some feedback from our academic partners is that reference managers are also not that easy to work with when using tags at a large scale, either adding or doing analysis on the basis of tags. They often end up using external workflows, such as R or Python or a spreadsheet, sometimes bypassing the reference manager altogether. The two-tier tags was also a request from these academic users (more info on the project here), because that is how they would like to codify data. If there are users that do not want two-tiered tags maybe we can provide the option to just have simple tags. |
No, I don't think it is a blocker. The only thing we need to determine is the difference between the 'tag' functionality and the 'notes' option. As long as the tags relate to the labeling decision there is no overlap between the two. If tags are added on the fly, what would you do with those records that have already received a labeling decision? |
Hello, sorry for delayed response, I was in the hospital last couple of weeks. Also I hit close button by accident, please disregard the notification 😄
|
Hi @PeterLombaers,
|
@Rensvandeschoot, I can answer that 😄 In our current implementation, we don't come back to old records in any case. If tags were added later, old records would have an empty list of selected tags, same as if new records were reviewed and none of tags were selected. |
@terrymyc, we were actually planning on doing that, but in the follow up PRs. I hope @yifeimichelle have answered the rest of your points. |
Oe... I hope you are recovered! all the best! |
Sorry for the confusion, I agree that default, pre-determined tags are not needed and screeners should have the flexibility to set tags. However, we should discuss/clarify the distinction between 'notes' and 'tags.' I think that Notes can be more free-form and spontaneous, while tags should be closely tied to the labeling decision or be an essential part of the record. Additionally, the absence of a tag could be interpreted as a 'zero' in the output file. This could either mean that the tag is genuinely not applicable, or it could indicate missing information because the user overlooked the tag. To avoid ambiguity, I would suggest making the tagging process more explicit, for example, in a pop-up after having made the labelling decision. (I think...) |
Indeed very minor for the technical implementation, helpful in the end for the user experience :-) |
Maybe at the same place as the text below Model? @terrymyc |
Add tags at the same time as creating categories, with additional validation.
eb3f82d
to
d678ef9
Compare
Hello @J535D165, our team made some changes to the PR according to our previous discussions:
Could you check our PR once more? 🙂 |
Great work! I can very much appreciate your progress!! I tested the new version and could add tags and export these in a data file without issues!!! I leave it up to @J535D165 to provide technical comments and merge this PR. We can always add more features at a later stage (like changing tags via the history panel). For now, I have one suggestion to consider: For the records selected as prior knowledge, the label for all tags is |
I don't know if it is better to do this in another PR, but could you also add support for tags as a new column in the data format? This way we could easily import/export data with tags in it. |
A big thanks to all!!! This PR is an excellent example of collaboration in science :) I made some changes to the PR because other works started to conflict with this merge. I think the work in #1224 helps resolve some of the UX suggestions in this PR. This work will be part of the upcoming 2.0 release with many other exciting features. Let us know if you or your team are willing to stay involved. We are welcoming all PRs, small and big ones (like this one). I will promise we will merge then fast(er than this one, sorry). Best Jonathan |
Please see #1505 for more details.
List of changes:
Review
page (see screenshot)results
table, so I bumpedstate_version
to 2 and made it so schema will automatically migrate to the new version if version is 1 (seeasreview/state/compatibility.py
)Please let us know what you think, we're looking forward to your feedback 🙂