Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement and Rebranding of Exploration Mode to Validation Mode #1617

Merged
merged 24 commits into from Jan 4, 2024

Conversation

Rensvandeschoot
Copy link
Member

@Rensvandeschoot Rensvandeschoot commented Dec 23, 2023

This Pull Request introduces improvements to what was previously known as the Exploration mode, now rebranded as the Validation mode.

Key Changes:

  • Partly Labeled Data Support: The Validation mode now supports datasets that are partially labeled, allowing the refinement of the dataset by correcting any potential misclassifications, which is useful for quality evaluation.
  • Expanded Subset Selection for Prior Knowledge: Users can now select from a broader subset of records, including unseen records, in addition to the previously available relevant and irrelevant ones.
  • Enhanced Review Screen Banner: The review screen banner has been upgraded for better clarity and user experience.

Documentation Updates:
The documentation has been thoroughly updated to reflect these changes and provide guidance on the various modes of operation in ASReview LAB. The revised documentation outlines the appropriate use cases for each mode:

  • Oracle Mode with Unlabeled Data: Ideal for screening a dataset from scratch.
  • Oracle Mode with Partly Labeled Data: Suitable for continuing the screening process, possibly with a different model.
  • Validation Mode with Partly or Fully Labeled Data: Best for validating labels provided by another screener or predicted by Large Language Models (e.g., ChatGPT).
  • Validation Mode with Fully Labeled Data: Useful for learning how the software functions with active learning.
  • Simulation Mode with Fully Labeled Data: Optimal for mimicking the screening process in simulation studies.

Screenshots
Additionally, screenshots in the documentation have been updated to reflect the latest interface and features.

setup_project_modes

project_screening_validation

setup_prior_knowledge_random_validate

@Rensvandeschoot Rensvandeschoot changed the title Introduce validation mode Enhancement and Rebranding of Exploration Mode to Validation Mode Dec 27, 2023
@Rensvandeschoot Rensvandeschoot marked this pull request as ready for review December 27, 2023 10:07
Copy link
Member

@J535D165 J535D165 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, nice work. Mostly small comments.

It might be good to double-check some text and docs for clarity and spelling.

asreview/webapp/api/projects.py Outdated Show resolved Hide resolved
asreview/webapp/api/projects.py Show resolved Hide resolved
asreview/webapp/api/projects.py Outdated Show resolved Hide resolved
asreview/webapp/api/projects.py Outdated Show resolved Hide resolved
asreview/webapp/src/ProjectComponents/ProjectModeSelect.js Outdated Show resolved Hide resolved
docs/source/project_create.rst Outdated Show resolved Hide resolved
docs/source/project_create.rst Outdated Show resolved Hide resolved
docs/source/screening.rst Outdated Show resolved Hide resolved
docs/source/screening.rst Outdated Show resolved Hide resolved
docs/source/screening.rst Outdated Show resolved Hide resolved
docs/source/screening.rst Outdated Show resolved Hide resolved
@J535D165 J535D165 added this to the v1.5 milestone Jan 3, 2024
@J535D165 J535D165 merged commit 4b5eb36 into asreview:master Jan 4, 2024
10 checks passed
@J535D165
Copy link
Member

J535D165 commented Jan 4, 2024

👯‍♂️

@Rensvandeschoot Rensvandeschoot deleted the validation_mode_on_master branch January 5, 2024 19:42
cskaandorp pushed a commit to cskaandorp/asreview that referenced this pull request Apr 22, 2024
Co-authored-by: Jonathan de Bruin <jonathandebruinos@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants