Skip to content

nature-reviewer-skills v2.0.1

Choose a tag to compare

View all tags
@GeoGeekLab GeoGeekLab released this 12 Jul 06:32
v2.0.1
5b3b8c4
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Overview

Nature Reviewer Skills v2.0.1 is a major architecture, evaluation, reliability, and security release.

This version introduces a shared reviewer core, repository-level continuous integration, structured reviewer-quality evaluation, improved reviewer-pattern retrieval, hardened manuscript processing, and standardized package contracts across all supported scientific domains.

v2.0.1 is the canonical release containing the new v2 architecture.

Highlights

Shared reviewer core

  • Added the src/nature_reviewer_core package.
  • Consolidated manuscript extraction, pattern retrieval, validation, report rendering, and evaluation logic.
  • Reduced duplicated implementations across individual reviewer-skill packages.
  • Standardized shared interfaces, configuration, and package contracts.

Improved reviewer-pattern retrieval

  • Replaced basic keyword-count matching with field-weighted BM25 ranking.
  • Added phrase weighting and query expansion.
  • Added retrieval-confidence indicators.
  • Added diversity-aware result selection to reduce repetitive reviewer concerns.
  • Improved field-level length normalization and ranking behavior.

Reviewer-quality evaluation

  • Added structured benchmark schemas and evaluation utilities.
  • Added major-issue recall measurement.
  • Added precision, recall, and F1 metrics.
  • Added severity-agreement evaluation.
  • Added evidence-anchor coverage measurement.
  • Added cross-reviewer redundancy analysis.
  • Added per-domain evaluation summaries.

Continuous integration and engineering quality

  • Added root-level GitHub Actions workflows.
  • Added testing across supported Python versions.
  • Added repository-wide package validation.
  • Added Ruff linting and formatting checks.
  • Added strict MyPy checks for the shared core.
  • Added Bandit security scanning.
  • Added independent validation for all seven reviewer skills.

Secure manuscript processing

  • Added defensive DOCX XML parsing.
  • Added PDF file-size and page-count limits.
  • Added encrypted-PDF rejection.
  • Added archive path-traversal protection.
  • Added symbolic-link rejection.
  • Added archive expansion-ratio limits.
  • Added extracted-text length limits.
  • Added atomic output writing.

Scientific reliability controls

  • Strengthened claim–evidence alignment requirements.
  • Added explicit prohibitions against fabricating figures, tables, page numbers, references, data, results, or misconduct allegations.
  • Added differentiated responsibilities for multiple simulated reviewers.
  • Added duplicate-concern detection across reviewer reports.
  • Clarified that simulated reviewers are not statistically independent expert reviewers.
  • Strengthened the distinction between correlation, causation, mechanism, and unsupported extrapolation.

Repository standardization

  • Standardized package manifests and metadata.
  • Aligned license declarations.
  • Added versioned documentation and migration guidance.
  • Added pinned development and documentation dependencies.
  • Preserved and normalized domain reviewer-pattern resources.
  • Moved shared Python implementation to a conventional src layout.

Included reviewer skills

This release includes reviewer skills for:

  • Atmospheric science
  • Chemistry
  • Climate and ecology
  • Engineering
  • Hydrology
  • Materials science
  • Remote sensing

Compatibility and migration

This release introduces substantial internal architecture changes.

Users who imported duplicated Python scripts directly from individual skill directories should migrate to the shared nature_reviewer_core interfaces.

Domain-specific SKILL.md files, reviewer guidance, gates, templates, and reviewer-pattern resources remain available.

Users who only install or invoke the reviewer skills through the documented skill interface should require fewer migration changes than users who depend on package-internal Python modules.

Validation

The release has been validated with:

  • Package validation for all seven reviewer skills
  • Shared-core automated tests
  • Independent skill-package tests
  • Ruff lint and formatting checks
  • Strict MyPy type checking
  • Bandit security scanning
  • DOCX extraction and report-generation tests
  • PDF anchor-extraction tests
  • CLI validation and retrieval tests
  • Benchmark-pipeline tests
  • Clean archive extraction and regression validation

Evaluation limitation

The included benchmark validates the operation of the evaluation pipeline and its metrics.

Synthetic benchmark results must not be interpreted as evidence that the system has achieved expert-reviewer performance.

Independent domain-expert evaluation, blinded manuscript testing, negative controls, inter-annotator agreement analysis, calibration studies, and prospective validation remain necessary.

The reviewer skills should be used as structured scientific quality-control assistants, not as replacements for qualified domain experts, editors, or formal peer review.

Recommended upgrade

Users of v1.x are encouraged to upgrade when they need:

  • Shared and reusable reviewer infrastructure
  • Repository-level CI and validation
  • More robust reviewer-pattern retrieval
  • Structured reviewer-quality metrics
  • Stronger manuscript-ingestion security
  • More consistent behavior across scientific domains

Review the migration documentation before upgrading integrations that depend directly on internal scripts from individual reviewer-skill directories.

Assets 2
Loading