Fix esm2 finetune loss

[yzhang123]

Description

1. fixed esm2 finetuning loss for token classification, regression, and sequence classification to work with data parallelism
   and classification to work with vocab tp parallelism

2. updated fine-tuning notebook to have test evaluation

3. added accuracy threshold to fine-tuning tests

Type of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Refactor
• Documentation update
• Other (please describe):

CI Pipeline Configuration

Configure CI behavior by applying the relevant labels:

• SKIP_CI - Skip all continuous integration tests
• INCLUDE_NOTEBOOKS_TESTS - Execute notebook validation tests in pytest
• INCLUDE_SLOW_TESTS - Execute tests labelled as slow in pytest for extensive testing

Note

By default, the notebooks validation tests are skipped unless explicitly enabled.

Authorizing CI Runs

We use copy-pr-bot to manage authorization of CI
runs on NVIDIA's compute resources.

• If a pull request is opened by a trusted user and contains only trusted changes, the pull request's code will
  automatically be copied to a pull-request/ prefixed branch in the source repository (e.g. pull-request/123)
• If a pull request is opened by an untrusted user or contains untrusted changes, an NVIDIA org member must leave an
  /ok to test comment on the pull request to trigger CI. This will need to be done for each new commit.

Usage

TODO: Add code snippet

Pre-submit Checklist

• I have tested these changes locally
• I have updated the documentation accordingly
• I have added/updated tests as needed
• All existing tests pass successfully

[codecov-commenter]

Codecov Report

❌ Patch coverage is 67.85714% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.18%. Comparing base (2d73084) to head (e58d630).
⚠️ Report is 377 commits behind head on main.

Files with missing lines	Patch %	Lines
...onemo-esm2/src/bionemo/esm2/model/finetune/loss.py	78.26%	5 Missing ⚠️
...mo-esm2/src/bionemo/esm2/model/finetune/dataset.py	20.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #959      +/-   ##
==========================================
- Coverage   84.18%   84.18%   -0.01%     
==========================================
  Files         144      144              
  Lines        9227     9231       +4     
==========================================
+ Hits         7768     7771       +3     
- Misses       1459     1460       +1     

Files with missing lines	Coverage Δ
...mo-esm2/src/bionemo/esm2/model/finetune/dataset.py	79.24% <20.00%> (+2.10%)	⬆️
...onemo-esm2/src/bionemo/esm2/model/finetune/loss.py	85.00% <78.26%> (-9.60%)	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[skothenhill-nv]

thanks yang!