Skip to content

docs: add Max's critique of April 24 experiment scope pivot#3

Merged
joshhickson merged 1 commit intoLogoMesh:mainfrom
max-toscano:max/experiment-pivot-critique
Apr 25, 2026
Merged

docs: add Max's critique of April 24 experiment scope pivot#3
joshhickson merged 1 commit intoLogoMesh:mainfrom
max-toscano:max/experiment-pivot-critique

Conversation

@max-toscano
Copy link
Copy Markdown
Contributor

Title: docs: Max's review of April 24 experiment scope pivot

Body:

Summary

  • Side-by-side review of Josh's proposed experiment cuts (commit 3ded7eb) against my independent analysis
  • Identifies where we agree (H100 drop, Exp 4/5 cut, B7 Option 2) and where we diverge (Experiment 3, 3B on RTX 3060, venue, DRAFT-HOLD visibility)
  • Risk analysis for both plans with specific failure modes
  • Two diagnostic tests that should run before committing to either strategy: 3B hardware validation and OEI alpha-sweep

Key disagreements

  1. Experiment 3: Josh cuts entirely. I say keep a simplified 3-point VRAM measurement (30 min to run) so the Proposition has empirical backing.
  2. Llama 3.2 3B on RTX 3060: Josh assumes it fits. Nobody has tested it. One 5-minute command settles this.
  3. Venue: Josh targets Main Track (May 6). I recommend NeurIPS Workshop (July-Aug) given the current evidence base.
  4. DRAFT-HOLD blocks: Josh leaves as internal LaTeX comments. I say convert to visible Limitations or fix the claims before submission.

Action requested

Review the critique, then let's run the two diagnostic tests (3B gate + OEI sweep) before finalizing the strategy.

Side-by-side review of Josh's proposed experiment cuts against an
independent analysis. Covers: where we agree (drop H100, cut Exp 4/5,
B7 Option 2), where we diverge (Experiment 3, 3B feasibility, venue
choice, DRAFT-HOLD visibility), risks in both plans, and recommended
next steps including two diagnostic tests before committing to strategy.
@joshhickson joshhickson merged commit f34ca18 into LogoMesh:main Apr 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants