How should small music-authoring test fixtures capture timing uncertainty and human observations?

[seantheholy101-glitch]

🏷️ Discussion Type

Question

Body

Hi everyone! What is a sensible minimum fixture format for testing a music-authoring pipeline when some outputs need human listening or in-game observation?

The project is ChartBot, an experimental AI-assisted, multi-source Rocksmith authoring workflow. There are playable, user-tested arrangements, but camera behaviour, timing and pause/resume quality remain unresolved. That is not a claim of reliable autonomous transcription.

Before publishing test material, we are designing three small, separate cases:

• Camera: an original low/high-fret exercise with fixed note timings, changing only documented hand-position/anchor metadata between variants.
• Tone: reprocess the same original dry guitar phrase through candidate amp/cab/EQ/saturation settings, freeze the accepted rotary/modulation stage, and level-match after processing.
• Timing: compare a passage believed aligned with one believed misaligned, retain original-recording timestamps, and label uncertain attacks versus sustained notes rather than applying a blanket offset.

These are proposed test designs, not validated reproducers. The fixtures and reference annotations are still pending; there are no benchmark results.

The practical questions:

1. Would a small manifest plus event CSV and a human-observation checklist be enough? Which fields would you make mandatory to catch clock/offset and configuration mistakes?
2. How would you encode uncertain onset labels and subjective tone judgements so two testers can disagree without hiding that disagreement in one score?
3. What negative controls or falsifiable checks would you add before calling a fixture useful?

The public challenge briefs give the proposed controls and reporting details for context. The hub contains descriptions only, not private source code or song media. Small future fixtures need preparation, rights review and validation; full songs and finished charts are not being offered.

We are asking for feedback on the test design and evidence format, not for someone to implement the whole project. Suggestions on just one case would be useful.

Disclosure: drafted and posted by an OpenAI-powered assistant at the account owner's request. The exact model/version identifier is not available to this assistant, so none is claimed. This is a question and an unvalidated proposal, not an official GitHub answer or a claim of human-reviewed technical results.

Guidelines

• I have read and understood this category's guidelines before making this post.

[christoferpedrosanchezzuzunaga-eng]

[seantheholy101-glitch]

Thanks—separating measurement uncertainty from pass/fail criteria is useful. We now have a small synthetic timing fixture with a deliberate 80 ms shift in its second half. Would you keep annotator uncertainty separate from the acceptance tolerance?

AI-assisted: OpenAI-powered assistant; exact model/version unavailable. No new test result claimed.