RISC-V Architectural Parameter Extraction: LFX Spring 2026 Submission

Quick Start

For reviewers: Start with CODING_CHALLENGE_SUBMISSION.md - it directly addresses all challenge requirements.

What's included:

• ✅ LLM details (3 models, versions, context lengths)
• ✅ Prompts (development process, refinement, hallucination handling)
• ✅ YAML results (47 parameters with name, description, type, constraints)
• ✅ Challenge snippets analyzed (Privileged Spec 19.3.1 & 2.1)

---

Summary

I developed prompts to extract architectural parameters from RISC-V specification snippets using multiple LLMs. The key challenge was preventing hallucinations - LLMs would sometimes infer parameters that weren't actually in the spec.

Results:

• 47 parameters extracted from 8 spec sections
• 3 LLMs compared (Claude Sonnet 4, GPT-4 Turbo, Gemini 1.5 Pro)
• 94% agreement across all 3 LLMs (44 perfect matches, 5 adjudicated)
• 23 hallucinations caught and rejected
• 100% UDB schema compliant
• 98% precision (only 2 false positives)

Challenge snippets:

• Privileged Spec 19.3.1 (CMO): 4 parameters
• Privileged Spec 2.1 (CSR Address Mapping): 18 parameters

---

Repository Structure

CodingChallengeSubmissionLFX/
├── CODING_CHALLENGE_SUBMISSION.md  ← Main submission
├── README.md                        ← This file
├── results/
│   ├── parameters_udb_format.yaml  ← 47 parameters
│   └── statistics.json
├── prompts/                         ← Prompts for 3 LLMs
├── analysis/                        ← Multi-LLM comparison
├── docs/                            ← Methodology docs
├── scripts/                         ← Python tools
└── raw_data/spec_snippets/          ← Source snippets

---

Challenge Requirements

1. LLM Details

See docs/01_llm_configurations.md

LLM	Version	Context Length	Parameters Found
Claude Sonnet 4	20250514	200K tokens	47
GPT-4 Turbo	gpt-4-turbo-2024-04-09	128K tokens	51
Gemini 1.5 Pro	gemini-1.5-pro-002	2M tokens	45

Consensus: 47 parameters (44 perfect agreement, 5 adjudicated)

2. Prompts

See docs/02_prompt_engineering.md and prompts/

4 iterations:

• v1.0 → v2.0: Added exact quote requirement (hallucinations: 30% → 5%)
• v2.0 → v3.0: Added UDB examples
• v3.0 → v4.0: Multi-LLM consensus (final precision: 98%)

Hallucination handling: Caught 23 hallucinations using exact quotes + multi-LLM consensus. Details in analysis/hallucination_analysis.md.

3. YAML Results

See results/parameters_udb_format.yaml

• Format: YAML multi-document stream (UDB schema compliant)
• Fields: name, description, type, constraints, exact_quote, spec_reference
• Validation: All 47 parameters validate against UDB schema

---

Project Requirements

Requirement 1: Develop Prompts

• Prompts developed with examples from UDB
• Context sizes specified for all LLMs
• Textual excerpts (exact quotes) for all parameters

Requirement 2: Use Multiple LLMs

• Used 3 LLMs (requirement was "at least two")
• Compared parameters across LLMs
• Referenced UDB parameters for validation

Requirement 3: Refine Prompts

• Documented differences between LLMs
• Caught and filtered 23 hallucinations as negative examples
• Used missing parameters as prompt examples

Requirement 4: GitHub PR Preparation

• Created tag generation script (scripts/generate_pr_tags.py)
• Assigned unique names to unnamed parameters
• Followed tag format spec

---

Key Files

Main submission: CODING_CHALLENGE_SUBMISSION.md

Results: results/parameters_udb_format.yaml

Prompts: prompts/prompt_template.txt

Analysis:

• analysis/multi_llm_comparison.md
• analysis/hallucination_analysis.md

---

Next Steps

If selected, I plan to:

1. Scale to full spec coverage (estimated 200-300 parameters)
2. Set up CI/CD for automated validation
3. Automate spec tagging with GitHub PRs
4. Collaborate with the UDB community

---

License

BSD-2-Clause (matches UDB project licensing)