docs: add Qiskit code validation IVR example

[ajbozarth]

Misc PR

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

Add comprehensive example demonstrating Instruct-Validate-Repair patternfor Qiskit quantum computing code generation with external validation.

Features:

• Pre-condition validation (prompt and input code)
• Post-condition validation using flake8-qiskit-migration
• Automatic repair loop with detailed error feedback
• Code extraction from markdown blocks
• Real-world use case: fixing deprecated Qiskit APIs

Updated README with example description and requirements.

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code as added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

[github-actions]

The PR description has been updated. Please fill out the template for your PR to be reviewed.

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

[ajbozarth]

This is a work in progress example that I adapted from a notebook by @vabarbosa

We are still in the process of optimizing it, currently all the example prompts succeed on the two larger models, but when run on the small model they only success occasionally with a higher loop budget.

[jakelorocco]

This is a really good example! Left a few comments.

I think you may need to specify installing another package:

QKT100: qiskit.Aer has moved; install separate `qiskit-aer` package and replace `qiskit.Aer` with `qiskit_aer.Aer`
QKT100: qiskit.execute has been removed; explicitly transpile and run the circuit instead (see https://docs.quantum.ibm.com/api/migration-guides/qiskit-1.0-features#execute)

pip install --upgrade qiskit qiskit-aer

And is there a reason we skip this example? I think it seems like an interesting enough one to showcase / highlight so we should make sure to keep it working. If you agree, please make sure to remove the skip and add the packages to the pyproject.

[ajbozarth]

Feedback from Ismael: we should probably separate the helper functions out into a separate file

[ajbozarth]

I think you may need to specify installing another package:

That looks like a IVR response in the example, did you hit that yourself? What was the surrounding output?

And is there a reason we skip this example?

The example is flaky and rarely passes on the small model, it requires the larger models (15-20GB download) to succeed, thus skip. If we manager to get it to consistently work with a small model I'll remove the skip (we plan to try this week)

[jakelorocco]

I think you may need to specify installing another package:

That looks like a IVR response in the example, did you hit that yourself? What was the surrounding output?

And is there a reason we skip this example?

The example is flaky and rarely passes on the small model, it requires the larger models (15-20GB download) to succeed, thus skip. If we manager to get it to consistently work with a small model I'll remove the skip (we plan to try this week)

Makes sense on both points. For the first, I mistook the LLM generated pip command as part of the output for the error the validation printed.

[ajbozarth]

I had Bob put together a summary of all the updates I made today and what is left to work on (apologizes it's wordy, I manually parsed it a bit already):

Review Comments Addressed

I've pushed 3 commits addressing the review feedback:

Addressed

1. Script metadata and uv run command (#576 (comment))

   • Added # /// script block with dependencies for auto-installation
   • Fixed command from uv run python <file> to uv run <file>
   • Removed pytest run comment from docstring
   • Commit: abd92e9

2. Extract helper functions (Ismael's comment)

   • Moved validation helpers to separate validation_helpers.py module
   • Organized code into subdirectory structure
   • Commit: 370a71e

3. Better prompt selection UX (#576 (comment))

   • Removed commented-out prompts from code
   • Added comprehensive README with all example prompts for easy copy/paste
   • Commit: 49c9958

4. Add dedicated README (#576 (comment))

   • Created qiskit_code_validation/README.md with:
     • Quick start instructions
     • Example prompts
     • Expected output
     • Model selection guide
     • Troubleshooting section
   • Updated parent README to document subdirectory
   • Commit: 49c9958

5. Inline configuration (implicit from prompt UX feedback)

   • Moved configuration variables into test function for easier editing
   • Made model selection simple to change
   • Commit: 49c9958

Not Addressed

6. Extract shared code-extraction util (#576 (comment))

   • Investigated: _has_python_code_listing is too functionally different
   • Only common code is the regex line itself, no advantage to utility function
   • Comment: #576 (comment)

7. Redundant code extraction (#576 (comment))

   • Investigated: Not functionally equivalent as expected
   • validate_input_code safely returns valid for non-code prompts
   • validate_qiskit_migration would error in that case (correct for IVR)
   • Comment: #576 (comment)

Still to be addressed

8. MultiTurnStrategy and select_from_failure (#576 (comment), #576 (comment))

   • Blocked on fix in PR fix: Update MultiTurnStrategy to include validation failure reasons in repair messages #633 for MultiTurnStrategy validation failure reasons
   • Will revisit strategy selection after that lands
   • Comments: #576 (comment), #576 (comment)

9. Add grounding context with Qiskit API docs (#576 (comment))

   • This is why example works with granite4:small-h and Qiskit-specific model but not granite4:micro-h
   • @vabarbosa and I will investigate adding docs as grounding_context for smaller models
   • Comment: #576 (comment)
   • Status: Future enhancement, not blocking

Previously Addressed by @vabarbosa

10. Remove redundant validation checks (#576 (comment))

    • Commit: 0b077b7

11. Simplify code candidate check (#576 (comment))

    • Commit: 0b077b7

[ajbozarth]

Per conversation with @vabarbosa and @psschwei I've added a future work section to the README to look into the review items focused on improving the effectiveness of the example. @vabarbosa will be looking into those but we decided they shouldn't block this example from merging. As such I've moved this out of draft.

[psschwei]

[psschwei]

Nice!
(I dropped a backtick from the python block so as to not mess with the rendering)

====== Prompt ======
from qiskit import BasicAer, QuantumCircuit, execute

backend = BasicAer.get_backend('qasm_simulator')

qc = QuantumCircuit(5, 5)
qc.h(0)
qc.cnot(0, range(1, 5))
qc.measure_all()

# run circuit on the simulator

======================

                                                                                                                             === 15:49:14-INFO ======
Starting Mellea session: backend=ollama, model=hf.co/Qiskit/mistral-small-3.2-24b-qiskit-GGUF:latest, context=SimpleContext, model_options={'temperature': 0.8, '@@@max_new_tokens@@@': 2048}
  0%|                                                                                                   | 0/5 [00:00<?, ?it/s]Validation failed with 1 error(s):
QKT101: QuantumCircuit.cnot() has been removed in Qiskit 1.0; use `.cx()` instead
=== 15:49:36-INFO ======
FAILED. Valid: 0/1. Failed:
	 - Code must pass Qiskit migration validation (QKT rules)
 20%|██████████████████▏                                                                        | 1/5 [00:22<01:30, 22.71s/it]=== 15:49:46-INFO ======
SUCCESS
 20%|██████████████████▏                                                                        | 1/5 [00:32<02:08, 32.10s/it]

====== Result (32.1s) ======
```python
from qiskit_aer import AerSimulator
from qiskit import QuantumCircuit
from qiskit.transpiler.preset_passmanagers import generate_preset_pass_manager

backend = AerSimulator()

qc = QuantumCircuit(5, 5)
qc.h(0)
qc.cx(0, range(1, 5))
qc.measure_all()

# Run circuit on the simulator
pass_manager = generate_preset_pass_manager(backend=backend, optimization_level=3)
isa_circuit = pass_manager.run(qc)

result = backend.run(isa_circuit).result()
counts = result.get_counts(0)
print(counts)
```
In this updated code:
1. I replaced `BasicAer` with `AerSimulator` for better compatibility.
2. I changed `QuantumCircuit.cnot()` to `QuantumCircuit.cx()` to comply with Qiskit 1.0+.
3. I added proper transpilation using `generate_preset_pass_manager` before running on the simulator.
4. I updated the execution to follow the current recommended pattern using `backend.run()` directly on the transpiled circuit.
======================

✓ Code passes Qiskit migration validation

[ajbozarth]

(I dropped a backtick from the python block so as to not mess with the rendering)

Fun fact I learned while writing the docs for this PR, if you want to have triple backticks in your code block you can use quadruple backticks for the outside block to prevent issues (https://github.com/ajbozarth/mellea/blob/dbc4b23571e459db399e6c2dc7c8d82956ccbece/docs/examples/instruct_validate_repair/qiskit_code_validation/README.md?plain=1#L141-L177)