SCENIC

SCENIC consolidates three research codebases for compact Chinese smart-home command generation under edge IoT constraints. The repository keeps only code that directly supports the three architecture tracks:

• encoder-decoder: ChatLM-mini-Chinese style seq2seq SFT, evaluation, pruning, and ONNX precision checks.
• decoder-only: causal LM SFT, command evaluation, pruning, sparsity experiments, and ONNX export.
• encoder-only: BERT-style response-selection SFT, EM@1/EM@5 evaluation, pruning, and ONNX export.

The consolidation keeps behavior-preserving architecture modules under scenic/architectures/ and exposes user-facing commands under scripts/.

Install

python -m pip install -U pip
python -m pip install -r requirements.txt

For H20, ONNX Runtime GPU, DeepSpeed, or TensorRT runs, install the matching optional stack for the host:

python -m pip install -e ".[h20,trt]"

Data

The shared SCENIC data files were byte-identical across the three source repos and are stored once:

examples/data/SCENIC_full_training_dataset.json
examples/data/SCENIC_full_anchor_positive_negative.json
examples/data/iot_instruction_benchmark_200.json

Smoke Checks

These commands validate local code paths without launching full training:

python -m compileall scenic
python -m pytest tests
python scripts/train/encoder_decoder.py --mode regular --dry-run
python scripts/prune/encoder_decoder.py --model /path/to/checkpoint --dry-run --max-train-examples 5 --max-benchmark-examples 5
python scripts/prune/sparsity_grid.py --architecture decoder_only --model /path/to/checkpoint

Model-backed smoke evaluation uses five examples:

python scripts/evaluate/encoder_decoder.py --model-path /path/to/seq2seq-checkpoint --dataset benchmark --limit 5
python scripts/evaluate/decoder_only.py --model-path /path/to/causal-checkpoint --dataset benchmark --limit 5
python scripts/evaluate/encoder_only.py --checkpoint /path/to/encoder-sft-checkpoint --dataset benchmark --limit 5

Full Evaluation

The standardized evaluator defaults to max_input_length=64, max_new_tokens=32, num_beams=5, num_return_sequences=5, command normalization, per-sample JSONL, aggregate JSON, and CSV outputs.

python scripts/evaluate/encoder_decoder.py --model-path /path/to/seq2seq-checkpoint --dataset benchmark
python scripts/evaluate/decoder_only.py --model-path /path/to/causal-checkpoint --dataset benchmark
python scripts/evaluate/encoder_only.py --checkpoint /path/to/encoder-sft-checkpoint --dataset benchmark

Training-set evaluation uses the same command with --dataset training.

Tables

Aggregate completed evaluation summaries and render a LaTeX table:

python scripts/tables/aggregate_eval.py outputs/evaluation --csv-output outputs/tables/evaluation_summary.csv
python scripts/tables/make_latex_tables.py --csv outputs/tables/evaluation_summary.csv

Reproduction Docs

See:

• docs/repo_inventory.md for what came from each source repo.
• docs/consolidation_report.md for kept/removed files, path changes, and risks.
• docs/reproduction_guide.md for architecture-specific commands.

This repository does not claim hardware sparse-kernel speedup from pruning alone. Use terms such as compression robustness, ONNX deployability, accuracy preservation, and sparsity readiness unless a backend run proves sparse-kernel use.