-
Notifications
You must be signed in to change notification settings - Fork 0
nemotron parse spark.stub
id: nemotron-parse-spark name: Nemotron Parse on DGX Spark repo_path: ~/Documents/sw30labs/repos/nemotron-parse-spark remote_url: https://github.com/sw30labs/nemotron-parse-spark.git primary_language: Python framework: transformers + vLLM (NVIDIA Nemotron Parse v1.2) category: developer-tools stacks: [converter, cli, dgx] ingested: 2026-04-22 last_commit_date: 2026-04-21
Test harness for NVIDIA Nemotron Parse v1.2 running on a DGX Spark (Grace Blackwell GB10, 128 GB unified memory). Nemotron Parse is a <1 B-parameter document-understanding model (ViT-H C-RADIO encoder → 1D conv adapter → mBART decoder) that extracts structured text, tables, bounding boxes, and per-element semantic class labels (Title, Section, Caption, Table, Image, Footnote, …) from document images. The repo provides CLI + interactive REPL + batch processor + benchmark over a curated set of arXiv papers and NIST reports, with both transformers and vLLM backends sharing a single postprocessing path.
-
Two backends, one postprocessor:
NemotronParseModel(transformers, default) andNemotronParseVLLM(high-throughput) both produce raw decoder strings and funnel them through_postprocess(), which importsextract_classes_bboxes,transform_bbox_to_original, andpostprocess_textfrom apostprocessingmodule that ships with the model weights. This module only becomes importable afterAutoModel.from_pretrained(..., trust_remote_code=True)— without that, output silently degrades to a singlecls="raw"element. -
Load-bearing v1.2 prompt:
DEFAULT_PROMPTandPROMPT_WITH_IMAGE_TEXTeach contain exactly four directive tokens after</s><s>(<predict_bbox>,<predict_classes>,<output_markdown>,<predict_no_text_in_pic>/<predict_text_in_pic>). The model card requires all four;TestPromptConstantsguards the constants. -
Image sizing contract:
resize_for_modelenforces the model card's 1024×1280 min / 1664×2048 max envelope while preserving aspect ratio.iter_document_imagesis the single entrypoint — routes PDFs through PyMuPDF (pdf2image fallback) at 200 DPI, images through PIL. -
Benchmark instrumentation: CUDA events (not wall-clock) for GPU timing;
torch.cuda.reset_peak_memory_stats()per-page for peak VRAM. Results serialized viadataclasses.asdict(BenchmarkSummary)so new fields auto-flow into JSON. -
DGX Spark specifics: 128 GB unified memory means full bf16 model fits without sharding/offload. On Blackwell, vLLM uses default FlashAttention — the
--attention-backend=TRITON_ATTNflag from the model card is A100/A10-only and is not added here.
-
transformers==4.51.3,accelerate==1.12.0,albumentations==2.0.8,timm==1.0.22(pinned — exact versions from the model card) -
open_clip_torch— required by C-RADIO encoder -
pdf2image>=1.17,pymupdf>=1.24,Pillow>=10— document loading -
vllm>=0.8— optional, commented out inrequirements.txt; install explicitly for--backend vllm -
psutil,tqdm,numpy— bookkeeping -
pytest>=8+pytest-timeout— unit + GPU-marked integration tests
No GitHub Actions. setup.sh creates .venv, installs dependencies, and installs poppler-utils system package. Two pytest markers (-m unit for no-GPU tests, -m gpu for the ~2 GB model-download integration tests) keep CI vs. local runs separable.
python, ocr, llm, cli, benchmark, converter
- developer-tools
- ollama-pdf2markdown — alternative local PDF→Markdown pipeline (Ollama vision models, CPU/Apple Silicon)
- deepseekvl2-pdf-ocr — another vision-LM PDF OCR path
- qwen3-vl-python-ocr-script-mlx — MLX/Apple-Silicon counterpart to this NVIDIA-Blackwell harness