Repository navigation
Disk backed conversion
For practical examples against other tools, see the resource usage comparison and its methodology.
A large mesh needs random access: faces refer to positions, texture coordinates and normals declared elsewhere in the source. objtools writes these attributes to temporary files, then maps those files into its address space. Geometry can therefore outgrow RAM while remaining available for direct lookup.
flowchart TB
obj["OBJ"] --> read["Stream source records"]
vrml["VRML"] --> intermediate["Temporary OBJ / MTL"]
intermediate --> read
read --> attributes["Attribute spools on disk"]
read --> faces["Face spool on disk"]
faces --> group["Group faces on disk"]
group --> replay["Replay faces and generate missing normals,<br/>then triangulate and deduplicate"]
attributes -->|"Memory-mapped lookups"| replay
replay --> binary["Write vertices and indices"]
binary --> output["Publish .bin and textures,<br/>then .gltf"]
The diagram follows geometry through the workspace. Image handling runs alongside this path; its policies are described in Images. Each stage reads or writes files instead of collecting the whole mesh in a heap-owned data structure.
These are different views of the importer's resource use:
| Resource | What it holds | How it grows |
|---|---|---|
| Heap | The current face, small records for each object/material/UV-layout key, and glTF metadata | Bounded per face, but grows with the number of objects and material combinations |
| Workspace files | Source attributes, faces, deduplication tables, generated normals and output under construction | Grows with the geometry and its seams |
| Memory-mapped pages | The portions of workspace files currently accessed | The operating system caches pages in RAM and can evict them under memory pressure |
Mapping a file makes its records addressable like an array; it does not require loading the entire file into RAM. Fixed-width attribute records allow a face corner to look up its position, UV or normal directly, without a separate read system call for each lookup.
Mapped pages count toward resident memory (RSS) while they are in RAM. They are backed by workspace files, so the operating system can reclaim them, writing modified pages back as needed. A high RSS can therefore include evictable file cache as well as the importer's heap. Virtual address space also grows with the mappings. See Development for monitoring both RSS and anonymous memory during an import.
flowchart TB
subgraph process["Process address space"]
heap["Heap: current face and metadata"]
mapping["File mappings: attributes, faces and tables"]
end
subgraph ram["Physical RAM"]
anonymous["Anonymous memory for the heap"]
cache["Resident file pages in the OS page cache"]
end
disk["Workspace files on disk"]
heap --> anonymous
mapping -->|"Accessed pages"| cache
disk -->|"Load pages on demand"| cache
cache -->|"Write modified pages back"| disk
Heap allocations and mappings both occupy process address space. Only resident pages consume physical RAM. The operating system can evict file pages from the cache when it needs RAM; their contents remain in the workspace files, with modified pages written back first. Eviction changes RAM use without changing the mapped file's logical size.
The geometry heap is bounded by one face plus small records per key. Metadata remains on the heap, so arbitrarily many objects or materials can still increase memory use. Image conversion has its own allocation limits and is described in Images.
OBJ is read once into separate position, UV and normal spools, a color spool when needed, and a sequential face spool. A spool is a temporary file written in source order. Negative indices are resolved as they appear; positive references are checked against the final attribute counts. The completed attribute spools are then memory-mapped.
VRML first streams through an IndexedFaceSet reader into temporary OBJ/MTL files. Those files follow the same pipeline, preserving the fixed-point coordinate path at the cost of extra disk space and I/O.
Faces are grouped by object, material and UV layout so each combination can become one glTF primitive. Objects are already contiguous in the face spool. When materials interleave within an object, a counting sort writes a second, mapped face spool that makes each combination contiguous while preserving its face order.
Missing normals require an additional replay of the faces. Their accumulated values live in a read-write mapped spool using 24 bytes per source position. The shading rules are described under Normals and smoothing.
Each group's faces are replayed and triangulated by ear clipping. A primitive holds one vertex per distinct (position, UV, normal) corner; a color belongs to its position. The exact triangulation rules handle concave and imperfect polygons without keeping an object in memory.
Deduplication uses a sparse mapped table with a 28-byte entry per source position. Each entry stores that position's first UV/normal combination and output index. Further combinations at the same position, such as UV seams, are chained in another sparse mapped file using 24 bytes per seam corner. A generation number resets the table between primitives without clearing the whole file.
Vertices stream directly to the binary output. Indices first go to a side spool as 32-bit values. At the end of each primitive, its vertex count determines the final index type:
| Vertex count | glTF index type |
|---|---|
| Up to 65,535 | UNSIGNED_SHORT |
| Above 65,535 | UNSIGNED_INT |
The cut-off follows glTF's reservation of each index type's maximum value for primitive restart. A primitive is split only if it would exceed 4,294,967,295 vertices, the 32-bit index limit, and this is reported as a warning. No global mesh optimization is performed: face order within each key and file order of keys are preserved.
Geometry is assembled in the private workspace before publication. The glTF JSON is written last, after its binary and any copied or converted textures. Existing output files are never overwritten. Publication of several files is not an atomic filesystem transaction; on a failed import, cleanup removes only output files created by that import.
Temporary workspace files are removed on success or error unless --keep-tmp is set.
Allow disk space for source attribute and face spools, deduplication tables, generated binary geometry and any converted images. Additional space depends on the source:
- Interleaved materials need a second face spool for grouping.
- UV and normal seams add entries to the seam table.
- Missing normals need the 24-byte-per-position normal spool.
- VRML needs intermediate OBJ/MTL files.
Deduplication files are sparse: their logical sizes and allocated disk space can differ. Actual workspace requirements depend on the model and filesystem, so there is no single input-to-workspace size ratio.
The workspace defaults to the output directory for import, and the input directory for inspect. Use --tmp-dir to choose another real disk. The operating system's temporary directory is often RAM-backed (tmpfs); putting the workspace there consumes memory rather than providing disk backing. An SSD helps once random attribute lookups outgrow the page cache.
If the disk fills while generated smooth normals are being written through their mapping, the process can end with a bus error (SIGBUS) rather than a normal error message.
To bound individual parsing operations, OBJ statements (including lines joined by a trailing \) and VRML tokens are limited to 1 MiB, VRML nesting to 1,024 levels, and polygons to 4,096 corners. Oversized records are rejected explicitly.