GC compaction: node/doc registry, IO callback context, and Reader.string all use stale addresses (GC corruption + SEGV)

[jeremy]

Summary

Three places in libxml-ruby 6.0.0 hand out an address into GC-managed memory and keep using it afterwards. GC compaction relocates the target, and the stale address is then dereferenced. The first is the most serious: it feeds a dead VALUE to rb_gc_mark from mark functions, so it corrupts the GC itself rather than just failing a user call.

All three are reproducible 3/3 with a passing control (same script, GC.verify_compaction_references line removed). Environment: libxml-ruby 6.0.0, libxml2 2.9.13, ruby 4.0.6 (2026-07-14) +PRISM arm64-darwin23.

---

1. The pointer→VALUE registry is invisible to compaction

ext/libxml/ruby_xml_registry.c:8 keeps a file-static st_table *rxml_registry mapping xmlDocPtr/xmlNodePtr to wrapper VALUEs stored as raw st_data_t:

static st_table *rxml_registry;
...
st_insert(rxml_registry, (st_data_t)ptr, (st_data_t)obj);

It is deliberately not a GC root — the header comment notes entries don't prevent collection — which is fine for liveness and fatal for compaction: nothing rewrites the stored VALUEs with rb_gc_location when the wrappers move.

Stored at ruby_xml_document.c:86 (rxml_document_wrap), ruby_xml_document.c:133, ruby_xml_node.c:110 (rxml_node_manage). Read back at ruby_xml_document.c:82, ruby_xml_node.c:136/:143 (inside rxml_node_mark), :154, :575, ruby_xml_attr.c:143, ruby_xml_attr_decl.c:46, ruby_xml_dtd.c:50 (rxml_dtd_mark), ruby_xml_reader.c:75 (rxml_reader_mark).

require "libxml-ruby"
$holder = [LibXML::XML::Document.string("<root><a/></root>")]
GC.verify_compaction_references(expand_heap: true, toward: :empty)
$holder[0].root.doc.root.name        # => NoMethodError on an unrelated object
$holder[0].root; GC.start(full_mark: true, immediate_sweep: true)   # => GC abort

Observed:

[BUG] heap_idx_for_size: allocation size too large (size=4899444305u ...)  -> Aborted
[BUG] try to mark T_NONE object (obj: 0x… T_NONE/, parent: … LibXML::XML::Node)
[BUG] Segmentation fault at 0x0000000000000008        (via XML::Document.file(path))
undefined method 'root' for an instance of Prism::LocalVariableOrWriteNode

Control prints "root" every time.

There is a masking effect worth knowing about: if a Node wrapper for a document-owned node happens to be alive at compaction time, rxml_node_mark calls the pinning rb_gc_mark(doc), the Document doesn't move, and the same script passes. So cases that appear to pass are latent, not safe — adding one GC.start after is enough to turn them into a crash.

Because dcompact receives the data pointer rather than the VALUE, it can't fix this on its own. The practical options are to pin every registered wrapper, or to back the registry with a compaction-aware structure — e.g. an ObjectSpace::WeakMap held in a global registered with rb_gc_register_address — instead of a bare st_table.

---

2. Ruby IO objects handed to libxml2 as raw VALUE callback context

rxml_read_callback (ext/libxml/ruby_xml_io.c:12) does VALUE io = (VALUE) context;. Three sites store that VALUE at construction and let libxml2 call back much later:

• ext/libxml/ruby_xml_parser_context.c:161 — XML::Parser::Context.io, consumed at #parse
• ext/libxml/ruby_xml_html_parser_context.c:200 — XML::HTMLParser::Context.io
• ext/libxml/ruby_xml_reader.c:217 — XML::Reader.io, consumed at every #read

Each does rb_ivar_set(result, IO_ATTR, io) "so it won't get freed", which keeps it alive but movable. The owning rb_data_type_ts (ruby_xml_parser_context.c:24, ruby_xml_html_parser_context.c:138, ruby_xml_reader.c:81) declare no dcompact, and their dmark never touches the io.

require "libxml-ruby"; require "stringio"
$holder = [LibXML::XML::Parser.io(StringIO.new("<root><a/></root>"))]
GC.verify_compaction_references(expand_heap: true, toward: :empty)
$holder[0].parse

Observed: [BUG] Segmentation fault at 0x0000000000000010, and on other runs undefined method 'read' for #<Class:0x…>. Control parses fine. XML::Reader.io, XML::HTMLParser.io and XML::SaxParser.io fail the same way.

Public APIs affected: XML::Parser.io, XML::Document.io, XML::SaxParser.io, XML::HTMLParser.io, XML::HTMLParser::Context.io, XML::Reader.io.

XML::Writer already does this correctly and is the model: it passes a malloc'd rxml_writer_object as the context and marks rwo->output with rb_gc_mark from its dmark (ruby_xml_writer.c:60-67). Verified safe under the same test.

---

3. XML::Reader.string retains a pointer into the Ruby String buffer

ext/libxml/ruby_xml_reader.c:280:

xmlReaderForMemory(StringValueCStr(string), RSTRING_LEN(string), …)

xmlReaderForMemory does not copy, the String is never retained in an ivar, and reads happen lazily on each #read. Compaction relocates the String's bytes.

require "libxml-ruby"
src = (+"<root>") << ("<a>#{"x" * 50}</a>" * 200) << "</root>"
$holder = [LibXML::XML::Reader.string(src)]
src = nil
GC.verify_compaction_references(expand_heap: true, toward: :empty)
20_000.times { |i| +"churn #{i}" }
n = 0; n += 1 while $holder[0].read

Observed, 3/3:

LibXML::XML::Error: Fatal error: Couldn't find end of Start Tag roow<garbage> at :1.

The corrupted tag name is the reused buffer showing through. Control reads all 401 nodes.

Note this needs a reasonably large, runtime-built string plus allocation churn to show — a short literal stays put and the same script passes, which is why it's easy to miss. This one is a dangling char * rather than a stale VALUE, so the fix is different: copy the buffer, or retain and pin the String on the reader.

By contrast XML::Parser::Context.string and XML::HTMLParser::Context.string are safe — xmlCreateMemoryParserCtxt copies.

---

Tested and found safe

XML::SaxParser#parse handler user-data (ruby_xml_sax_parser.c:82 — stored and consumed inside one synchronous xmlParseDocument, so it's stack-live and conservatively pinned; also verified with compaction called from inside on_start_document mid-parse), XML::Writer.io/.string, XML::Error.set_handler (userData is NULL; the block is fetched by name), XML::InputCallbacks.add_scheme (registered with rb_gc_register_address, context is a malloc'd struct), XML::Schema validation, and XML::Parser::Context.string/XML::HTMLParser::Context.string.

I also could not get the ~40 unregistered static VALUE cXML* class globals to misbehave — T_CLASS is movable in principle, but boot-time gem classes sit in dense heap pages that toward: :empty never sources from. Latent rather than demonstrated; mentioning it only so it isn't re-derived.

Context

Found while auditing this bug class across C extensions after fixing the equivalent problem in sqlite3-ruby (sparklemotion/sqlite3-ruby#723). Same root cause confirmed and reported in psych (ruby/psych#811), nokogiri (sparklemotion/nokogiri#3665), fiddle (ruby/fiddle#211) and pg (ged/ruby-pg#734). Happy to send PRs if that's more useful than a report.