Porting Zayit to Android: feasibility notes (search stack verified on ART)

[kdroidFilter]

Notes on what an Android port of Zayit would take, starting with its riskiest part: search. The search stack (lexical Lucene, plus the semantic addon's dense KNN on the same fused index) was tested on real Android runtimes (ART) to find the lowest Android version it can run on.

TL;DR

• Lucene 10.4 runs read-only on Android 15+ (API 35), with two Lucene classes patched. Lexical search, KNN and filtered KNN return exactly the same results as on desktop.
• Below API 35 it can't work: Lucene 10 targets Java 21 and calls APIs that ART only got in Android 15.
• The patching can be automated in Gradle, so we don't have to fork Lucene.
• The heavy parts of the port are elsewhere: data size, and a UI that is desktop-only today (Jewel, multi-window).

How it was tested

• The test index mirrors the fused index: text, book_id/is_base_book points, stored line_id, a 384-d COSINE vec field, 20k docs. It was written on desktop with Lucene 10.4.
• On each emulator (x86_64 google_apis), the test opens the index with NIOFSDirectory, then runs a TermQuery, a KnnFloatVectorQuery, and a KnnFloatVectorQuery with the same BooleanQuery filter as VectorSearcher.
• It runs straight on ART through dalvikvm64, with Lucene dexed by D8, in both --min-api 26 and --min-api 35 (the result was the same).

API	Stock Lucene	Patched Lucene
33 (Android 13)	❌	❌ java.lang.StackWalker / java.lang.Record missing
34 (Android 14)	❌	❌ HashMap.newHashMap missing (SegmentInfos): the index doesn't even open
35 (Android 15)	❌ Runtime.version()	✅ same results as the desktop JVM
36 (Android 16)	❌ Runtime.version()	✅ same results as the desktop JVM

"Same results" means the same line_ids in the same order for all three queries.

What breaks on ART, and the fixes

1. org.apache.lucene.internal.vectorization.VectorizationProvider: lookup() starts with Runtime.version().feature(), unguarded. ART has no Runtime.Version, so Lucene104Codec fails to instantiate. That kills everything, lexical search included: postings decoding, VectorUtil, the HNSW scorers.
   Fix: return DefaultVectorizationProvider first when !Constants.IS_HOTSPOT_VM. That is the scalar path; there is no Panama Vector API on Android anyway.
2. org.apache.lucene.util.HotspotVMOptions: on ART the HotSpotDiagnosticMXBean lookup fails as expected, but the catch block then calls Class.getModule()/ModuleLayer, which don't exist on ART. This class is loaded through Constants → RamUsageEstimator → ArrayUtil, so it is everywhere.
   Fix: in that catch, just log the warning.
3. MMapDirectory is Panama-only in Lucene 10. Use NIOFSDirectory, as we already do for GraalVM in LuceneSearchEngine, LuceneLookupSearchService and VectorSearcher. Only the condition needs widening (GraalVM or Android). DbDeltaUpdateService uses FSDirectory.open too.
4. Keep META-INF/services/* from the Lucene jars, and keep the codec, postings and vector format classes in R8. Without them, SPI finds no codec.

Not needed, and still broken: IndexWriter (it also calls Runtime.version()). The app only reads the index, so it doesn't matter.

Not tested yet: lucene-analysis-common, lucene-queryparser and lucene-highlighter, which the lexical engine also uses. They need the same check, and might need more patches.

Gradle patching

Idea: commit the two patched classes as sources (a copy of the upstream files plus the small diff), and swap them into lucene-core for the Android build only. Two possible ways:

A. AGP Instrumentation API (recommended). androidComponents.onVariants { it.instrumentation.transformClassesWith(LucenePatch::class.java, InstrumentationScope.ALL) {} }, with an AsmClassVisitorFactory:

• isInstrumentable() matches only the two class names.
• The visitor drops the original class and replays the patched bytecode (ClassReader(patchedBytes).accept(next, 0)).
• The patched classes are compiled in a tiny module against lucene-core (compileOnly) and loaded as resources.
• This is the standard AGP way: it runs on dependencies, is cached, and doesn't touch the desktop build.

B. Gradle artifact transform on the lucene-core jar (TransformAction + an attribute requested by the Android *RuntimeClasspath configurations). It rewrites the jar and replaces the two .class entries. This works with any toolchain, at the cost of a bit more attribute plumbing.

In both cases, a guard is needed: the patch records the SHA-256 of the upstream classes it replaces, and the build fails if they change. Then a Lucene bump can't silently ship a stale patch. Diff the upstream file, re-apply, update the hash.

Upstreaming is an option too: a guard in VectorizationProvider.lookup() and HotspotVMOptions costs little. But Lucene doesn't officially support Android, and API 35 would remain the floor because of newHashMap, StackWalker, List.getFirst, etc.

Semantic search (addon)

• ONNX Runtime: com.microsoft.onnxruntime:onnxruntime-android has the same ai.onnxruntime.* Java API as the desktop jar. Use the full AAR, not the -mobile one, which has a reduced op set and may lack the int8 quantized ops.
• Tokenizer: the DJL desktop jar only ships linux/win/mac natives. Either use ai.djl.android:tokenizer-native, or rewrite it in pure Kotlin. It is plain NFKC + whitespace + WordPiece, so the rewrite is small.
• Model delivery: the int8 model (>100 MB) can't be a Java resource in the APK, because of the Play size limits. Use Play Asset Delivery or a download with the DB bundle, and adapt the extraction step.
• Performance: one embedding per query is cheap. Re-snippeting dense-only hits embeds every clause of every hit on the page, so it should be batched (embedNormalized already accepts batches). To be measured on a real device.
• Not tested on an emulator yet.

Data size (the real constraint)

On desktop: seforim.db takes about 6.8 GB and seforim.db.lucene about 8.4 GB, of which about 1.9 GB is float32 vectors.

Options:

• Quantize the vectors (Lucene99HnswScalarQuantizedVectorsFormat, int8 ≈ ÷4). This must go with a recall check.
• Make the dense index optional on mobile.
• Offer a "lite" bundle (a subset of books), or remote search as a fallback.
• Delivery has to be an in-app download into app-specific storage, as on desktop.

The rest of the app

• SeforimLibrary core/dao already declare Android targets. The search module and the generator are JVM-only. SeforimApp/src/androidMain exists, but it's only a stub (manifest + icons).
• The UI is the big piece: Jewel (IntelliJ look), Nucleus, multi-window, virtual desktops, and tab drag & drop are desktop-only. Android needs its own shell (Material 3, single-activity navigation), reusing the ViewModels, use cases and commonMain state.
• The ViewModels currently live in jvmMain. Moving the platform-free parts to commonMain (or a shared Android/JVM source set) is a prerequisite.

Proposed next steps

1. Run the same ART test on analysis-common, queryparser and highlighter, then on a real slice of the production index (lexical + KNN), and measure latency with NIOFSDirectory.
2. Prototype the AGP instrumentation patch in a minimal Android module that opens the index (minSdk 35).
3. Check ONNX Runtime + the tokenizer on an emulator and on a real arm64 device.
4. Decide the data strategy (quantized vectors / lite bundle / download).
5. Then scope the UI work.

[kdroidFilter]

Update: ONNX Runtime + tokenizer verified on ART (API 35)

The semantic addon's embedding pipeline was checked the same way as Lucene: on an API 35 emulator (x86_64), run straight on ART through dalvikvm64.

The test reproduces the embedder exactly (v5 normalization → DJL HuggingFaceTokenizer → ONNX Runtime session, ALL_OPT), with the production int8 model and tokenizer.json, on 6 texts: modern queries, a vocalized verse, and text with HTML.

Android dependencies used:

• com.microsoft.onnxruntime:onnxruntime-android:1.20.0, the same ORT version as desktop
• ai.djl.huggingface:tokenizers:0.33.0 + ai.djl.android:tokenizer-native:0.33.0

	Desktop JVM (20 cores)	Emulator API 35 (4 cores, x86_64)
ORT session load	307 ms	176 ms
Tokenizer load	4.1 s	4.0 s
One embedding (warm, avg of 20)	5.6 ms	7.3 ms
Cosine(desktop vector, Android vector)	—	1.000000 on 4 texts, 0.99998 on 2

The max per-component difference is 9e-4, which is int8 kernel rounding. On a query/passage pair, the score goes from 0.5974 on desktop to 0.5978 on Android, so rankings don't change.

No patch needed, unlike Lucene. Things to know for the build:

• libc++_shared.so must be in the APK. DJL's Android AAR depends on it but doesn't bundle it, so it fails with UnsatisfiedLinkError otherwise. It comes from the NDK, or from any dependency that ships it.
• Keep the Java resources from jsoup (HTML entity tables, used by the normalizer) and DJL (META-INF/services), and keep them in R8.
• The tokenizer takes ~4 s to load, on desktop and on Android alike: that's the parsing of tokenizer.json. On mobile, it should be preloaded in the background, or replaced by a small pure-Kotlin WordPiece (it's only NFKC + whitespace + WordPiece). That would also remove the native library.

Still open:

• Real arm64 hardware: different int8 kernels, so expect tiny numeric differences, and the latency needs measuring.
• Lucene + ONNX together inside a real APK.

So the search stack, lexical and semantic, works on Android 15+, with the two Lucene class patches described above as the only code change.