Repository navigation
Releases
Releases · Computational-Hermeneutics/mallette
mallette 0.2.1
- Java's logarithm. mallette carries fdlibm's
log, which Java'sStrictMath.logis and itsMath.loggives (checked on 800 values with Java 27 on arm64), in place of JavaScript'sMath.log. JavaScript's differs from it in the last bit for about one value in two thousand, and from one engine to another (node 22 and Chrome 152 differ), so a model trained in a browser and the same model trained in node could part on a long run, and either could part from MALLET. The tests made with 0.2.0 matched MALLET because none of them met such a value; with 0.2.1 the same model comes out of every engine, as MALLET's.mallette.logis the function. - The Worker takes the diagnostics over MALLET's 20 top words (
diagnosticsWords), however many words of each topic it sends back (numWords): coherence and exclusivity depend on the number. highestOneBitwithoutMath.log2.npm test: 25 checks (Java's log on values where engines differ).
mallette 0.2.0
- Diagnostics, as MALLET's
--diagnostics-file(TopicModelDiagnostics):model.diagnostics(n)gives, for each topic, its tokens, document entropy, word length, coherence, distance from uniform and from the corpus, effective number of words, token–document difference, rank-1 documents, allocation ratio and count, and exclusivity, and for each of its top words the measures defined word by word;model.diagnosticsXML(n)writes MALLET's file. The same file, byte for byte, on every corpus and setting checked. - Inference for new documents (
TopicInferencer,infer-topics):model.inferencer(seed).infer(docs, { iterations, thinning, burnin }), anddocTopics(r)for MALLET's output. The documents are imported as the model's were; words the model does not hold are passed over. The same proportions as MALLET's for the same seed, on MALLET's sample and on 500 documents of Webster's 1913 the model was not trained on. - Documents as tokens:
importDocstakes{ name, tokens }as well as text, for a bench that makes its own tokens (lower-cased and stopwords removed as text's are). - A model as data:
model.toJSON()andmallette.inferencerFrom(data, seed). - A Web Worker:
mallette.trainInWorker(docs, k, o), with progress and cancel, loadingsrc/mallette-worker.js; it returns the result, the model as data, the topic keys and the diagnostics. - A test suite,
npm test, with no Java: MALLET's own outputs for its sample (test/expected/) compared byte for byte, and the rest checked;test/worker.htmlfor the Worker.test/compare.jsnow compares the diagnostics file and inferred topics too (--inferfor other documents), and refuses a seed that MALLET's command line cannot take (a negative one is read as missing). - The README's count of word types in MALLET's sample corrected: 893, not 891.
mallette 0.1.0
The first release of mallette: MALLET's topic model (LDA with hyperparameter optimisation, ParallelTopicModel on one thread) translated into JavaScript, to run in a web browser or in node with no Java and no server.
Given the same documents and the same random seed, mallette gives every token the same topic as MALLET 2.1.1 run with --num-threads 1, and so the same alpha, beta, topic keys and output files.
Checked against MALLET 2.1.1
| Corpus | Documents | Tokens | Topics | Result |
|---|---|---|---|---|
| MALLET's sample (web/en) | 12 | 1,245 | 3, 5, 8, 20; seeds 7, 123, −5, 1; with and without optimisation | all assignments, alpha, beta and topic keys identical |
| Webster's 1913, in 40-line documents | 2,000 | 274,486 | 20; 400 iterations; optimise every 20 | all assignments, alpha, beta and topic keys identical |
test/compare.js runs MALLET and mallette on the same file and reports the first difference, if any.
In this release
importDocs: MALLET's default token pattern, lower-casing and built-in English stoplist (524 words).TopicModel:estimate(), andestimateAsync()for the browser, which runs in steps of about 40 ms, reports progress and can be cancelled without changing the sequence of random numbers.- Output:
result(n), and MALLET's own formats fromtopicKeys(n),docTopics()andstate().
Planned: diagnostics, inference for new documents, reading and writing MALLET state files, a Web Worker, speed, a test suite, and an npm package. See the README.
Written for bench-kit. Apache License 2.0, as MALLET; see NOTICE.