Repository navigation
Releases
v0.0.16
PATCH: a new export before 1.0. The new entry @mark1russell7/text/references has decodeReferences. The root entry and the entry ./html do not change.
decodeReferences(text)decodes the character references of HTML content, as the tokenizer of the HTML Living Standard does.- The named references come from the table of the standard, with its 2231 names. The longest name wins, and the 106 legacy names are also correct with no semicolon.
- A decimal or hexadecimal reference gives its code point. The value 0, a surrogate and a value above
0x10FFFFgive U+FFFD, and0x80to0x9Fgive the characters of windows-1252. - An ampersand with no reference stays as text. The function does not throw, and it is linear in the length of the text.
- The script
scripts/html-references.tsmakes the table modulesrc/html-references.tsfromentities.jsonof the standard (CC BY 4.0). - The first prediction put the decoder in
./html, with a limit of 9000 bytes. The entry had 9425 bytes in CI. So the decoder is in its own entry,./references, with a limit of 9000 bytes. The limit of./htmlstays at 1100 bytes. - The first users are the alert sources with HTML descriptions in a feed. News feeds need the same function.
Runtime dependencies: no change against 0.0.15 (none).
v0.0.15
PATCH: a new export before 1.0. The entry @mark1russell7/text/search has rankerOf.
rankerOf(corpus, form)computes the IDF and the vector of each document one time. It gives a function(query) => DocumentScore[].- For each query, the ranker gives the same scores as
rankDocuments(corpus, query, form). A property test on 500 seeded corpora holds this. rankDocumentsusesrankerOf, so the two functions share one path. Its results do not change.
v0.0.14
PATCH: new exports before 1.0. The new entry @mark1russell7/text/search has a ranked search with TF-IDF. The root entry does not change.
searchWords(text, options)gives the search keys of the words of a text. The options are stop words, a test of a word and a stem.inverseDocumentFrequency(documents, form)gives the IDF of each term, in the formplain(the default) orsmooth.termVector(words, idf)andcosineSimilarity(a, b)give the TF-IDF vector of a text and the similarity of two vectors.rankDocuments(corpus, query, form)ranks the documents of a corpus for a query.- Property tests on seeded corpora: the order of the documents does not change a weight or a score. A term in each document has the plain weight 0.
v0.0.13
- Version 0.0.13 of
@mark1russell7/texton npm.
v0.0.12
PATCH: new exports before 1.0. The entry @mark1russell7/text/redact finds a secret by its context.
CONTEXT_SHAPESfinds the value of a secret by the text around it: a parameter, a variable or a field with the name of a secret (secret-parameter), a flag (secret-flag), the password of a URL (url-password), anAuthorizationheader (authorization) and the user option of curl (user-password). A match is the value only, so the name stays.isSecretName(name)tells if a name is the name of a secret, by the same rule.- A shape can give the span of a part of its match, with a group with the name
secret. The shapes ofSECRET_SHAPEShave no such group, so their matches do not change. - The default shapes of
findSecretShapesandredactSecretShapesdo not change. A caller addsCONTEXT_SHAPESto its list. - headroom redacts the command lines in its logs with these shapes after this version is on npm.
- The limit of the entry
./redactis 2400 bytes, predicted before the code. The measure is 2275 bytes.
Runtime dependencies: no change against 0.0.11 (none).
v0.0.11
PATCH: a new export before 1.0. The new entry @mark1russell7/text/english has countOf and the new listOf.
listOf(parts, options?)joins the parts into an English list: "a", "a and b" or "a, b and c". No part gives an empty text.- The option
conjunctionsets the word before the last part, and its default is "and". The optionserialCommaadds a comma before the conjunction in a list of three parts or more. - The list join comes from
listTextof@mark1russell7/site-pages. After this version is on npm, site-pages useslistOf,countOfandescapeAttributein place of its three copies. - The root entry does not change, because it has 663 of its 700 bytes. A test holds the entry
./englishat 300 bytes or fewer, a limit predicted before its first measure.
Runtime dependencies: no change against 0.0.10 (none).
v0.0.10
PATCH: new exports before 1.0. The new entry @mark1russell7/text/html has scanHtml, its types and HTML_SKIP.
scanHtmlscans HTML for its text and its start tags in one linear pass, with no DOM. Each start tag gives its name and its attributes.- The content of each element of the option
skipgives no text and no element. The default isscript,styleandtemplate. - The content of a script, a style, a title or a text area has no tags.
- The scan comes from
noScriptTextof@mark1russell7/site-pages. After this version is on npm, site-pages keeps only its floor of text, and its check of the icon link uses the same scan. - A test holds the entry
./htmlat 1100 bytes or fewer, a limit predicted before its first measure. The root entry does not change.
Runtime dependencies: no change against 0.0.9 (none).
v0.0.9
PATCH: new exports before 1.0. The new entry @mark1russell7/text/order has two orders of texts: compareCodeUnits and compareCodePoints.
compareCodeUnitscompares the UTF-16 code units, as the operator<and the default sort do.compareCodePointscompares the Unicode code points. A surrogate pair is one code point, and a lone surrogate is its own code point.- The two orders are different only in one case. At the first code point that is different, one text has a code point above U+FFFF. The other text has a lone surrogate or a code unit from U+E000 to U+FFFF.
- A test compares
compareCodePointswith a reference order for each two texts of 3 code units or fewer, with each pair and each lone surrogate. The same test holds the rule of the case where the two orders are different. - The six copies of the order of the code units in
mark1russell7/alertsmove tocompareCodeUnitsafter this version is on npm. - The orders are in their own entry, so the root entry does not change. With the orders, the root entry had 867 bytes after
esbuildandgzipat level 9, above its limit of 700 bytes. A test holds the entry./orderat 400 bytes or fewer.
Runtime dependencies: no change against 0.0.8 (none).
v0.0.8
- Version 0.0.8 of
@mark1russell7/texton npm.
v0.0.7
PATCH: a new export before 1.0. The new entry @mark1russell7/text/parts has partsAt and partsOf, and their type TextPart.
partsAtdivides a text into parts at ranges of code units. It sorts the ranges, joins the ranges that overlap and clamps each range to the text.partsOfdivides a text into parts at each copy of a text, from the start, with no overlap.- The two functions come from
@mark1russell7/site-pages, which drops its copies in a later version. - The entry
./partsis 326 bytes afteresbuildandgzipat level 9, and a test holds it at 450 bytes or fewer. The root entry does not change.
Runtime dependencies: no change against 0.0.6 (none).