Skip to content

Releases

v0.0.16

Choose a tag to compare

@github-actions github-actions released this 18 Nov 13:23
de12198

PATCH: a new export before 1.0. The new entry @mark1russell7/text/references has decodeReferences. The root entry and the entry ./html do not change.

  • decodeReferences(text) decodes the character references of HTML content, as the tokenizer of the HTML Living Standard does.
  • The named references come from the table of the standard, with its 2231 names. The longest name wins, and the 106 legacy names are also correct with no semicolon.
  • A decimal or hexadecimal reference gives its code point. The value 0, a surrogate and a value above 0x10FFFF give U+FFFD, and 0x80 to 0x9F give the characters of windows-1252.
  • An ampersand with no reference stays as text. The function does not throw, and it is linear in the length of the text.
  • The script scripts/html-references.ts makes the table module src/html-references.ts from entities.json of the standard (CC BY 4.0).
  • The first prediction put the decoder in ./html, with a limit of 9000 bytes. The entry had 9425 bytes in CI. So the decoder is in its own entry, ./references, with a limit of 9000 bytes. The limit of ./html stays at 1100 bytes.
  • The first users are the alert sources with HTML descriptions in a feed. News feeds need the same function.

Runtime dependencies: no change against 0.0.15 (none).

v0.0.15

Choose a tag to compare

@github-actions github-actions released this 18 Nov 06:51
712442a

PATCH: a new export before 1.0. The entry @mark1russell7/text/search has rankerOf.

  • rankerOf(corpus, form) computes the IDF and the vector of each document one time. It gives a function (query) => DocumentScore[].
  • For each query, the ranker gives the same scores as rankDocuments(corpus, query, form). A property test on 500 seeded corpora holds this.
  • rankDocuments uses rankerOf, so the two functions share one path. Its results do not change.

v0.0.14

Choose a tag to compare

@github-actions github-actions released this 18 Nov 05:27
50343a6

PATCH: new exports before 1.0. The new entry @mark1russell7/text/search has a ranked search with TF-IDF. The root entry does not change.

  • searchWords(text, options) gives the search keys of the words of a text. The options are stop words, a test of a word and a stem.
  • inverseDocumentFrequency(documents, form) gives the IDF of each term, in the form plain (the default) or smooth.
  • termVector(words, idf) and cosineSimilarity(a, b) give the TF-IDF vector of a text and the similarity of two vectors.
  • rankDocuments(corpus, query, form) ranks the documents of a corpus for a query.
  • Property tests on seeded corpora: the order of the documents does not change a weight or a score. A term in each document has the plain weight 0.

v0.0.13

Choose a tag to compare

@github-actions github-actions released this 17 Nov 00:21
8f4507a

v0.0.12

Choose a tag to compare

@github-actions github-actions released this 16 Nov 23:34
98449e8

PATCH: new exports before 1.0. The entry @mark1russell7/text/redact finds a secret by its context.

  • CONTEXT_SHAPES finds the value of a secret by the text around it: a parameter, a variable or a field with the name of a secret (secret-parameter), a flag (secret-flag), the password of a URL (url-password), an Authorization header (authorization) and the user option of curl (user-password). A match is the value only, so the name stays.
  • isSecretName(name) tells if a name is the name of a secret, by the same rule.
  • A shape can give the span of a part of its match, with a group with the name secret. The shapes of SECRET_SHAPES have no such group, so their matches do not change.
  • The default shapes of findSecretShapes and redactSecretShapes do not change. A caller adds CONTEXT_SHAPES to its list.
  • headroom redacts the command lines in its logs with these shapes after this version is on npm.
  • The limit of the entry ./redact is 2400 bytes, predicted before the code. The measure is 2275 bytes.

Runtime dependencies: no change against 0.0.11 (none).

v0.0.11

Choose a tag to compare

@github-actions github-actions released this 16 Nov 22:36
d4e758c

PATCH: a new export before 1.0. The new entry @mark1russell7/text/english has countOf and the new listOf.

  • listOf(parts, options?) joins the parts into an English list: "a", "a and b" or "a, b and c". No part gives an empty text.
  • The option conjunction sets the word before the last part, and its default is "and". The option serialComma adds a comma before the conjunction in a list of three parts or more.
  • The list join comes from listText of @mark1russell7/site-pages. After this version is on npm, site-pages uses listOf, countOf and escapeAttribute in place of its three copies.
  • The root entry does not change, because it has 663 of its 700 bytes. A test holds the entry ./english at 300 bytes or fewer, a limit predicted before its first measure.

Runtime dependencies: no change against 0.0.10 (none).

v0.0.10

Choose a tag to compare

@github-actions github-actions released this 16 Nov 14:06
3086b5c

PATCH: new exports before 1.0. The new entry @mark1russell7/text/html has scanHtml, its types and HTML_SKIP.

  • scanHtml scans HTML for its text and its start tags in one linear pass, with no DOM. Each start tag gives its name and its attributes.
  • The content of each element of the option skip gives no text and no element. The default is script, style and template.
  • The content of a script, a style, a title or a text area has no tags.
  • The scan comes from noScriptText of @mark1russell7/site-pages. After this version is on npm, site-pages keeps only its floor of text, and its check of the icon link uses the same scan.
  • A test holds the entry ./html at 1100 bytes or fewer, a limit predicted before its first measure. The root entry does not change.

Runtime dependencies: no change against 0.0.9 (none).

v0.0.9

Choose a tag to compare

@github-actions github-actions released this 16 Nov 11:41
02ce554

PATCH: new exports before 1.0. The new entry @mark1russell7/text/order has two orders of texts: compareCodeUnits and compareCodePoints.

  • compareCodeUnits compares the UTF-16 code units, as the operator < and the default sort do.
  • compareCodePoints compares the Unicode code points. A surrogate pair is one code point, and a lone surrogate is its own code point.
  • The two orders are different only in one case. At the first code point that is different, one text has a code point above U+FFFF. The other text has a lone surrogate or a code unit from U+E000 to U+FFFF.
  • A test compares compareCodePoints with a reference order for each two texts of 3 code units or fewer, with each pair and each lone surrogate. The same test holds the rule of the case where the two orders are different.
  • The six copies of the order of the code units in mark1russell7/alerts move to compareCodeUnits after this version is on npm.
  • The orders are in their own entry, so the root entry does not change. With the orders, the root entry had 867 bytes after esbuild and gzip at level 9, above its limit of 700 bytes. A test holds the entry ./order at 400 bytes or fewer.

Runtime dependencies: no change against 0.0.8 (none).

v0.0.8

Choose a tag to compare

@github-actions github-actions released this 16 Nov 08:54
95ae048

v0.0.7

Choose a tag to compare

@github-actions github-actions released this 16 Nov 02:52
cc57e6a

PATCH: a new export before 1.0. The new entry @mark1russell7/text/parts has partsAt and partsOf, and their type TextPart.

  • partsAt divides a text into parts at ranges of code units. It sorts the ranges, joins the ranges that overlap and clamps each range to the text.
  • partsOf divides a text into parts at each copy of a text, from the start, with no overlap.
  • The two functions come from @mark1russell7/site-pages, which drops its copies in a later version.
  • The entry ./parts is 326 bytes after esbuild and gzip at level 9, and a test holds it at 450 bytes or fewer. The root entry does not change.

Runtime dependencies: no change against 0.0.6 (none).