Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve jsonstringify and stringify operators docs: part 2 #7748

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
created: 20230922121858167
modified: 20230922122333325
tags: [[Operator Examples]] [[jsonstringify Operator]]
title: jsonstringify Operator (Examples)
type: text/vnd.tiddlywiki

Compare the encoding of quotes and control characters in the first example with the analogue [[example for the stringify operator|stringify Operator (Examples)]].
<<.operator-example 1 """[[Backslash \, double quote ", single quote ', tab , line feed
]] +[jsonstringify[]]""">>
<<.operator-example 2 """[[Accents and emojis -> äñøßπ ⌛🎄🍪🍓 without suffix]] +[jsonstringify[]]""">>
<<.operator-example 3 """[[Accents and emojis -> äñøßπ ⌛🎄🍪🍓 with rawunicode suffix]] +[jsonstringify:rawunicode[]]""">>
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
created: 20161017154944352
modified: 20230919124059118
modified: 20230922122319674
tags: [[Operator Examples]] [[stringify Operator]]
title: stringify Operator (Examples)
type: text/vnd.tiddlywiki

<<.operator-example 1 """[[Title with "double quotes" and single ' and \backslash]] +[stringify[]]""">>
Compare the encoding of quotes and control characters in the first example with the analogue [[example for the jsonstringify operator|jsonstringify Operator (Examples)]].
<<.operator-example 1 """[[Backslash \, double quote ", single quote ', tab , line feed
]] +[stringify[]]""">>
<<.operator-example 2 """[[Accents and emojis -> äñøßπ ⌛🎄🍪🍓 without suffix]] +[stringify[]]""">>
<<.operator-example 3 """[[Accents and emojis -> äñøßπ ⌛🎄🍪🍓 with rawunicode suffix]] +[stringify:rawunicode[]]""">>
27 changes: 25 additions & 2 deletions editions/tw5.com/tiddlers/filters/jsonstringify Operator.tid
Original file line number Diff line number Diff line change
@@ -1,12 +1,35 @@
caption: jsonstringify
created: 20171029155051467
from-version: 5.1.14
modified: 20230919124826880
modified: 20230922121404577
op-input: a [[selection of titles|Title Selection]]
op-output: the input with JSON string encodings applied
op-parameter:
op-parameter-name:
op-purpose: deprecated, use <<.olink stringify>> instead
op-purpose: apply JSON string encoding to a string, see also the similar <<.olink stringify>>
op-suffix: <<.from-version "5.1.23">> optionally, the keyword `rawunicode`
op-suffix-name: R
tags: [[Filter Operators]] [[String Operators]]
title: jsonstringify Operator
type: text/vnd.tiddlywiki

The following substitutions are made:

|!Character |!Replacement |!Condition |
|`\` |`\\` |Always |
|`"` |`\"` |Always |
|Carriage return (0x0d) |`\r` |Always |
|Line feed (0x0a) |`\n` |Always |
|Backspace (0x08) |`\b` |Always |
|Form field (0x0c) |`\f` |Always |
|Tab (0x09) |`\t` |Always|
|Characters from 0x00 to 0x1f, except listed above |`\u####` where #### is four hex digits |Always |
|Characters from from 0x80 to 0xffff|`\u####` where #### is four hex digits |If `rawunicode` suffix is not present (default) |
|Characters from 0x80 to 0xffff|<<.from-version "5.1.23">> Unchanged |If `rawunicode` suffix is present |

<<.from-version "5.1.23">> If the suffix `rawunicode` is present, Unicode characters above 0x80 (such as ß, ä, ñ or 🎄) will be passed through unchanged. Without the suffix, they will be substituted with `\u` codes, which was the default behavior before 5.1.23. Characters outside the Basic Multilingual Plane, such as 🎄 and other emojis, will be encoded as a UTF-16 surrogate pair, i.e. with two `\u` sequences.

<<.note """Mind the differences compared to <<.olink stringify>> in encoding of single quotes and control characters (0x00 to 0x1f).
""">>

<<.operator-examples "jsonstringify">>
19 changes: 8 additions & 11 deletions editions/tw5.com/tiddlers/filters/stringify_Operator.tid
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
caption: stringify
created: 20161017153038029
from-version: 5.1.14
modified: 20230919130847809
modified: 20230922121406947
op-input: a [[selection of titles|Title Selection]]
op-output: the input with ~JavaScript string encodings applied
op-parameter:
op-parameter-name:
op-purpose: apply ~JavaScript string encoding to a string
op-purpose: apply ~JavaScript string encoding to a string, see also the similar <<.olink jsonstringify>>
op-suffix: <<.from-version "5.1.23">> optionally, the keyword `rawunicode`
op-suffix-name: R
tags: [[Filter Operators]] [[String Operators]]
Expand All @@ -18,19 +18,16 @@ The following substitutions are made:
|!Character |!Replacement |!Condition |
|`\` |`\\` |Always |
|`"` |`\"` |Always |
|Carriage return (0x0d) |`\r` |Always |
|`'` |`\'` |Always |
|Line feed (0x0a) |`\n` |Always |
|Backspace (0x08) |`\b` |Always |
|Form field (0x0c) |`\f` |Always |
|Tab (0x09) |`\t` |Always |
|Characters from 0x00 to 0x1f |`\x##` where ## is two hex digits |Always |
|Carriage return (0x0d) |`\r` |Always |
|Characters from 0x00 to 0x1f, except listed above |`\x##` where ## is two hex digits |Always |
|Characters from 0x80 to 0xffff|`\u####` where #### is four hex digits |If `rawunicode` suffix is not present (default) |
|Characters from 0x80 to 0xffff|<<.from-version "5.1.23">> Unchanged |If `rawunicode` suffix is present |

<<.from-version "5.1.23">> If the suffix `rawunicode` is present, Unicode characters above 0x80 (such as ß, ä, ñ or 🎄) will be passed through unchanged. Without the suffix, they will be substituted with `\u` codes, which was the default behavior before 5.1.23.

<<.note """Characters outside the Basic Multilingual Plane, such as 🎄 and other emojis, will be encoded as a UTF-16 surrogate pair, i.e. with two `\u` sequences.""">>
<<.from-version "5.1.23">> If the suffix `rawunicode` is present, Unicode characters above 0x80 (such as ß, ä, ñ or 🎄) will be passed through unchanged. Without the suffix, they will be substituted with `\u` codes, which was the default behavior before 5.1.23. Characters outside the Basic Multilingual Plane, such as 🎄 and other emojis, will be encoded as a UTF-16 surrogate pair, i.e. with two `\u` sequences.
Copy link
Contributor

@CrossEye CrossEye Sep 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No critique here, just a question for the core team. How long do we keep these "New in 5.1.23" annotations around? There's been the entire 5.2.x train as well as 5.3.0 and 5.3.1, and this would likely appear in 5.3.2. Is there some official shelf-life?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! When we introduced the version tags, I was indeed thinking that we'd have some sort of statute of limitations where we'd drop the older ones. We could do it crudely by searching and replacing in an editor, but I'd prefer to update the macro so that it parses the version number and doesn't produce any output if the version is less than some threshold.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take a look into it, together with some other documentation-macro related edits I had in mind. Would a threshold of >= 5.2 be reasonable? That is, everything below 5.2 will be hidden?


<<.olink jsonstringify>> is considered deprecated, as it duplicates the functionality of <<.op stringify>>.
<<.note """Mind the differences compared to <<.olink jsonstringify>> in encoding of single quotes and control characters (0x00 to 0x1f).
""">>

<<.operator-examples "stringify">>
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
created: 20220427174702859
modified: 20230809113620964
modified: 20230922122551197
tags: [[JSON in TiddlyWiki]] Learning
title: Constructing JSON tiddlers

Expand All @@ -13,4 +13,4 @@ At a high level, we have several ways to generate JSON data in TiddlyWiki's own
* [[jsontiddler Macro]]
* [[jsontiddlers Macro]]

When constructing JSON data manually, the [[stringify Operator]] is needed to ensure that any special characters are properly escaped.
When constructing JSON data manually, the [[jsonstringify Operator]] is needed to ensure that any special characters are properly escaped.