Skip to content

Commit

Permalink
[doc/ref] Make doc/ref/toc-* look nicer
Browse files Browse the repository at this point in the history 
- Add linkify_stop_col to stop linkifying at a certain column
  - Apply it in certain sections in toc-*
- Simplify the regex to identify topics
  - We can be overly aggressive, and then apply the stop column

This gets rid of the 3 space rule, so we can do nicer alignment.
  • Loading branch information
Andy Chu committed May 23, 2024
1 parent b4965a1 commit 5c4a0b9
Show file tree
Hide file tree
Showing 7 changed files with 174 additions and 104 deletions.
28 changes: 15 additions & 13 deletions doc/ref/toc-data.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,16 @@ Siblings: [OSH Topics](toc-osh.html), [YSH Topics](toc-ysh.html)
</h2>

```chapter-links-j8
[J8 Strings] json-string "hi" json-escape \" \\ \u1234
[J8 Strings] json-string "hi"
json-escape \" \\ \u1234
surrogate-pair \ud83e\udd26
u-prefix u'hi' b-prefix b'hi'
j8-escape \u{1f926} \yff
no-prefix 'hi'
j8-escape \' \u{1f926} \yff
u-prefix u'hi'
b-prefix b'hi'
no-prefix 'hi'
[J8 Lines] unquoted-line
[JSON8] json8-num json8-str
json8-list json-dict
[JSON8] json8-num json8-str
json8-list json-dict
json8-comment
[TSV8] column-attrs column-types
```
Expand All @@ -49,8 +51,8 @@ All J8 notation is UTF-8.
</h2>

```chapter-links-packle
[Atoms] Null Bool Int Float Bytes Unicode
[Compound] List Dict
X [Atoms] Null Bool Int Float Bytes Unicode
X [Compound] List Dict
```

<h2 id="errors">
Expand All @@ -59,10 +61,10 @@ All J8 notation is UTF-8.
</h2>

```chapter-links-errors
[UTF8] err-utf8-encode err-utf8-decode
[J8 String] err-j8-str-encode err-j8-str-decode
[UTF8] err-utf8-encode err-utf8-decode
[J8 String] err-j8-str-encode err-j8-str-decode
[J8 Lines] err-j8-lines-encode err-j8-lines-decode
[JSON] err-json-encode err-json-decode
[JSON8] err-json8-encode err-json8-decode
X [Packle] err-packle-encode err-packle-decode
[JSON] err-json-encode err-json-decode
[JSON8] err-json8-encode err-json8-decode
X [Packle] err-packle-encode err-packle-decode
```
27 changes: 15 additions & 12 deletions doc/ref/toc-osh.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,10 @@ Siblings: [YSH Topics](toc-ysh.html), [Data Topics](toc-data.html)
[Iteration] while until for for-expr-sh ((
[Control Flow] break continue return exit
[Grouping] sh-func sh-block { subshell (
[Concurrency] pipe | X |&
ampersand &
[Redirects] redir-file > >> >| < <> X &>
redir-desc >& <&
here-doc << <<- <<<
[Concurrency] pipe | X pipe-amp |& ampersand &
[Redirects] redir-file > >> >| < <> X &>
redir-desc >& <&
here-doc << <<- <<<
[Other Command] dparen (( time X coproc X select
```

Expand All @@ -63,7 +62,9 @@ Siblings: [YSH Topics](toc-ysh.html), [Data Topics](toc-data.html)
Word Language (<a class="group-link" href="chap-word-lang.html">word-lang</a>)
</h2>

```chapter-links-word-lang
<!-- linkify_stop_col is 33 -->

```chapter-links-word-lang_33
[Quotes] osh-string 'abc' $'line\n' "$var"
[Substitutions] command-sub $(command) `command`
var-sub ${var} $0 $9
Expand All @@ -82,15 +83,17 @@ Siblings: [YSH Topics](toc-ysh.html), [Data Topics](toc-data.html)
Other Mini Languages (<a class="group-link" href="chap-mini-lang.html">mini-lang</a>)
</h2>

```chapter-links-mini-lang
<!-- linkify_stop_col is 33 -->

```chapter-links-mini-lang_33
[Arithmetic] arith-context Where legacy arithmetic is allowed
sh-numbers 0xFF 0755 etc.
sh-arith 1 + 2*3 a *= 2
sh-logical !a && b
sh-bitwise ~a ^ b
[Boolean] dbracket [[ vs. the test builtin
bool-expr test ! $x -a $y -o $z
[[ ! $x && $y || $z ]]
[Boolean] dbracket statically parsed expressions with [[
bool-expr [[ ! $x && $y || $z ]]
test ! $x -a $y -o $z
bool-infix [[ $a -nt $b ]] [[ $x == $y ]]
bool-path [[ -d /etc ]]
bool-str [[ -z '' ]]
Expand Down Expand Up @@ -151,7 +154,7 @@ X [Unsupported] enable
[Call Stack] @BASH_SOURCE @FUNCNAME @BASH_LINENO
X @BASH_ARGV X @BASH_ARGC
[Tracing] LINENO
[Process State] X BASHPID X PPID UID EUID
[Process State] X BASHPID X PPID UID EUID
X [Process Stack] BASH_SUBSHELL SHLVL
X [Shell State] BASH_CMDS @DIRSTACK
[Completion] @COMP_WORDS COMP_CWORD COMP_LINE COMP_POINT
Expand All @@ -169,7 +172,7 @@ X [Shell State] BASH_CMDS @DIRSTACK
</h2>

```chapter-links-plugin
[Signals] SIGTERM X SIGINT X SIGABRT SIG...
[Signals] SIGTERM X SIGINT X SIGABRT ...
[Traps] DEBUG ERR EXIT X RETURN
[Words] PS1 X PS2 X PS3 PS4
[Completion] complete
Expand Down
32 changes: 21 additions & 11 deletions doc/ref/toc-ysh.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@ Siblings: [OSH Topics](toc-osh.html), [Data Topics](toc-data.html)
Command Language (<a class="group-link" href="chap-cmd-lang.html">cmd-lang</a>)
</h2>

```chapter-links-cmd-lang
<!-- linkify_stop_col is 33 -->

```chapter-links-cmd-lang_33
[YSH Simple] typed-arg json write (x)
lazy-expr-arg assert [42 === x]
block-arg cd /tmp { echo $PWD }; cd /tmp (; ; blockexpr)
Expand All @@ -53,7 +55,9 @@ Siblings: [OSH Topics](toc-osh.html), [Data Topics](toc-data.html)
Expression Language and Assignments (<a class="group-link" href="chap-expr-lang.html">expr-lang</a>)
</h2>

```chapter-links-expr-lang
<!-- linkify_stop_col is 33 -->

```chapter-links-expr-lang_33
[Assignment] assign =
aug-assign += -= *= /= **= //= %=
&= |= ^= <<= >>=
Expand All @@ -78,7 +82,7 @@ Siblings: [OSH Topics](toc-osh.html), [Data Topics](toc-data.html)
ysh-equals === !== ~== is, is not
ysh-in in, not in
ysh-compare < <= > >= (numbers only)
ysh-logical not and or
ysh-logical not and or
ysh-arith + - * / // % **
ysh-bitwise ~ & | ^ << >>
ysh-ternary '+' if x >= 0 else '-'
Expand All @@ -105,7 +109,9 @@ Siblings: [OSH Topics](toc-osh.html), [Data Topics](toc-data.html)
Word Language (<a class="group-link" href="chap-word-lang.html">word-lang</a>)
</h2>

```chapter-links-word-lang
<!-- linkify_stop_col is 33 -->

```chapter-links-word-lang_33
[Quotes] ysh-string "x is $x" $"x is $x" r'[a-z]\n'
u'line\n' b'byte \yff'
triple-quoted """ $""" r''' u''' b'''
Expand All @@ -122,7 +128,9 @@ Siblings: [OSH Topics](toc-osh.html), [Data Topics](toc-data.html)
Builtin Commands (<a class="group-link" href="chap-builtin-cmd">builtin-cmd</a>)
</h2>

```chapter-links-builtin-cmd
<!-- linkify_stop_col is 42 -->

```chapter-links-builtin-cmd_42
[Memory] append Add elements to end of array
pp asdl cell X gc-stats line proc
[Handle Errors] try Run with errexit, set _status _error
Expand Down Expand Up @@ -168,7 +176,9 @@ X [External Lang] BEGIN END when (awk)
Shell Options (<a class="group-link" href="chap-option.html">option</a>)
</h2>

```chapter-links-option
<!-- linkify_stop_col is 42 -->

```chapter-links-option_42
[Option Groups] strict:all ysh:upgrade ysh:all
[Strictness] ... More Runtime Errors
strict_argv No empty argv
Expand Down Expand Up @@ -206,8 +216,8 @@ X [External Lang] BEGIN END when (awk)
X simple_trap Function name only
[YSH Breaking] ... The Full YSH Language
parse_at_all @ starting any word is an operator
parse_backslash (-u) Allow bad backslashes in "" and $''
parse_backticks (-u) Allow legacy syntax `echo hi`
parse_backslash (-u) Allow bad backslashes in "" and $''
parse_backticks (-u) Allow legacy syntax `echo hi`
parse_bare_word (-u) 'case unquoted' and 'for x in unquoted'
parse_dollar (-u) Allow bare $ to mean \$ (maybe $/d+/)
parse_dbracket (-u) Is legacy [[ allowed?
Expand All @@ -234,7 +244,7 @@ X [External Lang] BEGIN END when (awk)
</h2>

```chapter-links-special-var
[YSH Vars] ARGV X ENV X _ESCAPE
[YSH Vars] ARGV X ENV X _ESCAPE
_this_dir
[YSH Status] _status _error
_pipeline_status _process_sub_status
Expand All @@ -254,10 +264,10 @@ X [Builtin Sub] _buffer

```chapter-links-type-method
[Primitive] Bool Int Float Str Slice Range
[Str] X find(eggex) replace()
[Str] X find() replace()
trim() trimStart() trimEnd()
startsWith() endsWith()
upper() lower() # ascii or unicode
upper() lower()
search() leftMatch()
[Match] group() start() end()
X groups() X groupDict()
Expand Down
81 changes: 31 additions & 50 deletions doctools/help_gen.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,24 +60,31 @@
\]
''', re.VERBOSE)

# TODO:
#
# - Shouldn't be exactly 3 spaces - should be 3 or MORE
# - Three kinds of lines
# - Matches no topics
# - TOPIC+ $ - multiple topics and then EOL
# - TOPIC freeform - one topic, any other text is not linkified

# Complex heuristic to highlight topics.
TOPIC_RE = re.compile(r'''
(X[ ])? # optional deprecation symbol X, then a single space
@? # optional @array, e.g. @BASH_SOURCE
([a-zA-Z0-9:_-]+) # e.g. osh-usage, ysh:all, BASH_REMATCH
( [ ]\S+ # optional: single space then punctuation
([a-zA-Z_][a-zA-Z0-9:_-]+)
# topic names: osh-usage, _status, ysh:all, BASH_REMATCH
( [ ] [^a-zA-Z0-9 ] \S*
# trailer like >> or (make)
|
\(\) # or func()
\(\) # optional () for func()
)?
([ ][ ][ ])? # three spaces means we should keep highlighting
( # order of these 2 clauses matters
[ ]*\n # spaces/newline
|
[ ]+ # 1 or more spaces
)
''', re.VERBOSE)

"""
''', re.VERBOSE)
"""


def _StringToHref(s):
Expand All @@ -90,9 +97,10 @@ def _StringToHref(s):

class TopicHtmlRenderer(object):

def __init__(self, chapter, debug_out):
def __init__(self, chapter, debug_out, linkify_stop_col):
self.chapter = chapter
self.debug_out = debug_out
self.linkify_stop_col = linkify_stop_col

self.html_page = 'chap-%s.html' % chapter

Expand Down Expand Up @@ -172,51 +180,24 @@ def Render(self, line):

pos = m.end()

# Look ahead position
pos2 = pos

# Check the line consists of ONLY multiple topics
num_topics = 0
# Keep matching topics until it doesn't match.
while True:
m = TOPIC_RE.match(line, pos2)
m = TOPIC_RE.match(line, pos)

if not m:
break
num_topics += 1
pos2 = m.end()

if not m.group(4): # not 3 or more spaces
break

# 3 or more spaces
_NEWLINE_ONLY = re.compile(r'[ ]*\n$')
pos = m.end()

if num_topics > 0: # Some lines match zero topics
# The 1-based column number of the end of this topic
col = m.end(2) + 1
if self.linkify_stop_col != -1 and col > self.linkify_stop_col:
#log('STOPPING %d > %d' % (col, self.linkify_stop_col))
break

if _NEWLINE_ONLY.match(line, pos2):
line_is_all_topics = True
log('line is ALL topics - %d of them', num_topics)
else:
line_is_all_topics = False
# Enforced that this is just ONE topic, even though it might look like two
log('line starts with %d topics, but limited to 1 by %r', num_topics,
line[pos2:])

if line_is_all_topics:
# Print all topics
while True:
m = TOPIC_RE.match(line, pos)
if not m:
break

self._PrintTopic(m, out, line_info)
pos = m.end()
else:
# Print one topic
m = TOPIC_RE.match(line, pos)
assert m is not None
self._PrintTopic(m, out, line_info)

self._PrintTopic(m, out, line_info)
out.PrintTheRest()
#log('trailing %r', line[pos:])

out.PrintTheRest()
return f.getvalue()
Expand Down
Loading

0 comments on commit 5c4a0b9

Please sign in to comment.