[libc][docs] Introduce docgen #87682
Conversation
This script+config should help us generate more consistent documentation wrt.
what we currently support or not.
As an example usage:
$ ./libc/utils/docgen/docgen.py fenv.h
Will spit out an RST formatted table that can be copy+pasted into our docs.
The config is not filled out entirely, but doing so and then updating our docs
would be great beginner bugs for new contributors.
Having python+yaml generate things like docs, or headers (as imagined in
https://github.com/nickdesaulniers/llvm-project/tree/hdr-gen2) is perhaps
easier to work with than tablegen, and doesn't introduce a dependency on a host
tool that needs to be compiled from llvm sources before building the rest of
the libc. This can probably be merged with whatever we end up doing to replace
libc-hdrgen.
Please use
https://llvm.org/docs/CodingStandards.html#python-version-and-source-code-formatting
for keeping this file formatted.
|
@llvm/pr-subscribers-libc Author: Nick Desaulniers (nickdesaulniers) ChangesThis script+config should help us generate more consistent documentation wrt. As an example usage: Will spit out an RST formatted table that can be copy+pasted into our docs. The config is not filled out entirely, but doing so and then updating our docs Having python+yaml generate things like docs, or headers (as imagined in Please use Full diff: https://github.com/llvm/llvm-project/pull/87682.diff 4 Files Affected:
diff --git a/libc/docs/fenv.rst b/libc/docs/fenv.rst
new file mode 100644
index 00000000000000..b0518f7a0b257e
--- /dev/null
+++ b/libc/docs/fenv.rst
@@ -0,0 +1,64 @@
+.. include:: check.rst
+
+fenv.h Functions
+================
+
+.. list-table::
+ :widths: auto
+ :align: center
+ :header-rows: 1
+
+ * - Function
+ - Implemented
+ - Standard
+ * - fe_dec_getround
+ - |check|
+ - 7.6.5.3
+ * - fe_dec_setround
+ - |check|
+ - 7.6.5.6
+ * - feclearexcept
+ - |check|
+ - 7.6.4.1
+ * - fegetenv
+ - |check|
+ - 7.6.6.1
+ * - fegetexceptflag
+ - |check|
+ - 7.6.4.2
+ * - fegetmode
+ - |check|
+ - 7.6.5.1
+ * - fegetround
+ - |check|
+ - 7.6.5.2
+ * - feholdexcept
+ - |check|
+ - 7.6.6.2
+ * - feraiseexcept
+ - |check|
+ - 7.6.4.3
+ * - fesetenv
+ - |check|
+ - 7.6.6.3
+ * - fesetexcept
+ - |check|
+ - 7.6.4.4
+ * - fesetexceptflag
+ - |check|
+ - 7.6.4.5
+ * - fesetmode
+ - |check|
+ - 7.6.5.4
+ * - fesetround
+ - |check|
+ - 7.6.5.5
+ * - fetestexcept
+ - |check|
+ - 7.6.4.7
+ * - fetestexceptflag
+ - |check|
+ - 7.6.4.6
+ * - feupdateenv
+ - |check|
+ - 7.6.6.4
diff --git a/libc/docs/index.rst b/libc/docs/index.rst
index 370fcd843974e8..65ccb91e92ffe1 100644
--- a/libc/docs/index.rst
+++ b/libc/docs/index.rst
@@ -66,6 +66,7 @@ stages there is no ABI stability in any form.
strings
stdio
stdbit
+ fenv
libc_search
c23
diff --git a/libc/utils/docgen/api.yml b/libc/utils/docgen/api.yml
new file mode 100644
index 00000000000000..3e5429f68cdf8c
--- /dev/null
+++ b/libc/utils/docgen/api.yml
@@ -0,0 +1,107 @@
+ctype.h:
+ functions:
+ isalnum:
+ returns: int
+ params:
+ - int
+ isalpha:
+ returns: int
+ params:
+ - int
+ isblank:
+ returns: int
+ params:
+ - int
+ # TODO: fill me out!
+fenv.h:
+ macros:
+ - __STDC_VERSION_FENV_H__
+ functions:
+ feclearexcept:
+ returns: int
+ params:
+ - int
+ defined: '7.6.4.1'
+ fegetexceptflag:
+ returns: int
+ params:
+ - fexcept_t*
+ - int
+ defined: '7.6.4.2'
+ feraiseexcept:
+ returns: int
+ params:
+ - int
+ defined: '7.6.4.3'
+ fesetexcept:
+ returns: int
+ params:
+ - int
+ defined: '7.6.4.4'
+ fesetexceptflag:
+ returns: int
+ params:
+ - const fexcept_t*
+ - int
+ defined: '7.6.4.5'
+ fetestexceptflag:
+ returns: int
+ params:
+ - const fexcept_t*
+ - int
+ defined: '7.6.4.6'
+ fetestexcept:
+ returns: int
+ params:
+ - int
+ defined: '7.6.4.7'
+ fegetmode:
+ returns: int
+ params:
+ - femode_t*
+ defined: '7.6.5.1'
+ fegetround:
+ returns: int
+ params: []
+ defined: '7.6.5.2'
+ fe_dec_getround:
+ returns: int
+ params: []
+ defined: '7.6.5.3'
+ fesetmode:
+ returns: int
+ params:
+ - const femode_t*
+ defined: '7.6.5.4'
+ fesetround:
+ returns: int
+ params:
+ - int
+ defined: '7.6.5.5'
+ fe_dec_setround:
+ returns: int
+ params:
+ - int
+ defined: '7.6.5.6'
+ fegetenv:
+ returns: int
+ params:
+ - fenv_t*
+ defined: '7.6.6.1'
+ feholdexcept:
+ returns: int
+ params:
+ - fenv_t*
+ defined: '7.6.6.2'
+ fesetenv:
+ returns: int
+ params:
+ - const fentv_t*
+ defined: '7.6.6.3'
+ feupdateenv:
+ returns: int
+ params:
+ - const fenv_t*
+ defined: '7.6.6.4'
+
+# TODO: yay fill out moar stuff!!!1
diff --git a/libc/utils/docgen/docgen.py b/libc/utils/docgen/docgen.py
new file mode 100755
index 00000000000000..db5cdf1d327b82
--- /dev/null
+++ b/libc/utils/docgen/docgen.py
@@ -0,0 +1,77 @@
+#!/usr/bin/env python
+#
+# ====- Generate documentation for libc functions ------------*- python -*--==#
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+# ==-------------------------------------------------------------------------==#
+from argparse import ArgumentParser
+from pathlib import Path
+import sys
+import yaml
+
+
+def load_api():
+ p = Path(Path(__file__).resolve().parent, "api.yml")
+ api = p.read_text(encoding="utf-8")
+ return yaml.load(api, Loader=yaml.FullLoader)
+
+
+# TODO: we may need to get more sophisticated for less generic implementations.
+# Does libc/src/{hname minus .h suffix}/{fname}.cpp exist?
+def is_implemented(hname, fname):
+ return Path(
+ Path(__file__).resolve().parent.parent.parent,
+ "src",
+ hname.rstrip(".h"),
+ fname + ".cpp",
+ ).exists()
+
+
+def print_functions(header, functions):
+ for key in sorted(functions.keys()):
+ print(f" * - {key}")
+
+ if is_implemented(header, key):
+ print(" - |check|")
+ else:
+ print(" -")
+
+ if "defined" in functions[key]:
+ print(f' - {functions[key]["defined"]}')
+ else:
+ print(" -")
+
+
+def print_header(header, api):
+ fns = f"{header} Functions"
+ print(fns)
+ print("=" * (len(fns)))
+ print(
+ f"""
+.. list-table::
+ :widths: auto
+ :align: center
+ :header-rows: 1
+
+ * - Function
+ - Implemented
+ - Standard"""
+ )
+ # TODO: how do we want to signal implementation of macros?
+ print_functions(header, api["functions"])
+
+
+def parse_args(header_choices):
+ parser = ArgumentParser()
+ parser.add_argument("header_name", choices=header_choices)
+ return parser.parse_args()
+
+
+if __name__ == "__main__":
+ api = load_api()
+ args = parse_args(api.keys())
+
+ print_header(args.header_name, api[args.header_name])
|
libc/utils/docgen/api.yml
Outdated
| ctype.h: | ||
| functions: | ||
| isalnum: | ||
| returns: int | ||
| params: | ||
| - int | ||
| isalpha: | ||
| returns: int | ||
| params: | ||
| - int | ||
| isblank: | ||
| returns: int | ||
| params: | ||
| - int | ||
| # TODO: fill me out! | ||
| fenv.h: |
There was a problem hiding this comment.
Rather than have one long long running file, perhaps it would be better to rely on a convention where we have individual yaml files, one per header such as ctype.h.yml and fenv.h.yml (or ctype.yml fenv.yml). Then the header name is implied by the file name, and we can keep the individual files shorter and perhaps better organized?
libc/utils/docgen/api.yml
Outdated
| returns: int | ||
| params: | ||
| - int | ||
| defined: '7.6.4.1' |
There was a problem hiding this comment.
Perhaps defined should be a list. For example, if something is specified by both C and POSIX, maybe it makes sense to refer to BOTH sections?
The C standard can't be hyperlinked to AFAIK unfortunately, but POSIX can be IIRC. Maybe supporting hyperlinking would be nice.
libc/utils/docgen/api.yml
Outdated
| returns: int | ||
| params: | ||
| - int |
There was a problem hiding this comment.
Having the function signature encoded here makes more sense for replacing libc-hdrgen, but isn't strictly required for documentation purposes (at the moment; perhaps we could generate something akin to very very minimal man pages, eventually).
libc/utils/docgen/docgen.py
Outdated
|
|
||
| # TODO: we may need to get more sophisticated for less generic implementations. | ||
| # Does libc/src/{hname minus .h suffix}/{fname}.cpp exist? | ||
| def is_implemented(hname, fname): |
There was a problem hiding this comment.
nit: Do you want to do type annotation for our python scripts?
There was a problem hiding this comment.
Yes, I think it's a nice touch. I don't have much experience writing/checking these, but I have seen them before.
IIRC, there's a separate utility for checking these than the python vs/runtime itself? Got any pointers?
cc @JustinStitt
There was a problem hiding this comment.
The last time I checked, type hints were ignored by CPython/the Python interpreter, and just used by language servers.
There was a problem hiding this comment.
Or some linter tools to help enforcing them.
There was a problem hiding this comment.
@nickdesaulniers stuff like pylance, ruff or basedpyright can parse these type hints.
count me as a +1 for type-hinting everywhere!
This script+config should help us generate more consistent documentation wrt.
what we currently support or not.
As an example usage:
Will spit out an RST formatted table that can be copy+pasted into our docs.
The config is not filled out entirely, but doing so and then updating our docs
would be great beginner bugs for new contributors.
Having python+yaml generate things like docs, or headers (as imagined in
https://github.com/nickdesaulniers/llvm-project/tree/hdr-gen2) is perhaps
easier to work with than tablegen, and doesn't introduce a dependency on a host
tool that needs to be compiled from llvm sources before building the rest of
the libc. This can probably be merged with whatever we end up doing to replace
libc-hdrgen.
Please use
https://llvm.org/docs/CodingStandards.html#python-version-and-source-code-formatting
for keeping this file formatted.