Skip to content

Commit

Permalink
Merge branch 'docs-next' of git://git.lwn.net/linux
Browse files Browse the repository at this point in the history
Pull documentation updates from Jonathan Corbet:
 "After a fair amount of churn in the last couple of cycles, docs are
  taking it easier this time around. Lots of fixes and some new
  documentation, but nothing all that radical. Perhaps the most
  interesting change for many is the scripts/sphinx-pre-install tool
  from Mauro; it will tell you exactly which packages you need to
  install to get a working docs toolchain on your system.

  There are two little patches reaching outside of Documentation/; both
  just tweak kerneldoc comments to eliminate warnings and fix some
  dangling doc pointers"

* 'docs-next' of git://git.lwn.net/linux: (52 commits)
  Documentation/sphinx: fix kernel-doc decode for non-utf-8 locale
  genalloc: Fix an incorrect kerneldoc comment
  doc: Add documentation for the genalloc subsystem
  assoc_array: fix path to assoc_array documentation
  kernel-doc parser mishandles declarations split into lines
  docs: ReSTify table of contents in core.rst
  docs: process: drop git snapshots from applying-patches.rst
  Documentation:input: fix typo
  swap: Remove obsolete sentence
  sphinx.rst: Allow Sphinx version 1.6 at the docs
  docs-rst: fix verbatim font size on tables
  Documentation: stable-kernel-rules: fix broken git urls
  rtmutex: update rt-mutex
  rtmutex: update rt-mutex-design
  docs: fix minimal sphinx version in conf.py
  docs: fix nested numbering in the TOC
  NVMEM documentation fix: A minor typo
  docs-rst: pdf: use same vertical margin on all Sphinx versions
  doc: Makefile: if sphinx is not found, run a check script
  docs: Fix paths in security/keys
  ...
  • Loading branch information
torvalds committed Sep 4, 2017
2 parents fe91f28 + 86c0f04 commit 81a84ad
Show file tree
Hide file tree
Showing 39 changed files with 1,072 additions and 497 deletions.
12 changes: 2 additions & 10 deletions Documentation/Makefile
Expand Up @@ -22,6 +22,8 @@ ifeq ($(HAVE_SPHINX),0)

.DEFAULT:
$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
@echo
@./scripts/sphinx-pre-install
@echo " SKIP Sphinx $@ target."

else # HAVE_SPHINX
Expand Down Expand Up @@ -95,16 +97,6 @@ endif # HAVE_SPHINX
# The following targets are independent of HAVE_SPHINX, and the rules should
# work or silently pass without Sphinx.

# no-ops for the Sphinx toolchain
sgmldocs:
@:
psdocs:
@:
mandocs:
@:
installmandocs:
@:

cleandocs:
$(Q)rm -rf $(BUILDDIR)
$(Q)$(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/media clean
Expand Down
2 changes: 1 addition & 1 deletion Documentation/arm/firmware.txt
Expand Up @@ -60,7 +60,7 @@ Example of using a firmware operation:

/* some platform code, e.g. SMP initialization */

__raw_writel(virt_to_phys(exynos4_secondary_startup),
__raw_writel(__pa_symbol(exynos4_secondary_startup),
CPU1_BOOT_REG);

/* Call Exynos specific smc call */
Expand Down
6 changes: 3 additions & 3 deletions Documentation/conf.py
Expand Up @@ -29,7 +29,7 @@
# -- General configuration ------------------------------------------------

# If your documentation needs a minimal Sphinx version, state it here.
needs_sphinx = '1.2'
needs_sphinx = '1.3'

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
Expand Down Expand Up @@ -344,8 +344,8 @@
if major == 1 and minor <= 4:
latex_elements['preamble'] += '\\usepackage[margin=0.5in, top=1in, bottom=1in]{geometry}'
elif major == 1 and (minor > 5 or (minor == 5 and patch >= 3)):
latex_elements['sphinxsetup'] = 'hmargin=0.5in, vmargin=0.5in'

latex_elements['sphinxsetup'] = 'hmargin=0.5in, vmargin=1in'
latex_elements['preamble'] += '\\fvset{fontsize=auto}\n'

# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
Expand Down
144 changes: 144 additions & 0 deletions Documentation/core-api/genalloc.rst
@@ -0,0 +1,144 @@
The genalloc/genpool subsystem
==============================

There are a number of memory-allocation subsystems in the kernel, each
aimed at a specific need. Sometimes, however, a kernel developer needs to
implement a new allocator for a specific range of special-purpose memory;
often that memory is located on a device somewhere. The author of the
driver for that device can certainly write a little allocator to get the
job done, but that is the way to fill the kernel with dozens of poorly
tested allocators. Back in 2005, Jes Sorensen lifted one of those
allocators from the sym53c8xx_2 driver and posted_ it as a generic module
for the creation of ad hoc memory allocators. This code was merged
for the 2.6.13 release; it has been modified considerably since then.

.. _posted: https://lwn.net/Articles/125842/

Code using this allocator should include <linux/genalloc.h>. The action
begins with the creation of a pool using one of:

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_create

.. kernel-doc:: lib/genalloc.c
:functions: devm_gen_pool_create

A call to :c:func:`gen_pool_create` will create a pool. The granularity of
allocations is set with min_alloc_order; it is a log-base-2 number like
those used by the page allocator, but it refers to bytes rather than pages.
So, if min_alloc_order is passed as 3, then all allocations will be a
multiple of eight bytes. Increasing min_alloc_order decreases the memory
required to track the memory in the pool. The nid parameter specifies
which NUMA node should be used for the allocation of the housekeeping
structures; it can be -1 if the caller doesn't care.

The "managed" interface :c:func:`devm_gen_pool_create` ties the pool to a
specific device. Among other things, it will automatically clean up the
pool when the given device is destroyed.

A pool is shut down with:

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_destroy

It's worth noting that, if there are still allocations outstanding from the
given pool, this function will take the rather extreme step of invoking
BUG(), crashing the entire system. You have been warned.

A freshly created pool has no memory to allocate. It is fairly useless in
that state, so one of the first orders of business is usually to add memory
to the pool. That can be done with one of:

.. kernel-doc:: include/linux/genalloc.h
:functions: gen_pool_add

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_add_virt

A call to :c:func:`gen_pool_add` will place the size bytes of memory
starting at addr (in the kernel's virtual address space) into the given
pool, once again using nid as the node ID for ancillary memory allocations.
The :c:func:`gen_pool_add_virt` variant associates an explicit physical
address with the memory; this is only necessary if the pool will be used
for DMA allocations.

The functions for allocating memory from the pool (and putting it back)
are:

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_alloc

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_dma_alloc

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_free

As one would expect, :c:func:`gen_pool_alloc` will allocate size< bytes
from the given pool. The :c:func:`gen_pool_dma_alloc` variant allocates
memory for use with DMA operations, returning the associated physical
address in the space pointed to by dma. This will only work if the memory
was added with :c:func:`gen_pool_add_virt`. Note that this function
departs from the usual genpool pattern of using unsigned long values to
represent kernel addresses; it returns a void * instead.

That all seems relatively simple; indeed, some developers clearly found it
to be too simple. After all, the interface above provides no control over
how the allocation functions choose which specific piece of memory to
return. If that sort of control is needed, the following functions will be
of interest:

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_alloc_algo

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_set_algo

Allocations with :c:func:`gen_pool_alloc_algo` specify an algorithm to be
used to choose the memory to be allocated; the default algorithm can be set
with :c:func:`gen_pool_set_algo`. The data value is passed to the
algorithm; most ignore it, but it is occasionally needed. One can,
naturally, write a special-purpose algorithm, but there is a fair set
already available:

- gen_pool_first_fit is a simple first-fit allocator; this is the default
algorithm if none other has been specified.

- gen_pool_first_fit_align forces the allocation to have a specific
alignment (passed via data in a genpool_data_align structure).

- gen_pool_first_fit_order_align aligns the allocation to the order of the
size. A 60-byte allocation will thus be 64-byte aligned, for example.

- gen_pool_best_fit, as one would expect, is a simple best-fit allocator.

- gen_pool_fixed_alloc allocates at a specific offset (passed in a
genpool_data_fixed structure via the data parameter) within the pool.
If the indicated memory is not available the allocation fails.

There is a handful of other functions, mostly for purposes like querying
the space available in the pool or iterating through chunks of memory.
Most users, however, should not need much beyond what has been described
above. With luck, wider awareness of this module will help to prevent the
writing of special-purpose memory allocators in the future.

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_virt_to_phys

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_for_each_chunk

.. kernel-doc:: lib/genalloc.c
:functions: addr_in_gen_pool

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_avail

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_size

.. kernel-doc:: lib/genalloc.c
:functions: gen_pool_get

.. kernel-doc:: lib/genalloc.c
:functions: of_gen_pool_get
1 change: 1 addition & 0 deletions Documentation/core-api/index.rst
Expand Up @@ -20,6 +20,7 @@ Core utilities
genericirq
flexible-arrays
librs
genalloc

Interfaces for kernel debugging
===============================
Expand Down
6 changes: 4 additions & 2 deletions Documentation/dev-tools/gdb-kernel-debugging.rst
Expand Up @@ -31,11 +31,13 @@ Setup
CONFIG_DEBUG_INFO_REDUCED off. If your architecture supports
CONFIG_FRAME_POINTER, keep it enabled.

- Install that kernel on the guest.
- Install that kernel on the guest, turn off KASLR if necessary by adding
"nokaslr" to the kernel command line.
Alternatively, QEMU allows to boot the kernel directly using -kernel,
-append, -initrd command line switches. This is generally only useful if
you do not depend on modules. See QEMU documentation for more details on
this mode.
this mode. In this case, you should build the kernel with
CONFIG_RANDOMIZE_BASE disabled if the architecture supports KASLR.

- Enable the gdb stub of QEMU/KVM, either

Expand Down
11 changes: 10 additions & 1 deletion Documentation/dev-tools/kgdb.rst
Expand Up @@ -348,6 +348,15 @@ default behavior is always set to 0.
- ``echo 1 > /sys/module/debug_core/parameters/kgdbreboot``
- Enter the debugger on reboot notify.

Kernel parameter: ``nokaslr``
-----------------------------

If the architecture that you are using enable KASLR by default,
you should consider turning it off. KASLR randomizes the
virtual address where the kernel image is mapped and confuse
gdb which resolve kernel symbol address from symbol table
of vmlinux.

Using kdb
=========

Expand All @@ -358,7 +367,7 @@ This is a quick example of how to use kdb.

1. Configure kgdboc at boot using kernel parameters::

console=ttyS0,115200 kgdboc=ttyS0,115200
console=ttyS0,115200 kgdboc=ttyS0,115200 nokaslr

OR

Expand Down
107 changes: 106 additions & 1 deletion Documentation/doc-guide/sphinx.rst
Expand Up @@ -19,6 +19,110 @@ Finally, there are thousands of plain text documentation files scattered around
``Documentation``. Some of these will likely be converted to reStructuredText
over time, but the bulk of them will remain in plain text.

.. _sphinx_install:

Sphinx Install
==============

The ReST markups currently used by the Documentation/ files are meant to be
built with ``Sphinx`` version 1.3 or upper. If you're desiring to build
PDF outputs, it is recommended to use version 1.4.6 or upper.

There's a script that checks for the Spinx requirements. Please see
:ref:`sphinx-pre-install` for further details.

Most distributions are shipped with Sphinx, but its toolchain is fragile,
and it is not uncommon that upgrading it or some other Python packages
on your machine would cause the documentation build to break.

A way to get rid of that is to use a different version than the one shipped
on your distributions. In order to do that, it is recommended to install
Sphinx inside a virtual environment, using ``virtualenv-3``
or ``virtualenv``, depending on how your distribution packaged Python 3.

.. note::

#) Sphinx versions below 1.5 don't work properly with Python's
docutils version 0.13.1 or upper. So, if you're willing to use
those versions, you should run ``pip install 'docutils==0.12'``.

#) It is recommended to use the RTD theme for html output. Depending
on the Sphinx version, it should be installed in separate,
with ``pip install sphinx_rtd_theme``.

#) Some ReST pages contain math expressions. Due to the way Sphinx work,
those expressions are written using LaTeX notation. It needs texlive
installed with amdfonts and amsmath in order to evaluate them.

In summary, if you want to install Sphinx version 1.4.9, you should do::

$ virtualenv sphinx_1.4
$ . sphinx_1.4/bin/activate
(sphinx_1.4) $ pip install -r Documentation/sphinx/requirements.txt

After running ``. sphinx_1.4/bin/activate``, the prompt will change,
in order to indicate that you're using the new environment. If you
open a new shell, you need to rerun this command to enter again at
the virtual environment before building the documentation.

Image output
------------

The kernel documentation build system contains an extension that
handles images on both GraphViz and SVG formats (see
:ref:`sphinx_kfigure`).

For it to work, you need to install both GraphViz and ImageMagick
packages. If those packages are not installed, the build system will
still build the documentation, but won't include any images at the
output.

PDF and LaTeX builds
--------------------

Such builds are currently supported only with Sphinx versions 1.4 and upper.

For PDF and LaTeX output, you'll also need ``XeLaTeX`` version 3.14159265.

Depending on the distribution, you may also need to install a series of
``texlive`` packages that provide the minimal set of functionalities
required for ``XeLaTeX`` to work.

.. _sphinx-pre-install:

Checking for Sphinx dependencies
--------------------------------

There's a script that automatically check for Sphinx dependencies. If it can
recognize your distribution, it will also give a hint about the install
command line options for your distro::

$ ./scripts/sphinx-pre-install
Checking if the needed tools for Fedora release 26 (Twenty Six) are available
Warning: better to also install "texlive-luatex85".
You should run:

sudo dnf install -y texlive-luatex85
/usr/bin/virtualenv sphinx_1.4
. sphinx_1.4/bin/activate
pip install -r Documentation/sphinx/requirements.txt

Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.

By default, it checks all the requirements for both html and PDF, including
the requirements for images, math expressions and LaTeX build, and assumes
that a virtual Python environment will be used. The ones needed for html
builds are assumed to be mandatory; the others to be optional.

It supports two optional parameters:

``--no-pdf``
Disable checks for PDF;

``--no-virtualenv``
Use OS packaging for Sphinx instead of Python virtual environment.


Sphinx Build
============

Expand Down Expand Up @@ -118,7 +222,7 @@ Here are some specific guidelines for the kernel documentation:
the C domain
------------

The `Sphinx C Domain`_ (name c) is suited for documentation of C API. E.g. a
The **Sphinx C Domain** (name c) is suited for documentation of C API. E.g. a
function prototype:

.. code-block:: rst
Expand Down Expand Up @@ -229,6 +333,7 @@ Rendered as:

- column 3

.. _sphinx_kfigure:

Figures & Images
================
Expand Down
5 changes: 1 addition & 4 deletions Documentation/driver-api/basics.rst
Expand Up @@ -4,7 +4,7 @@ Driver Basics
Driver Entry and Exit points
----------------------------

.. kernel-doc:: include/linux/init.h
.. kernel-doc:: include/linux/module.h
:internal:

Driver device table
Expand Down Expand Up @@ -103,9 +103,6 @@ Kernel utility functions
.. kernel-doc:: kernel/panic.c
:export:

.. kernel-doc:: kernel/sys.c
:export:

.. kernel-doc:: kernel/rcu/tree.c
:export:

Expand Down

0 comments on commit 81a84ad

Please sign in to comment.