From 3113237481b70188900004c20059932e096f8e97 Mon Sep 17 00:00:00 2001 From: HereThereBeDragons Date: Fri, 12 Jan 2024 18:34:18 +0100 Subject: [PATCH 1/3] change xattr to table and add documentation about multipage attr, add example to restricting xattr access --- cpt-details.rst | 195 +++++++++++++++++++----------------------------- 1 file changed, 78 insertions(+), 117 deletions(-) diff --git a/cpt-details.rst b/cpt-details.rst index 308d931..e2f069c 100644 --- a/cpt-details.rst +++ b/cpt-details.rst @@ -661,134 +661,88 @@ information. In general, they can be displayed with a command like There are the following supported magic attributes: -**catalog\_counters** - Like ``repo_counters`` but only for the nested catalog that hosts the given path. +======================= ======================================================================= +**Parameter** **Meaning** +======================= ======================================================================= +``catalog_counters`` Like ``repo_counters`` but only for the nested catalog that hosts the given path. +``chunks`` Number of chunks of a regular file. +``chunk_list`` Hashes and sizes of the chunks of a regular (large) file. +``compression`` Compression algorithm, for regular files only. Either "zlib" or "none". +``expires`` Shows the remaining lifetime of the mounted root file catalog in minutes. +``external_file`` Indicates if a regular file is an external file or not. Either 0 or 1. +``external_host`` Like ``host`` but for the host settings to fetch external files. +``external_timeout`` Like ``timeout`` but for the host settings to fetch external files. +``fqrn`` Shows the fully qualified repository name of the mounted repository. +``hash`` Shows the cryptographic hash of a regular file as listed in the file catalog. +``host`` Shows the currently active HTTP server. +``host_list`` Shows the ordered list of HTTP servers. +``inode_max`` Shows the highest possible inode with the current set of loaded catalogs. +``lhash`` Shows the cryptographic hash of a regular file as stored in the local cache, if available. +``logbuffer`` Shows system log messages for the repository. +``maxfd`` Shows the maximum number of file descriptors available to file system clients. +``ncleanup24`` Shows the number of cache cleanups in the last 24 hours. +``nclg`` Shows the number of currently loaded nested catalogs. +``ndiropen`` Shows the overall number of opened directories. +``ndownload`` Shows the overall number of downloaded files since mounting. +``nioerr`` Shows the total number of I/O errors encountered since mounting. +``nopen`` Shows the overall number of ``open()`` calls since mounting. +``pid`` Shows the process ID of the CernVM-FS Fuse process. +``proxy`` Shows the currently active HTTP proxy. +``pubkeys`` The loaded public RSA keys used for repository whitelist verification. +``rawlink`` Shows unresolved variant symbolic links; only accessible from the root attribute namespace (use `attr -Rg rawlink`). +``repo_counters`` Shows the aggregate counters of the repository contents (number of files etc.) +``repo_metainfo`` Shows the :ref:`repository meta info ` file, if available +``revision`` Shows the file catalog revision of the mounted root catalog, an auto-increment counter increased on every repository publish. +``root_hash`` Shows the cryptographic hash of the root file catalog. +``rx`` Shows the overall amount of downloaded kilobytes. +``speed`` Shows the average download speed. +``tag`` The configured repository tag. +``timeout`` Shows the timeout for proxied connections in seconds. +``timeout_direct`` Shows the timeout for direct connections in seconds. +``uptime`` Shows the time passed since mounting in minutes. +``usedfd`` Shows the number of file descriptors currently issued to file system clients. +``version`` Shows the version of the loaded CernVM-FS binary. +======================= ======================================================================= -**chunks** - Number of chunks of a regular file. - -**chunk\_list** - Hashes and sizes of the chunks of a regular (large) file. - -**compression** - Compression algorithm, for regular files only. Either "zlib" or "none". - -**expires** - Shows the remaining lifetime of the mounted root file catalog in - minutes. - -**external\_file** - Indicates if a regular file is an external file or not. Either 0 or 1. - -**external\_host** - Like ``host`` but for the host settings to fetch external files. - -**external\_timeout** - Like ``timeout`` but for the host settings to fetch external files. - -**fqrn** - Shows the fully qualified repository name of the mounted repository. - -**hash** - Shows the cryptographic hash of a regular file as listed in the file - catalog. - -**host** - Shows the currently active HTTP server. - -**host\_list** - Shows the ordered list of HTTP servers. - -**inode\_max** - Shows the highest possible inode with the current set of loaded - catalogs. - -**lhash** - Shows the cryptographic hash of a regular file as stored in the - local cache, if available. - -**logbuffer** - Shows system log messages for the repository. - -**maxfd** - Shows the maximum number of file descriptors available to file - system clients. - -**ncleanup24** - Shows the number of cache cleanups in the last 24 hours. - -**nclg** - Shows the number of currently loaded nested catalogs. - -**ndiropen** - Shows the overall number of opened directories. - -**ndownload** - Shows the overall number of downloaded files since mounting. - -**nioerr** - Shows the total number of I/O errors encountered since mounting. - -**nopen** - Shows the overall number of ``open()`` calls since mounting. - -**pid** - Shows the process ID of the CernVM-FS Fuse process. - -**proxy** - Shows the currently active HTTP proxy. - -**pubkeys** - The loaded public RSA keys used for repository whitelist verification. - -**rawlink** - Shows unresolved variant symbolic links; only accessible from the - root attribute namespace (use `attr -Rg rawlink`). - -**repo\_counters** - Shows the aggregate counters of the repository contents (number of files - etc.) - -**repo\_metainfo** - Shows the :ref:`repository meta info ` file, if available - -**revision** - Shows the file catalog revision of the mounted root catalog, an - auto-increment counter increased on every repository publish. +Extended attributes can be queried using the ``attr`` command. For +instance, ``attr -g hash /cvmfs/atlas.cern.ch/ChangeLog`` returns the +cryptographic hash of the file at hand. The extended attributes are used +by the ``cvmfs_config stat`` command in order to show a current overview +of health and performance numbers. -**root\_hash** - Shows the cryptographic hash of the root file catalog. +Multipage Extended Attributes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -**rx** - Shows the overall amount of downloaded kilobytes. +Some extended attributes can be too large to be presented in a single request. +For this additional commands and output modes are available. -**speed** - Shows the average download speed. +The output mode can either be machine-readable (``~``) or human-readable (``@``). The +machine-readable output is designed to return output that can be easily parsed by a +machine and errors are returned as signals. The human-readable output includes a more +descriptive header, including how many pages are available and instructions how to access +them. Errors are returned as plaintext with possible instructions how to resolve the issue. -**tag** - The configured repository tag. +Furthermore, ``~?`` and ``@?`` allow retrieving information about the attribute. +At the moment, this consist only to get the number of pages the attribute has. -**timeout** - Shows the timeout for proxied connections in seconds. +Different pages of the attribute can be accessed with ``~`` and ``@`` +Pages start at 0. -**timeout\_direct** - Shows the timeout for direct connections in seconds. +The commands also work with single page attributes (page number is 0). -**uptime** - Shows the time passed since mounting in minutes. +======================= ======================================================================= +**Parameter** **Meaning** +======================= ======================================================================= +``@?`` Human-readable information about the attribute +``~?`` Machine-readable (CSV format) information about the attribute +``@`` Output of the attribute with descritive header. Page numbers starting from 0. Errors are returned as plaintext. +``~`` Output of the attribute. Page numbers starting from 0. Errors are returned as signals. +======================= ======================================================================= -**usedfd** - Shows the number of file descriptors currently issued to file system - clients. -**version** - Shows the version of the loaded CernVM-FS binary. -Extended attributes can be queried using the ``attr`` command. For -instance, ``attr -g hash /cvmfs/atlas.cern.ch/ChangeLog`` returns the -cryptographic hash of the file at hand. The extended attributes are used -by the ``cvmfs_config stat`` command in order to show a current overview -of health and performance numbers. +Restricting Access to Extended Attributes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Access to extended attributes can be restricted in the client config to ``root`` and users with a specific (main) ``gid`` listed by @@ -799,6 +753,13 @@ Note that those attributes must be listed in their full name, e.g. ``user.fqrn`` will have the prefix ``user.``. If uncertain, they can be looked up in the source code of ``cvmfs/magic_xattr.cc``. +Example that only users with ``gid=788`` (and ``root``) can access the repository name + +:: + + CVMFS_XATTR_PRIVILEGED_GIDS=788 + CVMFS_XATTR_PROTECTED_XATTRS=user.fqrn + Repository Publishing --------------------- From b774e79972ea433b5abb170e984d4bd8f23c2d97 Mon Sep 17 00:00:00 2001 From: HereThereBeDragons Date: Fri, 26 Jan 2024 16:24:57 +0100 Subject: [PATCH 2/3] add missing xattrs in doc --- cpt-details.rst | 90 ++++++++++++++++++++++++++----------------------- 1 file changed, 48 insertions(+), 42 deletions(-) diff --git a/cpt-details.rst b/cpt-details.rst index e2f069c..722c757 100644 --- a/cpt-details.rst +++ b/cpt-details.rst @@ -661,48 +661,54 @@ information. In general, they can be displayed with a command like There are the following supported magic attributes: -======================= ======================================================================= -**Parameter** **Meaning** -======================= ======================================================================= -``catalog_counters`` Like ``repo_counters`` but only for the nested catalog that hosts the given path. -``chunks`` Number of chunks of a regular file. -``chunk_list`` Hashes and sizes of the chunks of a regular (large) file. -``compression`` Compression algorithm, for regular files only. Either "zlib" or "none". -``expires`` Shows the remaining lifetime of the mounted root file catalog in minutes. -``external_file`` Indicates if a regular file is an external file or not. Either 0 or 1. -``external_host`` Like ``host`` but for the host settings to fetch external files. -``external_timeout`` Like ``timeout`` but for the host settings to fetch external files. -``fqrn`` Shows the fully qualified repository name of the mounted repository. -``hash`` Shows the cryptographic hash of a regular file as listed in the file catalog. -``host`` Shows the currently active HTTP server. -``host_list`` Shows the ordered list of HTTP servers. -``inode_max`` Shows the highest possible inode with the current set of loaded catalogs. -``lhash`` Shows the cryptographic hash of a regular file as stored in the local cache, if available. -``logbuffer`` Shows system log messages for the repository. -``maxfd`` Shows the maximum number of file descriptors available to file system clients. -``ncleanup24`` Shows the number of cache cleanups in the last 24 hours. -``nclg`` Shows the number of currently loaded nested catalogs. -``ndiropen`` Shows the overall number of opened directories. -``ndownload`` Shows the overall number of downloaded files since mounting. -``nioerr`` Shows the total number of I/O errors encountered since mounting. -``nopen`` Shows the overall number of ``open()`` calls since mounting. -``pid`` Shows the process ID of the CernVM-FS Fuse process. -``proxy`` Shows the currently active HTTP proxy. -``pubkeys`` The loaded public RSA keys used for repository whitelist verification. -``rawlink`` Shows unresolved variant symbolic links; only accessible from the root attribute namespace (use `attr -Rg rawlink`). -``repo_counters`` Shows the aggregate counters of the repository contents (number of files etc.) -``repo_metainfo`` Shows the :ref:`repository meta info ` file, if available -``revision`` Shows the file catalog revision of the mounted root catalog, an auto-increment counter increased on every repository publish. -``root_hash`` Shows the cryptographic hash of the root file catalog. -``rx`` Shows the overall amount of downloaded kilobytes. -``speed`` Shows the average download speed. -``tag`` The configured repository tag. -``timeout`` Shows the timeout for proxied connections in seconds. -``timeout_direct`` Shows the timeout for direct connections in seconds. -``uptime`` Shows the time passed since mounting in minutes. -``usedfd`` Shows the number of file descriptors currently issued to file system clients. -``version`` Shows the version of the loaded CernVM-FS binary. -======================= ======================================================================= +======================== ======================================================================= +**Parameter** **Meaning** +======================== ======================================================================= +``catalog_counters`` Like ``repo_counters`` but only for the nested catalog that hosts the given path. +``chunks`` Number of chunks of a regular file. +``chunk_list`` Hashes and sizes of the chunks of a regular (large) file. +``compression`` Compression algorithm, for regular files only. Either "zlib" or "none". +``direct_io`` Indicates if the current entry is using direct IO. Either 0 or 1. +``expires`` Shows the remaining lifetime of the mounted root file catalog in minutes. +``external_file`` Indicates if a regular file is an external file or not. Either 0 or 1. +``external_host`` Like ``host`` but for the host settings to fetch external files. +``external_timeout`` Like ``timeout`` but for the host settings to fetch external files. +``fqrn`` Shows the fully qualified repository name of the mounted repository. +``hash`` Shows the cryptographic hash of a regular file as listed in the file catalog. +``hitrate`` Shows overall cache hitrate since mounting the repository. +``host`` Shows the currently active HTTP server. +``host_list`` Shows the ordered list of HTTP servers. +``inode_max`` Shows the highest possible inode with the current set of loaded catalogs. +``lhash`` Shows the cryptographic hash of a regular file as stored in the local cache, if available. +``logbuffer`` Shows system log messages for the repository. +``maxfd`` Shows the maximum number of file descriptors available to file system clients. +``ncleanup24`` Shows the number of cache cleanups in the last 24 hours. +``nclg`` Shows the number of currently loaded nested catalogs. +``ndiropen`` Shows the overall number of opened directories. +``ndownload`` Shows the overall number of downloaded files since mounting. +``nioerr`` Shows the total number of I/O errors encountered since mounting. +``nopen`` Shows the overall number of ``open()`` calls since mounting. +``pid`` Shows the process ID of the CernVM-FS Fuse process. +``proxy`` Shows the currently active HTTP proxy. +``proxy_list`` Shows all registered proxies for this repository. Also contains fallback proxies. If none are used it shows ``DIRECT``. +``proxy_list_external`` Shows all registered proxies used for accessing external data. If none are used it shows ``DIRECT``. +``pubkeys`` The loaded public RSA keys used for repository whitelist verification. +``rawlink`` Shows unresolved variant symbolic links; only accessible from the root attribute namespace (use `attr -Rg rawlink`). +``repo_counters`` Shows the aggregate counters of the repository contents (number of files etc.) +``repo_metainfo`` Shows the :ref:`repository meta info ` file, if available +``revision`` Shows the file catalog revision of the mounted root catalog, an auto-increment counter increased on every repository publish. +``root_hash`` Shows the cryptographic hash of the root file catalog. +``rx`` Shows the overall amount of downloaded kilobytes. +``speed`` Shows the average download speed. +``tag`` The configured repository tag. +``timeout`` Shows the timeout for proxied connections in seconds. +``timestamp_last_ioerr`` Shows the timestamp when the last IO error occured. +``timeout_direct`` Shows the timeout for direct connections in seconds. +``uptime`` Shows the time passed since mounting in minutes. +``useddirp`` Shows the number of currently open directories. +``usedfd`` Shows the number of file descriptors currently issued to file system clients. +``version`` Shows the version of the loaded CernVM-FS binary. +======================== ======================================================================= Extended attributes can be queried using the ``attr`` command. For instance, ``attr -g hash /cvmfs/atlas.cern.ch/ChangeLog`` returns the From faadeb41d7fa1525d3c5dd70881ec460967cc11c Mon Sep 17 00:00:00 2001 From: HereThereBeDragons Date: Mon, 29 Jan 2024 09:34:11 +0100 Subject: [PATCH 3/3] small fixes of typos and better readability --- cpt-details.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/cpt-details.rst b/cpt-details.rst index 722c757..f94f4e8 100644 --- a/cpt-details.rst +++ b/cpt-details.rst @@ -728,8 +728,8 @@ machine and errors are returned as signals. The human-readable output includes a descriptive header, including how many pages are available and instructions how to access them. Errors are returned as plaintext with possible instructions how to resolve the issue. -Furthermore, ``~?`` and ``@?`` allow retrieving information about the attribute. -At the moment, this consist only to get the number of pages the attribute has. +Furthermore, ``~?`` and ``@?`` allow retrieving additional information about the attribute. +At the moment, this consist only of the number of pages the attribute has. Different pages of the attribute can be accessed with ``~`` and ``@`` Pages start at 0. @@ -739,10 +739,10 @@ The commands also work with single page attributes (page number is 0). ======================= ======================================================================= **Parameter** **Meaning** ======================= ======================================================================= -``@?`` Human-readable information about the attribute -``~?`` Machine-readable (CSV format) information about the attribute -``@`` Output of the attribute with descritive header. Page numbers starting from 0. Errors are returned as plaintext. -``~`` Output of the attribute. Page numbers starting from 0. Errors are returned as signals. +``@?`` Human-readable information about the attribute. +``~?`` Machine-readable (CSV format) information about the attribute. +``@`` Output of the attribute with a descriptive header. Page numbers are starting from 0. Errors are returned as plaintext. +``~`` Output of the attribute. Page numbers are starting from 0. Errors are returned as signals. ======================= ======================================================================= @@ -751,7 +751,7 @@ Restricting Access to Extended Attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Access to extended attributes can be restricted in the client config to -``root`` and users with a specific (main) ``gid`` listed by +``root`` (``gid=0``) and users with a specific (main) ``gid`` listed by ``CVMFS_XATTR_PRIVILEGED_GIDS``. Extended attributes to which this should apply are listed in ``CVMFS_XATTR_PROTECTED_XATTRS``. Note that those attributes must be listed in their full name, e.g. ``user.fqrn``, @@ -759,7 +759,7 @@ Note that those attributes must be listed in their full name, e.g. ``user.fqrn`` will have the prefix ``user.``. If uncertain, they can be looked up in the source code of ``cvmfs/magic_xattr.cc``. -Example that only users with ``gid=788`` (and ``root``) can access the repository name +Example: Only users with ``gid=788`` (and ``root``) can access the repository name ::