Skip to content

Commit

Permalink
Merge pull request #36527 from neha-ojha/wip-whitelist-blacklist
Browse files Browse the repository at this point in the history
Replace 'whitelist' and 'blacklist' language

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
  • Loading branch information
neha-ojha committed Aug 25, 2020
2 parents f144e5e + d7dfb8e commit fb44c2f
Show file tree
Hide file tree
Showing 459 changed files with 1,496 additions and 1,343 deletions.
25 changes: 25 additions & 0 deletions PendingReleaseNotes
Original file line number Diff line number Diff line change
Expand Up @@ -110,3 +110,28 @@
* fs: Names of new FSs, volumes, subvolumes and subvolume groups can only
contain alphanumeric and ``-``, ``_`` and ``.`` characters. Some commands
or CephX credentials may not work with old FSs with non-conformant names.

* `blacklist` has been replaced with `blocklist` throughout. The following commands have changed:

- ``ceph osd blacklist ...`` are now ``ceph osd blocklist ...``
- ``ceph <tell|daemon> osd.<NNN> dump_blacklist`` is now ``ceph <tell|daemon> osd.<NNN> dump_blocklist``

* The following config options have changed:

- ``mon osd blacklist default expire`` is now ``mon osd blocklist default expire``
- ``mon mds blacklist interval`` is now ``mon mds blocklist interval``
- ``mon mgr blacklist interval`` is now ''mon mgr blocklist interval``
- ``rbd blacklist on break lock`` is now ``rbd blocklist on break lock``
- ``rbd blacklist expire seconds`` is now ``rbd blocklist expire seconds``
- ``mds session blacklist on timeout`` is now ``mds session blocklist on timeout``
- ``mds session blacklist on evict`` is now ``mds session blocklist on evict``

* The following librados API calls have changed:

- ``rados_blacklist_add`` is now ``rados_blocklist_add``; the former will issue a deprecation warning and be removed in a future release.
- ``rados.blacklist_add`` is now ``rados.blocklist_add`` in the C++ API.

* The JSON output for the following commands now shows ``blocklist`` instead of ``blacklist``:

- ``ceph osd dump``
- ``ceph <tell|daemon> osd.<N> dump_blocklist``
44 changes: 22 additions & 22 deletions doc/cephfs/eviction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,30 +89,30 @@ do that using its unique ID, or various other attributes to identify it:
ceph tell mds.0 client evict client_metadata.=4305


Advanced: Un-blacklisting a client
Advanced: Un-blocklisting a client
==================================

Ordinarily, a blacklisted client may not reconnect to the servers: it
Ordinarily, a blocklisted client may not reconnect to the servers: it
must be unmounted and then mounted anew.

However, in some situations it may be useful to permit a client that
was evicted to attempt to reconnect.

Because CephFS uses the RADOS OSD blacklist to control client eviction,
Because CephFS uses the RADOS OSD blocklist to control client eviction,
CephFS clients can be permitted to reconnect by removing them from
the blacklist:
the blocklist:

::

$ ceph osd blacklist ls
$ ceph osd blocklist ls
listed 1 entries
127.0.0.1:0/3710147553 2018-03-19 11:32:24.716146
$ ceph osd blacklist rm 127.0.0.1:0/3710147553
un-blacklisting 127.0.0.1:0/3710147553
$ ceph osd blocklist rm 127.0.0.1:0/3710147553
un-blocklisting 127.0.0.1:0/3710147553


Doing this may put data integrity at risk if other clients have accessed
files that the blacklisted client was doing buffered IO to. It is also not
files that the blocklisted client was doing buffered IO to. It is also not
guaranteed to result in a fully functional client -- the best way to get
a fully healthy client back after an eviction is to unmount the client
and do a fresh mount.
Expand All @@ -121,7 +121,7 @@ If you are trying to reconnect clients in this way, you may also
find it useful to set ``client_reconnect_stale`` to true in the
FUSE client, to prompt the client to try to reconnect.

Advanced: Configuring blacklisting
Advanced: Configuring blocklisting
==================================

If you are experiencing frequent client evictions, due to slow
Expand All @@ -131,40 +131,40 @@ issue, then you may want to ask the MDS to be less strict.
It is possible to respond to slow clients by simply dropping their
MDS sessions, but permit them to re-open sessions and permit them
to continue talking to OSDs. To enable this mode, set
``mds_session_blacklist_on_timeout`` to false on your MDS nodes.
``mds_session_blocklist_on_timeout`` to false on your MDS nodes.

For the equivalent behaviour on manual evictions, set
``mds_session_blacklist_on_evict`` to false.
``mds_session_blocklist_on_evict`` to false.

Note that if blacklisting is disabled, then evicting a client will
Note that if blocklisting is disabled, then evicting a client will
only have an effect on the MDS you send the command to. On a system
with multiple active MDS daemons, you would need to send an
eviction command to each active daemon. When blacklisting is enabled
eviction command to each active daemon. When blocklisting is enabled
(the default), sending an eviction command to just a single
MDS is sufficient, because the blacklist propagates it to the others.
MDS is sufficient, because the blocklist propagates it to the others.

.. _background_blacklisting_and_osd_epoch_barrier:
.. _background_blocklisting_and_osd_epoch_barrier:

Background: Blacklisting and OSD epoch barrier
Background: Blocklisting and OSD epoch barrier
==============================================

After a client is blacklisted, it is necessary to make sure that
After a client is blocklisted, it is necessary to make sure that
other clients and MDS daemons have the latest OSDMap (including
the blacklist entry) before they try to access any data objects
that the blacklisted client might have been accessing.
the blocklist entry) before they try to access any data objects
that the blocklisted client might have been accessing.

This is ensured using an internal "osdmap epoch barrier" mechanism.

The purpose of the barrier is to ensure that when we hand out any
capabilities which might allow touching the same RADOS objects, the
clients we hand out the capabilities to must have a sufficiently recent
OSD map to not race with cancelled operations (from ENOSPC) or
blacklisted clients (from evictions).
blocklisted clients (from evictions).

More specifically, the cases where an epoch barrier is set are:

* Client eviction (where the client is blacklisted and other clients
must wait for a post-blacklist epoch to touch the same objects).
* Client eviction (where the client is blocklisted and other clients
must wait for a post-blocklist epoch to touch the same objects).
* OSD map full flag handling in the client (where the client may
cancel some OSD ops from a pre-full epoch, so other clients must
wait until the full epoch or later before touching the same objects).
Expand Down
2 changes: 1 addition & 1 deletion doc/cephfs/full.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ time the OSD full flag is sent. Clients update the ``osd_epoch_barrier``
when releasing capabilities on files affected by cancelled operations, in
order to ensure that these cancelled operations do not interfere with
subsequent access to the data objects by the MDS or other clients. For
more on the epoch barrier mechanism, see :ref:`background_blacklisting_and_osd_epoch_barrier`.
more on the epoch barrier mechanism, see :ref:`background_blocklisting_and_osd_epoch_barrier`.

Legacy (pre-hammer) behavior
----------------------------
Expand Down
2 changes: 1 addition & 1 deletion doc/cephfs/mdcache.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Clients can request capabilities and will generally get them, but when
there is competing access or memory pressure on the MDS, they may be
**revoked**. When a capability is revoked, the client is responsible for
returning it as soon as it is able. Clients that fail to do so in a
timely fashion may end up **blacklisted** and unable to communicate with
timely fashion may end up **blocklisted** and unable to communicate with
the cluster.

Since the cache is distributed, the MDS must take great care to ensure
Expand Down
12 changes: 6 additions & 6 deletions doc/cephfs/mds-config-ref.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,14 +69,14 @@
:Default: ``15``


``mds blacklist interval``
``mds blocklist interval``

:Description: The blacklist duration for failed MDSs in the OSD map. Note,
:Description: The blocklist duration for failed MDSs in the OSD map. Note,
this controls how long failed MDS daemons will stay in the
OSDMap blacklist. It has no effect on how long something is
blacklisted when the administrator blacklists it manually. For
example, ``ceph osd blacklist add`` will still use the default
blacklist time.
OSDMap blocklist. It has no effect on how long something is
blocklisted when the administrator blocklists it manually. For
example, ``ceph osd blocklist add`` will still use the default
blocklist time.
:Type: Float
:Default: ``24.0*60.0``

Expand Down
4 changes: 2 additions & 2 deletions doc/dev/osd_internals/manifest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -525,7 +525,7 @@ Operations:
Clears manifest chunks or redirect. Lazily releases references, may
leak.

do_osd_ops seems not to include it in the user_modify=false whitelist,
do_osd_ops seems not to include it in the user_modify=false ignorelist,
and so will trigger a snapshot. Note, this will be true even for a
redirect though SET_REDIRECT does not flip user_modify. This should
be fixed -- unset-manifest should not be a user_modify.
Expand All @@ -538,7 +538,7 @@ Operations:

rados -p base_pool tier-flush <obj-name>

Included in the user_modify=false whitelist, does not trigger a clone.
Included in the user_modify=false ignorelist, does not trigger a clone.

Does not evict the extents.

Expand Down
16 changes: 8 additions & 8 deletions doc/man/8/ceph.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Synopsis
| **ceph** **mon** [ *add* \| *dump* \| *getmap* \| *remove* \| *stat* ] ...
| **ceph** **osd** [ *blacklist* \| *blocked-by* \| *create* \| *new* \| *deep-scrub* \| *df* \| *down* \| *dump* \| *erasure-code-profile* \| *find* \| *getcrushmap* \| *getmap* \| *getmaxosd* \| *in* \| *ls* \| *lspools* \| *map* \| *metadata* \| *ok-to-stop* \| *out* \| *pause* \| *perf* \| *pg-temp* \| *force-create-pg* \| *primary-affinity* \| *primary-temp* \| *repair* \| *reweight* \| *reweight-by-pg* \| *rm* \| *destroy* \| *purge* \| *safe-to-destroy* \| *scrub* \| *set* \| *setcrushmap* \| *setmaxosd* \| *stat* \| *tree* \| *unpause* \| *unset* ] ...
| **ceph** **osd** [ *blocklist* \| *blocked-by* \| *create* \| *new* \| *deep-scrub* \| *df* \| *down* \| *dump* \| *erasure-code-profile* \| *find* \| *getcrushmap* \| *getmap* \| *getmaxosd* \| *in* \| *ls* \| *lspools* \| *map* \| *metadata* \| *ok-to-stop* \| *out* \| *pause* \| *perf* \| *pg-temp* \| *force-create-pg* \| *primary-affinity* \| *primary-temp* \| *repair* \| *reweight* \| *reweight-by-pg* \| *rm* \| *destroy* \| *purge* \| *safe-to-destroy* \| *scrub* \| *set* \| *setcrushmap* \| *setmaxosd* \| *stat* \| *tree* \| *unpause* \| *unset* ] ...
| **ceph** **osd** **crush** [ *add* \| *add-bucket* \| *create-or-move* \| *dump* \| *get-tunable* \| *link* \| *move* \| *remove* \| *rename-bucket* \| *reweight* \| *reweight-all* \| *reweight-subtree* \| *rm* \| *rule* \| *set* \| *set-tunable* \| *show-tunables* \| *tunables* \| *unlink* ] ...
Expand Down Expand Up @@ -613,27 +613,27 @@ osd
Manage OSD configuration and administration. It uses some additional
subcommands.

Subcommand ``blacklist`` manage blacklisted clients. It uses some additional
Subcommand ``blocklist`` manage blocklisted clients. It uses some additional
subcommands.

Subcommand ``add`` add <addr> to blacklist (optionally until <expire> seconds
Subcommand ``add`` add <addr> to blocklist (optionally until <expire> seconds
from now)

Usage::

ceph osd blacklist add <EntityAddr> {<float[0.0-]>}
ceph osd blocklist add <EntityAddr> {<float[0.0-]>}

Subcommand ``ls`` show blacklisted clients
Subcommand ``ls`` show blocklisted clients

Usage::

ceph osd blacklist ls
ceph osd blocklist ls

Subcommand ``rm`` remove <addr> from blacklist
Subcommand ``rm`` remove <addr> from blocklist

Usage::

ceph osd blacklist rm <EntityAddr>
ceph osd blocklist rm <EntityAddr>

Subcommand ``blocked-by`` prints a histogram of which OSDs are blocking their peers

Expand Down
6 changes: 3 additions & 3 deletions doc/man/8/mount.ceph.rst
Original file line number Diff line number Diff line change
Expand Up @@ -82,15 +82,15 @@ Basic
path to file containing the secret key to use with CephX

:command:`recover_session=<no|clean>`
Set auto reconnect mode in the case where the client is blacklisted. The
Set auto reconnect mode in the case where the client is blocklisted. The
available modes are ``no`` and ``clean``. The default is ``no``.

- ``no``: never attempt to reconnect when client detects that it has been
blacklisted. Blacklisted clients will not attempt to reconnect and
blocklisted. Blocklisted clients will not attempt to reconnect and
their operations will fail too.

- ``clean``: client reconnects to the Ceph cluster automatically when it
detects that it has been blacklisted. During reconnect, client drops
detects that it has been blocklisted. During reconnect, client drops
dirty data/metadata, invalidates page caches and writable file handles.
After reconnect, file locks become stale because the MDS loses track of
them. If an inode contains any stale file locks, read/write on the inode
Expand Down
14 changes: 7 additions & 7 deletions doc/rados/operations/control.rst
Original file line number Diff line number Diff line change
Expand Up @@ -237,18 +237,18 @@ Deployments utilizing Nautilus (or later revisions of Luminous and Mimic)
that have no pre-Luminous cients may instead wish to instead enable the
`balancer`` module for ``ceph-mgr``.

Add/remove an IP address to/from the blacklist. When adding an address,
you can specify how long it should be blacklisted in seconds; otherwise,
it will default to 1 hour. A blacklisted address is prevented from
connecting to any OSD. Blacklisting is most often used to prevent a
Add/remove an IP address to/from the blocklist. When adding an address,
you can specify how long it should be blocklisted in seconds; otherwise,
it will default to 1 hour. A blocklisted address is prevented from
connecting to any OSD. Blocklisting is most often used to prevent a
lagging metadata server from making bad changes to data on the OSDs.

These commands are mostly only useful for failure testing, as
blacklists are normally maintained automatically and shouldn't need
blocklists are normally maintained automatically and shouldn't need
manual intervention. ::

ceph osd blacklist add ADDRESS[:source_port] [TIME]
ceph osd blacklist rm ADDRESS[:source_port]
ceph osd blocklist add ADDRESS[:source_port] [TIME]
ceph osd blocklist rm ADDRESS[:source_port]

Creates/deletes a snapshot of a pool. ::

Expand Down
2 changes: 1 addition & 1 deletion doc/rados/operations/user-management.rst
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ The following entries describe valid capability profiles:
:Description: Gives a user permissions to manipulate RBD images. When used
as a Monitor cap, it provides the minimal privileges required
by an RBD client application; this includes the ability
to blacklist other client users. When used as an OSD cap, it
to blocklist other client users. When used as an OSD cap, it
provides read-write access to the specified pool to an
RBD client application. The Manager cap supports optional
``pool`` and ``namespace`` keyword arguments.
Expand Down
10 changes: 5 additions & 5 deletions doc/rbd/rbd-exclusive-locks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ accessing RBD data in an uncoordinated and destructive manner.

Thus, in the event that a lock cannot be acquired in the standard
graceful manner, the overtaking process not only breaks the lock, but
also blacklists the previous lock holder. This is negotiated between
the new client process and the Ceph Mon: upon receiving the blacklist
also blocklists the previous lock holder. This is negotiated between
the new client process and the Ceph Mon: upon receiving the blocklist
request,

* the Mon instructs the relevant OSDs to no longer serve requests from
Expand All @@ -73,10 +73,10 @@ request,
* once the new client has acquired the lock, it can commence writing
to the image.

Blacklisting is thus a form of storage-level resource `fencing`_.
Blocklisting is thus a form of storage-level resource `fencing`_.

In order for blacklisting to work, the client must have the ``osd
blacklist`` capability. This capability is included in the ``profile
In order for blocklisting to work, the client must have the ``osd
blocklist`` capability. This capability is included in the ``profile
rbd`` capability profile, which should generally be set on all Ceph
:ref:`client identities <user-management>` using RBD.

Expand Down
2 changes: 1 addition & 1 deletion qa/cephfs/overrides/whitelist_health.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
overrides:
ceph:
log-whitelist:
log-ignorelist:
- overall HEALTH_
- \(FS_DEGRADED\)
- \(MDS_FAILED\)
Expand Down
2 changes: 1 addition & 1 deletion qa/cephfs/overrides/whitelist_wrongly_marked_down.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
overrides:
ceph:
log-whitelist:
log-ignorelist:
- overall HEALTH_
- \(OSD_DOWN\)
- \(OSD_
Expand Down
2 changes: 1 addition & 1 deletion qa/cephfs/tasks/cfuse_workunit_suites_ffsb.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
overrides:
ceph:
log-whitelist:
log-ignorelist:
- SLOW_OPS
- slow request
conf:
Expand Down
2 changes: 1 addition & 1 deletion qa/overrides/2-size-2-min-size.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ overrides:
global:
osd_pool_default_size: 2
osd_pool_default_min_size: 2
log-whitelist:
log-ignorelist:
- \(REQUEST_STUCK\)
2 changes: 1 addition & 1 deletion qa/overrides/whitelist_wrongly_marked_down.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
overrides:
ceph:
log-whitelist:
log-ignorelist:
- but it is still running
conf:
mds:
Expand Down
2 changes: 1 addition & 1 deletion qa/releases/luminous.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@ overrides:
conf:
mon:
mon warn on osd down out interval zero: false
log-whitelist:
log-ignorelist:
- no active mgr
2 changes: 1 addition & 1 deletion qa/suites/big/rados-thrash/thrashers/default.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
overrides:
ceph:
log-whitelist:
log-ignorelist:
- but it is still running
- objects unfound and apparently lost
tasks:
Expand Down
2 changes: 1 addition & 1 deletion qa/suites/fs/basic_functional/tasks/alternate-pool.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

overrides:
ceph:
log-whitelist:
log-ignorelist:
- bad backtrace
- object missing on disk
- error reading table object
Expand Down
2 changes: 1 addition & 1 deletion qa/suites/fs/basic_functional/tasks/auto-repair.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
overrides:
ceph:
log-whitelist:
log-ignorelist:
- force file system read-only
- bad backtrace
- MDS in read-only mode
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
overrides:
ceph:
log-whitelist:
log-ignorelist:
- Replacing daemon mds
- Scrub error on inode
- Behind on trimming
Expand Down
2 changes: 1 addition & 1 deletion qa/suites/fs/basic_functional/tasks/client-limits.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

overrides:
ceph:
log-whitelist:
log-ignorelist:
- responding to mclientcaps\(revoke\)
- not advance its oldest_client_tid
- failing to advance its oldest client/flush tid
Expand Down
Loading

0 comments on commit fb44c2f

Please sign in to comment.