Removed potentially offensive terms

- Replaced the terms 'whitelist' and 'blacklist' throughout the code and docs,
  where possible.  No CLI argument changes, but some console output terminology
  changed for 'ksconf sort', no other changes noted.
- Fixed a few minor typos that were observed at the same time.

README.md

@@ -8,6 +8,8 @@
 [![Documentation Status](https://readthedocs.org/projects/ksconf/badge/?version=latest)](https://ksconf.readthedocs.io/en/latest/?badge=latest)
 [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/kintyre-splunk-conf.svg)](https://pypi.org/project/kintyre-splunk-conf/)
 ![PyPI - Downloads](https://img.shields.io/pypi/dm/kintyre-splunk-conf.svg)
+[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)
+
 
 ![Ksconf logo][logo]
 

docs/source/changelog.rst

@@ -11,22 +11,25 @@ New functionality, massive documentation improvements, metadata support, and Spl
 Release v0.7.8 (DRAFT)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
--   New automatic ``promote`` mode is now avaiable using CLI arguments!
+-   New automatic ``promote`` mode is now available using CLI arguments!
     This allows stanzas to be selected for promotion from the CLI in batch and interactive modes.
     This implementation borrows (and shares code) with the ``ksconf filter`` command so hopefully the CLI arguments look familiar.
-    It's possible to promote a single stanza, a stanza wildcard, regex or invert the matching logic and promote everything except for the named stanza (blacklist).
+    It's possible to promote a single stanza, a stanza wildcard, regex or invert the matching logic and promote everything except for the named stanza (blocklist).
     Right now ``--stanza`` is the only supporting matching mode, but more can be added as needed.
     A huge thanks to mthambipillai for providing a pull-request with an initial implementation of this feature!
 -   Added a new summary output mode (``ksconf promote --summary``) that will provide a quick summary of what content could be promoted.
     This can be used along side the new ``--stanza`` filtering options to show the names of stanzas that can be promoted.
+-   Replaced insensitive terminology with race-neutral terms.  Specifically the terms 'blacklist' and 'whitelist' have been replaced.
+    NOTE:  This does *not* change any CLI attributes, but in a few cases the standard output terminology is slighly different.
+    Also terminology in ``.conf`` files couldn't be updated as that's controlled by Splunk.
 -   Fixed bug in the ``unarchive`` command where a ``locale`` folder was blocked as a ``local`` folder.
 -   Fixed bug with ``minimize`` when the required ``--target`` argument is not given.  This now results in a reminder to the user rather than an unhandled exception.
 -   Splunk app packaging fix.  Write access to the app was previously not granted due to a spelling mistake in the metadata file.
 
 Release v0.7.7 (2020-03-05)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
--   Added new ``--follow-symlink`` option to the ``combine`` command so that input directory structures with sybolic links can be treated the same as proper directories.
+-   Added new ``--follow-symlink`` option to the ``combine`` command so that input directory structures with symbolic links can be treated the same as proper directories.
 -   Corrected Windows issue where wildcard (glob) patterns weren't expanded by for ``check`` and ``sort``.  This is primarily a difference in how a proper shells (e.g., bash, csh, zsh) handle expansion natively vs CMD on Windows does not.  However, since this is typically transparently handled by many CLI tools, we'll follow suite.  (BTW, running ksconf from the GIT Bash prompt is a great alternative.)  Only the most minimalistic expansion rules will be available, (so don't expect ``{props,transforms,app}.conf`` to work anytime soon), but this should be good enough for most use cases.  Thanks to SID800 for reporting this bug.
 -   Fixed issues with the ``unarchive`` command when ``git`` is not installed or an app is being unarchived (installed/upgrade) into a location not managed by Git.  Note that additional output is now enabled when the ``KSCONF_DEBUG`` environmental variable is set (in lieu of a proper verbose mode).  Bug report provided by SID800.
 -   Enhanced ``ksconf --version`` output to include Git executable path and version information; as well as a platform dump.  (Helpful for future bug reporting.)
@@ -186,7 +189,7 @@ Release v0.5.5 (2019-01-28)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 -  New :ref:`ksconf_cmd_filter` command added for slicing up a conf file into smaller pieces. Think of this as
-   GREP that’s stanza-aware. Can also whitelist or blacklist attributes, if desirable.
+   GREP that’s stanza-aware. Can also allow or block attributes, if desirable.
 -  Expanded ``rest-export`` CLI capabilities to include a new ``--delete`` option, pretty-printing,
    and now supports stdin by allowing the user to explicitly set the file type using ``--conf``.
 -  Refactored all CLI unittests for increased readability and long-term maintenance. Unit tests
@@ -346,7 +349,7 @@ rewritten.
 Release legacy-v1.0.1 (2018-04-20)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
--  Fixes to blacklist support and many enhancements to ``ksconf unarchive``.
+-  Fixes to blocklist support and many enhancements to ``ksconf unarchive``.
 -  Introduces parsing profiles.
 -  Lots of bug fixes to various subcommands.
 -  Added automatic detection of ‘subcommands’ for CLI documentation helper script.

docs/source/cmd_promote.rst

@@ -62,7 +62,7 @@ Key features include:
     Inversion
         The ``--invert-match`` option allows for the selection to be inverted.
         In this mode, it's possible to select which stanzas should *not* be promoted.
-        This can be used as a blacklist to prevent accidental promotions.
+        This can be used as a blocklist to prevent accidental promotions.
 
 
 

docs/source/dyn/cli.rst

@@ -245,7 +245,7 @@ ksconf filter
     
     Attribute selection:
       Include or exclude attributes passed through. By default, all attributes
-      are preserved. Whitelist (keep) operations are preformed before blacklist
+      are preserved. Allowlist (keep) operations are preformed before blocklist
       (reject) operations.
     
       --keep-attrs WC-ATTR  Select which attribute(s) will be preserved. This
@@ -455,7 +455,7 @@ ksconf sort
     config file will be echoed to the screen.  (2) the config files are updated
     in-place when the '-i' option is used.
     
-    Manually managed conf files can be blacklisted by adding a comment containing the
+    Manually managed conf files can be protected against changes by adding a comment containing the
     string 'KSCONF-NO-SORT' to the top of any .conf file.
     
     positional arguments:

ksconf/commands/combine.py

@@ -122,7 +122,7 @@ def run(self, args):
         for src_root in args.source:
             for (root, dirs, files) in relwalk(src_root, followlinks=args.follow_symlink):
                 for fn in files:
-                    # Todo: Add blacklist CLI support:  defaults to consider: *sw[po], .git*, .bak, .~
+                    # Todo: Add blocklist CLI support:  defaults to consider: *sw[po], .git*, .bak, .~
                     if fn.endswith(".swp") or fn.endswith("*.bak"):
                         continue  # pragma: no cover  (peephole optimization)
                     src_file = os.path.join(root, fn)
@@ -135,7 +135,7 @@ def run(self, args):
             for fn in files:
                 tgt_file = os.path.join(root, fn)
                 if tgt_file not in src_file_index:
-                    # Todo:  Add support for additional blacklist wildcards (using fnmatch)
+                    # Todo:  Add support for additional blocklist wildcards (using fnmatch)
                     if fn == CONTROLLED_DIR_MARKER or fn.endswith(".bak"):
                         continue  # pragma: no cover (peephole optimization)
                     target_extra_files.add(tgt_file)

ksconf/commands/filter.py

@@ -122,7 +122,7 @@ def register_args(self, parser):
         pg_con = parser.add_argument_group("Attribute selection", dedent("""\
             Include or exclude attributes passed through.
             By default, all attributes are preserved.
-            Whitelist (keep) operations are preformed before blacklist (reject) operations."""))
+            Allowlist (keep) operations are preformed before blocklist (reject) operations."""))
 
         pg_con.add_argument("--keep-attrs", metavar="WC-ATTR", default=[], action="append",
                             help=dedent("""\
@@ -151,7 +151,7 @@ def prep_filters(self, args):
             self.attrs_keep_filter = FilterListWildcard(flags)
             for attrs in args.keep_attrs:
                 self.attrs_keep_filter.feedall(attrs.split(" "))
-            self.attrs_reject_filter = FilterListWildcard(FilteredList.BLACKLIST | flags)
+            self.attrs_reject_filter = FilterListWildcard(FilteredList.INVERT | flags)
             for attrs in args.reject_attrs:
                 self.attrs_reject_filter.feedall(attrs.split(" "))
         else:

ksconf/commands/sort.py

@@ -30,7 +30,7 @@ class SortCmd(KsconfCmd):
     config file will be echoed to the screen.  (2) the config files are updated
     in-place when the ``-i`` option is used.
 
-    Manually managed conf files can be blacklisted by adding a comment containing the
+    Manually managed conf files can be protected against changes by adding a comment containing the
     string ``KSCONF-NO-SORT`` to the top of any .conf file.
     """)
     format = "manual"
@@ -86,7 +86,7 @@ def run(self, args):
                 try:
                     if not args.force and _has_nosort_marker(conf):
                         if not args.quiet:
-                            self.stderr.write("Skipping blacklisted file {}\n".format(conf))
+                            self.stderr.write("Skipping no-sort file {}\n".format(conf))
                         continue
                     c = self.parse_conf(conf, mode='r+', raw_exec=True)
                     #c = parse_conf(conf, profile=PARSECONF_STRICT)

ksconf/commands/unarchive.py

@@ -310,7 +310,7 @@ def fixup_pattern_bw(patterns, prefix=None):
         self.stdout.write("Extracting app now...\n")
         for gaf in files_iter:
             if match_bwlist(gaf.path, excludes, escape=False):
-                self.stdout.write("Skipping [blacklist] {}\n".format(gaf.path))
+                self.stdout.write("Skipping [blocklist] {}\n".format(gaf.path))
                 continue
             if not is_git or args.git_mode in ("nochange", "stage"):
                 self.stdout.write("{0:60s} {2:o} {1:-6d}\n".format(gaf.path, gaf.size, gaf.mode))
@@ -332,7 +332,7 @@ def fixup_pattern_bw(patterns, prefix=None):
         self.stdout.write("Extracted {} files:  {} new, {} existing, and {} removed\n".format(
             len(installed_files), len(files_new), len(files_upd), len(files_del)))
 
-        # Filer out "removed" files; and let us keep some based on a keep-whitelist:  This should
+        # Filer out "removed" files; and let us keep some based on a keep-allowlist:  This should
         # include things like local, ".gitignore", ".gitattributes" and so on
 
         keep_list = [".git*"]
@@ -355,7 +355,7 @@ def fixup_pattern_bw(patterns, prefix=None):
             else:
                 files_to_delete.append(fn)
         if files_to_keep:
-            self.stdout.write("Keeping {} of {} files marked for deletion due to whitelist.\n"
+            self.stdout.write("Keeping {} of {} files marked for deletion due to allow list.\n"
                               .format(len(files_to_keep), len(files_del)))
         git_rm_queue = []
 

ksconf/filter.py

@@ -11,9 +11,9 @@
 
 
 class FilteredList(object):
-    IGNORECASE = I = 1
-    BLACKLIST = B = 2
-    VERBOSE = V = 4
+    IGNORECASE = 1
+    INVERT = 2
+    VERBOSE = 4
 
     def __init__(self, flags=0):
         self.data = []
@@ -75,7 +75,7 @@ def match(self, item):
         else:
             #  No patterns defined.  No filter rule(s) => allow all through
             return True
-        if self.flags & self.BLACKLIST:
+        if self.flags & self.INVERT:
             return not result
         else:
             return result

make_splunk_app

@@ -59,7 +59,7 @@ echo "Installing packages...."
 #        be listed as a 'install_requires' for that to work correctly.  (Although if it's missing
 #        at runtime, only one command will be impacted)
 #
-# There is no way to simply "blacklist" a single package:  https://github.com/pypa/pip/issues/3090
+# There is no way to simply block a single package:  https://github.com/pypa/pip/issues/3090
 $PY2 -m pip install --isolated --disable-pip-version-check --no-deps --target="$PIP_TARGET" \
     "$wheel_dir"/*.whl entrypoints splunk-sdk
 

tests/test_cli_filter.py

@@ -450,7 +450,7 @@ def props03(self):
         JSON_TRIM_BRACES_IN_ARRAY_NAMES = true
         """)
 
-    def test_filter_attrs_whitelist(self):
+    def test_filter_attrs_keep(self):
         with ksconf_cli:
             ko = ksconf_cli("filter", self.props03, "--keep-attrs", "SHOULD_LINEMERGE")
             self.assertEqual(ko.returncode, EXIT_CODE_SUCCESS)
@@ -460,7 +460,7 @@ def test_filter_attrs_whitelist(self):
             self.assertEqual(len(out["kvstore"]), 1)
             self.assertEqual(list(out["kvstore"]), ["SHOULD_LINEMERGE"])
 
-    def test_filter_attrs_blacklist(self):
+    def test_filter_attrs_rejected(self):
         with ksconf_cli:
             ko = ksconf_cli("filter", self.props03,
                             "--reject-attrs", "METRICS_PROTOCOL NO_BINARY_CHECK")
@@ -475,7 +475,7 @@ def test_filter_attrs_blacklist(self):
             self.assertIn("JSON_TRIM_BRACES_IN_ARRAY_NAMES", keys)
 
     def test_filter_attrs_whbllist(self):
-        """ Confirm that blacklist is applied after whitelist """
+        """ Confirm that 'reject' is applied after 'keep'"""
         with ksconf_cli:
             ko = ksconf_cli("filter", self.props03, self.props02, "--ignore-case",
                             "--keep-attrs", "*TIME*",

tests/test_cli_sort.py

@@ -94,7 +94,7 @@ def test_sort_mixed(self):
             self.assertEqual(ko.returncode, EXIT_CODE_BAD_CONF_FILE)
             self.assertRegex(ko.stderr,
                              r"Error [^\r\n]+? file [^\r\n]+?[/\\]badfile\.conf[^\r\n]+ \[BAD_STANZA")
-            self.assertRegex(ko.stderr, r"Skipping blacklisted file [^ ]+[/\\]transforms\.conf")
+            self.assertRegex(ko.stderr, r"Skipping no-sort file [^ ]+[/\\]transforms\.conf")
 
     def test_sort_stdout(self):
         # Not yet implemented.  Currently relying on the shell to do this.

tests/test_filter.py

@@ -42,9 +42,9 @@ def test_string(self):
         fl, res = self.filter("string", ("ftp:auth", "http:auth"), self.sample01)
         self.assertEqual(res, ["ftp:auth", "http:auth"])
 
-    def test_string_blackslist(self):
+    def test_string_invert(self):
         fl, res = self.filter("string", ("ftp:auth", "http:auth"), self.sample01,
-                              flags=FilteredList.BLACKLIST)
+                              flags=FilteredList.INVERT)
         self.assertEqual(res, ["ftp:exchange", "ftp:transfer", "ftp:debug", "http:exchange"])
 
     def test_regex(self):
@@ -93,10 +93,10 @@ def test_wildcard_counter(self):
         self.assertEqual(fl.counter["*nomatch*"], 0)
         self.assertEqual(len(fl.counter), 2)
 
-    def test_string_blacklist_counter(self):
-        # Note that blacklist (match inversion) doesn't change the counter numbers calculation.
+    def test_string_invert_counter(self):
+        # Note that match inversion doesn't change the counter numbers calculation.
         fl, res = self.filter("string", ("http:auth", "ftp:auth", "ftp:bogus"), self.sample01,
-                              flags=FilteredList.BLACKLIST)
+                              flags=FilteredList.INVERT)
         self.assertEqual(res, ["ftp:exchange", "ftp:transfer", "ftp:debug", "http:exchange"])
         self.assertEqual(fl.counter["ftp:auth"], 1)
         self.assertEqual(fl.counter["http:auth"], 1)
@@ -105,7 +105,6 @@ def test_string_blacklist_counter(self):
         self.assertEqual(len(fl.counter), 3)
 
     def test_string_ignorecase_counter(self):
-        # Note that blacklist (match inversion) doesn't change the counter numbers calculation.
         sample = list(self.sample01)
         sample[4] = sample[4].upper()
         fl, res = self.filter("string", ("http:AUTH", "fTp:AuTh"), self.sample01, flags=FilteredList.IGNORECASE)