Guard optional plugin specs and modernize Ruby URI parsing

.github/workflows/ci.yml

@@ -1,10 +1,18 @@
 name: CI
 
-# Installs the bundle and runs the test suite on every supported Ruby version.
+# Installs the bundle, builds the gem and runs the default test suite on the
+# Ruby versions this project validates continuously.
+#
+# The supported range and the validated set are two different statements. The
+# code is written for Ruby 3.3 through 4.0, which is what required_ruby_version
+# says; the matrix below runs the ends of that range and the release in the
+# middle, rather than every intermediate release. See doc/POLICY.md section 11
+# and doc/REQUIREMENTS.md section 20.
 #
 # No credential is configured here and nothing reaches an external service: the
-# suite must need neither (doc/POLICY.md Invariant 6). Specs tagged :network
-# are excluded by default and are not run here.
+# suite must need neither (doc/POLICY.md Invariant 6). Examples tagged :network
+# are excluded by default, and the Gemfile's optional :plugins group is not
+# installed, so no plugin's own gem is a condition of this workflow passing.
 
 on:
   push:
@@ -22,9 +30,10 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        # Keep this in step with required_ruby_version in automatic.gemspec
-        # and with the supported environment section of README.md.
-        ruby: ['3.3', '3.4', '3.5']
+        # The continuously validated versions. Keep this in step with the
+        # supported environment section of README.md; the floor here and
+        # required_ruby_version in automatic.gemspec are the same version.
+        ruby: ['3.3', '3.4', '4.0']
 
     steps:
       - uses: actions/checkout@v4
@@ -43,6 +52,9 @@ jobs:
       - name: Check that the gemspec is valid and buildable
         run: gem build automatic.gemspec
 
+      - name: Check that the library loads
+        run: bundle exec ruby -Ilib -e "require 'automatic'"
+
       - name: Check that the CLI runs
         run: |
           bundle exec bin/automatic --version

Gemfile

@@ -11,10 +11,19 @@ gemspec
 # runtime dependencies of the gem: installing automatic does not install them,
 # and a Recipe that does not use the plugin does not need them.
 #
-# Uncomment what you use in a checkout. The table of which plugin needs which
-# gem, and which of those plugins still work, is in doc/DEPLOYMENT.md and
-# doc/PLUGINS.md section 6.
+# The group is optional, so `bundle install` does not install it and neither
+# the default test suite nor CI depends on it. Install it deliberately, and the
+# specs of the plugins that need it then run as part of the ordinary suite:
+#
+#   BUNDLE_WITH=plugins bundle install
+#   bundle exec rake
+#
+# The table of which plugin needs which gem, and which of those plugins still
+# work, is in doc/DEPLOYMENT.md and doc/PLUGINS.md section 6.
 group :plugins, optional: true do
+  gem 'nkf'               # FilterDescriptionLink
+  gem 'sanitize'          # FilterSanitize
+
   # PublishAmazonS3 and the s3n:// path of StoreFile call AWS::S3, which only
   # AWS SDK for Ruby v1 provided. No currently published gem satisfies them, so
   # there is nothing to uncomment; they need rework. See doc/PLUGINS.md.

README.md

@@ -160,14 +160,23 @@ The full account is [`doc/BASIC_DESIGN.md`](doc/BASIC_DESIGN.md).
 
 ## 4. Supported environment
 
-- **Ruby 3.3 or later.** Tested on 3.3, 3.4 and 3.5.
+- **Ruby 3.3 through 4.0.** CI validates 3.3, 3.4 and 4.0.
 - A Unix-like system. GNU/Linux and macOS are what it is used on. Windows is not
   supported.
 - A compiler, if `nokogiri` or `sqlite3` build from source on your platform.
 
 Ruby 3.3 is the floor: it is the oldest maintained release the dependencies are
-resolved and tested against. Nothing older is tested or supported. A newer Ruby than
-the matrix covers is permitted by the gemspec, which sets a lower bound only.
+resolved and tested against. Nothing older is tested or supported.
+
+Two statements, and they are not the same one:
+
+- **Supported range.** The code is written for Ruby 3.3 through 4.0, using APIs
+  the whole range shares. `required_ruby_version` is `>= 3.3.0` and has no upper
+  bound, so a Ruby newer than the matrix is permitted rather than refused.
+- **Continuously validated versions.** CI runs the ends of the range and the
+  release in the middle — 3.3, 3.4 and 4.0 — rather than every intermediate
+  release. A version's absence from the matrix means it is not verified on every
+  commit; it does not mean it is expected to fail.
 
 ## 5. Installation
 
@@ -463,6 +472,15 @@ COVERAGE=on bundle exec rake spec
   AUTOMATIC_NETWORK_SPECS=1 bundle exec rake spec
   ```
 
+- A plugin whose gem the Gemfile declares in its optional `:plugins` group is
+  **not verified by the default suite**, because that group is not installed.
+  Install it to run those specs as part of the ordinary suite:
+
+  ```sh
+  BUNDLE_WITH=plugins bundle install
+  bundle exec rake
+  ```
+
 - A spec whose plugin needs a gem that is not installed is skipped, and says
   which gem is missing. That absence is the signal; a plugin whose service no
   longer exists is never stubbed into passing.
@@ -471,8 +489,11 @@ COVERAGE=on bundle exec rake spec
   in CI. Most need a credential, a dead service, or both — read one before
   running it.
 
-CI runs `bundle install` and the suite on every supported Ruby version, from
-[`.github/workflows/ci.yml`](.github/workflows/ci.yml).
+CI installs the bundle, builds the gem, loads the library, runs the CLI and runs
+the default suite on each validated Ruby version, from
+[`.github/workflows/ci.yml`](.github/workflows/ci.yml). It configures no secret
+and installs no optional plugin gem, so no plugin's own dependency is a
+condition of a green build.
 
 ## 13. Development
 

automatic.gemspec

@@ -39,9 +39,12 @@ Gem::Specification.new do |spec|
   }
 
   # Ruby 3.3 is the floor: the oldest maintained release the dependencies are
-  # resolved and tested against. This is a lower bound only, so a newer Ruby is
-  # permitted before it reaches the CI matrix. See doc/REQUIREMENTS.md
-  # section 20.
+  # resolved and tested against. The code is written for Ruby 3.3 through 4.0.
+  #
+  # A lower bound only, and deliberately so. An upper bound would refuse a Ruby
+  # this code has every reason to work on, on the day it is released, and the
+  # only way to lift it would be a new release of this gem. What CI validates
+  # is a separate and narrower statement; see doc/REQUIREMENTS.md section 20.
   spec.required_ruby_version = '>= 3.3.0'
 
   # Shipped files. Derived from what Git sees, so that the list cannot drift
@@ -83,24 +86,30 @@ Gem::Specification.new do |spec|
   spec.require_paths      = ['lib']
   spec.extra_rdoc_files   = ['README.md', 'doc/LICENSE.md']
 
-  # Runtime dependencies: what the framework itself needs, plus the store
-  # plugins, which nearly every Recipe uses to avoid repeating its work.
+  # Runtime dependencies: what the framework itself needs, plus what the
+  # documented primary workflow needs — the store plugins, which nearly every
+  # Recipe uses to avoid repeating its work, and the Markdown publisher.
   #
   # A gem needed by a single plugin is NOT declared here. It is required inside
   # that plugin's own file and installed by the operator who uses the plugin.
   # See doc/POLICY.md section 9.1 and doc/DEPLOYMENT.md.
+  #
+  # rexml and rss left the standard library and became gems over the 3.x
+  # series, and nkf followed after 3.3. Each gem listed here is listed because
+  # something committed here requires it, and a library's move out of the
+  # standard library is not by itself a reason to declare it: nkf is a plugin's
+  # dependency and is in the Gemfile's optional group instead.
   spec.add_dependency 'activerecord',  '>= 7.1', '< 9.0'   # store plugins
   spec.add_dependency 'activesupport', '>= 7.1', '< 9.0'   # plugin loader, XML subscription
   spec.add_dependency 'feedbag',       '>= 1.0', '< 2.0'   # autodiscovery subcommand
   spec.add_dependency 'hashie',        '>= 4.0', '< 6.0'   # Recipe
-  # nkf stopped being a default gem in Ruby 3.4 and is needed by
-  # FilterDescriptionLink, which is a Supported plugin; it is a standard
-  # library extraction with no transitive dependencies.
-  spec.add_dependency 'nkf',           '>= 0.1', '< 1.0'    # FilterDescriptionLink
-  spec.add_dependency 'nokogiri',      '>= 1.15', '< 2.0'  # HTML parsing
+  # Used by no framework file on the way in: requiring `automatic` loads no
+  # HTML parser. It is here because Supported plugins that an installed gem
+  # must be able to run need it -- PublishMarkdown, and FeedParser.parse_html
+  # for SubscriptionLink and SubscriptionTumblr.
+  spec.add_dependency 'nokogiri',      '>= 1.15', '< 2.0'  # HTML parsing, in plugins
   spec.add_dependency 'rexml',         '>= 3.2', '< 4.0'   # OPML parser
   spec.add_dependency 'rss',           '>= 0.3', '< 1.0'   # the pipeline value
-  spec.add_dependency 'sanitize',      '>= 6.0', '< 8.0'   # FilterSanitize
   spec.add_dependency 'sqlite3',       '>= 1.7', '< 3.0'   # store plugins
 
   spec.add_development_dependency 'rake',                      '~> 13.0'

doc/DEPLOYMENT.md

@@ -28,8 +28,10 @@ nothing to stop.
 
 - A Unix-like system. GNU/Linux and macOS are what this is used on; Windows is
   not supported.
-- **Ruby 3.3 or later.** Check with `ruby -v`. The supported versions are 3.3,
-  3.4 and 3.5.
+- **Ruby 3.3 through 4.0.** Check with `ruby -v`. CI validates 3.3, 3.4 and 4.0;
+  a version between them is supported and is simply not checked on every commit,
+  and a Ruby newer than 4.0 is permitted rather than refused. See
+  [`REQUIREMENTS.md`](REQUIREMENTS.md) section 20.
 - A build environment for native extensions, because `nokogiri` and `sqlite3`
   may build from source:
 
@@ -395,6 +397,8 @@ several of these plugins talk to services that no longer exist.
 
 | Plugin | Needs | Status |
 | --- | --- | --- |
+| `FilterSanitize` | `sanitize` | Supported |
+| `FilterDescriptionLink` | `nkf` | Supported |
 | `CustomFeedSVNLog` | `xml-simple`, and the `svn` command | Supported (external) |
 | `ProvideFluentd`, `PublishFluentd` | `fluent-logger`, and a Fluentd instance | Supported (external) |
 | `PublishMemcached` | `dalli`, and a memcached server | Supported (external) |
@@ -409,6 +413,8 @@ several of these plugins talk to services that no longer exist.
 | `SubscriptionWeather` | — | Unsupported |
 
 ```sh
+gem install sanitize             # for FilterSanitize
+gem install nkf                  # for FilterDescriptionLink
 gem install fluent-logger        # for the Fluentd plugins
 gem install dalli                # for PublishMemcached
 gem install xml-simple           # for CustomFeedSVNLog
@@ -420,8 +426,16 @@ They call `AWS::S3`, which AWS SDK for Ruby version 1 provided and the current
 need rework. `StoreFile` makes that requirement lazily, so its ordinary HTTP
 download path works with no AWS gem installed at all.
 
-In a checkout, uncomment the `plugins` group in the `Gemfile` instead and run
-`bundle install`.
+In a checkout, install the `Gemfile`'s optional `plugins` group instead —
+uncommenting the entry first, where the gem is one of the commented ones:
+
+```sh
+BUNDLE_WITH=plugins bundle install
+```
+
+That group is not installed by default and is not installed in CI, so these
+plugins are outside what the default test suite verifies. Installing it also
+brings their specs into the ordinary `bundle exec rake` run.
 
 ## Your own plugins
 

doc/PLUGINS.md

@@ -388,7 +388,9 @@ module Automatic::Plugin
 
 That is what keeps a gem needed by one plugin out of everyone else's
 installation. A gem used by a single plugin is not added to the framework's
-runtime dependencies; see [`POLICY.md`](POLICY.md) section 9.
+runtime dependencies; it goes in the `Gemfile`'s optional `:plugins` group and
+the operator who uses the plugin installs it. See [`POLICY.md`](POLICY.md)
+section 9.
 
 Where a plugin has an optional capability that needs a heavier library — S3
 support in `StoreFile`, for instance — the `require` goes inside the branch that
@@ -520,6 +522,14 @@ Two rules govern this table, and they are the reason it exists at all:
   framework was used for, and several remain useful as templates for a
   replacement. Removal is a separate, deliberate decision.
 
+**Supported is not the same as covered by CI.** A Supported plugin whose gem is
+an optional plugin dependency — `FilterSanitize` and `FilterDescriptionLink` —
+works, and is simply not part of what a green build guarantees, because the
+default bundle does not install that gem. Its entry says so, and installing the
+gem runs its spec as part of the ordinary suite. Nothing here is classified by
+what CI happens to run; a plugin is not demoted for needing a gem, and is not
+promoted by a test that CI never executes.
+
 **This classification is a snapshot taken in August 2026,** based on the
 published status of each service and on what each plugin's code actually calls.
 The statuses in the "external service" rows depend on the outside world and can
@@ -759,6 +769,11 @@ page the link points at. Fetching pages means network access. No settings.
 | --- | --- | --- |
 | `mode` | string | `basic`, `relaxed`, or `restricted`. Default `restricted`. |
 
+Needs the `sanitize` gem, which is an optional plugin dependency and is not
+installed with the framework. Its spec is therefore outside the default suite
+and outside CI; installing the gem brings the spec back into the ordinary run.
+See [`DEPLOYMENT.md`](DEPLOYMENT.md).
+
 #### FilterTumblrResize — **Supported**
 
 `filter/tumblr_resize.rb`. Rewrites a Tumblr image link to the 1280-pixel
@@ -779,6 +794,12 @@ the body.
 `get_title` makes one request per item; use `FilterOne` or a store plugin before
 it on a large feed.
 
+Needs the `nkf` gem, which the plugin uses to normalize a fetched page's
+encoding. `nkf` left the standard library after Ruby 3.3 and is an optional
+plugin dependency rather than a framework one, so it is not installed with the
+framework, and this plugin's spec is outside the default suite and outside CI.
+See [`DEPLOYMENT.md`](DEPLOYMENT.md).
+
 #### FilterFullFeed — **Supported (external)**
 
 `filter/full_feed.rb`. Replaces a summary with the article body, by matching the
@@ -997,11 +1018,14 @@ arbitrary markup back into equivalent Markdown — tables, nested lists, inline
 links, images — is a large job with a large library behind it, and a library
 that size does not become a dependency for one plugin
 ([`POLICY.md`](POLICY.md) section 9.1). Reducing markup to text needs nothing
-that is not already installed: `nokogiri` is a runtime dependency, used by the
-framework's own feed adapters. The result is defined by its two ends — the
-text survives, the markup does not — which is what both a reader and a program
-reading the file want from it. A link inside a body becomes its own text; the
-item's own link is in the metadata list, where nothing loses it.
+beyond `nokogiri`, which `gem install automatic` installs: it is a runtime
+dependency of this gem precisely so that the Supported plugins an installed gem
+must be able to run — this one, and `FeedParser.parse_html` for
+`SubscriptionLink` and `SubscriptionTumblr` — work with nothing else added.
+Requiring `automatic` itself loads no HTML parser. The result is defined by its
+two ends — the text survives, the markup does not — which is what both a reader
+and a program reading the file want from it. A link inside a body becomes its
+own text; the item's own link is in the metadata list, where nothing loses it.
 
 Where a different treatment is wanted, the pipeline already has the means:
 `FilterSanitize` before this plugin decides what markup survives into the