Conditional Slicing

doc/admin-guide/plugins/slice.en.rst

@@ -153,6 +153,41 @@ The slice plugin supports the following options::
         Enable slice plugin to strip Range header for HEAD requests.
         -h for short
 
+    --minimum-size (optional)
+    --metadata-cache-size (optional)
+    --stats-prefix (optional)
+        In combination, these three options allow for conditional slice.
+        Specify the minimum size object to slice with --minimum-size.  Allowed
+        values are the same as --blockbytes.  Conditional slicing uses a cache
+        of object sizes to make the decision of whether to slice.  The cache
+        will only store the URL of large objects as they are discovered in
+        origin responses.  You should set the --metadata-cache-size to by
+        estimating the working set size of large objects.  You can use
+        stats to determine whether --metadata-cache-size was set optimally.
+        Stat names are prefixed with the value of --stats-prefix.  The names
+        are:
+
+        <prefix>.metadata_cache.true_large_objects - large object cache hits
+        <prefix>.metadata_cache.true_small_objects - small object cache hits
+        <prefix>.metadata_cache.false_large_objects - large object cache misses
+        <prefix>.metadata_cache.false_small_objects - small object cache misses
+        <prefix>.metadata_cache.no_content_length - number of responses without content length
+        <prefix>.metadata_cache.bad_content_length - number of responses with invalid content length
+        <prefix>.metadata_cache.no_url - number of responses where URL parsing failed
+
+        If an object size is not found in the object size cache, the plugin
+        will not slice the object, and will turn off ATS cache on this request.
+        The object size will be cached in following requests, and slice will
+        proceed normally if the object meets the minimum size requirement.
+
+        Range requests from the client for small objects are passed through the
+        plugin unchanged.  If you use the `cache_range_requests` plugin, slice plugin
+        will communicate with `cache_range_requests` using an internal header
+        that causes `cache_range_requests` to be bypassed in such requests, and
+        allow ATS to handle those range requests internally.
+
+
+
 Examples::
 
     @plugin=slice.so @pparam=--blockbytes=1000000 @plugin=cache_range_requests.so
@@ -307,6 +342,36 @@ The functionality works with `--ref-relative` both enabled and disabled. If `--r
 disabled (using slice 0 as the reference block), requesting to PURGE a block that does not have
 slice 0 in its range will still PURGE the slice 0 block, as the reference block is always processed.
 
+Conditional Slicing
+-------------------
+
+The goal of conditional slicing is to slice large objects and avoid the cost of slicing on small
+objects.  If `--minimum-size` is specified, conditional slicing is enabled and works as follows.
+
+The plugin builds a object size cache in memory.  The key is the URL of the object.  Only
+large object URLs are written to the cache.  The object size cache uses CLOCK eviction algorithm
+in order to have lazy promotion behavior.
+
+When a URL not found in the object size cache, the plugin treats the object as a small object.  It
+will not intercept the request.  The request is processed by ATS without any slice logic.  Upon
+receiving a response, the slice plugin will check the response content length to update the object
+size cache if necessary.
+
+When a large URL is requested for the first time, conditional slicing will not intercept that
+request since the URL is not known to be large.  This will cause an ATS cache miss and the request
+will go to origin server.  Slice plugin will turn off writing to cache for this response, because
+it expects to slice this object in future requests.
+
+If the object size cache evicts a URL, the size of the object for that URL will need to be learned
+again in a subsequent request, and the behavior above will happen again.
+
+If the URL is found in the object size cache, conditional slicing treats the object as a large object
+and will activate the slicing logic as described in the rest of this document.
+
+If the client sends a range request, and that URL is not in the object size cache, the slice plugin
+will forward the range request to ATS core.  It also attaches an internal header in order to deactivate
+the `cache_range_requests` plugin for this range request.
+
 Important Notes
 ===============
 

plugins/cache_range_requests/cache_range_requests.cc

@@ -50,9 +50,10 @@ enum parent_select_mode_t {
   PS_CACHEKEY_URL, // Set parent selection url to cache_key url
 };
 
-constexpr std::string_view DefaultImsHeader = {"X-Crr-Ims"};
-constexpr std::string_view SLICE_CRR_HEADER = {"Slice-Crr-Status"};
-constexpr std::string_view SLICE_CRR_VAL    = "1";
+constexpr std::string_view DefaultImsHeader  = {"X-Crr-Ims"};
+constexpr std::string_view SLICE_CRR_HEADER  = {"Slice-Crr-Status"};
+constexpr std::string_view SLICE_CRR_VAL     = "1";
+constexpr std::string_view SKIP_CRR_HDR_NAME = {"X-Skip-Crr"};
 
 struct pluginconfig {
   parent_select_mode_t ps_mode{PS_DEFAULT};
@@ -86,6 +87,7 @@ bool                 set_header(TSMBuffer, TSMLoc, const char *, int, const char
 int                  transaction_handler(TSCont, TSEvent, void *);
 struct pluginconfig *create_pluginconfig(int argc, char *const argv[]);
 void                 delete_pluginconfig(pluginconfig *const);
+static bool          has_skip_crr_header(TSHttpTxn);
 
 /**
  * Creates pluginconfig data structure
@@ -192,12 +194,32 @@ handle_read_request_header(TSCont /* txn_contp ATS_UNUSED */, TSEvent /* event A
 {
   TSHttpTxn txnp = static_cast<TSHttpTxn>(edata);
 
-  range_header_check(txnp, gPluginConfig);
+  if (!has_skip_crr_header(txnp)) {
+    range_header_check(txnp, gPluginConfig);
+  }
 
   TSHttpTxnReenable(txnp, TS_EVENT_HTTP_CONTINUE);
   return 0;
 }
 
+static bool
+has_skip_crr_header(TSHttpTxn txnp)
+{
+  TSMBuffer hdr_buf = nullptr;
+  TSMLoc    hdr_loc = TS_NULL_MLOC;
+  bool      ret     = false;
+
+  if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &hdr_buf, &hdr_loc)) {
+    TSMLoc const skip_crr_loc = TSMimeHdrFieldFind(hdr_buf, hdr_loc, SKIP_CRR_HDR_NAME.data(), SKIP_CRR_HDR_NAME.length());
+    if (TS_NULL_MLOC != skip_crr_loc) {
+      TSHandleMLocRelease(hdr_buf, hdr_loc, skip_crr_loc);
+      ret = true;
+    }
+    TSHandleMLocRelease(hdr_buf, TS_NULL_MLOC, hdr_loc);
+  }
+  return ret;
+}
+
 /**
  * Reads the client request header and if this is a range request:
  *
@@ -682,7 +704,9 @@ TSRemapDoRemap(void *ih, TSHttpTxn txnp, TSRemapRequestInfo * /* rri */)
 {
   pluginconfig *const pc = static_cast<pluginconfig *>(ih);
 
-  range_header_check(txnp, pc);
+  if (!has_skip_crr_header(txnp)) {
+    range_header_check(txnp, pc);
+  }
 
   return TSREMAP_NO_REMAP;
 }

plugins/slice/CMakeLists.txt

@@ -31,6 +31,7 @@ add_atsplugin(
   slice.cc
   transfer.cc
   util.cc
+  ObjectSizeCache.cc
 )
 
 target_link_libraries(slice PRIVATE PCRE::PCRE)

plugins/slice/Config.cc

@@ -18,6 +18,7 @@
 
 #include "Config.h"
 
+#include <cassert>
 #include <cctype>
 #include <cinttypes>
 #include <cstdlib>
@@ -123,13 +124,16 @@ Config::fromArgs(int const argc, char const *const argv[])
     {const_cast<char *>("blockbytes-test"),      required_argument, nullptr, 't'},
     {const_cast<char *>("prefetch-count"),       required_argument, nullptr, 'f'},
     {const_cast<char *>("strip-range-for-head"), no_argument,       nullptr, 'h'},
+    {const_cast<char *>("minimum-size"),         required_argument, nullptr, 'm'},
+    {const_cast<char *>("metadata-cache-size"),  required_argument, nullptr, 'z'},
+    {const_cast<char *>("stats-prefix"),         required_argument, nullptr, 'x'},
     {nullptr,                                    0,                 nullptr, 0  },
   };
 
   // getopt assumes args start at '1' so this hack is needed
   char *const *argvp = (const_cast<char *const *>(argv) - 1);
   for (;;) {
-    int const opt = getopt_long(argc + 1, argvp, "b:dc:e:i:lp:r:s:t:", longopts, nullptr);
+    int const opt = getopt_long(argc + 1, argvp, "b:dc:e:i:lm:p:r:s:t:x:z:", longopts, nullptr);
     if (-1 == opt) {
       break;
     }
@@ -228,6 +232,29 @@ Config::fromArgs(int const argc, char const *const argv[])
     case 'h': {
       m_head_strip_range = true;
     } break;
+    case 'm': {
+      int64_t const bytesread = bytesFrom(optarg);
+      if (bytesread < 0) {
+        DEBUG_LOG("Invalid minimum-size: %s", optarg);
+      }
+      m_min_size_to_slice = bytesread;
+      DEBUG_LOG("Only slicing objects %" PRIu64 " bytes or larger", m_min_size_to_slice);
+    } break;
+    case 'z': {
+      try {
+        size_t size = std::stoul(optarg);
+        setCacheSize(size);
+        DEBUG_LOG("Metadata cache size: %zu entries", size);
+      } catch (const std::invalid_argument &e) {
+        ERROR_LOG("Invalid metadata cache size argument: %s", optarg);
+      } catch (const std::out_of_range &e) {
+        ERROR_LOG("Metadata cache size out of range: %s", optarg);
+      }
+    } break;
+    case 'x': {
+      stat_prefix = optarg;
+      DEBUG_LOG("Stat prefix: %s", stat_prefix.c_str());
+    } break;
     default:
       break;
     }
@@ -256,6 +283,15 @@ Config::fromArgs(int const argc, char const *const argv[])
     DEBUG_LOG("Using default slice skip header %s", m_skip_header.c_str());
   }
 
+  if (m_min_size_to_slice > 0) {
+    if (m_oscache.has_value()) {
+      DEBUG_LOG("Metadata cache size: %zu", m_oscache->cache_capacity());
+    } else {
+      ERROR_LOG("--metadata-cache-size is required when --minimum-size is specified!  Using a default size of 16384.");
+      setCacheSize(16384);
+    }
+  }
+
   return true;
 }
 
@@ -309,3 +345,51 @@ Config::matchesRegex(char const *const url, int const urllen) const
 
   return matches;
 }
+
+void
+Config::setCacheSize(size_t entries)
+{
+  if (entries == 0) {
+    m_oscache.reset();
+  } else {
+    m_oscache.emplace(entries);
+  }
+}
+
+bool
+Config::isKnownLargeObj(std::string_view url)
+{
+  if (m_min_size_to_slice <= 0) {
+    // If conditional slicing is not set, all objects are large enough to slice
+    return true;
+  }
+
+  assert(m_oscache.has_value()); // object size cache is always present when conditionally slicing
+  std::optional<uint64_t> size = m_oscache->get(url);
+  if (size.has_value()) {
+    DEBUG_LOG("Found url in cache: %.*s -> %" PRIu64, static_cast<int>(url.size()), url.data(), size.value());
+    if (size.value() >= m_min_size_to_slice) {
+      return true;
+    }
+  }
+
+  return false;
+}
+
+void
+Config::sizeCacheAdd(std::string_view url, uint64_t size)
+{
+  if (m_oscache) {
+    DEBUG_LOG("Adding url to cache: %.*s -> %" PRIu64, static_cast<int>(url.size()), url.data(), size);
+    m_oscache->set(url, size);
+  }
+}
+
+void
+Config::sizeCacheRemove(std::string_view url)
+{
+  if (m_oscache) {
+    DEBUG_LOG("Removing url from cache: %.*s", static_cast<int>(url.size()), url.data());
+    m_oscache->remove(url);
+  }
+}

plugins/slice/Config.h

@@ -19,6 +19,7 @@
 #pragma once
 
 #include "slice.h"
+#include "ObjectSizeCache.h"
 
 #ifdef HAVE_PCRE_PCRE_H
 #include <pcre/pcre.h>
@@ -45,8 +46,9 @@ struct Config {
   int         m_paceerrsecs{0};   // -1 disable logging, 0 no pacing, max 60s
   int         m_prefetchcount{0}; // 0 disables prefetching
   enum RefType { First, Relative };
-  RefType m_reftype{First};          // reference slice is relative to request
-  bool    m_head_strip_range{false}; // strip range header for head requests
+  RefType  m_reftype{First};          // reference slice is relative to request
+  bool     m_head_strip_range{false}; // strip range header for head requests
+  uint64_t m_min_size_to_slice{0};    // Only strip objects larger than this
 
   std::string m_skip_header;
   std::string m_crr_ims_header;
@@ -73,7 +75,23 @@ struct Config {
   // If no null reg, true, otherwise check against regex
   bool matchesRegex(char const *const url, int const urllen) const;
 
+  // Add an object size to cache
+  void sizeCacheAdd(std::string_view url, uint64_t size);
+
+  // Remove an object size
+  void sizeCacheRemove(std::string_view url);
+
+  // Did we cache this internally as a small object?
+  bool isKnownLargeObj(std::string_view url);
+
+  // Metadata cache stats
+  std::string stat_prefix{};
+  int         stat_TP{0}, stat_TN{0}, stat_FP{0}, stat_FN{0}, stat_no_cl{0}, stat_bad_cl{0}, stat_no_url{0};
+  bool        stats_enabled{false};
+
 private:
-  TSHRTime   m_nextlogtime{0}; // next time to log in ns
-  std::mutex m_mutex;
+  TSHRTime                       m_nextlogtime{0}; // next time to log in ns
+  std::mutex                     m_mutex;
+  std::optional<ObjectSizeCache> m_oscache;
+  void                           setCacheSize(size_t entries);
 };