Skip to content

v0.29
Compare
Choose a tag to compare
@spladug spladug released this 30 Mar 05:46
v0.29.0
0f2cf46

Important

Apache Thrift

This is a major, breaking change. The Thrift implementation used has been changed from an ancient version of FBThrift to modern Apache Thrift. This comes with all the same bells and whistles (THeaderProtocol) and should generally be of comparable performance, but allows us to keep up with upstream much more effectively and generally gives us more flexibility. See the "Upgrading" section below for details of how to upgrade.

New Features

Runtime metrics

This is the first release of Baseplate that automatically sends per-process server metrics outside of those your application sends. The first metric to send is a gauge indicating the number of active requests being handled in each process. More server/runtime metrics will be coming very soon.

Pyramid CSRF Policy

A CSRF implementation for Pyramid suitable for use in an intranet environment is now included in Baseplate.

Changes

  • BREAKING CHANGE The max_concurrency setting for servers is now mandatory. It is very important to configure this to something meaningful for your application based on the IO vs. CPU usage of requests.
  • A client or server raising Thrift exceptions defined in the IDL will no longer count those as failed RPCs for metrics purposes.
  • Thrift RPC failures now include service and method names in the error log.
  • Setting up thrift clients on the request context now takes far less work. This performs significantly better on services with many clients.
  • The secrets store will only re-check the file on disk for modifications once per request. This performs significantly better on services that use many secrets per request.
  • When baseplate-serve binds its own socket, SO_REUSEPORT is now used to improve the balance of load across processes on the same host. This has no effect when running under an Einhorn configured to bind the socket (--bind).
  • The method which Pyramid servers determine if they'll accept Trace headers from a client can now be controlled by the application.
  • The secret fetcher sidecar now supports Vault 0.9+ authentication.
  • FileWatcher, SecretsStore, and experiments config all have a timeout parameter that control how long they will block waiting for the underlying file to be available during initial startup.
  • The experiments framework no longer creates a local span each time bucketing happens.
  • When Vault authentication fails, the error message gives some advice about how to resolve the situation.
  • Event publishers now use exponential backoff when publishing to the event collectors.

Bug fixes

  • The live data sidecar will now wait a while for the secrets store to be available rather than crashing immediately if it happens to come up first.
  • Traces missing the Sampled header will no longer be thrown out. This was always meant to be optional.

Upgrading

Apache Thrift

Apache Thrift does not support the event handler interface that older versions of FBThrift did. Baseplate's integration with the Thrift server is different as a result. See below for an example diff covering what needs to change.

--- a/reddit_service_activity/__init__.py
+++ b/reddit_service_activity/__init__.py
@@ -14,7 +14,7 @@
     tracing_client_from_config,
 )
 from baseplate.context.redis import RedisContextFactory
-from baseplate.integration.thrift import BaseplateProcessorEventHandler
+from baseplate.integration.thrift import baseplateify_processor
 
 from .activity_thrift import ActivityService, ttypes
 from .counter import ActivityCounter
@@ -48,7 +48,7 @@ def from_json(cls, value):
         )
 
 
-class Handler(ActivityService.ContextIface):
+class Handler(ActivityService.Iface):
     def __init__(self, counter):
         self.counter = counter
 
@@ -142,8 +142,5 @@ def make_processor(app_config):  # pragma: nocover
 
     counter = ActivityCounter(cfg.activity.window.total_seconds())
     handler = Handler(counter=counter)
-    processor = ActivityService.ContextProcessor(handler)
-    event_handler = BaseplateProcessorEventHandler(logger, baseplate)
-    processor.setEventHandler(event_handler)
-
-    return processor
+    processor = ActivityService.Processor(handler)
+    return baseplateify_processor(processor, logger, baseplate)
diff --git a/requirements.txt b/requirements.txt
index 5f0ab24..bd3baf9 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,5 +1,5 @@
-baseplate==0.28.0
+baseplate==0.29.0

@@ -18,5 +18,5 @@ raven==5.27.0
-Thrift==0.1
+thrift==0.12.1

Additionally, the Thrift compiler for Apache Thrift is called thrift rather than thrift1. If you're using the compiler directly you'll need to update this. Baseplate's built in thriftfile compilation steps handle this automatically.

This new updated compiler has a few differences which your thrift IDL and application code will need to take into account:

  • The float type in FBThrift isn't available in Apache Thrift, only the larger double type is. Unfortunately, this is a breaking change on the wire as the two types have quite different byte representations due to their different sizes. For an actively used field, you can make a new double-typed field and have your application populate or read both the float and double fields. Once all clients are using the new field you can drop the old one and then move to the new Baseplate.
  • Optional arguments to RPC methods do not get a default =None in the generated code anymore. Clients will need to ensure they're passing values for all parameters.
  • A list of keywords from various languages (e.g. next) is now blacklisted for use in field names in Thrift. If you have any fields with names like this, the new compiler will balk. Thankfully this is a purely code-side change and has no effect on how things look on the wire so you can just update your code without worrying about clients.
Assets 2