Skip to content
Permalink
Branch: develop
Commits on Jul 23, 2019
  1. finagle-http: Optimize HttpMuxer.route

    kevinoliver authored and jenkins committed Jul 23, 2019
    Problem
    
    `c.t.f.http.HttpMuxer.route` could be faster.
    
    Solution
    
    Optimize it to avoid allocations and work.
    
    Result
    
    For exact path matches, 45% faster and 0 allocations vs 96 bytes/call.
    
    JIRA Issues: CSL-8520
    
    Differential Revision: https://phabricator.twitter.biz/D344258
  2. finagle-http2: Handle upgrade events properly in APCH

    bryce-anderson authored and jenkins committed Jul 23, 2019
    Problem / Solution
    
    The AdapterProxyChannelHandler is relying on a Netty but that was fixed
    sometime between version 4.1.35 and 4.1.37. We're not upgrading to 4.1.37
    but we should fix the behavior anyway.
    
    JIRA Issues: CSL-8436
    
    Differential Revision: https://phabricator.twitter.biz/D336428
Commits on Jul 22, 2019
  1. finagle-http: Optimize HttpMuxer.normalize

    kevinoliver authored and jenkins committed Jul 22, 2019
    Problem
    
    `c.t.f.http.HttpMuxer.normalize` is slower than it could be.
    
    Solution
    
    Optimize it.
    
    Result
    
    Significantly faster and far less allocations. The common case when
    the path is already normalized is 96% faster and has 0 allocations.
    
    JIRA Issues: CSL-8520
    
    Differential Revision: https://phabricator.twitter.biz/D343261
  2. finagle-core: Split the ClientDispatcher.scala file into parts

    bryce-anderson authored and jenkins committed Jul 22, 2019
    Problem
    
    We have a lot of different things going on in ClientDispatcher.scala that
    don't need to be in the same file.
    
    Solution
    
    Split it into multiple files.
    
    Differential Revision: https://phabricator.twitter.biz/D342883
Commits on Jul 19, 2019
  1. Twitter-oss: update OSS libraries post-release to 19.8.0-SNAPSHOT

    dotordogh authored and jenkins committed Jul 19, 2019
    Problem
    
    We want to update to the next SNAPSHOT version of our Twitter OSS libraries: 19.8.0-SNAPSHOT
     - util
     - scrooge
     - finagle
     - twitter-server
     - finatra
    
    Solution
    
    Prepare libraries for their next SNAPSHOT version.
    
    Differential Revision: https://phabricator.twitter.biz/D342926
Commits on Jul 18, 2019
  1. finagle: Correct Streaming Metric Name in Doc

    jyanJing authored and jenkins committed Jul 18, 2019
    Problem
    We did not specify the `http` prefix on some streaming metrics in
    go/finaglemetrics
    
    Solution
    Add the `http` prefix in the metrics description
    
    JIRA Issues: CSL-8508
    
    Differential Revision: https://phabricator.twitter.biz/D342374
Commits on Jul 17, 2019
  1. Twitter-oss: Prepare OSS libraries for release 19.7.0

    roanta authored and jenkins committed Jul 17, 2019
    Problem
    
    We want to release the next versions of our Twitter OSS libraries 19.7.0
     - util
     - scrooge
     - finagle
     - twitter-server
     - finatra
    
    Solution
    
    Prepare libraries for their next releases.
    
    Differential Revision: https://phabricator.twitter.biz/D342141
Commits on Jul 16, 2019
  1. finagle-core: Remove synchronization from CachingPool

    kevinoliver authored and jenkins committed Jul 16, 2019
    Problem
    
    High throughput clients see lock contention in `CachingPool`.
    
    Solution
    
    Change the data structure used into one that does not need
    synchronization.
    
    JIRA Issues: CSL-7979
    
    Differential Revision: https://phabricator.twitter.biz/D339657
Commits on Jul 15, 2019
  1. finagle-http: Move the creation of the endpointer out of the Http object

    bryce-anderson authored and jenkins committed Jul 15, 2019
    Problem
    
    The code in the Http.scala file is getting pretty hairy, making it tough
    to make changes that may benefit HTTP/2.
    
    Solution
    
    Change the http client implementation configuration to be based on
    providing an endpointer instead of transporter oriented tooling and
    put this all in a new object, ClientEndpointer.
    
    Differential Revision: https://phabricator.twitter.biz/D337136
Commits on Jul 12, 2019
  1. finagle-exp: add ConcurrencyLimitFilter

    tigerlily-he authored and jenkins committed Jul 12, 2019
    Problem
    
    Concurrency limits are statically set by users. This leads to great variation
    and suboptimal concurrency limits.
    
    Solution
    
    This change adds ConcurrencyLimitFilter. Filter depends on Netflix
    concurrency-limits library, which provides algorithms for calculating
    concurrency limit based off the round trip time of requests.
    
    Result
    
    The new filter continuously estimates concurrency limit and rejects incoming
    requests if the service has reached its concurrency limit.
    
    JIRA Issues: CSL-8302, CSL-8303
    
    Differential Revision: https://phabricator.twitter.biz/D328815
  2. finagle-core: Remove extra comma for sbt compile

    yufangong authored and jenkins committed Jul 12, 2019
    Problem/Solution
    
    The trailing comma failed sbt scala 2.11 compile.
    Remove it.
    
    Differential Revision: https://phabricator.twitter.biz/D339825
  3. finagle-core: OffloadFilter flag to reduce network contention

    fwbrasil authored and jenkins committed Jul 11, 2019
    **Problem**
    
    We simulated a scenario when the service is close to its redline and profiled it with vtune. One of the top methods is a lock queue used for networking (`queued_spin_lock_slowpath`)
    
    **Solution**
    
    The contention seemed to be due to finagle applications reusing the netty event loop to run application code. Vladimir mentioned the new `OffloadFilter` that was created to deal with applications that do blocking operations and we decided to re-use it to test the hypothesis. The results are promising. After the change to use the offload filter with a fixed thread pool by default, there is a significant reduction in contention
    
    **Notes**
    
    - Looking at the [[ https://t.co/Ls5YZYpX2n?amp=1 | source code ]] of the system call, we noticed that the contention seems to happen when there are concurrent flushes and that the lock happens at the level of the network device. Our theory is that having fewer threads allocated to the event loop reduces the concurrency and makes the mux batching more effective, reducing the rate of flushes. It's possible that applications running with this configuration will become better neighbors when running in a shared aurora host since the lock happens at the network device level.
    
    - Additionally, with this new configuration the netty threads are always available to handle channel events, helping to reduce the latency. Since a channel is always pinned to a specific thread by netty, without the offload filter channels eventually have to wait for the thread to finish executing the application code to have their network events handled.
    
    - I haven't created any tests since the stack parameter is initialized statically. Please let me know if you think of a way to unit test the change.
    
    Differential Revision: https://phabricator.twitter.biz/D331502
Commits on Jul 11, 2019
  1. finagle-http: PayloadSizeFilter now measures streaming messages

    yufangong authored and jenkins committed Jul 11, 2019
    Problem
    
    We used to pass by streaming messages (isChunked) in PayloadSizeFilter.
    
    Solution
    
    Fork the filter in HTTP and measure chunked messages. Wrap the reader with an
    onRead callback which measures the payload size. Return a request proxy wrapping
    the reader we have callback there.
    
    JIRA Issues: CSL-8134
    
    Differential Revision: https://phabricator.twitter.biz/D337877
Commits on Jul 10, 2019
  1. util: Default to bypassing scheduler in Future.map

    luciferous authored and jenkins committed Jul 10, 2019
    Problem / Solution
    
    Bypassing the scheduler in ConstFuture's map is toggled on (via
    c.t.u.BypassScheduler) by default. This has been in production for over
    a quarter with minimal issue. Removing the toggle so that we always
    bypass the scheduler from now on.
    
    JIRA Issues: CSL-7806
    
    Differential Revision: https://phabricator.twitter.biz/D338487
Commits on Jul 8, 2019
  1. finagle-http2: Stop relying on synthetic connection headers to close …

    bryce-anderson authored and jenkins committed Jul 8, 2019
    …h2 client streams
    
    Problem
    
    We use a `Connection: close` header to trigger a stream channel to
    close, but this will bubble up to the application.
    
    Solution
    
    Introduce `Http2Transport` that it knows that it's a single-dispatch
    construct and manages state accordingly.
    
    JIRA Issues: CSL-7931
    
    Differential Revision: https://phabricator.twitter.biz/D333564
  2. finagle-core: Allow to use SOCKS proxy for localhost

    gpoulin authored and jenkins committed Jul 8, 2019
    Problem
    
    SOCKS proxy is bypassed when connecting to localhost and there's no way to
    override this behaviour. Using a SOCKS proxy when connecting to localhost can be
    useful to easily capture communication between local services for debugging.
    
    Solution
    
    Add a flags to enable SOCKS proxy for localhost.
    
    Result
    
    You can now force the use of SOCKS proxy for localhost with
    -com.twitter.finagle.socks.socksProxyForLocalhost. Without this flag, the
    behaviour of bypassing the SOCKS proxy is kept.
    
    Signed-off-by: Moses Nakamura <mnakamura@twitter.com>
    
    JIRA Issues: CSL-8450
    
    Differential Revision: https://phabricator.twitter.biz/D337073
Commits on Jul 3, 2019
  1. finagle-http2: Remove gauges when a H2 client sessions closes

    bryce-anderson authored and jenkins committed Jul 3, 2019
    Problem / Solution
    
    We have some H2 client gauges that should be explicitly removed when we
    close the session so lets just close them.
    
    Differential Revision: https://phabricator.twitter.biz/D336589
Commits on Jul 2, 2019
  1. finagle-core: Make details of SingletonPool private

    bryce-anderson authored and jenkins committed Jul 2, 2019
    Problem / Solution
    
    We don't need to leak the state machine of the c.t.f.pool.SingletonPool
    so let's make it private.
    
    Differential Revision: https://phabricator.twitter.biz/D335782
Commits on Jul 1, 2019
  1. finagle-http: reduce overhead of creating Http Dispatchers

    enbnt authored and jenkins committed Jul 1, 2019
    Problem
    
    There is a large overhead in creating new HTTP/2
    streams. The Dispatchers that get created
    will generate new Loggers and  StatsReceivers,
    per instance - in HTTP/2 these are created per
    Request/Response stream.
    
    Solution
    
    Move Loggers to companion objects, create and
    re-use a single StatsReceiver for client and
    server dispatchers, and modify dispatchers to
    use a `ExceptionStatsHandler` for rolling up
    errors, instead of allocating new scoped
    `StatsReceiver`s per instance.
    
    Result
    
    Fewer allocations for all HTTP clients and Servers.
    
    JIRA Issues: CSL-8402
    
    Differential Revision: https://phabricator.twitter.biz/D335114
  2. finagle-http2: MultiplexCodec stream pipelines configure backpressure…

    bryce-anderson authored and jenkins committed Jul 1, 2019
    … correctly
    
    Problem
    
    We don't currently support the configured mode of backpressure in H2
    stream channels, instead just enabling it regardless of what
    configuration the params have.
    
    Solution
    
    Honor the backpressure settings specified in the params in the
    MultiplexCodec stream channel bootstrap.
    
    JIRA Issues: CSL-8420
    
    Differential Revision: https://phabricator.twitter.biz/D334268
Commits on Jun 28, 2019
  1. finagle-base-http: Add support for new "b3" tracing header

    crispywalrus authored and jenkins committed Jun 28, 2019
    Problem
    
    OpenZipkin has created a new set of header to carry
    trace information. These headers are not compatible
    with the old ones and are needed to interoperate
    with services that use the new version of the header.
    
    Solution
    
    If a "b3" header exists in the request this extracts
    all the new information and populates finagles
    tracing header with the existing values.
    
    #749
    Signed-off-by: Ian Bennett <ibennett@twitter.com>
    
    Differential Revision: https://phabricator.twitter.biz/D334419
Commits on Jun 27, 2019
  1. finagle-core: Remove SslClientEngineFactory.getHostname

    ryanoneill authored and jenkins committed Jun 27, 2019
    Problem
    
    The use of `SslClientEngineFactory.getHostname` is highly discouraged
    because it uses the `getHostName` method of the passed in `InetSocketAddress`
    if the hostname within the `SslClientConfiguration` is None. The `getHostName`
    method of `InetSocketAddress` performs a reverse DNS lookup when provided with
    an IP address, which is undesriable.
    
    Solution
    
    Remove `SslClientEngineFactory.getHostname` and encourage folks to use
    `SslClientEngineFactory.getHostString` instead.
    
    JIRA Issues: CSL-8416
    
    Differential Revision: https://phabricator.twitter.biz/D334087
  2. finagle-http: Remove Use of request.response from ExceptionFilter

    ryanoneill authored and jenkins committed Jun 27, 2019
    Problem / Solution
    
    Use of 'request.response' within Finagle is an anti-pattern. Let's
    remove its use from finagle-http's `ExceptionFilter`.
    
    JIRA Issues: CSL-8391
    
    Differential Revision: https://phabricator.twitter.biz/D333509
Commits on Jun 26, 2019
  1. finagle/twitter-server: Add admin endpoint for info about clients con…

    mgodave authored and jenkins committed Jun 26, 2019
    …nected to servers
    
    Problem
    
    There is no way to get information about clients currently connected to a
    finagle server.
    
    Solution/Result
    
    Add an admin endpoint to list basic information about each client connected, as
    well as information about the connection's encryption status, for each listening
    server.
    
    JIRA Issues: CSL-8104
    
    Differential Revision: https://phabricator.twitter.biz/D329940
  2. finagle-core: Explicitly remove some gauges

    bryce-anderson authored and jenkins committed Jun 26, 2019
    Problem / Solution
    
    Gauges can be cleaned up by the GC but that can be costly since we
    cannot know when a GC cycle will occur. It's better to explicitly remove
    them as part of the life cycle of the object we're instrumenting.
    
    Differential Revision: https://phabricator.twitter.biz/D333057
  3. finagle-http: Remove Use of request.response from Cors

    ryanoneill authored and jenkins committed Jun 26, 2019
    Problem / Solution
    
    Use of 'request.response' is an anti-pattern in finagle-http. Let's
    remove its use in `com.twitter.finagle.http.Cors`.
    
    JIRA Issues: CSL-8389
    
    Differential Revision: https://phabricator.twitter.biz/D332765
Commits on Jun 25, 2019
  1. finagle-thriftmux: Remove dead method

    bryce-anderson authored and jenkins committed Jun 25, 2019
    Problem / Solution
    
    We have some dead code in VanillaThriftSession, so lets just get rid of
    it.
    
    Differential Revision: https://phabricator.twitter.biz/D333072
  2. finagle-base-http: Remove Use of request.response in DelayedReleaseSe…

    ryanoneill authored and jenkins committed Jun 25, 2019
    …rviceTest
    
    Problem / Solution
    
    request.response is an anti-pattern. Stop using it in `DelayedReleaseServiceTest`.
    
    JIRA Issues: CSL-8388
    
    Differential Revision: https://phabricator.twitter.biz/D332619
  3. finagle: Rewrite FlatSpec Tests as FunSuite

    ryanoneill authored and jenkins committed Jun 25, 2019
    Problem / Solution
    
    For Finagle, FunSuite is the recommended testing approach when
    using Scalatest. Let's rewrite the tests that are still using
    FlatSpec to use FunSuite instead.
    
    JIRA Issues: CSL-8383
    
    Differential Revision: https://phabricator.twitter.biz/D332802
Commits on Jun 24, 2019
  1. finagle-http2: Don't honor the Netty x-http2-stream-dependency-id header

    bryce-anderson authored and jenkins committed Jun 24, 2019
    Problem
    
    We don't provide an API for discovering stream id's so it's not possible
    for the users to set the relative priorities.
    
    Solution
    
    Don't honor the header.
    
    JIRA Issues: CSL-7930
    
    Differential Revision: https://phabricator.twitter.biz/D332191
  2. finagle-core: Remove all ServerBuilder#build deprecated methods

    ryanoneill authored and jenkins committed Jun 24, 2019
    There are multiple 'build' methods on `ServerBuilder` that have
    been deprecated for many years. Let's remove them.
    
    JIRA Issues: CSL-8337
    
    Differential Revision: https://phabricator.twitter.biz/D331011
  3. finagle-http2: Stop generating an unnecessary UnsupportedOperationExc…

    bryce-anderson authored and jenkins committed Jun 24, 2019
    …eption
    
    Problem
    
    We are calling `channel.connect` on stream channels in the h2
    `ClientSessionImpl` which is an unsupported operation. As such it
    generates an exception (which we promptly discard) which is a
    performance problem.
    
    Solution
    
    Don't call `connect` for new stream channels.
    
    JIRA Issues: CSL-7994
    
    Differential Revision: https://phabricator.twitter.biz/D332006
Commits on Jun 22, 2019
  1. finagle-core: ensure ClientDispatcher queueSize gauge is removed on t…

    enbnt authored and jenkins committed Jun 21, 2019
    …ransport close
    
    Problem
    
    The `GenSerialClientDispatcher` has a `queueSize` gauge
    that gets registered for every instance. The `HttpClientDispatcher`
    extends from this class, thus registering this gauge.
    For the HTTP/2 implementation, a `HttpClientDispatcher`
    is created for every new stream, increasing the number of
    registered gauges. As currently implemented, these gauges
    will be cleaned up upon GC.
    
    Solution
    
    Be a little more aggressive and remove the registered
    gauge on the close of the transport, instead of waiting
    for GC.
    
    Result
    
    Less memory pressure from gauges and less overhead
    during metrics collection.
    
    JIRA Issues: CSL-8378
    
    Differential Revision: https://phabricator.twitter.biz/D331923
Commits on Jun 21, 2019
  1. finagle-http: Stop Using Request.response in Tests

    ryanoneill authored and jenkins committed Jun 21, 2019
    Problem / Solution
    
    Using request.response is considered bad practice, because it
    can be incorrect. This is why it was deprecated in 2016. One
    day we would like to remove it, so let's remove its use in our
    own tests.
    
    JIRA Issues: CSL-8384
    
    Differential Revision: https://phabricator.twitter.biz/D331936
  2. Ensure read from files as bytes for Python 3-compatibility

    gregpoulos authored and jenkins committed Jun 21, 2019
    Use of the `includecode` directive was causing doc build failures in Python 3. The failures were related to differences in how Python 2 and 3 natively encode strings. The resolution is to enforce opening the file in binary mode within the extension.
    
    JIRA Issues: TECHDOCS-2451
    
    Differential Revision: https://phabricator.twitter.biz/D331358
Older
You can’t perform that action at this time.