Commits on Jul 7, 2016
  1. @vkostyukov

    csl: Release CSL libraries

    Problem / Solution
    
    Finagle 6.36
    Util 6.35
    Ostrich 9.19
    TwitterServer 1.21
    Finatra 2.2.0
    Scrooge 4.8.0
    
    RB_ID=849873
    vkostyukov committed with jenkins Jul 7, 2016
  2. @cacoco

    Twitter OSS: Add ISSUE_TEMPLATE

    Problem
    
    Open source contributors don't have a standardized way of including information we
    would like to be included when contributors open issues.
    
    Solution
    
    Create an ISSUE template for opening issues. The template is useful for customizing
    and standardizing information we'd like to have contributors include when opening
    an issue against a library.
    
    Result
    
    More help when creating issues in our Twitter OSS repositories.
    
    RB_ID=849239
    cacoco committed with jenkins Jul 6, 2016
Commits on Jul 6, 2016
  1. @yzhuang

    util-app: Fix bug parsing Java Float/Double flags

    RB_ID=848646
    yzhuang committed with jenkins Jul 5, 2016
Commits on Jun 29, 2016
  1. @mgodave

    util-core: Add merge operation to AsyncStream

    Problem
    
    Partitions in a pub-sub messaging system map naturally to individual infinite
    AsyncStreams. Furthermore, representing the partitions as a single stream and
    applying a form of back pressure (using mapConcurrent) on consumption of that
    stream and it's partitions is a clear and simple programming model.  Since these
    partitions are infinite and no current AsyncStream function matches this usage
    it became necessary to invent one.
    
    Solution
    
    A function for merging arbitrary, and possibly infinite, AsyncStreams
    
    Create a function merge, given a sequence of AsyncStreams, that returns the
    interleaving of events from it's inputs as they become available.
    
    RB_ID=846681
    mgodave committed with jenkins Jun 28, 2016
  2. @kevinoliver

    finagle-core: Prevent HashedWheelTimer.Default from being stopped

    Problem
    
    Finagle has an expectation that `HashedWheelTimer.Default` is always
    running. However we don't prevent users from shutting it down.
    
    Solution
    
    Make `HashedWheelTimer.Default` have a wrapper that prevents `stop()`
    and includes logging to help users track down the caller.
    
    RB_ID=846194
    kevinoliver committed with jenkins Jun 28, 2016
Commits on Jun 27, 2016
  1. @mjeffryes

    util-core: Memoize should not deadlock reentrant threads

    Problem:
    
    In order to ensure its underlying function is only called once for a given
    input, Memoize.apply blocks concurrent invocations behind a CountDownLatch until
    the first invoc ation completes. Under recursion, it's possible that the same
    thread might re-enter Memoize apply with the same input parameters and block on
    it's own latch. This effectively converts a bug that would have resulted in a
    StackOverflow into a hung thread.
    
    Solution:
    
    Record the thread id of the thread that created the Latch and throw an exception
    if the same thread tries to await on that latch.
    
    Result:
    
    Recursive invocations of Memoize's apply with identical input throws an
    IllegalStateError.
    
    RB_ID=845702
    mjeffryes committed with jenkins Jun 23, 2016
  2. @instanceofme

    Eval: add Serializable to wrapping class

    Allows the compiled code to be executed remotely, e.g. using Apache Spark.
    
    Another option would be to make `wrapCodeInClass` (and `codeWrapperLineOffset`) `protected` instead of `private[this]`.
    
    Signed-off-by: Ruben Oanta <roanta@twitter.com>
    
    RB_ID=844794
    instanceofme committed with jenkins Jun 22, 2016
  3. util-core: Add `Future.by(Time)`, complementary to `Future.within(Dur…

    …ation)`
    
    RB_ID=843298
    Joseph Boyd committed with jenkins Jun 20, 2016
Commits on Jun 20, 2016
  1. @ryanoneill

    util: Fix sbt Build for util-security

    Problem
    
    util-security is missing a dependency on util-logging in the sbt
    build file.
    
    Solution
    
    Add the dependency.
    
    RB_ID=843708
    ryanoneill committed with jenkins Jun 16, 2016
  2. @kevinoliver

    twitter-server: Add filtering to /admin/registry.json

    Problem
    
    There are times you are interested in a very specific subsection of
    the registry and you don't feel like using `jq` or parsing the output.
    
    Solution
    
    Introduce an optional HTTP request parameter `filter` allows for
    simple filtering of the returned data.  The value takes the form
    "path/to/data" and returns registry entries whose prefix matches the
    parameter. A glob, `*`, can be used to match any component of the
    path. For example, using "filter=registry/server/http" will output all
    Finagle Http servers.  Using "filter=registry/client/\*/user_service"
    would output all Finagle clients with a label of "user_service"
    regardless of protocol.
    
    RB_ID=842784
    kevinoliver committed with jenkins Jun 16, 2016
  3. @ryanoneill

    util-security: Add X509CertificateLoader and Introduce util-security

    Problem
    
    Java does not provide a simple way to load X509Certificates given a path
    on the filesytem.
    
    Solution
    
    Create X509CertificateLoader to easily load X509Certificates. Also,
    introduce the util-security project as a place to keep security related
    traits, classes, and objects.
    
    RB_ID=843070
    ryanoneill committed with jenkins Jun 16, 2016
  4. @mosesn

    util-core: Shave an alloc in Once

    Problem / Solution
    
    We can use this One Weird Trick to shave an alloc in Once
    
    RB_ID=843159
    mosesn committed with jenkins Jun 16, 2016
  5. Updating code ownership

    RB_ID=841907
    Dan Benediktson committed with jenkins Jun 15, 2016
  6. @mosesn

    util-core: Clarify Once behavior when we throw an exception

    RB_ID=842665
    TBR=true
    mosesn committed with jenkins Jun 15, 2016
  7. @mosesn

    util-core: Unbreak Once

    Problem
    
    Once is not actually synchronized, because synchronizing the apply
    method doesn't work for Functions (see
    https://issues.scala-lang.org/browse/SI-9814).
    
    Solution
    
    Use instance synchronization instead of method synchronization.
    
    RB_ID=842245
    mosesn committed with jenkins Jun 14, 2016
  8. util/util-core: Fix some unused import warnings in tests, change to u…

    …se the non-deprecated junit package
    
    RB_ID=842337
    Joseph Boyd committed with jenkins Jun 14, 2016
  9. util-stats: Fix typo in docs

    RB_ID=842076
    Yaohua Zhuo committed with jenkins Jun 14, 2016
Commits on Jun 13, 2016
  1. source: upgrade jackson to 2.6.5

    RB_ID=836819
    Matthew Bilotti committed with jenkins Jun 10, 2016
Commits on Jun 9, 2016
  1. @ryanoneill

    twitter-server: Exclude Sharded Duplicate Clients and Make Rules Test…

    …able
    
    Problem
    
    Lint rules in their current form are not testable. This makes adding new
    ones and modifying existing ones more difficult than it should be.
    Additionally, memcache clients should not show up in linting rules as
    duplicates, due to the commonality of sharded clients.
    
    Solution
    
    Make the existing set of lint rules in Twitter Server be testable.
    Modify the duplicate registry rule to exclude memcache.
    
    RB_ID=840753
    ryanoneill committed with jenkins Jun 9, 2016
Commits on Jun 8, 2016
  1. @cacoco

    util, ostrich, scrooge, finagle, twitter-server: Update to use codeco…

    …v.io
    
    Problem
    
    We'd like to update to different code coverage tooling.
    
    Solution
    
    Update the projects travis.yml and plugins.sbt files accordingly.
    
    Result
    
    New code coverage instrumentation.
    
    RB_ID=840597
    cacoco committed with jenkins Jun 8, 2016
  2. @cacoco

    util: Update to cross-compile for scala 2.12

    Problem
    
    We want to be able to cross-compile for scala 2.12 but scoverage
    isn't ready yet.
    
    Based on this:
    efcc8a3
    
    Which was reverted here:
    8e7e04d
    
    Solution
    
    Make scoverage resolve 2.11 artifacts when building for scala
    2.12 until 2.12 versions are available.
    
    Result
    
    We should be able to cross-build and this change doesn't appear
    to break any downstream projects.
    
    RB_ID=840265
    cacoco committed with jenkins Jun 7, 2016
  3. @vkostyukov

    util-core: Introduce AsyncMeter.perSecondLimited for limiting on exte…

    …rnal QPS
    
    Problem
    
    `AsyncMeter.perSecond(n, m)` can easily produce more than `n` permits per second because
    of the way its implemented. There is quite a useful trick to make sure we don't exceed
    the given value, but it's hard to figure this out w/o knowing the implementation details.
    
    Solution
    
    Introduce `AsyncMeter.perSecondLimited` an alias for `newMeter(1, 1.seconds / permits)`
    that might be used for limiting on an external resource that has a strict QPS requirement.
    Note that `perSecondLimited` may produce `permits + 1` over the very first second, but
    it should be smoothed back to `permits` after that.
    
    RB_ID=837752
    vkostyukov committed with jenkins Jun 7, 2016
Commits on Jun 7, 2016
  1. @cacoco

    util: Revert commit "Cross build for Scala 2.12"

    Problem
    
    This commit causes downstream projects to fail as the autoplugin
    mechanism for the scoverage plugin is wreaking havoc when downstream
    projects: ostrich, scrooge and finatra try to run their tests.
    FileNotFoundExceptions are generated (the scoverage Invoker is getting
    run during tests which fails due to no coverage data being available at
    that time). An example failure:
    https://s3.amazonaws.com/archive.travis-ci.org/jobs/135689956/log.txt
    
    Additionally upgrading to scoverage plugin version 1.3.5 does not work
    with TravisCI because of these issues:
    scoverage/sbt-scoverage#161
    scoverage/sbt-scoverage#146
    
    Which caused us to back out of upgrading to scoverage 1.3.5 previously.
    
    The odd thing is that util built fine with these changes, unfortunately
    the downstream projects did not.
    
    Solution
    
    Revert commit, RB 837525 and update the .travis.yml.
    
    Result
    
    Downstream projects no longer fail to build.
    
    RB_ID=839869
    cacoco committed with jenkins Jun 7, 2016
Commits on Jun 6, 2016
  1. @vkostyukov

    util-core: Fewer allocations for AsyncSemaphore

    Problem / Solution
    
    Quite an easy win in terms of allocations in `AsyncSemaphore` is to reuse
    a `Permit` instance, not build a new one every time it's needed.
    
    This change doesn't affect single-threaded running time so dramatically
    (it's hard to fill-up a heap and cause GC), but shows significant improvement
    (~30% running time, ~15% allocations) under a high contention.
    
    The new benchamark uses 1 for initial permits simulating our serial dispatchers.
    
    Before / After (1 thread)
    
    [info] acquireAndRelease     avgt   10  188.540 ±  32.454   ns/op
    [info] acquireAndRelease     avgt   10  185.416 ±  33.937   ns/op
    
    [info] acquireAndRelease:·gc.alloc.rate.norm    avgt   10  224.000 ±   0.001    B/op
    [info] acquireAndRelease:·gc.alloc.rate.norm    avgt   10  176.000 ±   0.001    B/op
    
    Before / After (10 threads)
    
    [info] acquireAndRelease     avgt   10  10738.301 ± 3063.429   ns/op
    [info] acquireAndRelease     avgt   10   7046.517 ± 1461.242   ns/op
    
    [info] acquireAndRelease:·gc.alloc.rate.norm    avgt   10   319.951 ±   20.883    B/op
    [info] acquireAndRelease:·gc.alloc.rate.norm    avgt   10   257.192 ±   19.268    B/op
    
    RB_ID=839348
    vkostyukov committed with jenkins Jun 6, 2016
  2. @mosesn

    util-codec: Stop depending on apache commons codec

    Problem
    
    commons-codec is a pretty big dependency, and now that we're on
    jdk8, we don't need it!
    
    Solution
    
    rm it
    
    Result
    
    Smaller dependency tree!
    
    RB_ID=833478
    TBR=true
    mosesn committed with jenkins Jun 5, 2016
  3. @fwbrasil

    I'm working on improving the performance of an internal system and, d…

    …uring the profiling sessions, I've identified a few potential optimizations of `Future` and `Promise`.
    
    - cache functions if possible
    - use `iterator` + `while` instead of `map`/`foreach`/`filterNot`
    - avoid tuple allocations (`zipWithIndex`) in `Future.collect`
    - cache empty collection futures (`emptySeq`/`emptyMap`)
    - use `ev` directly in `future.flatten` to avoid the function allocation
    - don't use the `ArrayBuffer`'s default size of 16 in `Promise.runq`
    
    RB_ID=838527
    fwbrasil committed with jenkins Jun 3, 2016
  4. @jeffreyolchovy

    Cross build for Scala 2.12

    Problem
    
    There exists a need to publish artifacts that are
    binary compatible with Scala 2.12.
    
    Solution
    
    - Modify sbt build defintion to cross build for 2.12.
    - Update scalatest and scalacheck libraries to versions
    that support 2.12.
    
    Signed-off-by: Moses Nakamura <mnakamura@twitter.com>
    
    RB_ID=837525
    jeffreyolchovy committed with jenkins Jun 2, 2016
  5. @vkostyukov

    util-core: Add Future.joinWith

    Problem
    
    `Future.join` is a popular way of joining two futures in-parallel. Although,
    in 99% of the cases, an intermediate tuple is mapped a domain type / case class
    so it often looks as: `a.join(b).map { case (a, b) => .... }`.
    
    Solution
    
    Introducing `Future.joinWith` that also accepts a function for mering the joined
    results, allows save allocations on a tuple as well as provides a reasonable
    API allowing users to write less code.
    
    Credits
    
    This work is inspired by Scala's `Future.zipWith` featured by Viktor Klang at his
    SD NYC talk.
    
    RB_ID=838169
    vkostyukov committed with jenkins Jun 2, 2016
Commits on Jun 1, 2016
  1. @kevinoliver

    util-core: Remove deprecated Future.isReturn and isThrow

    Problem
    
    `c.t.util.Future.isReturn` and `Future.isThrow` are deprecated because
    they make it a little too easy to do a blocking operation.
    
    Solution
    
    Remove the methods as users can use the more explicit
    `c.t.util.Await.result(Future.liftToTry).isReturn` and
    `c.t.util.Await.result(Future.liftToTry).isThrow`.
    
    RB_ID=837329
    TBR=true
    kevinoliver committed with jenkins Jun 1, 2016
  2. @vkostyukov

    source: NoStacktrace is dead. Long live NoStackTrace

    Problem / Solution
    
    Both Finagle and Util have their own versions of `NoStacktrace`, which are
    basically copies of the one provided by Scala's standard library. Let's use
    what's already out there and reduce both the amount of code we need to maintain
    and confusion around three different options. Also, Scala's implementation
    supports re-enabling stacktraces via a system-wide option, which might be
    useful for debugging.
    
    RB_ID=833188
    TBR=true
    NO_USER_HOOK=1
    vkostyukov committed with jenkins Jun 1, 2016
  3. @mosesn

    util-core: Fixes AsyncMeter behavior with small burst sizes and small…

    … periods
    
    Problem
    
    AsyncMeter's process is that it fills up the token bucket, and then
    uses the token bucket to give out permission to continue.  The
    tricky thing here is that if the token bucket can only hold 1, then
    even if it's given permission to hold 2, it won't distribute load
    for two.  Since we don't have arbitrary granularity timers, folks
    who select a maximum burst size of 1, but a frequency smaller than
    the granularity our timers can actually handle will not see the
    throughput they expect.
    
    Although we could simulate the right result by choosing a new burst
    size that better fits the timer's granularity, this introduces the
    problem of figuring out the timer's granularity, which can fluctuate
    given load.
    
    Solution
    
    When the meter is refilled, it allows the full disbursement of
    tokens to be used, instead of just what fits in the token bucket.
    
    RB_ID=836742
    mosesn committed with jenkins May 31, 2016
Commits on May 30, 2016
  1. @mosesn

    csl: Bump scalacheck to 1.12.5

    RB_ID=836280
    mosesn committed with jenkins May 26, 2016
  2. @kevinoliver

    util-core: Remove deprecated method Future.get(Duration)

    Problem
    
    `c.t.util.Future.get(Duration)` is deprecated because it makes it a
    little too easy to do a blocking operation.
    
    Solution
    
    Remove the method as users can use the more explicit
    `c.t.util.Await.result(Future.liftToTry, Duration)`.
    
    RB_ID=836066
    TBR=true
    kevinoliver committed with jenkins May 26, 2016
  3. @mosesn

    util-core: Elaborate in docs on failure in FutureTransformer

    Problem
    
    In the docs for FutureTransformer, we advise people to throw an
    exception, but it's not marked with the @throws annotation, so they
    can't throw an exception.
    
    Solution
    
    Throwing in a future callback isn't a best practice, so let's change
    the docs to make this explicit.
    
    RB_ID=835853
    mosesn committed with jenkins May 25, 2016
  4. @mikepii

    util-zk: Fix race condition in ZkAsyncSemaphore

    Problem
    
    Before acquiring a permit, ZkAsyncSemaphore gets the list of existing
    permits and then checks each permit ZNode’s data. This raised an error
    when one of the existing permits is released between the time the list
    was gotten and the data was checked. This was an issue for us because we
    are using ZkAsyncSemaphore in Hadoop containers. When many containers
    are closing while others are opening, many permits get released rapidly.
    
    Solution
    
    Change the shard permit consensus logic to count lost existing permits
    as having not been seen in the first place.
    
    Signed-off-by: Moses Nakamura <mnakamura@twitter.com>
    
    RB_ID=835856
    mikepii committed with jenkins May 25, 2016