Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
branch: master
Commits on Apr 10, 2015
  1. @viswanathgs

    Bump version to 35:0

    viswanathgs authored
  2. @nbronson @viswanathgs

    SharedMutex potential lost wakeup with exactly 3 or 4 contending writers

    nbronson authored viswanathgs committed
    Summary:
    SharedMutex used a saturating counter that records the number of
    waiting lock() calls, but an ABA problem on futexWait could lead to a lost
    wakeup when there was exactly 3 or 4 threads contending on the RW lock
    in W mode.  This diff changes the kWaitingE count to be heuristic (it is
    possible that the count says 1 but there are two waiters), saturates at
    2 instead of 3 (because there is no benefit from differentiating those
    two), and doesn't decrement the count on a successful wakeup.
    
    Also, I noticed while debugging this that boost::noncopyable was causing
    SharedMutex to be 8 bytes when it should only be 4.
    
    One way the wakeup could be lost in the old code:
    
    1. A calls lock()
    2. A updates state <- kHasE
    3. A returns
    4. B calls lock()
    5. B spins
    6. B updates state <- kHasE + 1 * kIncrWaitingE
    7. A calls unlock()
    8. A updates state <- 0
    9. A calls futexWake(), which returns 0
    10. A calls lock()
    11. A updates state <- kHasE
    12. A returns
    13. C calls lock()
    14. C spins
    15. C updates state <- kHasE + 1 * kIncrWaitingE
    16. C calls futexWait, expecting kHasE + 1 * kIncrWaitingE
    17. B calls futexWait, expecting kHasE + 1 * kIncrWaitingE
    18. A calls unlock()
    19. A updates state <- 0
    20. A calls futexWake(), which returns 1
    21. C receives the wakeup
    22. C updates state <- kHasE
    23. C returns
    24. C calls unlock()
    25. C updates state <- 0
    
    B missed the wakeup that was intended for it (sent at step 9, wait
    started at step 17), but went to sleep anyway because it saw the write
    state at step 17. Now there are two waiters but only 1 recorded in the
    SharedMutex, at which point failure is inevitable.
    
    Test Plan:
    1. DeterministicSchedule test using uniformSubset that can repro the problem
    2. Test in production scenario that produced occasional deadlocks under high stress
    
    Reviewed By: yfeldblum@fb.com
    
    Subscribers: folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1980210
    
    Tasks: 6720328
    
    Signature: t1:1980210:1428623932:ef1c00c3f88154578b2b253ac0cfdbadf9f31d8c
  3. @djwatson @viswanathgs

    Move StringHandler to its own file

    djwatson authored viswanathgs committed
    Summary: Move this from a test to its own file in codec folder
    
    Test Plan:
    already has a unittest
    fbconfig -r folly/wangle; fbmake runtests
    
    Reviewed By: hans@fb.com
    
    Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
    
    FB internal diff: D1959152
    
    Signature: t1:1959152:1427935371:9d1338986df97fd94f98ad2bf08c731d229bc2b1
  4. @djwatson @viswanathgs

    LineBasedFrameDecoder

    djwatson authored viswanathgs committed
    Summary: Copy of netty's line based decoder.
    
    Test Plan:
    unittests
    fbconfig folly/wangle/codec; fbmake runtests
    
    Reviewed By: hans@fb.com
    
    Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
    
    FB internal diff: D1959155
    
    Signature: t1:1959155:1427935150:e11280c5567df9ad9964dbb656aa090267856f57
  5. @djwatson @viswanathgs

    Pipeline OutputBufferingHandler

    djwatson authored viswanathgs committed
    Summary: Part 2 of pipeline.  Use OutputBufferingHandler, which is exactly the same output buffering thrift does currently.
    
    Test Plan:
    A couple canaries, unittests.
    Specific issues of previous pipeline diffs I think are fixed already and should be fine with this one:
    * Previously there were missing DestructorGuards.  I don't think anything in OutputBUfferHandler needs a DG
    * previously broke http GET handling.  There is a unittest for it now.
    
    Canary
    
    Reviewed By: alandau@fb.com
    
    Subscribers: doug, fugalh, alandau, bmatheny, mshneer, folly-diffs@, jsedgwick, yfeldblum, chalfant, vloh, andreib
    
    FB internal diff: D1970949
    
    Signature: t1:1970949:1428360621:a0db142032894525f69c2b144cf946d63f790fe1
  6. @luciang @viswanathgs

    folly: MemoryMapping: add offset/size params to advise

    luciang authored viswanathgs committed
    Test Plan: ran it
    
    Reviewed By: philipp@fb.com
    
    Subscribers: folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1974936
    
    Signature: t1:1974936:1428453023:1a83ed8336f75b745bffc633b8471231423b0fb0
  7. @luciang @viswanathgs

    folly: MemoryMapping: respect writable when opening the file

    luciang authored viswanathgs committed
    Summary: the file needs to be opened with write-permissions before it can be mmaped with PROT_WRITE.
    
    Test Plan: n/a
    
    Reviewed By: philipp@fb.com
    
    Subscribers: folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1974895
    
    Signature: t1:1974895:1428451664:1106019abf02b9de659ce0d305e3d788ae7d60bc
  8. @djwatson @viswanathgs

    AsyncSocketHandler cacheChainLength

    djwatson authored viswanathgs committed
    Summary:
    Is there any reason we shouldn't just cache the chain length everywhere?
    
    Currently the pipeline does a lot of passing of buffers between IOBufQueues, I wonder if we can get even smarter about passing the length around
    
    Test Plan: used in telnet server diff
    
    Reviewed By: hans@fb.com
    
    Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
    
    FB internal diff: D1959167
    
    Signature: t1:1959167:1427934568:eb9a49514b2bb7d157879bb8efa99f12092d5e95
  9. @andriigrynenko @viswanathgs

    Remove FiberManager move-constructor

    andriigrynenko authored viswanathgs committed
    Summary:
    LoopController and Fibers keep references to FiberManager, so implementing move-constructor for it is non-trivial.
    The only purpose of move constructor was to have a create() static method, replaced it with a constructor accepting a type tag.
    
    Test Plan: unit test + tao build
    
    Reviewed By: stepan@fb.com, pavlo@fb.com
    
    Subscribers: folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1971881
    
    Signature: t1:1971881:1428374073:945d913e69eaa6f957dace981c23835105d91935
  10. @djwatson @viswanathgs

    AsyncUDPServerSocket passes socket in callback

    djwatson authored viswanathgs committed
    Summary: AsyncUDPServerSocket doesn't make it easy to write to the same socket you read from.  Add the socket as a callback param, similar to AsyncServerSocket
    
    Test Plan:
    fbconfig -r folly; fbmake dbg
    
    Will fixup any other spots contbuild finds
    
    Reviewed By: hans@fb.com
    
    Subscribers: bmatheny, doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
    
    FB internal diff: D1948936
    
    Signature: t1:1948936:1427841651:20d13d73c06d31c75056624f051a6fd35b9701fb
  11. @viswanathgs

    Add ability to yield execution of the currently running fiber

    Brian Watling authored viswanathgs committed
    Summary: This diff allows fibers to explicitly yield execution to other fibers and the event loop
    
    Test Plan: unit tests
    
    Reviewed By: andrii@fb.com
    
    Subscribers: folly-diffs@, yfeldblum, chalfant, tao-eng@
    
    FB internal diff: D1965297
    
    Signature: t1:1965297:1428082686:e524e9dd21b3fb951e1d3556e4cb3eedc3e6511a
  12. @andriigrynenko @viswanathgs

    Fix open-source Makefiles after fibers were moved to folly

    andriigrynenko authored viswanathgs committed
    Test Plan: contbuild
    
    Reviewed By: pavlo@fb.com
    
    Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1966698
    
    Signature: t1:1966698:1428367351:62f2218c23d4911c15055917c0fbae31a2420903
  13. @fugalh @viswanathgs

    GlobalExecutor.h include Executor.h and IOExecutor.h

    fugalh authored viswanathgs committed
    Summary: We were forward-declaring `Executor` and `IOExecutor` for great compilation speed. However, this leaves users of this header to include at least one of those headers in their cpp file to actually use it, and that's kind of annoying. `Executor` and `IOExecutor` are not templated and are quite small, so it seems like the compilation speed benefit is minimal compared to the convenience factor, so this diff just includes the two headers. (Probably including `Executor.h` is redundant since `IOExecutor.h` probably includes it, but for completeness and explicitness, and in case `IOExecutor` also just forward-declares `Executor` now or in the future.)
    
    Test Plan: stuff still builds
    
    Reviewed By: davejwatson@fb.com
    
    Subscribers: fugalh, exa, folly-diffs@, jsedgwick, yfeldblum, chalfant
    
    FB internal diff: D1970179
    
    Signature: t1:1970179:1428351076:e9c3700a868688035e7ff50486b8cead751d3c02
  14. @andriigrynenko @viswanathgs

    Add FiberManager local type

    andriigrynenko authored viswanathgs committed
    Summary: This adds local type for each FiberManager. Only local of given type can be created on fibers of this FiberManager. Locals of other types will be just treated as regualar thread-locals.
    
    Test Plan: unit test
    
    Reviewed By: bwatling@fb.com
    
    Subscribers: folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1969739
    
    Signature: t1:1969739:1428345931:aff5deb526c179158e5881b29330ff3f6698149a
  15. @pavlo-fb @viswanathgs

    Option to record precise stack on every N fiber

    pavlo-fb authored viswanathgs committed
    Summary: title + some logging around
    
    Test Plan: mcrouter unit tests
    
    Reviewed By: andrii@fb.com
    
    Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1965097
    
    Signature: t1:1965097:1428103637:341ff705c9d11c730197828256ec0b790ee7afaa
  16. @djwatson @viswanathgs

    shared ptr vector sockets

    djwatson authored viswanathgs committed
    Summary: promote the sockets vector to a shared_ptr, since both ServerWorkerPool and ServerBootstrap use it.  Otherwise there are destruction order issues between ServerBootstrap and any IOThreadPoolExecutor you use
    
    Test Plan: Saw use after free in D1942242, gone after this.
    
    Reviewed By: yfeldblum@fb.com
    
    Subscribers: chalfant, doug, fugalh, folly-diffs@, jsedgwick, yfeldblum
    
    FB internal diff: D1947553
    
    Signature: t1:1947553:1427484417:5b78f5c9c70d244d3f52a6f71b6d1fab7b29d106
  17. @andriigrynenko @viswanathgs

    Use Fiber locals for TAO Fiber-locals

    andriigrynenko authored viswanathgs committed
    Summary:
    This simplifies TAO fibers locals, using FiberManager based fiber-local storage.
    This depends on D1958135.
    
    Test Plan: integration tests
    
    Reviewed By: bwatling@fb.com
    
    Subscribers: alikhtarov
    
    FB internal diff: D1962214
    
    Signature: t1:1962214:1427997755:b546b9039382a7600f234b0a2a60cc96da34e662
  18. @andriigrynenko @viswanathgs

    Fiber-local context

    andriigrynenko authored viswanathgs committed
    Summary: This adds fiber-local context, which behaves more like static objects+fork rather than thread-locals.
    
    Test Plan: unit test
    
    Reviewed By: pavlo@fb.com, bwatling@fb.com
    
    Subscribers: rushix, alikhtarov, bwatling
    
    FB internal diff: D1958135
    
    Signature: t1:1958135:1427999426:8e4b89f4af53a1a119b2e5a765fb549dd8442c50
  19. @viswanathgs

    Add static method to skip SSL init

    Joel Goguen authored viswanathgs committed
    Summary:
    Some things like gSOAP, initialize SSL only to have SSLContext stomp over their
    work. Add a method to allow flagging SSLContext as "initialized" to use the
    existing global state.
    
    Test Plan:
    Tested running mcrouter and paymentnegined to verify existing
    functionality appears normal.
    
    Reviewed By: subodh@fb.com
    
    Subscribers: ssl-diffs@, folly-diffs@, yfeldblum, chalfant, dihde
    
    FB internal diff: D1959936
    
    Tasks: 6655251
    
    Signature: t1:1959936:1427994292:8b2be1742ef17a5ec2c7b3ad37787d2cf4d142a6
Commits on Apr 2, 2015
  1. @afrind

    Bump version to 34:0

    afrind authored
  2. @afrind

    Update Makefile.am for AsyncSignalHandler

    afrind authored
    Summary: facepalm
    
    Test Plan: Built proxygen OSS that depends on this symbol
    
    Reviewed By: davejwatson@fb.com
    
    Subscribers: doug, folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1963679
    
    Signature: t1:1963679:1428006035:2c4ceffc046b995431cde2fbcef757fa7ea1ea0b
  3. @rushidesai @afrind

    Move mcrouter/lib/fibers to folly/experimental/fibers

    rushidesai authored afrind committed
    Summary:
    The mcrouter fibers library is quite general purpose and reusable by other projects. Hence I'm moving it into folly.
    
    Test Plan:
    fbconfig -r folly/experimental/fibers && fbmake runtests
    fbconfig -r mcrouter && fbmake runtests
    fbconfig -r tao && fbmake
    fbconfig -r ti && fbmake
    
    Reviewed By: andrii@fb.com
    
    Subscribers: vikas, zhuohuang, jmkaldor, jhunt, pavlo, int, aap, trunkagent, fredemmott, alikhtarov, folly-diffs@, jsedgwick, yfeldblum, chalfant, chip
    
    FB internal diff: D1958061
    
    Signature: t1:1958061:1428005194:b57bfecfe9678e81c48526f57e6197270e2b5a27
  4. @afrind

    Bump version to 33:0

    afrind authored
  5. @djwatson @afrind

    UDP bind via port or address

    djwatson authored afrind committed
    Summary: Bind via port or address, similar to tcp
    
    Test Plan: I wrote a unittest that tested v4 vs v6 binding - can't check it in because we have v6 only clusters though.  Not sure how eles to write a unittest without picking specific ports
    
    Reviewed By: hans@fb.com
    
    Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant
    
    FB internal diff: D1948938
    
    Signature: t1:1948938:1427841721:621031320981bf366a8252a0ea1d4491b694ea24
  6. @afrind

    Move TAsyncSignalHandler into folly

    afrind authored
    Summary:
    TODO item, trying to remove unecessary dependencies on thrift
    
    Test Plan: Unit tests
    
    Reviewed By: davejwatson@fb.com
    
    Subscribers: doug, fbcode-common-diffs@, davejwatson, andrewcox, alandau, bmatheny, anca, darshan, mshneer, folly-diffs@, bil, yfeldblum, haijunz, chalfant
    
    FB internal diff: D1960215
    
    Signature: t1:1960215:1427920934:8abd7e94c50676b05bf7ff79800df0db1bd04266
  7. @afrind

    Add socket mocks

    afrind authored
    Summary: These parallel the MockTAsync* from thrift/lib/cpp/test.  Once all the users of the thrift versions are gone they can be deleted
    
    Test Plan: Unit tests
    
    Reviewed By: yfeldblum@fb.com, alandau@fb.com
    
    Subscribers: doug, net-systems@, folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1959979
    
    Signature: t1:1959979:1427917909:19af219f88dd6847a064da986dd30765e29bdc99
  8. @afrind

    Move AsyncSocket tests from thrift to folly

    afrind authored
    Summary: These tests belong with the code that they test.  The old tests had a couple dependencies on TSocket/TSSLSocket, so I wrote a BlockingSocket wrapper for AsyncSocket/AsyncSSLSocket
    
    Test Plan: Ran the tests
    
    Reviewed By: alandau@fb.com
    
    Subscribers: doug, net-systems@, alandau, bmatheny, mshneer, folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1959955
    
    Signature: t1:1959955:1427917833:73d334846cf248f8bb215f3eb5b596df7f7cee4f
  9. @rushidesai @afrind

    Move AtomicLinkedList to folly

    rushidesai authored afrind committed
    Summary:
    Adding new atomic data structure from the mcrouter project.
    
    Test Plan: fbconfig -r mcrouter && fbmake runtests
    
    Reviewed By: andrii@fb.com
    
    Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant, achao, jpearce
    
    FB internal diff: D1960219
    
    Signature: t1:1960219:1427921204:78e5313ea916b8a249d32e31cc7ba1aa9e890d95
  10. @hannesroth @afrind

    (Wangle) Swap order of Try<T> and T matching

    hannesroth authored afrind committed
    Summary:
    See the test case and D1958860.
    
    I don't really know why I chose one over the other. This fixes a bug. So
    it's better?
    
    Test Plan: Run all the tests?
    
    Reviewed By: yfeldblum@fb.com
    
    Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant, over
    
    FB internal diff: D1960740
    
    Signature: t1:1960740:1427927644:25093b049a1519d7c869ee7043f3caced4cc971e
  11. @afrind

    Add support for writev for AsyncUDPSocket

    Naizhi Li authored afrind committed
    Summary:
    I have 2 local buffers and instead of allocating iobufs
    on the heap, I can simply use iovec on the stack to send.
    
    Test Plan: Unit tests and turn server.
    
    Reviewed By: davejwatson@fb.com
    
    Subscribers: folly-diffs@, ehrhardt, yfeldblum, chalfant, paramr
    
    FB internal diff: D1956201
    
    Signature: t1:1956201:1427918649:09f00399fd9e8ed3fe62be43b19adbf0a5a3f3b0
  12. @afrind

    Minor fixes from documentation

    Tom Jackson authored afrind committed
    Test Plan: unittests
    
    Reviewed By: jbrewer@fb.com, ashishsharma@fb.com
    
    Subscribers: maxime, folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1956101
    
    Signature: t1:1956101:1427828321:2d828e6f2c06c6dcd6e7baa3ed5e72ee59d9b77f
  13. @afrind

    improve split() docs

    Mark McDuff authored afrind committed
    Summary:
    From reading the docs, I assumed that casting exceptions would be caught.  Let's clarify this.
    
    Test Plan: readingiskey
    
    Reviewed By: yfeldblum@fb.com
    
    Subscribers: folly-diffs@, yfeldblum, chalfant
    
    FB internal diff: D1960186
    
    Signature: t1:1960186:1427920471:b7aa2af38a90a986418b98294fc0f69e678f0d1e
  14. @hannesroth @afrind

    (Wangle) Allocate lambda space inside Core instead of inside std::fun…

    hannesroth authored afrind committed
    …ction
    
    Summary:
    Taking this trick that is used in the fibers library. We can keep 64
    bytes of space inside `Core` to allocate the callback lambda into,
    instead of having `std::function` do another `malloc`. This seems to
    greatly improve the synthethic benchmark, and hopefully production
    workloads, too, by reducing the number of mallocs.
    
    64 bytes were picked because it fits all the lambdas in the futures
    tests. We might want to adjust this based on production data...?
    
    https://fb.intern.facebook.com/groups/715931878455430/permalink/837898842925399/
    
    Test Plan: Run all the tests for all platforms, compilers, and Windtunnel.
    
    Reviewed By: hans@fb.com
    
    Subscribers: chalfant, meisner, folly-diffs@, jsedgwick, yfeldblum
    
    FB internal diff: D1931620
    
    Signature: t1:1931620:1427841595:6ec667b58980be232dfb116bc316148bb67de4fc
  15. @djwatson @afrind

    stop in same thread

    djwatson authored afrind committed
    Summary: In a couple places in ServerBootstrap there is code that needs to run in the EB, but if we are already in the EB, it's fine to run it inline.  Maybe this should be a method on EB directly?  There is runInEventBaseThreadAndWait(), but it explicitly disallows this usage.
    
    Test Plan: used in D1942242, deadlocks without this, since stop() was called in the same eb
    
    Reviewed By: yfeldblum@fb.com
    
    Subscribers: chalfant, doug, fugalh, folly-diffs@, jsedgwick, yfeldblum
    
    FB internal diff: D1947581
    
    Signature: t1:1947581:1427831021:d1d08ff9a4a00501d6be60670709fcb17af04134
  16. @afrind

    Move OpenSSL uninitialization to a separate function

    Joel Goguen authored afrind committed
    Summary:
    The current behaviour of SSLContext is subject to a bug where an instance going
    out of scope at the wrong time can decrement the global refcount to 0, causing
    the destructor to destroy the SSL context. Instead, this requires callers to
    explicitly uninitialize the SSLContext instance and assume the risk of wrongly
    destroying the global SSL context.
    
    Test Plan:
    Tested with mcrouter
    
    Reviewed By: subodh@fb.com
    
    Subscribers: afrind, ssl-diffs@, folly-diffs@, yfeldblum, chalfant, dihde, melitam
    
    FB internal diff: D1949649
    
    Tasks: 6358211
    
    Signature: t1:1949649:1427854689:aea2dc801f63256ff64188b0f7a15121dcecee69
Something went wrong with that request. Please try again.