Latest commit 4d37864 Aug 23, 2016 Heng Hong Lee committed with Facebook Github Bot AsyncSSLSocket getRawBytes fix
Summary:
While adding logging around our socket and looking into `AsyncSSLSocket` it seems like the data that is actually written into the socket is not correctly attributed.  I
added logs and printed out what happens on the socket layer, P56563098 in this paste you can see that the `[fishhook]` logs are actually those added in D3698728.

Those are the actual bytes written onto the socket, in the paste, the bytes that are written by the AsyncSocket::bioWrite method are
the ones that actually get written onto the socket. some of the bytes written into the bio comes from bf_buff.c and bss_mem.c which are not eventually attributed to a
socket message and will be incorrectly counted when getRawBytesWritten/Read invoked on AsyncSSLSocket.cpp

Unfortunately/Fortunately this issue is not symmetrical and does not manifest in the getRawBytesReceived in AsyncSSLSocket, reading the bio for read bytes
correctly attributes the actual number of bytes written on the socket. moreover, pulling the asyncsocket data for getRawBytesRead doesnt give the full read bytes on
wire because SSL_connect and SSL_read dont return the number of bytes they read but return the number of bytes without the TLS bytes used.

siyengar seems like a right person to look at this. so adding him here.

Would love to discuss more about this and am open to iterating more on this solution

Reviewed By: knekritz

Differential Revision: D3698744

fbshipit-source-id: 541aa478778b9607f51db194fcbfe28bd23c737f
..
Failed to load latest commit information.
build Update generate_varint_tables to support MSVC May 24, 2016
detail Move detail/FunctionalExcept to portability/BitsFunctexcept Aug 19, 2016
docs An intro to the upgrade mutex in the Synchronized docs Aug 20, 2016
experimental Get ObserverManager compiling under MSVC Aug 21, 2016
fibers Split up FibersTest into test and Benchmark. Aug 16, 2016
futures prepare for folly::Executor taking folly::Function Aug 12, 2016
gen Don't use ?: Jul 22, 2016
init folly: allow folly::init to build on systems without the symbolizer (… Jul 20, 2016
io AsyncSSLSocket getRawBytes fix Aug 23, 2016
m4 Generate a pkg-config file for Folly Jul 19, 2016
portability Make it build on Ubuntu 16.04 Aug 23, 2016
ssl Wrappers for some of OpenSSL's crypto hash functions Jun 17, 2016
stats Add constructor for MultiLevelTimeSeries class that uses initializer_… Jun 15, 2016
test Use a constexpr char* under MSVC for the constexpr StringPiece test Aug 22, 2016
ApplyTuple.h Fix applyTuple to work under MSVC again Aug 15, 2016
Arena-inl.h folly copyright 2015 -> copyright 2016 Feb 12, 2016
Arena.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
Array.h Add make_array to folly Apr 19, 2016
Assume.cpp Remove the glog header include from Assume.h Aug 1, 2016
Assume.h Remove the glog header include from Assume.h Aug 1, 2016
AtomicBitSet.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
AtomicHashArray-inl.h Allow const mapped types in folly::AtomicHash(Array|Map) Jun 9, 2016
AtomicHashArray.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
AtomicHashMap-inl.h folly copyright 2015 -> copyright 2016 Feb 12, 2016
AtomicHashMap.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
AtomicIntrusiveLinkedList.h Don't attempt to implicitly convert an atomic<T> to bool May 4, 2016
AtomicLinkedList.h Non-intrusive AtomicLinkedList Apr 16, 2016
AtomicStruct.h Allow constexpr construction of AtomicStruct under MSVC Jun 4, 2016
AtomicUnorderedMap.h Switch uses of <unistd.h> to <folly/portability/Unistd.h> May 30, 2016
Baton.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
Benchmark.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
Benchmark.h Fix a few typos all over the code May 10, 2016
Bits.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
Bits.h Assume p is not nullptr in storeUnaligned Aug 4, 2016
CPortability.h folly: define FOLLY_SANITIZE_THREAD convenience macro for TSAN Jun 6, 2016
CallOnce.h Add call_once, wrapper around std::call_once with a fast path Feb 17, 2016
Checksum.cpp Use the standard intrinsics for crc32c Jun 16, 2016
Checksum.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
ClockGettimeWrappers.cpp Wrappers folly::chrono::clock_gettime and clock_gettime_ns Jul 19, 2016
ClockGettimeWrappers.h Wrappers folly::chrono::clock_gettime and clock_gettime_ns Jul 19, 2016
ConcurrentSkipList-inl.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
ConcurrentSkipList.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
ContainerTraits.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
Conv.cpp Mark the long overload of digits_to as noexcept Aug 19, 2016
Conv.h Fix folly::to<> under MSVC Aug 21, 2016
CppAttributes.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
CpuId.h Include intrin.h in CpuId for the cpuid builtins Aug 11, 2016
Demangle.cpp Limit symbol length in demangle() Jul 29, 2016
Demangle.h folly copyright 2015 -> copyright 2016 Feb 12, 2016
DiscriminatedPtr.h Added DiscriminatedPtr support to arm64 Aug 3, 2016
DynamicConverter.h Use std::string in folly::dynamic Apr 26, 2016
Enumerate.h Fix Enumerate C++17 Support Aug 4, 2016
EvictingCacheMap.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
Exception.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
ExceptionString.h Get rid of circular dependency when including ExceptionWrapper.h from… Jun 10, 2016
ExceptionWrapper.h Get rid of circular dependency when including ExceptionWrapper.h from… Jun 10, 2016
Executor.h folly copyright 2015 -> copyright 2016 Feb 12, 2016
Expected.h Fix folly::Expected under MSVC Aug 18, 2016
FBString.h Make fbstring::assign() tight, and simplify operator=() and append() Aug 23, 2016
FBVector.h Fix typos in FBVector's comments Aug 13, 2016
File.cpp Switch uses of <unistd.h> to <folly/portability/Unistd.h> May 30, 2016
File.h Switch uses of <unistd.h> to <folly/portability/Unistd.h> May 30, 2016
FileUtil.cpp Use the socket portability layer when needed. Aug 5, 2016
FileUtil.h Include the fcntl.h portability header where it's needed Aug 4, 2016
Fingerprint.h folly: fix clang's -Wundefined-var-template Jun 25, 2016
Foreach.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
Format-inl.h Include <folly/portability/Windows.h> in folly::format May 26, 2016
Format.cpp Start compiling a bit of `-Wshadow` Apr 6, 2016
Format.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
FormatArg.h Mark FormatArg::error as noreturn Aug 10, 2016
FormatTraits.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
Function.h Don't name a member small Aug 4, 2016
GroupVarint.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
GroupVarint.h Enable GroupVarint on Arm64 Jul 14, 2016
Hash.h fix bug for negative shift value Jun 24, 2016
IPAddress.cpp Move IPAddress definitions to source files Jul 9, 2016
IPAddress.h Move IPAddress definitions to source files Jul 9, 2016
IPAddressException.h Move IPAddress definitions to source files Jul 9, 2016
IPAddressV4.cpp Move IPAddress definitions to source files Jul 9, 2016
IPAddressV4.h Move IPAddress definitions to source files Jul 9, 2016
IPAddressV6.cpp Move IPAddress definitions to source files Jul 9, 2016
IPAddressV6.h Move IPAddress definitions to source files Jul 9, 2016
Indestructible.h Get MSVC able to construct an Indestructible Jul 26, 2016
IndexedMemPool.h Switch uses of <unistd.h> to <folly/portability/Unistd.h> May 30, 2016
IntrusiveList.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
Lazy.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
LifoSem.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
LifoSem.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
Likely.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
LockTraits.h Adding policies for all the upgrade and downgrade mutex transitions t… Aug 5, 2016
LockTraitsBoost.h Adding support for upgradable mutexes to LockTraits Aug 5, 2016
Logging.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
MPMCPipeline.h folly copyright 2015 -> copyright 2016 Feb 12, 2016
MPMCQueue.h Switch uses of <unistd.h> to <folly/portability/Unistd.h> May 30, 2016
MacAddress.cpp Move IPAddress definitions to source files Jul 9, 2016
MacAddress.h Move IPAddress definitions to source files Jul 9, 2016
Makefile.am Fixup a reference to detail/FunctionalExcept.cpp Aug 22, 2016
MallctlHelper.cpp Add helper functions for mallctl Jul 24, 2016
MallctlHelper.h Add helper functions for mallctl Jul 24, 2016
Malloc.h Make Malloc.h self-contained again Aug 23, 2016
MapUtil.h get_or_throw(map, key) returns references Jul 16, 2016
Memory.h allocate_sys_buffer Jul 20, 2016
MemoryMapping.cpp Switch gflags to portability/GFlags.h May 5, 2016
MemoryMapping.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
Merge.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
MicroLock.cpp Additional changes to MicroLock Aug 12, 2016
MicroLock.h Additional changes to MicroLock Aug 12, 2016
MicroSpinLock.h Assert that MicroSpinLock is a POD type Jun 1, 2016
MoveWrapper.h folly copyright 2015 -> copyright 2016 Feb 12, 2016
Optional.h Add folly::Expected, an alternative to exceptions for non-throwing AP… Aug 15, 2016
PackedSyncPtr.h Pack PackedSyncPtr Jun 8, 2016
Padded.h Don't use ?: Jul 22, 2016
Partial.h explicit return types in Partial to work around gcc bug Aug 16, 2016
PicoSpinLock.h Fix a typo in PicoSpinLock Aug 3, 2016
Portability.h Move detail/FunctionalExcept to portability/BitsFunctexcept Aug 19, 2016
Preprocessor.h Update FB_ONE_OR_NONE in Preproc.h to work with MSVC Jul 22, 2016
ProducerConsumerQueue.h Fix test/ProducerConsumerQueueTest.cpp build break May 28, 2016
RWSpinLock.h update Synchronized to use LockTraits Jul 6, 2016
Random-inl.h Correctly deduce RNG type in folly::Random Apr 15, 2016
Random.cpp Handle creating the default crypto context if it doesn't already exist Aug 5, 2016
Random.h Correctly deduce RNG type in folly::Random Apr 15, 2016
Range.h Move IPAddress definitions to source files Jul 9, 2016
SafeAssert.cpp Let FOLLY_SAFE_DCHECK be erased by the optimizer in release builds Aug 15, 2016
SafeAssert.h Let FOLLY_SAFE_DCHECK be erased by the optimizer in release builds Aug 15, 2016
ScopeGuard.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
SharedMutex.cpp folly: fix clang's -Wundefined-var-template Jun 25, 2016
SharedMutex.h Fix typo in SharedMutex comment Jul 24, 2016
Singleton-inl.h Don't allow getting singleton after shutdown in strict mode Aug 19, 2016
Singleton.cpp Use ReadMostlyMainPtrDeleter in folly::Singleton Aug 10, 2016
Singleton.h Don't allow getting singleton after shutdown in strict mode Aug 19, 2016
SingletonStackTrace.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
SingletonThreadLocal.h Fix SingletonThreadLocal Feb 29, 2016
SingletonVault_c.cpp SingletonVault_c.cpp is always C++ Jun 1, 2016
SingletonVault_c.h folly copyright 2015 -> copyright 2016 Feb 12, 2016
SmallLocks.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
SocketAddress.cpp Use the socket portability layer when needed. Aug 5, 2016
SocketAddress.h Use the socket portability layer when needed. Aug 5, 2016
SparseByteSet.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
SpinLock.h update Synchronized to use LockTraits Jul 6, 2016
SpookyHashV1.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
SpookyHashV1.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
SpookyHashV2.cpp Make Bits.h respect FOLLY_HAVE_UNALIGNED_ACCESS. Mar 9, 2016
SpookyHashV2.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
String-inl.h Use folly::Expected to implement folly::tryTo, a non-throwing variant… Aug 16, 2016
String.cpp Extensibility for folly::to<> through ADL Apr 7, 2016
String.h Use folly::Expected to implement folly::tryTo, a non-throwing variant… Aug 16, 2016
StringBase.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
Subprocess.cpp takeOwnershipOfPipes method releases memory Jul 29, 2016
Subprocess.h add a default Subprocess constructor May 26, 2016
Synchronized.h Adding upgradable locks to Synchronized Aug 19, 2016
ThreadCachedArena.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
ThreadCachedArena.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
ThreadCachedInt.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
ThreadLocal.h Transfer ownership from a unique_ptr to a ThreadLocalPtr May 12, 2016
ThreadName.h folly: improve setThreadName for macOS Jul 18, 2016
TimeoutQueue.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
TimeoutQueue.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
TokenBucket.h folly copyright 2015 -> copyright 2016 Feb 12, 2016
Traits.h Use std::is_nothrow_swappable under MSVC Aug 16, 2016
Try-inl.h Extract Try to top-level May 19, 2016
Try.h Extract Try to top-level May 19, 2016
Unicode.cpp Use std::string in folly::dynamic Apr 26, 2016
Unicode.h Use std::string in folly::dynamic Apr 26, 2016
Unit.h Flesh out the comments about Unit May 18, 2016
Uri-inl.h folly copyright 2015 -> copyright 2016 Feb 12, 2016
Uri.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
Uri.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
VERSION Bump version to 57:0 Aug 31, 2015
Varint.h folly: ubsan: cast to unsigned to avoid negative-left-shift May 10, 2016
Version.cpp folly copyright 2015 -> copyright 2016 Feb 12, 2016
VersionCheck.h folly: replace old-style header guards with "pragma once" Apr 1, 2016
configure.ac Make it build on Ubuntu 16.04 Aug 23, 2016
dynamic-inl.h Add a pair of overloads to dynamic::setDefault Aug 15, 2016
dynamic.cpp Use std::string in folly::dynamic Apr 26, 2016
dynamic.h Add a pair of overloads to dynamic::setDefault Aug 15, 2016
json.cpp Folly parseJson doesn't handle minInt properly Jun 27, 2016
json.h Add recursion limit to folly::parseJson. Apr 27, 2016
libfolly.pc.in Generate a pkg-config file for Folly Jul 19, 2016
memcpy.S folly copyright 2015 -> copyright 2016 Feb 12, 2016
small_vector.h Move the type_traits portability header into its own portability header Aug 17, 2016
sorted_vector_types.h Switch away from things that are removed in C++17 Jul 14, 2016
stop_watch.h stop_watch is moved to folly Jul 1, 2016