core: add msgpack encoder

[Kyle-Verhoog]

What?

This patch introduces the following:

Packer

A Cython msgpack encoder mostly derived from the official msgpack library's 0.6.2 implementation. The changes made were:

• Removed functionality we don't require
• Small optimizations since we have knowledge of what a trace and span roughly look like
  • Add custom span encoding which will intern the common strings (service, name, resource), etc
  • Process spans objects directly in Cython rather than serializing to a dict and then encoding

TraceMsgPackEncoder

A trace encoder using the above Packer to encode traces.

Why?

We support Python 2 but the msgpack package has dropped support for the C-extension in Python 2. Instead a pure-python fallback is used which is over an order of magnitude slower. This results in an unacceptable performance overhead to Python 2 projects using the tracer.

This also opens the door to further optimizations we can make to encoding.

Benchmarks

Python 2

We see a 23x improvement (244.49ms -> 10.35ms).

Note here that custom is the implementation added, fallback is the default that a user would have if they installed the tracer today without specifying their own version of msgpack.

-------------------------------------------------------------------------------------- benchmark 'encoding': 3 tests ---------------------------------------------------------------------------------------
Name (time in ms)                             Min                 Max                Mean            StdDev              Median               IQR            Outliers      OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_1000_span_trace_custom         9.8290 (1.0)       12.4619 (1.0)       10.4968 (1.0)      0.4931 (1.0)       10.3540 (1.0)      0.5691 (1.18)         19;4  95.2674 (1.0)          89           1
test_encode_1000_span_trace               12.2190 (1.24)      22.5139 (1.81)      12.9208 (1.23)     1.2610 (2.56)      12.6935 (1.23)     0.4821 (1.0)           2;5  77.3948 (0.81)         70           1
test_encode_1000_span_trace_fallback     239.6259 (24.38)    257.7190 (20.68)    244.4946 (23.29)    7.5664 (15.34)    241.5009 (23.32)    7.3441 (15.23)         1;0   4.0901 (0.04)          5           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------ benchmark 'encoding.join_encoded': 2 tests -----------------------------------------------------------------------------
Name (time in us)                 Min                 Max                Mean             StdDev              Median                IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_join_encoded_custom     175.7908 (1.0)      212.5597 (1.0)      184.4673 (1.0)       7.9693 (1.0)      181.3912 (1.0)       9.2912 (1.0)           8;2        5.4210 (1.0)          47         100
test_join_encoded            179.4696 (1.02)     228.1904 (1.07)     193.5662 (1.05)     10.2773 (1.29)     191.0090 (1.05)     11.3487 (1.22)         12;3        5.1662 (0.95)         55         100
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'encoding.small': 2 tests ------------------------------------------------------------------------------------
Name (time in us)                       Min                 Max                Mean             StdDev              Median                IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_trace_small_custom      83.5681 (1.0)      150.4898 (1.0)       95.6845 (1.0)      10.5787 (1.0)       93.4911 (1.0)       7.1502 (1.0)          15;8       10.4510 (1.0)         108         100
test_encode_trace_small            131.4497 (1.57)     260.9396 (1.73)     144.0758 (1.51)     16.5305 (1.56)     140.1711 (1.50)     10.0195 (1.40)          3;3        6.9408 (0.66)         69         100
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------- benchmark 'encoding.small.multi': 2 tests ------------------------------------------------------------------------------
Name (time in ms)                           Min                Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_trace_small_multi_custom     4.1935 (1.0)       5.5765 (1.0)      4.5640 (1.0)      0.2989 (1.0)      4.4820 (1.0)      0.3368 (1.0)          27;5  219.1047 (1.0)         111           2
test_encode_trace_small_multi            6.9890 (1.67)     16.7830 (3.01)     7.9098 (1.73)     1.3714 (4.59)     7.6439 (1.71)     0.4773 (1.42)          3;5  126.4259 (0.58)        117           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Python 3

We get about a 1.4-2.0x improvement with the optimizations introduced.

---------------------------------------------------------------------------------------- benchmark 'encoding': 3 tests ----------------------------------------------------------------------------------------
Name (time in ms)                             Min                 Max                Mean             StdDev              Median                IQR            Outliers       OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_1000_span_trace_custom         5.9540 (1.0)       17.5237 (1.0)        6.8113 (1.0)       1.8651 (1.0)        6.2934 (1.0)       0.4221 (1.0)          9;17  146.8155 (1.0)         138           1
test_encode_1000_span_trace                8.6126 (1.45)      25.9230 (1.48)      10.8887 (1.60)      3.7886 (2.03)       9.1269 (1.45)      2.1952 (5.20)          6;6   91.8384 (0.63)         58           1
test_encode_1000_span_trace_fallback     185.2778 (31.12)    228.0931 (13.02)    199.9352 (29.35)    15.5621 (8.34)     196.9757 (31.30)    17.7143 (41.96)         1;0    5.0016 (0.03)          6           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'encoding.join_encoded': 2 tests -------------------------------------------------------------------------------
Name (time in us)                 Min                   Max                Mean              StdDev              Median                IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_join_encoded            176.3751 (1.0)        291.8260 (1.0)      194.2263 (1.0)       24.1176 (1.0)      186.0701 (1.0)      11.4473 (1.0)           6;7        5.1486 (1.0)          55         100
test_join_encoded_custom     183.8962 (1.04)     1,650.5986 (5.66)     383.0385 (1.97)     264.9502 (10.99)    303.6385 (1.63)     78.7307 (6.88)        20;25        2.6107 (0.51)        191          17
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'encoding.small': 2 tests ------------------------------------------------------------------------------------
Name (time in us)                       Min                 Max                Mean             StdDev              Median                IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_trace_small_custom      60.7158 (1.0)      102.8957 (1.0)       68.4087 (1.0)       7.0093 (1.0)       66.6154 (1.0)       4.6086 (1.0)         12;10       14.6180 (1.0)         156         100
test_encode_trace_small            121.2871 (2.00)     265.2528 (2.58)     141.9907 (2.08)     26.6266 (3.80)     131.9370 (1.98)     18.8154 (4.08)          8;8        7.0427 (0.48)         65         100
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------- benchmark 'encoding.small.multi': 2 tests ------------------------------------------------------------------------------
Name (time in ms)                           Min                Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_encode_trace_small_multi_custom     3.0415 (1.0)      10.9975 (1.0)      4.5682 (1.0)      1.9561 (1.0)      3.6383 (1.0)      1.3941 (2.16)        24;23  218.9026 (1.0)         151           2
test_encode_trace_small_multi            6.0442 (1.99)     26.2392 (2.39)     7.0495 (1.54)     2.3930 (1.22)     6.6648 (1.83)     0.6442 (1.0)           2;7  141.8542 (0.65)        131           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Further optimizations

Outside the scope of this PR

• LRU cache common strings
• Reduce memory allocations with a larger initial buffer
• Reduce memory allocations to 1 by using just one buffer over the lifespan of the tracer
• Fix payloads encoding and dropping data
• inlining more calls
• join_encoded is actually pretty slow and can be quite easily done manually
  • could be done in-place if we take advantage of the encoder/payload being stateful

TODO

• copy setup.py config from msgpack
• further tests
• license issues?

[brettlangdon]

👍 on most of your responses, good to keep things 100% compatible now, then we can make riskier optimizations later

[Kyle-Verhoog]

Was able to run the tests under Windows:

@brettlangdon I think we're good for a final review 😄