[BEAM-3042] Adding time tracking of batch side inputs [low priority]

[pabloem]

This PR improves Cython tags for some classes, and uses them for tracking of time spent reading side inputs.

It also adds a state sampler thread to make the side input microbenchmark a more realistic representation of the worker environment (as there may be contention due to the state switching).

NOTE: This PR should add flag versioning before merging in any case.

Current performance with 500 runs:

• Average runtime: 0.604471780777
• Time per element: 3.77794862986e-05
• Regression: 0% (it's the baseline)

With change and flag deactivated:

• Average runtime: 0.614184889793
• Time per element: 3.83865556121e-05
• Regression: 1.6%

With change and flag activated:

• Average runtime: 0.611651549339
• Time per element: 3.82282218337e-05
• Regression: 1.187%

This represents a really small regression in a microbenchmark that specifically exercises this feature.

[pabloem]

Well, it seems that this code is going to disappear. Oh well.....

[pabloem]

The code is not going away after all. Reopening.

[pabloem]

Run Python PostCommit

[pabloem]

r: @aaltay
Preempting a question that you may have: I believe the reason why this change is faster compared to the previous attempt is the improved cython annotations in the state sampler, and in the counter classes.

I understand that @boyuanzz has made extensive research on perf diagnosing of a new code change.
What do you think is a good path for validating whether the perf footprint is small enough be accepted in? (beyond validating with the microbenchmark)

[aaltay]

Change LGTM. The PR description refers to a flag but I do not see the flag in the code, what are you referring to ?

[pabloem]

Gating on a new flag. I've added a commit for that. How does it all look?

[pabloem]

Flag pushed.

[pabloem]

With 2.5.0 branch cut, I'd like to merge this soon. LMK if things look good

[pabloem]

ping

[aaltay]

Was there any change after my LGTM on May 11? If not feel free to self merge when ready.