tracing: utxocache tracepoints #22902

arnabsen1729 · 2021-09-06T14:35:17Z

This PR adds some of the UTXO set cache tracepoints proposed in #20981 (comment). The first tracepoints were added in #22006.

tracepoint	description
`utxocache:flush`	Is called after the caches and indexes are flushed
`utxocache:add`	when a new coin is added to the UTXO cache
`utxocache:spent`	when a coin is spent
`utxocache:uncache`	when coin is removed from the UTXO cache

The tracepoints are further documented in docs/tracing.md and the usage is shown via the two newly added example scripts in contrib/tracing/.

0xB10C · 2021-09-06T15:25:12Z

Concept ACK.

@arnabsen1729 is working on this as part of his summerofbitcoin.org project. I've mentored him along the way. Some earlier review is in 0xB10C#1.

I suspect hooking into the add and spend tracepoints during IBD could have a noticeable performance impact. I'll test this. If is is the case, then it should be documented. Not hooking into the tracepoints shouldn't have any performance impact.

jb55 · 2021-09-06T16:26:26Z

src/validation.cpp

@@ -2120,6 +2120,13 @@ bool CChainState::FlushStateToDisk(
            nLastFlush = nNow;
            full_flush_completed = true;
        }
+        TRACE6(utxocache, flush,
+               (int64_t)(GetTimeMicros() - nNow.count()), // in microseconds (µs)


what's the purpose of this argument?

The purpose was to get the duration of the flush, this value can be used for benchmarking purposes.

The nNow was declared here:

bitcoin/src/validation.cpp

Line 1969 in 384d076

const auto nNow = GetTime<std::chrono::microseconds>();

jb55 · 2021-09-06T16:36:38Z

src/coins.cpp

@@ -95,6 +96,15 @@ void CCoinsViewCache::AddCoin(const COutPoint &outpoint, Coin&& coin, bool possi
    it->second.coin = std::move(coin);
    it->second.flags |= CCoinsCacheEntry::DIRTY | (fresh ? CCoinsCacheEntry::FRESH : 0);
    cachedCoinsUsage += it->second.coin.DynamicMemoryUsage();
+    TRACE8(utxocache, add,
+           outpoint.hash.ToString().c_str(),


do we need the hash string here (and the ones below)? we should avoid doing non-negligible computations in traces.

We need if we want to support the current versions of bpftrace as it can't print binary data in hex (or at all really).

related discussion #22006 (comment)

might be worth thinking about adding support for this in bpftrace rather than this work around here

I agree that outpoint.hash.ToString().c_str() here is called a lot more than the hash.data().toString().c_str() in the connect_block tracepoint. This could hurt performance. Might need to drop this.

How about only using the first 64bit of the hash: outpoint.hash.GetUint64(0). This should be very fast and maybe unique enough, at least for the purpose of tracing

We already pass outpoint.hash.data() as the last argument, that's not the issue. The issue here is the ToString() part which might be to expensive in such a frequently called function while being only a work around for bpftrace not supporting hex-formatting.

I just came across bpftrace/bpftrace#659 (comment). We don't need to use the built-in hex-formatting as it's quite simple just printf("%02x") each byte of the txid in an unroll () {...}. This even allows us to print the txids in big-endian (the block-explorer format).

$p = $txid + 31; unroll(32) { $b = *(uint8*)$p; printf("%02x", $b); $p-=1; }

This means we can drop the outpoint.hash.ToString().c_str() from the tracepoints added here and use the outpoint.hash.data() as the first argument.

Nice, I was trying to use unroll like this for printing data in bpftrace but could never figure it out. I was never able to do it based on a length argument, probably because it wasn't a literal. This case works fine though since we know the length of the hash.

We already pass outpoint.hash.data() as the last argument, that's not the issue. The issue here is the ToString() part which might be to expensive in such a frequently called function while being only a work around for bpftrace not supporting hex-formatting.

I agree here. Let's remove this ToString argument. It's too much overhead to allocate, construct, deallocate a string here every time even if the tracepoint is disabled.

DrahtBot · 2021-09-07T00:28:44Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#9384 (CCoinsViewCache code cleanup & deduplication by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

practicalswift · 2021-09-08T07:58:08Z

Concept ACK

theStack · 2021-09-08T14:28:20Z

Concept ACK

15b8ba5 [tracing] tracepoints to utxocache add, spent and uncache (Arnab Sen) 56f0a04 [tracing] tracepoint for utxocache flushes (Arnab Sen) Pull request description: This PR adds some of the UTXO set cache tracepoints proposed in bitcoin/bitcoin#20981 (comment). The first tracepoints were added in bitcoin#22006. tracepoint | description -- | -- `utxocache:flush` | Is called after the caches and indexes are flushed `utxocache:add` | when a new coin is added to the UTXO cache `utxocache:spent` | when a coin is spent `utxocache:uncache` | when coin is removed from the UTXO cache The tracepoints are further documented in `docs/tracing.md` and the usage is shown via the two newly added example scripts in `contrib/tracing/`. Top commit has no ACKs. Tree-SHA512: 53ba6c2811c7fed08aa608a888bff83214e754b0f50621e83e4f81e95bc22d21f7f956096d12858d8e85240aa226431845995682eaaf55e93016c5b8f4c9282b

0xB10C · 2021-09-10T09:32:29Z

My benchmarks (4x reindex-chainstate till block 500000 for both the mergebase 92aad53 and 15b8ba5) show that this PR increases time spent in ConnectBlock by about 3.4%. This is without hooking into the tracepoints.

As discussed in #22902 (comment) we can drop the first argument of the add, spent, and uncache tracepoints which should improve the performance. Will re-benchmark then.

0xB10C · 2021-09-11T23:00:52Z

Did some more profiling on a node in IBD running with this PR (15b8ba5) to make sure the ToString() is really the issue here.

In CCoinsViewCache::SpendCoin we spend more than half of the time in ToString().

In CCoinsViewCache::AddCoin we spend a lot less time in ToString() compared to CCoinsViewCache::SpendCoin, but still significant time.

(please ignore the bitcoind-22902-mergebase binary name shown here. It's not the mergebase, the binary is build from 15b8ba5)

laanwj · 2021-10-01T07:29:25Z

I think in general good advice is to pass binary/raw data into the trace functions directly. Don't spent effort formatting, you can do this on the other side (and only when necessary, just before presentation to the user).

This tracepoint was introduced in bitcoin#22006. The first argument was the hash of the connected block as a pointer to a C-like String. The last argument passed the hash of the connected block as pointer to 32 bytes. The hash was passed both as pointer to a string and to bytes to allow `bpftrace` scripts to print the String. It was (incorrectly) assumed that `bpftrace` is not able to hex-format and print the block hash given only the hash bytes. The block hash can be printed in `bpftrace` by calling `printf("%02x")` for each byte of the hash in an `unroll () {...}`. By starting from the last byte of the hash it can be printed in in big-endian (the block-explorer format). ```C $p = $hash + 31; unroll(32) { $b = *(uint8*)$p; printf("%02x", $b); $p-=1; } ``` See also: bitcoin#22902 (comment) This is a breaking change to the connect_block tracepoint API, however there exist no release including the tracepoint yet.

This tracepoint was introduced in bitcoin#22006. The first argument was the hash of the connected block as a pointer to a C-like String. The last argument passed the hash of the connected block as a pointer to 32 bytes. The hash was passed both as a pointer to a string and bytes to allow `bpftrace` scripts to print the String. It was (incorrectly) assumed that `bpftrace` cannot hex-format and print the block hash given only the hash as bytes. The block hash can be printed in `bpftrace` by calling `printf("%02x")` for each byte of the hash in an `unroll () {...}`. By starting from the last byte of the hash, it can be printed in big-endian (the block-explorer format). ```C $p = $hash + 31; unroll(32) { $b = *(uint8*)$p; printf("%02x", $b); $p-=1; } ``` See also: bitcoin#22902 (comment) This is a breaking change to the connect_block tracepoint API, however there exist no release, including the tracepoint, yet.

The tracepoint `validation:block_connected` was introduced in bitcoin#22006. The first argument was the hash of the connected block as a pointer to a C-like String. The last argument passed the hash of the connected block as a pointer to 32 bytes. The hash was passed both as a pointer to a string and bytes to allow `bpftrace` scripts to print the String. It was (incorrectly) assumed that `bpftrace` cannot hex- format and print the block hash given only the hash as bytes. The block hash can be printed in `bpftrace` by calling `printf("%02x")` for each byte of the hash in an `unroll () {...}`. By starting from the last byte of the hash, it can be printed in big-endian (the block-explorer format). ```C $p = $hash + 31; unroll(32) { $b = *(uint8*)$p; printf("%02x", $b); $p-=1; } ``` See also: bitcoin#22902 (comment) This is a breaking change to the block_connected tracepoint API, however this tracepoint has not yet been included in a release.

The tracepoint `validation:block_connected` was introduced in bitcoin#22006. The first argument was the hash of the connected block as a pointer to a C-like String. The last argument passed the hash of the connected block as a pointer to 32 bytes. The hash was only passed as string to allow `bpftrace` scripts to print the hash. It was (incorrectly) assumed that `bpftrace` cannot hex-format and print the block hash given only the hash as bytes. The block hash can be printed in `bpftrace` by calling `printf("%02x")` for each byte of the hash in an `unroll () {...}`. By starting from the last byte of the hash, it can be printed in big-endian (the block-explorer format). ```C $p = $hash + 31; unroll(32) { $b = *(uint8*)$p; printf("%02x", $b); $p-=1; } ``` See also: bitcoin#22902 (comment) This is a breaking change to the block_connected tracepoint API, however this tracepoint has not yet been included in a release.

The tracepoint `validation:block_connected` was introduced in bitcoin#22006. The first argument was the hash of the connected block as a pointer to a C-like String. The last argument passed the hash of the connected block as a pointer to 32 bytes. The hash was only passed as string to allow `bpftrace` scripts to print the hash. It was (incorrectly) assumed that `bpftrace` cannot hex-format and print the block hash given only the hash as bytes. The block hash can be printed in `bpftrace` by calling `printf("%02x")` for each byte of the hash in an `unroll () {...}`. By starting from the last byte of the hash, it can be printed in big-endian (the block-explorer format). ```C $p = $hash + 31; unroll(32) { $b = *(uint8*)$p; printf("%02x", $b); $p -= 1; } ``` See also: bitcoin#22902 (comment) This is a breaking change to the block_connected tracepoint API, however this tracepoint has not yet been included in a release.

…alidation:block_connected` tracepoint 53c9fa9 tracing: drop block_connected hash.toString() arg (0xb10c) Pull request description: The tracepoint `validation:block_connected` was introduced in #22006. The first argument was the hash of the connected block as a pointer to a C-like String. The last argument passed the hash of the connected block as a pointer to 32 bytes. The hash was only passed as string to allow `bpftrace` scripts to print the hash. It was (incorrectly) assumed that `bpftrace` cannot hex-format and print the block hash given only the hash as bytes. The block hash can be printed in `bpftrace` by calling `printf("%02x")` for each byte of the hash in an `unroll () {...}`. By starting from the last byte of the hash, it can be printed in big-endian (the block-explorer format). ```C $p = $hash + 31; unroll(32) { $b = *(uint8*)$p; printf("%02x", $b); $p -= 1; } ``` See also: #22902 (comment) This is a breaking change to the block_connected tracepoint API, however this tracepoint has not yet been included in a release. ACKs for top commit: laanwj: Concept and code review ACK 53c9fa9 jb55: ACK 53c9fa9 Tree-SHA512: f1b9e4e0ee45aae892e8bf38e04b5ee5fbc643d6e7e27d011b829ed8701dacf966a99b7c877c46cca8666b894a375633e62582c552c8203614c6f2b9c4087585

… the `validation:block_connected` tracepoint 53c9fa9 tracing: drop block_connected hash.toString() arg (0xb10c) Pull request description: The tracepoint `validation:block_connected` was introduced in bitcoin#22006. The first argument was the hash of the connected block as a pointer to a C-like String. The last argument passed the hash of the connected block as a pointer to 32 bytes. The hash was only passed as string to allow `bpftrace` scripts to print the hash. It was (incorrectly) assumed that `bpftrace` cannot hex-format and print the block hash given only the hash as bytes. The block hash can be printed in `bpftrace` by calling `printf("%02x")` for each byte of the hash in an `unroll () {...}`. By starting from the last byte of the hash, it can be printed in big-endian (the block-explorer format). ```C $p = $hash + 31; unroll(32) { $b = *(uint8*)$p; printf("%02x", $b); $p -= 1; } ``` See also: bitcoin#22902 (comment) This is a breaking change to the block_connected tracepoint API, however this tracepoint has not yet been included in a release. ACKs for top commit: laanwj: Concept and code review ACK 53c9fa9 jb55: ACK 53c9fa9 Tree-SHA512: f1b9e4e0ee45aae892e8bf38e04b5ee5fbc643d6e7e27d011b829ed8701dacf966a99b7c877c46cca8666b894a375633e62582c552c8203614c6f2b9c4087585

laanwj · 2021-11-10T18:39:49Z

Concept and light code review ACK, thanks for adding tracepoints!

Edit: see discussion please remove the ToString.

The tracepoint `validation:block_connected` was introduced in #22006. The first argument was the hash of the connected block as a pointer to a C-like String. The last argument passed the hash of the connected block as a pointer to 32 bytes. The hash was only passed as string to allow `bpftrace` scripts to print the hash. It was (incorrectly) assumed that `bpftrace` cannot hex-format and print the block hash given only the hash as bytes. The block hash can be printed in `bpftrace` by calling `printf("%02x")` for each byte of the hash in an `unroll () {...}`. By starting from the last byte of the hash, it can be printed in big-endian (the block-explorer format). ```C $p = $hash + 31; unroll(32) { $b = *(uint8*)$p; printf("%02x", $b); $p -= 1; } ``` See also: bitcoin/bitcoin#22902 (comment) This is a breaking change to the block_connected tracepoint API, however this tracepoint has not yet been included in a release.

Signed-off-by: Arnab Sen <arnabsen1729@gmail.com>

0xB10C · 2021-12-06T11:12:53Z

Agree that a follow-up is better suited here.

0xB10C · 2021-12-06T11:28:32Z

I've just tested the log_utxos.bt on mainnet and it's giving me this output:

Spent  0b9ab1dc44cf8ff9a3b9d4Lost 20 events
94Lost 36 events
f6Lost 33 events
b241Lost 32 events
b2Lost 32 events
Spent  6ae7e6eb7cffLost 28 events
bcLost 43 events
faLost 46 events
09Lost 24 events
37Lost 19 events
2fLost 32 events
Spent  1bfaf48e84d8b3dc2c3ebe282d8842Lost 34 events
c0Lost 18 events
5e17Lost 17 events
8aLost 42 events
0dLost 39 events
7ee9Lost 31 events
Spent  bbc2c4ccbd1c3d6583cacd4a8572b2Lost 44 events

The tracepoints utxocache:add and utxocache:spent tracepoints are called very frequently during block connections causing events to be lost as the buffer fills before it read. Haven't tested if attaching to the two tracepoints via a BCC Python script or Rust program behaves differently.

I'm seeing events being lost during testnet and signet IBD too, however not as many as on mainnet (as the blocks contain fewer transactions and thus fewer changes to the UTXO set).

0xB10C · 2021-12-06T12:01:08Z

Let's handle it in a follow-up PR.

fwiw: @arnabsen1729 is working on a follow-up

jb55 · 2021-12-06T18:29:41Z

On Mon, Dec 06, 2021 at 03:28:44AM -0800, 0xB10C wrote: I've just tested the `log_utxos.bt` on mainnet and it's giving me this output: 94Lost 36 events [..] The tracepoints `utxocache:add` and `utxocache:spent` tracepoints are called very frequently during block connections causing events to be lost as the buffer fills before it read. Haven't tested if attaching to the two tracepoints via a BCC Python script or Rust program behaves differently.

Interesting, it looks like this is a fundamental limitation of the ring buffer. If you're generating events faster than the script can process them, it looks like events will be dropped. Looking at the script, calling printf 32 times might be pretty slow relative to the incoming events. Is it possible to call printf once with 32 %02x entries in the format string?

0xB10C · 2021-12-06T19:12:39Z

Looking at the script, calling printf 32 times might be pretty slow relative to the incoming events. Is it possible to call printf once with 32 %02x entries in the format string?

Maybe, yes.

In BCC you can set the page count of the ring buffer. That worked for me in the past. You can control bpftrace's ring buffer page count with BPFTRACE_PERF_RB_PAGES (needs to be set to a power of two).

Running BPFTRACE_PERF_RB_PAGES=16384 bpftrace contrib/tracing/log_utxos.bt is the lowest page size that works for me here without dropping events. Default is 64.

jb55 · 2021-12-06T19:14:33Z

@0xB10C that seems to be a better solution, since I imagine people will have harder workloads than 32 printfs...

0xB10C · 2021-12-06T19:18:48Z

yes, I guess mentioning this on either the docs or examples page would be good. Could be done in @arnabsen1729 follow-up.

jb55 · 2021-12-06T19:20:01Z

if the page size is 4096 (some quick googling, not sure if correct), then 16384 pages is ~67 MB. seems pretty high but not unreasonable...

See these comments: bitcoin#22902 (comment)

The documentation and examples for the fFlushForPrune argument of the utxocache flush tracepoint weren't clear without looking at the code. See these comments: bitcoin#22902 (comment)

4769342 [tracing] tracepoints to utxocache add, spent and uncache (Arnab Sen) 7d79993 [tracing] tracepoint for utxocache flushes (Arnab Sen) Pull request description: This PR adds some of the UTXO set cache tracepoints proposed in bitcoin/bitcoin#20981 (comment). The first tracepoints were added in bitcoin#22006. tracepoint | description -- | -- `utxocache:flush` | Is called after the caches and indexes are flushed `utxocache:add` | when a new coin is added to the UTXO cache `utxocache:spent` | when a coin is spent `utxocache:uncache` | when coin is removed from the UTXO cache The tracepoints are further documented in `docs/tracing.md` and the usage is shown via the two newly added example scripts in `contrib/tracing/`. ACKs for top commit: laanwj: Code and documentation review ACK 4769342 Tree-SHA512: d6b4f435d3260de4c48b36956f9311f65ab3b52cd03b1e0a4ba9cf47a774d8c4b31878e222b11e0ba5d233a68f7567f8a367b12a6392f688c10c11529341e837

- mention 'Lost X events' workaround - clarify flush tracepoint docs - fix typo in tracepoint context - clarify flush for prune The documentation and examples for the `fFlushForPrune` argument of the utxocache flush tracepoint weren't clear without looking at the code. See these comments: bitcoin#22902 (comment)

- mention 'Lost X events' workaround - clarify flush tracepoint docs - fix typo in tracepoint context - clarify flush for prune The documentation and examples for the `fFlushForPrune` argument of the utxocache flush tracepoint weren't clear without looking at the code. See these comments: bitcoin#22902 (comment) - doc: note that there can be temporary UTXO caches Bitcoin Core uses temporary clones of it's _main_ UTXO cache in some places. The utxocache:add and :spent tracepoints are triggered when temporary caches are changed too. This is documented.

… for #22902 799968e tracing: misc follow-ups to 22902 (0xb10c) 36a6584 tracing: correctly scope utxocache:flush tracepoint (Arnab Sen) Pull request description: This PR is a follow-up to the [#22902](bitcoin/bitcoin#22902). Previously, the tracepoint `utxocache:flush` was called, even when it was not flushing. So, the tracepoint is now scoped to write only when coins cache to disk. ACKs for top commit: 0xB10C: ACK 799968e Tree-SHA512: ebb096cbf991c551c81e4339821f10d9768c14cf3d8cb14d0ad851acff5980962228a1c746914c6aba3bdb27e8be53b33349c41efe8bab5542f639916e437b5f

…coin#22902 799968e tracing: misc follow-ups to 22902 (0xb10c) 36a6584 tracing: correctly scope utxocache:flush tracepoint (Arnab Sen) Pull request description: This PR is a follow-up to the [bitcoin#22902](bitcoin#22902). Previously, the tracepoint `utxocache:flush` was called, even when it was not flushing. So, the tracepoint is now scoped to write only when coins cache to disk. ACKs for top commit: 0xB10C: ACK 799968e Tree-SHA512: ebb096cbf991c551c81e4339821f10d9768c14cf3d8cb14d0ad851acff5980962228a1c746914c6aba3bdb27e8be53b33349c41efe8bab5542f639916e437b5f

76c60d7 test: validation:block_connected tracepoint test (0xb10c) 260e28e test: utxocache:* tracepoint tests (0xb10c) 34b27ba test: net:in/out_message tracepoint tests (0xb10c) c934087 test: checks for tracepoint tests (0xb10c) Pull request description: This adds functional tests for the USDT tracepoints added in bitcoin/bitcoin#22006 and bitcoin/bitcoin#22902. This partially fixes #23296. The tests **are probably skipped** on most systems as these tests require: - a Linux system with a kernel that supports BPF (and available kernel headers) - that Bitcoin Core is compiled with tracepoints for USDT support (default when compiled with depends) - [bcc](https://github.com/iovisor/bcc) installed - the tests are run with a privileged user that is able to e.g. do BPF syscalls and load BPF maps The tests are not yet run in our CI as the CirrusCI containers lack the required permissions (see bitcoin/bitcoin#23296 (comment)). Running the tests in a VM in the CI could work, but I haven't experimented with this yet. The priority was to get the actual tests done first to ensure the tracepoints work as intended for the v23.0 release. Running the tracepoint tests in the CI is planned as the next step to finish #23296. The tests can, however, be run against e.g. release candidates by hand. Additionally, they provide a starting point for tests for future tracepoints. PRs adding new tracepoint should include tests. This makes reviewing these PRs easier. The tests require privileges to execute BPF sycalls (`CAP_SYS_ADMIN` before Linux kernel 5.8 and `CAP_BPF` and `CAP_PERFMON` on 5.8+) and permissions to `/sys/kernel/debug/tracing/`. It's currently recommended to run the tests in a virtual machine (or on a VPS) where it's sensible to use the `root` user to gain these privileges. Never run python scripts you haven't carefully reviewed with `root` permissions! It's unclear if a non-root user can even gain the required privileges. This needs more experimenting. The goal here is to test the tracepoint interface to make sure the [documented interface](https://github.com/bitcoin/bitcoin/blob/master/doc/tracing.md#tracepoint-documentation) does not break by accident. The tracepoints expose implementation details. This means we also need to rely on implementation details of Bitcoin Core in these functional tests to trigger the tracepoints. An example is the test of the `utxocache:flush` tracepoint: On Bitcoin Core shutdown, the UTXO cache is flushed twice. The corresponding tracepoint test expects two flushes, too - if not, the test fails. Changing implementation details could cause these tests to fail and the tracepoint API to break. However, we purposefully treat the tracepoints only as [**semi-stable**](https://github.com/bitcoin/bitcoin/blob/master/doc/tracing.md#semi-stable-api). The tracepoints should not block refactors or changes to other internals. ACKs for top commit: jb55: tACK 76c60d7 laanwj: Tested ACK 76c60d7 Tree-SHA512: 9a63d945c68102e59d751bd8d2805ddd7b37185408fa831d28a9cb6641b701961389b55f216c475df7d4771154e735625067ee957fc74f454ad7a7921255364c

- mention 'Lost X events' workaround - clarify flush tracepoint docs - fix typo in tracepoint context - clarify flush for prune The documentation and examples for the `fFlushForPrune` argument of the utxocache flush tracepoint weren't clear without looking at the code. See these comments: bitcoin/bitcoin#22902 (comment) - doc: note that there can be temporary UTXO caches Bitcoin Core uses temporary clones of it's _main_ UTXO cache in some places. The utxocache:add and :spent tracepoints are triggered when temporary caches are changed too. This is documented.

Summary: ``` The tracepoint `validation:block_connected` was introduced in #22006. The first argument was the hash of the connected block as a pointer to a C-like String. The last argument passed the hash of the connected block as a pointer to 32 bytes. The hash was only passed as string to allow `bpftrace` scripts to print the hash. It was (incorrectly) assumed that `bpftrace` cannot hex-format and print the block hash given only the hash as bytes. The block hash can be printed in `bpftrace` by calling `printf("%02x")` for each byte of the hash in an `unroll () {...}`. By starting from the last byte of the hash, it can be printed in big-endian (the block-explorer format). $p = $hash + 31; unroll(32) { $b = *(uint8*)$p; printf("%02x", $b); $p -= 1; } See also: bitcoin/bitcoin#22902 (comment) This is a breaking change to the block_connected tracepoint API, however this tracepoint has not yet been included in a release. ``` Backport of [[bitcoin/bitcoin#23302 | core#23302]]. Also updated sigops => sigchecks in the bt script, and updated the detection so it's more obvious that it's enabled. Test Plan: Build and run with tracing enabled, then: sudo bpftrace ../contrib/tracing/connectblock_benchmark.bt 0 0 25 Reviewers: #bitcoin_abc, sdulfari Reviewed By: #bitcoin_abc, sdulfari Subscribers: sdulfari Differential Revision: https://reviews.bitcoinabc.org/D12674

Summary: ``` The tracepoints are further documented in docs/tracing.md and the usage is shown via the two newly added example scripts in contrib/tracing/. ``` Backport of [[bitcoin/bitcoin#22902 | core#22902]]. Backport of [[bitcoin/bitcoin#23907 | core#23907]]. Notes: - The amount is passed as string because our type can't be converted to int - CTxOut and CCoin accessors are used where applicable - I removed the casts that are both useless and even mixing stding and posix types Test Plan: Build with tracing, run bitcoind, then: sudo ../contrib/tracing/log_utxocache_flush.py ./src/bitcoind sudo bpftrace ../contrib/tracing/log_utxos.bt Reviewers: #bitcoin_abc, PiRK Reviewed By: #bitcoin_abc, PiRK Subscribers: PiRK Differential Revision: https://reviews.bitcoinabc.org/D12715

arnabsen1729 force-pushed the 2021-09-utxocache-usdt branch from 24b054b to 9fd9d30 Compare September 6, 2021 14:39

laanwj added the Utils/log/libs label Sep 6, 2021

arnabsen1729 force-pushed the 2021-09-utxocache-usdt branch from 9fd9d30 to 15b8ba5 Compare September 6, 2021 15:17

This was referenced Sep 6, 2021

tracing: add utxocache tracepoints 0xB10C/bitcoin#1

Closed

tracing: additional utxocache tracepoints 0xB10C/bitcoin#2

Closed

jb55 reviewed Sep 6, 2021

View reviewed changes

DrahtBot mentioned this pull request Sep 7, 2021

CCoinsViewCache code cleanup & deduplication #9384

Closed

0xB10C mentioned this pull request Oct 18, 2021

tracing: drop GetHash().ToString() argument from the validation:block_connected tracepoint #23302

Merged

[tracing] tracepoint for utxocache flushes

a26e8ee

Signed-off-by: Arnab Sen <arnabsen1729@gmail.com>

arnabsen1729 force-pushed the 2021-09-utxocache-usdt branch from 15b8ba5 to 786c300 Compare November 28, 2021 06:05

[tracing] tracepoints to utxocache add, spent and uncache

2bc51c5

Signed-off-by: Arnab Sen <arnabsen1729@gmail.com>

arnabsen1729 force-pushed the 2021-09-utxocache-usdt branch from 786c300 to 2bc51c5 Compare November 28, 2021 07:46

arnabsen1729 mentioned this pull request Dec 6, 2021

tracing: utxocache tracepoints follow-up 0xB10C/bitcoin#3

Closed

0xB10C added a commit to 0xB10C/bitcoin that referenced this pull request Dec 13, 2021

tracing: mention 'Lost X events' workaround

21fc279

See these comments: bitcoin#22902 (comment)

0xB10C added a commit to 0xB10C/bitcoin that referenced this pull request Dec 13, 2021

tracing: mention 'Lost X events' workaround

f1d51b3

See these comments: bitcoin#22902 (comment)

0xB10C added a commit to 0xB10C/bitcoin that referenced this pull request Dec 13, 2021

tracing: mention 'Lost X events' workaround

0407530

See these comments: bitcoin#22902 (comment)

arnabsen1729 pushed a commit to arnabsen1729/bitcoin that referenced this pull request Dec 17, 2021

tracing: mention 'Lost X events' workaround

29685d0

See these comments: bitcoin#22902 (comment)

0xB10C mentioned this pull request Dec 18, 2021

build: add systemtap's sys/sdt.h as depends for GUIX builds with USDT tracepoints #23724

Merged

arnabsen1729 mentioned this pull request Dec 30, 2021

tracing: utxocache tracepoints follow up for #22902 #23907

Merged

0xB10C mentioned this pull request Feb 16, 2022

test: USDT tracepoint interface tests #24358

Merged

bitcoin locked and limited conversation to collaborators Dec 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tracing: utxocache tracepoints #22902

tracing: utxocache tracepoints #22902

arnabsen1729 commented Sep 6, 2021

0xB10C commented Sep 6, 2021 •

edited

Loading

jb55 Sep 6, 2021

arnabsen1729 Sep 10, 2021

jb55 Sep 6, 2021

0xB10C Sep 6, 2021 •

edited

Loading

0xB10C Sep 6, 2021 •

edited

Loading

martinus Sep 9, 2021

0xB10C Sep 9, 2021 •

edited

Loading

0xB10C Sep 9, 2021

jb55 Sep 10, 2021

laanwj Nov 10, 2021

DrahtBot commented Sep 7, 2021

practicalswift commented Sep 8, 2021

theStack commented Sep 8, 2021

0xB10C commented Sep 10, 2021 •

edited

Loading

0xB10C commented Sep 11, 2021

laanwj commented Oct 1, 2021 •

edited

Loading

laanwj commented Nov 10, 2021 •

edited

Loading

0xB10C commented Dec 6, 2021

0xB10C commented Dec 6, 2021 •

edited

Loading

0xB10C commented Dec 6, 2021

jb55 commented Dec 6, 2021 via email •

edited

Loading

0xB10C commented Dec 6, 2021 •

edited

Loading

jb55 commented Dec 6, 2021

0xB10C commented Dec 6, 2021

jb55 commented Dec 6, 2021

tracing: utxocache tracepoints #22902

tracing: utxocache tracepoints #22902

Conversation

arnabsen1729 commented Sep 6, 2021

0xB10C commented Sep 6, 2021 • edited Loading

jb55 Sep 6, 2021

Choose a reason for hiding this comment

arnabsen1729 Sep 10, 2021

Choose a reason for hiding this comment

jb55 Sep 6, 2021

Choose a reason for hiding this comment

0xB10C Sep 6, 2021 • edited Loading

Choose a reason for hiding this comment

0xB10C Sep 6, 2021 • edited Loading

Choose a reason for hiding this comment

martinus Sep 9, 2021

Choose a reason for hiding this comment

0xB10C Sep 9, 2021 • edited Loading

Choose a reason for hiding this comment

0xB10C Sep 9, 2021

Choose a reason for hiding this comment

jb55 Sep 10, 2021

Choose a reason for hiding this comment

laanwj Nov 10, 2021

Choose a reason for hiding this comment

DrahtBot commented Sep 7, 2021

Conflicts

practicalswift commented Sep 8, 2021

theStack commented Sep 8, 2021

0xB10C commented Sep 10, 2021 • edited Loading

0xB10C commented Sep 11, 2021

laanwj commented Oct 1, 2021 • edited Loading

laanwj commented Nov 10, 2021 • edited Loading

0xB10C commented Dec 6, 2021

0xB10C commented Dec 6, 2021 • edited Loading

0xB10C commented Dec 6, 2021

jb55 commented Dec 6, 2021 via email • edited Loading

0xB10C commented Dec 6, 2021 • edited Loading

jb55 commented Dec 6, 2021

0xB10C commented Dec 6, 2021

jb55 commented Dec 6, 2021

0xB10C commented Sep 6, 2021 •

edited

Loading

0xB10C Sep 6, 2021 •

edited

Loading

0xB10C Sep 6, 2021 •

edited

Loading

0xB10C Sep 9, 2021 •

edited

Loading

0xB10C commented Sep 10, 2021 •

edited

Loading

laanwj commented Oct 1, 2021 •

edited

Loading

laanwj commented Nov 10, 2021 •

edited

Loading

0xB10C commented Dec 6, 2021 •

edited

Loading

jb55 commented Dec 6, 2021 via email •

edited

Loading

0xB10C commented Dec 6, 2021 •

edited

Loading