My experimental bitcoind-observer tool is a Bitcoin Core Prometheus metrics exporter utilizing and demonstrating the newly added tracepoints in Bitcoin Core.

The idea for a Bitcoin Core Prometheus metrics exporter based on the new tracepoints originated in a discussion on potential tracepoints between jb55 and laanwj in the Bitcoin Core issue 20981. Similar tools exist, for example, Jameson Lopp’s website statoshi.info. However, one downside with Statoshi is the patch-set that has to be rebased with every release. This can be cumbersome work if there are, for example, refactors in Bitcoin Core.

jb55: I was looking at this a long time ago. In theory you could get the same functionality as statoshi (stats gathering via prometheus) with https://github.com/cloudflare/ebpf_exporter Would be an interesting project. source

laanwj: I agree! This would be my preferred way to do this kind of reporting, to be honest. It'd flexible and built into the operating system. It has low impact on the code and low impact on performance. It could potentially work for all software.

One reason Statoshi never got merged is because it's just too much hassle to integrate and maintain a Bitcoin Core specific framework for this. Not because it's not a great thing to have. source

The tracepoints in Bitcoin Core have a semi-stable API. Upgrading applications that utilize the tracepoints to a new Bitcoin Core release should be easy. The tracepoints are flexible enough that we don’t instrument Bitcoin Core with an additional interface for a single application. The tracepoints can be used in debugging, review, for education about internals, monitoring, and data collection in small and large projects.

The bitcoind-observer tool attaches to multiple tracepoints in Bitcoin Core and transforms the tracepoint arguments to Prometheus metrics. It uses the bcc (BPF compiler collection) Rust crate to interact with BPF. For me, it also serves as a test bed for working with the tracepoints in Rust and for evaluating newly proposed tracepoints.

I build an experimental Bitcoin Core Prometheus metrics exporter based on the eBPF and USDT tracepoints I add in PR #22006 (https://t.co/1QGb7jgsnZ).https://t.co/RTLdhCLEf6

Grafana demo available on https://t.co/uEgE7d7Z7q

— 0xB10C (@0xB10C) July 27, 2021

And block connection timings during IBD and data about validated blocks/second, transactions/second, inputs/second, and sigops/second.https://t.co/syfJ0pEAJZ pic.twitter.com/sGuIv3NavH

— 0xB10C (@0xB10C) July 27, 2021

I’ve used the bitcoind-observer to provide feedback to Gleb for his work on Erlay by measuring the bandwidth consumption between erlay and non-erlay nodes. My measurements match Glebs calculations showing that we can archive the hoped bandwidth savings in this specific setup.