OpenSats Work-Log 10

Friday, July 31, 2026

This is a copy of the 10th work-log I sent to OpenSats for my LTS grant.

Disclaimer: Some information that is not (or not yet) meant to be published may have been redacted.

What did you work on?

In May, June, July of 2026 I’ve worked on peer-observer and it’s infrastructure to collect more data that will be helpful in analyzing network problems in the future. This includes, for example, the peer-observer archiver, but also snapshotting and archiving the contents of the honey-pot node address managers. With the help of an LLM agent, I was able compile a x86_64 bitcoind binary that matches the GUIX build release binary bit-by-bit on Nix. This is something I never thought was feasible to archive, and then attempted to re-create the full Bitcoin Core v31.0 release tarballs (including macOS, Windows, aarch64, …), and ended up matching it bit-by-bit. This was a big win for cross-toolchain reproducibility! I’ve also spent time working on simulations and a fix for a practical eclipse attack (see issue #34019) against Bitcoin Core. While I’ve made a lot of progress, this needs more work until I can open a PR with the fix. With the BIP-110 mandatory activation and the ecash hardfork coming up in August, I’ve spent some time brainstorming what monitoring and data collection would be useful for these. This resulted in me picking up a bunch of features that were on my wishlist for fork-observer. With more and more activity on the BNOC forum, I’ve decided to finish my tooling for BNOC and delvingbitcoin archives and mirroring. I’ve also spend some time setting up https://data.bnoc.xyz and have been uploading data to the public and hidden BNOC data share.

Publications

  • TCP hole punching for Bitcoin nodes behind home NATs?: At CoreDev Barcelona, an idea for using TCP hole punching to connect Bitcoin nodes behind home-NATs came up. To share the idea and continue the discussion, I posted this to delving. While it’s an interesting idea, I don’t think it’s time has come yet.
  • Bitcoin Optech Newsletter #406 Recap Podcast: I joined Murch, Gustavo and Mike on the Bitcoin Optech Newsletter #406 Recap Podcast to talk about TCP holepunching and peer-observer P2P monitoring tools, which both were mentioned in the Optech Newsletter #406.
  • Matching Hashes: Reproducing the Guix-built Bitcoin Core release binary with Nix: wrote about my adventure of building a Bitcoin Core release binary using Nix that match the official Guix ones. At this point I didn’t know that I’d manage to build all release artifacts matching bit-by-bit a few weeks later.
  • While visiting, I gave a short talk at the Autonomous University of Barcelona to update the students and faculty members about my current projects.

BNOC

peer-observer

peer-observer (tooling)

The peer-observer tooling extracts and processes data from Bitcoin nodes. I’ve been lucky to have a growing number of contributors to peer-observer, so much of my time spend on it was review and testing, as well as integrating the changes into the existing infrastructure (see below).

archiver: The peer-observer archiver archives events to disk for future processing. The main work for me was to review and improve an implementation started by Octavio Lucca.

  • #429: fix: archiver build.rs use GIT_HASH or fallback
  • #439: archiver: tracking and followup issue
  • #450: archiver: remove manifest.toml writing
  • #451: archiver: rework ArchiveHeader and implement an iterator-based ArchiveReader
  • #444: remove: drop duplicate dev dependencies from archiver
  • #445: move: archiver and replayer into tools/archive as sub-tools
  • #443: add(replayer): sequence diagrams, CSV export, and P2P message filtering
  • #455: refactor(replayer): use clap and log, return errors instead of unwrap
  • #464: refactor(archive): split replayer into library module and use anyhow
  • #463: fix: correctly read protobuf message varints in archives

ipc-extractor: The ipc-extractor extracts from Bitcoin Core’s new IPC interface. This needed a bit of polishing after the initial implemenation , which was mainly contributed by xyzconstant.

  • #436: ipc-extractor: tracking issue and followups
  • #452: Add metrics for IPC extractor
  • #453: Improve disconnection handling in ipc-extractor
  • #457: ipc-extractor: Use bitcoin-capnp-types as capnp schema provider
  • #441: Extend ipc-extractor to test bitcoin/bitcoin#29409

reducing intermitting integration test failures: The peer-observer tooling has extensive integration test coverage. Some of the integration tests would fail intermittently and deadmanoz started fixing these. I reviewed this work.

  • #423: fix(rpc-extractor): use OS-assigned port in integration tests
  • #424: fix(metrics): use OS-assigned port in integration tests
  • #425: test(rpc-extractor): replace brittle getchaintxstats constants with relational check
  • #430: fix(p2p): force tx-inv trickle in integration test via setmocktime
  • #447: ci: free disk space on ubuntu runners
  • #449: test(shared): add post-IBD tip wait helper
  • #456: intermittent integration test failure: test_integration_logger* log does not contain line *
  • #446: rustc-LLVM ERROR: IO failure on output stream: No space left on device
  • #458: test(logger): wait on the logger instead of fixed sleeps
  • #468: Intermitting failure in test_integration_logextractor_saw_new_header
  • #469: fix: Remove redundant peer_id assertion in integration test

maintenance: The project requires too keep up-to-date with e.g. Bitcoin Core releases, dependencies, and more.

  • #427: Upgrade corepc crates (with custom patches)
  • #435: Update custom corepc branch to support bitcoind v31.0 download (and remove manual download from CI)
  • #478: fix: linter warnings
  • #466: update: CI Nix channel to version 26.05
  • #467: fix: CI workflow permissions can be reduced
  • #437: clean: move tokio-util to shared and re-export
  • #438: cleanup: In tests, pass structs for the checks we want to enable instead of bools
  • #442: fix: metric description for distinct source ASN

ideas: A few new ideas that came up.

  • #426: tools: P2P traffic contract testing and pattern matching
  • #470: metrics-tool: AS lookups for IPs
  • #440: metrics idea: median timestamp age per addrman table (new/tried) per node

in-progress:

  • #459: Add addr-entries spammer detection to the alerts tool
  • #465: Add dashboards for compact block reconstruction metrics

infra-library

The peer-observer infra-library defines a way to run Bitcoin nodes with peer-observer and more monitoring tools attached.

profiling evaluation: The peer-observer node hosts run continues profiling along with the nodes. This allows us to track down where CPU time was spend in the bitcoind process if there are problems or e.g. a DoS attack. I was using parca.dev for this for a while, but this wasn’t optimal for multiple reasons (ephemeral storage, frequent profiling crashes, …). I looked into both wprof and samply as alternatives. I documented my evaluations in the following GH issues.

  • #149: Evaluating wprof as an alternative to parca.dev
  • #154: Evaluating samply as an alternative to parca.dev
  • #169: Use samply for continues profiling instead of parca.dev
  • #207: fix: don’t let samply profiling resurrect a crashed bitcoind

data collection: While analyzing e.g. the Bitprojects eclipse attack, I noticed that historical address manager snapshots would be useful. So I’ve setup collection and archival of both peers.dat files and the output of getrawaddrman.

  • #160: add: daily getrawaddrman snapshots
  • #173: nodes: Keep weekly peers.dat snapshots on all nodes
  • #174: change: enable addrman snapshots by default
  • #208: add: weekly peers.dat snapshots
  • #182: add: expose addrman snapshots via web servers

bitcoind work: The peer-observer infrastructure can different version of the bitcoind binary with different configurations. Added some logging, extra sanitizer suppressions, updated default version, and more.

  • #159: update: default bitcoind commit to recent master
  • #138: bug: build bitcoind with human readable debug symbols again
  • #161: add: run bitcoind with sanitizer suppressions
  • #171: add: enable leveldb debug logs
  • #190: Do-not-restart on crash did not work for OOM-kill
  • #166: change: test CJDNS again

dependency updates: Tools and OS updates as well as more work on automated update PRs.

  • #140: Update flake input: b10c-nix
  • #153: Update flake input: b10c-nix
  • #155: update: nixpkgs flake input
  • #192: upgrade: actions checkout to v7
  • #194: Configure Dependabot for Nix package ecosystem
  • #164: update: NixOS 26.05
  • #195: build(deps): bump b10c-nix from c32110d to 9983ffc
  • #198: Add GitHub Actions to Dependabot configuration
  • #199: Update flake input: b10c-nix

integration of peer-observer tooling: Integration of the peer-observer tools requires changes and maintenance. For example, supporting a new RPC in the whitelist or enabling the ipc-extractor by default on all nodes

  • #143: Support for alerts and archiver tool
  • #156: allow estimatesmartfee RPC
  • #157: add: enable ipc-extractor by default
  • #158: add: add estimatesmartfee RPC to whitelist
  • #172: add: support for IPC-extractor metrics
  • #188: RPC User forkobserver not allowed to call method waitfornewblock
  • #189: Add ‘waitfornewblock’ to rpcwhitelist for forkobserver

ideas: new ideas based on problems observed

  • #213: Detect deadlocks of nodes
  • #214: Detect internet outages of nodes

0xB10C/fork-observer

The fork-observer tool connects to multiple backends/nodes (Bitcoin Core, Knots, electrum, mempool-space, btcd, …) on different networks (mainnet, signet, testnet, …) and shows a blocktree visualization and what the nodes consider as chain tip. This is the perfect tool for showing an BIP-110 soft-fork chain-split. I prioritized working on this in July to get it ready for the two forks in August.

backend:

  • #130: change: bitcoincore-rpc -> corepc-cilent: The bitcoincore-rpc crate is outdated and the corepc-client is a good drop-in replacement for it.
  • #131: add: basic integration test for Bitcoin Core node: helps to make sure nothing breaks drastically.
  • #132: add: use waitfornewblock on Bitcoin Core nodes: This is a long-pool RPC that returns when the node knows about a new block. This means faster notifications about tip changes.
  • #140: Add ‘waitfornewblock’ to docs & whitelist example
  • #133: fix: ensure a schema is set for rpc_host
  • #138: Expose stale headers and stale blocks in API: This allows fetching this information from e.g. bitcoin-data/stale-blocks
  • #141: add: support for mempool as a backend: There are a few official and community mempool-space instances that are useful to tap into to have extra network coverage of forks.
  • #143: add: support for a remote fork-observer backend: combining multiple fork-observer instances together is an easy way of running a single “reference” instance of fork-observer with many nodes and good stale-block and fork coverage.
  • #145: Activity log of changes to nodes: As of writing, this is still a work-in-progress. Merging this would allow to replay for example reorgs and chain-splits.
  • #146: add: block-dn backend: An additional viewpoint on the network based on what the block delivery network considers to be the active chain tip. Additionally, can be used to download blocks if all other nodes are pruned.
  • #147: refactor: node -> backend and split into multiple files: the node.rs file got to big. Refactoring it helps to keep this maintainable.

frontend:

  • #134: Frontend fixups and improvements: A few quality of live changes that e.g. make the headertree deterministic
  • #136: change: modernize frontend: 3D blocks and a better node overview.
  • #137: Show next block based on stratum jobs: Using stratum jobs, we can show on which branch of a fork miners are mining on. This initial attempt was based on my stratum-observer websocket.
  • #144: add: show next block based on stratum jobs (stratum.work edition): Same as above, but based on bborst’s stratum.work data stream.
  • #139: Support for friendly network URLs: redirects fork.observer/mainnet to mainnet and similar for signet
  • #142: add: per-network height countdown: a countdown for special block heights: soft-fork activations, halvenings, …
  • #151: change: show the full user agent (if possible): previously, we might not have shown the full user agent of e.g. a knots node.

0xB10C/nix

While most of the package updates in my nix-packages and modules repository run automatically, I sometimes need to fixup or add new features for changed packages.

updates: These are package or module updates that needed special attention.

  • #361: peer-observer: 2026-05-12 automated package update
  • #362: Update the peer-observer package and module to support the archiver and alerts tool
  • #371: update: ckpool v0.9.9 -> v1.1.1
  • #391: peer-observer: 2026-06-03 automated package update
  • #411: fork-observer: 2026-06-27 automated package update

package additions:

  • #369: add: wprof package: evaluated this for continues profiling
  • #380: init: samply at unstable-2026-05-13: evaluated this for continues profiling

misc fixes, updates and additions:

  • #364: fix: correctly create archiver tempfiles 
  • #367: misc Github metdata data mirror and backup things
  • #376: ci: don’t –build in update job
  • #377: upgrade: NixOS 26.05
  • #378: add peer-observer ipc-extractor support
  • #388: change: passthrough peer-observer static files
  • #399: fix: rm: Argument list too long
  • #418: discourse-archive: automatically delete .metadata.json from time to time to do a full re-sync
  • #426: remove: macos-intel CI run with NixOS unstable
  • #427: add: new fork-observer options
  • #431: fix: discourse-archive build on nixpkgs-unstable

Bitcoin Core

bitcoin/bitcoin

PR:

  • #35285: bench: add benchmark for GetMappedAS(): Added a benchmark for IP->AS lookups Bitcoin Core does. At CoreDev, it was mentioned that this part of the code might be refactored at some point, so having performance measurements beforehand is useful.

netgroup issue simulations and fix:

  • #34019: RFC: randomize over netgroups in outbound peer selection: handcrafted detailed simulations that showed the issue based on historical data that I was able to source from my nodes and fellow contributors. This is also the basis for a fix I’ve been working on, and allows to check if and what the fix actually fixes, once finished.

guix builds for PRs:

  • #35197: guix: add -Wl,–icf=safe to darwin build
  • #35261: guix: disable LTO in GCC
  • #35821: guix: followups to #35537

review & feedback:

  • #35252: net: send decoy transactions via private broadcast
  • #35368: tracing: add block header and compact block tracepoints
  • #35298: chainstate leveldb is rewritten every 30 or 60 minutes: provided disk io statistics for this
  • #35369: Structured logging output
  • #35550: net_processing: fix BIP152 first integer interpretation: setup monitoring for this and
  • #35558: p2p: Prefill compact blocks: left some initial review comments
  • #35578: net: don’t self advertise tor exit node ip addresses in outbound connections: Left a detailed Concept ACK

bitcoin-core/asmap-data

Participated in two asmap runs.

  • #51: Collaborative Launch 1780588800 (Thursday June 4, 2026 16:00 UTC)
  • #60: Collaborative Launch 1783008000 (Thursday July 2, 2026 16:00 UTC)

I’ve also published two Rust crates for working with asmap:

bitcoin-core/guix.sigs

Uploaded my release artifact sigs built with Nix instead of GUIX for v31.1 and its release candidate.

  • #2465: add v31.1rc1 (using nix)
  • #2518: add: v31.1 using nix

0xB10C/bitcoind-gunix

I’ve picked up work on this four year old side-project and with the help of LLMs & agents, I was able to not only match the GUIX bitcoind binary in Nix, but was also able to match the full bitcoin-31.0-x86_64-linux-gnu.tar.gz tarball and later even the full release. The future of this is uncertain, as it eats quite a bit of usage of my LLM subscription, but I might do it again for the v32.0 release.

  • #6: Next steps & follow-ups
  • #7: build matching bitcoin-31.0-x86_64-linux-gnu.tar.gz tarball
  • #8: build matching bitcoin-31.0-aarch64-linux-gnu.tar.gz tarball
  • #9: Update to NixOS 26.05
  • #10: cross-to-self of x86_64 and drop CRC patching (full x86_64 debug build reproducible)
  • #11: aarch64: byte-identical .dbg files - drop the CRC patch, add .#debugTarballAarch64
  • #12: Reproducible riscv64, armhf, and powerpc64 tarballs
  • #13: x86_64 and aarch64 macOS reproducible
  • #14: win64 reproducible
  • #15: clean up and structure the nix files
  • #16: re-export source archives & make nix build build the full .#sha256sum output
  • #17: fix build on aarch64 host

bitcoin-data/stale-blocks

A dataset of stale blocks observed on the Bitcoin network. While addition is partly automated, it still requires manual review, and (currently often manual) addition of the full stale block. I’ve looked the following dataset additions:

  • #100: Automated: add recent stale block(s) 2026-05-14
  • #102: add: stale block at 950517
  • #103: Automated: add recent stale block(s) 2026-05-23
  • #105: Automated: add recent stale block(s) 2026-05-26
  • #106: automated: add recent stale block(s) 2026-05-26
  • #107: Automated: add recent stale block(s) 2026-06-25
  • #108: Automated: add recent stale block(s) 2026-06-26
  • #111: Automated: add recent stale block(s) 2026-07-02
  • #113: Automated: add recent stale block(s) 2026-07-24
  • #114: Automated: add recent stale block(s) 2026-07-25
  • #115: Automated: add recent stale block(s) 2026-07-26
  • #116: Automated: add recent stale block(s) 2026-07-27

0xB10C/mainnet-observer

Only basic maintenance for mainnet-observer.

  • #169: chart: coinjoin number and volumes on-chain
  • #171: fix: clippy warning about into_iter()
  • #172: update: chart BIP-54 Coinbase Locktime set tags
  • #174: missing-chart: Rewrite html to markdown to fix hugo build failure ☹
  • #175: chart: Combine transaction input-output count charts
  • #176: chart: Direct address reuse
  • #177: Update thumbnails 2026-06-26
  • #178: Update chart thumbnails around 2026-11-15

0xB10C/discourse-archive

This is an archive and mirroring tool for discourse based forums. An archive without anyone looking at it isn’t worth much, as it might be outdated or incomplete. A mirror can help there. I added the functionality and set up the mirroring on e.g. https://mirror.b10c.me.

  • #2: add: a mirror that produces static HTML pages
  • #3: Make aware of missing posts in mirror
  • #4: Show gaps in post history
  • #5: Post rendering updates

NixOS/nixpkgs

I’ve been maintaining and updating a few Nix packages. This includes btcd and Bitcoin Core.

  • #533154: btcd: 0.25.0 -> 0.26.0
  • #545455: btcd: 0.26.0 -> 0.26.2
  • #539651: bitcoin: 31.0 -> 31.1
  • #538128: nixos/discourse: discourse-assets-2026.1.4 build fails with “mini_racer_extension.so: undefined symbol: _ZN6icu_788ByteSink15GetAppendBufferEiiPciPi”: Ran a computationally expensive git bisect to point out where a package build breaks. This was worthwhile for me, as someone else ended up fixing the discourse package I use to run bnoc.xyz. 🤝

0xB10C/github-metadata-mirror

Added a plain markdown rendering of issues and PRs to allow better consumption by LLMs during training and interference.

  • #17: Add optional plain markdown rendering of issues and PRs
  • #18: fix: order of contributor PRs and Issues
  • #19: add: support for project v2 things

bitcoinfibre/bitcoinfibre

I’ve been running a Bitcoin Fibre node in collaboration with Localhost Research in my peer-observer nodes set and peer-observer caught two bugs which I reported upstream. Both were fatal with one being a OOM crash and the other a deadlock.

  • #22: v30.0 Fibre OOM crash
  • #23: v31.0 Fibre ring-buffer logging deadlock

Review of plans for last quarter

Commenting on my Plans for Next Quarter from Report 9:

> peer-observer: continue reviewing PRs from contributors and pick up some of the issues I’ve hadn’t had time to work on during the last months.

I worked down the PR backlog quite a bit during May and June. There’s still quite a few open issues (and new ones) to tackle. In July, peer-observer hasn’t been a big priority.

> BNOC: Over the past few months, the BNOC forum has attracted regular visitors and posters. I want to continue building this out, posting there, moderating, and working on adjacent topics like the BNOC data share. A strech goal includes an unofficial, optional, but well documented banlist for IP addresses of mass-connectors, spy-nodes, and similar that exist on the network today.

BNOC forum has been going strong. The data.bnoc.xyz data-share too! There was some activity in the banlist repository, but this needs more time and care.

> I want to work on simulations for [https://github.com/bitcoin/bitcoin/issues/34019](https://github.com/bitcoin/bitcoin/issues/34019), as I happen to have some addrman snapshots that are useful. The simulation would also show that a potential fix mitigates the problem.

Did work a lot on the simulations for this and also started working on a fix, but didn’t have enough time and focus to finish it. This will have to continue over the next months. It probably also not make before feature freeze for v32.0. Since it’s a rather big change, maybe it’s better to have it in master for a while to get more testing on it. Can also be backported.

> IPC based tracing interface: I hope to get to benchmarking the feasibility of an IPC based Events/tracing interface that allows us to decide if we want to move forward with it or not.

I did not get to this. Still think it’s interesting and worthwhile to explore, but fixing bugs and monitoring steady state was more important than adding a big, new feature.

> p2p-observer: I want to bring this into a runnable, and well rounded place so we can start capturing block propagation measurements.

Neither did I manage to put time into this. This is a bit higher on my list for next month though.

> work on a benchmark for ASMap lookups in Bitcoin Core and do the same for a Rust asmap library I’ve been thinking about. This library will come in handy for peer-observer and p2p-observer in the future. Likely adding AS information (name, type) to it as well.

Did work on ASMap benchmarks in Bitcoin Core and finished two libraries for asmap lookups and AS info lookups! Did not integrate it into peer-observer nor p2p-observer, but that will happen once I focus on these tools again.

> discourse-archive: set up an archive and mirroring of delvingbitcoin and the BNOC forum.

Did this! So much useful information on Delving and BNOC. Would be a shame if that was lost or not indexable by LLMs for training.

Plans for Next Quarter

  • fork monitoring: With the upcoming BIP-110 mandatory signaling and ecash hard-fork I plan to run monitoring software and livestream these. I’ve been doing livestreams for e.g. Taproot activation, the last halvening, and other events before. These are usually well-received and provide a way of revisiting what happened during these events.
  • fixing #34019: My plan is continue working on a (partly) netgroup-randomized outbound connection selection for Bitcoin Core. This would close a practical (but expensive) way of eclipsing a node.
  • explore block-propagation dataset: KIT DNS shared a dataset with block propagation timings with me. This covers the last few years and likely allows to extract a lot of useful information about block propagation on the Bitcoin network. A few examples are propagation of blocks with non-standard transactions, propagation of blocks with not-previously-seen-transactions. propagation-by-pool, and propagation-by-block-size, as well as propagation-by-block-validation-speed.
  • p2p-observer: To collect more block propagation data going forward, having a measurement tool that supports, for example, compact blocks, is needed. The p2p-observer tool aims to be that, and while it showed that it works during the slow-blocks-on-signet testing, it needs a lot of polishing for mainnet deployment.
  • peer-observer: There are a few features I want to have before tagging version 1.0. I’ll probably spend some time on them. Additionally, infrastructure maintenance and collecting and archiving more data (e.g. multiple getpeerinfo snapshots per day) are planned.
  • likely some work on fork-observer, mainnet-observer and other projects I’ve been working on.
  • review and collect data for a few Bitcoin Core PRs. One PR that comes to mind immediately is https://github.com/bitcoin/bitcoin/pull/35558… Possibly also moving the IPC-tracing or structured logging issues forward.
  • Will continue to work on BNOC. Maybe a wiki next. Additionally, revisiting the banlist might be good.

Use of Funds

As mentioned in Work Log 9, I’m cutting expenses to have a bit more money left over for myself after expenses. This quarter, I’ve spend some time e.g. decommissioning a Hetzner dedicated box I’ve been renting for the last five years. While this means a bit less “cloud” compute and storage available for development and data-science work, it also saves a good chunk of money. Additionally, I’ve started to retrieve my gzipped, historical debug.logs from backup cloud storage and have been re-compressing them with a very high level of zstd-compression. This roughly halved the compressed size, saving me money in the long run. I’ve also made use of a CoreDev accommodation sponsorship.

Use-of-funds:

  • living expenses (i.e. a modest salary in my region)
  • server, hosting, and cloud storage expenses
  • hardware
  • travel expenses (e.g. conferences & CoreDev)
  • office rent
  • accounting expenses
  • paying taxes
  • insurances (health, liability, …)
  • Claude Pro subscription


My open-source work is currently funded by an OpenSats LTS grant. You can learn more about my funding and how to support my work on my funding page.

Creative Commons License Text and images on this page are licensed under the Creative Commons Attribution-ShareAlike 4.0 International License

Previous

Image for OpenSats Work-Log 9

April 30, 2026

OpenSats Work-Log 9

This is a copy of the 9th work-log I sent to OpenSats for my LTS grant.