Compare commits

...

41 Commits

Author SHA1 Message Date
Gud Boi be108ccc32 Draft the `/tipc` multiaddr protocol
Propose a fixed-width service endpoint carrying the TIPC type,
instance and publication scope, with one canonical structured value
that generic multiaddr parsers can compose normally.

Retain the kernel-standard `tipc` name while using “Cluster Domain
Sockets” as explanatory terminology. Document the binary and text
encodings, WireGuard composition, deployment-management boundary,
upstream sequence, test vector and open maintainer questions.

Prompt-IO: ai/prompt-io/opencode/20260819T003328Z_53516b09_prompt_io.md

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-19 15:03:00 -04:00
Gud Boi ff2b20b7d1 Harden the `tipc` two-host smoke test
Turn the physical-host sketch into an operator runbook covering
cluster identity, interface and bearer setup, link validation,
failure/rejoin testing, diagnostic capture and cleanup.

Explain the cluster-domain-socket analogy and identify a future
`pyroute2` TIPC codec as the path from manual `tipc(8)` commands to
the same netlink management stack planned for WireGuard.

Authorize `host_a_srv` by its stable import name so direct script
execution does not expose only `__main__` while host B requests the
callable's actual `NamespacePath`.

Prompt-IO: ai/prompt-io/opencode/20260819T003327Z_53516b09_prompt_io.md

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-19 14:56:48 -04:00
Gud Boi caf9c3d6d3 Make the `tipc` CI leg blocking
The refreshed PR matrix passes on Ubuntu with the TIPC kernel
module loaded, along with the TCP, UDS and macOS legs. Remove the
temporary `continue-on-error` expression so future TIPC
regressions block CI.

Prompt-IO: ai/prompt-io/opencode/20260819T003326Z_53516b09_prompt_io.md

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-19 14:52:49 -04:00
Gud Boi 53516b094c Abort `TIPC` topology streams on overflow
Keep `_stream_name_events()` non-blocking so a slow memory-channel
consumer cannot back up the kernel topology queue. Raise
`TIPCNameEventOverflow` and end the subscription rather than drop a
transition or let the socket reader stall. Discovery consumers must
then resubscribe and rebuild their name-table view.

Also,
- document topology semantics and scope with Linux references
- diagram the `.connect()`/`.getpeername()` withdrawal schedules
- explain the child-service and callable requirements in the
  two-host example

Review: PR #493 (copilot-pull-request-reviewer[bot],goodboy)
https://github.com/goodboy/tractor/pull/493

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-18 14:20:47 -04:00
Gud Boi c1501a36d5 Fix `TIPC` two-host service example
A registrar root does not register itself in its own actor-name
registry, so host B could never discover the advertised `host_a`.
Boot that service as a child actor under the `TIPC` registrar
instead.

Import and pass the enabled `echo` callable to `.open_context()`;
the prior module-path string could not produce a `NamespacePath`.

Review: PR #493 (copilot-pull-request-reviewer[bot],goodboy)
https://github.com/goodboy/tractor/pull/493

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-18 13:45:54 -04:00
Gud Boi be6f9e86d1 Make `TIPCNameEvent` delivery lossless
Stop labeling topology events with caller-supplied scope that the
kernel never reports. Event addresses now carry an explicit unknown
scope instead of fabricated reachability.

Apply memory-channel backpressure rather than silently dropping
publish/withdraw transitions, and close the stream after delivering
the terminal event from a finite subscription.

Review: PR #493 (copilot-pull-request-reviewer[bot],goodboy)
https://github.com/goodboy/tractor/pull/493

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-18 13:44:17 -04:00
Gud Boi 145782d38c Harden `TIPC` socket setup races
Reject TIPC availability outside Linux before probing the fallback
socket-family integer, which can alias an unrelated family on
another OS.

Keep dialled sockets under setup ownership through transport
construction, then reuse the constructor's tolerant peer
observation. A peer withdrawing after `.connect()` can no longer
trigger a second raw `getpeername()` or leak setup resources.

Review: PR #493 (copilot-pull-request-reviewer[bot],goodboy)
https://github.com/goodboy/tractor/pull/493

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-18 13:34:11 -04:00
Gud Boi d52c78c106 Key `TIPCAddress` instances by actor UUID
TIPC service names span the cluster while PIDs remain host-local.
Hashing only `(name, pid)` could therefore make same-named actors
on different hosts silently share one round-robin service name.

Derive the live-runtime seed from `Aid.uid` so the actor UUID
separates those names while keeping each identity reproducible.
Pin both properties with a deterministic regression test.

Review: PR #493 (copilot-pull-request-reviewer[bot],goodboy)
https://github.com/goodboy/tractor/pull/493

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-18 13:28:02 -04:00
Gud Boi a19a639ddf Tighten `TIPC` address-shape dispatch
Restrict proto-key matching to numeric 3- or 4-element
descriptors so a UDS directory named `tipc` stays UDS.

Route `/tipc` parsing through `TIPCAddress.from_addr()` to
normalize zone scope and report malformed input clearly. Also
align UDS unwrapped metadata with its actual `(str, str)` shape.

Keep the TIPC test module portable by importing `SOL_TIPC` from
the backend's UAPI fallback instead of the host `socket` module.

Review: PR #493 (copilot-pull-request-reviewer[bot],goodboy)
https://github.com/goodboy/tractor/pull/493

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-18 04:50:00 -04:00
Gud Boi 1298ba945f Refresh the `TIPC` landing handoff
Record #493's current draft head, #492's advanced planning
tip and the exact restack sequence before final landing.

Also,
- keep the unrelated `pformat` red-test/fix pair ordered for
  its standalone `main` PR
- distinguish the 17 substantive arc commits from the
  local-cache ignore
- make the in-repo handoff authoritative over agent memory
- preserve digest/drift checks for already-authorized forge
  writes

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-17 19:57:04 -04:00
Gud Boi c7ae606504 Clarify the `wg`-over-TIPC motivation, harden caveats
Follow-on to 4aa7a890 now that the encryption premise is
corrected: reframe *why* we want a `wg` mesh under TIPC (#502)
rather than leaving a "wg adds the crypto TIPC lacks" reading
lying around, since that reading is flat wrong.

The motivation is different but still real,
- TIPC's keys are **symmetric + pre-shared**, so distribution,
  rotation and revocation are all on the operator; `wg` brings
  public-key identity and a handshake.
- `wg` is an overlay *every* tpt can sit on (tcp now, quic
  later), not a TIPC-only mechanism.
- NAT traversal / roaming, which raw TIPC bearers have no story
  for at all.

Which to actually default to wants **benchmarking** — native
crypto skips a tunnel hop and may win for LAN-local clusters.

Also lean much harder on the udp-bearer-only caveat in the
handoff doc; it's the one that bites. A wg iface is L3/`tun` w/
no L2 addr, so there's no device for `media eth` to name — which
means #378's "ethernet bearers pair most excellently w/ wg
tunnelling" framing does NOT hold: on a given link the L2 path
and the wg path are mutually exclusive. Any design assuming both
is broken from the start.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-17 17:44:58 -04:00
Gud Boi 4aa7a890cb Document TIPC-over-`wg`, add a handoff doc
Anticipating gh #502 — TIPC over a WireGuard mesh as our go-to
multihost tpt deployment — plus a cold-start handoff for whoever
(or whatever) picks this up next.

The wg deats, both verified locally,
- a wg iface is L3/`tun` (`POINTOPOINT,NOARP`, `link/none`, no
  L2 addr) so TIPC's `eth` media **cannot** bind it; the udp
  bearer is *mandatory* over wg, not merely an alternative.
  Also its ~1420 MTU sits under ethernet's 1500.
- the composed deployment maddr is
  `/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>`.
  XXX note the tipc segment has NO locative part unlike tcp's
  inner `/ip4/../tcp/..` — a service name is
  location-independent, so wg carries routing and tipc carries
  identity. That's the argument for one `/tipc` proto w/ a
  structured value in the #498 spec proposal.

XXX ALSO correcting a premise: TIPC is **not** unencrypted. It
ships AES-GCM crypto (`tipc node set key`, linux 5.9+) w/
cluster/master/per-node keys + rekeying. Those keys are
symmetric+pre-shared tho, so wg is still preferred for
public-key identity, NAT traversal, and one overlay every tpt
can share.

`01_tipc_HANDOFF.md` is deliberately provider-neutral: env
setup, the hard-won kernel facts table, the two closed design
decisions (+why), what landed, the pre-land TODOs and the repo's
working conventions.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-17 17:38:05 -04:00
Gud Boi 1802641e41 Ignore the `gish` `github/` local issue cache
`gh issue create` caches bodies under the
`<backend>/<repo>/<kind>/<num>.md` path — and that dir is named
for the *service*, not the CLI, so the existing `gitea/` + `gh/`
entries never covered it. Filing the `tipc` follow-ups (#495-501)
is what surfaced it.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-17 17:37:24 -04:00
Gud Boi 2d082373bd Add a `--tpt-proto=tipc` CI matrix leg
Plan 01 §7.3's last item. The module ships w/ the standard
ubuntu kernel package but is NOT loaded by default, so the leg
gets a gated `sudo modprobe tipc` step plus a verify that
asserts `TIPCAddress.is_available()` before the suite runs —
i.e. a missing module fails w/ an actionable line instead of a
few hundred connect timeouts.

Deats,
- `tipc` added to the `tpt_proto` matrix axis, and excluded on
  `macos-latest` bc `AF_TIPC` is a linux-kernel proto that
  doesn't exist on darwin at all.
- `continue-on-error` is scoped to just this leg via
  `${{ matrix.tpt_proto == 'tipc' }}` — GH's runners have never
  been asked to `modprobe` for us, so it lands NON-blocking
  until it's had a few green runs. Drop the gate then.
- if the runners do refuse, the documented fallback is a
  container job w/ `--cap-add NET_ADMIN`.

Cross-node (bearer) TIPC still can't be CI'd; that stays the
manual smoke test in `examples/multihost/tipc_cluster/README.md`.

Partially addresses #420.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-17 14:24:44 -04:00
Gud Boi 1526372e37 Add a `tipc` guide page, roster it in the docs
Plan 01 §8's docs deliverable: `docs/guide/tipc.rst`, leading
w/ the `tipc nametable show` demo as the plan asked.

Frames the backend by what makes it different — every other tpt
gives you a pipe and leaves discovery to the registrar, whereas
TIPC's service names live in a kernel-maintained cluster-wide
name table, so a `.bind()` IS registration and a `.connect()` IS
the lookup. Then: push-based discovery via
`open_topology_events()`, scope-as-`.bindspace`, bearer setup
for spanning hosts, and the gotchas.

Also,
- roster it in `guide/index.rst` (prose list + toctree)
- `api/ipc.rst`'s transport line said `['tcp' | 'uds']` and
  described only 2 unwrapped-addr shapes; now mentions `tipc`
  and its proto-keyed `('tipc', stype, instance, scope)`.

Verified w/ a full `sphinx -b html` build: succeeded, page
renders, internal refs resolve.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-15 20:08:12 -04:00
Gud Boi b14332017d Add the `tipc_cluster` example set
Plan 01 §8's deployment deliverable, under
`examples/multihost/` (like the `wg_lan` set) since these need
the `tipc` kernel module — and, for the 2-host pair, a live
bearer — so they can't satisfy `test_docs_examples.py`'s
"walk `examples/` and assert rc == 0". `'multihost'` is already
in that test's exclusion list.

- `single_host.py` — boots a 4-actor tree and shells out to
  `tipc nametable show` before/during/after. Watching 4 service
  names appear in the KERNEL's table and vanish on teardown,
  entirely outside any `tractor` API, is the single best demo
  this backend has.
- `watch_nametable.py` — the same story push-based, via
  `open_topology_events()`: live `[+] published` / `[-]
  withdrawn` as actors come and go.
- `host_a_srv.py` + `host_b_client.py` — the cross-node pair.
  Note what's absent from both: any IP, hostname or port. Both
  sides name the same *service* and the kernel routes it.
- `README.md` — the manual smoke test (bearer setup, `tipc link
  list` verify) per §7.3, plus the gotchas: silent crosstalk,
  graceful-close-looks-like-`ECONNRESET`, the interim maddr.

Both single-host scripts were RUN against a live kernel and
their real output is what's pasted in the README.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-15 20:07:24 -04:00
Gud Boi 33a040b312 Add `open_topology_events()`, push-based discovery
Second half of layer B: an `@acm` yielding a `trio` receive-chan
of `TIPCNameEvent` fed by a nursery-spawned reader on a
`SOCK_SEQPACKET` conn to `TIPC_TOP_SRV`.

This is the bit that makes #378's "end game cluster proto" claim
real — the kernel *tells* us when any actor anywhere in the
cluster publishes or withdraws a service name, so a registrar
never has to poll `find_actor()`. Groundwork for the push
registry in `discovery/_registry.py` (gh #184, #216).

Deats,
- `filt` selects granularity; `TIPC_SUB_SERVICE` is one event
  per *name*, `TIPC_SUB_PORTS` one per *publisher* — the latter
  makes the §2.3 duplicate-name/round-robin crosstalk case
  externally observable, which is how a push-registry could
  ever detect it.
- a full event buf **drops** w/ a loud warning rather than
  blocking the reader; stalling it just backs up the kernel's
  own queue and loses the event less visibly.
- `SOCK_SEQPACKET` is fine here bc this sock never goes through
  `MsgpackTransport` — the contract's "`SOCK_STREAM` only" rule
  is about `MsgTransport` streams, not this.

XXX teardown order is load-bearing: cancel the nursery BEFORE
closing the fd. `.close()`ing out from under a pending
`.recv()` races — trio's retry can land on an already-freed fd
and raise a bare `OSError(EBADF)` instead of the
`ClosedResourceError` the reader guards for, which then escapes
the nursery as an eg.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-15 01:39:37 -04:00
Gud Boi e269bbf871 Add the `TIPC_TOP_SRV` name-event wire codec
First half of plan 01 §5.2 (layer B): the `struct` layouts and
the `TIPCNameEvent` type for the kernel's *push-based* name
table, w/o any socket plumbing yet. Pure-python, so it tests
w/o a loaded `tipc` module.

Deats,
- `_SUBSCR_FMT = '=5I8s'` (28B `struct tipc_subscr`) and
  `_EVENT_FMT = '=10I8s'` (48B `struct tipc_event`).
- `_mk_subscr()` masks the timeout: python exposes
  `TIPC_WAIT_FOREVER` as **`-1`** which `struct` flat refuses
  to pack into an unsigned `'I'`.
- `_decode_name_event()` *drops* runt frames and unknown event
  codes rather than raising — a confused kernel must not be
  able to kill the reader task.

XXX two corrections to what the plan §5.2 sketch claimed, both
verified against a live kernel,
- the event is **48B** (`4+4+4+8+28`), NOT 40.
- native (`'='`) byte-order is **accepted**; publish+withdraw
  both round-tripped w/ the 28B subscription echoed back
  intact. So the proposed `_detect_topsrv_endianness()` `'>'`
  retry-probe is unnecessary and is NOT implemented.

Note the event carries no *scope* — the name-table doesn't
report one — so the decoded `.addr` echoes the subscription's
own rather than pretending to observe it.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-15 01:37:13 -04:00
Gud Boi 7e20585f59 Reconcile plan 01 w/ the live-kernel findings
Per contract §0 ("if this doc disagrees with the code, the code
wins; fix it in the same PR"), fold the step-0 probe results and
the as-landed impl back into `01_tipc_backend.md`.

Settled the two claims §9 flagged as unverified,
- `SO_ACCEPTCONN` on `AF_TIPC` **works** (answers `1`); we never
  needed trio's `except OSError` carve-out.
- dup-name bind → **silent crosstalk is real**: both binds
  succeed and dials alternate strictly, so a `.get_random()`
  clash is never `EADDRINUSE`.

Corrections where the plan was wrong,
- §5.2's `tipc_event` is **48B not 40B** (`4+4+4+8+28`), and
  python exposes `TIPC_WAIT_FOREVER` as `-1` so it needs masking
  before packing as `'I'`.
- §7.2's pytest mark goes in `_testing/pytest.py::
  pytest_configure()`, NOT `pyproject.toml` — the repo has no
  `markers` ini table.
- §7.4's "10k → 10k distinct" is a ~1.2% flaky assert by
  birthday bound on a 32b instance space; use `>= n-2` w/ the
  arithmetic documented.
- §2.2's `unwrapped_type` and §3.2's `from_addr()` sketch still
  showed the 2-tuple + the `'tipc:<stype>:<scope>'` prefix hack
  that §2.2 itself had already withdrawn.

Two hazards the plan never anticipated, now recorded in §9,
- an unpublished-name dial answers `EHOSTUNREACH` which python
  maps to a **bare `OSError`**, NOT a `ConnectionError` subtype,
  so the `_reraise_as_connerr()` wrap is contract-§4 mandatory.
- a connect-then-drop peer answers `ENOTCONN` from
  `getpeername()`, which — since `.get_stream_addrs()` runs
  BEFORE the handshake — used to kill the whole actor.

Also withdraw §9's "fold a 6-byte digest into `(stype_low,
instance)`" escalation: varying `_stype` per-actor would need
65536 topology subscriptions and kills layer B outright.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 22:13:52 -04:00
Gud Boi 51d7133f47 Add the interim `/tipc/` maddr grammar
`mk_maddr()`/`parse_maddr()` learn,

    /tipc/<stype>/<instance>/<scope>

mirroring how `uds` maps onto the spec-legal `/unix`.

XXX `str`-ONLY for now: there is no registered `/tipc` proto
in the multiaddr table (upstream track gh #483 +
multiformats/py-multiaddr#107) and `Multiaddr()` rejects an
unregistered name outright. `MsgTransport.maddr`s return type
is already `Multiaddr|str` (and `MsgpackUDSStream` already
exercises the `str` branch), so this fits — but it IS why gh

`parse_maddr()` therefore special-cases the `/tipc/` prefix
BEFORE handing anything to `Multiaddr()`.

Also drive the maddr mapping-table tests off `_address_types`
instead of a hardcoded len/dict so the next backend can't
fail them for the wrong reason.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 10:02:44 -04:00
Gud Boi e0f66616cd Register `tipc` in the tpt tables + test harness
Wire the backend through every registration site (contract §2)
so `--tpt-proto tipc` is a first-class suite mode,
- `_state.TransportProtocolKey` gains the key
- `_addr._address_types` + `._default_lo_addrs`
- `_addr.wrap_address()` gets a `case ('tipc', *_)`; being a
  4-elem seq it can't collide w/ `tcp`s or `uds`s 2-tuple
  cases, so NO ordering hazard (and a bare seq-pattern matches
  the `list` form `msgpack` decodes to).
- `_types`: the `Address` union, `_msg_transports`,
  `_key_to_transport`, `_addr_to_transport` and the
  `transport_from_stream()` family match. That last one keys
  off `._tipc.AF_TIPC` (which carries the uapi fallback) NOT
  `socket.AF_TIPC` which is linux-only.

Test-harness side,
- `get_rando_addr()` gains a `tipc` branch; `.get_random()`
  already salts w/ `uuid4`+pid so both within- and cross-proc
  isolation come for free.
- the `tpt_protos` fixture calls an addr-type's optional
  `.is_available()` and `pytest.fail()`s w/ its reason. Keeps
  a module-less box from turning `--tpt-proto tipc` into a few
  hundred confusing connect-timeouts. Generic on purpose —
  plans 02/03 need the same hook.
- the discovery `daemon` fixture's readiness probe learns to
  dial a TIPC service name (it previously assumed tcp-or-uds
  and blew up on the 4-tuple).

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 10:02:44 -04:00
Gud Boi 2204979492 Widen `UnwrappedAddress` to admit proto-keyed addrs
`SpawnSpec.reg_addrs`/`.bind_addrs` pinned the wire shape to
a 2-tuple, so a `tipc` addr (`('tipc', stype, inst, scope)`)
died at the child w/ `msgspec.ValidationError: Expected array
of length 2, got 4` -> `invalid SpawnSpec IPC msg`.

Point those fields at `UnwrappedAddress` (which `SpawnSpec`s
own TODO already asked for) and widen the alias.

XXX VARIADIC (`tuple[str|int, ...]`) rather than a union of
the two concrete shapes, bc `msgspec` refuses a union holding
more than one array-like type.

?TODO, the real fix is the full proto-key migration (contract
§1.1) after which this becomes a tagged union keyed off elem
0 and per-proto validation comes back.

Note the alias is declared TWICE — `.msg.types` re-declares it
to dodge a circular import (`._addr` -> `.ipc._tcp` -> `.msg`)
and *that* copy is what actually validates the wire msg.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 10:02:44 -04:00
Gud Boi 8c0ae140cd Add `MsgpackTIPCStream`, the `AF_TIPC` `MsgTransport`
Wire `.connect_to()` (dial by service name), `.connected()`
and `.get_stream_addrs()` on top of `MsgpackTransport` so
`trio.SocketStream` + the existing `<I`-prefix framing carry
`msgpack` msgs over TIPC unchanged.

XXX both ends of a connected TIPC sock answer `TIPC_ADDR_ID`
port-ids and a port-id carries NO service name, so,
- the *dialling* side re-asserts the name it actually dialled
  over `._raddr` (same move as `MsgpackUDSStream`s peer-pid
  re-assign),
- the *accepting* side keeps a `TIPC_NAME_UNKNOWN` sentinel
  plus the observed `(node, ref)`. It doesn't need more — the
  `Aid` from `._do_handshake()` already carries the peer's
  logical identity.

Also normalize dial failures: TIPC answers an unpublished-name
lookup with `EHOSTUNREACH`, which python maps to a **bare**
`OSError` and NOT a `ConnectionError` subtype the way
`ECONNREFUSED` maps to `ConnectionRefusedError`. The
discovery-ping path needs the `ConnectionError` shape, so the
`_reraise_as_connerr()` wrap is load-bearing, not polish.

XXX ALSO tolerate a dead peer in `.get_stream_addrs()`!
Unlike tcp/uds — where the kernel keeps answering the peer
addr until *we* close — TIPC answers `ENOTCONN` once the peer
is gone. Since `MsgpackTransport.__init__()` calls
`.get_stream_addrs()` (via `Channel.from_stream()`) BEFORE the
handshake, an unguarded `OSError` there escapes
`handle_stream_from_peer()`s handshake tolerance (contract §4)
and tears down the WHOLE actor. Any connect-then-drop peer — a
port scan, a liveness probe, a cancelled dial — was a remote
actor-kill. A dead peer must cost us an addr, not the runtime.

Deats,
- `TIPC_IMPORTANCE` exposed as a `.connect_to()` kwarg — TIPC
  can rank a conn's traffic under congestion, which no other
  backend can do. Defaulted to the kernel default for now;
  wiring the parent<->child chan to `HIGH` is a follow-up.
- `TIPC_DEST_DROPPABLE = 0` so undeliverable msgs surface as
  errors instead of being silently dropped.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 10:02:44 -04:00
Gud Boi e3089ba356 Add `TIPCAddress` + `start_listener()`, gh #378
First slice of the `AF_TIPC` tpt backend: the addr type, the
`is_tipc_available()` capability predicate and the
name-publishing listener. No `MsgTransport` yet.

An actor's TIPC addr is a *service name* `(stype, instance)`:
`.bind()`ing the singleton `TIPC_ADDR_NAMESEQ` range IS the
service registration (it shows up in `tipc nametable show`)
and a peer's `.connect()`-by-name IS the lookup — so the
kernel does discovery for us, no registrar hop.

Deats,
- `.unwrap()` is proto-keyed as `('tipc', stype, inst, scope)`
  using the `multiaddr` proto spelling so `wrap_address()`
  can't confuse it with `tcp`s or `uds`s 2-tuples.
- `.rebind_from_sockname = False` bc `getsockname()` answers
  a port-id; `.from_addr()` raises on a bare `TIPC_ADDR_ID`
  rather than fabricate an un-dialable addr.
- `.bindspace` is the TIPC *scope*, i.e. literally the set of
  hosts a published name is reachable from. `ZONE` scope is
  deprecated/aliased so fold it to `CLUSTER` on input.
- mod stays importable on non-linux (uapi-value fallbacks,
  the `_uds.SO_PASSCRED` precedent) bc `._addr` builds its
  registration tables at import time.

XXX a `.get_random()` clash does NOT raise `EADDRINUSE` —
TIPC accepts multiple publishers of one name and round-robins
connects between them (verified against a live kernel), so a
collision is *silent crosstalk*. Hence the `blake2b` digest
and its (birthday-bounded) collision test.

Also,
- a generic `.is_available() -> (ok, why_not)` classmethod;
  deliberately spelled generically (NOT `is_tipc_*`) so the
  sibling env-dependent backends — `quic`/`iroh` (gh #353)
  and the `wg` netns bindspace (gh #482) — get the same gate
  for free. Its consumer lands w/ the reg tables.
- register a `tipc` pytest mark; the kernel-touching cases
  self-skip unless `sudo modprobe tipc` has been run.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 10:02:44 -04:00
Gud Boi cca3a70de4 Add `Address.rebind_from_sockname` opt-out
Gate `Endpoint.start_listener()`s `getsockname()`-vs-`.addr`
reconciliation on a new per-addr-type `ClassVar[bool]`, set
`True` on both `TCPAddress` and `UDSAddress` so existing
behaviour is bit-for-bit unchanged.

That reconciliation exists ONLY to learn a kernel-assigned
port from a `port=0` tcp bind (its own comment says so). The
incoming `tipc` backend (gh #378) has no late-binding
analogue AND its `getsockname()` answers a `TIPC_ADDR_ID`
port-id rather than the name-seq it published — rebinding
from that would swap a dialable service name for an
un-dialable, un-reconstructable port id.

So opting out is semantically right rather than a hack.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 10:02:44 -04:00
Gud Boi d4737e957f Pin `Endpoint` addr-reconciliation for tcp/uds
Guard test for `.start_listener()`s post-bind
`getsockname()`-vs-`.addr` round-trip, landed *before* that
reconciliation gets gated on an opt-out `ClassVar`.

- tcp: a `port=0` bind MUST still learn the kernel-picked
  port, since the reconciliation is the only path that ever
  does.
- uds: the sock-file path must survive the `.from_addr()`
  round-trip unchanged.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 10:02:44 -04:00
Gud Boi f9f98eeb06 Fix `pformat_caller_frame()`s bogus `indent` kwarg
Just drop it — `pformat_boxed_tb()` spells its knobs
`tb_box_indent`/`tb_body_indent`, and that fn's default
(1-space box indent) is what the caller wanted anyway.

Regressed-by: 888af602 (`pformat_cs()` mv into `.devx.pformat`)
Found-via: `/run-tests` test_pformat_caller_frame_renders

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 10:02:44 -04:00
Gud Boi 22ef362d9f Add a `pformat_caller_frame()` render guard test
`pformat_boxed_tb()` has never accepted an `indent` kwarg but
`pformat_caller_frame(box_tb=True)` has been passing one since
`888af602`. Nothing in the suite covered the branch, so the
`TypeError` only ever surfaced from `_mk_send_mte()` — i.e.
EVERY send-side `MsgTypeError` blew up while formatting itself
and masked the real msg-spec violation behind a bogus
`TypeError`.

Red on purpose per the test-first convention; the 1-line fix
lands next.

Also pin `pformat_boxed_tb()`s signature so a future typo'd
kwarg fails loudly at the call site instead of only when some
rare error path runs.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 10:02:44 -04:00
Gud Boi ee17ed9f6e Update `wg` docs for the merged py-multiaddr#108
it lands" framing in plan-03 and the example README was stale in
both directions: the branch pin is obsolete, yet you still can't
just `pip install multiaddr`.

Deats,
- §3.2's grammar table is now re-verified against the upstream
  merge (`f86519da`) rather than only `baudco@wg_support` in a
  throwaway venv. Also notes the codec enforces a 32-byte key,
  so a truncated one is a `StringParseError` and not a silently
  mangled parse.
- §1 says merged-but-unreleased; the still-open work is spec
  registration (py-multiaddr#107 + gh #483).
- §3.4 swaps "pin the branch" for the `[tool.uv.sources]` `rev`
  pin, and fixes the `_have_wg_maddr_proto()` recipe it
  suggested — probing w/ `Multiaddr('/wg/uAAAA')` now ALWAYS
  raises bc the codec wants 32B, i.e. that feature-detect would
  report `False` even w/ the proto perfectly well known.
- risk table row goes "#108 not merged" -> "merged but
  unreleased".
- example README: `uv sync` alone now suffices bc of the pin;
  documents the 32B check and points at
  `_have_wg_maddr_proto()` as the gate.

The one surviving `baudco` mention is deliberate, it records
where the grammar was *first* verified.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 09:51:36 -04:00
Gud Boi 7d6e79551e Fix silently-corrupt keys in `parse_wg_maddr()`
`_segments()` called `Multiaddr(maddr)` purely to validate, then
swallowed every failure under `except Exception: pass`. That was
harmless pre-#108 — w/o a `wg` codec there was nothing to
validate — but now that the codec is pinned in, the swallow is
load-bearing and disabled: a malformed key sails past validation
into `wg8_pubkey()`, which happily emits a corrupt b64 str, and
the returned struct then fails its own `.maddr` round-trip. No
raise, just quietly wrong output.

Deats,
- add `_have_wg_maddr_proto()`, the gate plan-03 already
  referenced but which never actually existed. Impl'd as
  `protocols.protocol_with_name('wg')` under
  `except ProtocolNotFoundError` and cached in a mod global,
  same shape as the TIPC plan's `is_tipc_available()`.
- only validate when that gate is `True`, and let
  `StringParseError` propagate — a maddr which doesn't parse
  must NOT reach `wg8_pubkey()`.
- keep the degraded split for a pre-#108 install, now w/ an
  explicit `XXX` naming the validation you give up.

So parsing stays pure but becomes total-or-raises. Our own
`ValueError`s (missing `/wg/` seg, bare tunnel w/o an overlay
ep) are unaffected, as is the `wg(8)` b64 round-trip.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 09:50:48 -04:00
Gud Boi cc85f17f5f Pin `multiaddr` to the merged `wg` codec rev
py-multiaddr#108 (the `/wg/u<key>` maddr proto) merged upstream
on 2026-07-28 as `f86519da`, but ships in no release yet — the
latest `0.2.0` predates it by ~4 months and carries no `wg`
codec at all. So `examples/multihost/wg_lan/` can't parse its
own maddrs off PyPI.

Pinned by `rev` and not `branch` so CI stays reproducible. Note
the lock now records the git source *instead of* the `>=0.2.0`
specifier, i.e. the dep floor above is fully overridden for as
long as this pin lives.

TODO, drop the pin (and bump that floor) the moment a release
carries the codec; the only consumer is the `wg_lan` example
set.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-14 09:38:58 -04:00
Gud Boi 0c96f92396 Log prompt-io for the tpt-backend planning arc
One record covering all 9 commits on this branch, per the NLNet
generative-AI policy and the existing `ai/prompt-io/claude/`
convention.

Uses diff-ref mode for both the plan docs and the example code
(`git diff main..ng_tpts_planning -- <path>`) rather than
duplicating content already in `git log -p`. Kept verbatim in
the `.raw.md`: the four verified findings (trio's
family-agnostic `SocketStream`/`SocketListener`, the round-trip
table proving `/wg/` is infix, the proto-key `UnwrappedAddress`
rationale, and `setns(2)`'s per-thread reality), since those are
reasoning rather than diffable output.

`## Human edits` records that the steering here was substantial
and mid-session rather than post-hoc: two model claims about wg
maddr semantics were challenged and retracted (incl. in an
already-posted issue comment), and the proto-key +
netns-as-runtime-config framings were human-directed. Also notes
the one model-initiated correction — a pre-publication
self-review that downgraded the `uniffi`/asyncio thesis and the
TIPC duplicate-binder claim to explicitly-flagged assumptions.

Prompt-IO: ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-12 20:36:03 -04:00
Gud Boi 27c34aebb6 Move the `wg_lan` examples under `examples/multihost/`
`tests/test_docs_examples.py` walks `examples/` **recursively**
and subproc-runs every collected file asserting `rc == 0`. Ran
its exact filter against the tree: all 4 of our files were being
collected — including `README.md`, since the filter never checks
the extension, so CI would have literally tried `python
README.md`. These need a real second host + a live `wg` tunnel,
so they can't ever satisfy that gate.

`'multihost' not in p[0]` is already in the test's exclusion
list w/ no dir yet using it, so this is a pure `git mv` — zero
test changes — and it's what the exclusion was plainly there
for. Collection drops 24 -> 20 files, 0 of them ours.

Also records *why* in the two places someone would look before
adding the next one: a callout at the top of the example README
and a note on plan 03's §3.4 deliverables. Anything needing a
second host or live tunnel goes under `examples/multihost/`.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-12 20:08:15 -04:00
Gud Boi bf974c9870 Add a `wg`-tunnelled 2-host example set
Re-renders gh #482's examples w/ the corrected (infix) maddr
grammar, as the "layer A" slice of the wg plan: declarative
maddrs only, tunnel pre-provisioned out-of-band, zero runtime
changes.

- `wg_maddr.py`: a `frozen=True` `msgspec.Struct` addr carrying
  `bearer`/`peer_pubkey`/`inner` (+ `inner_proto`), a `.maddr`
  property that re-renders the canonical form, and pure
  `mb_pubkey()`/`wg8_pubkey()`/`parse_wg_maddr()`. The parser
  rejects #482's inverted suffix form w/ an actionable error and
  stays **side-effect free** — `verify_wg_peer()` is a separate,
  explicitly impure step the caller composes, never something a
  parse path shells out to.
- `host_a_srv.py`/`host_b_client.py`: the two-host runs, passing
  only `addr.inner` into `open_nursery()`/`open_root_actor()`,
  which is the whole point — the bearer + key layers are already
  established before any bind happens.
- `README.md`: the grammar + the 3-owners table, the `#108`
  branch install line, tunnel setup, and a "what changed vs
  #482" section enumerating the corrections.

Runnable-shaped but **not yet run against a live tunnel**; that's
next, and the reason these sit on the planning branch rather than
in `examples/` proper. `_segments()` marks its stopgap for when
the `wg` codec isn't installed.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-12 20:08:15 -04:00
Gud Boi 41d08d04a6 Fix the `wg` maddr grammar, `/wg/` is *infix*
The prior revision (and gh #482's examples) had it as a suffix,
`/ip4/10.0.11.1/tcp/1616/wg/u<key>`. Wrong: verified against
`baudco/py-multiaddr@wg_support` (py-multiaddr#108) installed in
a throwaway venv, the canonical form is

  /ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616

where segs *before* `/wg/` are the **bearer** — the underlay
`(ip, udp-port)` `wg(8)` itself listens on (`ListenPort`), per
the codec docstring's own example — and segs *after* are the
**overlay** ep, the only part we ever bind. The suffix form does
parse, which is why it slipped through, but it's semantically
inverted: overlay addr where the bearer belongs, `tcp` where
wg's `udp` goes, and no overlay ep declared at all.

Records the observed `[p.name for p in m.protocols()]` lists so
the `match` can be written against fact, and replaces the
"composed vs not" framing w/ what's actually the design axis:
three parts, three **owners** — bearer bound by the kernel via
`wg-quick`/`pyroute2`, `/wg/u<key>` bound by nothing (it's an
identity, verified out-of-band), overlay bound by our
`IPCServer` as `.inner`. `_peel_tunnel_segs()` correspondingly
grows a 3rd return, splitting *at* the tunnel seg so nested
tunnels fall out for free.

Also hoists the netns conclusion to the top of §5.3 where it
can't be missed: netns is a **runtime-level config API, not an
actor-app-code one**. It's a spawn/boot-time input alongside
`enable_transports`/`tpt_bind_addrs`, deliberately w/ no
`await actor.enter_netns(...)`, because `setns(2)` neither moves
already-created sockets nor applies beyond the calling thread —
so a mid-life API would silently leave the IPC server bound in
the old ns.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-12 20:08:15 -04:00
Gud Boi 208876cc3b Proto-key the unwrapped-addr form in the plans
Shape-matching in `wrap_address()` doesn't survive 4 backends and
the plans were papering over it: TIPC's natural unwrapped form is
a `(str, int)`, indistinguishable from `TCPAddress`, and iroh's
is a `(str, str)`, which the *existing* UDS case
(`case (_, filename) if type(filename) is str`) already swallows.

So the contract doc (§1.1) now carries the conclusion as a
**recommended prerequisite for all three backends**: make the
unwrapped form carry an explicit proto-key spelled with the
`multiaddr` protocol name — `('tcp', host, port)`,
`('unix', path)`, `('tipc', stype, inst, scope)`. `wrap_address()`
then collapses from an order-sensitive `match` to
`_address_types[addr[0]]` and the whole collision class stops
existing, while the on-wire form finally agrees w/
`mk_maddr()`/`parse_maddr()` instead of being an independent
invention.

Two consequences spelled out: it's a wire-format change
(`SpawnSpec`, `_root_mailbox`, `_registry_addrs`) + every fixture
+ downstream config, so it wants its own migration commit landed
*before* any new backend; and it's the moment to stop handing raw
tuples to users at all — `Address` becomes the public currency
and `UnwrappedAddress` an internal serialization detail, the same
discipline `ipaddress` uses (you pass `IPv4Address`, never a
4-tuple).

Plan 01 §2.2 is rewritten to match and to explicitly **retract**
its own earlier `('tipc:<stype>:<scope>', instance)` self-tagging
prefix hack — it keeps `wrap_address()` order-sensitive and does
nothing for the iroh/UDS collision, so the doc says don't
resurrect it. Registration checklist item 4 likewise becomes "do
the migration first, then this is a one-line `_address_types`
entry".

Also seeds a `/tipc` multiaddr-spec submission as a follow-up,
mirroring the `wg` track (multiformats/py-multiaddr#107/#108 + gh

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-12 20:08:15 -04:00
Gud Boi f60bdc44c7 Index the tpt-backend plans w/ a README
Landing page for `ai/tpt-backends/`: points at the contract spec
as required first reading, tables the 3 plans against their
issues/deps/size, and states the landing order + why.

Deats,
- TIPC first as the cheap proof the table-registration story
  generalizes to a genuinely new proto (stdlib-only, and
  `trio`'s sock wrappers are family-agnostic).
- `wg` layer-A next since it's deployable-today doc/example work.
- QUIC last, gated on its own prep PR.
- notes that plans 01 and 02 both want the same
  `Address.rebind_from_sockname` gate, so whichever lands first
  ships it.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-12 20:08:15 -04:00
Gud Boi 7806b60fff Add `wg`-as-nested-bindspace plan doc
Plan doc for gh #482 + the tunnelled-maddr item of #443. Pushes
back on the framing that `wg` is a tpt: it's transparent to
`socket(2)`, so it belongs as a *bindspace* — a scoped
`@acm`-managed net ctx that an existing L4 tpt binds *inside* —
and it's what finally implements the long-spec'd (never
implemented) `Address.namespace`.

Deats, 3 independently-shippable layers,
- A) declarative: commit #482's examples, teach `parse_maddr()`
  the `/…/wg/u<key>` suffix -> a `TunnelledAddress` wrapper whose
  `.proto_key`/`.unwrap()` delegate to `.inner` so nothing new
  crosses the wire and every existing table lookup keeps working.
- B) swap the `subprocess.run(['sudo', 'wg', 'show'])` shelling
  for `pyroute2`. Default to `trio.to_thread` around the sync API
  (these are one-shot ops at bind/teardown, never hot-path), w/
  sans-io codecs + a trio `AF_NETLINK` sock as the follow-up for
  the read paths. Explicitly forbids dragging `trio-asyncio` in.
- C) `open_bindspace()`/`open_netns()`/`open_wg_iface()` `@acm`s
  folded w/ an `AsyncExitStack`, + filling in the
  `# !TODO, always be ns aware!` placeholder already sitting in
  `Endpoint.pformat()`.

Also flags the subtlest bug in the whole thing: `setns(2)` is
*per-thread*, so a `pyroute2` query issued via `trio.to_thread`
lands in the *original* netns. Test-first, per usual.

Further, designs for the generalization (`TunnelSpec` union +
`match` dispatch) while only implementing `wg`+netns, and calls
out `veth`-in-netns as the better *first* one bc it makes a
fully self-contained two-"host" integration test possible w/o
`wg` at all.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-12 20:08:15 -04:00
Gud Boi 7fab27de29 Add `QUIC`-via-`iroh` tpt-backend plan
Plan doc for gh #353. Picks `iroh` (the `uniffi` FFI pkg) over
`aioquic`/`quiche` bc node-id addressing + hole-punching + relay
fallback is the whole point; `aioquic` stays documented as the
fallback since ~90% of the adapters here are reusable against a
sans-io core.

Deats,
- the layering: iroh `Endpoint` per actor, `Connection` per peer
  (pooled via `trionics.maybe_open_context()`, not a hand-rolled
  cache), one bi-stream per `Channel`. 4-byte prefix framing
  stays so `MsgpackTransport` is untouched.
- `_uniffi_trio.py`: uniffi only uses `asyncio` as the executor
  for its rust-future poll loop, so a ~40-line
  `TrioToken.run_sync_soon()` bridge replaces it. Spells out the
  real hazards — strong ref on the `ctypes` trampoline, poll-code
  propagation, and a *bounded* shielded cancel-drain so a wedged
  rust future can't make an actor un-cancellable.
- `IrohAddress` w/ ALPN as the `.bindspace`, the `(str, str)`
  unwrapped form's collision w/ the UDS match-case, and why
  `get_root()` needs a persisted secret key -> a lazy
  `default_lo_addrs()` + a pure-getter/explicit-setter split.
- `QuicMsgStream(trio.abc.HalfCloseableStream)` +
  `QuicListener(trio.abc.Listener)`, incl. the exact
  EOF/reset/use-after-close semantics `_transport.py` already
  match-cases on, and hanging the acceptor tasks off the
  existing `Endpoint.listen_tn`.
- a prep-PR boundary: annotation widening, the shared
  `rebind_from_sockname` gate and a `tpt_key`-based
  `transport_from_stream()` dispatch, all landable w/ tcp/uds as
  the only backends.

Further, notes this is our first tpt w/ real transport security
+ peer auth, so an inbound node-id allowlist hook belongs here —
and that it says nothing about the other backends.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-12 20:08:15 -04:00
Gud Boi f7e3fdea1a Add `TIPC` tpt-backend impl plan
Plan doc for gh #378, the cheapest new backend we can add: it's
stdlib-only (CPython ships `AF_TIPC` + 23 `TIPC_*` consts) and
per the contract doc `trio`'s stream/listener wrappers don't care
about the addr family, so `MsgpackTransport` framing and
`trio.serve_listeners()` are reused verbatim.

Deats,
- `TIPCAddress` as a *service name* `(type, instance)` w/ scope
  as the `.bindspace`; `bind()` publishes the singleton
  name-range, peers `connect()` by name and the kernel resolves
  + load-balances. I.e. registration/lookup for free, no
  registrar in the loop.
- the self-tagging `('tipc:<stype>:<scope>', instance)` unwrapped
  form + why it must be match-ordered before `TCPAddress`'s.
- `get_random()` via a blake2b digest of the actor id (there's no
  `port=0` analogue) and the silent-crosstalk risk that follows:
  TIPC *allows* dup binders and round-robins, so a collision
  doesn't `EADDRINUSE`, it cross-talks.
- an `Address.rebind_from_sockname` ClassVar to opt out of
  `Endpoint.start_listener()`'s `getsockname()` reconcile, which
  for TIPC always returns a port-id, never the bound name.
- the `TIPC_TOP_SRV` topology-service subscription as an `@acm`
  yielding a chan of typed name-table events — push-based
  register/dereg, the real "end game cluster proto" bit.
- commit sequencing, hard capability gating (`modprobe tipc`;
  bare `AF_TIPC` is `EAFNOSUPPORT` on a stock box), CI matrix
  notes, risks + follow-up seeds.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-12 20:08:15 -04:00
Gud Boi 402e74b97a Add the `.ipc` tpt-backend contract spec
First doc of a new `ai/tpt-backends/` set: the normative
description of what a `tractor` tpt backend *is* as of `main`,
written so the 3 sibling plans (TIPC, QUIC, `wg`) can be worked
independently (by another model/provider) w/o design drift.

Deats,
- the backend duck-type as empirically derived from
  `_tcp.py`/`_uds.py`: the `Address` protocol surface, the
  mod-level `start_listener()`/`close_listener()` pair and
  `Msgpack<Proto>Stream(MsgpackTransport)`.
- the ONE reflection you can't break:
  `Endpoint.start_listener()` resolves the tpt mod via
  `inspect.getmodule(self.addr)`, so an `Address` type and its
  listener fns MUST live in the same mod.
- a 10-item registration checklist (`_address_types`,
  `_key_to_transport`, `_addr_to_transport`, `wrap_address()`
  match-cases, `TransportProtocolKey`, maddr tables, ..) incl.
  the import-time `_default_lo_addrs` trap.
- where the `trio.SocketListener` assumption is *actually*
  load-bearing (just the `getsockname()` reconcile) vs. merely
  annotated.
- the handshake/discovery invariants a new backend inherits,
  dep policy (extras + import-laziness per the #470 boot-latency
  budget), `--tpt-proto` harness plumbing and code style.

Also, records a verified finding the plans lean on hard:
`trio.SocketStream`/`SocketListener` are addr-*family* agnostic
— the only ctor checks are "is a trio sock" + `SOCK_STREAM` (+
an `OSError`-suppressed `SO_ACCEPTCONN`) — so any `SOCK_STREAM`
family CPython can make drops into the existing
`trio.serve_listeners()` path unmodified.

(this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-12 20:08:15 -04:00
48 changed files with 7421 additions and 27 deletions

View File

@ -117,6 +117,7 @@ jobs:
tpt_proto: [ tpt_proto: [
'tcp', 'tcp',
'uds', 'uds',
'tipc',
] ]
# https://github.com/orgs/community/discussions/26253#discussioncomment-3250989 # https://github.com/orgs/community/discussions/26253#discussioncomment-3250989
exclude: exclude:
@ -124,6 +125,11 @@ jobs:
- os: macos-latest - os: macos-latest
tpt_proto: 'uds' tpt_proto: 'uds'
# XXX `AF_TIPC` is a linux-kernel protocol; the address
# family doesn't exist on darwin at all.
- os: macos-latest
tpt_proto: 'tipc'
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
@ -150,6 +156,20 @@ jobs:
- name: List deps tree - name: List deps tree
run: uv tree run: uv tree
# XXX `AF_TIPC` needs its kernel module loaded! It ships
# with the standard ubuntu kernel package but is NOT loaded
# by default, so a bare `--tpt-proto=tipc` run would
# otherwise fail at the first `.bind()`.
#
# If GH's runners ever refuse the `modprobe`, the fallback
# is a container job w/ `--cap-add NET_ADMIN`.
- name: 'Load the `tipc` kernel module'
if: matrix.tpt_proto == 'tipc'
run: |
sudo modprobe tipc
tipc node get address
uv run python -c "from tractor.ipc._tipc import TIPCAddress; ok, why = TIPCAddress.is_available(); assert ok, why; print('AF_TIPC available')"
- name: Run tests - name: Run tests
run: > run: >
uv run uv run

4
.gitignore vendored
View File

@ -161,6 +161,10 @@ Session.vim
# -[ ] everything should be put under a `.gish/` no? # -[ ] everything should be put under a `.gish/` no?
gitea/ gitea/
gh/ gh/
# NOTE, `gh issue create` writes its local cache under the
# `<backend>/<repo>/<kind>/<num>.md` path, so the dir is named for
# the *service* not the CLI.
github/
# ------ macOS ------ # ------ macOS ------
# Finder metadata # Finder metadata

View File

@ -0,0 +1,136 @@
---
model: claude-opus-5
service: claude
session: 7b9c97c4-fff7-4ac4-97fb-35720453308e
timestamp: 2026-08-13T00:11:02Z
git_ref: 27c34aeb
scope: docs+code
substantive: true
raw_file: 20260813T001102Z_27c34aeb_prompt_io.raw.md
---
## Prompt
> draft hyper detailed implementation plans for [three]
> prospective new transport (tpt) backends for tractor's `.ipc`
> layer, from four GitHub issues: TIPC (gh #378) using built-in
> linux socket API w/ `trio` interfacing, leveraging TIPC's
> built-in discovery machinery; QUIC (gh #353) using the `iroh`
> lib, ideally with the py asyncio support (via ffi) rewritten
> for trio; wg (gh #482 and/or #443) with other shuttle-able
> tpts, using `pyroute2`, as much trio wrapping as possible
> where any other async support can be replaced.
With constraints: "be only slightly speculative"; realistic given
the linked info; the plans must be workable "by another
model/provider without much discrepancy on the design or lib
selections needed"; do it all in a new `/open-wkt` off main; and
be "as modular and modern-async-python-as-nearly-functional-style
as possible."
Key follow-up prompts that changed the design mid-session:
> wg requires binding a socket on both ends of the tunnel, the
> new support we added to `py-multiaddr` in their #108 enables
> declaring this in the maddr? […] so the
> `/ip4/10.0.0.1/udp/666/ip4/192.168.0.13/tcp/80` bullet example
> from #443 isn't correct for a wg tunnel maddr and would
> actually end up looking like,
> `/ip4/10.0.0.1/udp/666/wg/<pub-key>/ip4/192.168.0.13/tcp/80` no?
> right, we're likely going to need less `tuple`-formed
> `UnwrappedAddress` usage to make them all work and/or to extend
> the unwrapped form with a proto-str, likely the key from
> `multiaddrs` no?
> better encapsulating `UnwrappedAddress` for use externally is
> really way overdue; we should likely be encouraging (if not
> preventing unwrapped) wrapped addrs always much like other
> (std) libs, `ipaddress` comes to mind.
> so iirc netns mgmt becomes a lower-level runtime config-api
> rather then an actor-app-code API which can be dynamically
> changed once the actor is up ya?
## Response summary
A research-and-planning branch, `ng_tpts_planning` — **no runtime
code touched**. Five plan docs under `ai/tpt-backends/` plus a
corrected `wg` example set under `examples/multihost/wg_lan/`,
across 9 commits.
Three design conclusions came out of the planning that outlive
the branch:
1. **`wg` is a bindspace, not a `MsgTransport`.** The `/wg/`
maddr segment is *infix*: bearer (underlay ip+udp
`ListenPort`, bound by the kernel) → `/wg/u<key>` (an
identity, bound by nothing) → overlay (the only part tractor
binds). Verified empirically by installing
`baudco/py-multiaddr@wg_support` (py-multiaddr#108) in a
throwaway venv and round-tripping each candidate form; gh
#482's original suffix form parses but is semantically
inverted.
2. **Proto-key the `UnwrappedAddress` form**, spelled with the
`multiaddr` protocol names, dispatched via
`_address_types[addr[0]]`. Kills a whole collision class
(TIPC's `(str, int)``TCPAddress`; iroh's `(str, str)`
swallowed by the UDS case) and is the recommended migration
*before* any new backend lands.
3. **netns is a runtime/boot-time config API, not an app-code
one** — `setns(2)` is per-thread and won't move
already-created sockets, so there is deliberately no
`await actor.enter_netns(...)`.
Also verified that `trio.SocketStream`/`SocketListener` are
address-family agnostic (no `AF_*` check anywhere), which is what
makes TIPC the cheapest of the three backends to add.
Four related issues were annotated with the results (#378, #353,
#482, #443); #443's body was rewritten to reflect the corrected
grammar, with no existing checkbox state changed.
## Files changed
- `ai/tpt-backends/00_shared_backend_contract.md` — normative
backend duck-type contract, registration checklist, §1.1
proto-key conclusion
- `ai/tpt-backends/01_tipc_backend.md` — TIPC plan; service
addressing, `TIPC_TOP_SRV` push registry, instance-collision
hazard, step-0 probe
- `ai/tpt-backends/02_quic_iroh_backend.md``iroh` plan;
`uniffi`→`trio` bridge, listener/stream adapters, API-truth
table
- `ai/tpt-backends/03_wg_tunnel_bindspace.md``wg`-as-bindspace
plan; verified maddr grammar, 3-owner split, netns reality
- `ai/tpt-backends/README.md` — index
- `examples/multihost/wg_lan/wg_maddr.py` — frozen `msgspec`
tunnelled addr + pure parse/render helpers; impure
`verify_wg_peer()` kept separate
- `examples/multihost/wg_lan/host_a_srv.py` — host-A actor tree
- `examples/multihost/wg_lan/host_b_client.py` — host-B dialer
- `examples/multihost/wg_lan/README.md` — grammar, owner table,
setup, "what changed vs #482"
## Human edits
Substantial human steering rather than post-hoc editing; the
corrections were applied by the model in-session after being
challenged:
- rejected an initial claim that `wg` has "nothing to bind at the
tunnel layer" and supplied the correct composed maddr form,
which forced a rewrite of plan 03 §3.2 and a retraction in the
already-posted #443 comment
- rejected a supporting claim that `/ip4/../udp/443/quic-v1` was
"also composed"
- directed the proto-key/`ipaddress`-discipline conclusion and
the netns-as-runtime-config framing, both of which were then
folded back into the docs
- chose the commit boundaries and authored all commits; ran every
`git` mutation (commit, rebase, push) themselves
One model-initiated correction pre-publication: a self-review
downgraded two overconfident claims (the `uniffi`/asyncio thesis
and TIPC duplicate-binder behaviour) to explicitly-flagged
assumptions before the #353/#378 comments were posted.

View File

@ -0,0 +1,165 @@
---
model: claude-opus-5
service: claude
timestamp: 2026-08-13T00:11:02Z
git_ref: 27c34aeb
diff_cmd: git diff main..ng_tpts_planning
---
# Raw output — next-gen tpt-backend implementation plans
## Generated planning docs
> `git diff main..ng_tpts_planning -- ai/tpt-backends/`
Five markdown docs. `00_shared_backend_contract.md` is normative
and the other three are written against it so they can be worked
independently:
- **`00_shared_backend_contract.md`** — the backend duck-type
(`<Proto>Address(msgspec.Struct, frozen=True)` + module-level
`start_listener()`/`close_listener()` + a
`Msgpack<Proto>Stream(MsgpackTransport)`), the
`inspect.getmodule(self.addr)` reflection in
`Endpoint.start_listener()` that forces the Address class and
its listener fns to share a module, a 10-item registration
checklist, the dep policy, the test-harness shape, and §1.1's
proto-key conclusion (below).
- **`01_tipc_backend.md`** — service addressing via
`TIPC_ADDR_NAMESEQ` (bind/publish) and `TIPC_ADDR_NAME`
(connect/lookup), `TIPC_TOP_SRV` topology subscriptions as a
push-based registry, the `get_random()` instance-collision
hazard, and a step-0 capability-probe spike.
- **`02_quic_iroh_backend.md`** — `iroh` over
`aioquic`/`quiche`/`trio-asyncio`, a `_uniffi_trio.py` bridge
built on `TrioToken.run_sync_soon()`, `trio.abc.Listener`/
`HalfCloseableStream` adapters, and an API-truth table to fill
in during step 0.
- **`03_wg_tunnel_bindspace.md`** — `wg` as a *bindspace* rather
than a `MsgTransport`, a `TunnelledAddress` wrapper delegating
`.proto_key`/`.unwrap()` to `.inner`, `pyroute2` for layer B,
and `@acm`-managed netns/iface for layer C.
- **`README.md`** — index.
## Generated example code
> `git diff main..ng_tpts_planning -- examples/multihost/wg_lan/`
- `wg_maddr.py``WGTunnelledAddr(msgspec.Struct, frozen=True)`
carrying `bearer: tuple[str, int]`, `peer_pubkey: str`,
`inner: tuple[str, int]`, `inner_proto: Literal['tcp']`, plus a
`.maddr` property that re-renders the canonical form. Pure
helpers `mb_pubkey()`, `wg8_pubkey()`, `parse_wg_maddr()`, and
`_segments()` (with a marked stopgap for when the `wg` codec
isn't installed). `verify_wg_peer()` is impure **by design** and
kept out of the parse path.
- `host_a_srv.py` / `host_b_client.py` — the two-host runs; both
pass only `addr.inner` to `open_nursery()`/`open_root_actor()`.
- `README.md` — grammar, owner table, `#108`-branch install line,
tunnel setup, "what changed vs #482".
## Verified findings (non-code, verbatim)
### `trio` is address-family agnostic
Read against the installed `trio`. `SocketStream`/`SocketListener`
ctor checks are only "is a trio sock object" + `type ==
SOCK_STREAM`, plus an `OSError`-**suppressed** `SO_ACCEPTCONN`
probe. No `AF_*` check anywhere; `TCP_NODELAY`/`TCP_NOTSENT_LOWAT`
are set under `suppress(OSError)`. A TIPC `SOCK_STREAM` sock should
therefore drop straight into `trio.serve_listeners()` with the
existing `MsgpackTransport` framing, making TIPC mostly
table-registration boilerplate w/ zero new deps.
### the `wg` maddr grammar — `/wg/` is infix, not suffix
Installed `baudco/py-multiaddr@wg_support` (PR
multiformats/py-multiaddr#108) into a throwaway venv and
round-tripped every candidate form:
| maddr | `[p.name for p in m.protocols()]` |
| --- | --- |
| `/ip4/1.2.3.4/udp/51820/wg/u<k>` | `['ip4','udp','wg']` |
| `/ip4/../udp/../wg/u<k>/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` |
| `/ip4/10.0.11.1/tcp/1616/wg/u<k>` | `['ip4','tcp','wg']` |
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\_______ bearer __________/\__ key __/\______ overlay ______/
```
Segments *before* `/wg/` are the bearer — the underlay
`(ip, udp-port)` that `wg(8)` itself listens on (`ListenPort`).
Segments *after* are the overlay endpoint, the only part tractor
binds. The third row above is #482's original suffix form: it
parses, but is semantically inverted.
Three parts, three owners — and only one is an `Endpoint`:
| part | bound by | in the runtime? |
| --- | --- | --- |
| bearer | kernel, via `wg-quick`/`pyroute2` | no |
| `/wg/u<key>` | nothing — an identity | no, verified out-of-band |
| overlay | `tractor`'s `IPCServer` | yes, as `.inner` |
### proto-key-tagged `UnwrappedAddress`
Shape-matching in `wrap_address()` does not survive four backends.
TIPC's natural unwrapped form is a `(str, int)`, indistinguishable
from `TCPAddress`; iroh's is a `(str, str)`, already swallowed by
the existing UDS case (`case (_, filename) if type(filename) is
str`). Ordering hacks and prefix-tagging only paper over it.
Recommended prerequisite for all three backends: carry an explicit
proto-key spelled with the `multiaddr` protocol name —
`('tcp', host, port)`, `('unix', path)`,
`('tipc', stype, inst, scope)` — so `wrap_address()` collapses to
`_address_types[addr[0]]` and the collision class stops existing.
This also makes the on-wire form agree with
`mk_maddr()`/`parse_maddr()` instead of being an independent
invention. It is a wire-format change (`SpawnSpec`,
`_root_mailbox`, `_registry_addrs`) plus every fixture and
downstream config, so it wants its own migration commit landed
before any new backend — and it is the moment to stop handing raw
tuples to users at all, making `Address` the public currency and
`UnwrappedAddress` an internal serialization detail (the
discipline `ipaddress` uses).
### netns is a runtime-level config API
`setns(2)` affects the calling thread only and does not move
already-created sockets. So a netns is a spawn/boot-time input
alongside `enable_transports`/`tpt_bind_addrs`, and there is
deliberately no `await actor.enter_netns(...)` — a mid-life API
would silently leave the IPC server bound in the old namespace.
Corollary for layer B: pass `netns=` down to `pyroute2` rather
than assuming a `trio.to_thread` worker inherits it.
### `examples/` collection would have failed CI
`tests/test_docs_examples.py` walks `examples/` recursively and
subproc-runs every collected file asserting `rc == 0`. Its filter
never checks the extension, so all four `wg_lan` files were
collected — including `README.md`, which would have been run as
`python README.md`. `'multihost' not in p[0]` was already in the
exclusion list with no directory using it. Moving the set under
`examples/multihost/wg_lan/` drops collection 24 → 20 with zero
test changes; confirmed via `pytest --collect-only`.
## Corrections applied during the session
The human corrected two claims that had been asserted without
verification, both since retracted in-place in the docs and in the
posted issue comments:
1. that `wg` has "nothing to bind at the tunnel layer, exactly one
bind" — wrong; a wg stack is genuinely composed, and the real
axis is *who owns* each layer's endpoint.
2. that `/ip4/../udp/443/quic-v1` was "also composed" — wrong;
that is one endpoint with a protocol qualifier, not a tunnel.
A self-review before publication also downgraded two
overconfident claims to explicitly-flagged assumptions: the
`uniffi`-uses-asyncio-only-as-executor thesis (contradicted that
plan's own "do not guess from memory" step 0) and TIPC's
duplicate-binder round-robin behaviour (unverified).

View File

@ -0,0 +1,29 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: a998acff-af66-4969-a65e-f77cbf5ef8dd
timestamp: 2026-08-19T00:33:26Z
git_ref: wkt/tipc_backend_378
scope: config
substantive: true
raw_file: 20260819T003326Z_53516b09_prompt_io.raw.md
---
## Prompt
Confirm whether the successful TIPC CI leg can become blocking.
## Response summary
Removed the temporary TIPC-only `continue-on-error` expression after
every refreshed PR #493 check passed.
## Files changed
- `.github/workflows/ci.yml` - make TIPC failures block CI.
## Human edits
The human explicitly requested reconsidering the gate after the CI
run and chose to leave the generated change local for review before
requesting this commit plan.

View File

@ -0,0 +1,23 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-19T00:33:26Z
git_ref: wkt/tipc_backend_378
diff_cmd: git diff HEAD~1..HEAD
---
## Prompt
The user asked whether the now-green TIPC CI leg could stop using
`continue-on-error`.
## Response
All checks on PR #493 passed, including the Ubuntu TIPC matrix job.
The generated config change removes the temporary non-blocking gate
and its obsolete explanatory comment:
> `git diff HEAD~1..HEAD -- .github/workflows/ci.yml`
The remaining matrix, Linux-only exclusion and `modprobe tipc` setup
stay unchanged.

View File

@ -0,0 +1,37 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: a998acff-af66-4969-a65e-f77cbf5ef8dd
timestamp: 2026-08-19T00:33:27Z
git_ref: wkt/tipc_backend_378
scope: code
substantive: true
raw_file: 20260819T003327Z_53516b09_prompt_io.raw.md
---
## Prompt
Develop a practical bootstrap plan for testing TIPC across two
physical motherboards.
## Response summary
Expanded the two-host example into a reproducible smoke-test runbook
and made host A authorize the stable module name imported by host B.
Documented the cluster-domain-socket analogy and the prospective
`pyroute2` management path.
## Files changed
- `examples/multihost/tipc_cluster/README.md` - document setup,
validation, resilience testing, capture and cleanup.
- `examples/multihost/tipc_cluster/host_a_srv.py` - authorize the
`host_a_srv` RPC module under direct script execution.
## Human edits
The human selected two separate physical motherboards as the target
environment and requested that the resulting local work be packaged
as its own commit boundary. The human also proposed “Cluster Domain
Sockets” as clearer terminology and reusing the planned `pyroute2`
dependency for TIPC management.

View File

@ -0,0 +1,27 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-19T00:33:27Z
git_ref: wkt/tipc_backend_378
diff_cmd: git diff HEAD~1..HEAD
---
## Prompt
The user asked for a bootstrap plan to try TIPC with two real,
physically separate Linux motherboards.
## Response
The generated changes turn the existing sketch into an operator
runbook and correct the RPC module authorization used by direct
script execution:
> `git diff HEAD~1..HEAD -- examples/multihost/tipc_cluster/README.md examples/multihost/tipc_cluster/host_a_srv.py`
The runbook covers matching revisions, cluster identity, interface
selection, Ethernet bearer setup, link and name-table validation,
RPC, failure/rejoin testing, diagnostics, cleanup and a future
network-namespace fixture. It also records “Cluster Domain Sockets”
as explanatory terminology and identifies a future `pyroute2` TIPC
generic-netlink codec as the path away from manual `tipc(8)` calls.

View File

@ -0,0 +1,37 @@
---
model: openai/gpt-5.6-sol
service: opencode
session: a998acff-af66-4969-a65e-f77cbf5ef8dd
timestamp: 2026-08-19T00:33:28Z
git_ref: wkt/tipc_backend_378
scope: docs
substantive: true
raw_file: 20260819T003328Z_53516b09_prompt_io.raw.md
---
## Prompt
Draft an upstream `/tipc` multiaddr issue modeled on the prior `wg`
proposal.
## Response summary
Added a reviewable upstream issue draft with a concrete binary and
text encoding, composition examples, rollout plan and open design
questions. Retained the ecosystem-standard `tipc` name while using
“Cluster Domain Sockets” as explanatory terminology, and separated
the future `pyroute2` management track from address encoding.
## Files changed
- `ai/tpt-backends/04_tipc_multiaddr_upstream.md` - candidate
`multiformats/multiaddr` issue body.
- `ai/tpt-backends/README.md` - roster the upstream proposal.
## Human edits
The human chose to track the draft under `ai/tpt-backends/` instead
of leaving it in the ignored gish cache, and explicitly requested a
Prompt-IO record for this docs-only commit. The human proposed the
`cds` alternative and unifying TIPC/WireGuard management through
`pyroute2`.

View File

@ -0,0 +1,26 @@
---
model: openai/gpt-5.6-sol
service: opencode
timestamp: 2026-08-19T00:33:28Z
git_ref: wkt/tipc_backend_378
diff_cmd: git diff HEAD~1..HEAD
---
## Prompt
The user asked to begin drafting an upstream `/tipc` multiaddr issue
similar to the existing WireGuard proposal, then chose to track the
draft as an AI planning document.
## Response
The generated proposal is intended for `multiformats/multiaddr`:
> `git diff HEAD~1..HEAD -- ai/tpt-backends/04_tipc_multiaddr_upstream.md ai/tpt-backends/README.md`
It proposes a code allocated upstream, a fixed 72-bit value holding
the service type, instance and scope, and the canonical text form
`/tipc/<type>:<instance>:<scope>`. It includes WireGuard composition,
security boundaries, the `tipc` versus `cds` naming decision,
deployment-management separation, implementation phases, test data
and open questions for maintainers.

View File

@ -0,0 +1,403 @@
# `tractor.ipc` next-gen transport backends: the shared contract
Status: design doc / implementation spec.
Audience: any model or human implementing one of the three
sibling plans in this directory.
- [`01_tipc_backend.md`](./01_tipc_backend.md) — `AF_TIPC`
(gh #378)
- [`02_quic_iroh_backend.md`](./02_quic_iroh_backend.md) — QUIC
via `iroh` FFI, uniffi-async rewritten onto `trio` (gh #353)
- [`03_wg_tunnel_bindspace.md`](./03_wg_tunnel_bindspace.md) —
WireGuard (and other shuttle-able) tunnels as a *nested
bindspace* layer via `pyroute2` (gh #482, #443)
This doc is the **normative** description of what a `tractor`
transport backend *is* as of `main@83b34884`. Each sibling plan
assumes it and only documents its own deltas. Read this first;
do not re-derive it from the code.
---
## 0. Why a shared contract doc
The three plans are meant to be implementable *independently and
concurrently* by different models/providers without design
drift. Everything they share — the backend duck-type, the
registration tables, the test harness plumbing, the naming and
code-style rules — lives here exactly once. If an implementer
finds this doc disagrees with `main`, **the code wins**; fix this
doc in the same PR.
---
## 1. The backend duck-type (empirical, from `_tcp.py`/`_uds.py`)
A transport backend is **one module** under `tractor/ipc/`
exposing exactly four things. There is no ABC to subclass and no
plugin entrypoint; wiring is by explicit table registration
(§2) plus one piece of reflection (§1.3).
### 1.1 `class <Proto>Address(msgspec.Struct, frozen=True)`
Structurally conforms to the `Address` `Protocol` in
`tractor/discovery/_addr.py:82`. Required surface:
| member | kind | notes |
| --- | --- | --- |
| `proto_key` | `ClassVar[str]` | the wire/registry key, e.g. `'tcp'`, `'uds'` |
| `unwrapped_type` | `ClassVar[type]` | the primitive tuple shape |
| `def_bindspace` | `ClassVar` | default bindspace value |
| `is_valid` | `@property -> bool` | "is this a *dialable/bindable* addr" |
| `bindspace` | `@property` | the "set of hosts"-ish scope (see below) |
| `from_addr(cls, addr)` | `@classmethod` | primitive -> wrapped, `match`-based |
| `unwrap(self)` | method | wrapped -> primitive (must be msgpack-native!) |
| `get_random(cls, bindspace=...)` | `@classmethod` | per-subactor ephemeral addr |
| `get_root(cls)` | `@classmethod` | host-singleton default registrar addr |
| `__repr__` | method | `f'{type(self).__name__}[{...}]'` house style |
Hard constraints learned from the existing two:
- **`frozen=True`.** Addresses are dict keys
(`Server.epsdict()`, `Endpoint.peer_tpts`) and are compared by
value all over the runtime.
- **`.unwrap()` output must round-trip through `msgspec` and
through `wrap_address()`.** It is what actually crosses the
wire in `SpawnSpec`/`_root_mailbox`/`_registry_addrs`, and it
is what `Actor.reg_addrs` and every test compares against. If
your unwrapped form is not *uniquely* pattern-matchable
against the other backends' forms in
`wrap_address()` (`_addr.py:230`), you have a bug that
manifests as the wrong transport being loaded — the file's own
`XXX NOTE` warns about precisely this.
⚠️ **and shape-matching does not survive 4 backends.** Adding
TIPC and iroh breaks it outright: TIPC's natural form is a
`(str, int)` — indistinguishable from `TCPAddress` — and
iroh's is a `(str, str)`, which the *existing* UDS case
(`case (_, filename) if type(filename) is str`) already
swallows. Ordering hacks and prefix-tagging (an earlier
revision of plan 01 proposed `('tipc:<stype>:<scope>', inst)`)
paper over it at best.
**The fix, and the recommended prerequisite for all three
backends: make the unwrapped form carry an explicit
proto-key, using the `multiaddr` protocol name as the
canonical spelling** — `('tcp', host, port)`,
`('unix', path)`, `('udp', ...)`, `('tipc', stype, inst,
scope)`. Then `wrap_address()` collapses from an
order-sensitive `match` to `_address_types[addr[0]]`, and the
whole collision class stops existing. Note this *also* aligns
the on-wire form with `mk_maddr()`/`parse_maddr()`, so the two
representations stop being independent inventions.
Two consequences to plan for:
- it's a **wire-format change** (`SpawnSpec`,
`_root_mailbox`, `_registry_addrs`) plus every test fixture
and downstream config (`piker`'s `[network]` table). It
wants its **own migration commit, landed before any new
backend**, not smuggled into one.
- it's the moment to **stop handing raw unwrapped tuples to
users at all.** The long-term shape is: `Address` subtypes
are the public currency and `UnwrappedAddress` becomes an
internal serialization detail — the same discipline
`ipaddress` uses (you pass `IPv4Address`, not a 4-tuple).
Public API should accept `Address|maddr-str` and treat bare
tuples as legacy-tolerated input, ideally deprecated.
- **`.get_random()` must be collision-free without a live
runtime.** See the `UDSAddress.get_random()` uuid-token
comment (`_uds.py:207-220`): with no `current_actor()` the
sockname degenerates to a pure fn of `(prefix, pid)` and two
calls in one proc alias. Mix in a `uuid4().hex[:8]` token.
- **`.bindspace` semantics**: "the address' bindable space" —
ip/host for `tcp`, the socket-file *directory* for `uds`. For
the new backends: the TIPC *scope* (§1 of plan 01), the iroh
*ALPN + relay/discovery realm* (plan 02), the netns (plan 03).
`Address.namespace` is already spec'd in the Protocol as
"the if-available OS-specific network namespace key" and is
currently unimplemented by both backends — plan 03 is the
first real consumer.
### 1.2 module-level listener lifecycle
```python
async def start_listener(
addr: <Proto>Address,
**kwargs,
) -> trio.SocketListener # or a trio.abc.Listener, see §3
...
def close_listener( # OPTIONAL
addr: <Proto>Address,
lstnr: trio.abc.Listener,
) -> None:
...
```
`close_listener()` is optional; `Endpoint.close_listener()`
(`_server.py:674`) `getattr`s it and treats absence as "closing
is implicit". `uds` needs it (unlinks the sock-file), `tcp`
does not.
### 1.3 the ONE piece of reflection you must not break
`Endpoint.start_listener()` (`_server.py:656`):
```python
tpt_mod: ModuleType = inspect.getmodule(self.addr)
lstnr = await tpt_mod.start_listener(addr=self.addr)
```
The transport module is found by `inspect.getmodule()` **on the
`Address` instance**. Therefore: *the `Address` class and its
`start_listener()`/`close_listener()` MUST live in the same
module.* Do not define the address type in `_types.py` or a
`_addrs.py` and the listener elsewhere.
Immediately after, the same method does:
```python
if (unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap():
self.addr = self.addr.from_addr(unwrapped)
```
i.e. it assumes `lstnr.socket.getsockname()` exists and that its
return value is a valid `from_addr()` input. This is fine for
TIPC (§3 of plan 01) and **is the main integration hazard for
iroh** (§3 of plan 02) — plans that break it must say so
explicitly and propose the upstream `_server.py` patch.
### 1.4 `class Msgpack<Proto>Stream(MsgpackTransport)`
Subclass `tractor.ipc._transport.MsgpackTransport`. You inherit
all framing (`<I` 4-byte little-endian length prefix),
`msgspec` codec ctx-var lookup, `TransportClosed` normalization,
`.drain()`, `__aiter__`. You implement only:
| member | notes |
| --- | --- |
| `address_type` | the `<Proto>Address` class |
| `layer_key: int` | OSI-ish layer, `4` for both current backends |
| `maddr` `@property` | `-> Multiaddr\|str`, via `mk_maddr(self.raddr)` |
| `connected(self) -> bool` | `tcp`/`uds` both use `self.stream.socket.fileno() != -1` |
| `connect_to(cls, addr, prefix_size=4, codec=None, **kw)` | `@classmethod`, returns an instance |
| `get_stream_addrs(cls, stream) -> (laddr, raddr)` | `@classmethod`, called from `MsgpackTransport.__init__` |
`MsgpackTransport.__init__` requires the object passed as
`stream` to satisfy:
- `await stream.send_all(bytes)`
- usable as `tricycle.BufferedReceiveStream(transport_stream=stream)`,
i.e. `await stream.receive_some(n)`
- `trio.BrokenResourceError` / `trio.ClosedResourceError` /
`ValueError('...unclean EOF...')` on the failure paths that
`_iter_packets()` and `send()` already `match` on
(`_transport.py:221-304`, `:436-499`).
That is **`trio.abc.Stream`, not `trio.SocketStream`**. The
`MsgTransport` Protocol's `stream: trio.SocketStream`
annotation (`_transport.py:83`) is a lie of convenience — the
actual `MsgpackTransport.__init__` param is typed
`trio.abc.Stream` and nothing in the msg path touches
`.socket`. Only `connected()` (which each backend defines) and
`Endpoint.start_listener()`'s `getsockname()` do.
### 1.5 verified-good news for socket-family backends
Both `trio.SocketStream` and `trio.SocketListener` are
**address-family agnostic**. Verified against the installed
`trio` (`trio/_highlevel_socket.py`): the only constructor
checks are
- `isinstance(socket, trio.socket.SocketType)`
- `socket.type == SOCK_STREAM`
- (listener) `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)` is truthy,
with `OSError` **suppressed** (the macOS carve-out, which
also covers exotic families that reject the opt)
There is no `AF_*` check and no `IPPROTO_TCP` hard dependency
(`TCP_NODELAY`/`TCP_NOTSENT_LOWAT` are set under
`suppress(OSError)`). Consequence: **any `SOCK_STREAM` family
CPython can create — including `AF_TIPC` — drops straight into
the existing `trio.SocketStream` + `trio.serve_listeners()`
path.** This is why plan 01 is small and plan 02 is not.
---
## 2. Registration tables (the full wiring checklist)
Adding a backend touches these and only these:
1. `tractor/runtime/_state.py:46`
`TransportProtocolKey = Literal['tcp', 'uds', ...]` — add the
key. This `Literal` is the canonical set; `_testing/pytest.py`
drives `--tpt-proto` validation off `_addr._address_types`,
and the spawn-backend fixture already models the
"drive-the-set-from-the-Literal" pattern
(`pytest.py:870-880`) — do the same rather than hardcoding.
2. `tractor/discovery/_addr.py:173` `_address_types: bidict`
`{'<key>': <Proto>Address}`. Note it is a **`bidict`**, so
the mapping must stay 1:1.
3. `tractor/discovery/_addr.py:181` `_default_lo_addrs`
`'<key>': <Proto>Address.get_root().unwrap()`.
⚠️ this dict is built at **import time**, so
`get_root()` must not require a live runtime, a loaded kernel
module, or network I/O. (`UDSAddress.def_bindspace =
get_rt_dir()` is the precedent for "cheap, pure, filesystem-
ish".) A backend whose root addr needs I/O must make this
entry lazy — propose that refactor explicitly.
4. `tractor/discovery/_addr.py:230` `wrap_address()` `match`
add a case iff your `unwrapped_type` isn't already uniquely
matched. **Preferably do the proto-key migration in §1.1
first**, after which this step becomes a one-line
`_address_types` entry instead of an order-sensitive `case`.
5. `tractor/ipc/_types.py``Address` union alias,
`_msg_transports` list, `_key_to_transport[('msgpack', key)]`,
`_addr_to_transport[<Proto>Address]`.
6. `tractor/ipc/_types.py:92` `transport_from_stream()` — the
`sock.family` `match`. For a non-socket stream type (iroh)
this needs a different discriminator; see plan 02 §3.3.
7. `tractor/discovery/_multiaddr.py`
`_tpt_proto_to_maddr`, and a `case` in both `mk_maddr()` and
`parse_maddr()`.
8. `tractor/ipc/__init__.py` — re-export if the backend has a
public surface.
9. `tractor/_testing/addr.py::get_rando_addr()` — per-proto
branch so the whole suite can run under `--tpt-proto <key>`.
10. `pyproject.toml` — new deps go in an **optional extra**, never
in `[project].dependencies`. See §5.
## 3. Where the `trio.SocketListener` assumption is load-bearing
`_serve_ipc_eps()` (`_server.py:1041`) annotates
`listener: trio.abc.Listener` and hands the list to
`trio.serve_listeners(handler=handle_stream_from_peer,
listeners=..., handler_nursery=stream_handler_tn)`.
`trio.serve_listeners` itself is generic over
`trio.abc.Listener`. So the *only* `SocketListener`-specific
code in the server path is the `getsockname()` reconciliation in
`Endpoint.start_listener()` (§1.3) and the type annotations.
`handle_stream_from_peer()` (`_server.py:298`) then does
`Channel.from_stream(stream)`
`transport_from_stream(stream)``sock.family` match (§2.6).
**Therefore**: a non-socket backend needs (a) a
`trio.abc.Listener` subclass, (b) a change to
`Endpoint.start_listener()` to not blindly `getsockname()`, and
(c) a change to `transport_from_stream()`'s discrimination.
All three are small, upstream-able, and *should be landed as
their own prep PR* before the backend itself — see plan 02 §3.
## 4. Handshake / discovery invariants you inherit
- Every accepted stream immediately does
`chan._do_handshake(aid=actor.aid)`; a peer that fails it is
logged at `runtime` and dropped, **not** raised
(`_server.py:334-365`). Discovery-sys "pings" rely on this,
so your `connect_to()` must raise something that normalizes
to `TransportClosed`/`ConnectionError` on a dead peer, never
a novel exception type.
- `_root.py:381-406` fail-fasts when a `registry_addrs` entry's
`proto_key` is not in `enable_transports`. Your key must be
spellable in both.
- `_root.py:256` currently enforces `len(enable_transports) == 1`.
Multi-tpt actors are a separate work item; none of these three
plans may depend on lifting it.
- Sub-actor bind addrs come from
`_runtime.py:1600-1610`: for each key in the parent-supplied
`enable_transports`, `get_address_cls(key).get_random()`.
So `get_random()` runs *in the child, post-fork, pre-listen*.
Anything it needs (kernel module, netns membership, an iroh
secret key) must already be true at that moment.
## 5. Dependency policy
`[project].dependencies` stays lean (see the boot-latency work,
gh #470: `import tractor` is budgeted at ~0.145s). Every new
backend dep is an extra:
```toml
[project.optional-dependencies]
tipc = [] # stdlib-only!
quic = ["iroh>=0.35"] # pin per plan 02 §1
wg = ["pyroute2>=0.9"] # pin per plan 03 §1
```
and every backend module must be **import-lazy**: a
`tractor/ipc/_<proto>.py` that imports its 3rd-party dep at
module scope must not be imported by `tractor/__init__.py`,
`tractor/ipc/__init__.py`, or `tractor/discovery/_addr.py`'s
import-time table construction. The `_addr._default_lo_addrs`
eager-dict (§2.3) is the trap: keep the backend's `get_root()`
dep-free, or make that table lazy.
## 6. Test-harness plumbing (identical for all three)
- `--tpt-proto <key>` (`_testing/pytest.py:409`) selects the
session-wide proto; the `tpt_proto` fixture mutates
`_state._def_tpt_proto` + `_runtime_vars['_enable_tpts']`
(`pytest.py:807-835`). Adding the key to `_address_types` is
what makes `--tpt-proto <key>` legal (`pytest.py:795-800`
asserts the lookup).
- The **acceptance bar** for every backend is: the *entire*
existing suite passes under `--tpt-proto <key>`, unmodified.
That is the whole point of the abstraction. Backend-specific
unit tests go in `tests/ipc/test_each_tpt.py` (the existing
`test_uds_bindspace_created_implicitly` /
`test_uds_double_listen_raises_connerr` are the model).
- Capability gating: each backend needs a **cheap, pure
predicate** + a `pytest.mark.skipif`, because these are all
environment-dependent. Verified example: on this dev box
`socket.socket(AF_TIPC, SOCK_STREAM)` raises
`OSError(97, 'Address family not supported by protocol')`
because the `tipc` module isn't loaded. Put the predicate in
the backend module (so apps can use it too), not in the test.
- New pytest marks must be registered in `pyproject.toml`, per
the project's fix-warnings-at-source rule (gh #469).
## 7. Code style (non-negotiable, matches the repo)
- module header tagline: `# tractor: distributed structured
concurrency.` for **new** files (not the legacy
`structured concurrent "actors".` form the existing `_tcp.py`
carries).
- AGPL header block copied verbatim from `_tcp.py`.
- `from __future__ import annotations` first.
- annotate *everything*, including locals:
`sockpath: Path = addr.sockpath`.
- `match`/`case` over `isinstance` chains for address and
error dispatch.
- multi-line call/`import` style with trailing commas.
- never emit a whitespace-only line.
- error messages are multi-line f-strings ending in `\n`, with
the `f'...\n' f'...\n'` implicit-concat layout and the
`>[`/`[>`/`<=(` nested-op sigils where a `nest_from_op()` is
in play.
- prefer pure functions + module-level helpers over methods;
keep `Address` types data-only. Where a helper needs
scoped setup/teardown, it's an `@acm` — not a class with
`.start()`/`.stop()`.
- pure getters: no `get_*(..., mutate=True)` flags; split into
a read-only getter and an explicit sibling setter.
---
## 8. Cross-plan sequencing
The three are independent *except*:
- plan 02 (iroh) needs the `_server.py` /
`transport_from_stream()` generalization (§3) — plan 01 does
**not**, and should therefore land first as the cheap proof
that the table-registration story works for a genuinely new
proto.
- plan 03 (wg) composes *under* whatever L4 tpt is in use and
its netns work is what finally implements
`Address.namespace`. It can land before or after 02, but its
`TunnelledAddress` design must be reviewed against plan 02's
address shape so the "tunnelled maddr" grammar (gh #443)
covers `/…/quic-v1/p2p/…` inner addrs too.
- All three want first-class `wg`/`quic`/`tipc` protos in
`py-multiaddr`; that upstream track is gh #483 and
multiformats/py-multiaddr#107/#108.

View File

@ -0,0 +1,319 @@
# TIPC backend — handoff
Status: **PR [#493] is feature-complete and green**; what remains
is landing logistics plus a named follow-up track.
Takeover snapshot (2026-08-17): [#493] remains a draft at
`c7ae6065`, targeting `ng_tpts_planning` ([#492]). Its branch
point is `ee17ed9f`; #492 has since advanced by two commits to
`d9a6e2e9`, so #493 still needs rebasing onto that current
[#492] head before final landing work. The latest `tipc` CI leg
passed; the workflow as a whole is red only because the macOS
`tcp` leg failed.
Audience: any agent or human picking this up cold, from any
provider. Nothing here assumes a particular harness or tooling.
Read in this order,
1. this file (orientation + what's already settled)
2. [`00_shared_backend_contract.md`](./00_shared_backend_contract.md)
**normative** description of what a `tractor` transport
backend *is*
3. [`01_tipc_backend.md`](./01_tipc_backend.md) — the plan,
already reconciled against the live-kernel findings
Do **not** re-derive the design or re-select libraries. Where
this doc and the code disagree, **the code wins** — fix the doc
in the same change (contract §0).
[#493]: https://github.com/goodboy/tractor/pull/493
---
## 1. Environment
The backend needs a linux kernel module that is **not loaded by
default**:
```bash
sudo modprobe tipc
tipc node get address # confirms the module is live
```
Everything else is stdlib — TIPC adds **zero** dependencies.
This repo is a git worktree with a `uv`-managed venv at
`./py313`. Run things through it:
```bash
./py313/bin/python -m pytest tests/ipc -q
./py313/bin/python -m pytest tests/ -q --tpt-proto tipc
```
To rebuild docs you need the docs dep-group, which *mutates*
that venv — `uv sync` afterwards to restore it:
```bash
UV_PROJECT_ENVIRONMENT=py313 uv run --group docs \
python -m sphinx -b html docs /tmp/docbuild
UV_PROJECT_ENVIRONMENT=py313 uv sync
```
Without the module, `--tpt-proto tipc` fails loudly and
immediately (by design); `tests/ipc/test_tipc.py`'s
kernel-touching cases self-skip.
## 2. What the backend is, in three sentences
An actor's TIPC address is a **service name** `(stype, instance)`
— no host, no port. Binding the singleton `TIPC_ADDR_NAMESEQ`
range *publishes* it into a kernel-maintained cluster-wide name
table (visible via `tipc nametable show`), and a peer's
`.connect()`-by-name *is* the discovery lookup, resolved
in-kernel. `TIPC_ADDR_ID` port-ids are only ever **observed**,
never user-facing.
Everything lives in `tractor/ipc/_tipc.py`.
## 3. Hard-won facts — do not re-litigate these
All verified against a live kernel. Several contradict what the
plan originally assumed.
| fact | why it matters |
| --- | --- |
| A duplicate name bind **succeeds**, and connects **round-robin** between publishers | An instance collision is *silent crosstalk*, never `EADDRINUSE`. Hence the `blake2b` instance digest. |
| Dialing an unpublished name gives `EHOSTUNREACH` **instantly**, as a **bare `OSError`** — not a `ConnectionError` subtype | `_reraise_as_connerr()` is REQUIRED by contract §4, not polish |
| `getsockname()` always answers a `TIPC_ADDR_ID` port-id, even pre-bind | why `TIPCAddress.rebind_from_sockname = False` |
| A connect-then-drop peer makes `getpeername()` raise `ENOTCONN` | Unguarded, this **kills the whole actor**`.get_stream_addrs()` runs *before* the handshake, so it escapes handshake tolerance. See `_maybe_sockaddr()`. |
| `SO_ACCEPTCONN` works fine (answers `1`) | trio's `except OSError` carve-out is not load-bearing here |
| Graceful peer close arrives as `BrokenResourceError`/`ECONNRESET`, not a clean 0-byte EOF | benign; `_iter_packets()` already classifies it as a normal disconnect |
| Topology `struct tipc_event` is **48 bytes** (`4+4+4+8+28`) | the plan said 40 |
| The topology server **accepts native `'='` byte-order** | the plan's proposed `'>'`-retry endianness probe was deleted as unnecessary |
| `TIPC_WAIT_FOREVER` is `-1` in python | must be masked (`& 0xFFFFFFFF`) before packing as unsigned |
| TIPC **has** AES-GCM encryption (`tipc node set key`, linux 5.9+) | cluster/master/per-node keys + rekeying. So "wg adds the encryption TIPC lacks" is **false** — see §6 for the real motivation. |
| A wg interface is L3/`tun` (`POINTOPOINT,NOARP`, `link/none`) | TIPC's `eth` media **cannot** bind it — udp media is *mandatory* over wg. See the §6 caveat; this one bites. |
Two design decisions that are **closed**, with reasons:
- **The accepting side does not learn the peer's service name.**
A port-id can't be reversed into one. It gets a
`TIPC_NAME_UNKNOWN` sentinel plus the observed `(node, ref)`,
and that's fine — the `Aid` from the handshake already carries
the peer's logical identity. (`uds` has the same wart.)
- **Do not fold digest bits into the service `_stype` to widen
the collision space.** A topology subscription can only watch
**one** `stype`, so varying it per-actor would need 65536
subscriptions and kills the push-registry work outright. If
crosstalk ever bites, the escalation is a post-bind
verification handshake ([#501]).
[#501]: https://github.com/goodboy/tractor/issues/501
## 4. What landed
All §6 steps 17 landed on `wkt/tipc_backend_378`, based on
`ng_tpts_planning` (PR [#492], docs-only). The completed arc is
17 substantive commits from `22ef362d` through `c7ae6065`, plus
the incidental `1802641e` local-cache ignore commit. The plan's
§6 status text is the historical snapshot at `7e20585f`; its
claim that steps 67 remain is no longer current.
- `TIPCAddress` + `is_tipc_available()` + `start_listener()`
- `MsgpackTIPCStream` (`connect_to()`, `get_stream_addrs()`)
- `open_topology_events()` — the `TIPC_TOP_SRV` push feed
- registration across every table in contract §2
- interim `str`-only `/tipc/…` maddr grammar
- a `--tpt-proto=tipc` CI leg, **non-blocking** for now
- `docs/guide/tipc.rst` + `examples/multihost/tipc_cluster/`
Two fixes fell out that are **not** TIPC-specific:
- `devx/pformat.py``pformat_caller_frame()` passed an
`indent=''` kwarg `pformat_boxed_tb()` never accepted, so every
send-side `MsgTypeError` died with a `TypeError` while
formatting itself. On `main` and every branch since
`888af602`. **Wants cherry-picking out of this stack.**
- `SpawnSpec.reg_addrs`/`.bind_addrs` pinned the wire shape to a
2-tuple. Widened to `UnwrappedAddress`, which had to become
**variadic** (`tuple[str|int, ...]`) because `msgspec` refuses
a union holding more than one array-like type.
**Acceptance bar met**: 122 passed / 1 xfailed / 2 xpassed under
`--tpt-proto tipc` across `ipc`, `discovery`, `runtime`,
`spawning`, `local`, `rpc`, `cancellation`. `tcp`/`uds`
unchanged.
[#492]: https://github.com/goodboy/tractor/pull/492
## 5. Immediate next steps (pre-land)
These live on [#493]'s body as `### TODOs before landing`. They
are **not** mirrored into an issue — if the PR is ever superseded
they need re-homing.
1. **Cherry-pick the `pformat` fix onto `main`** as its own PR,
and land it before [#493] —
`22ef362d` (red guard test) then `f9f98eeb` (the 1-line fix).
Preserve that order. The pair is unrelated to TIPC, every
branch has the bug, and this fix must not disappear if #493
is superseded.
2. **Watch the `tipc` CI leg.** It's gated
`continue-on-error: ${{ matrix.tpt_proto == 'tipc' }}` because
GH's runners have never been asked to `modprobe` for us. Once
it has a few green runs, drop the gate. If the runners refuse
the `modprobe`, fall back to a container job with
`--cap-add NET_ADMIN`.
3. **Rebase #493 onto #492's current head first.** At this
snapshot that means moving from the `ee17ed9f` branch point
onto `d9a6e2e9`. #492 is checked out in another worktree, so
refresh its authoritative head and coordinate before changing
history. Once #492 merges, rebase #493 onto `main` for final
landing.
## 6. The follow-up track
All filed with the `follow-up` label.
| issue | what |
| --- | --- |
| [#495] | `TIPC_IMPORTANCE` supervision QoS on the parent↔child chan |
| [#496] | `TIPC_TOP_SRV` push registry in `discovery._registry` |
| [#497] | dual-link resiliency / multi-homing |
| [#498] | `/tipc` multiaddr spec submission |
| [#499] | registrar-less discovery via name derivation |
| [#500] | multicast/group msging as a *broadcast* transport |
| [#501] | post-bind verification for instance collisions |
| [#502] | **TIPC over a `wg` mesh — the reference multihost deployment** |
### the wg direction
[#502] is the strategic one. The intent is that TIPC-over-`wg`
becomes our go-to multihost transport deployment.
> ⚠️ **Do not repeat the claim that wg adds encryption TIPC
> lacks.** We assumed that initially and it is **wrong**. TIPC
> ships AES-GCM crypto of its own (`tipc node set key`, linux
> 5.9+) with cluster/master/per-node keys and rekeying.
>
> The motivation is different but still real:
> - **key management** — TIPC keys are symmetric and
> *pre-shared*; distribution, rotation and revocation are the
> operator's problem. wg gives public-key identity + handshake.
> - **uniformity** — wg is an overlay *every* backend can sit on
> (tcp now, quic later), not a TIPC-only mechanism.
> - **NAT traversal / roaming**, which raw TIPC bearers have no
> story for.
>
> Which to default to should be **benchmarked**, not assumed:
> TIPC-native crypto avoids a tunnel hop and may win on latency
> for LAN-local clusters.
> ⚠️ **`udp` media is MANDATORY over wg — `eth` cannot work.**
> A wg interface is L3/`tun`: `POINTOPOINT,NOARP`, `link/none`,
> no L2 address at all. There is no device for `tipc bearer
> enable media eth device …` to name.
>
> ```bash
> # impossible over wg
> sudo tipc bearer enable media eth device wg0
> # required, bound to the wg overlay IP
> sudo tipc bearer enable media udp name wgmesh localip 10.0.11.1
> ```
>
> Consequence worth internalizing: #378's "ethernet bearers pair
> most excellently with wireguard tunnelling" framing does **not**
> hold — on a given link the low-latency L2 path and the wg path
> are *mutually exclusive*. Any design that assumes both is
> broken from the start.
>
> Also mind the MTU: wg links are typically 1420, under
> ethernet's 1500, so TIPC link MTU wants checking not assuming.
Composed addresses take the form:
```
/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>
\____ wg bearer ________/\_key_/\______ tractor ep ________/
```
Note the structural point, which matters for the spec proposal
in [#498]: the tcp equivalent repeats an `/ip4/…/tcp/…` inner
segment because a tcp endpoint is *located*. The tipc inner
segment has **no locative component at all** — a service name is
location-independent by design. So in the composed form the wg
segments carry all the routing and the tipc segment carries pure
*identity*.
Prerequisites already established:
- py-multiaddr [#108] (merged) proved the composed `wg` + tcp
form parses and round-trips (`['ip4','udp','wg','ip4','tcp']`)
- `examples/multihost/wg_lan/` is the existing wg example set to
generalize from rather than duplicate
- the udp-bearer-only constraint and MTU caveat are documented in
both `docs/guide/tipc.rst` and the `tipc_cluster` README
[#495]: https://github.com/goodboy/tractor/issues/495
[#496]: https://github.com/goodboy/tractor/issues/496
[#497]: https://github.com/goodboy/tractor/issues/497
[#498]: https://github.com/goodboy/tractor/issues/498
[#499]: https://github.com/goodboy/tractor/issues/499
[#500]: https://github.com/goodboy/tractor/issues/500
[#502]: https://github.com/goodboy/tractor/issues/502
[#108]: https://github.com/multiformats/py-multiaddr/pull/108
## 7. Working conventions in this repo
Provider-neutral, but they *are* enforced by review:
- **Never commit, push, rebase or amend on your own.** Prepare
changes, report them, and let the maintainer stage. Asking
"should we commit?" is a question *for you to answer*, not
permission to act.
- **Do not re-ask for an exact forge write already authorized in
the current request.** Use the provider adapter's snapshot,
digest and drift checks, perform the named edit, then report
what was published. This does not authorize unrelated or
destructive forge actions.
- **A failing/guard test lands in its own commit before the fix
it guards.** Red first, then green.
- **One commit per logical step**, so history shows *why*. Never
squash unrelated changes.
- Commit subjects: present-tense verb, ~50 chars (hard max 67),
backticks around every code element. Bodies wrap at 67 cols.
- **Never write a line containing only whitespace.**
- Annotate everything including locals; prefer `match`/`case`
over `isinstance` chains; multi-line calls with trailing
commas; `@acm` over classes with `.start()`/`.stop()`.
- New modules get the `# tractor: distributed structured
concurrency.` header tagline plus the AGPL block.
- **Do not change task/checkbox state** in issues, plans or
trackers unless explicitly asked for that exact transition.
- Fix warnings at source; only genuinely-unfixable ones get
filtered, with a documented reason.
## 8. Where things are
```
tractor/ipc/_tipc.py the whole backend
tests/ipc/test_tipc.py 28 backend unit tests
tests/ipc/test_server.py the reconciliation guard
tests/devx/test_pformat.py the cherry-pick candidate
docs/guide/tipc.rst the docs page
examples/multihost/tipc_cluster/ runnable demos + manual
smoke test
ai/tpt-backends/00_shared_backend_contract.md
ai/tpt-backends/01_tipc_backend.md the (reconciled) plan
.github/workflows/ci.yml the gated tipc leg
```
Both single-host examples have been **run against a live
kernel** — the output pasted in their README is real, not
illustrative.
External agent memory deliberately contains only a project
pointer back to this handoff, not a competing copy of the project
state. Treat this file as the durable source of truth and update
it when the branch topology or landing sequence changes.

View File

@ -0,0 +1,821 @@
# Plan 01 — `TIPC` transport backend (`tractor/ipc/_tipc.py`)
Tracks gh [#378]. Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: TIPC is the *cheapest* new backend we can add and
simultaneously the only one that gives us cluster-wide service
discovery **for free, in the kernel**, replacing (for
TIPC-capable deployments) the whole `tractor.discovery`
registrar round-trip with a `bind()`/`connect()` on a
*service name*. It is stdlib-only: zero new dependencies.
[#378]: https://github.com/goodboy/tractor/issues/378
---
## 1. Why this is small: three verified facts
1. **CPython already speaks TIPC.** `socket.AF_TIPC` plus 23
`TIPC_*` constants are present in the stdlib on Linux
(verified on the dev box, py3.13):
`AF_TIPC, SOL_TIPC, TIPC_ADDR_ID, TIPC_ADDR_NAME,
TIPC_ADDR_NAMESEQ, TIPC_CFG_SRV, TIPC_CLUSTER_SCOPE,
TIPC_CONN_TIMEOUT, TIPC_{CRITICAL,HIGH,MEDIUM,LOW}_IMPORTANCE,
TIPC_DEST_DROPPABLE, TIPC_IMPORTANCE, TIPC_NODE_SCOPE,
TIPC_PUBLISHED, TIPC_SRC_DROPPABLE, TIPC_SUBSCR_TIMEOUT,
TIPC_SUB_CANCEL, TIPC_SUB_PORTS, TIPC_SUB_SERVICE,
TIPC_TOP_SRV, TIPC_WAIT_FOREVER, TIPC_WITHDRAWN,
TIPC_ZONE_SCOPE`.
`sock.bind()/connect()/getsockname()` take/return the
5-tuple `(addr_type, v1, v2, v3, scope)` — the last element
is optional on input and defaults to `0`.
2. **`trio` doesn't care about the address family.** Per
contract §1.5, `trio.SocketStream` and `trio.SocketListener`
only require a trio socket object of type `SOCK_STREAM`.
TIPC's `SOCK_STREAM` is a real connection-oriented reliable
byte stream. So we reuse `trio.SocketStream`,
`trio.SocketListener`, `trio.serve_listeners()`,
`MsgpackTransport`'s framing — *all of it*.
3. **It is not available by default.** On this box
`socket.socket(AF_TIPC, SOCK_STREAM)`
`OSError(97, 'Address family not supported by protocol')`
with no `tipc` in `/proc/modules`. `modprobe tipc` is
required; cross-node needs a bearer
(`tipc bearer enable media eth device <if>` or
`media udp name <n> localip <ip>`). Everything about this
plan's testability hinges on gating (§7).
Non-goals: `SOCK_RDM`/`SOCK_DGRAM`/`SOCK_SEQPACKET` message
modes, multicast fan-out, and TIPC group messaging. They are
genuinely interesting for a future `tractor` broadcast/pubsub
transport but they do **not** fit `MsgTransport`'s
stream-of-length-prefixed-msgs shape. Note them in the
follow-up issue, do not build them here.
---
## 2. `TIPCAddress`
### 2.1 the three TIPC address flavours, and which we use
| flavour | tuple | meaning |
| --- | --- | --- |
| `TIPC_ADDR_NAMESEQ` | `(type, lower, upper, scope)` | a *published range* — what a server `bind()`s |
| `TIPC_ADDR_NAME` | `(type, instance, domain, scope)` | a *lookup* — what a client `connect()`s |
| `TIPC_ADDR_ID` | `(node, ref, 0, scope)` | a concrete port id — the "physical" address |
The design decision that makes this backend coherent:
> **A `tractor` actor's TIPC address is a *service name*
> `(type, instance)`; `bind()` publishes the singleton range
> `(type, instance, instance)`; peers `connect()` by name and
> the kernel resolves + load-balances. `TIPC_ADDR_ID` is only
> ever an *observed* address (`getpeername()`), never a
> user-facing one.**
This is exactly the "leverage the built-in discovery machinery"
ask in #378: publishing a bind *is* registration, and
`connect()` on a name *is* a lookup, with no registrar actor in
the loop.
### 2.2 the struct
```python
class TIPCAddress(
msgspec.Struct,
frozen=True,
):
_stype: int # TIPC "type" == service class
_instance: int # service instance within the type
_scope: int = TIPC_CLUSTER_SCOPE
# observed-only, never part of identity/equality-by-intent
maybe_node: int|None = None # from TIPC_ADDR_ID getpeername()
maybe_ref: int|None = None
proto_key: ClassVar[str] = 'tipc'
unwrapped_type: ClassVar[type] = tuple[str, int, int, int]
def_bindspace: ClassVar[int] = TIPC_CLUSTER_SCOPE
```
**Unwrapped form** (the wire/`SpawnSpec` shape).
TIPC's natural form is `(stype, instance, scope)` — but a
2-tuple squeeze of it is a `(str, int)`, i.e. *the same coarse
shape as `TCPAddress`*, so `wrap_address()`'s
`case (str(), int())` steals it. This backend is therefore the
forcing function for the contract-doc's conclusion (§1.1):
> **make the unwrapped form carry an explicit proto-key, spelled
> with the `multiaddr` protocol name.**
```python
def unwrap(self) -> tuple[str, int, int, int]:
return ('tipc', self._stype, self._instance, self._scope)
```
`wrap_address()` then dispatches `_address_types[addr[0]]` and
the collision class disappears. **This is a prerequisite
migration commit, not part of this backend** — see contract §1.1
for its blast radius (wire format + every fixture + `piker`
config) and for the follow-on "stop handing raw tuples to users
at all, à la `ipaddress`" direction.
⚠️ an earlier revision of this plan proposed a self-tagging
`('tipc:<stype>:<scope>', instance)` string-prefix hack with an
ordered `case` guard. **Dropped** — it papers over the problem,
keeps `wrap_address()` order-sensitive, and doesn't help iroh's
`(str, str)`-vs-UDS collision at all. Do not resurrect it.
Note `TIPCAddress` is the first backend where `.unwrap()` is
**not** a lossless view of the live socket — `maybe_node`/
`maybe_ref` are observed metadata, exactly like
`UDSAddress.maybe_pid` (which is likewise excluded from
`.unwrap()`). Follow that precedent, including its `__repr__`
treatment (`_uds.py:242`).
### 2.3 how to pick `_stype` and `_instance`
- `_stype` = a `tractor`-reserved service class. TIPC reserves
0..63 for internal use (`TIPC_TOP_SRV == 1`,
`TIPC_CFG_SRV == 0`). Use a module constant
`TRACTOR_STYPE: int = 0x74_72_00_00` ("tr\0\0") as the default
and make it overridable via `TIPCAddress._stype` so an app
can partition service classes. Document that two `tractor`
trees sharing a cluster **and** a `_stype` share a namespace.
- `_instance` for `get_root()`: `1616` — mirrors the
`TCPAddress.get_root()` port and the `registry@1616.sock`
UDS filename, so the "1616 is tractor's registrar" idiom
holds across all backends.
- `_instance` for `get_random()`: TIPC gives us no
kernel-assigned-instance analogue of `port=0`, so we must
choose. Use a *pure* fn of the actor identity so it is
reproducible and collision-free:
```python
# 32-bit instance derived from the actor's uuid4 (+ pid when
# there's no live runtime, per the UDS precedent).
inst: int = int.from_bytes(
blake2b(seed.encode(), digest_size=4).digest(),
'big',
)
```
where `seed = f'{actor.aid.name}@{pid}'` if
`current_actor(err_on_no_runtime=False)` else
`f'{prefix}.{uuid4().hex[:8]}@{pid}'`. Must avoid the reserved
low range: `inst = 64 + (inst % (2**32 - 64))`.
⚠️ *unlike* `port=0`, a collision here surfaces as a
successful-but-shared publication (TIPC allows multiple
binders on the same name and round-robins!) rather than
`EADDRINUSE`. That is a silent-crosstalk failure mode; §7 has
the test that proves the 4-byte digest is enough and §9 has
the mitigation if it isn't.
- `_scope`: `TIPC_NODE_SCOPE` for a same-host-only actor (the
UDS-equivalent), `TIPC_CLUSTER_SCOPE` (default) for
cluster-visible. **This is `.bindspace`**:
```python
@property
def bindspace(self) -> int:
return self._scope
```
It is the honest analogue of "the set of hosts this bind is
reachable from", which is precisely the docstring in
`Address.bindspace`. (`TIPC_ZONE_SCOPE` is deprecated/aliased
to cluster in modern kernels — accept it on input, normalize
to cluster, log at `transport` level.)
### 2.4 `is_valid`
```python
@property
def is_valid(self) -> bool:
return (
self._instance != 0
and
self._stype not in _tipc_reserved_stypes # {0, 1, ...}
and
self._scope in (TIPC_NODE_SCOPE, TIPC_CLUSTER_SCOPE)
)
```
---
## 3. Listener + stream
### 3.1 `start_listener()`
```python
async def start_listener(
addr: TIPCAddress,
backlog: int = 128,
**kwargs,
) -> SocketListener:
sock = trio.socket.socket(
socket.AF_TIPC,
socket.SOCK_STREAM,
)
# publish the singleton name-range == "register the service"
await sock.bind((
socket.TIPC_ADDR_NAMESEQ,
addr._stype,
addr._instance,
addr._instance,
addr._scope,
))
sock.listen(backlog)
return SocketListener(sock)
```
Notes / hazards:
- `bind()` on `AF_TIPC` is **not** a filesystem or port-table
operation and can't block on DNS, but keep it `await`ed
through `trio.socket` anyway for uniformity.
- `backlog=128` matching `_uds.start_listener()`'s hard-won
value (see its comment at `_uds.py:317-331` re: concurrent
deregistration storms). Do not use `1`.
- **no `close_listener()` needed** — nothing to unlink. Omit the
function entirely (contract §1.2: absence means implicit).
Withdrawal of the published name happens on socket close.
- ✅ **SETTLED** (step-0 probe, live kernel): `SocketListener.
__init__`'s `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)` **works**
on `AF_TIPC` and answers `1`. We do *not* rely on trio's
`except OSError: pass` carve-out at all. Pinned by
`test_listener_tolerates_so_acceptconn`.
- Wrap the bind in a `_reraise_as_connerr()`-style `@cm` (copy
the `_uds.py:256` pattern) so `EADDRINUSE`-ish and
`EAFNOSUPPORT` become `ConnectionError` with the addr in the
message. `EAFNOSUPPORT` here means "kernel module not
loaded" and deserves a *specifically actionable* message:
`'TIPC unavailable — try `sudo modprobe tipc`\n'`.
### 3.2 the `getsockname()` reconciliation
`Endpoint.start_listener()` does
`if lstnr.socket.getsockname() != self.addr.unwrap(): self.addr =
self.addr.from_addr(unwrapped)`.
For TIPC, `getsockname()` on a bound-but-listening socket
returns a `TIPC_ADDR_ID`-flavoured 5-tuple (the port id), *not*
the name-seq we bound. So the `!=` is **always true** and
`from_addr()` will be handed a 5-tuple.
Handle it inside `TIPCAddress.from_addr()` — do **not** patch
`_server.py`:
⚠️ the sketch that stood here used the `'tipc:<stype>:<scope>'`
string-prefix hack §2.2 explicitly **withdrew**. Corrected to
the proto-keyed form (and note a bare seq-pattern matches the
`list` that `msgpack` decodes our tuples back to, so no
separate `[...]` alternative is needed):
```python
@classmethod
def from_addr(cls, addr) -> TIPCAddress:
match addr:
# our own proto-keyed unwrapped form
case ('tipc', int() as stype, int() as inst, int() as scope):
return TIPCAddress(stype, inst, _norm_scope(scope))
# ..w/ the scope defaulted
case ('tipc', int() as stype, int() as inst):
return TIPCAddress(stype, inst)
# a kernel-observed TIPC_ADDR_ID 5-tuple
case (int() as atype, *_) if atype == TIPC_ADDR_ID:
...
```
The `TIPC_ADDR_ID` case cannot reconstruct `(stype, instance)`
— that info isn't in a port id. So `from_addr()` alone is
insufficient for the reconciliation path. **Resolution**: make
`from_addr()` raise a clear `ValueError` for the bare
`TIPC_ADDR_ID` case, and instead prevent the reconciliation
from firing by having `start_listener()` return a listener
whose `getsockname()` we never need — i.e. land this two-line
upstream fix in `_server.py:664`:
```python
if (
(unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap()
and
self.addr.rebind_from_sockname # ClassVar[bool] = True on tcp/uds
):
```
with `TIPCAddress.rebind_from_sockname: ClassVar[bool] = False`
(and `True` on `TCPAddress`/`UDSAddress`, preserving today's
behaviour exactly). Rationale: the reconciliation exists *only*
to learn the kernel-assigned port for `port=0` TCP binds (its
own comment says so, `_server.py:662`); TIPC has no such
late-binding, so opting out is semantically right rather than a
hack. **Land this as its own commit, ahead of the backend**,
with a test that `tcp`'s `port=0` behaviour is unchanged.
Keep the observed port-id available anyway: annotate
`ep.addr = ep.addr.with_port_id(*getsockname()[1:3])` (a pure
`msgspec.structs.replace()` helper) purely for logging/repr.
### 3.3 `MsgpackTIPCStream`
```python
class MsgpackTIPCStream(MsgpackTransport):
address_type = TIPCAddress
layer_key: int = 4
@property
def maddr(self) -> Multiaddr|str:
return mk_maddr(self.raddr)
def connected(self) -> bool:
return self.stream.socket.fileno() != -1
@classmethod
async def connect_to(
cls,
destaddr: TIPCAddress,
prefix_size: int = 4,
codec: MsgCodec|None = None,
**kwargs,
) -> MsgpackTIPCStream:
sock = trio.socket.socket(AF_TIPC, SOCK_STREAM)
with close_on_error(sock):
# NOTE: connect by *name* -> kernel does the lookup,
# so this is our "discovery" call.
await sock.connect((
socket.TIPC_ADDR_NAME,
destaddr._stype,
destaddr._instance,
0, # domain: 0 == "anywhere in scope"
destaddr._scope,
))
return cls(
trio.SocketStream(sock),
prefix_size=prefix_size,
codec=codec,
)
```
- reuse `trio._highlevel_open_unix_stream.close_on_error` (the
UDS backend already imports it) or inline the equivalent
`try/except: sock.close(); raise`.
- `SO_/TIPC_` opts worth setting and documenting:
- `setsockopt(SOL_TIPC, TIPC_IMPORTANCE, TIPC_HIGH_IMPORTANCE)`
for the *parent<->child* lifetime channel — this is a real
win TIPC gives us that TCP can't: the runtime's
supervision channel can outrank bulk app traffic under
congestion. Wire it as a `connect_to(..., importance=...)`
kwarg defaulted from a module constant, and have
`_runtime.py`'s parent-chan path pass the high value **in a
follow-up** (don't couple it to this PR).
- `TIPC_CONN_TIMEOUT` — the kernel-side connect timeout;
leave at default, we have `trio` cancel scopes.
- `TIPC_DEST_DROPPABLE = 0` on the connection so undeliverable
msgs come back as errors rather than being silently dropped.
- ✅ **SETTLED****`connect_to()` on a name with no
publisher**: TIPC answers `EHOSTUNREACH` (113) *instantly*
(no SYN-timeout wait), which is indeed better discovery-ping
behaviour than TCP.
⚠️ BUT the errno matters more than expected: python maps
`EHOSTUNREACH` to a **bare `OSError`**, NOT to a
`ConnectionError` subtype the way it maps `ECONNREFUSED` ->
`ConnectionRefusedError`. So the `_reraise_as_connerr()` wrap
is **load-bearing for contract §4**, not cosmetic polish —
without it the registrar ping path sees a foreign exc type.
(For contrast, dialling a bogus *port-id* — as opposed to a
name — does give `ECONNREFUSED`.)
### 3.4 `get_stream_addrs()`
```python
@classmethod
def get_stream_addrs(cls, stream) -> tuple[TIPCAddress, TIPCAddress]:
sock = stream.socket
# both return TIPC_ADDR_ID 5-tuples for a connected sock
l_id = sock.getsockname()
r_id = sock.getpeername()
...
```
Problem: neither end's port-id tells us the *service name*. The
`laddr`/`raddr` are used for logging, `Channel.raddr`,
`Server._peers` keying-adjacent repr, and `maddr`. Design:
- the **connecting** side knows the destaddr it dialled →
`connect_to()` overrides `_raddr` after construction with the
known-good `TIPCAddress`, exactly as
`MsgpackUDSStream.connect_to()` does for the peer-pid case
(`_uds.py:539-543`).
- the **accepting** side does not know the peer's service name
from the socket. Two honest options:
- **(a) accept it: `raddr` carries only `(node, ref)`** via
`maybe_node`/`maybe_ref`, `_stype/_instance` set to a
sentinel `-1`, and `__repr__` renders
`TIPCAddress[<peer-node:0x...>:<ref>]`. The `Aid` from the
handshake already gives us the peer's logical identity, so
nothing in the runtime actually *needs* the peer's service
name. **Recommended.**
- (b) piggyback the peer's own bound name in the handshake.
Rejected for this PR: touches `Aid`/msg-spec.
- `laddr` on the accepting side: the `Endpoint` knows its own
`addr`; but `get_stream_addrs()` is a `@classmethod` with only
the stream. Use `TIPC_ADDR_ID` for `laddr` too and let
`Endpoint.peer_tpts` keying (which is by *peer* addr) still
work. Verify nothing asserts `laddr == ep.addr` — grep for
`.laddr` uses before committing (`_server.py`'s
`con_status` logging, `Channel.pformat()`).
✅ grepped: `.laddr` is repr/logging-ONLY. `.raddr` has three
real consumers (`discovery/_api.py:277`'s `query_actor()`
yield, plus two test asserts) — and note `uds` *already* has
this same wart (its accepting-side `raddr` is the listener's
own sockpath), so (a) is consistent with the status quo.
- 🐛 **HAZARD the original draft missed — a dropped peer must
not kill the actor.** Unlike tcp/uds — where the kernel keeps
answering the peer addr until *we* close — a TIPC socket
whose peer has already gone answers **`ENOTCONN`** from
`getpeername()`.
That's fatal as written, because
`MsgpackTransport.__init__()` calls `get_stream_addrs()` (via
`Channel.from_stream()`) **before** the handshake, so the
`OSError` escapes `handle_stream_from_peer()`'s
handshake-failure tolerance (contract §4) and tears down the
**whole actor**. i.e. any connect-then-immediately-drop peer
— a port scan, a liveness probe, a cancelled dial — is a
remote actor-kill.
Wrap both `getsockname`/`getpeername` in a tolerant helper
and degrade to a port-id-less addr. A dead peer must cost us
an addr, not the runtime.
NOTE this is *not* hypothetical: the discovery suite's own
`daemon` readiness probe
(`tests/discovery/conftest.py`) does exactly this, which is
how it was found.
---
## 4. Multiaddr representation
There is no `/tipc` in the multiaddr protocol table. Interim
grammar, mirroring how `uds` maps to the spec-legal `/unix`:
```
/tipc/<stype>/<instance> # scope implied = cluster
/tipc/<stype>/<instance>/<scope> # explicit
```
- `_tpt_proto_to_maddr['tipc'] = 'tipc'` and a `mk_maddr()`
`case 'tipc':` building the above.
- `parse_maddr()` gets `case ['tipc']:` — but note
`py-multiaddr` will reject an unregistered protocol name
outright, so this **requires an upstream registration** (same
track as the `wg` work, gh #483 /
multiformats/py-multiaddr#107). Until that lands:
- `MsgpackTIPCStream.maddr` returns the **`str`** form (the
`MsgTransport.maddr` return type is already
`Multiaddr|str`, and `MsgpackUDSStream.maddr` already
exercises the `str` branch), and
- `parse_maddr()` special-cases the `/tipc/` prefix *before*
handing the string to `Multiaddr()`.
Document this as the reason gh #443's "standardize on
returning `Multiaddr` everywhere" item stays blocked.
Propose `/tipc/` upstream as: name `tipc`, code TBD, size
variable, value `<stype>:<instance>:<scope>` — or as three
composed protos. Prefer *one* proto with a structured value so
the maddr stays 2-segment like `/unix/...`.
---
## 5. Discovery: the actually-interesting part
Two independently-shippable layers. **Layer A is in scope for
the first PR; layer B is a fast-follow.**
### 5.1 Layer A — "discovery by bind" (free)
Because `bind(TIPC_ADDR_NAMESEQ)` publishes and
`connect(TIPC_ADDR_NAME)` resolves, a `tractor` tree whose
`registry_addrs` are TIPC service names needs **no registrar
liveness at all** for the connect path: `find_actor()`'s
"connect to the registrar and ask" becomes "connect to the
service name directly". Concretely:
- `tractor.discovery._api.find_actor()` etc. keep working
unchanged (they go through the registrar), *and*
- a new, TIPC-only fast path becomes possible: derive an actor's
service name from its `(name, uuid)` and dial it without any
registrar hop.
Do **not** build the fast path in PR 1. Instead, prove the
property with a test (§7.4) and file the follow-up: it changes
`discovery` semantics (name→instance derivation must be a
documented, stable, cross-language-able hash) and deserves its
own design.
### 5.2 Layer B — the topology service (`TIPC_TOP_SRV`)
This is what makes #378's "end game cluster proto" claim real:
a *subscription* to name-table events, i.e. push-based
`register`/`deregister` for free, replacing the registrar's
polled `find_actor()`.
Mechanics (verify each field against
`linux/include/uapi/linux/tipc.h` + `net/tipc/topsrv.c` at
implementation time — the struct layout below is from the uapi
header and the byte-order caveat is real):
```python
# SOCK_SEQPACKET connected to the topology server
sock = trio.socket.socket(AF_TIPC, SOCK_SEQPACKET)
await sock.connect((
socket.TIPC_ADDR_NAME,
socket.TIPC_TOP_SRV, # == 1
socket.TIPC_TOP_SRV,
0,
))
# struct tipc_subscr {
# struct tipc_name_seq seq; /* 3 * __u32: type, lower, upper */
# __u32 timeout; /* TIPC_WAIT_FOREVER == ~0 */
# __u32 filter; /* TIPC_SUB_{PORTS,SERVICE,CANCEL} */
# char usr_handle[8];
# } /* == 28 bytes */
_SUBSCR_FMT: str = '=IIIII8s' # ⚠ 5*I is 20 -> use '=5I8s'
```
- **byte order**: the topology server historically accepts both
host and swapped order and auto-detects; modern kernels are
strict-ish. Pack native (`'='`) first, and if the server
closes the connection immediately, retry with `'>'`. Encode
that as a one-time probe helper
`_detect_topsrv_endianness()` cached at module level — and
put a `# ?TODO` pointing at `net/tipc/topsrv.c` for someone
to make it deterministic.
- **events**: `struct tipc_event` is `event: u32`,
`found_lower: u32`, `found_upper: u32`,
`port: {ref: u32, node: u32}`, then the 28-byte subscription
echo. `event ∈ {TIPC_PUBLISHED, TIPC_WITHDRAWN,
TIPC_SUBSCR_TIMEOUT}`.
⚠️ **CORRECTION**: that totals **48** bytes
(`4 + 4 + 4 + 8 + 28`), not the 40 an earlier revision of this
plan claimed. Verified via `struct.calcsize()` at step 0. Use
`'=5I8s'` (28) for the subscription and a 48-byte read for the
event.
⚠️ also: python exposes `TIPC_WAIT_FOREVER` as **`-1`**, not
`0xFFFFFFFF`, so it must be masked (`& 0xFFFFFFFF`) before
packing into an unsigned `'I'` field.
- **trio shape** — this is where the "nearly-functional,
modern-async" style pays off; expose it as an `@acm` yielding
a `trio` receive-channel of typed events, *not* a class:
```python
@acm
async def open_topology_events(
stype: int = TRACTOR_STYPE,
lower: int = 0,
upper: int = 0xFFFFFFFF,
filter: int = TIPC_SUB_SERVICE,
timeout: int = TIPC_WAIT_FOREVER,
buf_size: int = 64,
) -> AsyncGenerator[
trio.MemoryReceiveChannel[TIPCNameEvent],
None,
]:
...
```
with `TIPCNameEvent(msgspec.Struct, frozen=True)` fields
`kind: Literal['published','withdrawn','timeout']`,
`addr: TIPCAddress`, `node: int`, `ref: int`. One
`trio.lowlevel`-free implementation: a nursery-spawned reader
task doing `await sock.recv(48)` in a loop and
`send_nowait()`ing decoded events, with the `@acm` closing the
socket on exit → reader gets `ClosedResourceError` → cancel
scope collapses. Standard `tractor` `@acm` discipline.
- **consumer**: `tractor/discovery/_registry.py` gains an
optional "watch" mode so a registrar (or any actor) can keep
a live view of the actor set without polling. Sketch the
integration in the follow-up issue; do not wire it in PR 1.
- **`SOCK_SEQPACKET` is fine here** because this socket never
goes through `MsgpackTransport` — it's a plain trio socket
used with `recv()`. The contract's "`SOCK_STREAM` only"
constraint applies to `MsgTransport` streams, not to this.
---
## 6. Commit sequencing (each independently reviewable + green)
**STATUS** (gh PR #493, stacked on #492): steps 1-5 landed as
9 commits, `22ef362d..51d7133f`. Acceptance bar met — 122
passed / 1 xfailed / 2 xpassed under `--tpt-proto tipc` across
`ipc`, `discovery`, `runtime`, `spawning`, `local`, `rpc`,
`cancellation`; `tcp`/`uds` unchanged. Steps 6-7 remain.
Two commits fell out that this plan did NOT anticipate,
- a wire-spec widening for the 4-tuple (§9), and
- an unrelated `devx.pformat` crasher that masked EVERY
send-side `MsgTypeError`; it's on `main` and every branch,
so it wants cherry-picking out of this stack.
1. `_server.py`: add `Address.rebind_from_sockname:
ClassVar[bool]`, gate the `getsockname()` reconciliation on
it, `True` for tcp/uds. Test: tcp `port=0` unchanged.
2. `tractor/ipc/_tipc.py`: `TIPCAddress` + `is_tipc_available()`
predicate + `start_listener()`. No transport yet.
Tests: address round-trip (`unwrap`/`from_addr`/`wrap_address`),
`get_random()` uniqueness, bind/listen + `SO_ACCEPTCONN`
tolerance, `EAFNOSUPPORT` → actionable `ConnectionError`.
3. `MsgpackTIPCStream` + `connect_to()` + `get_stream_addrs()`.
Test: two `trio` tasks in one proc exchange a msg over
`Msgpack` framing (no `tractor` runtime).
4. registration tables (contract §2 items 1-6, 9) +
`pyproject.toml` mark/extra. Test: full suite under
`--tpt-proto tipc` (§7.3).
5. maddr support (`str` form + prefix special-case) + docs.
6. `open_topology_events()` @acm + its tests (layer B).
7. docs page + `docs/` example.
Per project convention, a reproducing/guard test lands in its
own commit **before** the fix it guards.
---
## 7. Testing
### 7.1 the capability predicate (in `_tipc.py`, public)
```python
def is_tipc_available() -> bool:
'''
True iff this kernel can create an `AF_TIPC` socket, i.e.
the `tipc` module is loaded.
'''
try:
socket.socket(socket.AF_TIPC, socket.SOCK_STREAM).close()
return True
except OSError:
return False
```
Cache it in a module global (it can't change without a
`modprobe`, and a cold call costs a syscall). Pure predicate, no
side effects, no logging.
### 7.2 gating
- `pytest.mark.tipc` registered in
`_testing/pytest.py::pytest_configure()` alongside `no_tpt`,
`skipon_spawn_backend` et al.
⚠️ **CORRECTION**: an earlier revision said `pyproject.toml`;
the repo has no `[tool.pytest.ini_options] markers` table and
registers every custom mark via `config.addinivalue_line()`.
Per contract §0, the code wins.
- module-level
`pytestmark = pytest.mark.skipif(not is_tipc_available(),
reason='`tipc` kernel module not loaded (`modprobe tipc`)')`
in `tests/ipc/test_tipc.py`.
- `--tpt-proto tipc` with no module must fail **loudly and
early** with the actionable message, not with 400 confusing
timeouts. Add the check to the `tpt_protos` fixture's existing
per-proto validation loop (`_testing/pytest.py:795`): if the
chosen `Address` type exposes an `is_available()`-style
classmethod, call it and `pytest.fail()` with its reason.
Generalize (don't special-case tipc) — plans 02/03 need the
same hook.
### 7.3 CI
- add a job matrix entry `--tpt-proto tipc` that runs
`sudo modprobe tipc` in a `before` step. GH's
`ubuntu-latest` runners do allow `modprobe tipc` (the module
ships with the standard Ubuntu kernel package); verify in a
throwaway workflow before wiring the matrix. If it turns out
to be unavailable, fall back to a container job with
`--privileged`/`--cap-add NET_ADMIN`, and mark the job
`continue-on-error` until it's proven stable.
- cross-node TIPC (bearer) cannot be CI'd; cover it with a
documented manual smoke test in the docs page, in the style
of gh #482's LAN examples.
### 7.4 backend-specific tests worth writing
- **name-publication is discovery**: bind a listener on
`(stype, inst)`, then from a second task `connect()` by name
and assert it lands — *without* any `tractor` registrar.
- **`get_random()` collision resistance**: 10k `get_random()`
calls with no live runtime.
⚠️ **CORRECTION**: asserting **10k distinct** is a ~1.2%
flaky test, not a guarantee —
`P(collision) ≈ 1 - exp(-n²/2^33) ≈ 1.16e-2` for `n=10k` in a
32-bit instance space. That's ~1-in-86 runs red, which the
project's fix-flakes-at-source rule forbids. Assert
`>= n - 2` instead (`P(>2 collisions) ≈ 1e-7`) and document
the arithmetic inline.
Also add a *deterministic* sibling asserting the derivation
is a pure fn of the seed, which is the property the (§5.1)
registrar-less fast path will actually depend on.
⚠️ do **NOT** take §9's "fold a 6-byte digest into
`(stype_low, instance)`" escalation: §5.2's topology
subscription can only watch **one** service type, so varying
`_stype` per-actor would need 65536 subscriptions and kills
layer B outright. The instance space is 32b and that's that;
if crosstalk ever bites for real, the answer is the post-bind
verification handshake, not stype bits.
- ✅ **SETTLED — round-robin surprise is REAL**: two listeners
bound to the *same* `(stype, inst)` both bind fine and
connects alternate strictly (`b,a,b,a,b,a` observed over 6
dials). So a `get_random()` clash is *silent crosstalk*, never
`EADDRINUSE`. Assert the observed behaviour and reference it
from the `get_random()` docstring so the next reader knows
why the hash matters.
- **scope isolation**: a `TIPC_NODE_SCOPE` bind is not visible
to a cluster-scope lookup from another node (manual/marked).
- **importance opt** round-trips via `getsockopt`.
- **graceful + abrupt close** produce `TransportClosed` with the
same `loglevel` classification as tcp/uds — i.e. re-run the
relevant `tests/ipc/test_each_tpt.py` cases parametrized over
the new proto rather than writing new ones.
---
## 8. Deployment / docs deliverable
A `docs/` page (and/or an `examples/` script) covering:
```bash
# single host, node-scope only
sudo modprobe tipc
tipc node get addr
# multi-host over ethernet (pairs beautifully with plan 03's wg)
sudo tipc bearer enable media eth device eth0
# ...or over UDP when L2 isn't available:
sudo tipc bearer enable media udp name uc localip 10.0.11.1
tipc link list
tipc nametable show # <- see tractor's published services!
```
`tipc nametable show` displaying live `tractor` actors is the
single best demo this backend has; lead with it.
---
## 9. Known risks + escalations
Status column reconciled against the **step-0 probe on a live
kernel** (`modprobe tipc`, py3.13) plus the landed impl. Rows
marked ⚠️ are the ones whose *stated* mitigation turned out to
be wrong or insufficient.
| risk | status | mitigation |
| --- | --- | --- |
| `_instance` hash collision → silent crosstalk (two actors share a service name, TIPC round-robins connects between them) | ✅ **confirmed real** — dup binds both succeed, dials alternate strictly | `blake2b` 32b digest + §7.4 tests. ⚠️ the "6-byte digest folded into `(stype_low, instance)`" escalation is **withdrawn** — it breaks §5.2's single-type subscription. Real escalation is a post-bind verification handshake |
| kernel/module unavailability everywhere (dev boxes, macOS, CI) | ✅ handled | `is_tipc_available()` + the generic `Address.is_available() -> (ok, why_not)` hook consumed by the `tpt_protos` fixture; module stays importable on non-linux via uapi-value fallbacks. TIPC is *opt-in cluster* only, never a default |
| `getsockname()` returns port-id not name | ✅ confirmed (true even *pre*-bind) | `rebind_from_sockname` opt-out (§3.2), landed first |
| dial of an unpublished name doesn't normalize | ⚠️ **worse than stated**`EHOSTUNREACH` is a **bare `OSError`**, not a `ConnectionError` subtype | `_reraise_as_connerr()` is REQUIRED for contract §4, not polish (§3.3) |
| a connect-then-drop peer kills the whole actor via `ENOTCONN` from `getpeername()` | ⚠️ **NOT in the original plan; found by our own test harness** | tolerant `getsockname`/`getpeername` helper degrading to a port-id-less addr (§3.4) |
| the unwrapped 4-tuple doesn't fit the wire msg-spec | ⚠️ **NOT in the original plan**`SpawnSpec.reg_addrs`/`.bind_addrs` pinned a 2-tuple | widen to `UnwrappedAddress`, **variadic** `tuple[str\|int, ...]` since `msgspec` refuses a union w/ >1 array-like type. Own commit; first real bite of contract §1.1 |
| `SO_ACCEPTCONN` rejected by `AF_TIPC` | ✅ **non-issue** — answers `1` | none needed; pinned by a test anyway |
| unregistered `/tipc` multiaddr proto | ✅ handled (interim) | `str` maddr + `parse_maddr()` prefix special-case *before* `Multiaddr()` (§4); upstream track gh #483. Keeps gh #443 blocked |
| stale docs (#378 notes tipc.io docs may be out of date) | ✅ still true | treat `include/uapi/linux/tipc.h` + `net/tipc/` as the only normative source; cite file+symbol in code comments |
| `SOCK_SEQPACKET` topology framing byte-order | ⏳ open (layer B) | probe helper + `?TODO` (§5.2). Note the event struct is **48B not 40B** and `TIPC_WAIT_FOREVER` is `-1` in python |
Non-risks worth recording so nobody re-litigates them:
- **graceful peer close arrives as `BrokenResourceError`
/`ECONNRESET`, not a clean 0-byte EOF** like tcp/uds. Benign:
`MsgpackTransport._iter_packets()` already `match`es
`'Connection reset by peer'` into the `loglevel='transport'`
"normal operation breakage" branch, so `TransportClosed`
classification is unchanged. Worth a sentence in the docs
page (§8) since it *looks* alarming in transport logs.
- **`tipc nametable show` really does list our published
services** (type `1953628160` == `0x74720000`), so the §8 demo
works as advertised.
## 10. Follow-up issue seeds
- **register `/tipc` in the multiaddr spec**, mirroring the `wg`
track (multiformats/py-multiaddr#107/#108 + gh #483). Same
shape of work: propose the proto + code, land a codec in
`py-multiaddr`, then drop our `str`-maddr fallback (§4). Worth
filing *alongside* the `wg` spec-submission issue so both
proposals go up together rather than as one-offs.
- registrar-less discovery fast path via name derivation (§5.1)
- `TIPC_TOP_SRV`-driven push registry in
`discovery/_registry.py` (§5.2)
- `TIPC_IMPORTANCE` for the parent<->child lifetime channel
(§3.3) — genuinely novel supervision QoS, no other backend
can do it
- TIPC multicast / group messaging as a *broadcast* transport
for `tractor.trionics` fan-out (explicitly not `MsgTransport`)
- dual-link resiliency / multi-homing (#378's "hybrid dual link")
once bearers are scripted in the docs

View File

@ -0,0 +1,566 @@
# Plan 02 — QUIC backend via `iroh` FFI, uniffi-async rewritten onto `trio`
Tracks gh [#353]. Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: the value of `iroh` over "just QUIC" is
`NodeId`-addressed, NAT-traversing, relay-fallback endpoints —
i.e. a `tractor` actor tree that spans hosts *without* a
reachable listening socket. The cost is that `iroh`'s python
surface is `uniffi`-generated **asyncio** and its listener is not
a socket. This plan spends its complexity budget in exactly two
places: a `trio`-native uniffi future bridge, and a
`trio.abc.Listener`/`Stream` adapter pair. Everything else is
contract boilerplate.
[#353]: https://github.com/goodboy/tractor/issues/353
---
## 1. Library selection (decided, with the rejected alternatives)
**Chosen: `iroh` (PyPI, from `n0-computer/iroh-ffi`), pinned to
a single minor.** The `iroh` python package is a `uniffi`
binding over the rust `iroh` crate (QUIC via `quinn`/`noq`).
Rejected, and why — record these so the next implementer doesn't
relitigate:
- **`aioquic`** (sans-io + asyncio): genuinely trio-portable
(`hypercorn` already pairs its sans-io core with a trio UDP
server, see the links in #353) and dependency-light. But it
gives us *only* QUIC — no NodeId identity, no hole punching,
no relay. We'd be reimplementing iroh's whole reason for
existing. **Keep as the documented fallback** if the FFI
bridge (§2) proves unmaintainable; the `MsgTransport` and
`Listener` adapters from §3 are ~90% reusable against an
`aioquic` core, which is a deliberate design property of this
plan.
- **`quiche` / `quinn` via a hand-rolled PyO3 ext**: strictly
more work than reusing `iroh-ffi`, and puts us in the
build-wheels business.
- **`trio-asyncio`**: viable *shortcut* to run the asyncio-shaped
bindings under trio, and `tractor` already ships
infected-asyncio machinery (`tractor.to_asyncio`,
`tests/test_infected_asyncio.py`). Rejected as the *primary*
design because it makes every IPC send/recv cross a
loop-boundary shim in the hot path, and because #353 asks
explicitly for the asyncio support to be "rewritten for trio".
**But**: build it first as the throwaway spike (§6 step 0) to
de-risk the iroh API surface before writing the bridge.
Version pinning: `iroh` moves fast and has had breaking
API renames across minors. Pin `iroh>=X.Y,<X.Y+1` in a `quic`
extra, and **write down the exact resolved version + the
generated `iroh/_uniffi*` module layout** in the module
docstring, because §2 depends on generated-code internals.
**Step 0 of implementation is an API-truth pass**: install the
pinned `iroh`, `python -c "import iroh; help(iroh)"`, and record
in this doc's §1.1 the real names of: endpoint builder, secret
key type, `connect`/`accept`, bi-stream open/accept, the
send/recv methods and their exact signatures/return types, and
whether they're `async def`. Everything below uses *provisional*
names and must be reconciled. Do not skip this; do not guess
from memory.
### 1.1 API-truth table (fill in during step 0)
| concept | provisional name | actual (fill in) |
| --- | --- | --- |
| secret key | `iroh.SecretKey.generate()` | |
| endpoint builder | `iroh.Endpoint.builder(...).bind()` | |
| node id | `endpoint.node_id() -> str` | |
| node addr (relay + direct) | `iroh.NodeAddr` | |
| dial | `await endpoint.connect(node_addr, alpn)` | |
| accept conn | `await endpoint.accept()` | |
| open bi-stream | `await conn.open_bi()` | |
| accept bi-stream | `await conn.accept_bi()` | |
| send | `await send_stream.write_all(b)` | |
| recv | `await recv_stream.read(n) -> bytes\|None` | |
| half-close | `await send_stream.finish()` | |
---
## 2. The `trio`-native uniffi future bridge (`tractor/ipc/_uniffi_trio.py`)
### 2.1 what uniffi actually generates
`uniffi`'s async support does not use asyncio *semantically*
it uses asyncio only as the *executor* for a poll loop. The
generated python for an `async fn` is, in shape:
1. call `_uniffi_..._<method>(...)` → returns an opaque
`RustFuture` handle (a `void*`/`u64`).
2. loop: call
`ffi_..._rust_future_poll_<T>(handle, callback, callback_data)`.
The callback is a C-ABI fn pointer invoked **from an
arbitrary rust thread** with a poll-result code
(`READY`/`MAYBE_READY`).
3. the generated glue's callback resolves an
`asyncio.Future` via `loop.call_soon_threadsafe(...)`; the
coroutine awaits it, then re-polls.
4. on ready: `ffi_..._rust_future_complete_<T>(handle,
&call_status)` → the value; then
`ffi_..._rust_future_free_<T>(handle)`.
**The asyncio dependency is confined to step 3.** That is the
whole insight: the bridge is ~40 lines.
### 2.2 the trio version
```python
async def await_rust_future(
poll: Callable, # ffi_..._rust_future_poll_<T>
complete: Callable, # ffi_..._rust_future_complete_<T>
free: Callable, # ffi_..._rust_future_free_<T>
handle: int,
lift: Callable[[Any], Any],
) -> Any:
'''
Drive a `uniffi` rust-future to completion on the current
`trio` task, bridging rust-thread wakeups via
`TrioToken.run_sync_soon()`.
'''
token = trio.lowlevel.current_trio_token()
while True:
wake = trio.Event()
# NOTE, invoked from a *rust* thread!
def _cb(_data, poll_code):
token.run_sync_soon(wake.set)
cb = _UNIFFI_FUTURE_CALLBACK(_cb) # keep a strong ref!
poll(handle, cb, 0)
await wake.wait()
if <poll_code was READY>:
break
try:
status = _UniffiRustCallStatus.default()
res = complete(handle, status)
_uniffi_check_call_status(status) # reuse generated helper
return lift(res)
finally:
free(handle)
```
Critical details, each a real bug if missed:
- **`token.run_sync_soon()` is the only trio API callable from a
foreign thread**, and it is documented as such. Use it; do
*not* use `trio.from_thread.run_sync` (requires a trio thread
context) and do not touch the `Event` directly from the
callback.
- **the poll code must reach the trio side.** Capture it in a
`nonlocal`/1-slot list written by the callback *before*
`run_sync_soon`, since the callback owns the value. Handle
`MAYBE_READY` by re-polling (the loop above does).
- **keep the `ctypes` callback object alive** across the await —
a GC'd `CFUNCTYPE` trampoline is a segfault. Bind it to a
local *and* make sure the local outlives the `poll()` call
window.
- **cancellation.** `await wake.wait()` is a trio checkpoint, so
a `Cancelled` can fire while rust still owns the future. On
cancel we must still `free(handle)` — and per uniffi, the
correct sequence is to call the generated
`ffi_..._rust_future_cancel_<T>(handle)` then continue
polling to completion before `free`. Wrap the whole thing so
the cancel path does:
`with trio.CancelScope(shield=True): cancel(handle); <drain
poll loop>; free(handle)`. **Bounded** shield (add a
`trio.move_on_after()` with a module-level constant) so a
wedged rust future can't make an actor un-cancellable —
`tractor` is SC-first and an unbounded shield here would
violate that.
- **`trio.lowlevel.current_trio_token()`** must be captured on
the trio side (not in the callback).
### 2.3 how to apply it to the generated bindings
Do **not** fork/vendor the generated `iroh` python. Instead ship
a *narrow* re-dispatch shim:
- write `tractor/ipc/_uniffi_trio.py` with `await_rust_future()`
plus a `@cm patch_uniffi_for_trio()` that monkey-patches the
generated module's single async-driver entrypoint (in current
uniffi that's `_uniffi_rust_call_async` / `_rust_call_async`,
one function) to the trio implementation.
- verify at import time that the expected symbol exists and
raise a clear, actionable error naming the pinned `iroh`
version if not. A silent fallback to asyncio would be a
nightmare to debug.
- **plan for this to break on `iroh`/`uniffi` upgrades.** Mitigate
with (a) a unit test that drives one trivial `iroh` async call
under bare `trio.run()` and asserts no event loop was ever
created (`asyncio.get_event_loop_policy()` untouched /
`asyncio._get_running_loop() is None`), and (b) a docstring
pointing at the uniffi codegen template this mirrors.
If step 0 reveals the generated code is *structurally* hostile
to this (e.g. `asyncio` imported and used at module scope for
more than the driver), fall back to option (b): run iroh under
`tractor.to_asyncio` infected mode and open the follow-up to
revisit. Say so in the PR rather than fighting it.
---
## 3. Mapping QUIC onto `MsgTransport`
### 3.1 the layering decision
QUIC natively multiplexes streams inside one connection. The
mapping that preserves *all* existing `tractor` semantics with
the least new code:
```
iroh Endpoint == one per actor (process) -> the "listener"
iroh Connection == one per peer actor -> pooled
iroh bi-stream == one `Channel`/`MsgTransport` -> 1:1
```
- keep the 4-byte `<I` length-prefix framing **unchanged**. It's
redundant-ish over a QUIC stream but it means
`MsgpackTransport` is reused verbatim, and framing is cheap.
Revisit only after it works.
- **one-task-per-stream** falls out naturally, which is exactly
the #353 note about QUIC sub-stream QoS/cancellation fitting
`trio`.
- `layer_key: int = 4` still (QUIC is L4-ish); note in a comment
that this backend is really 4+security+multiplex.
**Connection pooling** is the one place we add state the other
backends don't have: dialing the same peer twice should reuse
the `Connection` and open a second bi-stream. Implement as a
module-level `dict[NodeId, Connection]` guarded by a
`trio.Lock`... **no** — that's a per-process cache with
lifetime/teardown hazards. Instead reuse the codebase's existing
idiom: `tractor.trionics.maybe_open_context()` keyed on the
node-id, which already solves exactly this (one-cached-resource-
per-key, refcounted, teardown-on-last-exit) and whose teardown
semantics were just hardened (gh #488). Use it; do not hand-roll
a cache. Anything concurrency-subtle here should get the
`conc-anal` skill run over it.
### 3.2 `IrohAddress`
```python
class IrohAddress(
msgspec.Struct,
frozen=True,
):
_node_id: str # 32B ed25519 pubkey, hex or z32
_alpn: str = 'tractor/0' # the bindspace!
# optional dial hints; NOT part of identity
maybe_relay_url: str|None = None
maybe_direct_addrs: tuple[str, ...] = ()
proto_key: ClassVar[str] = 'iroh' # ?or 'quic'; see §3.2.1
unwrapped_type: ClassVar[type] = tuple[str, str]
def_bindspace: ClassVar[str] = 'tractor/0'
```
- **`.unwrap() -> (node_id_str, alpn_str)`** — a `(str, str)`
tuple, which is *unambiguously distinct* from
`TCPAddress`'s `(str, int)`. But careful:
`wrap_address()`'s UDS case is
`case (_, filename) if type(filename) is str` — which
**already catches `(str, str)`**. So the iroh `case` MUST be
ordered *before* the UDS case and guarded, e.g.
`case (str() as nid, str() as alpn) if _is_node_id(nid):`
with `_is_node_id()` a cheap length+alphabet check. Add a
regression test asserting a UDS `(dir, filename)` pair still
wraps to `UDSAddress` — this is the exact "wrong transport
loaded" hazard `_addr.py:214` warns about.
- `.bindspace``self._alpn`. This is the honest analogue:
the ALPN is the set of endpoints willing to talk to you, and
two `tractor` deployments sharing an iroh network are
separated by ALPN exactly as two UDS deployments are
separated by directory. Include a `tractor` version/proto
epoch in the default ALPN so incompatible runtimes can't
handshake.
- `.is_valid` → node-id parses, alpn non-empty.
- **`get_root()` is the hard one.** There is no
well-known-port analogue: an iroh node id is a *keypair*, so
"the host's default registrar addr" requires a *persisted
secret key*. Design:
- the root/registrar's secret key lives at
`get_rt_dir() / 'iroh_registrar.key'` (0600), created on
first use.
- `get_root()` must stay **pure and import-time-safe**
(contract §2.3: `_default_lo_addrs` is built at import!).
So `get_root()` *reads* the key file if present and
otherwise returns an `IrohAddress` with
`_node_id=''`/sentinel, and the **generation** happens in
an explicit sibling — `ensure_registrar_key() ->
IrohAddress` — called from the listen path. Pure getter,
explicit setter; do not smuggle key generation into
`get_root()`.
- this almost certainly means `_default_lo_addrs` must become
lazy for this backend. **Land that refactor as its own prep
commit** (a `default_lo_addrs()` that computes per-call
instead of the import-time dict) — it also unblocks plan
03's netns-scoped defaults.
- `get_random()`: generate a fresh `SecretKey` per subactor and
return its node-id. Note this runs post-fork pre-listen
(contract §4) and costs an ed25519 keygen (~µs, fine). The
*secret* can't live in a frozen `Address`, so it must be
stashed where the listen path can find it: a module-level
`dict[node_id, SecretKey]` populated by `get_random()` and
consumed+popped by `start_listener()`. Ugly but honest;
document it and note the alternative (thread the key through
`Endpoint`) as a follow-up.
#### 3.2.1 `proto_key`: `'iroh'` vs `'quic'`
Use **`'quic'`** for the `proto_key`/`--tpt-proto` name and
name the module `_quic.py`, with `iroh` as the *implementation*.
Rationale: it keeps the door open for the `aioquic` fallback
(§1) without a user-visible rename, and it matches how `uds` is
a proto name rather than a lib name. Put `iroh`-specific bits
behind an internal `_iroh` submodule if the file gets big.
### 3.3 the `trio.abc` adapters — where the real work is
Contract §3 says a non-socket backend needs three upstream
generalizations. Land them **as a prep PR, before any iroh
code**, so they can be reviewed on their own merits with
tcp/uds still the only backends:
1. **`Endpoint.start_listener()` must not assume
`.socket.getsockname()`.** Use the same
`Address.rebind_from_sockname: ClassVar[bool]` gate that
plan 01 §3.2 introduces — coordinate so it lands once. (If
plan 01 lands first, this is free.)
2. **`transport_from_stream()` (`_types.py:92`) must not assume
`trio.SocketStream`.** Replace the `sock.family` match with:
check `isinstance(stream, trio.SocketStream)` → existing
family match; else look for a
`stream.tpt_key: ClassVar[MsgTransportKey]` attribute on the
adapter and use it. Keeps the existing path byte-identical
and makes new stream types self-describing (a much better
shape than growing an `isinstance` ladder).
3. **type annotations**: `handle_stream_from_peer(stream:
trio.SocketStream)` → `trio.abc.Stream`; `Endpoint._listener:
SocketListener|None` → `trio.abc.Listener|None`;
`MsgTransport.stream: trio.SocketStream`
`trio.abc.Stream`. Annotation-only, zero behaviour change.
Then the adapters:
```python
class QuicMsgStream(trio.abc.HalfCloseableStream):
'''
A single `iroh` bi-directional QUIC stream presented as
a `trio` byte-stream so `MsgpackTransport` can frame over
it unmodified.
'''
tpt_key: ClassVar[MsgTransportKey] = ('msgpack', 'quic')
def __init__(self, conn, send, recv) -> None: ...
async def send_all(self, data: bytes) -> None: ...
async def wait_send_all_might_not_block(self) -> None: ...
async def receive_some(self, max_bytes: int|None = None) -> bytes: ...
async def send_eof(self) -> None: ...
async def aclose(self) -> None: ...
```
Non-negotiable behaviours (each maps to a `match` case that
already exists in `_transport.py` and must keep working):
- `receive_some()` returns `b''` at clean EOF →
`MsgpackTransport._iter_packets()` sees `header == b''` and
raises `TransportClosed(loglevel='transport')`. **This is the
graceful-disconnect path the whole runtime relies on**; get it
right first.
- a reset/aborted stream → raise `trio.BrokenResourceError`.
- use after local close → raise `trio.ClosedResourceError`
(ideally with `'another task closed this fd'`-equivalent text
absent, so the `raise_on_report` branch at
`_transport.py:290` stays quiet).
- `send_all()` on a closed peer → `trio.BrokenResourceError`.
- honour `trio`'s one-task-per-direction rule: guard with
`trio._util.ConflictDetector` equivalents (or just document +
assert), because `MsgpackTransport` already serializes sends
with a `StrictFIFOLock` but recvs are single-task by
construction.
- **buffering**: if iroh's `read()` doesn't support
"read up to n", `receive_some()` must maintain an internal
leftover buffer. Note `MsgpackTransport` wraps us in
`tricycle.BufferedReceiveStream` anyway, so `receive_some()`
just needs *some* nonzero-progress contract.
```python
class QuicListener(trio.abc.Listener):
'''
Accepts iroh `Connection`s and yields one `QuicMsgStream`
per accepted bi-stream, so `trio.serve_listeners()` spawns
one `handle_stream_from_peer()` per `Channel`.
'''
async def accept(self) -> QuicMsgStream: ...
async def aclose(self) -> None: ...
```
The accept-side subtlety: `trio.abc.Listener.accept()` yields
one stream per call, but iroh gives us *connections* which then
yield *streams*. So `QuicListener` needs an internal
`trio.MemoryReceiveChannel[QuicMsgStream]` fed by a background
task-pair (one task accepting connections, one per connection
accepting bi-streams). `trio.abc.Listener` has no nursery, so:
make the listener **constructed by an `@acm`** that owns the
nursery, and have `start_listener()` be that `@acm`'s driver.
⚠️ this collides with `Endpoint.start_listener()` being a plain
`async def` returning a listener. Two options:
- **(a)** hang the nursery off the `Endpoint`'s existing
`listen_tn``_serve_ipc_eps()` already creates `listen_tn`
and passes it into every `Endpoint` (`_server.py:1063-1074`),
and `Endpoint.listen_tn` is right there. So
`start_listener()` can `self.listen_tn.start_soon(...)` the
acceptor tasks. **Recommended**: no upstream signature change,
correct lifetime (dies with the ep group), and it's why
`listen_tn` is on the struct in the first place.
- (b) change `start_listener()` to a `@acm`. Bigger blast
radius; only if (a) proves insufficient.
Since `start_listener()` is called via
`inspect.getmodule(addr)` with only `addr=` (contract §1.3),
option (a) needs the `Endpoint` itself. Either add `ep=` to the
module-level `start_listener()` call signature (all backends
ignore it except quic → small upstream change, do it as part of
the prep PR and make it keyword-only with a default) or have
`QuicListener.accept()` lazily spawn via
`trio.lowlevel.current_task().parent_nursery` (**rejected** —
fragile, implicit). Do the explicit `ep=` kwarg.
### 3.4 `maddr`
Multiaddr already standardizes the pieces:
```
/ip4/<h>/udp/<p>/quic-v1 # direct
/ip4/<h>/udp/<p>/quic-v1/p2p/<node-id> # direct + identity
/dns/<relay-host>/tcp/443/tls/ws/p2p/<node> # relay-ish
```
- primary form: `/p2p/<node-id>` alone is a legal maddr and is
the *only* required component for iroh dialling — relay +
direct addrs are discovery hints. So `mk_maddr()` emits
`/p2p/<node_id>` and, when known, prefixes the direct
`/ip4/../udp/../quic-v1/`.
- `/p2p/` values are multihash-encoded peer ids; an iroh node-id
is a raw ed25519 key. Converting requires the identity
multihash + libp2p key protobuf wrapper. **Decide**: emit the
raw node-id under a *tractor-local* `/iroh/<node-id>` segment
(needs upstream registration, same track as `wg`/`tipc`,
gh #483) rather than pretending to be a libp2p peer-id we
can't round-trip. Return the `str` form until upstream lands
(`MsgTransport.maddr` is `Multiaddr|str`).
- this backend is the strongest argument for gh #443's
**tunnelled/composed maddr** item: `/ip4/../udp/../quic-v1/..`
*is* a composed stack. Cross-reference plan 03 §5 so the two
grammars land compatibly.
---
## 4. Discovery integration
- iroh's node-id addressing means the `tractor` registrar can
hold `IrohAddress`es that are **reachable from anywhere** with
no port-forwarding — that is the headline feature. The
registrar itself works unchanged.
- iroh has its own discovery (DNS/pkarr/mdns). **Out of scope**;
note in the follow-up that `tractor.discovery` could
eventually delegate to it, which would be the direct analogue
of plan 01's TIPC-topology idea.
- relay servers: default to n0's public relays for the demo,
document self-hosting (docs.iroh.computer's dedicated-infra
page is linked from #353), and make the relay set a
`start_listener()` kwarg.
## 5. Security note
QUIC is TLS-1.3-always and iroh authenticates by node-id, so
this backend is the first `tractor` transport with real
transport security and peer authentication. Two things follow:
1. an **allowlist hook** — an actor should be able to reject
inbound connections from unknown node-ids *before* the
`Aid` handshake. Natural home: a predicate kwarg on
`start_listener()`, evaluated in `QuicListener`'s connection
acceptor task. Sketch it; ship it in PR 1 if cheap (it is).
2. do **not** claim any security property for the other
backends by association. `tcp`/`uds`/`tipc` remain
unauthenticated; that's what plan 03 (wg) is for.
## 6. Commit sequencing
0. **spike (throwaway, not committed)**: drive iroh under
`trio-asyncio`/`tractor.to_asyncio`, echo bytes over a
bi-stream between two procs. Fills in §1.1. Timebox it.
1. prep PR: annotation widening + `rebind_from_sockname` gate +
`transport_from_stream()` `tpt_key` dispatch + `ep=` kwarg on
`start_listener()` + lazy `default_lo_addrs()`. **No new
backend.** Full suite green on tcp *and* uds.
2. `_uniffi_trio.py` + its tests (drive one iroh async call
under bare `trio.run()`; assert no asyncio loop; assert
cancellation frees the future).
3. `QuicMsgStream` + tests against a *loopback* iroh endpoint
pair in one process (no `tractor` runtime): send/recv, clean
EOF → `b''`, reset → `BrokenResourceError`, use-after-close
`ClosedResourceError`.
4. `QuicListener` + `start_listener()` + `IrohAddress` +
key-file mgmt.
5. `MsgpackQuicStream(MsgpackTransport)` + `connect_to()` +
`maybe_open_context()` connection pooling.
6. registration tables + `--tpt-proto quic` + full suite.
7. maddr + docs + a two-host example (pairs with #482's format).
## 7. Testing
- capability predicate `is_quic_available()``iroh` importable
*and* the uniffi driver symbol present at the pinned version.
Same `pytest.fail`-early hook as plan 01 §7.2.
- **the acceptance bar is the same**: whole suite green under
`--tpt-proto quic`. Expect this to shake out real bugs in the
adapters (esp. teardown ordering and `TransportClosed`
classification) — that's the point.
- expect to need **timeout headroom**: iroh endpoint bind +
first connect (relay discovery) is orders of magnitude slower
than a UDS bind. Before touching any test deadline, rule out
the CPU-throttle false-positive (see the project's
`env_cpu_throttle_masquerades_as_regression` note); then, if
real, add a per-proto timeout multiplier to the test harness
rather than editing individual tests.
- a no-network test mode: iroh with relays disabled +
loopback direct addrs only, so CI doesn't depend on n0's
infra. **Make this the default in CI**; mark the relay tests
`pytest.mark.net` and keep them out of the default run.
- leak checks: assert every `SecretKey`/`Endpoint` is closed on
actor teardown (an `Endpoint` left open holds UDP sockets and
relay connections; a leak here shows up as hung tests, not
errors).
## 8. Risks
| risk | mitigation |
| --- | --- |
| uniffi codegen internals shift on upgrade | pinned minor, symbol assertion at import, the "no asyncio loop" test, documented fallback to `to_asyncio` |
| rust-thread callback → trio wakeup mishandled (segfault / lost wakeup / un-cancellable task) | strong ref on the ctypes trampoline; `run_sync_soon` only; **bounded** shielded cancel-drain; run the `conc-anal` skill over the bridge |
| `iroh` wheel availability for 3.13/3.14 on linux+macos | verify in step 0; if missing, that alone may force the `aioquic` fallback |
| QUIC latency/jitter destabilizes the existing suite's timing assumptions | per-proto timeout multiplier, relay-less CI mode |
| `(str, str)` unwrapped form collides with UDS in `wrap_address()` | guarded case ordered first + explicit regression test (§3.2) |
| scope creep into iroh's docs/blobs/gossip crates | this backend is `Endpoint`+`Connection`+bi-streams only; anything else is a separate issue |
## 9. Follow-up issue seeds
- `tractor.discovery` delegating to iroh discovery (DNS/pkarr/mdns)
- per-`Context` QUIC sub-streams: today one `Channel` == one
stream; QUIC would let each `tractor.Context` own its own
stream with independent flow-control and cancellation — this
is the genuinely novel win #353 gestures at, and it's a
runtime-layer change, not a transport one
- unreliable QUIC datagrams for a lossy-ok broadcast transport
(pairs with plan 01's TIPC-multicast seed)
- node-id allowlist → a real `tractor` authz story
- `aioquic` sans-io backend reusing §3's adapters

View File

@ -0,0 +1,464 @@
# Plan 03 — WireGuard (and other tunnels) as a *nested bindspace* via `pyroute2`
Tracks gh [#482] + the tunnelled-maddr item of [#443].
Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: WireGuard is **not** a `MsgTransport`. It is an
interface-layer tunnel that is transparent to `socket(2)`, so
the correct abstraction is a *bindspace* — a scoped,
`@acm`-managed network context that an existing L4 transport
(`tcp`, and later `quic`/`tipc`-over-UDP-bearer) binds *inside*.
This plan implements `Address.namespace` (spec'd but unused
since day one) and the composed/tunnelled maddr grammar, with
`pyroute2` as the netlink codec and as much of the I/O moved
onto `trio` as the library's sans-io layer allows.
[#482]: https://github.com/goodboy/tractor/issues/482
[#443]: https://github.com/goodboy/tractor/issues/443
---
## 1. What exists today (verified, per #482)
- `wrap_address()` accepts maddr `str`s (leading-`/` dispatch,
`_addr.py:262`) but `parse_maddr()` only knows
`/ip4|ip6/<h>/tcp/<p>` and `/unix/<p>`; a `.../wg/u<key>`
maddr raises `ValueError('Unsupported multiaddr protocol
combo')`.
- there is no `wg` proto in the multiaddr *spec* yet, but
multiformats/py-multiaddr#108 (key form `u<base64url>`) is
**merged** as of 2026-07-28 (`f86519da`) — and unreleased, the
latest `0.2.0` predating it. Spec registration is still tracked
by multiformats/py-multiaddr#107 and gh #483.
- so **today's deployable story is declarative**: run `wg-quick`
out-of-band, parse the maddr, strip to the inner
`(host, port)`, verify the pubkey against the live tunnel,
hand the inner addr to `registry_addrs=`/`tpt_bind_addrs=`.
#482 already contains working example code for exactly this.
- `Address.namespace` exists in the Protocol
(`_addr.py:94-101`, "the if-available OS-specific network
namespace key") and **no backend implements it**. This plan is
its first consumer.
## 2. Three layers, three PRs
| layer | what | dep | ships |
| --- | --- | --- | --- |
| **A. declarative** | commit #482's examples; `parse_maddr()` learns `/wg/u<key>` → inner `Address` + verified pubkey | `multiaddr` (already), `wg(8)` CLI | first |
| **B. `pyroute2` read/verify** | replace the `subprocess.run(['sudo','wg','show'])` shelling with netlink queries | `pyroute2` extra | second |
| **C. `@acm` lifecycle** | create/configure/tear down wg ifaces + netns *from the runtime*, as nested bindspaces; implement `Address.namespace` | `pyroute2` + `CAP_NET_ADMIN` | third |
Each is independently valuable and independently reviewable.
**Do not attempt C first** — the interesting design (nested
bindspace `@acm`s) is only well-posed once A has pinned the
address grammar and B has proven the netlink path under trio.
---
## 3. Layer A — declarative `wg` maddrs
### 3.1 the address shape
The decision: **a wg segment annotates an existing address, it
does not create a new address type.** Two candidate encodings;
**pick (a)**:
- **(a) `TunnelledAddress` wrapper** (recommended):
```python
class TunnelledAddress(
msgspec.Struct,
frozen=True,
):
inner: Address # e.g. TCPAddress
tunnel: WGTunnelSpec # proto-specific, frozen
```
with `.proto_key` **delegating to `inner.proto_key`** so every
existing table lookup (`_addr_to_transport`,
`enable_transports` guard at `_root.py:391`,
`transport_from_addr()`) keeps working untouched, and
`.unwrap()` delegating to `inner.unwrap()` so **nothing new
crosses the wire**. `.namespace` and `.bindspace` come from
the tunnel spec. The wrapper is stripped (`→ .inner`) at the
moment of bind/connect.
- ⚠️ `is_wrapped_addr()` (`_addr.py:194`) tests
`type(addr) in _address_types.values()` — a `bidict` of
proto_key→type. `TunnelledAddress` isn't in it and must not
be (it's not 1:1 with a proto). So either add an explicit
`isinstance(addr, TunnelledAddress)` clause there, or give
the wrapper a marker and test structurally. Do the former;
it's two lines and honest.
- the reflection in `Endpoint.start_listener()`
(`inspect.getmodule(self.addr)`) would resolve to the
*wrapper's* module, not the transport's. **So the wrapper
must be unwrapped before it reaches `Endpoint`** — i.e. by
the bindspace `@acm` (layer C) or by `parse_maddr()`
(layer A). State this loudly in the docstring; it's the #1
way to get this wrong.
- (b) add fields to each existing `Address` type. Rejected:
duplicates tunnel logic per-backend and pollutes `.unwrap()`.
```python
class WGTunnelSpec(
msgspec.Struct,
frozen=True,
):
peer_pubkey: str # std-base64 `wg(8)` form
iface: str = 'wg0'
netns: str|None = None
# layer-C-only fields, unset in layer A
maybe_endpoint: tuple[str, int]|None = None
maybe_allowed_ips: tuple[str, ...] = ()
```
### 3.2 `parse_maddr()`/`mk_maddr()`
Grammar — **verified** against py-multiaddr#108, first on the
`baudco/py-multiaddr@wg_support` branch and re-verified after it
merged upstream (`multiformats/py-multiaddr@f86519da`); all three
forms below parse *and* round-trip. Note the codec also validates
that the key decodes to exactly 32 bytes, so a truncated key is a
`StringParseError`, not a silently-mangled parse:
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\_______ bearer __________/\__ key __/\______ overlay ______/
underlay, wg `ListenPort` the ONLY part we bind
```
The `/wg/` segment is **infix, not suffix** — the segments
*before* it are the wg **bearer** (the underlay `(ip, udp-port)`
that `wg(8)` itself listens on, per the codec docstring's own
`/ip4/1.2.3.4/udp/51820/wg/{key}` example), and the segments
*after* are the **overlay** endpoint that `tractor` binds.
⚠️ **CORRECTION** — an earlier revision of this plan (and the
examples in gh #482) used a *suffix* form
`/ip4/10.0.11.1/tcp/1616/wg/u<key>`. That parses, but it is
semantically inverted: it puts the overlay addr where the bearer
belongs, `tcp` where wg's `udp` `ListenPort` goes, and declares
no overlay endpoint at all. `parse_wg_maddr()` in
`examples/multihost/wg_lan/` now rejects it with an actionable
error.
Observed protocol-name lists, for writing the `match`:
| maddr | `[p.name for p in m.protocols()]` |
| --- | --- |
| `/ip4/1.2.3.4/udp/51820/wg/u<k>` | `['ip4','udp','wg']` |
| `/ip4/../udp/../wg/u<k>/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` |
- so the three parts have **three different owners**, and only the
third is an `Endpoint`:
| part | bound by | in the runtime? |
| --- | --- | --- |
| bearer | kernel, via `wg-quick`/`pyroute2` | no |
| `/wg/u<key>` | nothing — it's an identity | no, verified out-of-band |
| overlay | `tractor`'s `IPCServer` | **yes**, as `.inner` |
This owner-split is the real axis of the design, *not* whether
the maddr stack is "composed" (it is).
- `parse_maddr()` gains a case on
`[('ip4'|'ip6'), 'udp', 'wg', ('ip4'|'ip6'), <inner-l4>]`
build the inner `Address` from the trailing segments, decode
the multibase key to std-base64, and return
`TunnelledAddress(inner=..., tunnel=WGTunnelSpec(...))` with
the bearer recorded in the spec.
- keep the existing 2-proto cases byte-identical; add the new
case *after* them.
- generalize by **peeling at the tunnel segment**: split
`proto_names` at `'wg'`, hand the trailing list to the existing
inner-stack logic, and recurse for nested tunnels. Write it as
a small pure fn `_peel_tunnel_segs(proto_names) ->
(bearer_names, tunnel_specs, inner_names)`. This is also what
makes a wg-inside-wg stack fall out for free.
- `mk_maddr()` inverse for `TunnelledAddress`.
- **pending an upstream release**: py-multiaddr#108 is merged, so
`Multiaddr('/…/wg/u…')` parses — but off a `[tool.uv.sources]`
`rev` pin, since no release carries the codec. Gate the tests
on `_have_wg_maddr_proto()`, implemented as
`protocols.protocol_with_name('wg')` under
`except ProtocolNotFoundError`. Do **not** probe by parsing a
dummy like `Multiaddr('/wg/uAAAA')` — the codec enforces a
32-byte key, so that raises even when the proto *is* known. Do
**not** hand-roll a `wg` parser in `tractor` — the whole point
of #429 was dropping the NIH parser.
### 3.3 verification helper (pure, composable)
Port #482 §2's helpers into `tractor/discovery/_tunnel.py` as
*pure functions* + one impure probe, cleanly separated:
```python
def parse_wg_maddr(maddr: str) -> TunnelledAddress: ... # pure
def wg8_pubkey(multibase_key: str) -> str: ... # pure
def verify_wg_peer(spec: WGTunnelSpec) -> bool: ... # impure probe
```
In layer A `verify_wg_peer()` may shell out (`wg show <if>
peers`), but it must be a *single* function so layer B swaps
only its body. Never call it implicitly from
`wrap_address()`/`parse_maddr()` — parsing must stay pure and
side-effect-free; verification is the *caller's* explicit step
(and later, the bindspace `@acm`'s).
### 3.4 deliverables
- `examples/` scripts distilled from #482 §§3-5 (this is the
unchecked "commit examples from ^" bullet in #443). They live
under `examples/multihost/``test_docs_examples.py` walks
`examples/` recursively and runs every collected file as a
subproc asserting `rc == 0` (it doesn't even filter by
extension, so a stray `README.md` would be `python`-run too),
and `'multihost' not in p[0]` is already in its exclusion
list. Anything needing a real second host or a live tunnel
belongs there.
- a `docs/` page: tunnel setup, the maddr form, the two-host
run. Keep prose in the docs; keep the examples runnable and
minimal.
- tests: maddr round-trip, `TunnelledAddress` delegation
(`proto_key`/`unwrap` identical to inner), `wrap_address()`
regression (a tunnelled maddr `str``TunnelledAddress`; a
plain one → unchanged), and **a real end-to-end over a
locally-created wg pair** gated on `CAP_NET_ADMIN` (see §5.3).
---
## 4. Layer B — `pyroute2` under `trio`
### 4.1 the library situation (verify at implementation time)
`pyroute2` ≥0.9 rewrote its core onto **asyncio**
(`AsyncIPRoute`; the sync `IPRoute` wraps it with its own loop).
It also ships a `WireGuard` netlink (generic-netlink) module
supporting `.set(iface, private_key=..., peer={...})` and
`.info(iface)`, plus `pyroute2.netns` / `NetNS` for namespaces,
and `IPRoute.link('add', kind='wireguard', ifname=...)`.
Three integration options, in increasing trio-nativeness:
- **(1) `trio.to_thread.run_sync()` around the sync API.**
Netlink ops here are one-shot, sub-millisecond, and happen at
bind/teardown time only — *not* in the msg hot path. This is
the **correct default**: it's ~10 lines, uses a battle-tested
API, and costs nothing where it's used.
- **(2) sans-io: `trio.socket` + pyroute2's message codecs.**
`pyroute2`'s message classes
(`pyroute2.netlink.rtnl.*`, `pyroute2.netlink.generic.wireguard.wgmsg`)
encode/decode independently of its I/O core. So a
`tractor/ipc/_netlink.py` with a small trio `NetlinkSocket`
(`trio.socket.socket(AF_NETLINK, SOCK_RAW|SOCK_DGRAM, proto)`,
`sendto`/`recv`, seq/pid matching, `NLMSG_DONE`/`NLMSG_ERROR`
handling) + pyroute2 codecs is very achievable and is the
honest reading of "as much trio wrapping as possible where any
other async support can be replaced".
**Do this for the paths we actually need** (link add/del,
addr add, wg get/set, netns bind) and *only* those — a
general netlink client is out of scope.
- (3) reimplement the codecs. Never.
**Recommended split**: ship (1) first so layer B is a small,
reviewable, behaviour-preserving swap of `verify_wg_peer()`'s
body; then land (2) as a follow-up commit for the read path
(`wg get`, `link get`) where the sans-io surface is smallest,
and keep (1) for the privileged mutating ops. Measure before
converting anything else — there is no perf argument here, only
a "no foreign event loop in a trio actor" argument, which (1)
already satisfies (a thread is not an event loop).
Explicitly **do not** pull in `trio-asyncio` for pyroute2: it
would be the one place in the runtime where an asyncio loop
exists for no reason.
### 4.2 API shape
Pure-ish, functional, `@acm` for anything with teardown:
```python
async def read_wg_peers(
iface: str = 'wg0',
netns: str|None = None,
) -> tuple[str, ...]: ... # base64 pubkeys
async def read_wg_pubkey(iface: str = 'wg0', ...) -> str: ...
```
and `verify_wg_peer()` becomes a thin composition over the two.
Note the pure-getter rule: no `read_wg_peers(..., create=True)`.
---
## 5. Layer C — nested bindspace `@acm`s + `Address.namespace`
This is the part #443 and `multiaddr_declare_eps.md` actually
ask for: *"for any tunneled maddr-`str`-entry we deliver a
data-structure which can easily be passed to nested `@acm`s
which consecutively setup nested net bindspaces for binding the
endpoint addrs"*.
### 5.1 the composition
```python
@acm
async def open_bindspace(
addr: TunnelledAddress,
) -> AsyncGenerator[Address, None]:
'''
Enter the net-bindspace implied by `addr`'s tunnel stack,
yielding the *inner* `Address` ready to bind/connect.
Nests: one `@acm` per tunnel segment, outermost-first, so
a 2-deep stack is just two nested `async with`s and the
teardown order is guaranteed by `trio`.
'''
```
with per-tunnel-kind implementations:
```python
@acm
async def open_netns(name: str) -> AsyncGenerator[None, None]: ...
@acm
async def open_wg_iface(spec: WGTunnelSpec) -> AsyncGenerator[WGTunnelSpec, None]: ...
```
and a driver that folds a list of specs into nested contexts
(`contextlib.AsyncExitStack` for the N-deep case). The
`parse_endpoints()` API (`_multiaddr.py:153`) is the front door:
it already returns `dict[name, list[Address]]` and the
`multiaddr_declare_eps.md` sketch anticipates the recursive
`dict[str, list[Address]]|dict[...]` return for tunnelled
entries. Extend it to carry the tunnel stack, not to *enter* it.
### 5.2 `Address.namespace`, at last
- `TunnelledAddress.namespace``(kind, id)` e.g.
`('netns', 'tractor-wg0')`.
- **and** the existing backends should implement it as `None`
explicitly (they currently just don't define it), so the
Protocol stops lying.
- consumers to audit: nothing reads `.namespace` today — so
adding it is safe, but the *point* is that
`Endpoint`/`Server.pformat()` should start showing it (there's
already a `# !TODO, always be ns aware!` +
`f'|_netns: {netns}\n'` placeholder sitting in
`Endpoint.pformat()`, `_server.py:645`). Fill that in; it's
the cheapest possible proof the layer is wired.
### 5.3 the netns/process reality — read this before designing
**The headline consequence, stated up front**: netns is a
**runtime-level config API, not an actor-app-code API.** It is
declared as part of how an actor process is *brought up* — a
spawn-time/boot-time input alongside `enable_transports` and
`tpt_bind_addrs` — and it is **not** dynamically re-enterable by
app code once the actor is live. There is deliberately no
`await actor.enter_netns(...)`. Two hard reasons, both below:
`setns(2)` doesn't retroactively move existing sockets, and it's
per-thread rather than per-process. Anything that *looks* like a
mid-life API here would be a footgun that silently leaves the IPC
server bound in the old namespace.
- `setns(2)` with `CLONE_NEWNET` affects **the calling thread
only**, and sockets already created keep their original netns.
A trio actor is effectively single-threaded for our purposes,
so "enter the netns, *then* bind" works — but any
`to_thread` worker (§4.1 option 1!) is in the **original**
netns unless it also `setns`. Concretely: a wg query issued
via `trio.to_thread` will hit the wrong namespace. Either
pass `netns=` down to `pyroute2` (which does the
fork/setns dance itself) or pin a dedicated worker. **This is
the single subtlest bug in this plan — write the test first.**
- entering a netns is *process-global-ish and irreversible-ish*
in practice. Therefore: **netns membership belongs to the
actor process, decided before the runtime binds**, not to a
mid-life `@acm`. Design:
- the root/parent decides the netns for a subactor and passes
it in the spawn spec (there's already
`enable_transports`/`accept_addrs` plumbing at
`_runtime.py:1595-1615` — the netns rides alongside).
- the child, in `_runtime.async_main()` **before**
`IPCServer.listen_on()`, enters it.
- the mid-life `@acm` form is then only for the *root* /
single-actor case, and for iface creation (which is
genuinely scoped).
- document the constraint rather than hiding it; a
`RuntimeError` if `open_netns()` is entered after any
listener exists.
- privileges: iface/netns creation needs `CAP_NET_ADMIN`.
Never `sudo` from inside the runtime. Two supported modes:
(i) pre-provisioned out-of-band (layers A/B — the default,
and what #482 documents), (ii) runtime-managed when the
process already holds the cap. Detect with a cheap
`os.geteuid()==0 or CAP_NET_ADMIN in /proc/self/status`
probe and *fail loudly with an actionable message* otherwise.
- teardown must be idempotent and tolerant: an iface/netns
already gone must not strand the rest of the teardown — the
exact lesson `_uds.close_listener()`'s `FileNotFoundError`
tolerance and `_serve_ipc_eps()`'s per-ep `try/except`
encode. Mirror both.
### 5.4 tests for layer C
- unit: fold-N-tunnel-specs-into-nested-`@acm`s, with fakes; assert
enter/exit ordering (outermost-last-out) via a trace list.
- integration, gated on `CAP_NET_ADMIN` (skip otherwise, and in
CI run it in a `--cap-add NET_ADMIN` container job): create two
netns + a wg pair entirely in-process, boot a `tractor` root in
one and a subactor in the other, `find_actor()` across the
tunnel. This is a *fantastic* test to have and is fully
self-contained — no second host, no `sudo` in the test body.
- the `to_thread`-netns-mismatch regression from §5.3, written
**first** (red), then the fix (green), per project convention.
---
## 6. "Other shuttle-able tpts"
The generalization the #482 follow-up gestures at: once
`TunnelledAddress` + `open_bindspace()` exist, the same
machinery covers any iface-layer tunnel `pyroute2` can drive —
`ipip`/`gre`/`sit`/`vxlan`/`geneve`/`bridge`/`veth`. Keep
`WGTunnelSpec` as *one* frozen struct among a
`TunnelSpec = WGTunnelSpec|VxlanTunnelSpec|...` union with a
`kind: ClassVar[str]`, and dispatch `open_*` by `match` on it.
Design for it now (union + `match`), implement only `wg` +
`netns`. `veth`-pairs-in-netns is the natural second one because
it makes the §5.4 integration test possible without wg at all —
consider doing it *first* for exactly that reason.
## 7. Non-goals
- no wg userspace implementation, no key exchange, no
`wg-quick` reimplementation (config-file parsing is
out of scope; take structured input).
- no persistence of private keys beyond what layer C's iface
creation needs (and that stays in `get_rt_dir()`, 0600).
- macOS/Windows: layers B/C are Linux-only. Layer A (declarative)
works anywhere `wg` does. Gate accordingly and say so in the
docs — do not silently no-op.
## 8. Risks
| risk | mitigation |
| --- | --- |
| `to_thread` worker runs in the wrong netns | §5.3; pass `netns=` to pyroute2 or pin a worker; test-first |
| py-multiaddr#108 merged but unreleased | `[tool.uv.sources]` `rev` pin + `_have_wg_maddr_proto()` gate; layer A's inner-addr path works regardless |
| `TunnelledAddress` leaks into `Endpoint` and breaks `inspect.getmodule()` | unwrap at parse/bindspace boundary; assert `not isinstance(ep.addr, TunnelledAddress)` in `Endpoint.__post_init__` |
| privileged ops in a library | never `sudo`; explicit cap probe + actionable error; pre-provisioned is the default |
| pyroute2 0.9 asyncio core drags a loop into the actor | option (1) is a *thread*, not a loop; forbid `trio-asyncio` here (§4.1) |
| netns teardown strands actor teardown | idempotent/tolerant teardown mirroring `_uds.close_listener()` |
## 9. Follow-up issue seeds
- `veth`-in-netns bindspace (unblocks capless-ish integration
testing, and is a great local multi-"host" test rig)
- composed/tunnelled maddr grammar shared with plan 02's
`/…/quic-v1/…` stacks (gh #443)
- `wg` proto into the multiaddr **spec** (gh #483), then flip
`MsgTransport.maddr` to always return `Multiaddr` (the third
#443 bullet)
- runtime-managed wg key rotation / peer add-remove as a
`tractor` service actor — the natural "actor that owns the
network" demo

View File

@ -0,0 +1,191 @@
# `/tipc` multiaddr protocol: upstream issue draft
Candidate issue for `multiformats/multiaddr`, to be submitted after
the encoding questions below have been reviewed locally.
## Context
Linux TIPC (Transparent Inter-Process Communication) addresses a
service by a location-independent `(service type, instance)` name.
A server publishes that name into the kernel-maintained cluster name
table and a client connects by the same name; no host or transport
port forms part of the service identity.
TIPC is also known as **Cluster Domain Sockets**, a useful description
of its relationship to Unix-domain sockets. The registered protocol
name should nevertheless remain `tipc`: it matches Linux's
`AF_TIPC`, socket constants, kernel module and iproute2 frontend.
Registering `cds` would create an ecosystem-specific alias that is
harder to map back to the normative kernel API.
We would like to register a `tipc` multiaddr component so these
service endpoints can be represented directly and composed with a
bearer or tunnel description:
```text
/tipc/1953628160:1616:2
/ip4/192.168.1.50/udp/51820/wg/u<key>/tipc/1953628160:1616:2
\____________ WireGuard bearer ____________/\____ TIPC service ____/
```
In the composed form, the components through `/wg/<key>` identify
the routed bearer and tunnel peer. The `/tipc/...` component is pure
service identity, resolved to a current publisher by the TIPC kernel
name table. Unlike a TCP endpoint, it deliberately has no inner IP
address or port.
This proposal does not imply that parsing the multiaddr configures a
TIPC bearer. In particular, TIPC over a WireGuard interface requires
a separately configured TIPC UDP bearer; WireGuard interfaces are L3
devices and cannot carry TIPC Ethernet media directly.
Today that bearer is configured through iproute2's `tipc` frontend,
which speaks the kernel's `TIPCv2` generic-netlink family. `pyroute2`
already provides WireGuard support and generic-netlink primitives but
has no TIPC codec/module; adding one is a complementary deployment
automation track, not part of this address-format proposal.
## Proposed protocol
- Name: `tipc`
- Code: TBD, allocated in `multiformats/multicodec` under the
`multiaddr` tag before implementations stabilize one
- Size: 72 bits
- Value: service type, service instance and publication scope
### Binary form
Exactly nine bytes with no value-length prefix:
| Offset | Size | Field | Encoding |
| ---: | ---: | --- | --- |
| 0 | 4 bytes | service type | unsigned 32-bit big-endian |
| 4 | 4 bytes | service instance | unsigned 32-bit big-endian |
| 8 | 1 byte | publication scope | unsigned enum byte |
```text
tipc-value = uint32be(type) || uint32be(instance) || uint8(scope)
```
For type `1953628160` (`0x74720000`), instance `1616` and
cluster scope `2`, the payload is:
```text
74 72 00 00 00 00 06 50 02
```
### String form
Use one multiaddr value segment containing three canonical decimal
integers:
```text
/tipc/<type>:<instance>:<scope>
```
Canonical values have no sign, whitespace, alternate radix or
leading zeroes, except that zero itself is `0`. `type` and `instance`
must fit unsigned 32-bit fields. Scope is one of:
- `2`: `TIPC_CLUSTER_SCOPE`
- `3`: `TIPC_NODE_SCOPE`
The existing experimental spelling
`/tipc/<type>/<instance>/<scope>` cannot be registered as one normal
multiaddr protocol: generic parsing treats each slash-delimited name
as another protocol component. A single structured value preserves
TIPC's atomic service-address semantics without registering three
artificial protocols.
## Why scope is included
TIPC scope controls where a bound service publication is visible.
The same address representation is used for listener configuration
and dialing, so retaining scope lets a multiaddr round-trip the full
socket address rather than silently turning a node-local bind into a
cluster publication.
Modern Linux UAPI defines cluster and node scopes. The deprecated
zone spelling should not receive a new wire value; implementations
may normalize legacy input to cluster scope before encoding.
## Composition
Standalone service:
```text
/tipc/1953628160:1616:2
```
TIPC service reached through a WireGuard bearer:
```text
/ip4/192.168.1.50/udp/51820/wg/u<key>/tipc/1953628160:1616:2
```
This differs intentionally from TCP over WireGuard:
```text
/ip4/192.168.1.50/udp/51820/wg/u<key>/ip4/10.0.11.1/tcp/1616
```
TCP repeats an inner locative address. TIPC does not: its service
name is resolved and load-balanced in-kernel across current
publishers.
## Semantics and security
- A TIPC service name identifies a service, not a unique process.
Multiple publishers may bind the same name and connections can be
distributed among them.
- Publication scope is reachability metadata, not authentication.
- A composed `/wg` key authenticates the tunnel peer, not the TIPC
service publisher.
- TIPC's optional native AES-GCM link encryption is independent of
this address codec and of WireGuard.
- Codec implementations should validate field widths and canonical
text only; cluster membership and publisher authorization remain
deployment concerns.
## Implementation plan
1. Reserve a `multiaddr`-tagged code in
`multiformats/multicodec`.
2. Add the fixed-size protocol row and normative encoding text to
`multiformats/multiaddr`.
3. Add codecs and cross-language test vectors, beginning with
`multiformats/py-multiaddr`.
4. Verify standalone and composed `wg` + `tipc` string/binary
round-trips.
## Open questions
1. Is a fixed 72-bit value preferred over a self-describing or
variable-width tuple for this kernel-defined address?
2. Should node scope be representable in a generally shareable
multiaddr, or should the registered form be cluster-only?
3. Does multiaddr have an existing convention for structured numeric
values that should replace the colon-separated text form?
4. Should the specification describe TIPC service *ranges*, or keep
this protocol limited to singleton service names used for
connection endpoints?
## References
- Linux TIPC documentation:
https://docs.kernel.org/networking/tipc.html
- Cluster Domain Sockets terminology:
https://en.wikipedia.org/wiki/Transparent_Inter-process_Communication
- Linux socket UAPI:
https://github.com/torvalds/linux/blob/master/include/uapi/linux/tipc.h
- Linux TIPC generic-netlink UAPI:
https://github.com/torvalds/linux/blob/master/include/uapi/linux/tipc_netlink.h
- pyroute2 WireGuard and generic-netlink APIs:
https://docs.pyroute2.org/wireguard.html
- WireGuard multiaddr implementation discussion:
https://github.com/multiformats/py-multiaddr/issues/107
- WireGuard codec implementation:
https://github.com/multiformats/py-multiaddr/pull/108
- Downstream tracking and prototype:
https://github.com/goodboy/tractor/issues/498

View File

@ -0,0 +1,58 @@
# next-gen `tractor.ipc` transport backend plans
Implementation specs for three prospective `.ipc` transport
backends, written so each can be worked independently (by a
different model/provider) without design or lib-selection drift.
**Read [`00_shared_backend_contract.md`](./00_shared_backend_contract.md)
first** — it is the normative description of what a `tractor`
transport backend *is* as of `main@83b34884` (the backend
duck-type, the 10-item registration checklist, the test-harness
plumbing, the code-style rules). The three plans assume it and
document only their own deltas.
| plan | issue | dep | size | lands |
| --- | --- | --- | --- | --- |
| [01 — TIPC](./01_tipc_backend.md) | [#378] | **none** (stdlib) | small | **landed**, PR [#493] — see the [handoff](./01_tipc_HANDOFF.md) |
| [02 — QUIC/`iroh`](./02_quic_iroh_backend.md) | [#353] | `iroh` (uniffi FFI) | large | needs a prep PR |
| [03 — `wg` bindspace](./03_wg_tunnel_bindspace.md) | [#482], [#443] | `pyroute2` | medium, 3 layers | layer A now |
| [04 — `/tipc` multiaddr](./04_tipc_multiaddr_upstream.md) | [#498] | upstream `multiformats` | issue draft | local review |
Headline conclusions:
- **TIPC is the cheap win.** Verified: `trio.SocketStream` and
`trio.SocketListener` are address-family agnostic (only
`SOCK_STREAM` + a trio socket), and CPython ships `AF_TIPC` +
23 `TIPC_*` constants. So the backend is ~one module of
contract boilerplate, zero new deps, and it buys
*kernel-native* service discovery: `bind()` publishes,
`connect()`-by-name resolves — no registrar in the loop.
(`modprobe tipc` is required; hard-gate everything.)
- **QUIC's cost is entirely in two adapters**, not in QUIC. The
`iroh` python bindings are `uniffi`-generated asyncio, but the
asyncio dependency is confined to *one* future-poll callback —
a ~40-line `trio` bridge (`TrioToken.run_sync_soon`) replaces
it. The second cost is that an iroh listener isn't a socket,
which needs a small, independently-reviewable prep PR to
`_server.py`/`_types.py`.
- **WireGuard is not a transport.** It's an iface-layer tunnel,
so it belongs as a *nested bindspace* (`TunnelledAddress` +
`open_bindspace()` `@acm`s) wrapping whatever L4 tpt is in
use — which is also what finally implements the long-spec'd
`Address.namespace`, and what generalizes to
`veth`/`vxlan`/`gre`.
Ordering rationale: plan 01 first as the cheap proof the
table-registration story generalizes to a genuinely new proto;
plan 03 layer A is already deployable-today doc/example work;
plan 02 last (and gated on its prep PR). Plans 01 and 02 both
want the same `Address.rebind_from_sockname` gate — whichever
lands first ships it.
[#378]: https://github.com/goodboy/tractor/issues/378
[#353]: https://github.com/goodboy/tractor/issues/353
[#482]: https://github.com/goodboy/tractor/issues/482
[#443]: https://github.com/goodboy/tractor/issues/443
[#498]: https://github.com/goodboy/tractor/issues/498
[#493]: https://github.com/goodboy/tractor/pull/493

View File

@ -4,10 +4,12 @@ IPC and logging
Under every portal, context and stream sits a per-peer Under every portal, context and stream sits a per-peer
:class:`~tractor.Channel`: a msgpack-typed messaging link wrapping :class:`~tractor.Channel`: a msgpack-typed messaging link wrapping
one OS transport connection. Transports are pluggable per actor one OS transport connection. Transports are pluggable per actor
via ``enable_transports=['tcp' | 'uds']`` — TCP is the default, via ``enable_transports=['tcp' | 'uds' | 'tipc']`` — TCP is the
UDS (unix domain sockets) gives you port-less, same-host IPC with default, UDS (unix domain sockets) gives you port-less, same-host
kernel-provided peer credentials for free — and exactly **one** IPC with kernel-provided peer credentials for free, and TIPC is an
transport may currently be enabled per actor. opt-in linux cluster protocol where the address *is* a
kernel-published service name (see :doc:`/guide/tipc`) — and
exactly **one** transport may currently be enabled per actor.
.. d2:: diagrams/runtime_stack.d2 .. d2:: diagrams/runtime_stack.d2
:caption: Where ``Channel`` sits in the runtime stack. :caption: Where ``Channel`` sits in the runtime stack.
@ -15,7 +17,8 @@ transport may currently be enabled per actor.
:alt: layered runtime stack from app code down to transports :alt: layered runtime stack from app code down to transports
Addresses are "unwrapped" tuples at the API edges: Addresses are "unwrapped" tuples at the API edges:
``('host', port)`` for TCP, filesystem-path pairs for UDS. For ``('host', port)`` for TCP, filesystem-path pairs for UDS and the
proto-keyed ``('tipc', stype, instance, scope)`` for TIPC. For
the full layering story — transport protocols, the IPC server, the full layering story — transport protocols, the IPC server,
address types and the msg loop — see address types and the msg loop — see
:doc:`/explain/architecture`. :doc:`/explain/architecture`.

View File

@ -31,6 +31,8 @@ order,
SC-supervise ``asyncio`` tasks from ``trio``. SC-supervise ``asyncio`` tasks from ``trio``.
- :doc:`msging` — typed IPC payloads, the wire - :doc:`msging` — typed IPC payloads, the wire
msg-spec and custom codecs. msg-spec and custom codecs.
- :doc:`tipc` — the ``AF_TIPC`` cluster backend,
where the kernel does discovery for you.
- :doc:`testing` — running + monitoring the - :doc:`testing` — running + monitoring the
test suite (and testing your own actor apps). test suite (and testing your own actor apps).
@ -49,4 +51,5 @@ order,
parallelism parallelism
asyncio asyncio
msging msging
tipc
testing testing

276
docs/guide/tipc.rst 100644
View File

@ -0,0 +1,276 @@
TIPC: when the kernel does discovery
====================================
Every other ``tractor`` transport gives you a *pipe* and leaves
discovery to us: the registrar actor, the ``find_actor()``
round-trip, the whole :doc:`discovery` story. TIPC_
(Transparent Inter-Process Communication) is different — it's a
linux-kernel cluster protocol whose **service names live in a
cluster-wide name table the kernel itself maintains**.
Which flips the model:
- an actor's IPC address *is* a service name ``(stype,
instance)`` — no host, no port,
- ``.bind()``-ing that name **is** service registration,
- a peer's ``.connect()``-by-name **is** the lookup, resolved
and load-balanced in-kernel.
So for TIPC-capable deployments the registrar round-trip stops
being the only way peers find each other. Enable it per actor
like any other backend,
.. code:: python
async with tractor.open_nursery(
enable_transports=['tipc'],
) as an:
...
.. warning::
TIPC is **opt-in and linux-only**. The ``tipc`` kernel module
is not loaded on most boxes (``sudo modprobe tipc``), and the
address family doesn't exist off-linux at all. Check
:func:`tractor.ipc._tipc.is_tipc_available` before assuming;
``tractor`` never selects this backend for you.
.. _TIPC: https://en.wikipedia.org/wiki/Transparent_Inter-process_Communication
Your actor tree, in the kernel's name table
-------------------------------------------
The single best demo this backend has needs no ``tractor`` API
at all — boot a tree and ask ``tipc(8)`` what it sees:
.. code:: bash
sudo modprobe tipc
python examples/multihost/tipc_cluster/single_host.py
.. code:: text
--- `tipc nametable show` :: root + 3 subactors ---
Type Lower Upper Scope Port
1953628160 1616 1616 cluster 3161982128
1953628160 1219427151 1219427151 cluster 1587358717
1953628160 2641339936 2641339936 cluster 1864021571
1953628160 3344505866 3344505866 cluster 3816483388
--- `tipc nametable show` :: after teardown (all withdrawn) ---
Type Lower Upper Scope Port
Reading the rows,
- ``1953628160`` is ``0x74720000``, ``tractor``'s reserved
service *type* — ascii ``tr`` in the high half, with the low
16 bits free so an app can partition its own service classes
via ``TIPCAddress._stype``,
- ``1616`` is the host-singleton registrar instance, the same
"1616 is tractor's registrar" idiom as the TCP port and the
``registry@1616.sock`` UDS filename,
- the other three are per-actor instances derived from a
``blake2b`` digest of the actor's identity (see
`Silent crosstalk`_),
- ``Scope`` is the address' :attr:`bindspace` — see `Scope is
the bindspace`_.
Push-based discovery
--------------------
TIPC also exposes a *topology service*: subscribe and the kernel
pushes you name-table transitions as they happen.
:func:`tractor.ipc._tipc.open_topology_events` wraps it as an
``@acm`` yielding a ``trio`` receive-channel,
.. code:: python
from tractor.ipc._tipc import open_topology_events
async with open_topology_events() as events:
async for ev in events:
print(f'{ev.kind}: {ev.addr}')
.. code:: text
watching the TIPC name table..
[+] published instance=1616 port=0x00000000:2375440573
spawning subactors..
[+] published instance=186947472 port=0x00000000:3960753074
[+] published instance=2191362136 port=0x00000000:2263898853
tearing down..
[-] withdrawn instance=186947472 port=0x00000000:3960753074
No polling, no registrar round-trip — this is the groundwork for
a registrar that keeps a live view of the actor set without ever
calling ``find_actor()``.
``filt`` picks the granularity: ``TIPC_SUB_SERVICE`` gives one
event per *name* becoming (un)available, ``TIPC_SUB_PORTS`` one
per *publisher* — which is what makes the duplicate-name case
below externally observable.
Scope is the bindspace
----------------------
Every ``tractor`` address type has a ``.bindspace`` — "the set
of hosts this bind is reachable from". For TCP that's the IP,
for UDS the socket-file directory. For TIPC it's the *scope*,
which is about as literal a reading of that docstring as exists:
.. list-table::
:header-rows: 1
:widths: 30 70
* - scope
- meaning
* - ``TIPC_NODE_SCOPE``
- same host only — the UDS analogue
* - ``TIPC_CLUSTER_SCOPE``
- cluster-visible (the default)
``TIPC_ZONE_SCOPE`` is deprecated and aliased to cluster-scope
by modern kernels; ``tractor`` accepts it on input, folds it to
cluster and logs at ``transport`` level.
Spanning hosts
--------------
Single-host TIPC needs only ``modprobe``. Crossing hosts needs a
**bearer** enabled on both — an ethernet (L2) or UDP underlay
the kernel routes service names over:
.. code:: bash
# on BOTH hosts
sudo tipc bearer enable media eth device eth0
# ..or, when L2 isn't available:
sudo tipc bearer enable media udp name uc localip 10.0.11.1
tipc link list # must list the peer before you proceed
The two-host example pair then talks with **no IP, hostname or
port anywhere in either script** — both sides name the same
service and the kernel routes it. Move the server to a third
node and the client's dial keeps working, unchanged. See
``examples/multihost/tipc_cluster/`` for the full walkthrough
(that directory is excluded from CI precisely because it needs
real hardware).
Over a WireGuard mesh
~~~~~~~~~~~~~~~~~~~~~
TIPC over a `wg` mesh is the intended reference deployment for
multihost ``tractor`` (see gh #502), composing with the tunnel
examples in ``examples/multihost/wg_lan/``.
.. warning::
A wg interface is L3/``tun````POINTOPOINT,NOARP`` with
``link/none`` and no L2 address — so TIPC's ``eth`` media
**cannot** bind it. Over wg the udp bearer is *mandatory*,
not merely an alternative:
.. code:: bash
# NOT possible over wg
sudo tipc bearer enable media eth device wg0
# required instead, bound to the wg overlay IP
sudo tipc bearer enable media udp name wgmesh \
localip 10.0.11.1
Mind the MTU too: wg links typically sit at 1420, under
ethernet's 1500.
.. note::
**wg is not about confidentiality here.** TIPC ships AES-GCM
crypto of its own (``tipc node set key``, linux 5.9+) with
cluster, master and per-node keys plus rekeying intervals, so
"wg adds the encryption TIPC lacks" is simply wrong.
The motivation is different but real: TIPC's keys are
*symmetric and pre-shared*, leaving distribution, rotation and
revocation to the operator, whereas wg brings public-key
identity and a handshake — plus NAT traversal, and one overlay
that *every* transport can share rather than a TIPC-only
mechanism. Which to prefer is worth benchmarking; native
crypto avoids a tunnel hop entirely.
Gotchas
-------
.. _Silent crosstalk:
**Silent crosstalk.** Unlike every other backend, a duplicate
bind does *not* raise ``EADDRINUSE``. TIPC accepts multiple
publishers of one name and **round-robins** connects between
them — verified: six dials alternated strictly between two
listeners. So an instance collision splits traffic silently
instead of erroring. That's why ``TIPCAddress.get_random()``
derives its instance from a ``blake2b`` digest of the actor
identity rather than a counter, and why two ``tractor`` trees
sharing both a cluster **and** an ``_stype`` share a namespace —
partition them with a distinct ``_stype``.
**Graceful close looks like a reset.** A peer closing cleanly
surfaces as ``BrokenResourceError``/``ECONNRESET`` rather than
the clean 0-byte EOF TCP and UDS give you. Benign — the
transport layer already classifies it as a normal disconnect —
but it does look alarming in ``transport``-level logs.
**Dialing an unpublished name** answers ``EHOSTUNREACH``
*instantly*, with no SYN-timeout wait. That's markedly better
discovery-ping behaviour than TCP; ``tractor`` normalizes it to
``ConnectionError`` so the usual lookup paths work unchanged.
**Multiaddrs are interim.** There's no registered ``/tipc``
protocol in the multiaddr table yet, so the grammar is
``str``-only:
.. code:: text
/tipc/<stype>/<instance>/<scope>
Composed with a wg bearer — the form that actually matters for
multihost — that becomes:
.. code:: text
/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>
Note the tipc segment carries **no** locative component, unlike
the ``/ip4/../tcp/..`` inner segment of the equivalent tcp maddr
— a TIPC service name is location-independent by design, so the
wg segments carry all the routing and the tipc segment is pure
identity.
Running the suite over TIPC
---------------------------
The backend is a first-class suite mode — the *entire* existing
test suite runs over it unmodified, which is the acceptance bar
for any ``tractor`` transport:
.. code:: bash
sudo modprobe tipc
pytest --tpt-proto tipc
Without the module that fails loudly and immediately with an
actionable message rather than a few hundred confusing connect
timeouts. Backend-specific unit tests live in
``tests/ipc/test_tipc.py`` and self-skip when the module is
absent.
Normative references
--------------------
The tipc.io documentation is stale in places. Treat the kernel
sources as the only authority:
- ``include/uapi/linux/tipc.h`` — address flavours, sockopts,
the topology ``struct``\s
- ``net/tipc/socket.c``, ``net/tipc/topsrv.c``
- ``man 8 tipc``

View File

@ -0,0 +1,321 @@
# `tractor` over `AF_TIPC`, where the address *is* the service name
TIPC is a linux-kernel cluster IPC protocol whose service names
live in a **cluster-wide name table maintained by the kernel**.
It is also described as **Cluster Domain Sockets**: the Unix-domain
socket model extended from one kernel to a cluster. That name is a
useful explanation for new users, while the code keeps `tipc` as its
protocol key to match Linux's `AF_TIPC`, kernel module and tooling.
For `tractor` that means:
- an actor's IPC address is a service name `(stype, instance)`,
not a host/port,
- `.bind()`ing it **is** service registration,
- a peer's `.connect()`-by-name **is** the lookup.
So the discovery machinery `tractor.discovery` normally
implements with a registrar actor comes for free, in-kernel —
which is the ask in gh
[#378](https://github.com/goodboy/tractor/issues/378).
> **Why `examples/multihost/`?** `tests/test_docs_examples.py`
> walks `examples/` recursively and runs everything it collects
> as a subproc, asserting `rc == 0`. These need the `tipc`
> kernel module (and, for the two-host pair, a live bearer), so
> they can't satisfy that; `'multihost' not in p[0]` is already
> in the test's exclusion list, which is what keeps them out of
> CI. See "CI" below for the separate matrix-entry plan.
## the single best demo
```bash
sudo modprobe tipc
python single_host.py
```
Four actors boot, four service names appear in the kernel's
table, and all four are withdrawn on teardown — observed with
`tipc(8)`, entirely outside `tractor`:
```
--- `tipc nametable show` :: root + 3 subactors ---
Type Lower Upper Scope Port
1953628160 1616 1616 cluster 3161982128
1953628160 1219427151 1219427151 cluster 1587358717
1953628160 2641339936 2641339936 cluster 1864021571
1953628160 3344505866 3344505866 cluster 3816483388
--- `tipc nametable show` :: after teardown (all withdrawn) ---
Type Lower Upper Scope Port
```
`1953628160` is `0x74720000``tractor`'s reserved service
type, ascii `tr` in the high half. `1616` is the host-singleton
registrar, the same idiom as the TCP port and the
`registry@1616.sock` UDS filename. The other three instances are
per-actor digests (see "silent crosstalk" below).
## push-based discovery
```bash
python watch_nametable.py
```
Subscribes to the kernel's *topology service* and prints name
table transitions as they happen — no polling, no registrar
round-trip:
```
watching the TIPC name table..
[+] published instance=1616 port=0x00000000:2375440573
spawning subactors..
[+] published instance=186947472 port=0x00000000:3960753074
[+] published instance=2191362136 port=0x00000000:2263898853
[+] published instance=3484369663 port=0x00000000:2126817956
tearing down..
[-] withdrawn instance=186947472 port=0x00000000:3960753074
...
```
This is the groundwork for a push registry in
`tractor.discovery._registry` (gh
[#184](https://github.com/goodboy/tractor/issues/184),
[#216](https://github.com/goodboy/tractor/issues/216)) — a
registrar that *never polls* `find_actor()`.
## two hosts
Everything above is single-node (`modprobe` is enough). To span
hosts you need a **bearer** on both, which is the one thing that
can't be CI'd.
For the first physical test, use two wired Linux hosts on the same
L2 segment. Prefer a direct cable or uncomplicated switch; avoid
Wi-Fi, guest VLANs and port isolation until the basic link works.
Use the same checkout and Python environment on both hosts:
```bash
# on BOTH hosts
git rev-parse HEAD # must match on A and B
uv sync --all-extras --dev
sudo modprobe tipc
# choose the real wired iface; do not assume `eth0`
ip -br link
IFACE=enp3s0
# inspect existing cluster identity before changing anything
tipc node get address # must differ between hosts
tipc node get netid # must match between hosts
# use one private test netid on BOTH hosts, before enabling bearers
sudo tipc node set netid 37801
# ethernet is simplest when the hosts share an L2 segment
sudo tipc bearer enable media eth device "$IFACE"
# ..or over UDP when L2 isn't available — and MANDATORY over a
# `wg` mesh, see below
sudo tipc bearer enable media udp name uc localip 10.0.11.1
# verify BEFORE running anything: this must list the peer
tipc bearer list
tipc link list
tipc node list
```
If the link does not appear, first verify carrier, a common TIPC
network ID, distinct node addresses, a common VLAN and compatible
MTUs. Ethernet TIPC uses EtherType traffic rather than IP routing,
so a successful `ping` alone does not prove the bearer can work.
Run each command from `examples/multihost/tipc_cluster/`:
```bash
# host A
uv run python host_a_srv.py
# host B
watch -n 0.5 tipc nametable show # optional second terminal
uv run python host_b_client.py
```
Note what's absent from both scripts: any IP, hostname or port.
Both sides name the *same service*, and the kernel routes it.
Move `host_a_srv.py` to a third node and host B's dial keeps
working, unchanged.
For a first resilience pass, use a local console or separate
management link so the test does not cut off your own SSH session:
```bash
# host B: record the healthy baseline
tipc link list
tipc link statistics show
# either host: withdraw and recreate the bearer
sudo tipc bearer disable media eth device "$IFACE"
tipc link list
sudo tipc bearer enable media eth device "$IFACE"
tipc link list
# prove name withdrawal/republication and RPC recovery
tipc nametable show
uv run python host_b_client.py
```
Capture `uname -a`, both node addresses, `tipc bearer list`,
`tipc link list`, `tipc link statistics show`, the name table and
both Python transcripts. Those artifacts distinguish an actor bug
from bearer discovery, cluster identity or switch configuration.
Clean up a disposable Ethernet test on both hosts with:
```bash
sudo tipc bearer disable media eth device "$IFACE"
tipc link list
```
Restore any pre-existing network ID only after all bearers are
disabled. The first useful automation target is a two-node network
namespace fixture that asserts link-up, remote publication, RPC,
withdrawal and republication in that order; physical hardware then
remains the validation layer for real NIC and switch behaviour.
The commands above use iproute2's `tipc` frontend, which speaks the
kernel's `TIPCv2` generic-netlink family. The planned `pyroute2`
dependency already manages WireGuard, interfaces and namespaces and
provides generic-netlink primitives, but it does not currently ship a
TIPC message codec. Adding one upstream would let `tractor` replace
these manual commands with one Python netlink stack instead of
shelling out; until then, `tipc(8)` remains the canonical frontend.
### over a `wg` mesh
TIPC over WireGuard is the intended reference multihost
deployment (gh #502). One hard constraint: a wg interface is
L3/`tun` — `POINTOPOINT,NOARP`, `link/none`, no L2 address — so
TIPC's `eth` media **cannot** bind it. The udp bearer is
mandatory there, bound to the wg overlay IP, and wg's typical
1420 MTU sits under ethernet's 1500 so link MTU wants checking.
Composed, the deployment address is:
```
/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>
\____ wg bearer ________/\_key_/\______ tractor ep ________/
```
Note the tipc segment has no locative part, unlike tcp's inner
`/ip4/../tcp/..` — a service name is location-independent, so wg
carries the routing and tipc carries identity.
**wg here is not about confidentiality.** TIPC ships AES-GCM
crypto of its own (`tipc node set key`, linux 5.9+) with
cluster/master/per-node keys and rekeying, so "wg adds the
encryption TIPC lacks" is wrong.
The motivation is different but real: TIPC's keys are *symmetric
and pre-shared* — distribution, rotation and revocation are all
on you — whereas wg gives public-key identity and a handshake,
NAT traversal, and one overlay shared by every transport instead
of a TIPC-only mechanism. Worth benchmarking either way; native
crypto skips the tunnel hop.
Note too that the ethernet-bearer pairing #378 imagined does
**not** apply over wg: on a given link the L2 path and the wg
path are mutually exclusive.
### scope
`TIPCAddress._scope` is the backend's `.bindspace` — literally
"the set of hosts this published name is reachable from":
| scope | meaning |
| --- | --- |
| `TIPC_NODE_SCOPE` | same host only — the UDS analogue |
| `TIPC_CLUSTER_SCOPE` | cluster-visible (the default) |
`TIPC_ZONE_SCOPE` is deprecated and aliased to cluster by modern
kernels; `tractor` accepts it on input and folds it, logging at
`transport` level.
## gotchas worth knowing before you deploy
**Silent crosstalk.** Unlike every other backend, a duplicate
bind does **not** raise `EADDRINUSE` — TIPC happily accepts
multiple publishers of one name and *round-robins* connects
between them (verified: 6 dials alternated `b,a,b,a,b,a`). So an
instance collision is silent traffic-splitting, not an error.
That's why `TIPCAddress.get_random()` derives the instance from
a `blake2b` digest of the actor identity rather than a counter.
Two `tractor` trees sharing both a cluster **and** an `_stype`
share a name space; partition them by passing a distinct
`_stype`.
**Graceful close looks like a reset.** A peer closing cleanly
surfaces as `BrokenResourceError`/`ECONNRESET` rather than the
clean 0-byte EOF you get from TCP/UDS. It's benign — the
transport layer already classifies it as a normal disconnect —
but it does look alarming in `transport`-level logs.
**Dialing an unpublished name** answers `EHOSTUNREACH`
*instantly* (no SYN-timeout wait), which is much better
discovery-ping behaviour than TCP. `tractor` normalizes it to
`ConnectionError`.
**It's opt-in, never a default.** The module isn't loaded on
most boxes and doesn't exist off-linux, so
`enable_transports=['tipc']` is always explicit. Check
`tractor.ipc._tipc.is_tipc_available()` before assuming.
## maddr form
There is no registered `/tipc` protocol in the multiaddr table
yet (upstream track: gh
[#483](https://github.com/goodboy/tractor/issues/483) +
multiformats/py-multiaddr#107), so the grammar is interim and
`str`-only:
```
/tipc/<stype>/<instance>/<scope>
```
`parse_maddr()` special-cases this prefix *before* handing
anything to `Multiaddr()`, which would otherwise reject the
unregistered name outright. Registering it upstream is what
would unblock gh
[#443](https://github.com/goodboy/tractor/issues/443)'s
"return `Multiaddr` everywhere" item.
## running the suite over TIPC
The whole test suite runs under the backend:
```bash
sudo modprobe tipc
pytest --tpt-proto tipc
```
Without the module that fails loudly and immediately with an
actionable message rather than a few hundred connect timeouts.
Backend-specific unit tests live in `tests/ipc/test_tipc.py` and
self-skip when the module is absent.
## CI
Single-host TIPC *is* CI-able — the module ships with the
standard Ubuntu kernel package. CI loads it with `sudo modprobe
tipc` and runs the suite with `--tpt-proto tipc` as a blocking
matrix leg. Cross-node bearer testing stays manual — this README
is that smoke test.
## normative refs
The tipc.io docs are stale in places (gh #378 says as much).
Treat the kernel sources as the only normative reference:
- `include/uapi/linux/tipc.h` — address flavours, sockopts, the
topology `struct`s
- `net/tipc/socket.c`, `net/tipc/topsrv.c`
- `man 8 tipc`

View File

@ -0,0 +1,78 @@
'''
HOST A publish a `tractor` service on a cluster-scoped TIPC
service name.
Note what is NOT in this file: any IP address, hostname or port.
The actor's address IS the service name `(stype, instance)`, and
the kernel routes it over whatever bearer you enabled. Move this
process to another node and host B's dial keeps working,
unchanged.
Prereqs on BOTH hosts (see README.md),
sudo modprobe tipc
sudo tipc bearer enable media eth device <iface>
tipc link list # must show a link to the peer
Then here,
python host_a_srv.py
'''
from __future__ import annotations
import trio
import tractor
from tractor.ipc._tipc import (
TIPCAddress,
is_tipc_available,
)
@tractor.context
async def echo(
ctx: tractor.Context,
) -> None:
await ctx.started()
async with ctx.open_stream() as stream:
async for msg in stream:
print(f'host-a <- {msg!r}')
await stream.send(f'{msg} (from host A)')
async def main() -> None:
# the host-singleton registrar name, `instance=1616` —
# the same "1616 is tractor's registrar" idiom as the tcp
# port and the `registry@1616.sock` UDS filename.
reg: TIPCAddress = TIPCAddress.get_root()
print(f'host A publishing {reg}')
# A registrar root is not an entry in its own
# `Registrar._registry`, so host B cannot discover that root by
# its actor name. `open_nursery()` keeps the root as registrar
# while the named child below registers as the dialable service.
async with tractor.open_nursery(
enable_transports=['tipc'],
registry_addrs=[reg.unwrap()],
) as an:
await an.start_actor(
'host_a',
# Host B imports this same module name to construct the
# RPC `NamespacePath`; direct script execution would
# otherwise expose it as `__main__` on host A.
enable_modules=['host_a_srv'],
)
print(
'host_a up — `tipc nametable show` on EITHER host\n'
'should now list this service. ctrl-c to stop.'
)
await trio.sleep_forever()
if __name__ == '__main__':
if not is_tipc_available():
raise RuntimeError(
'The `tipc` kernel module is not loaded!\n'
' |_try: `sudo modprobe tipc`\n'
)
trio.run(main)

View File

@ -0,0 +1,64 @@
'''
HOST B dial host A's service *by name*, across the cluster.
The `.connect()` on a TIPC service name IS the discovery lookup:
the kernel resolves the published name to whichever node serves
it. So this client needs no IP, no port and no idea where host A
actually is.
Prereqs: same bearer setup as `host_a_srv.py`, and that script
already running on the other node.
python host_b_client.py
'''
from __future__ import annotations
import trio
import tractor
from host_a_srv import echo
from tractor.ipc._tipc import (
TIPCAddress,
is_tipc_available,
)
async def main() -> None:
reg: TIPCAddress = TIPCAddress.get_root()
print(f'host B dialling {reg} (by NAME, not address)')
async with tractor.open_root_actor(
name='host_b',
enable_transports=['tipc'],
registry_addrs=[reg.unwrap()],
):
async with tractor.find_actor('host_a') as ptl:
if ptl is None:
raise RuntimeError(
'No `host_a` in the cluster name table!\n'
' |_is `host_a_srv.py` running?\n'
' |_does `tipc link list` show the peer?\n'
)
# `.open_context()` derives a `NamespacePath` from a
# callable. Importing `echo` also loads its module path
# locally, while host A's `enable_modules` authorizes the
# corresponding remote callable.
async with (
ptl.open_context(
echo,
) as (ctx, _),
ctx.open_stream() as stream,
):
for msg in ('hello', 'from', 'the other node'):
await stream.send(msg)
print(f'host-b <- {await stream.receive()!r}')
if __name__ == '__main__':
if not is_tipc_available():
raise RuntimeError(
'The `tipc` kernel module is not loaded!\n'
' |_try: `sudo modprobe tipc`\n'
)
trio.run(main)

View File

@ -0,0 +1,104 @@
'''
`tractor` over `AF_TIPC` on a single host.
Every actor's IPC address is a TIPC *service name*, and binding
one publishes it into the kernel's cluster-wide name table. So
`tipc nametable show` lists your live actor tree no registrar
query, no `tractor` API, just the kernel telling you what's up.
Run,
sudo modprobe tipc
python single_host.py
'''
from __future__ import annotations
import subprocess
import trio
import tractor
from tractor.ipc._tipc import (
TRACTOR_STYPE,
is_tipc_available,
)
def show_nametable(tag: str) -> None:
'''
Dump the kernel name-table rows belonging to `tractor`.
'''
print(f'\n--- `tipc nametable show` :: {tag} ---')
out = subprocess.run(
['tipc', 'nametable', 'show'],
capture_output=True,
text=True,
)
for line in out.stdout.splitlines():
# header, or one of *our* service-type rows
if (
line.startswith('Type')
or
line.startswith(str(TRACTOR_STYPE))
):
print(f' {line}')
@tractor.context
async def wait_until_cancelled(
ctx: tractor.Context,
) -> None:
await ctx.started()
await trio.sleep_forever()
async def main() -> None:
async with tractor.open_nursery(
enable_transports=['tipc'],
) as an:
show_nametable('root only')
portals: list[tractor.Portal] = []
for name in ('donny', 'walter', 'dude'):
portals.append(
await an.start_actor(
name,
enable_modules=[__name__],
)
)
async with trio.open_nursery() as tn:
for ptl in portals:
tn.start_soon(
_hold_open,
ptl,
)
await trio.sleep(0.5)
# XXX the money shot: 4 actors, 4 published names
show_nametable('root + 3 subactors')
tn.cancel_scope.cancel()
await an.cancel()
show_nametable('after teardown (all withdrawn)')
async def _hold_open(
ptl: tractor.Portal,
) -> None:
async with ptl.open_context(
wait_until_cancelled,
) as (ctx, _):
await trio.sleep_forever()
if __name__ == '__main__':
if not is_tipc_available():
raise RuntimeError(
'The `tipc` kernel module is not loaded!\n'
' |_try: `sudo modprobe tipc`\n'
)
trio.run(main)

View File

@ -0,0 +1,93 @@
'''
Watch `tractor` actors (de)register themselves, live, via TIPC's
topology service.
`open_topology_events()` subscribes to the kernel's name table
and yields a `trio` receive-channel of `publish`/`withdraw`
events. That's **push-based** service discovery: no registrar
round-trip, no polling the kernel tells you the instant any
actor anywhere in the cluster comes or goes.
Run,
sudo modprobe tipc
python watch_nametable.py
'''
from __future__ import annotations
import trio
import tractor
from tractor.ipc._tipc import (
TIPCNameEvent,
is_tipc_available,
open_topology_events,
)
@tractor.context
async def wait_until_cancelled(
ctx: tractor.Context,
) -> None:
await ctx.started()
await trio.sleep_forever()
async def print_events(
events: trio.MemoryReceiveChannel[TIPCNameEvent],
) -> None:
glyphs: dict[str, str] = {
'published': '[+]',
'withdrawn': '[-]',
'timeout': '[!]',
}
async for ev in events:
print(
f' {glyphs.get(ev.kind, "[?]")} {ev.kind:<10} '
f'instance={ev.addr._instance:<12} '
f'port=0x{ev.node:08x}:{ev.ref}'
)
async def main() -> None:
# NOTE, subscribe BEFORE booting the runtime so we catch the
# root actor's own publication too.
async with open_topology_events() as events:
async with trio.open_nursery() as tn:
tn.start_soon(print_events, events)
print('watching the TIPC name table..\n')
async with tractor.open_nursery(
enable_transports=['tipc'],
) as an:
await trio.sleep(0.3)
print('\nspawning subactors..')
portals: list[tractor.Portal] = []
for name in ('donny', 'walter', 'dude'):
portals.append(
await an.start_actor(
name,
enable_modules=[__name__],
)
)
await trio.sleep(0.2)
print('\ntearing down..')
for ptl in portals:
await ptl.cancel_actor()
await trio.sleep(0.2)
await an.cancel()
await trio.sleep(0.5)
tn.cancel_scope.cancel()
if __name__ == '__main__':
if not is_tipc_available():
raise RuntimeError(
'The `tipc` kernel module is not loaded!\n'
' |_try: `sudo modprobe tipc`\n'
)
trio.run(main)

View File

@ -0,0 +1,171 @@
# `tractor` over a WireGuard tunnel, declared as one maddr
A two-host LAN setup: a `tractor` actor tree on host A, dialed
from host B, with the endpoint declared as a single `wg`
multiaddr.
Supersedes the example set in gh
[#482](https://github.com/goodboy/tractor/issues/482) — see
[what changed](#what-changed-vs-482).
> **Why `examples/multihost/`?** `tests/test_docs_examples.py`
> walks `examples/` recursively and runs everything it collects
> as a subproc, asserting `rc == 0`. These need a real second
> host and a live `wg` tunnel, so they can't satisfy that;
> `'multihost' not in p[0]` is already in the test's exclusion
> list, which is what keeps them out of CI.
## the maddr form
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\____ wg bearer ___________/\__ key __/\____ tractor ep _____/
underlay, wg `ListenPort` overlay, on the wg iface
(kernel/`wg(8)` owns it) (the ONLY part tractor binds)
```
Three parts, three different owners:
| part | who binds it | in the runtime? |
| --- | --- | --- |
| `/ip4/../udp/51820` bearer | kernel via `wg-quick`/`pyroute2` | no |
| `/wg/u<key>` | nothing — it's an identity | no, verified out-of-band |
| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | **yes**, as `.inner` |
Verified against py-multiaddr
[#108](https://github.com/multiformats/py-multiaddr/pull/108):
this composed form parses and round-trips
(`['ip4','udp','wg','ip4','tcp']`).
## requirements
py-multiaddr #108 is **merged** (2026-07-28) but ships in no
release yet — the latest `0.2.0` (2026-03-17) predates it and has
no `wg` codec. So `pyproject.toml` carries a temporary
`[tool.uv.sources]` `rev` pin at the merge commit, and a plain
```bash
uv sync
```
gets you a `wg`-aware `multiaddr`. That pin goes away once a
release carries the codec. You also need `multibase`:
```bash
uv pip install multibase
```
Without the codec `wg_maddr.py` degrades to a plain segment split
— the examples still run, but you lose per-segment validation
(incl. the 32-byte key-length check), so a malformed key reaches
the returned struct instead of raising. `_have_wg_maddr_proto()`
is the gate. It deliberately does **not** hand-roll a `wg` codec
(gh #429 was about *dropping* our NIH parser).
## 0. tunnel setup (out-of-band, both hosts)
Host A is the service host (underlay e.g. `192.168.1.50`), host B
your workstation. Overlay net `10.0.11.0/24`.
```bash
umask 077
wg genkey | tee wg_priv.key | wg pubkey > wg_pub.key
```
`/etc/wireguard/wg0.conf` on **host A**:
```ini
[Interface]
PrivateKey = <A_priv>
Address = 10.0.11.1/24
ListenPort = 51820
```
```ini
[Peer]
PublicKey = <B_pub>
AllowedIPs = 10.0.11.2/32
```
on **host B**:
```ini
[Interface]
PrivateKey = <B_priv>
Address = 10.0.11.2/24
```
```ini
[Peer]
PublicKey = <A_pub>
Endpoint = 192.168.1.50:51820
AllowedIPs = 10.0.11.1/32
PersistentKeepalive = 25
```
Note how `ListenPort` and `Endpoint` are exactly the maddr's
bearer segment, and `[Interface] Address` is its overlay host.
```bash
sudo wg-quick up wg0 # both hosts
ping -c1 10.0.11.1 # from B
```
## 1. get your pubkey into the maddr
```bash
python -c "
import base64, multibase
key = open('wg_pub.key').read().strip()
print(multibase.encode('base64url', base64.b64decode(key)).decode())
"
```
Paste the `u...` output into `WG_MADDR` in both scripts (they use
the same string — A's bearer, A's key, A's overlay ep).
## 2. run
```bash
# host A
python host_a_srv.py
# host B
python host_b_client.py
```
`host_a_srv.py` must be importable on host B too, since
`portal.run()` refs the fn by module path — standard `tractor`
RPC semantics.
## what changed vs #482
Four corrections, all from
`ai/tpt-backends/03_wg_tunnel_bindspace.md`:
1. **the maddr semantics were inverted.** #482 used
`/ip4/10.0.11.1/tcp/1616/wg/u<key>` — that parses, but it puts
the *overlay* addr where the bearer belongs and `tcp` where
wg's `udp` `ListenPort` goes, and it declares no overlay ep at
all. `parse_wg_maddr()` now rejects it with an actionable
error.
2. **parsing is pure.** #482's helper had the key-check adjacent
to the parse; `verify_wg_peer()` is now a separate, explicitly
composed step that the caller invokes. A parser that shells
out is a nasty surprise.
3. **no `sudo`.** #482 ran `sudo wg show`; a library/example must
never escalate. `wg show` works unprivileged for read on most
setups; if yours needs root, run the script as root rather
than embedding `sudo`.
4. **no new `Address` proto-type.** The tunnel rides *beside* the
inner addr in a frozen `WGTunnelledAddr`, and only `.inner`
crosses into `open_nursery()`. #482 §6 floated a `WGAddress`
registered in `_address_types` — that table is a `bidict`
(1:1 proto-key↔type) and `_addr_to_transport` wants a
`MsgTransport` per addr-type, which `wg` doesn't have.
## next
`WGTunnelledAddr` is deliberately example-local. Promoting it to
`tractor.discovery` as a `TunnelledAddress` whose
`.proto_key`/`.unwrap()` delegate to `.inner`, plus
`open_bindspace()` `@acm`s that create/tear down the iface +
netns via `pyroute2`, is layers A→C of the plan doc.

View File

@ -0,0 +1,61 @@
# tractor: distributed structured concurrency.
'''
Host A: the service host, reachable over a `wg` tunnel.
Binds `tractor`'s registrar + an `echo_srv` sub-actor on the
tunnel's *overlay* addr, declared as a single `wg` maddr.
'''
from __future__ import annotations
import tractor
import trio
from wg_maddr import (
parse_wg_maddr,
verify_wg_peer,
WGTunnelledAddr,
)
# bearer = host A's underlay `(ip, wg ListenPort)`
# key = host A's OWN tunnel pubkey
# overlay = the ep `tractor` binds, on the wg iface's addr
WG_MADDR: str = (
'/ip4/192.168.1.50/udp/51820'
'/wg/u<A_pub_b64url>'
'/ip4/10.0.11.1/tcp/1616'
)
async def echo(msg: str) -> str:
actor = tractor.current_actor()
return f'{actor.aid.name!r} echoes: {msg}'
async def main():
addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR)
assert verify_wg_peer(addr), (
f'wg pubkey from maddr not active on wg0 !\n'
f'maddr: {WG_MADDR}\n'
f'key: {addr.peer_pubkey}\n'
)
print(
f'wg bearer (kernel-owned): {addr.bearer}\n'
f'tractor overlay ep: {addr.inner}\n'
)
async with tractor.open_nursery(
# XXX only `.inner` crosses into the runtime; the bearer
# + key are iface-layer concerns `tractor` never binds.
registry_addrs=[addr.inner],
enable_transports=[addr.inner_proto],
) as an:
await an.start_actor(
'echo_srv',
enable_modules=[__name__],
)
print(f'echo_srv up on\n {addr.maddr}\n')
await trio.sleep_forever()
if __name__ == '__main__':
trio.run(main)

View File

@ -0,0 +1,52 @@
# tractor: distributed structured concurrency.
'''
Host B: workstation dialing host A's actor tree through the
`wg` tunnel.
'''
from __future__ import annotations
import tractor
import trio
from host_a_srv import echo # noqa: F401 (RPC refs it by mod path)
from wg_maddr import (
parse_wg_maddr,
verify_wg_peer,
WGTunnelledAddr,
)
# same maddr as host A: A's bearer, A's key, A's overlay ep
WG_MADDR: str = (
'/ip4/192.168.1.50/udp/51820'
'/wg/u<A_pub_b64url>'
'/ip4/10.0.11.1/tcp/1616'
)
async def main():
addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR)
assert verify_wg_peer(addr), (
f'wg pubkey from maddr not a peer on wg0 !\n'
f'maddr: {WG_MADDR}\n'
)
async with (
tractor.open_root_actor(
name='wg_client',
registry_addrs=[addr.inner],
enable_transports=[addr.inner_proto],
),
tractor.find_actor(
'echo_srv',
registry_addrs=[addr.inner],
) as portal,
):
res: str = await portal.run(
echo,
msg='hello over wg!',
)
print(res)
if __name__ == '__main__':
trio.run(main)

View File

@ -0,0 +1,248 @@
# tractor: distributed structured concurrency.
r'''
Parse `wg`-tunnelled multiaddrs into `tractor`-ready addrs.
The canonical form (per py-multiaddr #108, verified to parse +
round-trip against its upstream merge) nests the *overlay*
endpoint **after** the `/wg/` segment:
/ip4/10.0.0.1/udp/51820/wg/u<key>/ip4/10.0.11.1/tcp/1616
\_______ wg bearer ______/\_ key _/\____ tractor ep _____/
(underlay, wg
`ListenPort`)
- the segments *before* `/wg/` are the **bearer**: the underlay
`(ip, udp-port)` that `wg(8)` itself listens on. Nothing in
`tractor` ever binds this the kernel/`wg` iface owns it.
- `/wg/u<key>` carries the tunnel peer's Curve25519 pubkey as
multibase base64url (std base64 from `wg(8)` contains `/` and
can't go in a `/`-delimited maddr).
- the segments *after* are the **overlay** endpoint, i.e. the
addr `tractor` actually binds/dials. This is the only part the
runtime sees.
XXX NOTE, `tractor`'s own `parse_maddr()` can't parse this yet
(`ValueError('Unsupported multiaddr protocol combo')`), which is
why this module exists: parse here, hand `.inner` to the runtime.
Design rules this module follows (see
`ai/tpt-backends/03_wg_tunnel_bindspace.md`):
- **parsing is pure**. `parse_wg_maddr()` does no I/O, no
`subprocess`, no netlink. A parser that shells out is a nasty
surprise.
- **verification is an explicit, separate step**. The caller
composes `verify_wg_peer()` when it wants it; nothing implicit.
- **no new `Address` proto-type**. `wg` gets no entry in
`tractor.discovery._addr._address_types` (a `bidict`, so 1:1
proto-key<->type) bc it has no `MsgTransport` of its own. The
tunnel is a *bindspace*, so we carry it beside the inner addr
and strip to `.inner` at bind/dial time.
'''
from __future__ import annotations
import base64
import subprocess
from typing import Literal
import msgspec
class WGTunnelledAddr(
msgspec.Struct,
frozen=True,
):
'''
A `wg`-tunnelled endpoint: the underlay bearer, the tunnel
peer key, and the overlay addr `tractor` binds/dials.
'''
# underlay, owned by `wg(8)`/the kernel — NEVER bound by us
bearer: tuple[str, int]
# tunnel peer pubkey in the std-base64 `wg(8)` form, i.e.
# directly comparable to `wg show <if> peers` output
peer_pubkey: str
# overlay ep: an `UnwrappedAddress` as accepted by
# `tractor.discovery.wrap_address()`
inner: tuple[str, int]
inner_proto: Literal['tcp'] = 'tcp'
@property
def maddr(self) -> str:
'''
Re-render the canonical maddr `str` form.
'''
b_host, b_port = self.bearer
i_host, i_port = self.inner
return (
f'/ip4/{b_host}/udp/{b_port}'
f'/wg/{mb_pubkey(self.peer_pubkey)}'
f'/ip4/{i_host}/{self.inner_proto}/{i_port}'
)
def mb_pubkey(wg8_key: str) -> str:
'''
`wg(8)` std-base64 pubkey -> multibase base64url (`u`-prefixed).
'''
import multibase
raw: bytes = base64.b64decode(wg8_key)
return multibase.encode('base64url', raw).decode('ascii')
def wg8_pubkey(mb_key: str) -> str:
'''
Inverse of `mb_pubkey()`: multibase -> `wg(8)` std-base64.
'''
import multibase
raw: bytes = multibase.decode(mb_key)
return base64.b64encode(raw).decode('ascii')
def parse_wg_maddr(
maddr: str,
) -> WGTunnelledAddr:
'''
Split a `wg`-tunnelled maddr into its bearer/key/overlay
parts. Pure no I/O.
Total-or-raises: with a `wg`-aware `py-multiaddr` (#108) an
unparseable maddr raises instead of yielding a struct built
from garbage segments. See `_segments()` for the degraded
pre-#108 path.
'''
segs: list[str] = _segments(maddr)
try:
wg_at: int = segs.index('wg')
except ValueError:
raise ValueError(
f'Not a `wg`-tunnelled maddr, no `/wg/` segment ??\n'
f'maddr: {maddr!r}\n'
)
bearer_segs: list[str] = segs[:wg_at]
mb_key: str = segs[wg_at + 1]
inner_segs: list[str] = segs[wg_at + 2:]
match bearer_segs:
case ['ip4'|'ip6', str() as b_host, 'udp', str() as b_port]:
bearer = (b_host, int(b_port))
case _:
raise ValueError(
f'Bad `wg` bearer, expected `/ip4|ip6/<h>/udp/<p>`\n'
f'got: {"/".join(bearer_segs)!r}\n'
f'from maddr: {maddr!r}\n'
)
match inner_segs:
case ['ip4'|'ip6', str() as i_host, 'tcp', str() as i_port]:
inner = (i_host, int(i_port))
inner_proto = 'tcp'
case []:
raise ValueError(
f'`wg` maddr declares no overlay endpoint!\n'
f'A bare `/…/wg/<key>` names only the tunnel; '
f'append the ep `tractor` should bind, e.g.\n'
f' {maddr}/ip4/10.0.11.1/tcp/1616\n'
)
case _:
raise ValueError(
f'Unsupported `wg` overlay proto combo\n'
f'got: {"/".join(inner_segs)!r}\n'
f'from maddr: {maddr!r}\n'
)
return WGTunnelledAddr(
bearer=bearer,
peer_pubkey=wg8_pubkey(mb_key),
inner=inner,
inner_proto=inner_proto,
)
_wg_proto_known: bool|None = None
def _have_wg_maddr_proto() -> bool:
'''
True iff the installed `py-multiaddr` knows the `/wg/` proto,
i.e. carries py-multiaddr#108.
Merged upstream 2026-07-28 but in no release as of `0.2.0`,
hence the `[tool.uv.sources]` `rev` pin.
Pure predicate; result cached since it can't change without a
reinstall.
'''
global _wg_proto_known
if _wg_proto_known is None:
from multiaddr.protocols import protocol_with_name
from multiaddr.exceptions import ProtocolNotFoundError
try:
protocol_with_name('wg')
_wg_proto_known = True
except ProtocolNotFoundError:
_wg_proto_known = False
return _wg_proto_known
def _segments(maddr: str) -> list[str]:
'''
Deliver a maddr's `/`-split segments, validating via the real
parser whenever it knows `wg`.
'''
if _have_wg_maddr_proto():
from multiaddr import Multiaddr
# the real thing: validates every proto + value, incl.
# that the `wg` key decodes to exactly 32 bytes. Let it
# raise — a maddr that doesn't parse must NOT reach
# `wg8_pubkey()`, which would happily emit a corrupt key.
Multiaddr(maddr)
# XXX, degraded path for a pre-#108 `py-multiaddr` ONLY: no
# per-segment validation, so a malformed key survives to the
# returned struct. We deliberately DON'T hand-roll a `wg`
# codec (the whole point of gh #429 was dropping the NIH
# parser) — install the pinned rev to get validation back.
return [s for s in maddr.split('/') if s]
def verify_wg_peer(
addr: WGTunnelledAddr,
iface: str = 'wg0',
) -> bool:
'''
True iff `addr.peer_pubkey` is a configured peer (or our own
pubkey) on `iface`.
IMPURE + explicit by design: never called from
`parse_wg_maddr()`.
?TODO, per plan-03 layer B, swap this body for `pyroute2`
(keeping the signature) and note `setns(2)` is *per-thread*,
so a query issued via `trio.to_thread` lands in the ORIGINAL
netns unless `netns=` is passed down.
'''
def _wg(*args: str) -> str:
return subprocess.run(
['wg', 'show', iface, *args],
capture_output=True,
text=True,
check=True,
).stdout
return (
addr.peer_pubkey in _wg('peers').split()
or
addr.peer_pubkey == _wg('public-key').strip()
)

View File

@ -166,6 +166,17 @@ sync_pause = {requires-python = ">=3.13, <3.14"}
# linux kernel networking # linux kernel networking
# 'pyroute2 # 'pyroute2
# XXX TEMP, the `/wg/u<key>` maddr proto is MERGED upstream (in
# py-multiaddr#108, 2026-07-28) but is in NO release yet; the
# latest `0.2.0` (2026-03-17) predates the merge by ~4 months.
# Pinned by `rev` (not `branch`) so CI stays reproducible.
#
# Drop this pin (and bump the `multiaddr` dep floor above) the
# moment a release carries the `wg` codec; the only consumer is
# `examples/multihost/wg_lan/`.
# |_https://github.com/multiformats/py-multiaddr/pull/108
multiaddr = { git = 'https://github.com/multiformats/py-multiaddr.git', rev = 'f86519daaa21699023d0037c58cdff600313dd09' }
# ------ tool.uv.sources ------ # ------ tool.uv.sources ------
[tool.uv] [tool.uv]

View File

@ -0,0 +1,52 @@
'''
Unit tests for the `tractor.devx.pformat` render helpers.
'''
from __future__ import annotations
import pytest
from tractor.devx.pformat import (
pformat_boxed_tb,
pformat_caller_frame,
)
@pytest.mark.parametrize(
'box_tb',
[True, False],
ids=['boxed', 'bare'],
)
def test_pformat_caller_frame_renders(box_tb: bool):
'''
`pformat_caller_frame()` must render, not raise.
XXX the `box_tb=True` branch was passing an `indent=''` kwarg
that `pformat_boxed_tb()` never accepted, so it blew up with
a `TypeError`. Nothing in the test suite covered it, and the
only caller is `_mk_send_mte()` i.e. EVERY send-side
`MsgTypeError` died while formatting itself, masking the real
msg-spec violation behind a bogus `TypeError`.
'''
report: str = pformat_caller_frame(
stack_limit=3,
box_tb=box_tb,
)
assert isinstance(report, str)
assert 'test_pformat_caller_frame_renders' in report
def test_pformat_boxed_tb_rejects_unknown_kwargs():
'''
Pin the signature so a future typo'd kwarg fails loudly at the
call site rather than only when some rare error path runs.
'''
assert pformat_boxed_tb(tb_str='doggy\n')
with pytest.raises(TypeError):
pformat_boxed_tb(
tb_str='doggy\n',
indent='',
)

View File

@ -84,6 +84,35 @@ def _wait_for_daemon_ready(
timeout=poll_interval, timeout=poll_interval,
): ):
return return
elif tpt_proto == 'tipc':
# TIPC — `reg_addr` is the proto-keyed
# `('tipc', stype, instance, scope)` per
# `tractor.ipc._tipc.TIPCAddress.unwrap()`.
#
# NOTE, connecting *by name* IS the readiness
# probe: until the daemon `.bind()`s (i.e.
# publishes) the name, the kernel answers
# `EHOSTUNREACH` immediately — no timeout wait.
from tractor.ipc._tipc import (
AF_TIPC,
TIPC_ADDR_NAME,
)
_, stype, instance, scope = reg_addr
sock = socket.socket(AF_TIPC, socket.SOCK_STREAM)
try:
sock.settimeout(poll_interval)
sock.connect((
TIPC_ADDR_NAME,
stype,
instance,
0, # domain: 0 == "anywhere in scope"
scope,
))
return
finally:
sock.close()
else: else:
# UDS — `reg_addr` is a `(filedir, sockname)` # UDS — `reg_addr` is a `(filedir, sockname)`
# tuple per `tractor.ipc._uds.UDSAddress.unwrap`. # tuple per `tractor.ipc._uds.UDSAddress.unwrap`.

View File

@ -19,7 +19,10 @@ from tractor.discovery._multiaddr import (
_tpt_proto_to_maddr, _tpt_proto_to_maddr,
_maddr_to_tpt_proto, _maddr_to_tpt_proto,
) )
from tractor.discovery._addr import wrap_address from tractor.discovery._addr import (
wrap_address,
_address_types,
)
def test_tpt_proto_to_maddr_mapping(): def test_tpt_proto_to_maddr_mapping():
@ -30,7 +33,12 @@ def test_tpt_proto_to_maddr_mapping():
''' '''
assert _tpt_proto_to_maddr['tcp'] == 'tcp' assert _tpt_proto_to_maddr['tcp'] == 'tcp'
assert _tpt_proto_to_maddr['uds'] == 'unix' assert _tpt_proto_to_maddr['uds'] == 'unix'
assert len(_tpt_proto_to_maddr) == 2 assert _tpt_proto_to_maddr['tipc'] == 'tipc'
# NOTE, drive the expected set off the registration table
# (per the "drive-the-set-from-the-`Literal`" pattern) so
# adding a backend can't fail this for the wrong reason.
assert set(_tpt_proto_to_maddr) == set(_address_types)
def test_mk_maddr_tcp_ipv4(): def test_mk_maddr_tcp_ipv4():
@ -153,9 +161,12 @@ def test_maddr_to_tpt_proto_mapping():
''' '''
assert _maddr_to_tpt_proto == { assert _maddr_to_tpt_proto == {
'tcp': 'tcp', maddr_proto: proto_key
'unix': 'uds', for proto_key, maddr_proto in _tpt_proto_to_maddr.items()
} }
assert _maddr_to_tpt_proto['tcp'] == 'tcp'
assert _maddr_to_tpt_proto['unix'] == 'uds'
assert _maddr_to_tpt_proto['tipc'] == 'tipc'
def test_parse_maddr_tcp_ipv4(): def test_parse_maddr_tcp_ipv4():

View File

@ -14,6 +14,7 @@ from tractor import (
from tractor._testing.addr import ( from tractor._testing.addr import (
get_rando_addr, get_rando_addr,
) )
from tractor.ipc._tcp import TCPAddress
# TODO, use/check-roundtripping with some of these wrapper types? # TODO, use/check-roundtripping with some of these wrapper types?
# #
# from .._addr import Address # from .._addr import Address
@ -70,3 +71,71 @@ def test_basic_ipc_server(
pdb=debug_mode, pdb=debug_mode,
): ):
trio.run(main) trio.run(main)
@pytest.mark.parametrize(
'_tpt_proto',
['uds', 'tcp']
)
def test_ep_addr_reconciled_from_sockname(
_tpt_proto: str,
debug_mode: bool,
):
'''
Guard `Endpoint.start_listener()`'s post-bind reconciliation of
`.addr` against the listener's `socket.getsockname()`.
For `tcp` that reconciliation is the ONLY way a kernel-assigned
port (from a `port=0` bind) is ever learned, so it must keep
firing; for `uds` the sock-file path must survive the
round-trip through `.from_addr()` unchanged.
Both are pinned here *before* the reconciliation gets gated on
an `Address.rebind_from_sockname` opt-out (for backends whose
`getsockname()` reports something other than what was bound).
'''
async def main():
async with ipc._server.open_ipc_server() as server:
accept_addr: tuple[str, int|str]
match _tpt_proto:
# XXX the whole point: ask the kernel to pick.
case 'tcp':
accept_addr = (
TCPAddress.def_bindspace,
0,
)
case 'uds':
accept_addr = get_rando_addr(
tpt_proto=_tpt_proto,
)
eps: list[ipc._server.Endpoint] = await server.listen_on(
accept_addrs=[accept_addr],
stream_handler_nursery=None,
)
assert len(eps) == 1
ep: ipc._server.Endpoint = eps[0]
sockname = ep._listener.socket.getsockname()
match _tpt_proto:
case 'tcp':
# the bind req was for "any port"..
assert accept_addr[1] == 0
# ..and the ep learned the real one.
assert ep.addr._port != 0
assert ep.addr.unwrap() == tuple(sockname[:2])
case 'uds':
# sock-file path is stable across the
# `.from_addr()` round-trip.
assert ep.addr.unwrap() == accept_addr
assert str(ep.addr.sockpath) == sockname
server._parent_tn.cancel_scope.cancel()
with devx.maybe_open_crash_handler(
pdb=debug_mode,
):
trio.run(main)

File diff suppressed because it is too large Load Diff

View File

@ -99,6 +99,20 @@ def get_rando_addr(
assert addr.sockpath.resolve() assert addr.sockpath.resolve()
testrun_reg_addr = addr.unwrap() testrun_reg_addr = addr.unwrap()
# NOTE, `.get_random()` already derives the service
# *instance* from a `uuid4`+pid-salted seed, so both the
# within- and cross-proc isolation the other 2 protos
# hand-roll above comes for free.
#
# XXX matters MORE here than for tcp/uds: a TIPC name
# clash doesn't raise `EADDRINUSE`, it silently
# round-robins connects between both publishers.
case 'tipc':
from tractor.ipc._tipc import TIPCAddress
addr: TIPCAddress = addr_type.get_random()
assert addr.is_valid
testrun_reg_addr = addr.unwrap()
# XXX, as sanity it should never the same as the default for the # XXX, as sanity it should never the same as the default for the
# host-singleton registry actor. # host-singleton registry actor.
assert def_reg_addr != testrun_reg_addr assert def_reg_addr != testrun_reg_addr

View File

@ -501,6 +501,12 @@ def pytest_configure(
'trio: legacy mark for tests meant to run under the `trio` ' 'trio: legacy mark for tests meant to run under the `trio` '
'spawn backend (e.g. `test_local.py`).' 'spawn backend (e.g. `test_local.py`).'
) )
config.addinivalue_line(
'markers',
'tipc: test targets the `AF_TIPC` tpt backend; the kernel- '
'touching cases self-skip unless the `tipc` module is loaded '
'(`sudo modprobe tipc`).'
)
# `--enable-stackscope`: install SIGUSR1 → trio task-tree # `--enable-stackscope`: install SIGUSR1 → trio task-tree
# dump in pytest itself + propagate to every subactor via # dump in pytest itself + propagate to every subactor via
@ -797,6 +803,27 @@ def tpt_protos(
addr_type = _addr._address_types[proto_key] addr_type = _addr._address_types[proto_key]
assert addr_type.proto_key == proto_key assert addr_type.proto_key == proto_key
# XXX, generic capability gate: an env-dependent tpt
# whose backing kernel-mod/lib/netns isn't present here
# must fail LOUDLY and EARLY rather than as a few hundred
# confusing connect-timeouts downstream.
#
# Any `Address` type MAY expose `.is_available()`
# returning `(ok, why_not)`; absence means "always
# available" (i.e. tcp/uds).
is_avail = getattr(
addr_type,
'is_available',
None,
)
if is_avail:
avail, why_not = is_avail()
if not avail:
pytest.fail(
f'--tpt-proto={proto_key!r} is NOT usable here!\n'
f'{why_not}\n'
)
yield proto_keys yield proto_keys

View File

@ -215,7 +215,6 @@ def pformat_caller_frame(
tb_str: str = pformat_boxed_tb( tb_str: str = pformat_boxed_tb(
tb_str=tb_str, tb_str=tb_str,
field_prefix=' ', field_prefix=' ',
indent='',
) )
return tb_str return tb_str

View File

@ -33,6 +33,7 @@ from ..runtime._state import (
) )
from ..ipc._tcp import TCPAddress from ..ipc._tcp import TCPAddress
from ..ipc._uds import UDSAddress from ..ipc._uds import UDSAddress
from ..ipc._tipc import TIPCAddress
if TYPE_CHECKING: if TYPE_CHECKING:
from ..runtime._runtime import Actor from ..runtime._runtime import Actor
@ -65,9 +66,22 @@ log = get_logger()
# #
UnwrappedAddress = ( UnwrappedAddress = (
# tcp/udp/uds # tcp/udp/uds
# ('127.0.0.1', 1616)
# ('/run/user/1000/tractor', 'registry@1616.sock')
#
# ..and the explicitly proto-keyed (`multiaddr`-spelled)
# form, which is where ALL backends should eventually land
# per the note below,
# ('tipc', 1953628160, 1616, 2)
#
# XXX VARIADIC bc `msgspec` refuses a union of >1 array-like
# type, so the two shapes can't be spelled as a union. Keep
# in sync with `.msg.types.UnwrappedAddress` which
# re-declares this to dodge a circular import AND is what
# actually validates the `SpawnSpec` wire msg!
tuple[ tuple[
str, # host/domain(tcp), filesys-dir(uds) str|int,
int|str, # port/path(uds) ...,
] ]
# ?TODO? should we also include another 2 fields from # ?TODO? should we also include another 2 fields from
# our `Aid` msg such that we include the runtime `Actor.uid` # our `Aid` msg such that we include the runtime `Actor.uid`
@ -83,6 +97,17 @@ class Address(Protocol):
proto_key: ClassVar[str] proto_key: ClassVar[str]
unwrapped_type: ClassVar[UnwrappedAddress] unwrapped_type: ClassVar[UnwrappedAddress]
# whether `.ipc._server.Endpoint.start_listener()` should
# reconcile a bound `.addr` against its listener's
# `socket.getsockname()`.
#
# XXX NOTE, that reconciliation exists ONLY to learn the
# kernel-*assigned* port from a `port=0` tcp bind; a backend
# whose `getsockname()` reports a categorically different thing
# than what was `.bind()`ed must opt out with `False`, else the
# ep's addr gets clobbered by an un-dialable one.
rebind_from_sockname: ClassVar[bool]
# TODO, i feel like an `.is_bound()` is a better thing to # TODO, i feel like an `.is_bound()` is a better thing to
# support? # support?
# Lke, what use does this have besides a noop and if it's not # Lke, what use does this have besides a noop and if it's not
@ -172,7 +197,8 @@ class Address(Protocol):
_address_types: bidict[str, Type[Address]] = { _address_types: bidict[str, Type[Address]] = {
'tcp': TCPAddress, 'tcp': TCPAddress,
'uds': UDSAddress 'uds': UDSAddress,
'tipc': TIPCAddress,
} }
@ -184,6 +210,9 @@ _default_lo_addrs: dict[
] = { ] = {
'tcp': TCPAddress.get_root().unwrap(), 'tcp': TCPAddress.get_root().unwrap(),
'uds': UDSAddress.get_root().unwrap(), 'uds': UDSAddress.get_root().unwrap(),
# NOTE, pure/cheap: a service-name pair, no kernel module
# nor I/O required at import time.
'tipc': TIPCAddress.get_root().unwrap(),
} }
@ -229,6 +258,20 @@ def wrap_address(
# import pdbp; pdbp.set_trace() # import pdbp; pdbp.set_trace()
match addr: match addr:
# XXX, the explicitly proto-keyed form (spelled with the
# `multiaddr` proto name) which is where ALL backends
# should eventually land per the `UnwrappedAddress`
# migration note above.
#
# NOTE, a bare seq-pattern matches `list` too, which is
# what `msgpack` decodes our tuples back to.
case (
('tipc', int(), int())
|
('tipc', int(), int(), int())
):
cls = TIPCAddress
# classic network socket-address as tuple/list # classic network socket-address as tuple/list
case ( case (
(str(), int()) (str(), int())

View File

@ -38,8 +38,23 @@ if TYPE_CHECKING:
_tpt_proto_to_maddr: dict[str, str] = { _tpt_proto_to_maddr: dict[str, str] = {
'tcp': 'tcp', 'tcp': 'tcp',
'uds': 'unix', 'uds': 'unix',
'tipc': 'tipc',
} }
# XXX, there is NO `/tipc` in the multiaddr protocol table yet
# (upstream track: gh #483 + multiformats/py-multiaddr#107), and
# `Multiaddr()` rejects an unregistered proto name outright.
#
# So until that lands `tipc` maddrs stay **`str`**-only — which
# `MsgTransport.maddr`s `Multiaddr|str` return type already
# allows and `MsgpackUDSStream.maddr` already exercises — and
# `parse_maddr()` special-cases the prefix BEFORE handing
# anything to `Multiaddr()`.
#
# This is also why gh #443's "always return `Multiaddr`" item
# stays blocked.
_tipc_maddr_prefix: str = '/tipc/'
# reverse mapping: multiaddr protocol name -> tractor proto_key # reverse mapping: multiaddr protocol name -> tractor proto_key
_maddr_to_tpt_proto: dict[str, str] = { _maddr_to_tpt_proto: dict[str, str] = {
v: k for k, v in _tpt_proto_to_maddr.items() v: k for k, v in _tpt_proto_to_maddr.items()
@ -49,12 +64,15 @@ _maddr_to_tpt_proto: dict[str, str] = {
def mk_maddr( def mk_maddr(
addr: 'Address', addr: 'Address',
) -> Multiaddr: ) -> Multiaddr|str:
''' '''
Construct a `Multiaddr` from a tractor `Address` instance, Construct a `Multiaddr` from a tractor `Address` instance,
dispatching on the `.proto_key` to build the correct dispatching on the `.proto_key` to build the correct
multiaddr-spec-compliant protocol path. multiaddr-spec-compliant protocol path.
Return a `Multiaddr` for registered protocols. TIPC remains
an interim `str` until its upstream multiaddr protocol lands.
''' '''
proto_key: str = addr.proto_key proto_key: str = addr.proto_key
maddr_proto: str|None = _tpt_proto_to_maddr.get(proto_key) maddr_proto: str|None = _tpt_proto_to_maddr.get(proto_key)
@ -75,6 +93,18 @@ def mk_maddr(
f'/{net_proto}/{host}/{maddr_proto}/{port}' f'/{net_proto}/{host}/{maddr_proto}/{port}'
) )
# NOTE, interim `str`-only grammar (see the
# `_tipc_maddr_prefix` note above),
#
# /tipc/<stype>/<instance>/<scope>
#
# mirroring how `uds` maps onto the spec-legal `/unix`.
case 'tipc':
_, stype, instance, scope = addr.unwrap()
return (
f'/{maddr_proto}/{stype}/{instance}/{scope}'
)
case 'uds': case 'uds':
filedir, filename = addr.unwrap() filedir, filename = addr.unwrap()
filepath = Path(filedir) / filename filepath = Path(filedir) / filename
@ -100,6 +130,23 @@ def parse_maddr(
# lazy imports to avoid circular deps # lazy imports to avoid circular deps
from tractor.ipc._tcp import TCPAddress from tractor.ipc._tcp import TCPAddress
from tractor.ipc._uds import UDSAddress from tractor.ipc._uds import UDSAddress
from tractor.ipc._tipc import TIPCAddress
# XXX MUST come before `Multiaddr()` which rejects the
# not-yet-registered `/tipc` proto name outright.
if maddr_str.startswith(_tipc_maddr_prefix):
try:
_, _, stype, instance, scope = maddr_str.split('/')
return TIPCAddress.from_addr((
'tipc',
int(stype),
int(instance),
int(scope),
))
except (TypeError, ValueError) as src_err:
raise ValueError(
f'Invalid TIPC multiaddr: {maddr_str!r}'
) from src_err
maddr = Multiaddr(maddr_str) maddr = Multiaddr(maddr_str)
proto_names: list[str] = [ proto_names: list[str] = [

View File

@ -661,7 +661,16 @@ class Endpoint(Struct):
# NOTE, for handling the resolved non-0 port for # NOTE, for handling the resolved non-0 port for
# TCP/UDP network sockets. # TCP/UDP network sockets.
#
# XXX, gated on the addr-type's opt-in since for some
# backends `getsockname()` does NOT answer "the addr you
# bound"; `tipc` reports a `TIPC_ADDR_ID` port-id instead
# of the published name-seq, so rebinding from it would
# replace a dialable service-name with an un-dialable
# (and un-reconstructable) port-id.
if ( if (
self.addr.rebind_from_sockname
and
(unwrapped := lstnr.socket.getsockname()) (unwrapped := lstnr.socket.getsockname())
!= !=
self.addr.unwrap() self.addr.unwrap()

View File

@ -65,6 +65,10 @@ class TCPAddress(
unwrapped_type: ClassVar[type] = tuple[str, int] unwrapped_type: ClassVar[type] = tuple[str, int]
def_bindspace: ClassVar[str] = '127.0.0.1' def_bindspace: ClassVar[str] = '127.0.0.1'
# XXX, REQUIRED here since a `port=0` bind means the kernel
# picks and `getsockname()` is the only way we learn it.
rebind_from_sockname: ClassVar[bool] = True
# ?TODO, actually validate ipv4/6 with stdlib's `ipaddress` # ?TODO, actually validate ipv4/6 with stdlib's `ipaddress`
@property @property
def is_valid(self) -> bool: def is_valid(self) -> bool:

1145
tractor/ipc/_tipc.py 100644

File diff suppressed because it is too large Load Diff

View File

@ -38,17 +38,23 @@ from tractor.ipc._uds import (
UDSAddress, UDSAddress,
MsgpackUDSStream, MsgpackUDSStream,
) )
from tractor.ipc._tipc import (
AF_TIPC,
TIPCAddress,
MsgpackTIPCStream,
)
# if TYPE_CHECKING: # if TYPE_CHECKING:
# from tractor._addr import Address # from tractor._addr import Address
Address = TCPAddress|UDSAddress Address = TCPAddress|UDSAddress|TIPCAddress
# manually updated list of all supported msg transport types # manually updated list of all supported msg transport types
_msg_transports = [ _msg_transports = [
MsgpackTCPStream, MsgpackTCPStream,
MsgpackUDSStream MsgpackUDSStream,
MsgpackTIPCStream,
] ]
@ -59,15 +65,17 @@ _key_to_transport: dict[
] = { ] = {
('msgpack', 'tcp'): MsgpackTCPStream, ('msgpack', 'tcp'): MsgpackTCPStream,
('msgpack', 'uds'): MsgpackUDSStream, ('msgpack', 'uds'): MsgpackUDSStream,
('msgpack', 'tipc'): MsgpackTIPCStream,
} }
# convert an Address wrapper to its corresponding transport type # convert an Address wrapper to its corresponding transport type
_addr_to_transport: dict[ _addr_to_transport: dict[
Type[TCPAddress|UDSAddress], Type[TCPAddress|UDSAddress|TIPCAddress],
Type[MsgTransport] Type[MsgTransport]
] = { ] = {
TCPAddress: MsgpackTCPStream, TCPAddress: MsgpackTCPStream,
UDSAddress: MsgpackUDSStream, UDSAddress: MsgpackUDSStream,
TIPCAddress: MsgpackTIPCStream,
} }
@ -108,6 +116,12 @@ def transport_from_stream(
case socket.AF_UNIX: case socket.AF_UNIX:
transport = 'uds' transport = 'uds'
# NOTE, `AF_TIPC` is linux-only in CPython so we
# match the `._tipc` constant (which carries a uapi
# fallback) rather than `socket.AF_TIPC`.
case _ if sock.family == AF_TIPC:
transport = 'tipc'
case _: case _:
raise NotImplementedError( raise NotImplementedError(
f'Unsupported socket family: {sock.family}' f'Unsupported socket family: {sock.family}'

View File

@ -114,9 +114,15 @@ class UDSAddress(
# -[ ] need to check what other mult-transport frameworks do # -[ ] need to check what other mult-transport frameworks do
# like zmq, nng, uri-spec et al! # like zmq, nng, uri-spec et al!
proto_key: ClassVar[str] = 'uds' proto_key: ClassVar[str] = 'uds'
unwrapped_type: ClassVar[type] = tuple[str, int] unwrapped_type: ClassVar[type] = tuple[str, str]
def_bindspace: ClassVar[Path] = get_rt_dir() def_bindspace: ClassVar[Path] = get_rt_dir()
# NOTE, `getsockname()` answers the sock-file path as a `str`
# which never `==` our 2-tuple `.unwrap()`, so the round-trip
# always fires; it's a no-op modulo `.maybe_pid` and is kept
# `True` to preserve pre-existing behaviour exactly.
rebind_from_sockname: ClassVar[bool] = True
@property @property
def bindspace(self) -> Path: def bindspace(self) -> Path:
''' '''
@ -170,7 +176,7 @@ class UDSAddress(
f'{addr!r}\n' f'{addr!r}\n'
) )
def unwrap(self) -> tuple[str, int]: def unwrap(self) -> tuple[str, str]:
# XXX NOTE, since this gets passed DIRECTLY to # XXX NOTE, since this gets passed DIRECTLY to
# `.ipc._uds.open_unix_socket_w_passcred()` # `.ipc._uds.open_unix_socket_w_passcred()`
return ( return (

View File

@ -188,6 +188,31 @@ class Aid(
__repr__ = pretty_struct.Struct.__repr__ __repr__ = pretty_struct.Struct.__repr__
# NOTE, mirrors `.discovery._addr.UnwrappedAddress` but is
# re-declared here to dodge the circular import
# (`._addr` -> `.ipc._tcp` -> `.msg`).
#
# XXX this is the **wire** shape, so widening it is a wire-format
# change; keep the two decls in sync.
# XXX VARIADIC on purpose! `msgspec` rejects a union holding
# more than one array-like type ("Type unions may not contain
# more than one array-like (list, set, tuple) type"), so the two
# concrete shapes,
#
# ('127.0.0.1', 1616) # tcp
# ('/run/user/1000/tractor', 'x.sock') # uds
# ('tipc', 1953628160, 1616, 2) # proto-keyed (tipc)
#
# can't be spelled as `tuple[str, str|int]|tuple[str, int, int,
# int]`. Widen to one homogeneous variadic tuple instead.
#
# ?TODO, the real fix is the `UnwrappedAddress` proto-key
# migration (see `.discovery._addr`) after which this becomes a
# tagged union keyed off elem 0 and full per-proto validation
# comes back.
UnwrappedAddress = tuple[str|int, ...]
class SpawnSpec( class SpawnSpec(
pretty_struct.Struct, pretty_struct.Struct,
tag=True, tag=True,
@ -213,8 +238,8 @@ class SpawnSpec(
# TODO: not just sockaddr pairs? # TODO: not just sockaddr pairs?
# -[ ] abstract into a `TransportAddr` type? # -[ ] abstract into a `TransportAddr` type?
reg_addrs: list[tuple[str, str|int]] reg_addrs: list[UnwrappedAddress]
bind_addrs: list[tuple[str, str|int]]|None bind_addrs: list[UnwrappedAddress]|None
# TODO: caps based RPC support in the payload? # TODO: caps based RPC support in the payload?

View File

@ -47,6 +47,7 @@ if TYPE_CHECKING:
TransportProtocolKey = Literal[ TransportProtocolKey = Literal[
'tcp', 'tcp',
'uds', 'uds',
'tipc',
] ]
_def_tpt_proto: TransportProtocolKey = 'tcp' _def_tpt_proto: TransportProtocolKey = 'tcp'

View File

@ -518,7 +518,7 @@ wheels = [
[[package]] [[package]]
name = "multiaddr" name = "multiaddr"
version = "0.2.0" version = "0.2.0"
source = { registry = "https://pypi.org/simple" } source = { git = "https://github.com/multiformats/py-multiaddr.git?rev=f86519daaa21699023d0037c58cdff600313dd09#f86519daaa21699023d0037c58cdff600313dd09" }
dependencies = [ dependencies = [
{ name = "base58" }, { name = "base58" },
{ name = "dnspython" }, { name = "dnspython" },
@ -533,10 +533,6 @@ dependencies = [
{ name = "trio-typing" }, { name = "trio-typing" },
{ name = "varint" }, { name = "varint" },
] ]
sdist = { url = "https://files.pythonhosted.org/packages/c7/10/4e26a8577cfce1c0febc8d83087e1373e93c695c6e73ad010546fb67e229/multiaddr-0.2.0.tar.gz", hash = "sha256:acb6b25c332ec1b2f1f8fef8d03a8c63385d34a87d690df0f4bba43cdf6efe8d", size = 58356, upload-time = "2026-03-17T21:51:00.274Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b5/13/56e503d01218d1ca27ea9fda862045a4b400cae5e756f47315f5aaba0eee/multiaddr-0.2.0-py3-none-any.whl", hash = "sha256:bcff7bf3d7de3d6da0b865b25423bcb411de1d20d70cc6abfacf75170d17866c", size = 40424, upload-time = "2026-03-17T21:50:58.833Z" },
]
[[package]] [[package]]
name = "mypy-extensions" name = "mypy-extensions"
@ -1178,7 +1174,7 @@ requires-dist = [
{ name = "bidict", specifier = ">=0.23.1" }, { name = "bidict", specifier = ">=0.23.1" },
{ name = "colorlog", specifier = ">=6.8.2,<7" }, { name = "colorlog", specifier = ">=6.8.2,<7" },
{ name = "msgspec", specifier = ">=0.20.0" }, { name = "msgspec", specifier = ">=0.20.0" },
{ name = "multiaddr", specifier = ">=0.2.0" }, { name = "multiaddr", git = "https://github.com/multiformats/py-multiaddr.git?rev=f86519daaa21699023d0037c58cdff600313dd09" },
{ name = "pdbp", specifier = ">=1.8.2,<2" }, { name = "pdbp", specifier = ">=1.8.2,<2" },
{ name = "platformdirs", specifier = ">=4.4.0" }, { name = "platformdirs", specifier = ">=4.4.0" },
{ name = "setproctitle", specifier = ">=1.3,<2" }, { name = "setproctitle", specifier = ">=1.3,<2" },