Compare commits
41 Commits
80e1ec6461
...
be108ccc32
| Author | SHA1 | Date |
|---|---|---|
|
|
be108ccc32 | |
|
|
ff2b20b7d1 | |
|
|
caf9c3d6d3 | |
|
|
53516b094c | |
|
|
c1501a36d5 | |
|
|
be6f9e86d1 | |
|
|
145782d38c | |
|
|
d52c78c106 | |
|
|
a19a639ddf | |
|
|
1298ba945f | |
|
|
c7ae606504 | |
|
|
4aa7a890cb | |
|
|
1802641e41 | |
|
|
2d082373bd | |
|
|
1526372e37 | |
|
|
b14332017d | |
|
|
33a040b312 | |
|
|
e269bbf871 | |
|
|
7e20585f59 | |
|
|
51d7133f47 | |
|
|
e0f66616cd | |
|
|
2204979492 | |
|
|
8c0ae140cd | |
|
|
e3089ba356 | |
|
|
cca3a70de4 | |
|
|
d4737e957f | |
|
|
f9f98eeb06 | |
|
|
22ef362d9f | |
|
|
ee17ed9f6e | |
|
|
7d6e79551e | |
|
|
cc85f17f5f | |
|
|
0c96f92396 | |
|
|
27c34aebb6 | |
|
|
bf974c9870 | |
|
|
41d08d04a6 | |
|
|
208876cc3b | |
|
|
f60bdc44c7 | |
|
|
7806b60fff | |
|
|
7fab27de29 | |
|
|
f7e3fdea1a | |
|
|
402e74b97a |
|
|
@ -117,6 +117,7 @@ jobs:
|
||||||
tpt_proto: [
|
tpt_proto: [
|
||||||
'tcp',
|
'tcp',
|
||||||
'uds',
|
'uds',
|
||||||
|
'tipc',
|
||||||
]
|
]
|
||||||
# https://github.com/orgs/community/discussions/26253#discussioncomment-3250989
|
# https://github.com/orgs/community/discussions/26253#discussioncomment-3250989
|
||||||
exclude:
|
exclude:
|
||||||
|
|
@ -124,6 +125,11 @@ jobs:
|
||||||
- os: macos-latest
|
- os: macos-latest
|
||||||
tpt_proto: 'uds'
|
tpt_proto: 'uds'
|
||||||
|
|
||||||
|
# XXX `AF_TIPC` is a linux-kernel protocol; the address
|
||||||
|
# family doesn't exist on darwin at all.
|
||||||
|
- os: macos-latest
|
||||||
|
tpt_proto: 'tipc'
|
||||||
|
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
|
@ -150,6 +156,20 @@ jobs:
|
||||||
- name: List deps tree
|
- name: List deps tree
|
||||||
run: uv tree
|
run: uv tree
|
||||||
|
|
||||||
|
# XXX `AF_TIPC` needs its kernel module loaded! It ships
|
||||||
|
# with the standard ubuntu kernel package but is NOT loaded
|
||||||
|
# by default, so a bare `--tpt-proto=tipc` run would
|
||||||
|
# otherwise fail at the first `.bind()`.
|
||||||
|
#
|
||||||
|
# If GH's runners ever refuse the `modprobe`, the fallback
|
||||||
|
# is a container job w/ `--cap-add NET_ADMIN`.
|
||||||
|
- name: 'Load the `tipc` kernel module'
|
||||||
|
if: matrix.tpt_proto == 'tipc'
|
||||||
|
run: |
|
||||||
|
sudo modprobe tipc
|
||||||
|
tipc node get address
|
||||||
|
uv run python -c "from tractor.ipc._tipc import TIPCAddress; ok, why = TIPCAddress.is_available(); assert ok, why; print('AF_TIPC available')"
|
||||||
|
|
||||||
- name: Run tests
|
- name: Run tests
|
||||||
run: >
|
run: >
|
||||||
uv run
|
uv run
|
||||||
|
|
|
||||||
|
|
@ -161,6 +161,10 @@ Session.vim
|
||||||
# -[ ] everything should be put under a `.gish/` no?
|
# -[ ] everything should be put under a `.gish/` no?
|
||||||
gitea/
|
gitea/
|
||||||
gh/
|
gh/
|
||||||
|
# NOTE, `gh issue create` writes its local cache under the
|
||||||
|
# `<backend>/<repo>/<kind>/<num>.md` path, so the dir is named for
|
||||||
|
# the *service* not the CLI.
|
||||||
|
github/
|
||||||
|
|
||||||
# ------ macOS ------
|
# ------ macOS ------
|
||||||
# Finder metadata
|
# Finder metadata
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,136 @@
|
||||||
|
---
|
||||||
|
model: claude-opus-5
|
||||||
|
service: claude
|
||||||
|
session: 7b9c97c4-fff7-4ac4-97fb-35720453308e
|
||||||
|
timestamp: 2026-08-13T00:11:02Z
|
||||||
|
git_ref: 27c34aeb
|
||||||
|
scope: docs+code
|
||||||
|
substantive: true
|
||||||
|
raw_file: 20260813T001102Z_27c34aeb_prompt_io.raw.md
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prompt
|
||||||
|
|
||||||
|
> draft hyper detailed implementation plans for [three]
|
||||||
|
> prospective new transport (tpt) backends for tractor's `.ipc`
|
||||||
|
> layer, from four GitHub issues: TIPC (gh #378) using built-in
|
||||||
|
> linux socket API w/ `trio` interfacing, leveraging TIPC's
|
||||||
|
> built-in discovery machinery; QUIC (gh #353) using the `iroh`
|
||||||
|
> lib, ideally with the py asyncio support (via ffi) rewritten
|
||||||
|
> for trio; wg (gh #482 and/or #443) with other shuttle-able
|
||||||
|
> tpts, using `pyroute2`, as much trio wrapping as possible
|
||||||
|
> where any other async support can be replaced.
|
||||||
|
|
||||||
|
With constraints: "be only slightly speculative"; realistic given
|
||||||
|
the linked info; the plans must be workable "by another
|
||||||
|
model/provider without much discrepancy on the design or lib
|
||||||
|
selections needed"; do it all in a new `/open-wkt` off main; and
|
||||||
|
be "as modular and modern-async-python-as-nearly-functional-style
|
||||||
|
as possible."
|
||||||
|
|
||||||
|
Key follow-up prompts that changed the design mid-session:
|
||||||
|
|
||||||
|
> wg requires binding a socket on both ends of the tunnel, the
|
||||||
|
> new support we added to `py-multiaddr` in their #108 enables
|
||||||
|
> declaring this in the maddr? […] so the
|
||||||
|
> `/ip4/10.0.0.1/udp/666/ip4/192.168.0.13/tcp/80` bullet example
|
||||||
|
> from #443 isn't correct for a wg tunnel maddr and would
|
||||||
|
> actually end up looking like,
|
||||||
|
> `/ip4/10.0.0.1/udp/666/wg/<pub-key>/ip4/192.168.0.13/tcp/80` no?
|
||||||
|
|
||||||
|
> right, we're likely going to need less `tuple`-formed
|
||||||
|
> `UnwrappedAddress` usage to make them all work and/or to extend
|
||||||
|
> the unwrapped form with a proto-str, likely the key from
|
||||||
|
> `multiaddrs` no?
|
||||||
|
|
||||||
|
> better encapsulating `UnwrappedAddress` for use externally is
|
||||||
|
> really way overdue; we should likely be encouraging (if not
|
||||||
|
> preventing unwrapped) wrapped addrs always much like other
|
||||||
|
> (std) libs, `ipaddress` comes to mind.
|
||||||
|
|
||||||
|
> so iirc netns mgmt becomes a lower-level runtime config-api
|
||||||
|
> rather then an actor-app-code API which can be dynamically
|
||||||
|
> changed once the actor is up ya?
|
||||||
|
|
||||||
|
## Response summary
|
||||||
|
|
||||||
|
A research-and-planning branch, `ng_tpts_planning` — **no runtime
|
||||||
|
code touched**. Five plan docs under `ai/tpt-backends/` plus a
|
||||||
|
corrected `wg` example set under `examples/multihost/wg_lan/`,
|
||||||
|
across 9 commits.
|
||||||
|
|
||||||
|
Three design conclusions came out of the planning that outlive
|
||||||
|
the branch:
|
||||||
|
|
||||||
|
1. **`wg` is a bindspace, not a `MsgTransport`.** The `/wg/`
|
||||||
|
maddr segment is *infix*: bearer (underlay ip+udp
|
||||||
|
`ListenPort`, bound by the kernel) → `/wg/u<key>` (an
|
||||||
|
identity, bound by nothing) → overlay (the only part tractor
|
||||||
|
binds). Verified empirically by installing
|
||||||
|
`baudco/py-multiaddr@wg_support` (py-multiaddr#108) in a
|
||||||
|
throwaway venv and round-tripping each candidate form; gh
|
||||||
|
#482's original suffix form parses but is semantically
|
||||||
|
inverted.
|
||||||
|
2. **Proto-key the `UnwrappedAddress` form**, spelled with the
|
||||||
|
`multiaddr` protocol names, dispatched via
|
||||||
|
`_address_types[addr[0]]`. Kills a whole collision class
|
||||||
|
(TIPC's `(str, int)` ≡ `TCPAddress`; iroh's `(str, str)`
|
||||||
|
swallowed by the UDS case) and is the recommended migration
|
||||||
|
*before* any new backend lands.
|
||||||
|
3. **netns is a runtime/boot-time config API, not an app-code
|
||||||
|
one** — `setns(2)` is per-thread and won't move
|
||||||
|
already-created sockets, so there is deliberately no
|
||||||
|
`await actor.enter_netns(...)`.
|
||||||
|
|
||||||
|
Also verified that `trio.SocketStream`/`SocketListener` are
|
||||||
|
address-family agnostic (no `AF_*` check anywhere), which is what
|
||||||
|
makes TIPC the cheapest of the three backends to add.
|
||||||
|
|
||||||
|
Four related issues were annotated with the results (#378, #353,
|
||||||
|
#482, #443); #443's body was rewritten to reflect the corrected
|
||||||
|
grammar, with no existing checkbox state changed.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `ai/tpt-backends/00_shared_backend_contract.md` — normative
|
||||||
|
backend duck-type contract, registration checklist, §1.1
|
||||||
|
proto-key conclusion
|
||||||
|
- `ai/tpt-backends/01_tipc_backend.md` — TIPC plan; service
|
||||||
|
addressing, `TIPC_TOP_SRV` push registry, instance-collision
|
||||||
|
hazard, step-0 probe
|
||||||
|
- `ai/tpt-backends/02_quic_iroh_backend.md` — `iroh` plan;
|
||||||
|
`uniffi`→`trio` bridge, listener/stream adapters, API-truth
|
||||||
|
table
|
||||||
|
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` — `wg`-as-bindspace
|
||||||
|
plan; verified maddr grammar, 3-owner split, netns reality
|
||||||
|
- `ai/tpt-backends/README.md` — index
|
||||||
|
- `examples/multihost/wg_lan/wg_maddr.py` — frozen `msgspec`
|
||||||
|
tunnelled addr + pure parse/render helpers; impure
|
||||||
|
`verify_wg_peer()` kept separate
|
||||||
|
- `examples/multihost/wg_lan/host_a_srv.py` — host-A actor tree
|
||||||
|
- `examples/multihost/wg_lan/host_b_client.py` — host-B dialer
|
||||||
|
- `examples/multihost/wg_lan/README.md` — grammar, owner table,
|
||||||
|
setup, "what changed vs #482"
|
||||||
|
|
||||||
|
## Human edits
|
||||||
|
|
||||||
|
Substantial human steering rather than post-hoc editing; the
|
||||||
|
corrections were applied by the model in-session after being
|
||||||
|
challenged:
|
||||||
|
|
||||||
|
- rejected an initial claim that `wg` has "nothing to bind at the
|
||||||
|
tunnel layer" and supplied the correct composed maddr form,
|
||||||
|
which forced a rewrite of plan 03 §3.2 and a retraction in the
|
||||||
|
already-posted #443 comment
|
||||||
|
- rejected a supporting claim that `/ip4/../udp/443/quic-v1` was
|
||||||
|
"also composed"
|
||||||
|
- directed the proto-key/`ipaddress`-discipline conclusion and
|
||||||
|
the netns-as-runtime-config framing, both of which were then
|
||||||
|
folded back into the docs
|
||||||
|
- chose the commit boundaries and authored all commits; ran every
|
||||||
|
`git` mutation (commit, rebase, push) themselves
|
||||||
|
|
||||||
|
One model-initiated correction pre-publication: a self-review
|
||||||
|
downgraded two overconfident claims (the `uniffi`/asyncio thesis
|
||||||
|
and TIPC duplicate-binder behaviour) to explicitly-flagged
|
||||||
|
assumptions before the #353/#378 comments were posted.
|
||||||
|
|
@ -0,0 +1,165 @@
|
||||||
|
---
|
||||||
|
model: claude-opus-5
|
||||||
|
service: claude
|
||||||
|
timestamp: 2026-08-13T00:11:02Z
|
||||||
|
git_ref: 27c34aeb
|
||||||
|
diff_cmd: git diff main..ng_tpts_planning
|
||||||
|
---
|
||||||
|
|
||||||
|
# Raw output — next-gen tpt-backend implementation plans
|
||||||
|
|
||||||
|
## Generated planning docs
|
||||||
|
|
||||||
|
> `git diff main..ng_tpts_planning -- ai/tpt-backends/`
|
||||||
|
|
||||||
|
Five markdown docs. `00_shared_backend_contract.md` is normative
|
||||||
|
and the other three are written against it so they can be worked
|
||||||
|
independently:
|
||||||
|
|
||||||
|
- **`00_shared_backend_contract.md`** — the backend duck-type
|
||||||
|
(`<Proto>Address(msgspec.Struct, frozen=True)` + module-level
|
||||||
|
`start_listener()`/`close_listener()` + a
|
||||||
|
`Msgpack<Proto>Stream(MsgpackTransport)`), the
|
||||||
|
`inspect.getmodule(self.addr)` reflection in
|
||||||
|
`Endpoint.start_listener()` that forces the Address class and
|
||||||
|
its listener fns to share a module, a 10-item registration
|
||||||
|
checklist, the dep policy, the test-harness shape, and §1.1's
|
||||||
|
proto-key conclusion (below).
|
||||||
|
- **`01_tipc_backend.md`** — service addressing via
|
||||||
|
`TIPC_ADDR_NAMESEQ` (bind/publish) and `TIPC_ADDR_NAME`
|
||||||
|
(connect/lookup), `TIPC_TOP_SRV` topology subscriptions as a
|
||||||
|
push-based registry, the `get_random()` instance-collision
|
||||||
|
hazard, and a step-0 capability-probe spike.
|
||||||
|
- **`02_quic_iroh_backend.md`** — `iroh` over
|
||||||
|
`aioquic`/`quiche`/`trio-asyncio`, a `_uniffi_trio.py` bridge
|
||||||
|
built on `TrioToken.run_sync_soon()`, `trio.abc.Listener`/
|
||||||
|
`HalfCloseableStream` adapters, and an API-truth table to fill
|
||||||
|
in during step 0.
|
||||||
|
- **`03_wg_tunnel_bindspace.md`** — `wg` as a *bindspace* rather
|
||||||
|
than a `MsgTransport`, a `TunnelledAddress` wrapper delegating
|
||||||
|
`.proto_key`/`.unwrap()` to `.inner`, `pyroute2` for layer B,
|
||||||
|
and `@acm`-managed netns/iface for layer C.
|
||||||
|
- **`README.md`** — index.
|
||||||
|
|
||||||
|
## Generated example code
|
||||||
|
|
||||||
|
> `git diff main..ng_tpts_planning -- examples/multihost/wg_lan/`
|
||||||
|
|
||||||
|
- `wg_maddr.py` — `WGTunnelledAddr(msgspec.Struct, frozen=True)`
|
||||||
|
carrying `bearer: tuple[str, int]`, `peer_pubkey: str`,
|
||||||
|
`inner: tuple[str, int]`, `inner_proto: Literal['tcp']`, plus a
|
||||||
|
`.maddr` property that re-renders the canonical form. Pure
|
||||||
|
helpers `mb_pubkey()`, `wg8_pubkey()`, `parse_wg_maddr()`, and
|
||||||
|
`_segments()` (with a marked stopgap for when the `wg` codec
|
||||||
|
isn't installed). `verify_wg_peer()` is impure **by design** and
|
||||||
|
kept out of the parse path.
|
||||||
|
- `host_a_srv.py` / `host_b_client.py` — the two-host runs; both
|
||||||
|
pass only `addr.inner` to `open_nursery()`/`open_root_actor()`.
|
||||||
|
- `README.md` — grammar, owner table, `#108`-branch install line,
|
||||||
|
tunnel setup, "what changed vs #482".
|
||||||
|
|
||||||
|
## Verified findings (non-code, verbatim)
|
||||||
|
|
||||||
|
### `trio` is address-family agnostic
|
||||||
|
|
||||||
|
Read against the installed `trio`. `SocketStream`/`SocketListener`
|
||||||
|
ctor checks are only "is a trio sock object" + `type ==
|
||||||
|
SOCK_STREAM`, plus an `OSError`-**suppressed** `SO_ACCEPTCONN`
|
||||||
|
probe. No `AF_*` check anywhere; `TCP_NODELAY`/`TCP_NOTSENT_LOWAT`
|
||||||
|
are set under `suppress(OSError)`. A TIPC `SOCK_STREAM` sock should
|
||||||
|
therefore drop straight into `trio.serve_listeners()` with the
|
||||||
|
existing `MsgpackTransport` framing, making TIPC mostly
|
||||||
|
table-registration boilerplate w/ zero new deps.
|
||||||
|
|
||||||
|
### the `wg` maddr grammar — `/wg/` is infix, not suffix
|
||||||
|
|
||||||
|
Installed `baudco/py-multiaddr@wg_support` (PR
|
||||||
|
multiformats/py-multiaddr#108) into a throwaway venv and
|
||||||
|
round-tripped every candidate form:
|
||||||
|
|
||||||
|
| maddr | `[p.name for p in m.protocols()]` |
|
||||||
|
| --- | --- |
|
||||||
|
| `/ip4/1.2.3.4/udp/51820/wg/u<k>` | `['ip4','udp','wg']` |
|
||||||
|
| `/ip4/../udp/../wg/u<k>/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` |
|
||||||
|
| `/ip4/10.0.11.1/tcp/1616/wg/u<k>` | `['ip4','tcp','wg']` |
|
||||||
|
|
||||||
|
```
|
||||||
|
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
|
||||||
|
\_______ bearer __________/\__ key __/\______ overlay ______/
|
||||||
|
```
|
||||||
|
|
||||||
|
Segments *before* `/wg/` are the bearer — the underlay
|
||||||
|
`(ip, udp-port)` that `wg(8)` itself listens on (`ListenPort`).
|
||||||
|
Segments *after* are the overlay endpoint, the only part tractor
|
||||||
|
binds. The third row above is #482's original suffix form: it
|
||||||
|
parses, but is semantically inverted.
|
||||||
|
|
||||||
|
Three parts, three owners — and only one is an `Endpoint`:
|
||||||
|
|
||||||
|
| part | bound by | in the runtime? |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| bearer | kernel, via `wg-quick`/`pyroute2` | no |
|
||||||
|
| `/wg/u<key>` | nothing — an identity | no, verified out-of-band |
|
||||||
|
| overlay | `tractor`'s `IPCServer` | yes, as `.inner` |
|
||||||
|
|
||||||
|
### proto-key-tagged `UnwrappedAddress`
|
||||||
|
|
||||||
|
Shape-matching in `wrap_address()` does not survive four backends.
|
||||||
|
TIPC's natural unwrapped form is a `(str, int)`, indistinguishable
|
||||||
|
from `TCPAddress`; iroh's is a `(str, str)`, already swallowed by
|
||||||
|
the existing UDS case (`case (_, filename) if type(filename) is
|
||||||
|
str`). Ordering hacks and prefix-tagging only paper over it.
|
||||||
|
|
||||||
|
Recommended prerequisite for all three backends: carry an explicit
|
||||||
|
proto-key spelled with the `multiaddr` protocol name —
|
||||||
|
`('tcp', host, port)`, `('unix', path)`,
|
||||||
|
`('tipc', stype, inst, scope)` — so `wrap_address()` collapses to
|
||||||
|
`_address_types[addr[0]]` and the collision class stops existing.
|
||||||
|
This also makes the on-wire form agree with
|
||||||
|
`mk_maddr()`/`parse_maddr()` instead of being an independent
|
||||||
|
invention. It is a wire-format change (`SpawnSpec`,
|
||||||
|
`_root_mailbox`, `_registry_addrs`) plus every fixture and
|
||||||
|
downstream config, so it wants its own migration commit landed
|
||||||
|
before any new backend — and it is the moment to stop handing raw
|
||||||
|
tuples to users at all, making `Address` the public currency and
|
||||||
|
`UnwrappedAddress` an internal serialization detail (the
|
||||||
|
discipline `ipaddress` uses).
|
||||||
|
|
||||||
|
### netns is a runtime-level config API
|
||||||
|
|
||||||
|
`setns(2)` affects the calling thread only and does not move
|
||||||
|
already-created sockets. So a netns is a spawn/boot-time input
|
||||||
|
alongside `enable_transports`/`tpt_bind_addrs`, and there is
|
||||||
|
deliberately no `await actor.enter_netns(...)` — a mid-life API
|
||||||
|
would silently leave the IPC server bound in the old namespace.
|
||||||
|
Corollary for layer B: pass `netns=` down to `pyroute2` rather
|
||||||
|
than assuming a `trio.to_thread` worker inherits it.
|
||||||
|
|
||||||
|
### `examples/` collection would have failed CI
|
||||||
|
|
||||||
|
`tests/test_docs_examples.py` walks `examples/` recursively and
|
||||||
|
subproc-runs every collected file asserting `rc == 0`. Its filter
|
||||||
|
never checks the extension, so all four `wg_lan` files were
|
||||||
|
collected — including `README.md`, which would have been run as
|
||||||
|
`python README.md`. `'multihost' not in p[0]` was already in the
|
||||||
|
exclusion list with no directory using it. Moving the set under
|
||||||
|
`examples/multihost/wg_lan/` drops collection 24 → 20 with zero
|
||||||
|
test changes; confirmed via `pytest --collect-only`.
|
||||||
|
|
||||||
|
## Corrections applied during the session
|
||||||
|
|
||||||
|
The human corrected two claims that had been asserted without
|
||||||
|
verification, both since retracted in-place in the docs and in the
|
||||||
|
posted issue comments:
|
||||||
|
|
||||||
|
1. that `wg` has "nothing to bind at the tunnel layer, exactly one
|
||||||
|
bind" — wrong; a wg stack is genuinely composed, and the real
|
||||||
|
axis is *who owns* each layer's endpoint.
|
||||||
|
2. that `/ip4/../udp/443/quic-v1` was "also composed" — wrong;
|
||||||
|
that is one endpoint with a protocol qualifier, not a tunnel.
|
||||||
|
|
||||||
|
A self-review before publication also downgraded two
|
||||||
|
overconfident claims to explicitly-flagged assumptions: the
|
||||||
|
`uniffi`-uses-asyncio-only-as-executor thesis (contradicted that
|
||||||
|
plan's own "do not guess from memory" step 0) and TIPC's
|
||||||
|
duplicate-binder round-robin behaviour (unverified).
|
||||||
|
|
@ -0,0 +1,29 @@
|
||||||
|
---
|
||||||
|
model: openai/gpt-5.6-sol
|
||||||
|
service: opencode
|
||||||
|
session: a998acff-af66-4969-a65e-f77cbf5ef8dd
|
||||||
|
timestamp: 2026-08-19T00:33:26Z
|
||||||
|
git_ref: wkt/tipc_backend_378
|
||||||
|
scope: config
|
||||||
|
substantive: true
|
||||||
|
raw_file: 20260819T003326Z_53516b09_prompt_io.raw.md
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prompt
|
||||||
|
|
||||||
|
Confirm whether the successful TIPC CI leg can become blocking.
|
||||||
|
|
||||||
|
## Response summary
|
||||||
|
|
||||||
|
Removed the temporary TIPC-only `continue-on-error` expression after
|
||||||
|
every refreshed PR #493 check passed.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `.github/workflows/ci.yml` - make TIPC failures block CI.
|
||||||
|
|
||||||
|
## Human edits
|
||||||
|
|
||||||
|
The human explicitly requested reconsidering the gate after the CI
|
||||||
|
run and chose to leave the generated change local for review before
|
||||||
|
requesting this commit plan.
|
||||||
|
|
@ -0,0 +1,23 @@
|
||||||
|
---
|
||||||
|
model: openai/gpt-5.6-sol
|
||||||
|
service: opencode
|
||||||
|
timestamp: 2026-08-19T00:33:26Z
|
||||||
|
git_ref: wkt/tipc_backend_378
|
||||||
|
diff_cmd: git diff HEAD~1..HEAD
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prompt
|
||||||
|
|
||||||
|
The user asked whether the now-green TIPC CI leg could stop using
|
||||||
|
`continue-on-error`.
|
||||||
|
|
||||||
|
## Response
|
||||||
|
|
||||||
|
All checks on PR #493 passed, including the Ubuntu TIPC matrix job.
|
||||||
|
The generated config change removes the temporary non-blocking gate
|
||||||
|
and its obsolete explanatory comment:
|
||||||
|
|
||||||
|
> `git diff HEAD~1..HEAD -- .github/workflows/ci.yml`
|
||||||
|
|
||||||
|
The remaining matrix, Linux-only exclusion and `modprobe tipc` setup
|
||||||
|
stay unchanged.
|
||||||
|
|
@ -0,0 +1,37 @@
|
||||||
|
---
|
||||||
|
model: openai/gpt-5.6-sol
|
||||||
|
service: opencode
|
||||||
|
session: a998acff-af66-4969-a65e-f77cbf5ef8dd
|
||||||
|
timestamp: 2026-08-19T00:33:27Z
|
||||||
|
git_ref: wkt/tipc_backend_378
|
||||||
|
scope: code
|
||||||
|
substantive: true
|
||||||
|
raw_file: 20260819T003327Z_53516b09_prompt_io.raw.md
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prompt
|
||||||
|
|
||||||
|
Develop a practical bootstrap plan for testing TIPC across two
|
||||||
|
physical motherboards.
|
||||||
|
|
||||||
|
## Response summary
|
||||||
|
|
||||||
|
Expanded the two-host example into a reproducible smoke-test runbook
|
||||||
|
and made host A authorize the stable module name imported by host B.
|
||||||
|
Documented the cluster-domain-socket analogy and the prospective
|
||||||
|
`pyroute2` management path.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `examples/multihost/tipc_cluster/README.md` - document setup,
|
||||||
|
validation, resilience testing, capture and cleanup.
|
||||||
|
- `examples/multihost/tipc_cluster/host_a_srv.py` - authorize the
|
||||||
|
`host_a_srv` RPC module under direct script execution.
|
||||||
|
|
||||||
|
## Human edits
|
||||||
|
|
||||||
|
The human selected two separate physical motherboards as the target
|
||||||
|
environment and requested that the resulting local work be packaged
|
||||||
|
as its own commit boundary. The human also proposed “Cluster Domain
|
||||||
|
Sockets” as clearer terminology and reusing the planned `pyroute2`
|
||||||
|
dependency for TIPC management.
|
||||||
|
|
@ -0,0 +1,27 @@
|
||||||
|
---
|
||||||
|
model: openai/gpt-5.6-sol
|
||||||
|
service: opencode
|
||||||
|
timestamp: 2026-08-19T00:33:27Z
|
||||||
|
git_ref: wkt/tipc_backend_378
|
||||||
|
diff_cmd: git diff HEAD~1..HEAD
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prompt
|
||||||
|
|
||||||
|
The user asked for a bootstrap plan to try TIPC with two real,
|
||||||
|
physically separate Linux motherboards.
|
||||||
|
|
||||||
|
## Response
|
||||||
|
|
||||||
|
The generated changes turn the existing sketch into an operator
|
||||||
|
runbook and correct the RPC module authorization used by direct
|
||||||
|
script execution:
|
||||||
|
|
||||||
|
> `git diff HEAD~1..HEAD -- examples/multihost/tipc_cluster/README.md examples/multihost/tipc_cluster/host_a_srv.py`
|
||||||
|
|
||||||
|
The runbook covers matching revisions, cluster identity, interface
|
||||||
|
selection, Ethernet bearer setup, link and name-table validation,
|
||||||
|
RPC, failure/rejoin testing, diagnostics, cleanup and a future
|
||||||
|
network-namespace fixture. It also records “Cluster Domain Sockets”
|
||||||
|
as explanatory terminology and identifies a future `pyroute2` TIPC
|
||||||
|
generic-netlink codec as the path away from manual `tipc(8)` calls.
|
||||||
|
|
@ -0,0 +1,37 @@
|
||||||
|
---
|
||||||
|
model: openai/gpt-5.6-sol
|
||||||
|
service: opencode
|
||||||
|
session: a998acff-af66-4969-a65e-f77cbf5ef8dd
|
||||||
|
timestamp: 2026-08-19T00:33:28Z
|
||||||
|
git_ref: wkt/tipc_backend_378
|
||||||
|
scope: docs
|
||||||
|
substantive: true
|
||||||
|
raw_file: 20260819T003328Z_53516b09_prompt_io.raw.md
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prompt
|
||||||
|
|
||||||
|
Draft an upstream `/tipc` multiaddr issue modeled on the prior `wg`
|
||||||
|
proposal.
|
||||||
|
|
||||||
|
## Response summary
|
||||||
|
|
||||||
|
Added a reviewable upstream issue draft with a concrete binary and
|
||||||
|
text encoding, composition examples, rollout plan and open design
|
||||||
|
questions. Retained the ecosystem-standard `tipc` name while using
|
||||||
|
“Cluster Domain Sockets” as explanatory terminology, and separated
|
||||||
|
the future `pyroute2` management track from address encoding.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `ai/tpt-backends/04_tipc_multiaddr_upstream.md` - candidate
|
||||||
|
`multiformats/multiaddr` issue body.
|
||||||
|
- `ai/tpt-backends/README.md` - roster the upstream proposal.
|
||||||
|
|
||||||
|
## Human edits
|
||||||
|
|
||||||
|
The human chose to track the draft under `ai/tpt-backends/` instead
|
||||||
|
of leaving it in the ignored gish cache, and explicitly requested a
|
||||||
|
Prompt-IO record for this docs-only commit. The human proposed the
|
||||||
|
`cds` alternative and unifying TIPC/WireGuard management through
|
||||||
|
`pyroute2`.
|
||||||
|
|
@ -0,0 +1,26 @@
|
||||||
|
---
|
||||||
|
model: openai/gpt-5.6-sol
|
||||||
|
service: opencode
|
||||||
|
timestamp: 2026-08-19T00:33:28Z
|
||||||
|
git_ref: wkt/tipc_backend_378
|
||||||
|
diff_cmd: git diff HEAD~1..HEAD
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prompt
|
||||||
|
|
||||||
|
The user asked to begin drafting an upstream `/tipc` multiaddr issue
|
||||||
|
similar to the existing WireGuard proposal, then chose to track the
|
||||||
|
draft as an AI planning document.
|
||||||
|
|
||||||
|
## Response
|
||||||
|
|
||||||
|
The generated proposal is intended for `multiformats/multiaddr`:
|
||||||
|
|
||||||
|
> `git diff HEAD~1..HEAD -- ai/tpt-backends/04_tipc_multiaddr_upstream.md ai/tpt-backends/README.md`
|
||||||
|
|
||||||
|
It proposes a code allocated upstream, a fixed 72-bit value holding
|
||||||
|
the service type, instance and scope, and the canonical text form
|
||||||
|
`/tipc/<type>:<instance>:<scope>`. It includes WireGuard composition,
|
||||||
|
security boundaries, the `tipc` versus `cds` naming decision,
|
||||||
|
deployment-management separation, implementation phases, test data
|
||||||
|
and open questions for maintainers.
|
||||||
|
|
@ -0,0 +1,403 @@
|
||||||
|
# `tractor.ipc` next-gen transport backends: the shared contract
|
||||||
|
|
||||||
|
Status: design doc / implementation spec.
|
||||||
|
Audience: any model or human implementing one of the three
|
||||||
|
sibling plans in this directory.
|
||||||
|
|
||||||
|
- [`01_tipc_backend.md`](./01_tipc_backend.md) — `AF_TIPC`
|
||||||
|
(gh #378)
|
||||||
|
- [`02_quic_iroh_backend.md`](./02_quic_iroh_backend.md) — QUIC
|
||||||
|
via `iroh` FFI, uniffi-async rewritten onto `trio` (gh #353)
|
||||||
|
- [`03_wg_tunnel_bindspace.md`](./03_wg_tunnel_bindspace.md) —
|
||||||
|
WireGuard (and other shuttle-able) tunnels as a *nested
|
||||||
|
bindspace* layer via `pyroute2` (gh #482, #443)
|
||||||
|
|
||||||
|
This doc is the **normative** description of what a `tractor`
|
||||||
|
transport backend *is* as of `main@83b34884`. Each sibling plan
|
||||||
|
assumes it and only documents its own deltas. Read this first;
|
||||||
|
do not re-derive it from the code.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. Why a shared contract doc
|
||||||
|
|
||||||
|
The three plans are meant to be implementable *independently and
|
||||||
|
concurrently* by different models/providers without design
|
||||||
|
drift. Everything they share — the backend duck-type, the
|
||||||
|
registration tables, the test harness plumbing, the naming and
|
||||||
|
code-style rules — lives here exactly once. If an implementer
|
||||||
|
finds this doc disagrees with `main`, **the code wins**; fix this
|
||||||
|
doc in the same PR.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The backend duck-type (empirical, from `_tcp.py`/`_uds.py`)
|
||||||
|
|
||||||
|
A transport backend is **one module** under `tractor/ipc/`
|
||||||
|
exposing exactly four things. There is no ABC to subclass and no
|
||||||
|
plugin entrypoint; wiring is by explicit table registration
|
||||||
|
(§2) plus one piece of reflection (§1.3).
|
||||||
|
|
||||||
|
### 1.1 `class <Proto>Address(msgspec.Struct, frozen=True)`
|
||||||
|
|
||||||
|
Structurally conforms to the `Address` `Protocol` in
|
||||||
|
`tractor/discovery/_addr.py:82`. Required surface:
|
||||||
|
|
||||||
|
| member | kind | notes |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `proto_key` | `ClassVar[str]` | the wire/registry key, e.g. `'tcp'`, `'uds'` |
|
||||||
|
| `unwrapped_type` | `ClassVar[type]` | the primitive tuple shape |
|
||||||
|
| `def_bindspace` | `ClassVar` | default bindspace value |
|
||||||
|
| `is_valid` | `@property -> bool` | "is this a *dialable/bindable* addr" |
|
||||||
|
| `bindspace` | `@property` | the "set of hosts"-ish scope (see below) |
|
||||||
|
| `from_addr(cls, addr)` | `@classmethod` | primitive -> wrapped, `match`-based |
|
||||||
|
| `unwrap(self)` | method | wrapped -> primitive (must be msgpack-native!) |
|
||||||
|
| `get_random(cls, bindspace=...)` | `@classmethod` | per-subactor ephemeral addr |
|
||||||
|
| `get_root(cls)` | `@classmethod` | host-singleton default registrar addr |
|
||||||
|
| `__repr__` | method | `f'{type(self).__name__}[{...}]'` house style |
|
||||||
|
|
||||||
|
Hard constraints learned from the existing two:
|
||||||
|
|
||||||
|
- **`frozen=True`.** Addresses are dict keys
|
||||||
|
(`Server.epsdict()`, `Endpoint.peer_tpts`) and are compared by
|
||||||
|
value all over the runtime.
|
||||||
|
- **`.unwrap()` output must round-trip through `msgspec` and
|
||||||
|
through `wrap_address()`.** It is what actually crosses the
|
||||||
|
wire in `SpawnSpec`/`_root_mailbox`/`_registry_addrs`, and it
|
||||||
|
is what `Actor.reg_addrs` and every test compares against. If
|
||||||
|
your unwrapped form is not *uniquely* pattern-matchable
|
||||||
|
against the other backends' forms in
|
||||||
|
`wrap_address()` (`_addr.py:230`), you have a bug that
|
||||||
|
manifests as the wrong transport being loaded — the file's own
|
||||||
|
`XXX NOTE` warns about precisely this.
|
||||||
|
|
||||||
|
⚠️ **and shape-matching does not survive 4 backends.** Adding
|
||||||
|
TIPC and iroh breaks it outright: TIPC's natural form is a
|
||||||
|
`(str, int)` — indistinguishable from `TCPAddress` — and
|
||||||
|
iroh's is a `(str, str)`, which the *existing* UDS case
|
||||||
|
(`case (_, filename) if type(filename) is str`) already
|
||||||
|
swallows. Ordering hacks and prefix-tagging (an earlier
|
||||||
|
revision of plan 01 proposed `('tipc:<stype>:<scope>', inst)`)
|
||||||
|
paper over it at best.
|
||||||
|
|
||||||
|
**The fix, and the recommended prerequisite for all three
|
||||||
|
backends: make the unwrapped form carry an explicit
|
||||||
|
proto-key, using the `multiaddr` protocol name as the
|
||||||
|
canonical spelling** — `('tcp', host, port)`,
|
||||||
|
`('unix', path)`, `('udp', ...)`, `('tipc', stype, inst,
|
||||||
|
scope)`. Then `wrap_address()` collapses from an
|
||||||
|
order-sensitive `match` to `_address_types[addr[0]]`, and the
|
||||||
|
whole collision class stops existing. Note this *also* aligns
|
||||||
|
the on-wire form with `mk_maddr()`/`parse_maddr()`, so the two
|
||||||
|
representations stop being independent inventions.
|
||||||
|
|
||||||
|
Two consequences to plan for:
|
||||||
|
- it's a **wire-format change** (`SpawnSpec`,
|
||||||
|
`_root_mailbox`, `_registry_addrs`) plus every test fixture
|
||||||
|
and downstream config (`piker`'s `[network]` table). It
|
||||||
|
wants its **own migration commit, landed before any new
|
||||||
|
backend**, not smuggled into one.
|
||||||
|
- it's the moment to **stop handing raw unwrapped tuples to
|
||||||
|
users at all.** The long-term shape is: `Address` subtypes
|
||||||
|
are the public currency and `UnwrappedAddress` becomes an
|
||||||
|
internal serialization detail — the same discipline
|
||||||
|
`ipaddress` uses (you pass `IPv4Address`, not a 4-tuple).
|
||||||
|
Public API should accept `Address|maddr-str` and treat bare
|
||||||
|
tuples as legacy-tolerated input, ideally deprecated.
|
||||||
|
- **`.get_random()` must be collision-free without a live
|
||||||
|
runtime.** See the `UDSAddress.get_random()` uuid-token
|
||||||
|
comment (`_uds.py:207-220`): with no `current_actor()` the
|
||||||
|
sockname degenerates to a pure fn of `(prefix, pid)` and two
|
||||||
|
calls in one proc alias. Mix in a `uuid4().hex[:8]` token.
|
||||||
|
- **`.bindspace` semantics**: "the address' bindable space" —
|
||||||
|
ip/host for `tcp`, the socket-file *directory* for `uds`. For
|
||||||
|
the new backends: the TIPC *scope* (§1 of plan 01), the iroh
|
||||||
|
*ALPN + relay/discovery realm* (plan 02), the netns (plan 03).
|
||||||
|
`Address.namespace` is already spec'd in the Protocol as
|
||||||
|
"the if-available OS-specific network namespace key" and is
|
||||||
|
currently unimplemented by both backends — plan 03 is the
|
||||||
|
first real consumer.
|
||||||
|
|
||||||
|
### 1.2 module-level listener lifecycle
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def start_listener(
|
||||||
|
addr: <Proto>Address,
|
||||||
|
**kwargs,
|
||||||
|
) -> trio.SocketListener # or a trio.abc.Listener, see §3
|
||||||
|
...
|
||||||
|
|
||||||
|
def close_listener( # OPTIONAL
|
||||||
|
addr: <Proto>Address,
|
||||||
|
lstnr: trio.abc.Listener,
|
||||||
|
) -> None:
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
`close_listener()` is optional; `Endpoint.close_listener()`
|
||||||
|
(`_server.py:674`) `getattr`s it and treats absence as "closing
|
||||||
|
is implicit". `uds` needs it (unlinks the sock-file), `tcp`
|
||||||
|
does not.
|
||||||
|
|
||||||
|
### 1.3 the ONE piece of reflection you must not break
|
||||||
|
|
||||||
|
`Endpoint.start_listener()` (`_server.py:656`):
|
||||||
|
|
||||||
|
```python
|
||||||
|
tpt_mod: ModuleType = inspect.getmodule(self.addr)
|
||||||
|
lstnr = await tpt_mod.start_listener(addr=self.addr)
|
||||||
|
```
|
||||||
|
|
||||||
|
The transport module is found by `inspect.getmodule()` **on the
|
||||||
|
`Address` instance**. Therefore: *the `Address` class and its
|
||||||
|
`start_listener()`/`close_listener()` MUST live in the same
|
||||||
|
module.* Do not define the address type in `_types.py` or a
|
||||||
|
`_addrs.py` and the listener elsewhere.
|
||||||
|
|
||||||
|
Immediately after, the same method does:
|
||||||
|
|
||||||
|
```python
|
||||||
|
if (unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap():
|
||||||
|
self.addr = self.addr.from_addr(unwrapped)
|
||||||
|
```
|
||||||
|
|
||||||
|
i.e. it assumes `lstnr.socket.getsockname()` exists and that its
|
||||||
|
return value is a valid `from_addr()` input. This is fine for
|
||||||
|
TIPC (§3 of plan 01) and **is the main integration hazard for
|
||||||
|
iroh** (§3 of plan 02) — plans that break it must say so
|
||||||
|
explicitly and propose the upstream `_server.py` patch.
|
||||||
|
|
||||||
|
### 1.4 `class Msgpack<Proto>Stream(MsgpackTransport)`
|
||||||
|
|
||||||
|
Subclass `tractor.ipc._transport.MsgpackTransport`. You inherit
|
||||||
|
all framing (`<I` 4-byte little-endian length prefix),
|
||||||
|
`msgspec` codec ctx-var lookup, `TransportClosed` normalization,
|
||||||
|
`.drain()`, `__aiter__`. You implement only:
|
||||||
|
|
||||||
|
| member | notes |
|
||||||
|
| --- | --- |
|
||||||
|
| `address_type` | the `<Proto>Address` class |
|
||||||
|
| `layer_key: int` | OSI-ish layer, `4` for both current backends |
|
||||||
|
| `maddr` `@property` | `-> Multiaddr\|str`, via `mk_maddr(self.raddr)` |
|
||||||
|
| `connected(self) -> bool` | `tcp`/`uds` both use `self.stream.socket.fileno() != -1` |
|
||||||
|
| `connect_to(cls, addr, prefix_size=4, codec=None, **kw)` | `@classmethod`, returns an instance |
|
||||||
|
| `get_stream_addrs(cls, stream) -> (laddr, raddr)` | `@classmethod`, called from `MsgpackTransport.__init__` |
|
||||||
|
|
||||||
|
`MsgpackTransport.__init__` requires the object passed as
|
||||||
|
`stream` to satisfy:
|
||||||
|
|
||||||
|
- `await stream.send_all(bytes)`
|
||||||
|
- usable as `tricycle.BufferedReceiveStream(transport_stream=stream)`,
|
||||||
|
i.e. `await stream.receive_some(n)`
|
||||||
|
- `trio.BrokenResourceError` / `trio.ClosedResourceError` /
|
||||||
|
`ValueError('...unclean EOF...')` on the failure paths that
|
||||||
|
`_iter_packets()` and `send()` already `match` on
|
||||||
|
(`_transport.py:221-304`, `:436-499`).
|
||||||
|
|
||||||
|
That is **`trio.abc.Stream`, not `trio.SocketStream`**. The
|
||||||
|
`MsgTransport` Protocol's `stream: trio.SocketStream`
|
||||||
|
annotation (`_transport.py:83`) is a lie of convenience — the
|
||||||
|
actual `MsgpackTransport.__init__` param is typed
|
||||||
|
`trio.abc.Stream` and nothing in the msg path touches
|
||||||
|
`.socket`. Only `connected()` (which each backend defines) and
|
||||||
|
`Endpoint.start_listener()`'s `getsockname()` do.
|
||||||
|
|
||||||
|
### 1.5 verified-good news for socket-family backends
|
||||||
|
|
||||||
|
Both `trio.SocketStream` and `trio.SocketListener` are
|
||||||
|
**address-family agnostic**. Verified against the installed
|
||||||
|
`trio` (`trio/_highlevel_socket.py`): the only constructor
|
||||||
|
checks are
|
||||||
|
|
||||||
|
- `isinstance(socket, trio.socket.SocketType)`
|
||||||
|
- `socket.type == SOCK_STREAM`
|
||||||
|
- (listener) `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)` is truthy,
|
||||||
|
with `OSError` **suppressed** (the macOS carve-out, which
|
||||||
|
also covers exotic families that reject the opt)
|
||||||
|
|
||||||
|
There is no `AF_*` check and no `IPPROTO_TCP` hard dependency
|
||||||
|
(`TCP_NODELAY`/`TCP_NOTSENT_LOWAT` are set under
|
||||||
|
`suppress(OSError)`). Consequence: **any `SOCK_STREAM` family
|
||||||
|
CPython can create — including `AF_TIPC` — drops straight into
|
||||||
|
the existing `trio.SocketStream` + `trio.serve_listeners()`
|
||||||
|
path.** This is why plan 01 is small and plan 02 is not.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Registration tables (the full wiring checklist)
|
||||||
|
|
||||||
|
Adding a backend touches these and only these:
|
||||||
|
|
||||||
|
1. `tractor/runtime/_state.py:46`
|
||||||
|
`TransportProtocolKey = Literal['tcp', 'uds', ...]` — add the
|
||||||
|
key. This `Literal` is the canonical set; `_testing/pytest.py`
|
||||||
|
drives `--tpt-proto` validation off `_addr._address_types`,
|
||||||
|
and the spawn-backend fixture already models the
|
||||||
|
"drive-the-set-from-the-Literal" pattern
|
||||||
|
(`pytest.py:870-880`) — do the same rather than hardcoding.
|
||||||
|
2. `tractor/discovery/_addr.py:173` `_address_types: bidict` —
|
||||||
|
`{'<key>': <Proto>Address}`. Note it is a **`bidict`**, so
|
||||||
|
the mapping must stay 1:1.
|
||||||
|
3. `tractor/discovery/_addr.py:181` `_default_lo_addrs` —
|
||||||
|
`'<key>': <Proto>Address.get_root().unwrap()`.
|
||||||
|
⚠️ this dict is built at **import time**, so
|
||||||
|
`get_root()` must not require a live runtime, a loaded kernel
|
||||||
|
module, or network I/O. (`UDSAddress.def_bindspace =
|
||||||
|
get_rt_dir()` is the precedent for "cheap, pure, filesystem-
|
||||||
|
ish".) A backend whose root addr needs I/O must make this
|
||||||
|
entry lazy — propose that refactor explicitly.
|
||||||
|
4. `tractor/discovery/_addr.py:230` `wrap_address()` `match` —
|
||||||
|
add a case iff your `unwrapped_type` isn't already uniquely
|
||||||
|
matched. **Preferably do the proto-key migration in §1.1
|
||||||
|
first**, after which this step becomes a one-line
|
||||||
|
`_address_types` entry instead of an order-sensitive `case`.
|
||||||
|
5. `tractor/ipc/_types.py` — `Address` union alias,
|
||||||
|
`_msg_transports` list, `_key_to_transport[('msgpack', key)]`,
|
||||||
|
`_addr_to_transport[<Proto>Address]`.
|
||||||
|
6. `tractor/ipc/_types.py:92` `transport_from_stream()` — the
|
||||||
|
`sock.family` `match`. For a non-socket stream type (iroh)
|
||||||
|
this needs a different discriminator; see plan 02 §3.3.
|
||||||
|
7. `tractor/discovery/_multiaddr.py` —
|
||||||
|
`_tpt_proto_to_maddr`, and a `case` in both `mk_maddr()` and
|
||||||
|
`parse_maddr()`.
|
||||||
|
8. `tractor/ipc/__init__.py` — re-export if the backend has a
|
||||||
|
public surface.
|
||||||
|
9. `tractor/_testing/addr.py::get_rando_addr()` — per-proto
|
||||||
|
branch so the whole suite can run under `--tpt-proto <key>`.
|
||||||
|
10. `pyproject.toml` — new deps go in an **optional extra**, never
|
||||||
|
in `[project].dependencies`. See §5.
|
||||||
|
|
||||||
|
## 3. Where the `trio.SocketListener` assumption is load-bearing
|
||||||
|
|
||||||
|
`_serve_ipc_eps()` (`_server.py:1041`) annotates
|
||||||
|
`listener: trio.abc.Listener` and hands the list to
|
||||||
|
`trio.serve_listeners(handler=handle_stream_from_peer,
|
||||||
|
listeners=..., handler_nursery=stream_handler_tn)`.
|
||||||
|
`trio.serve_listeners` itself is generic over
|
||||||
|
`trio.abc.Listener`. So the *only* `SocketListener`-specific
|
||||||
|
code in the server path is the `getsockname()` reconciliation in
|
||||||
|
`Endpoint.start_listener()` (§1.3) and the type annotations.
|
||||||
|
|
||||||
|
`handle_stream_from_peer()` (`_server.py:298`) then does
|
||||||
|
`Channel.from_stream(stream)` →
|
||||||
|
`transport_from_stream(stream)` → `sock.family` match (§2.6).
|
||||||
|
|
||||||
|
**Therefore**: a non-socket backend needs (a) a
|
||||||
|
`trio.abc.Listener` subclass, (b) a change to
|
||||||
|
`Endpoint.start_listener()` to not blindly `getsockname()`, and
|
||||||
|
(c) a change to `transport_from_stream()`'s discrimination.
|
||||||
|
All three are small, upstream-able, and *should be landed as
|
||||||
|
their own prep PR* before the backend itself — see plan 02 §3.
|
||||||
|
|
||||||
|
## 4. Handshake / discovery invariants you inherit
|
||||||
|
|
||||||
|
- Every accepted stream immediately does
|
||||||
|
`chan._do_handshake(aid=actor.aid)`; a peer that fails it is
|
||||||
|
logged at `runtime` and dropped, **not** raised
|
||||||
|
(`_server.py:334-365`). Discovery-sys "pings" rely on this,
|
||||||
|
so your `connect_to()` must raise something that normalizes
|
||||||
|
to `TransportClosed`/`ConnectionError` on a dead peer, never
|
||||||
|
a novel exception type.
|
||||||
|
- `_root.py:381-406` fail-fasts when a `registry_addrs` entry's
|
||||||
|
`proto_key` is not in `enable_transports`. Your key must be
|
||||||
|
spellable in both.
|
||||||
|
- `_root.py:256` currently enforces `len(enable_transports) == 1`.
|
||||||
|
Multi-tpt actors are a separate work item; none of these three
|
||||||
|
plans may depend on lifting it.
|
||||||
|
- Sub-actor bind addrs come from
|
||||||
|
`_runtime.py:1600-1610`: for each key in the parent-supplied
|
||||||
|
`enable_transports`, `get_address_cls(key).get_random()`.
|
||||||
|
So `get_random()` runs *in the child, post-fork, pre-listen*.
|
||||||
|
Anything it needs (kernel module, netns membership, an iroh
|
||||||
|
secret key) must already be true at that moment.
|
||||||
|
|
||||||
|
## 5. Dependency policy
|
||||||
|
|
||||||
|
`[project].dependencies` stays lean (see the boot-latency work,
|
||||||
|
gh #470: `import tractor` is budgeted at ~0.145s). Every new
|
||||||
|
backend dep is an extra:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[project.optional-dependencies]
|
||||||
|
tipc = [] # stdlib-only!
|
||||||
|
quic = ["iroh>=0.35"] # pin per plan 02 §1
|
||||||
|
wg = ["pyroute2>=0.9"] # pin per plan 03 §1
|
||||||
|
```
|
||||||
|
|
||||||
|
and every backend module must be **import-lazy**: a
|
||||||
|
`tractor/ipc/_<proto>.py` that imports its 3rd-party dep at
|
||||||
|
module scope must not be imported by `tractor/__init__.py`,
|
||||||
|
`tractor/ipc/__init__.py`, or `tractor/discovery/_addr.py`'s
|
||||||
|
import-time table construction. The `_addr._default_lo_addrs`
|
||||||
|
eager-dict (§2.3) is the trap: keep the backend's `get_root()`
|
||||||
|
dep-free, or make that table lazy.
|
||||||
|
|
||||||
|
## 6. Test-harness plumbing (identical for all three)
|
||||||
|
|
||||||
|
- `--tpt-proto <key>` (`_testing/pytest.py:409`) selects the
|
||||||
|
session-wide proto; the `tpt_proto` fixture mutates
|
||||||
|
`_state._def_tpt_proto` + `_runtime_vars['_enable_tpts']`
|
||||||
|
(`pytest.py:807-835`). Adding the key to `_address_types` is
|
||||||
|
what makes `--tpt-proto <key>` legal (`pytest.py:795-800`
|
||||||
|
asserts the lookup).
|
||||||
|
- The **acceptance bar** for every backend is: the *entire*
|
||||||
|
existing suite passes under `--tpt-proto <key>`, unmodified.
|
||||||
|
That is the whole point of the abstraction. Backend-specific
|
||||||
|
unit tests go in `tests/ipc/test_each_tpt.py` (the existing
|
||||||
|
`test_uds_bindspace_created_implicitly` /
|
||||||
|
`test_uds_double_listen_raises_connerr` are the model).
|
||||||
|
- Capability gating: each backend needs a **cheap, pure
|
||||||
|
predicate** + a `pytest.mark.skipif`, because these are all
|
||||||
|
environment-dependent. Verified example: on this dev box
|
||||||
|
`socket.socket(AF_TIPC, SOCK_STREAM)` raises
|
||||||
|
`OSError(97, 'Address family not supported by protocol')`
|
||||||
|
because the `tipc` module isn't loaded. Put the predicate in
|
||||||
|
the backend module (so apps can use it too), not in the test.
|
||||||
|
- New pytest marks must be registered in `pyproject.toml`, per
|
||||||
|
the project's fix-warnings-at-source rule (gh #469).
|
||||||
|
|
||||||
|
## 7. Code style (non-negotiable, matches the repo)
|
||||||
|
|
||||||
|
- module header tagline: `# tractor: distributed structured
|
||||||
|
concurrency.` for **new** files (not the legacy
|
||||||
|
`structured concurrent "actors".` form the existing `_tcp.py`
|
||||||
|
carries).
|
||||||
|
- AGPL header block copied verbatim from `_tcp.py`.
|
||||||
|
- `from __future__ import annotations` first.
|
||||||
|
- annotate *everything*, including locals:
|
||||||
|
`sockpath: Path = addr.sockpath`.
|
||||||
|
- `match`/`case` over `isinstance` chains for address and
|
||||||
|
error dispatch.
|
||||||
|
- multi-line call/`import` style with trailing commas.
|
||||||
|
- never emit a whitespace-only line.
|
||||||
|
- error messages are multi-line f-strings ending in `\n`, with
|
||||||
|
the `f'...\n' f'...\n'` implicit-concat layout and the
|
||||||
|
`>[`/`[>`/`<=(` nested-op sigils where a `nest_from_op()` is
|
||||||
|
in play.
|
||||||
|
- prefer pure functions + module-level helpers over methods;
|
||||||
|
keep `Address` types data-only. Where a helper needs
|
||||||
|
scoped setup/teardown, it's an `@acm` — not a class with
|
||||||
|
`.start()`/`.stop()`.
|
||||||
|
- pure getters: no `get_*(..., mutate=True)` flags; split into
|
||||||
|
a read-only getter and an explicit sibling setter.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Cross-plan sequencing
|
||||||
|
|
||||||
|
The three are independent *except*:
|
||||||
|
|
||||||
|
- plan 02 (iroh) needs the `_server.py` /
|
||||||
|
`transport_from_stream()` generalization (§3) — plan 01 does
|
||||||
|
**not**, and should therefore land first as the cheap proof
|
||||||
|
that the table-registration story works for a genuinely new
|
||||||
|
proto.
|
||||||
|
- plan 03 (wg) composes *under* whatever L4 tpt is in use and
|
||||||
|
its netns work is what finally implements
|
||||||
|
`Address.namespace`. It can land before or after 02, but its
|
||||||
|
`TunnelledAddress` design must be reviewed against plan 02's
|
||||||
|
address shape so the "tunnelled maddr" grammar (gh #443)
|
||||||
|
covers `/…/quic-v1/p2p/…` inner addrs too.
|
||||||
|
- All three want first-class `wg`/`quic`/`tipc` protos in
|
||||||
|
`py-multiaddr`; that upstream track is gh #483 and
|
||||||
|
multiformats/py-multiaddr#107/#108.
|
||||||
|
|
@ -0,0 +1,319 @@
|
||||||
|
# TIPC backend — handoff
|
||||||
|
|
||||||
|
Status: **PR [#493] is feature-complete and green**; what remains
|
||||||
|
is landing logistics plus a named follow-up track.
|
||||||
|
|
||||||
|
Takeover snapshot (2026-08-17): [#493] remains a draft at
|
||||||
|
`c7ae6065`, targeting `ng_tpts_planning` ([#492]). Its branch
|
||||||
|
point is `ee17ed9f`; #492 has since advanced by two commits to
|
||||||
|
`d9a6e2e9`, so #493 still needs rebasing onto that current
|
||||||
|
[#492] head before final landing work. The latest `tipc` CI leg
|
||||||
|
passed; the workflow as a whole is red only because the macOS
|
||||||
|
`tcp` leg failed.
|
||||||
|
|
||||||
|
Audience: any agent or human picking this up cold, from any
|
||||||
|
provider. Nothing here assumes a particular harness or tooling.
|
||||||
|
|
||||||
|
Read in this order,
|
||||||
|
|
||||||
|
1. this file (orientation + what's already settled)
|
||||||
|
2. [`00_shared_backend_contract.md`](./00_shared_backend_contract.md)
|
||||||
|
— **normative** description of what a `tractor` transport
|
||||||
|
backend *is*
|
||||||
|
3. [`01_tipc_backend.md`](./01_tipc_backend.md) — the plan,
|
||||||
|
already reconciled against the live-kernel findings
|
||||||
|
|
||||||
|
Do **not** re-derive the design or re-select libraries. Where
|
||||||
|
this doc and the code disagree, **the code wins** — fix the doc
|
||||||
|
in the same change (contract §0).
|
||||||
|
|
||||||
|
[#493]: https://github.com/goodboy/tractor/pull/493
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Environment
|
||||||
|
|
||||||
|
The backend needs a linux kernel module that is **not loaded by
|
||||||
|
default**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo modprobe tipc
|
||||||
|
tipc node get address # confirms the module is live
|
||||||
|
```
|
||||||
|
|
||||||
|
Everything else is stdlib — TIPC adds **zero** dependencies.
|
||||||
|
|
||||||
|
This repo is a git worktree with a `uv`-managed venv at
|
||||||
|
`./py313`. Run things through it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./py313/bin/python -m pytest tests/ipc -q
|
||||||
|
./py313/bin/python -m pytest tests/ -q --tpt-proto tipc
|
||||||
|
```
|
||||||
|
|
||||||
|
To rebuild docs you need the docs dep-group, which *mutates*
|
||||||
|
that venv — `uv sync` afterwards to restore it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
UV_PROJECT_ENVIRONMENT=py313 uv run --group docs \
|
||||||
|
python -m sphinx -b html docs /tmp/docbuild
|
||||||
|
UV_PROJECT_ENVIRONMENT=py313 uv sync
|
||||||
|
```
|
||||||
|
|
||||||
|
Without the module, `--tpt-proto tipc` fails loudly and
|
||||||
|
immediately (by design); `tests/ipc/test_tipc.py`'s
|
||||||
|
kernel-touching cases self-skip.
|
||||||
|
|
||||||
|
## 2. What the backend is, in three sentences
|
||||||
|
|
||||||
|
An actor's TIPC address is a **service name** `(stype, instance)`
|
||||||
|
— no host, no port. Binding the singleton `TIPC_ADDR_NAMESEQ`
|
||||||
|
range *publishes* it into a kernel-maintained cluster-wide name
|
||||||
|
table (visible via `tipc nametable show`), and a peer's
|
||||||
|
`.connect()`-by-name *is* the discovery lookup, resolved
|
||||||
|
in-kernel. `TIPC_ADDR_ID` port-ids are only ever **observed**,
|
||||||
|
never user-facing.
|
||||||
|
|
||||||
|
Everything lives in `tractor/ipc/_tipc.py`.
|
||||||
|
|
||||||
|
## 3. Hard-won facts — do not re-litigate these
|
||||||
|
|
||||||
|
All verified against a live kernel. Several contradict what the
|
||||||
|
plan originally assumed.
|
||||||
|
|
||||||
|
| fact | why it matters |
|
||||||
|
| --- | --- |
|
||||||
|
| A duplicate name bind **succeeds**, and connects **round-robin** between publishers | An instance collision is *silent crosstalk*, never `EADDRINUSE`. Hence the `blake2b` instance digest. |
|
||||||
|
| Dialing an unpublished name gives `EHOSTUNREACH` **instantly**, as a **bare `OSError`** — not a `ConnectionError` subtype | `_reraise_as_connerr()` is REQUIRED by contract §4, not polish |
|
||||||
|
| `getsockname()` always answers a `TIPC_ADDR_ID` port-id, even pre-bind | why `TIPCAddress.rebind_from_sockname = False` |
|
||||||
|
| A connect-then-drop peer makes `getpeername()` raise `ENOTCONN` | Unguarded, this **kills the whole actor** — `.get_stream_addrs()` runs *before* the handshake, so it escapes handshake tolerance. See `_maybe_sockaddr()`. |
|
||||||
|
| `SO_ACCEPTCONN` works fine (answers `1`) | trio's `except OSError` carve-out is not load-bearing here |
|
||||||
|
| Graceful peer close arrives as `BrokenResourceError`/`ECONNRESET`, not a clean 0-byte EOF | benign; `_iter_packets()` already classifies it as a normal disconnect |
|
||||||
|
| Topology `struct tipc_event` is **48 bytes** (`4+4+4+8+28`) | the plan said 40 |
|
||||||
|
| The topology server **accepts native `'='` byte-order** | the plan's proposed `'>'`-retry endianness probe was deleted as unnecessary |
|
||||||
|
| `TIPC_WAIT_FOREVER` is `-1` in python | must be masked (`& 0xFFFFFFFF`) before packing as unsigned |
|
||||||
|
| TIPC **has** AES-GCM encryption (`tipc node set key`, linux 5.9+) | cluster/master/per-node keys + rekeying. So "wg adds the encryption TIPC lacks" is **false** — see §6 for the real motivation. |
|
||||||
|
| A wg interface is L3/`tun` (`POINTOPOINT,NOARP`, `link/none`) | TIPC's `eth` media **cannot** bind it — udp media is *mandatory* over wg. See the §6 caveat; this one bites. |
|
||||||
|
|
||||||
|
Two design decisions that are **closed**, with reasons:
|
||||||
|
|
||||||
|
- **The accepting side does not learn the peer's service name.**
|
||||||
|
A port-id can't be reversed into one. It gets a
|
||||||
|
`TIPC_NAME_UNKNOWN` sentinel plus the observed `(node, ref)`,
|
||||||
|
and that's fine — the `Aid` from the handshake already carries
|
||||||
|
the peer's logical identity. (`uds` has the same wart.)
|
||||||
|
- **Do not fold digest bits into the service `_stype` to widen
|
||||||
|
the collision space.** A topology subscription can only watch
|
||||||
|
**one** `stype`, so varying it per-actor would need 65536
|
||||||
|
subscriptions and kills the push-registry work outright. If
|
||||||
|
crosstalk ever bites, the escalation is a post-bind
|
||||||
|
verification handshake ([#501]).
|
||||||
|
|
||||||
|
[#501]: https://github.com/goodboy/tractor/issues/501
|
||||||
|
|
||||||
|
## 4. What landed
|
||||||
|
|
||||||
|
All §6 steps 1–7 landed on `wkt/tipc_backend_378`, based on
|
||||||
|
`ng_tpts_planning` (PR [#492], docs-only). The completed arc is
|
||||||
|
17 substantive commits from `22ef362d` through `c7ae6065`, plus
|
||||||
|
the incidental `1802641e` local-cache ignore commit. The plan's
|
||||||
|
§6 status text is the historical snapshot at `7e20585f`; its
|
||||||
|
claim that steps 6–7 remain is no longer current.
|
||||||
|
|
||||||
|
- `TIPCAddress` + `is_tipc_available()` + `start_listener()`
|
||||||
|
- `MsgpackTIPCStream` (`connect_to()`, `get_stream_addrs()`)
|
||||||
|
- `open_topology_events()` — the `TIPC_TOP_SRV` push feed
|
||||||
|
- registration across every table in contract §2
|
||||||
|
- interim `str`-only `/tipc/…` maddr grammar
|
||||||
|
- a `--tpt-proto=tipc` CI leg, **non-blocking** for now
|
||||||
|
- `docs/guide/tipc.rst` + `examples/multihost/tipc_cluster/`
|
||||||
|
|
||||||
|
Two fixes fell out that are **not** TIPC-specific:
|
||||||
|
|
||||||
|
- `devx/pformat.py` — `pformat_caller_frame()` passed an
|
||||||
|
`indent=''` kwarg `pformat_boxed_tb()` never accepted, so every
|
||||||
|
send-side `MsgTypeError` died with a `TypeError` while
|
||||||
|
formatting itself. On `main` and every branch since
|
||||||
|
`888af602`. **Wants cherry-picking out of this stack.**
|
||||||
|
- `SpawnSpec.reg_addrs`/`.bind_addrs` pinned the wire shape to a
|
||||||
|
2-tuple. Widened to `UnwrappedAddress`, which had to become
|
||||||
|
**variadic** (`tuple[str|int, ...]`) because `msgspec` refuses
|
||||||
|
a union holding more than one array-like type.
|
||||||
|
|
||||||
|
**Acceptance bar met**: 122 passed / 1 xfailed / 2 xpassed under
|
||||||
|
`--tpt-proto tipc` across `ipc`, `discovery`, `runtime`,
|
||||||
|
`spawning`, `local`, `rpc`, `cancellation`. `tcp`/`uds`
|
||||||
|
unchanged.
|
||||||
|
|
||||||
|
[#492]: https://github.com/goodboy/tractor/pull/492
|
||||||
|
|
||||||
|
## 5. Immediate next steps (pre-land)
|
||||||
|
|
||||||
|
These live on [#493]'s body as `### TODOs before landing`. They
|
||||||
|
are **not** mirrored into an issue — if the PR is ever superseded
|
||||||
|
they need re-homing.
|
||||||
|
|
||||||
|
1. **Cherry-pick the `pformat` fix onto `main`** as its own PR,
|
||||||
|
and land it before [#493] —
|
||||||
|
`22ef362d` (red guard test) then `f9f98eeb` (the 1-line fix).
|
||||||
|
Preserve that order. The pair is unrelated to TIPC, every
|
||||||
|
branch has the bug, and this fix must not disappear if #493
|
||||||
|
is superseded.
|
||||||
|
2. **Watch the `tipc` CI leg.** It's gated
|
||||||
|
`continue-on-error: ${{ matrix.tpt_proto == 'tipc' }}` because
|
||||||
|
GH's runners have never been asked to `modprobe` for us. Once
|
||||||
|
it has a few green runs, drop the gate. If the runners refuse
|
||||||
|
the `modprobe`, fall back to a container job with
|
||||||
|
`--cap-add NET_ADMIN`.
|
||||||
|
3. **Rebase #493 onto #492's current head first.** At this
|
||||||
|
snapshot that means moving from the `ee17ed9f` branch point
|
||||||
|
onto `d9a6e2e9`. #492 is checked out in another worktree, so
|
||||||
|
refresh its authoritative head and coordinate before changing
|
||||||
|
history. Once #492 merges, rebase #493 onto `main` for final
|
||||||
|
landing.
|
||||||
|
|
||||||
|
## 6. The follow-up track
|
||||||
|
|
||||||
|
All filed with the `follow-up` label.
|
||||||
|
|
||||||
|
| issue | what |
|
||||||
|
| --- | --- |
|
||||||
|
| [#495] | `TIPC_IMPORTANCE` supervision QoS on the parent↔child chan |
|
||||||
|
| [#496] | `TIPC_TOP_SRV` push registry in `discovery._registry` |
|
||||||
|
| [#497] | dual-link resiliency / multi-homing |
|
||||||
|
| [#498] | `/tipc` multiaddr spec submission |
|
||||||
|
| [#499] | registrar-less discovery via name derivation |
|
||||||
|
| [#500] | multicast/group msging as a *broadcast* transport |
|
||||||
|
| [#501] | post-bind verification for instance collisions |
|
||||||
|
| [#502] | **TIPC over a `wg` mesh — the reference multihost deployment** |
|
||||||
|
|
||||||
|
### the wg direction
|
||||||
|
|
||||||
|
[#502] is the strategic one. The intent is that TIPC-over-`wg`
|
||||||
|
becomes our go-to multihost transport deployment.
|
||||||
|
|
||||||
|
> ⚠️ **Do not repeat the claim that wg adds encryption TIPC
|
||||||
|
> lacks.** We assumed that initially and it is **wrong**. TIPC
|
||||||
|
> ships AES-GCM crypto of its own (`tipc node set key`, linux
|
||||||
|
> 5.9+) with cluster/master/per-node keys and rekeying.
|
||||||
|
>
|
||||||
|
> The motivation is different but still real:
|
||||||
|
> - **key management** — TIPC keys are symmetric and
|
||||||
|
> *pre-shared*; distribution, rotation and revocation are the
|
||||||
|
> operator's problem. wg gives public-key identity + handshake.
|
||||||
|
> - **uniformity** — wg is an overlay *every* backend can sit on
|
||||||
|
> (tcp now, quic later), not a TIPC-only mechanism.
|
||||||
|
> - **NAT traversal / roaming**, which raw TIPC bearers have no
|
||||||
|
> story for.
|
||||||
|
>
|
||||||
|
> Which to default to should be **benchmarked**, not assumed:
|
||||||
|
> TIPC-native crypto avoids a tunnel hop and may win on latency
|
||||||
|
> for LAN-local clusters.
|
||||||
|
|
||||||
|
> ⚠️ **`udp` media is MANDATORY over wg — `eth` cannot work.**
|
||||||
|
> A wg interface is L3/`tun`: `POINTOPOINT,NOARP`, `link/none`,
|
||||||
|
> no L2 address at all. There is no device for `tipc bearer
|
||||||
|
> enable media eth device …` to name.
|
||||||
|
>
|
||||||
|
> ```bash
|
||||||
|
> # impossible over wg
|
||||||
|
> sudo tipc bearer enable media eth device wg0
|
||||||
|
> # required, bound to the wg overlay IP
|
||||||
|
> sudo tipc bearer enable media udp name wgmesh localip 10.0.11.1
|
||||||
|
> ```
|
||||||
|
>
|
||||||
|
> Consequence worth internalizing: #378's "ethernet bearers pair
|
||||||
|
> most excellently with wireguard tunnelling" framing does **not**
|
||||||
|
> hold — on a given link the low-latency L2 path and the wg path
|
||||||
|
> are *mutually exclusive*. Any design that assumes both is
|
||||||
|
> broken from the start.
|
||||||
|
>
|
||||||
|
> Also mind the MTU: wg links are typically 1420, under
|
||||||
|
> ethernet's 1500, so TIPC link MTU wants checking not assuming.
|
||||||
|
|
||||||
|
Composed addresses take the form:
|
||||||
|
|
||||||
|
```
|
||||||
|
/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>
|
||||||
|
\____ wg bearer ________/\_key_/\______ tractor ep ________/
|
||||||
|
```
|
||||||
|
|
||||||
|
Note the structural point, which matters for the spec proposal
|
||||||
|
in [#498]: the tcp equivalent repeats an `/ip4/…/tcp/…` inner
|
||||||
|
segment because a tcp endpoint is *located*. The tipc inner
|
||||||
|
segment has **no locative component at all** — a service name is
|
||||||
|
location-independent by design. So in the composed form the wg
|
||||||
|
segments carry all the routing and the tipc segment carries pure
|
||||||
|
*identity*.
|
||||||
|
|
||||||
|
Prerequisites already established:
|
||||||
|
|
||||||
|
- py-multiaddr [#108] (merged) proved the composed `wg` + tcp
|
||||||
|
form parses and round-trips (`['ip4','udp','wg','ip4','tcp']`)
|
||||||
|
- `examples/multihost/wg_lan/` is the existing wg example set to
|
||||||
|
generalize from rather than duplicate
|
||||||
|
- the udp-bearer-only constraint and MTU caveat are documented in
|
||||||
|
both `docs/guide/tipc.rst` and the `tipc_cluster` README
|
||||||
|
|
||||||
|
[#495]: https://github.com/goodboy/tractor/issues/495
|
||||||
|
[#496]: https://github.com/goodboy/tractor/issues/496
|
||||||
|
[#497]: https://github.com/goodboy/tractor/issues/497
|
||||||
|
[#498]: https://github.com/goodboy/tractor/issues/498
|
||||||
|
[#499]: https://github.com/goodboy/tractor/issues/499
|
||||||
|
[#500]: https://github.com/goodboy/tractor/issues/500
|
||||||
|
[#502]: https://github.com/goodboy/tractor/issues/502
|
||||||
|
[#108]: https://github.com/multiformats/py-multiaddr/pull/108
|
||||||
|
|
||||||
|
## 7. Working conventions in this repo
|
||||||
|
|
||||||
|
Provider-neutral, but they *are* enforced by review:
|
||||||
|
|
||||||
|
- **Never commit, push, rebase or amend on your own.** Prepare
|
||||||
|
changes, report them, and let the maintainer stage. Asking
|
||||||
|
"should we commit?" is a question *for you to answer*, not
|
||||||
|
permission to act.
|
||||||
|
- **Do not re-ask for an exact forge write already authorized in
|
||||||
|
the current request.** Use the provider adapter's snapshot,
|
||||||
|
digest and drift checks, perform the named edit, then report
|
||||||
|
what was published. This does not authorize unrelated or
|
||||||
|
destructive forge actions.
|
||||||
|
- **A failing/guard test lands in its own commit before the fix
|
||||||
|
it guards.** Red first, then green.
|
||||||
|
- **One commit per logical step**, so history shows *why*. Never
|
||||||
|
squash unrelated changes.
|
||||||
|
- Commit subjects: present-tense verb, ~50 chars (hard max 67),
|
||||||
|
backticks around every code element. Bodies wrap at 67 cols.
|
||||||
|
- **Never write a line containing only whitespace.**
|
||||||
|
- Annotate everything including locals; prefer `match`/`case`
|
||||||
|
over `isinstance` chains; multi-line calls with trailing
|
||||||
|
commas; `@acm` over classes with `.start()`/`.stop()`.
|
||||||
|
- New modules get the `# tractor: distributed structured
|
||||||
|
concurrency.` header tagline plus the AGPL block.
|
||||||
|
- **Do not change task/checkbox state** in issues, plans or
|
||||||
|
trackers unless explicitly asked for that exact transition.
|
||||||
|
- Fix warnings at source; only genuinely-unfixable ones get
|
||||||
|
filtered, with a documented reason.
|
||||||
|
|
||||||
|
## 8. Where things are
|
||||||
|
|
||||||
|
```
|
||||||
|
tractor/ipc/_tipc.py the whole backend
|
||||||
|
tests/ipc/test_tipc.py 28 backend unit tests
|
||||||
|
tests/ipc/test_server.py the reconciliation guard
|
||||||
|
tests/devx/test_pformat.py the cherry-pick candidate
|
||||||
|
docs/guide/tipc.rst the docs page
|
||||||
|
examples/multihost/tipc_cluster/ runnable demos + manual
|
||||||
|
smoke test
|
||||||
|
ai/tpt-backends/00_shared_backend_contract.md
|
||||||
|
ai/tpt-backends/01_tipc_backend.md the (reconciled) plan
|
||||||
|
.github/workflows/ci.yml the gated tipc leg
|
||||||
|
```
|
||||||
|
|
||||||
|
Both single-host examples have been **run against a live
|
||||||
|
kernel** — the output pasted in their README is real, not
|
||||||
|
illustrative.
|
||||||
|
|
||||||
|
External agent memory deliberately contains only a project
|
||||||
|
pointer back to this handoff, not a competing copy of the project
|
||||||
|
state. Treat this file as the durable source of truth and update
|
||||||
|
it when the branch topology or landing sequence changes.
|
||||||
|
|
@ -0,0 +1,821 @@
|
||||||
|
# Plan 01 — `TIPC` transport backend (`tractor/ipc/_tipc.py`)
|
||||||
|
|
||||||
|
Tracks gh [#378]. Prereq reading:
|
||||||
|
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
|
||||||
|
|
||||||
|
**Thesis**: TIPC is the *cheapest* new backend we can add and
|
||||||
|
simultaneously the only one that gives us cluster-wide service
|
||||||
|
discovery **for free, in the kernel**, replacing (for
|
||||||
|
TIPC-capable deployments) the whole `tractor.discovery`
|
||||||
|
registrar round-trip with a `bind()`/`connect()` on a
|
||||||
|
*service name*. It is stdlib-only: zero new dependencies.
|
||||||
|
|
||||||
|
[#378]: https://github.com/goodboy/tractor/issues/378
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Why this is small: three verified facts
|
||||||
|
|
||||||
|
1. **CPython already speaks TIPC.** `socket.AF_TIPC` plus 23
|
||||||
|
`TIPC_*` constants are present in the stdlib on Linux
|
||||||
|
(verified on the dev box, py3.13):
|
||||||
|
`AF_TIPC, SOL_TIPC, TIPC_ADDR_ID, TIPC_ADDR_NAME,
|
||||||
|
TIPC_ADDR_NAMESEQ, TIPC_CFG_SRV, TIPC_CLUSTER_SCOPE,
|
||||||
|
TIPC_CONN_TIMEOUT, TIPC_{CRITICAL,HIGH,MEDIUM,LOW}_IMPORTANCE,
|
||||||
|
TIPC_DEST_DROPPABLE, TIPC_IMPORTANCE, TIPC_NODE_SCOPE,
|
||||||
|
TIPC_PUBLISHED, TIPC_SRC_DROPPABLE, TIPC_SUBSCR_TIMEOUT,
|
||||||
|
TIPC_SUB_CANCEL, TIPC_SUB_PORTS, TIPC_SUB_SERVICE,
|
||||||
|
TIPC_TOP_SRV, TIPC_WAIT_FOREVER, TIPC_WITHDRAWN,
|
||||||
|
TIPC_ZONE_SCOPE`.
|
||||||
|
`sock.bind()/connect()/getsockname()` take/return the
|
||||||
|
5-tuple `(addr_type, v1, v2, v3, scope)` — the last element
|
||||||
|
is optional on input and defaults to `0`.
|
||||||
|
2. **`trio` doesn't care about the address family.** Per
|
||||||
|
contract §1.5, `trio.SocketStream` and `trio.SocketListener`
|
||||||
|
only require a trio socket object of type `SOCK_STREAM`.
|
||||||
|
TIPC's `SOCK_STREAM` is a real connection-oriented reliable
|
||||||
|
byte stream. So we reuse `trio.SocketStream`,
|
||||||
|
`trio.SocketListener`, `trio.serve_listeners()`,
|
||||||
|
`MsgpackTransport`'s framing — *all of it*.
|
||||||
|
3. **It is not available by default.** On this box
|
||||||
|
`socket.socket(AF_TIPC, SOCK_STREAM)` →
|
||||||
|
`OSError(97, 'Address family not supported by protocol')`
|
||||||
|
with no `tipc` in `/proc/modules`. `modprobe tipc` is
|
||||||
|
required; cross-node needs a bearer
|
||||||
|
(`tipc bearer enable media eth device <if>` or
|
||||||
|
`media udp name <n> localip <ip>`). Everything about this
|
||||||
|
plan's testability hinges on gating (§7).
|
||||||
|
|
||||||
|
Non-goals: `SOCK_RDM`/`SOCK_DGRAM`/`SOCK_SEQPACKET` message
|
||||||
|
modes, multicast fan-out, and TIPC group messaging. They are
|
||||||
|
genuinely interesting for a future `tractor` broadcast/pubsub
|
||||||
|
transport but they do **not** fit `MsgTransport`'s
|
||||||
|
stream-of-length-prefixed-msgs shape. Note them in the
|
||||||
|
follow-up issue, do not build them here.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. `TIPCAddress`
|
||||||
|
|
||||||
|
### 2.1 the three TIPC address flavours, and which we use
|
||||||
|
|
||||||
|
| flavour | tuple | meaning |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `TIPC_ADDR_NAMESEQ` | `(type, lower, upper, scope)` | a *published range* — what a server `bind()`s |
|
||||||
|
| `TIPC_ADDR_NAME` | `(type, instance, domain, scope)` | a *lookup* — what a client `connect()`s |
|
||||||
|
| `TIPC_ADDR_ID` | `(node, ref, 0, scope)` | a concrete port id — the "physical" address |
|
||||||
|
|
||||||
|
The design decision that makes this backend coherent:
|
||||||
|
|
||||||
|
> **A `tractor` actor's TIPC address is a *service name*
|
||||||
|
> `(type, instance)`; `bind()` publishes the singleton range
|
||||||
|
> `(type, instance, instance)`; peers `connect()` by name and
|
||||||
|
> the kernel resolves + load-balances. `TIPC_ADDR_ID` is only
|
||||||
|
> ever an *observed* address (`getpeername()`), never a
|
||||||
|
> user-facing one.**
|
||||||
|
|
||||||
|
This is exactly the "leverage the built-in discovery machinery"
|
||||||
|
ask in #378: publishing a bind *is* registration, and
|
||||||
|
`connect()` on a name *is* a lookup, with no registrar actor in
|
||||||
|
the loop.
|
||||||
|
|
||||||
|
### 2.2 the struct
|
||||||
|
|
||||||
|
```python
|
||||||
|
class TIPCAddress(
|
||||||
|
msgspec.Struct,
|
||||||
|
frozen=True,
|
||||||
|
):
|
||||||
|
_stype: int # TIPC "type" == service class
|
||||||
|
_instance: int # service instance within the type
|
||||||
|
_scope: int = TIPC_CLUSTER_SCOPE
|
||||||
|
# observed-only, never part of identity/equality-by-intent
|
||||||
|
maybe_node: int|None = None # from TIPC_ADDR_ID getpeername()
|
||||||
|
maybe_ref: int|None = None
|
||||||
|
|
||||||
|
proto_key: ClassVar[str] = 'tipc'
|
||||||
|
unwrapped_type: ClassVar[type] = tuple[str, int, int, int]
|
||||||
|
def_bindspace: ClassVar[int] = TIPC_CLUSTER_SCOPE
|
||||||
|
```
|
||||||
|
|
||||||
|
**Unwrapped form** (the wire/`SpawnSpec` shape).
|
||||||
|
|
||||||
|
TIPC's natural form is `(stype, instance, scope)` — but a
|
||||||
|
2-tuple squeeze of it is a `(str, int)`, i.e. *the same coarse
|
||||||
|
shape as `TCPAddress`*, so `wrap_address()`'s
|
||||||
|
`case (str(), int())` steals it. This backend is therefore the
|
||||||
|
forcing function for the contract-doc's conclusion (§1.1):
|
||||||
|
|
||||||
|
> **make the unwrapped form carry an explicit proto-key, spelled
|
||||||
|
> with the `multiaddr` protocol name.**
|
||||||
|
|
||||||
|
```python
|
||||||
|
def unwrap(self) -> tuple[str, int, int, int]:
|
||||||
|
return ('tipc', self._stype, self._instance, self._scope)
|
||||||
|
```
|
||||||
|
|
||||||
|
`wrap_address()` then dispatches `_address_types[addr[0]]` and
|
||||||
|
the collision class disappears. **This is a prerequisite
|
||||||
|
migration commit, not part of this backend** — see contract §1.1
|
||||||
|
for its blast radius (wire format + every fixture + `piker`
|
||||||
|
config) and for the follow-on "stop handing raw tuples to users
|
||||||
|
at all, à la `ipaddress`" direction.
|
||||||
|
|
||||||
|
⚠️ an earlier revision of this plan proposed a self-tagging
|
||||||
|
`('tipc:<stype>:<scope>', instance)` string-prefix hack with an
|
||||||
|
ordered `case` guard. **Dropped** — it papers over the problem,
|
||||||
|
keeps `wrap_address()` order-sensitive, and doesn't help iroh's
|
||||||
|
`(str, str)`-vs-UDS collision at all. Do not resurrect it.
|
||||||
|
|
||||||
|
Note `TIPCAddress` is the first backend where `.unwrap()` is
|
||||||
|
**not** a lossless view of the live socket — `maybe_node`/
|
||||||
|
`maybe_ref` are observed metadata, exactly like
|
||||||
|
`UDSAddress.maybe_pid` (which is likewise excluded from
|
||||||
|
`.unwrap()`). Follow that precedent, including its `__repr__`
|
||||||
|
treatment (`_uds.py:242`).
|
||||||
|
|
||||||
|
### 2.3 how to pick `_stype` and `_instance`
|
||||||
|
|
||||||
|
- `_stype` = a `tractor`-reserved service class. TIPC reserves
|
||||||
|
0..63 for internal use (`TIPC_TOP_SRV == 1`,
|
||||||
|
`TIPC_CFG_SRV == 0`). Use a module constant
|
||||||
|
`TRACTOR_STYPE: int = 0x74_72_00_00` ("tr\0\0") as the default
|
||||||
|
and make it overridable via `TIPCAddress._stype` so an app
|
||||||
|
can partition service classes. Document that two `tractor`
|
||||||
|
trees sharing a cluster **and** a `_stype` share a namespace.
|
||||||
|
- `_instance` for `get_root()`: `1616` — mirrors the
|
||||||
|
`TCPAddress.get_root()` port and the `registry@1616.sock`
|
||||||
|
UDS filename, so the "1616 is tractor's registrar" idiom
|
||||||
|
holds across all backends.
|
||||||
|
- `_instance` for `get_random()`: TIPC gives us no
|
||||||
|
kernel-assigned-instance analogue of `port=0`, so we must
|
||||||
|
choose. Use a *pure* fn of the actor identity so it is
|
||||||
|
reproducible and collision-free:
|
||||||
|
```python
|
||||||
|
# 32-bit instance derived from the actor's uuid4 (+ pid when
|
||||||
|
# there's no live runtime, per the UDS precedent).
|
||||||
|
inst: int = int.from_bytes(
|
||||||
|
blake2b(seed.encode(), digest_size=4).digest(),
|
||||||
|
'big',
|
||||||
|
)
|
||||||
|
```
|
||||||
|
where `seed = f'{actor.aid.name}@{pid}'` if
|
||||||
|
`current_actor(err_on_no_runtime=False)` else
|
||||||
|
`f'{prefix}.{uuid4().hex[:8]}@{pid}'`. Must avoid the reserved
|
||||||
|
low range: `inst = 64 + (inst % (2**32 - 64))`.
|
||||||
|
⚠️ *unlike* `port=0`, a collision here surfaces as a
|
||||||
|
successful-but-shared publication (TIPC allows multiple
|
||||||
|
binders on the same name and round-robins!) rather than
|
||||||
|
`EADDRINUSE`. That is a silent-crosstalk failure mode; §7 has
|
||||||
|
the test that proves the 4-byte digest is enough and §9 has
|
||||||
|
the mitigation if it isn't.
|
||||||
|
- `_scope`: `TIPC_NODE_SCOPE` for a same-host-only actor (the
|
||||||
|
UDS-equivalent), `TIPC_CLUSTER_SCOPE` (default) for
|
||||||
|
cluster-visible. **This is `.bindspace`**:
|
||||||
|
```python
|
||||||
|
@property
|
||||||
|
def bindspace(self) -> int:
|
||||||
|
return self._scope
|
||||||
|
```
|
||||||
|
It is the honest analogue of "the set of hosts this bind is
|
||||||
|
reachable from", which is precisely the docstring in
|
||||||
|
`Address.bindspace`. (`TIPC_ZONE_SCOPE` is deprecated/aliased
|
||||||
|
to cluster in modern kernels — accept it on input, normalize
|
||||||
|
to cluster, log at `transport` level.)
|
||||||
|
|
||||||
|
### 2.4 `is_valid`
|
||||||
|
|
||||||
|
```python
|
||||||
|
@property
|
||||||
|
def is_valid(self) -> bool:
|
||||||
|
return (
|
||||||
|
self._instance != 0
|
||||||
|
and
|
||||||
|
self._stype not in _tipc_reserved_stypes # {0, 1, ...}
|
||||||
|
and
|
||||||
|
self._scope in (TIPC_NODE_SCOPE, TIPC_CLUSTER_SCOPE)
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Listener + stream
|
||||||
|
|
||||||
|
### 3.1 `start_listener()`
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def start_listener(
|
||||||
|
addr: TIPCAddress,
|
||||||
|
backlog: int = 128,
|
||||||
|
**kwargs,
|
||||||
|
) -> SocketListener:
|
||||||
|
sock = trio.socket.socket(
|
||||||
|
socket.AF_TIPC,
|
||||||
|
socket.SOCK_STREAM,
|
||||||
|
)
|
||||||
|
# publish the singleton name-range == "register the service"
|
||||||
|
await sock.bind((
|
||||||
|
socket.TIPC_ADDR_NAMESEQ,
|
||||||
|
addr._stype,
|
||||||
|
addr._instance,
|
||||||
|
addr._instance,
|
||||||
|
addr._scope,
|
||||||
|
))
|
||||||
|
sock.listen(backlog)
|
||||||
|
return SocketListener(sock)
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes / hazards:
|
||||||
|
|
||||||
|
- `bind()` on `AF_TIPC` is **not** a filesystem or port-table
|
||||||
|
operation and can't block on DNS, but keep it `await`ed
|
||||||
|
through `trio.socket` anyway for uniformity.
|
||||||
|
- `backlog=128` matching `_uds.start_listener()`'s hard-won
|
||||||
|
value (see its comment at `_uds.py:317-331` re: concurrent
|
||||||
|
deregistration storms). Do not use `1`.
|
||||||
|
- **no `close_listener()` needed** — nothing to unlink. Omit the
|
||||||
|
function entirely (contract §1.2: absence means implicit).
|
||||||
|
Withdrawal of the published name happens on socket close.
|
||||||
|
- ✅ **SETTLED** (step-0 probe, live kernel): `SocketListener.
|
||||||
|
__init__`'s `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)` **works**
|
||||||
|
on `AF_TIPC` and answers `1`. We do *not* rely on trio's
|
||||||
|
`except OSError: pass` carve-out at all. Pinned by
|
||||||
|
`test_listener_tolerates_so_acceptconn`.
|
||||||
|
- Wrap the bind in a `_reraise_as_connerr()`-style `@cm` (copy
|
||||||
|
the `_uds.py:256` pattern) so `EADDRINUSE`-ish and
|
||||||
|
`EAFNOSUPPORT` become `ConnectionError` with the addr in the
|
||||||
|
message. `EAFNOSUPPORT` here means "kernel module not
|
||||||
|
loaded" and deserves a *specifically actionable* message:
|
||||||
|
`'TIPC unavailable — try `sudo modprobe tipc`\n'`.
|
||||||
|
|
||||||
|
### 3.2 the `getsockname()` reconciliation
|
||||||
|
|
||||||
|
`Endpoint.start_listener()` does
|
||||||
|
`if lstnr.socket.getsockname() != self.addr.unwrap(): self.addr =
|
||||||
|
self.addr.from_addr(unwrapped)`.
|
||||||
|
|
||||||
|
For TIPC, `getsockname()` on a bound-but-listening socket
|
||||||
|
returns a `TIPC_ADDR_ID`-flavoured 5-tuple (the port id), *not*
|
||||||
|
the name-seq we bound. So the `!=` is **always true** and
|
||||||
|
`from_addr()` will be handed a 5-tuple.
|
||||||
|
|
||||||
|
Handle it inside `TIPCAddress.from_addr()` — do **not** patch
|
||||||
|
`_server.py`:
|
||||||
|
|
||||||
|
⚠️ the sketch that stood here used the `'tipc:<stype>:<scope>'`
|
||||||
|
string-prefix hack §2.2 explicitly **withdrew**. Corrected to
|
||||||
|
the proto-keyed form (and note a bare seq-pattern matches the
|
||||||
|
`list` that `msgpack` decodes our tuples back to, so no
|
||||||
|
separate `[...]` alternative is needed):
|
||||||
|
|
||||||
|
```python
|
||||||
|
@classmethod
|
||||||
|
def from_addr(cls, addr) -> TIPCAddress:
|
||||||
|
match addr:
|
||||||
|
# our own proto-keyed unwrapped form
|
||||||
|
case ('tipc', int() as stype, int() as inst, int() as scope):
|
||||||
|
return TIPCAddress(stype, inst, _norm_scope(scope))
|
||||||
|
|
||||||
|
# ..w/ the scope defaulted
|
||||||
|
case ('tipc', int() as stype, int() as inst):
|
||||||
|
return TIPCAddress(stype, inst)
|
||||||
|
|
||||||
|
# a kernel-observed TIPC_ADDR_ID 5-tuple
|
||||||
|
case (int() as atype, *_) if atype == TIPC_ADDR_ID:
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
The `TIPC_ADDR_ID` case cannot reconstruct `(stype, instance)`
|
||||||
|
— that info isn't in a port id. So `from_addr()` alone is
|
||||||
|
insufficient for the reconciliation path. **Resolution**: make
|
||||||
|
`from_addr()` raise a clear `ValueError` for the bare
|
||||||
|
`TIPC_ADDR_ID` case, and instead prevent the reconciliation
|
||||||
|
from firing by having `start_listener()` return a listener
|
||||||
|
whose `getsockname()` we never need — i.e. land this two-line
|
||||||
|
upstream fix in `_server.py:664`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
if (
|
||||||
|
(unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap()
|
||||||
|
and
|
||||||
|
self.addr.rebind_from_sockname # ClassVar[bool] = True on tcp/uds
|
||||||
|
):
|
||||||
|
```
|
||||||
|
|
||||||
|
with `TIPCAddress.rebind_from_sockname: ClassVar[bool] = False`
|
||||||
|
(and `True` on `TCPAddress`/`UDSAddress`, preserving today's
|
||||||
|
behaviour exactly). Rationale: the reconciliation exists *only*
|
||||||
|
to learn the kernel-assigned port for `port=0` TCP binds (its
|
||||||
|
own comment says so, `_server.py:662`); TIPC has no such
|
||||||
|
late-binding, so opting out is semantically right rather than a
|
||||||
|
hack. **Land this as its own commit, ahead of the backend**,
|
||||||
|
with a test that `tcp`'s `port=0` behaviour is unchanged.
|
||||||
|
|
||||||
|
Keep the observed port-id available anyway: annotate
|
||||||
|
`ep.addr = ep.addr.with_port_id(*getsockname()[1:3])` (a pure
|
||||||
|
`msgspec.structs.replace()` helper) purely for logging/repr.
|
||||||
|
|
||||||
|
### 3.3 `MsgpackTIPCStream`
|
||||||
|
|
||||||
|
```python
|
||||||
|
class MsgpackTIPCStream(MsgpackTransport):
|
||||||
|
address_type = TIPCAddress
|
||||||
|
layer_key: int = 4
|
||||||
|
|
||||||
|
@property
|
||||||
|
def maddr(self) -> Multiaddr|str:
|
||||||
|
return mk_maddr(self.raddr)
|
||||||
|
|
||||||
|
def connected(self) -> bool:
|
||||||
|
return self.stream.socket.fileno() != -1
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
async def connect_to(
|
||||||
|
cls,
|
||||||
|
destaddr: TIPCAddress,
|
||||||
|
prefix_size: int = 4,
|
||||||
|
codec: MsgCodec|None = None,
|
||||||
|
**kwargs,
|
||||||
|
) -> MsgpackTIPCStream:
|
||||||
|
sock = trio.socket.socket(AF_TIPC, SOCK_STREAM)
|
||||||
|
with close_on_error(sock):
|
||||||
|
# NOTE: connect by *name* -> kernel does the lookup,
|
||||||
|
# so this is our "discovery" call.
|
||||||
|
await sock.connect((
|
||||||
|
socket.TIPC_ADDR_NAME,
|
||||||
|
destaddr._stype,
|
||||||
|
destaddr._instance,
|
||||||
|
0, # domain: 0 == "anywhere in scope"
|
||||||
|
destaddr._scope,
|
||||||
|
))
|
||||||
|
return cls(
|
||||||
|
trio.SocketStream(sock),
|
||||||
|
prefix_size=prefix_size,
|
||||||
|
codec=codec,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
- reuse `trio._highlevel_open_unix_stream.close_on_error` (the
|
||||||
|
UDS backend already imports it) or inline the equivalent
|
||||||
|
`try/except: sock.close(); raise`.
|
||||||
|
- `SO_/TIPC_` opts worth setting and documenting:
|
||||||
|
- `setsockopt(SOL_TIPC, TIPC_IMPORTANCE, TIPC_HIGH_IMPORTANCE)`
|
||||||
|
for the *parent<->child* lifetime channel — this is a real
|
||||||
|
win TIPC gives us that TCP can't: the runtime's
|
||||||
|
supervision channel can outrank bulk app traffic under
|
||||||
|
congestion. Wire it as a `connect_to(..., importance=...)`
|
||||||
|
kwarg defaulted from a module constant, and have
|
||||||
|
`_runtime.py`'s parent-chan path pass the high value **in a
|
||||||
|
follow-up** (don't couple it to this PR).
|
||||||
|
- `TIPC_CONN_TIMEOUT` — the kernel-side connect timeout;
|
||||||
|
leave at default, we have `trio` cancel scopes.
|
||||||
|
- `TIPC_DEST_DROPPABLE = 0` on the connection so undeliverable
|
||||||
|
msgs come back as errors rather than being silently dropped.
|
||||||
|
- ✅ **SETTLED** — **`connect_to()` on a name with no
|
||||||
|
publisher**: TIPC answers `EHOSTUNREACH` (113) *instantly*
|
||||||
|
(no SYN-timeout wait), which is indeed better discovery-ping
|
||||||
|
behaviour than TCP.
|
||||||
|
|
||||||
|
⚠️ BUT the errno matters more than expected: python maps
|
||||||
|
`EHOSTUNREACH` to a **bare `OSError`**, NOT to a
|
||||||
|
`ConnectionError` subtype the way it maps `ECONNREFUSED` ->
|
||||||
|
`ConnectionRefusedError`. So the `_reraise_as_connerr()` wrap
|
||||||
|
is **load-bearing for contract §4**, not cosmetic polish —
|
||||||
|
without it the registrar ping path sees a foreign exc type.
|
||||||
|
(For contrast, dialling a bogus *port-id* — as opposed to a
|
||||||
|
name — does give `ECONNREFUSED`.)
|
||||||
|
|
||||||
|
### 3.4 `get_stream_addrs()`
|
||||||
|
|
||||||
|
```python
|
||||||
|
@classmethod
|
||||||
|
def get_stream_addrs(cls, stream) -> tuple[TIPCAddress, TIPCAddress]:
|
||||||
|
sock = stream.socket
|
||||||
|
# both return TIPC_ADDR_ID 5-tuples for a connected sock
|
||||||
|
l_id = sock.getsockname()
|
||||||
|
r_id = sock.getpeername()
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Problem: neither end's port-id tells us the *service name*. The
|
||||||
|
`laddr`/`raddr` are used for logging, `Channel.raddr`,
|
||||||
|
`Server._peers` keying-adjacent repr, and `maddr`. Design:
|
||||||
|
|
||||||
|
- the **connecting** side knows the destaddr it dialled →
|
||||||
|
`connect_to()` overrides `_raddr` after construction with the
|
||||||
|
known-good `TIPCAddress`, exactly as
|
||||||
|
`MsgpackUDSStream.connect_to()` does for the peer-pid case
|
||||||
|
(`_uds.py:539-543`).
|
||||||
|
- the **accepting** side does not know the peer's service name
|
||||||
|
from the socket. Two honest options:
|
||||||
|
- **(a) accept it: `raddr` carries only `(node, ref)`** via
|
||||||
|
`maybe_node`/`maybe_ref`, `_stype/_instance` set to a
|
||||||
|
sentinel `-1`, and `__repr__` renders
|
||||||
|
`TIPCAddress[<peer-node:0x...>:<ref>]`. The `Aid` from the
|
||||||
|
handshake already gives us the peer's logical identity, so
|
||||||
|
nothing in the runtime actually *needs* the peer's service
|
||||||
|
name. **Recommended.**
|
||||||
|
- (b) piggyback the peer's own bound name in the handshake.
|
||||||
|
Rejected for this PR: touches `Aid`/msg-spec.
|
||||||
|
- `laddr` on the accepting side: the `Endpoint` knows its own
|
||||||
|
`addr`; but `get_stream_addrs()` is a `@classmethod` with only
|
||||||
|
the stream. Use `TIPC_ADDR_ID` for `laddr` too and let
|
||||||
|
`Endpoint.peer_tpts` keying (which is by *peer* addr) still
|
||||||
|
work. Verify nothing asserts `laddr == ep.addr` — grep for
|
||||||
|
`.laddr` uses before committing (`_server.py`'s
|
||||||
|
`con_status` logging, `Channel.pformat()`).
|
||||||
|
✅ grepped: `.laddr` is repr/logging-ONLY. `.raddr` has three
|
||||||
|
real consumers (`discovery/_api.py:277`'s `query_actor()`
|
||||||
|
yield, plus two test asserts) — and note `uds` *already* has
|
||||||
|
this same wart (its accepting-side `raddr` is the listener's
|
||||||
|
own sockpath), so (a) is consistent with the status quo.
|
||||||
|
|
||||||
|
- 🐛 **HAZARD the original draft missed — a dropped peer must
|
||||||
|
not kill the actor.** Unlike tcp/uds — where the kernel keeps
|
||||||
|
answering the peer addr until *we* close — a TIPC socket
|
||||||
|
whose peer has already gone answers **`ENOTCONN`** from
|
||||||
|
`getpeername()`.
|
||||||
|
|
||||||
|
That's fatal as written, because
|
||||||
|
`MsgpackTransport.__init__()` calls `get_stream_addrs()` (via
|
||||||
|
`Channel.from_stream()`) **before** the handshake, so the
|
||||||
|
`OSError` escapes `handle_stream_from_peer()`'s
|
||||||
|
handshake-failure tolerance (contract §4) and tears down the
|
||||||
|
**whole actor**. i.e. any connect-then-immediately-drop peer
|
||||||
|
— a port scan, a liveness probe, a cancelled dial — is a
|
||||||
|
remote actor-kill.
|
||||||
|
|
||||||
|
Wrap both `getsockname`/`getpeername` in a tolerant helper
|
||||||
|
and degrade to a port-id-less addr. A dead peer must cost us
|
||||||
|
an addr, not the runtime.
|
||||||
|
|
||||||
|
NOTE this is *not* hypothetical: the discovery suite's own
|
||||||
|
`daemon` readiness probe
|
||||||
|
(`tests/discovery/conftest.py`) does exactly this, which is
|
||||||
|
how it was found.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Multiaddr representation
|
||||||
|
|
||||||
|
There is no `/tipc` in the multiaddr protocol table. Interim
|
||||||
|
grammar, mirroring how `uds` maps to the spec-legal `/unix`:
|
||||||
|
|
||||||
|
```
|
||||||
|
/tipc/<stype>/<instance> # scope implied = cluster
|
||||||
|
/tipc/<stype>/<instance>/<scope> # explicit
|
||||||
|
```
|
||||||
|
|
||||||
|
- `_tpt_proto_to_maddr['tipc'] = 'tipc'` and a `mk_maddr()`
|
||||||
|
`case 'tipc':` building the above.
|
||||||
|
- `parse_maddr()` gets `case ['tipc']:` — but note
|
||||||
|
`py-multiaddr` will reject an unregistered protocol name
|
||||||
|
outright, so this **requires an upstream registration** (same
|
||||||
|
track as the `wg` work, gh #483 /
|
||||||
|
multiformats/py-multiaddr#107). Until that lands:
|
||||||
|
- `MsgpackTIPCStream.maddr` returns the **`str`** form (the
|
||||||
|
`MsgTransport.maddr` return type is already
|
||||||
|
`Multiaddr|str`, and `MsgpackUDSStream.maddr` already
|
||||||
|
exercises the `str` branch), and
|
||||||
|
- `parse_maddr()` special-cases the `/tipc/` prefix *before*
|
||||||
|
handing the string to `Multiaddr()`.
|
||||||
|
Document this as the reason gh #443's "standardize on
|
||||||
|
returning `Multiaddr` everywhere" item stays blocked.
|
||||||
|
|
||||||
|
Propose `/tipc/` upstream as: name `tipc`, code TBD, size
|
||||||
|
variable, value `<stype>:<instance>:<scope>` — or as three
|
||||||
|
composed protos. Prefer *one* proto with a structured value so
|
||||||
|
the maddr stays 2-segment like `/unix/...`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Discovery: the actually-interesting part
|
||||||
|
|
||||||
|
Two independently-shippable layers. **Layer A is in scope for
|
||||||
|
the first PR; layer B is a fast-follow.**
|
||||||
|
|
||||||
|
### 5.1 Layer A — "discovery by bind" (free)
|
||||||
|
|
||||||
|
Because `bind(TIPC_ADDR_NAMESEQ)` publishes and
|
||||||
|
`connect(TIPC_ADDR_NAME)` resolves, a `tractor` tree whose
|
||||||
|
`registry_addrs` are TIPC service names needs **no registrar
|
||||||
|
liveness at all** for the connect path: `find_actor()`'s
|
||||||
|
"connect to the registrar and ask" becomes "connect to the
|
||||||
|
service name directly". Concretely:
|
||||||
|
|
||||||
|
- `tractor.discovery._api.find_actor()` etc. keep working
|
||||||
|
unchanged (they go through the registrar), *and*
|
||||||
|
- a new, TIPC-only fast path becomes possible: derive an actor's
|
||||||
|
service name from its `(name, uuid)` and dial it without any
|
||||||
|
registrar hop.
|
||||||
|
|
||||||
|
Do **not** build the fast path in PR 1. Instead, prove the
|
||||||
|
property with a test (§7.4) and file the follow-up: it changes
|
||||||
|
`discovery` semantics (name→instance derivation must be a
|
||||||
|
documented, stable, cross-language-able hash) and deserves its
|
||||||
|
own design.
|
||||||
|
|
||||||
|
### 5.2 Layer B — the topology service (`TIPC_TOP_SRV`)
|
||||||
|
|
||||||
|
This is what makes #378's "end game cluster proto" claim real:
|
||||||
|
a *subscription* to name-table events, i.e. push-based
|
||||||
|
`register`/`deregister` for free, replacing the registrar's
|
||||||
|
polled `find_actor()`.
|
||||||
|
|
||||||
|
Mechanics (verify each field against
|
||||||
|
`linux/include/uapi/linux/tipc.h` + `net/tipc/topsrv.c` at
|
||||||
|
implementation time — the struct layout below is from the uapi
|
||||||
|
header and the byte-order caveat is real):
|
||||||
|
|
||||||
|
```python
|
||||||
|
# SOCK_SEQPACKET connected to the topology server
|
||||||
|
sock = trio.socket.socket(AF_TIPC, SOCK_SEQPACKET)
|
||||||
|
await sock.connect((
|
||||||
|
socket.TIPC_ADDR_NAME,
|
||||||
|
socket.TIPC_TOP_SRV, # == 1
|
||||||
|
socket.TIPC_TOP_SRV,
|
||||||
|
0,
|
||||||
|
))
|
||||||
|
|
||||||
|
# struct tipc_subscr {
|
||||||
|
# struct tipc_name_seq seq; /* 3 * __u32: type, lower, upper */
|
||||||
|
# __u32 timeout; /* TIPC_WAIT_FOREVER == ~0 */
|
||||||
|
# __u32 filter; /* TIPC_SUB_{PORTS,SERVICE,CANCEL} */
|
||||||
|
# char usr_handle[8];
|
||||||
|
# } /* == 28 bytes */
|
||||||
|
_SUBSCR_FMT: str = '=IIIII8s' # ⚠ 5*I is 20 -> use '=5I8s'
|
||||||
|
```
|
||||||
|
|
||||||
|
- **byte order**: the topology server historically accepts both
|
||||||
|
host and swapped order and auto-detects; modern kernels are
|
||||||
|
strict-ish. Pack native (`'='`) first, and if the server
|
||||||
|
closes the connection immediately, retry with `'>'`. Encode
|
||||||
|
that as a one-time probe helper
|
||||||
|
`_detect_topsrv_endianness()` cached at module level — and
|
||||||
|
put a `# ?TODO` pointing at `net/tipc/topsrv.c` for someone
|
||||||
|
to make it deterministic.
|
||||||
|
- **events**: `struct tipc_event` is `event: u32`,
|
||||||
|
`found_lower: u32`, `found_upper: u32`,
|
||||||
|
`port: {ref: u32, node: u32}`, then the 28-byte subscription
|
||||||
|
echo. `event ∈ {TIPC_PUBLISHED, TIPC_WITHDRAWN,
|
||||||
|
TIPC_SUBSCR_TIMEOUT}`.
|
||||||
|
|
||||||
|
⚠️ **CORRECTION**: that totals **48** bytes
|
||||||
|
(`4 + 4 + 4 + 8 + 28`), not the 40 an earlier revision of this
|
||||||
|
plan claimed. Verified via `struct.calcsize()` at step 0. Use
|
||||||
|
`'=5I8s'` (28) for the subscription and a 48-byte read for the
|
||||||
|
event.
|
||||||
|
|
||||||
|
⚠️ also: python exposes `TIPC_WAIT_FOREVER` as **`-1`**, not
|
||||||
|
`0xFFFFFFFF`, so it must be masked (`& 0xFFFFFFFF`) before
|
||||||
|
packing into an unsigned `'I'` field.
|
||||||
|
- **trio shape** — this is where the "nearly-functional,
|
||||||
|
modern-async" style pays off; expose it as an `@acm` yielding
|
||||||
|
a `trio` receive-channel of typed events, *not* a class:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@acm
|
||||||
|
async def open_topology_events(
|
||||||
|
stype: int = TRACTOR_STYPE,
|
||||||
|
lower: int = 0,
|
||||||
|
upper: int = 0xFFFFFFFF,
|
||||||
|
filter: int = TIPC_SUB_SERVICE,
|
||||||
|
timeout: int = TIPC_WAIT_FOREVER,
|
||||||
|
buf_size: int = 64,
|
||||||
|
) -> AsyncGenerator[
|
||||||
|
trio.MemoryReceiveChannel[TIPCNameEvent],
|
||||||
|
None,
|
||||||
|
]:
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
with `TIPCNameEvent(msgspec.Struct, frozen=True)` fields
|
||||||
|
`kind: Literal['published','withdrawn','timeout']`,
|
||||||
|
`addr: TIPCAddress`, `node: int`, `ref: int`. One
|
||||||
|
`trio.lowlevel`-free implementation: a nursery-spawned reader
|
||||||
|
task doing `await sock.recv(48)` in a loop and
|
||||||
|
`send_nowait()`ing decoded events, with the `@acm` closing the
|
||||||
|
socket on exit → reader gets `ClosedResourceError` → cancel
|
||||||
|
scope collapses. Standard `tractor` `@acm` discipline.
|
||||||
|
- **consumer**: `tractor/discovery/_registry.py` gains an
|
||||||
|
optional "watch" mode so a registrar (or any actor) can keep
|
||||||
|
a live view of the actor set without polling. Sketch the
|
||||||
|
integration in the follow-up issue; do not wire it in PR 1.
|
||||||
|
- **`SOCK_SEQPACKET` is fine here** because this socket never
|
||||||
|
goes through `MsgpackTransport` — it's a plain trio socket
|
||||||
|
used with `recv()`. The contract's "`SOCK_STREAM` only"
|
||||||
|
constraint applies to `MsgTransport` streams, not to this.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Commit sequencing (each independently reviewable + green)
|
||||||
|
|
||||||
|
**STATUS** (gh PR #493, stacked on #492): steps 1-5 landed as
|
||||||
|
9 commits, `22ef362d..51d7133f`. Acceptance bar met — 122
|
||||||
|
passed / 1 xfailed / 2 xpassed under `--tpt-proto tipc` across
|
||||||
|
`ipc`, `discovery`, `runtime`, `spawning`, `local`, `rpc`,
|
||||||
|
`cancellation`; `tcp`/`uds` unchanged. Steps 6-7 remain.
|
||||||
|
|
||||||
|
Two commits fell out that this plan did NOT anticipate,
|
||||||
|
- a wire-spec widening for the 4-tuple (§9), and
|
||||||
|
- an unrelated `devx.pformat` crasher that masked EVERY
|
||||||
|
send-side `MsgTypeError`; it's on `main` and every branch,
|
||||||
|
so it wants cherry-picking out of this stack.
|
||||||
|
|
||||||
|
1. `_server.py`: add `Address.rebind_from_sockname:
|
||||||
|
ClassVar[bool]`, gate the `getsockname()` reconciliation on
|
||||||
|
it, `True` for tcp/uds. Test: tcp `port=0` unchanged.
|
||||||
|
2. `tractor/ipc/_tipc.py`: `TIPCAddress` + `is_tipc_available()`
|
||||||
|
predicate + `start_listener()`. No transport yet.
|
||||||
|
Tests: address round-trip (`unwrap`/`from_addr`/`wrap_address`),
|
||||||
|
`get_random()` uniqueness, bind/listen + `SO_ACCEPTCONN`
|
||||||
|
tolerance, `EAFNOSUPPORT` → actionable `ConnectionError`.
|
||||||
|
3. `MsgpackTIPCStream` + `connect_to()` + `get_stream_addrs()`.
|
||||||
|
Test: two `trio` tasks in one proc exchange a msg over
|
||||||
|
`Msgpack` framing (no `tractor` runtime).
|
||||||
|
4. registration tables (contract §2 items 1-6, 9) +
|
||||||
|
`pyproject.toml` mark/extra. Test: full suite under
|
||||||
|
`--tpt-proto tipc` (§7.3).
|
||||||
|
5. maddr support (`str` form + prefix special-case) + docs.
|
||||||
|
6. `open_topology_events()` @acm + its tests (layer B).
|
||||||
|
7. docs page + `docs/` example.
|
||||||
|
|
||||||
|
Per project convention, a reproducing/guard test lands in its
|
||||||
|
own commit **before** the fix it guards.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Testing
|
||||||
|
|
||||||
|
### 7.1 the capability predicate (in `_tipc.py`, public)
|
||||||
|
|
||||||
|
```python
|
||||||
|
def is_tipc_available() -> bool:
|
||||||
|
'''
|
||||||
|
True iff this kernel can create an `AF_TIPC` socket, i.e.
|
||||||
|
the `tipc` module is loaded.
|
||||||
|
|
||||||
|
'''
|
||||||
|
try:
|
||||||
|
socket.socket(socket.AF_TIPC, socket.SOCK_STREAM).close()
|
||||||
|
return True
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
|
```
|
||||||
|
|
||||||
|
Cache it in a module global (it can't change without a
|
||||||
|
`modprobe`, and a cold call costs a syscall). Pure predicate, no
|
||||||
|
side effects, no logging.
|
||||||
|
|
||||||
|
### 7.2 gating
|
||||||
|
|
||||||
|
- `pytest.mark.tipc` registered in
|
||||||
|
`_testing/pytest.py::pytest_configure()` alongside `no_tpt`,
|
||||||
|
`skipon_spawn_backend` et al.
|
||||||
|
⚠️ **CORRECTION**: an earlier revision said `pyproject.toml`;
|
||||||
|
the repo has no `[tool.pytest.ini_options] markers` table and
|
||||||
|
registers every custom mark via `config.addinivalue_line()`.
|
||||||
|
Per contract §0, the code wins.
|
||||||
|
- module-level
|
||||||
|
`pytestmark = pytest.mark.skipif(not is_tipc_available(),
|
||||||
|
reason='`tipc` kernel module not loaded (`modprobe tipc`)')`
|
||||||
|
in `tests/ipc/test_tipc.py`.
|
||||||
|
- `--tpt-proto tipc` with no module must fail **loudly and
|
||||||
|
early** with the actionable message, not with 400 confusing
|
||||||
|
timeouts. Add the check to the `tpt_protos` fixture's existing
|
||||||
|
per-proto validation loop (`_testing/pytest.py:795`): if the
|
||||||
|
chosen `Address` type exposes an `is_available()`-style
|
||||||
|
classmethod, call it and `pytest.fail()` with its reason.
|
||||||
|
Generalize (don't special-case tipc) — plans 02/03 need the
|
||||||
|
same hook.
|
||||||
|
|
||||||
|
### 7.3 CI
|
||||||
|
|
||||||
|
- add a job matrix entry `--tpt-proto tipc` that runs
|
||||||
|
`sudo modprobe tipc` in a `before` step. GH's
|
||||||
|
`ubuntu-latest` runners do allow `modprobe tipc` (the module
|
||||||
|
ships with the standard Ubuntu kernel package); verify in a
|
||||||
|
throwaway workflow before wiring the matrix. If it turns out
|
||||||
|
to be unavailable, fall back to a container job with
|
||||||
|
`--privileged`/`--cap-add NET_ADMIN`, and mark the job
|
||||||
|
`continue-on-error` until it's proven stable.
|
||||||
|
- cross-node TIPC (bearer) cannot be CI'd; cover it with a
|
||||||
|
documented manual smoke test in the docs page, in the style
|
||||||
|
of gh #482's LAN examples.
|
||||||
|
|
||||||
|
### 7.4 backend-specific tests worth writing
|
||||||
|
|
||||||
|
- **name-publication is discovery**: bind a listener on
|
||||||
|
`(stype, inst)`, then from a second task `connect()` by name
|
||||||
|
and assert it lands — *without* any `tractor` registrar.
|
||||||
|
- **`get_random()` collision resistance**: 10k `get_random()`
|
||||||
|
calls with no live runtime.
|
||||||
|
⚠️ **CORRECTION**: asserting **10k distinct** is a ~1.2%
|
||||||
|
flaky test, not a guarantee —
|
||||||
|
`P(collision) ≈ 1 - exp(-n²/2^33) ≈ 1.16e-2` for `n=10k` in a
|
||||||
|
32-bit instance space. That's ~1-in-86 runs red, which the
|
||||||
|
project's fix-flakes-at-source rule forbids. Assert
|
||||||
|
`>= n - 2` instead (`P(>2 collisions) ≈ 1e-7`) and document
|
||||||
|
the arithmetic inline.
|
||||||
|
|
||||||
|
Also add a *deterministic* sibling asserting the derivation
|
||||||
|
is a pure fn of the seed, which is the property the (§5.1)
|
||||||
|
registrar-less fast path will actually depend on.
|
||||||
|
|
||||||
|
⚠️ do **NOT** take §9's "fold a 6-byte digest into
|
||||||
|
`(stype_low, instance)`" escalation: §5.2's topology
|
||||||
|
subscription can only watch **one** service type, so varying
|
||||||
|
`_stype` per-actor would need 65536 subscriptions and kills
|
||||||
|
layer B outright. The instance space is 32b and that's that;
|
||||||
|
if crosstalk ever bites for real, the answer is the post-bind
|
||||||
|
verification handshake, not stype bits.
|
||||||
|
- ✅ **SETTLED — round-robin surprise is REAL**: two listeners
|
||||||
|
bound to the *same* `(stype, inst)` both bind fine and
|
||||||
|
connects alternate strictly (`b,a,b,a,b,a` observed over 6
|
||||||
|
dials). So a `get_random()` clash is *silent crosstalk*, never
|
||||||
|
`EADDRINUSE`. Assert the observed behaviour and reference it
|
||||||
|
from the `get_random()` docstring so the next reader knows
|
||||||
|
why the hash matters.
|
||||||
|
- **scope isolation**: a `TIPC_NODE_SCOPE` bind is not visible
|
||||||
|
to a cluster-scope lookup from another node (manual/marked).
|
||||||
|
- **importance opt** round-trips via `getsockopt`.
|
||||||
|
- **graceful + abrupt close** produce `TransportClosed` with the
|
||||||
|
same `loglevel` classification as tcp/uds — i.e. re-run the
|
||||||
|
relevant `tests/ipc/test_each_tpt.py` cases parametrized over
|
||||||
|
the new proto rather than writing new ones.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Deployment / docs deliverable
|
||||||
|
|
||||||
|
A `docs/` page (and/or an `examples/` script) covering:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# single host, node-scope only
|
||||||
|
sudo modprobe tipc
|
||||||
|
tipc node get addr
|
||||||
|
|
||||||
|
# multi-host over ethernet (pairs beautifully with plan 03's wg)
|
||||||
|
sudo tipc bearer enable media eth device eth0
|
||||||
|
# ...or over UDP when L2 isn't available:
|
||||||
|
sudo tipc bearer enable media udp name uc localip 10.0.11.1
|
||||||
|
tipc link list
|
||||||
|
tipc nametable show # <- see tractor's published services!
|
||||||
|
```
|
||||||
|
|
||||||
|
`tipc nametable show` displaying live `tractor` actors is the
|
||||||
|
single best demo this backend has; lead with it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Known risks + escalations
|
||||||
|
|
||||||
|
Status column reconciled against the **step-0 probe on a live
|
||||||
|
kernel** (`modprobe tipc`, py3.13) plus the landed impl. Rows
|
||||||
|
marked ⚠️ are the ones whose *stated* mitigation turned out to
|
||||||
|
be wrong or insufficient.
|
||||||
|
|
||||||
|
| risk | status | mitigation |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `_instance` hash collision → silent crosstalk (two actors share a service name, TIPC round-robins connects between them) | ✅ **confirmed real** — dup binds both succeed, dials alternate strictly | `blake2b` 32b digest + §7.4 tests. ⚠️ the "6-byte digest folded into `(stype_low, instance)`" escalation is **withdrawn** — it breaks §5.2's single-type subscription. Real escalation is a post-bind verification handshake |
|
||||||
|
| kernel/module unavailability everywhere (dev boxes, macOS, CI) | ✅ handled | `is_tipc_available()` + the generic `Address.is_available() -> (ok, why_not)` hook consumed by the `tpt_protos` fixture; module stays importable on non-linux via uapi-value fallbacks. TIPC is *opt-in cluster* only, never a default |
|
||||||
|
| `getsockname()` returns port-id not name | ✅ confirmed (true even *pre*-bind) | `rebind_from_sockname` opt-out (§3.2), landed first |
|
||||||
|
| dial of an unpublished name doesn't normalize | ⚠️ **worse than stated** — `EHOSTUNREACH` is a **bare `OSError`**, not a `ConnectionError` subtype | `_reraise_as_connerr()` is REQUIRED for contract §4, not polish (§3.3) |
|
||||||
|
| a connect-then-drop peer kills the whole actor via `ENOTCONN` from `getpeername()` | ⚠️ **NOT in the original plan; found by our own test harness** | tolerant `getsockname`/`getpeername` helper degrading to a port-id-less addr (§3.4) |
|
||||||
|
| the unwrapped 4-tuple doesn't fit the wire msg-spec | ⚠️ **NOT in the original plan** — `SpawnSpec.reg_addrs`/`.bind_addrs` pinned a 2-tuple | widen to `UnwrappedAddress`, **variadic** `tuple[str\|int, ...]` since `msgspec` refuses a union w/ >1 array-like type. Own commit; first real bite of contract §1.1 |
|
||||||
|
| `SO_ACCEPTCONN` rejected by `AF_TIPC` | ✅ **non-issue** — answers `1` | none needed; pinned by a test anyway |
|
||||||
|
| unregistered `/tipc` multiaddr proto | ✅ handled (interim) | `str` maddr + `parse_maddr()` prefix special-case *before* `Multiaddr()` (§4); upstream track gh #483. Keeps gh #443 blocked |
|
||||||
|
| stale docs (#378 notes tipc.io docs may be out of date) | ✅ still true | treat `include/uapi/linux/tipc.h` + `net/tipc/` as the only normative source; cite file+symbol in code comments |
|
||||||
|
| `SOCK_SEQPACKET` topology framing byte-order | ⏳ open (layer B) | probe helper + `?TODO` (§5.2). Note the event struct is **48B not 40B** and `TIPC_WAIT_FOREVER` is `-1` in python |
|
||||||
|
|
||||||
|
Non-risks worth recording so nobody re-litigates them:
|
||||||
|
|
||||||
|
- **graceful peer close arrives as `BrokenResourceError`
|
||||||
|
/`ECONNRESET`, not a clean 0-byte EOF** like tcp/uds. Benign:
|
||||||
|
`MsgpackTransport._iter_packets()` already `match`es
|
||||||
|
`'Connection reset by peer'` into the `loglevel='transport'`
|
||||||
|
"normal operation breakage" branch, so `TransportClosed`
|
||||||
|
classification is unchanged. Worth a sentence in the docs
|
||||||
|
page (§8) since it *looks* alarming in transport logs.
|
||||||
|
- **`tipc nametable show` really does list our published
|
||||||
|
services** (type `1953628160` == `0x74720000`), so the §8 demo
|
||||||
|
works as advertised.
|
||||||
|
|
||||||
|
## 10. Follow-up issue seeds
|
||||||
|
|
||||||
|
- **register `/tipc` in the multiaddr spec**, mirroring the `wg`
|
||||||
|
track (multiformats/py-multiaddr#107/#108 + gh #483). Same
|
||||||
|
shape of work: propose the proto + code, land a codec in
|
||||||
|
`py-multiaddr`, then drop our `str`-maddr fallback (§4). Worth
|
||||||
|
filing *alongside* the `wg` spec-submission issue so both
|
||||||
|
proposals go up together rather than as one-offs.
|
||||||
|
- registrar-less discovery fast path via name derivation (§5.1)
|
||||||
|
- `TIPC_TOP_SRV`-driven push registry in
|
||||||
|
`discovery/_registry.py` (§5.2)
|
||||||
|
- `TIPC_IMPORTANCE` for the parent<->child lifetime channel
|
||||||
|
(§3.3) — genuinely novel supervision QoS, no other backend
|
||||||
|
can do it
|
||||||
|
- TIPC multicast / group messaging as a *broadcast* transport
|
||||||
|
for `tractor.trionics` fan-out (explicitly not `MsgTransport`)
|
||||||
|
- dual-link resiliency / multi-homing (#378's "hybrid dual link")
|
||||||
|
once bearers are scripted in the docs
|
||||||
|
|
@ -0,0 +1,566 @@
|
||||||
|
# Plan 02 — QUIC backend via `iroh` FFI, uniffi-async rewritten onto `trio`
|
||||||
|
|
||||||
|
Tracks gh [#353]. Prereq reading:
|
||||||
|
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
|
||||||
|
|
||||||
|
**Thesis**: the value of `iroh` over "just QUIC" is
|
||||||
|
`NodeId`-addressed, NAT-traversing, relay-fallback endpoints —
|
||||||
|
i.e. a `tractor` actor tree that spans hosts *without* a
|
||||||
|
reachable listening socket. The cost is that `iroh`'s python
|
||||||
|
surface is `uniffi`-generated **asyncio** and its listener is not
|
||||||
|
a socket. This plan spends its complexity budget in exactly two
|
||||||
|
places: a `trio`-native uniffi future bridge, and a
|
||||||
|
`trio.abc.Listener`/`Stream` adapter pair. Everything else is
|
||||||
|
contract boilerplate.
|
||||||
|
|
||||||
|
[#353]: https://github.com/goodboy/tractor/issues/353
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Library selection (decided, with the rejected alternatives)
|
||||||
|
|
||||||
|
**Chosen: `iroh` (PyPI, from `n0-computer/iroh-ffi`), pinned to
|
||||||
|
a single minor.** The `iroh` python package is a `uniffi`
|
||||||
|
binding over the rust `iroh` crate (QUIC via `quinn`/`noq`).
|
||||||
|
|
||||||
|
Rejected, and why — record these so the next implementer doesn't
|
||||||
|
relitigate:
|
||||||
|
|
||||||
|
- **`aioquic`** (sans-io + asyncio): genuinely trio-portable
|
||||||
|
(`hypercorn` already pairs its sans-io core with a trio UDP
|
||||||
|
server, see the links in #353) and dependency-light. But it
|
||||||
|
gives us *only* QUIC — no NodeId identity, no hole punching,
|
||||||
|
no relay. We'd be reimplementing iroh's whole reason for
|
||||||
|
existing. **Keep as the documented fallback** if the FFI
|
||||||
|
bridge (§2) proves unmaintainable; the `MsgTransport` and
|
||||||
|
`Listener` adapters from §3 are ~90% reusable against an
|
||||||
|
`aioquic` core, which is a deliberate design property of this
|
||||||
|
plan.
|
||||||
|
- **`quiche` / `quinn` via a hand-rolled PyO3 ext**: strictly
|
||||||
|
more work than reusing `iroh-ffi`, and puts us in the
|
||||||
|
build-wheels business.
|
||||||
|
- **`trio-asyncio`**: viable *shortcut* to run the asyncio-shaped
|
||||||
|
bindings under trio, and `tractor` already ships
|
||||||
|
infected-asyncio machinery (`tractor.to_asyncio`,
|
||||||
|
`tests/test_infected_asyncio.py`). Rejected as the *primary*
|
||||||
|
design because it makes every IPC send/recv cross a
|
||||||
|
loop-boundary shim in the hot path, and because #353 asks
|
||||||
|
explicitly for the asyncio support to be "rewritten for trio".
|
||||||
|
**But**: build it first as the throwaway spike (§6 step 0) to
|
||||||
|
de-risk the iroh API surface before writing the bridge.
|
||||||
|
|
||||||
|
Version pinning: `iroh` moves fast and has had breaking
|
||||||
|
API renames across minors. Pin `iroh>=X.Y,<X.Y+1` in a `quic`
|
||||||
|
extra, and **write down the exact resolved version + the
|
||||||
|
generated `iroh/_uniffi*` module layout** in the module
|
||||||
|
docstring, because §2 depends on generated-code internals.
|
||||||
|
|
||||||
|
**Step 0 of implementation is an API-truth pass**: install the
|
||||||
|
pinned `iroh`, `python -c "import iroh; help(iroh)"`, and record
|
||||||
|
in this doc's §1.1 the real names of: endpoint builder, secret
|
||||||
|
key type, `connect`/`accept`, bi-stream open/accept, the
|
||||||
|
send/recv methods and their exact signatures/return types, and
|
||||||
|
whether they're `async def`. Everything below uses *provisional*
|
||||||
|
names and must be reconciled. Do not skip this; do not guess
|
||||||
|
from memory.
|
||||||
|
|
||||||
|
### 1.1 API-truth table (fill in during step 0)
|
||||||
|
|
||||||
|
| concept | provisional name | actual (fill in) |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| secret key | `iroh.SecretKey.generate()` | |
|
||||||
|
| endpoint builder | `iroh.Endpoint.builder(...).bind()` | |
|
||||||
|
| node id | `endpoint.node_id() -> str` | |
|
||||||
|
| node addr (relay + direct) | `iroh.NodeAddr` | |
|
||||||
|
| dial | `await endpoint.connect(node_addr, alpn)` | |
|
||||||
|
| accept conn | `await endpoint.accept()` | |
|
||||||
|
| open bi-stream | `await conn.open_bi()` | |
|
||||||
|
| accept bi-stream | `await conn.accept_bi()` | |
|
||||||
|
| send | `await send_stream.write_all(b)` | |
|
||||||
|
| recv | `await recv_stream.read(n) -> bytes\|None` | |
|
||||||
|
| half-close | `await send_stream.finish()` | |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. The `trio`-native uniffi future bridge (`tractor/ipc/_uniffi_trio.py`)
|
||||||
|
|
||||||
|
### 2.1 what uniffi actually generates
|
||||||
|
|
||||||
|
`uniffi`'s async support does not use asyncio *semantically* —
|
||||||
|
it uses asyncio only as the *executor* for a poll loop. The
|
||||||
|
generated python for an `async fn` is, in shape:
|
||||||
|
|
||||||
|
1. call `_uniffi_..._<method>(...)` → returns an opaque
|
||||||
|
`RustFuture` handle (a `void*`/`u64`).
|
||||||
|
2. loop: call
|
||||||
|
`ffi_..._rust_future_poll_<T>(handle, callback, callback_data)`.
|
||||||
|
The callback is a C-ABI fn pointer invoked **from an
|
||||||
|
arbitrary rust thread** with a poll-result code
|
||||||
|
(`READY`/`MAYBE_READY`).
|
||||||
|
3. the generated glue's callback resolves an
|
||||||
|
`asyncio.Future` via `loop.call_soon_threadsafe(...)`; the
|
||||||
|
coroutine awaits it, then re-polls.
|
||||||
|
4. on ready: `ffi_..._rust_future_complete_<T>(handle,
|
||||||
|
&call_status)` → the value; then
|
||||||
|
`ffi_..._rust_future_free_<T>(handle)`.
|
||||||
|
|
||||||
|
**The asyncio dependency is confined to step 3.** That is the
|
||||||
|
whole insight: the bridge is ~40 lines.
|
||||||
|
|
||||||
|
### 2.2 the trio version
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def await_rust_future(
|
||||||
|
poll: Callable, # ffi_..._rust_future_poll_<T>
|
||||||
|
complete: Callable, # ffi_..._rust_future_complete_<T>
|
||||||
|
free: Callable, # ffi_..._rust_future_free_<T>
|
||||||
|
handle: int,
|
||||||
|
lift: Callable[[Any], Any],
|
||||||
|
) -> Any:
|
||||||
|
'''
|
||||||
|
Drive a `uniffi` rust-future to completion on the current
|
||||||
|
`trio` task, bridging rust-thread wakeups via
|
||||||
|
`TrioToken.run_sync_soon()`.
|
||||||
|
|
||||||
|
'''
|
||||||
|
token = trio.lowlevel.current_trio_token()
|
||||||
|
while True:
|
||||||
|
wake = trio.Event()
|
||||||
|
# NOTE, invoked from a *rust* thread!
|
||||||
|
def _cb(_data, poll_code):
|
||||||
|
token.run_sync_soon(wake.set)
|
||||||
|
|
||||||
|
cb = _UNIFFI_FUTURE_CALLBACK(_cb) # keep a strong ref!
|
||||||
|
poll(handle, cb, 0)
|
||||||
|
await wake.wait()
|
||||||
|
if <poll_code was READY>:
|
||||||
|
break
|
||||||
|
try:
|
||||||
|
status = _UniffiRustCallStatus.default()
|
||||||
|
res = complete(handle, status)
|
||||||
|
_uniffi_check_call_status(status) # reuse generated helper
|
||||||
|
return lift(res)
|
||||||
|
finally:
|
||||||
|
free(handle)
|
||||||
|
```
|
||||||
|
|
||||||
|
Critical details, each a real bug if missed:
|
||||||
|
|
||||||
|
- **`token.run_sync_soon()` is the only trio API callable from a
|
||||||
|
foreign thread**, and it is documented as such. Use it; do
|
||||||
|
*not* use `trio.from_thread.run_sync` (requires a trio thread
|
||||||
|
context) and do not touch the `Event` directly from the
|
||||||
|
callback.
|
||||||
|
- **the poll code must reach the trio side.** Capture it in a
|
||||||
|
`nonlocal`/1-slot list written by the callback *before*
|
||||||
|
`run_sync_soon`, since the callback owns the value. Handle
|
||||||
|
`MAYBE_READY` by re-polling (the loop above does).
|
||||||
|
- **keep the `ctypes` callback object alive** across the await —
|
||||||
|
a GC'd `CFUNCTYPE` trampoline is a segfault. Bind it to a
|
||||||
|
local *and* make sure the local outlives the `poll()` call
|
||||||
|
window.
|
||||||
|
- **cancellation.** `await wake.wait()` is a trio checkpoint, so
|
||||||
|
a `Cancelled` can fire while rust still owns the future. On
|
||||||
|
cancel we must still `free(handle)` — and per uniffi, the
|
||||||
|
correct sequence is to call the generated
|
||||||
|
`ffi_..._rust_future_cancel_<T>(handle)` then continue
|
||||||
|
polling to completion before `free`. Wrap the whole thing so
|
||||||
|
the cancel path does:
|
||||||
|
`with trio.CancelScope(shield=True): cancel(handle); <drain
|
||||||
|
poll loop>; free(handle)`. **Bounded** shield (add a
|
||||||
|
`trio.move_on_after()` with a module-level constant) so a
|
||||||
|
wedged rust future can't make an actor un-cancellable —
|
||||||
|
`tractor` is SC-first and an unbounded shield here would
|
||||||
|
violate that.
|
||||||
|
- **`trio.lowlevel.current_trio_token()`** must be captured on
|
||||||
|
the trio side (not in the callback).
|
||||||
|
|
||||||
|
### 2.3 how to apply it to the generated bindings
|
||||||
|
|
||||||
|
Do **not** fork/vendor the generated `iroh` python. Instead ship
|
||||||
|
a *narrow* re-dispatch shim:
|
||||||
|
|
||||||
|
- write `tractor/ipc/_uniffi_trio.py` with `await_rust_future()`
|
||||||
|
plus a `@cm patch_uniffi_for_trio()` that monkey-patches the
|
||||||
|
generated module's single async-driver entrypoint (in current
|
||||||
|
uniffi that's `_uniffi_rust_call_async` / `_rust_call_async`,
|
||||||
|
one function) to the trio implementation.
|
||||||
|
- verify at import time that the expected symbol exists and
|
||||||
|
raise a clear, actionable error naming the pinned `iroh`
|
||||||
|
version if not. A silent fallback to asyncio would be a
|
||||||
|
nightmare to debug.
|
||||||
|
- **plan for this to break on `iroh`/`uniffi` upgrades.** Mitigate
|
||||||
|
with (a) a unit test that drives one trivial `iroh` async call
|
||||||
|
under bare `trio.run()` and asserts no event loop was ever
|
||||||
|
created (`asyncio.get_event_loop_policy()` untouched /
|
||||||
|
`asyncio._get_running_loop() is None`), and (b) a docstring
|
||||||
|
pointing at the uniffi codegen template this mirrors.
|
||||||
|
|
||||||
|
If step 0 reveals the generated code is *structurally* hostile
|
||||||
|
to this (e.g. `asyncio` imported and used at module scope for
|
||||||
|
more than the driver), fall back to option (b): run iroh under
|
||||||
|
`tractor.to_asyncio` infected mode and open the follow-up to
|
||||||
|
revisit. Say so in the PR rather than fighting it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Mapping QUIC onto `MsgTransport`
|
||||||
|
|
||||||
|
### 3.1 the layering decision
|
||||||
|
|
||||||
|
QUIC natively multiplexes streams inside one connection. The
|
||||||
|
mapping that preserves *all* existing `tractor` semantics with
|
||||||
|
the least new code:
|
||||||
|
|
||||||
|
```
|
||||||
|
iroh Endpoint == one per actor (process) -> the "listener"
|
||||||
|
iroh Connection == one per peer actor -> pooled
|
||||||
|
iroh bi-stream == one `Channel`/`MsgTransport` -> 1:1
|
||||||
|
```
|
||||||
|
|
||||||
|
- keep the 4-byte `<I` length-prefix framing **unchanged**. It's
|
||||||
|
redundant-ish over a QUIC stream but it means
|
||||||
|
`MsgpackTransport` is reused verbatim, and framing is cheap.
|
||||||
|
Revisit only after it works.
|
||||||
|
- **one-task-per-stream** falls out naturally, which is exactly
|
||||||
|
the #353 note about QUIC sub-stream QoS/cancellation fitting
|
||||||
|
`trio`.
|
||||||
|
- `layer_key: int = 4` still (QUIC is L4-ish); note in a comment
|
||||||
|
that this backend is really 4+security+multiplex.
|
||||||
|
|
||||||
|
**Connection pooling** is the one place we add state the other
|
||||||
|
backends don't have: dialing the same peer twice should reuse
|
||||||
|
the `Connection` and open a second bi-stream. Implement as a
|
||||||
|
module-level `dict[NodeId, Connection]` guarded by a
|
||||||
|
`trio.Lock`... **no** — that's a per-process cache with
|
||||||
|
lifetime/teardown hazards. Instead reuse the codebase's existing
|
||||||
|
idiom: `tractor.trionics.maybe_open_context()` keyed on the
|
||||||
|
node-id, which already solves exactly this (one-cached-resource-
|
||||||
|
per-key, refcounted, teardown-on-last-exit) and whose teardown
|
||||||
|
semantics were just hardened (gh #488). Use it; do not hand-roll
|
||||||
|
a cache. Anything concurrency-subtle here should get the
|
||||||
|
`conc-anal` skill run over it.
|
||||||
|
|
||||||
|
### 3.2 `IrohAddress`
|
||||||
|
|
||||||
|
```python
|
||||||
|
class IrohAddress(
|
||||||
|
msgspec.Struct,
|
||||||
|
frozen=True,
|
||||||
|
):
|
||||||
|
_node_id: str # 32B ed25519 pubkey, hex or z32
|
||||||
|
_alpn: str = 'tractor/0' # the bindspace!
|
||||||
|
# optional dial hints; NOT part of identity
|
||||||
|
maybe_relay_url: str|None = None
|
||||||
|
maybe_direct_addrs: tuple[str, ...] = ()
|
||||||
|
|
||||||
|
proto_key: ClassVar[str] = 'iroh' # ?or 'quic'; see §3.2.1
|
||||||
|
unwrapped_type: ClassVar[type] = tuple[str, str]
|
||||||
|
def_bindspace: ClassVar[str] = 'tractor/0'
|
||||||
|
```
|
||||||
|
|
||||||
|
- **`.unwrap() -> (node_id_str, alpn_str)`** — a `(str, str)`
|
||||||
|
tuple, which is *unambiguously distinct* from
|
||||||
|
`TCPAddress`'s `(str, int)`. But careful:
|
||||||
|
`wrap_address()`'s UDS case is
|
||||||
|
`case (_, filename) if type(filename) is str` — which
|
||||||
|
**already catches `(str, str)`**. So the iroh `case` MUST be
|
||||||
|
ordered *before* the UDS case and guarded, e.g.
|
||||||
|
`case (str() as nid, str() as alpn) if _is_node_id(nid):`
|
||||||
|
with `_is_node_id()` a cheap length+alphabet check. Add a
|
||||||
|
regression test asserting a UDS `(dir, filename)` pair still
|
||||||
|
wraps to `UDSAddress` — this is the exact "wrong transport
|
||||||
|
loaded" hazard `_addr.py:214` warns about.
|
||||||
|
- `.bindspace` → `self._alpn`. This is the honest analogue:
|
||||||
|
the ALPN is the set of endpoints willing to talk to you, and
|
||||||
|
two `tractor` deployments sharing an iroh network are
|
||||||
|
separated by ALPN exactly as two UDS deployments are
|
||||||
|
separated by directory. Include a `tractor` version/proto
|
||||||
|
epoch in the default ALPN so incompatible runtimes can't
|
||||||
|
handshake.
|
||||||
|
- `.is_valid` → node-id parses, alpn non-empty.
|
||||||
|
- **`get_root()` is the hard one.** There is no
|
||||||
|
well-known-port analogue: an iroh node id is a *keypair*, so
|
||||||
|
"the host's default registrar addr" requires a *persisted
|
||||||
|
secret key*. Design:
|
||||||
|
- the root/registrar's secret key lives at
|
||||||
|
`get_rt_dir() / 'iroh_registrar.key'` (0600), created on
|
||||||
|
first use.
|
||||||
|
- `get_root()` must stay **pure and import-time-safe**
|
||||||
|
(contract §2.3: `_default_lo_addrs` is built at import!).
|
||||||
|
So `get_root()` *reads* the key file if present and
|
||||||
|
otherwise returns an `IrohAddress` with
|
||||||
|
`_node_id=''`/sentinel, and the **generation** happens in
|
||||||
|
an explicit sibling — `ensure_registrar_key() ->
|
||||||
|
IrohAddress` — called from the listen path. Pure getter,
|
||||||
|
explicit setter; do not smuggle key generation into
|
||||||
|
`get_root()`.
|
||||||
|
- this almost certainly means `_default_lo_addrs` must become
|
||||||
|
lazy for this backend. **Land that refactor as its own prep
|
||||||
|
commit** (a `default_lo_addrs()` that computes per-call
|
||||||
|
instead of the import-time dict) — it also unblocks plan
|
||||||
|
03's netns-scoped defaults.
|
||||||
|
- `get_random()`: generate a fresh `SecretKey` per subactor and
|
||||||
|
return its node-id. Note this runs post-fork pre-listen
|
||||||
|
(contract §4) and costs an ed25519 keygen (~µs, fine). The
|
||||||
|
*secret* can't live in a frozen `Address`, so it must be
|
||||||
|
stashed where the listen path can find it: a module-level
|
||||||
|
`dict[node_id, SecretKey]` populated by `get_random()` and
|
||||||
|
consumed+popped by `start_listener()`. Ugly but honest;
|
||||||
|
document it and note the alternative (thread the key through
|
||||||
|
`Endpoint`) as a follow-up.
|
||||||
|
|
||||||
|
#### 3.2.1 `proto_key`: `'iroh'` vs `'quic'`
|
||||||
|
|
||||||
|
Use **`'quic'`** for the `proto_key`/`--tpt-proto` name and
|
||||||
|
name the module `_quic.py`, with `iroh` as the *implementation*.
|
||||||
|
Rationale: it keeps the door open for the `aioquic` fallback
|
||||||
|
(§1) without a user-visible rename, and it matches how `uds` is
|
||||||
|
a proto name rather than a lib name. Put `iroh`-specific bits
|
||||||
|
behind an internal `_iroh` submodule if the file gets big.
|
||||||
|
|
||||||
|
### 3.3 the `trio.abc` adapters — where the real work is
|
||||||
|
|
||||||
|
Contract §3 says a non-socket backend needs three upstream
|
||||||
|
generalizations. Land them **as a prep PR, before any iroh
|
||||||
|
code**, so they can be reviewed on their own merits with
|
||||||
|
tcp/uds still the only backends:
|
||||||
|
|
||||||
|
1. **`Endpoint.start_listener()` must not assume
|
||||||
|
`.socket.getsockname()`.** Use the same
|
||||||
|
`Address.rebind_from_sockname: ClassVar[bool]` gate that
|
||||||
|
plan 01 §3.2 introduces — coordinate so it lands once. (If
|
||||||
|
plan 01 lands first, this is free.)
|
||||||
|
2. **`transport_from_stream()` (`_types.py:92`) must not assume
|
||||||
|
`trio.SocketStream`.** Replace the `sock.family` match with:
|
||||||
|
check `isinstance(stream, trio.SocketStream)` → existing
|
||||||
|
family match; else look for a
|
||||||
|
`stream.tpt_key: ClassVar[MsgTransportKey]` attribute on the
|
||||||
|
adapter and use it. Keeps the existing path byte-identical
|
||||||
|
and makes new stream types self-describing (a much better
|
||||||
|
shape than growing an `isinstance` ladder).
|
||||||
|
3. **type annotations**: `handle_stream_from_peer(stream:
|
||||||
|
trio.SocketStream)` → `trio.abc.Stream`; `Endpoint._listener:
|
||||||
|
SocketListener|None` → `trio.abc.Listener|None`;
|
||||||
|
`MsgTransport.stream: trio.SocketStream` →
|
||||||
|
`trio.abc.Stream`. Annotation-only, zero behaviour change.
|
||||||
|
|
||||||
|
Then the adapters:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class QuicMsgStream(trio.abc.HalfCloseableStream):
|
||||||
|
'''
|
||||||
|
A single `iroh` bi-directional QUIC stream presented as
|
||||||
|
a `trio` byte-stream so `MsgpackTransport` can frame over
|
||||||
|
it unmodified.
|
||||||
|
|
||||||
|
'''
|
||||||
|
tpt_key: ClassVar[MsgTransportKey] = ('msgpack', 'quic')
|
||||||
|
|
||||||
|
def __init__(self, conn, send, recv) -> None: ...
|
||||||
|
async def send_all(self, data: bytes) -> None: ...
|
||||||
|
async def wait_send_all_might_not_block(self) -> None: ...
|
||||||
|
async def receive_some(self, max_bytes: int|None = None) -> bytes: ...
|
||||||
|
async def send_eof(self) -> None: ...
|
||||||
|
async def aclose(self) -> None: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Non-negotiable behaviours (each maps to a `match` case that
|
||||||
|
already exists in `_transport.py` and must keep working):
|
||||||
|
|
||||||
|
- `receive_some()` returns `b''` at clean EOF →
|
||||||
|
`MsgpackTransport._iter_packets()` sees `header == b''` and
|
||||||
|
raises `TransportClosed(loglevel='transport')`. **This is the
|
||||||
|
graceful-disconnect path the whole runtime relies on**; get it
|
||||||
|
right first.
|
||||||
|
- a reset/aborted stream → raise `trio.BrokenResourceError`.
|
||||||
|
- use after local close → raise `trio.ClosedResourceError`
|
||||||
|
(ideally with `'another task closed this fd'`-equivalent text
|
||||||
|
absent, so the `raise_on_report` branch at
|
||||||
|
`_transport.py:290` stays quiet).
|
||||||
|
- `send_all()` on a closed peer → `trio.BrokenResourceError`.
|
||||||
|
- honour `trio`'s one-task-per-direction rule: guard with
|
||||||
|
`trio._util.ConflictDetector` equivalents (or just document +
|
||||||
|
assert), because `MsgpackTransport` already serializes sends
|
||||||
|
with a `StrictFIFOLock` but recvs are single-task by
|
||||||
|
construction.
|
||||||
|
- **buffering**: if iroh's `read()` doesn't support
|
||||||
|
"read up to n", `receive_some()` must maintain an internal
|
||||||
|
leftover buffer. Note `MsgpackTransport` wraps us in
|
||||||
|
`tricycle.BufferedReceiveStream` anyway, so `receive_some()`
|
||||||
|
just needs *some* nonzero-progress contract.
|
||||||
|
|
||||||
|
```python
|
||||||
|
class QuicListener(trio.abc.Listener):
|
||||||
|
'''
|
||||||
|
Accepts iroh `Connection`s and yields one `QuicMsgStream`
|
||||||
|
per accepted bi-stream, so `trio.serve_listeners()` spawns
|
||||||
|
one `handle_stream_from_peer()` per `Channel`.
|
||||||
|
|
||||||
|
'''
|
||||||
|
async def accept(self) -> QuicMsgStream: ...
|
||||||
|
async def aclose(self) -> None: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
The accept-side subtlety: `trio.abc.Listener.accept()` yields
|
||||||
|
one stream per call, but iroh gives us *connections* which then
|
||||||
|
yield *streams*. So `QuicListener` needs an internal
|
||||||
|
`trio.MemoryReceiveChannel[QuicMsgStream]` fed by a background
|
||||||
|
task-pair (one task accepting connections, one per connection
|
||||||
|
accepting bi-streams). `trio.abc.Listener` has no nursery, so:
|
||||||
|
make the listener **constructed by an `@acm`** that owns the
|
||||||
|
nursery, and have `start_listener()` be that `@acm`'s driver.
|
||||||
|
|
||||||
|
⚠️ this collides with `Endpoint.start_listener()` being a plain
|
||||||
|
`async def` returning a listener. Two options:
|
||||||
|
- **(a)** hang the nursery off the `Endpoint`'s existing
|
||||||
|
`listen_tn` — `_serve_ipc_eps()` already creates `listen_tn`
|
||||||
|
and passes it into every `Endpoint` (`_server.py:1063-1074`),
|
||||||
|
and `Endpoint.listen_tn` is right there. So
|
||||||
|
`start_listener()` can `self.listen_tn.start_soon(...)` the
|
||||||
|
acceptor tasks. **Recommended**: no upstream signature change,
|
||||||
|
correct lifetime (dies with the ep group), and it's why
|
||||||
|
`listen_tn` is on the struct in the first place.
|
||||||
|
- (b) change `start_listener()` to a `@acm`. Bigger blast
|
||||||
|
radius; only if (a) proves insufficient.
|
||||||
|
|
||||||
|
Since `start_listener()` is called via
|
||||||
|
`inspect.getmodule(addr)` with only `addr=` (contract §1.3),
|
||||||
|
option (a) needs the `Endpoint` itself. Either add `ep=` to the
|
||||||
|
module-level `start_listener()` call signature (all backends
|
||||||
|
ignore it except quic → small upstream change, do it as part of
|
||||||
|
the prep PR and make it keyword-only with a default) or have
|
||||||
|
`QuicListener.accept()` lazily spawn via
|
||||||
|
`trio.lowlevel.current_task().parent_nursery` (**rejected** —
|
||||||
|
fragile, implicit). Do the explicit `ep=` kwarg.
|
||||||
|
|
||||||
|
### 3.4 `maddr`
|
||||||
|
|
||||||
|
Multiaddr already standardizes the pieces:
|
||||||
|
|
||||||
|
```
|
||||||
|
/ip4/<h>/udp/<p>/quic-v1 # direct
|
||||||
|
/ip4/<h>/udp/<p>/quic-v1/p2p/<node-id> # direct + identity
|
||||||
|
/dns/<relay-host>/tcp/443/tls/ws/p2p/<node> # relay-ish
|
||||||
|
```
|
||||||
|
|
||||||
|
- primary form: `/p2p/<node-id>` alone is a legal maddr and is
|
||||||
|
the *only* required component for iroh dialling — relay +
|
||||||
|
direct addrs are discovery hints. So `mk_maddr()` emits
|
||||||
|
`/p2p/<node_id>` and, when known, prefixes the direct
|
||||||
|
`/ip4/../udp/../quic-v1/`.
|
||||||
|
- `/p2p/` values are multihash-encoded peer ids; an iroh node-id
|
||||||
|
is a raw ed25519 key. Converting requires the identity
|
||||||
|
multihash + libp2p key protobuf wrapper. **Decide**: emit the
|
||||||
|
raw node-id under a *tractor-local* `/iroh/<node-id>` segment
|
||||||
|
(needs upstream registration, same track as `wg`/`tipc`,
|
||||||
|
gh #483) rather than pretending to be a libp2p peer-id we
|
||||||
|
can't round-trip. Return the `str` form until upstream lands
|
||||||
|
(`MsgTransport.maddr` is `Multiaddr|str`).
|
||||||
|
- this backend is the strongest argument for gh #443's
|
||||||
|
**tunnelled/composed maddr** item: `/ip4/../udp/../quic-v1/..`
|
||||||
|
*is* a composed stack. Cross-reference plan 03 §5 so the two
|
||||||
|
grammars land compatibly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Discovery integration
|
||||||
|
|
||||||
|
- iroh's node-id addressing means the `tractor` registrar can
|
||||||
|
hold `IrohAddress`es that are **reachable from anywhere** with
|
||||||
|
no port-forwarding — that is the headline feature. The
|
||||||
|
registrar itself works unchanged.
|
||||||
|
- iroh has its own discovery (DNS/pkarr/mdns). **Out of scope**;
|
||||||
|
note in the follow-up that `tractor.discovery` could
|
||||||
|
eventually delegate to it, which would be the direct analogue
|
||||||
|
of plan 01's TIPC-topology idea.
|
||||||
|
- relay servers: default to n0's public relays for the demo,
|
||||||
|
document self-hosting (docs.iroh.computer's dedicated-infra
|
||||||
|
page is linked from #353), and make the relay set a
|
||||||
|
`start_listener()` kwarg.
|
||||||
|
|
||||||
|
## 5. Security note
|
||||||
|
|
||||||
|
QUIC is TLS-1.3-always and iroh authenticates by node-id, so
|
||||||
|
this backend is the first `tractor` transport with real
|
||||||
|
transport security and peer authentication. Two things follow:
|
||||||
|
1. an **allowlist hook** — an actor should be able to reject
|
||||||
|
inbound connections from unknown node-ids *before* the
|
||||||
|
`Aid` handshake. Natural home: a predicate kwarg on
|
||||||
|
`start_listener()`, evaluated in `QuicListener`'s connection
|
||||||
|
acceptor task. Sketch it; ship it in PR 1 if cheap (it is).
|
||||||
|
2. do **not** claim any security property for the other
|
||||||
|
backends by association. `tcp`/`uds`/`tipc` remain
|
||||||
|
unauthenticated; that's what plan 03 (wg) is for.
|
||||||
|
|
||||||
|
## 6. Commit sequencing
|
||||||
|
|
||||||
|
0. **spike (throwaway, not committed)**: drive iroh under
|
||||||
|
`trio-asyncio`/`tractor.to_asyncio`, echo bytes over a
|
||||||
|
bi-stream between two procs. Fills in §1.1. Timebox it.
|
||||||
|
1. prep PR: annotation widening + `rebind_from_sockname` gate +
|
||||||
|
`transport_from_stream()` `tpt_key` dispatch + `ep=` kwarg on
|
||||||
|
`start_listener()` + lazy `default_lo_addrs()`. **No new
|
||||||
|
backend.** Full suite green on tcp *and* uds.
|
||||||
|
2. `_uniffi_trio.py` + its tests (drive one iroh async call
|
||||||
|
under bare `trio.run()`; assert no asyncio loop; assert
|
||||||
|
cancellation frees the future).
|
||||||
|
3. `QuicMsgStream` + tests against a *loopback* iroh endpoint
|
||||||
|
pair in one process (no `tractor` runtime): send/recv, clean
|
||||||
|
EOF → `b''`, reset → `BrokenResourceError`, use-after-close
|
||||||
|
→ `ClosedResourceError`.
|
||||||
|
4. `QuicListener` + `start_listener()` + `IrohAddress` +
|
||||||
|
key-file mgmt.
|
||||||
|
5. `MsgpackQuicStream(MsgpackTransport)` + `connect_to()` +
|
||||||
|
`maybe_open_context()` connection pooling.
|
||||||
|
6. registration tables + `--tpt-proto quic` + full suite.
|
||||||
|
7. maddr + docs + a two-host example (pairs with #482's format).
|
||||||
|
|
||||||
|
## 7. Testing
|
||||||
|
|
||||||
|
- capability predicate `is_quic_available()` → `iroh` importable
|
||||||
|
*and* the uniffi driver symbol present at the pinned version.
|
||||||
|
Same `pytest.fail`-early hook as plan 01 §7.2.
|
||||||
|
- **the acceptance bar is the same**: whole suite green under
|
||||||
|
`--tpt-proto quic`. Expect this to shake out real bugs in the
|
||||||
|
adapters (esp. teardown ordering and `TransportClosed`
|
||||||
|
classification) — that's the point.
|
||||||
|
- expect to need **timeout headroom**: iroh endpoint bind +
|
||||||
|
first connect (relay discovery) is orders of magnitude slower
|
||||||
|
than a UDS bind. Before touching any test deadline, rule out
|
||||||
|
the CPU-throttle false-positive (see the project's
|
||||||
|
`env_cpu_throttle_masquerades_as_regression` note); then, if
|
||||||
|
real, add a per-proto timeout multiplier to the test harness
|
||||||
|
rather than editing individual tests.
|
||||||
|
- a no-network test mode: iroh with relays disabled +
|
||||||
|
loopback direct addrs only, so CI doesn't depend on n0's
|
||||||
|
infra. **Make this the default in CI**; mark the relay tests
|
||||||
|
`pytest.mark.net` and keep them out of the default run.
|
||||||
|
- leak checks: assert every `SecretKey`/`Endpoint` is closed on
|
||||||
|
actor teardown (an `Endpoint` left open holds UDP sockets and
|
||||||
|
relay connections; a leak here shows up as hung tests, not
|
||||||
|
errors).
|
||||||
|
|
||||||
|
## 8. Risks
|
||||||
|
|
||||||
|
| risk | mitigation |
|
||||||
|
| --- | --- |
|
||||||
|
| uniffi codegen internals shift on upgrade | pinned minor, symbol assertion at import, the "no asyncio loop" test, documented fallback to `to_asyncio` |
|
||||||
|
| rust-thread callback → trio wakeup mishandled (segfault / lost wakeup / un-cancellable task) | strong ref on the ctypes trampoline; `run_sync_soon` only; **bounded** shielded cancel-drain; run the `conc-anal` skill over the bridge |
|
||||||
|
| `iroh` wheel availability for 3.13/3.14 on linux+macos | verify in step 0; if missing, that alone may force the `aioquic` fallback |
|
||||||
|
| QUIC latency/jitter destabilizes the existing suite's timing assumptions | per-proto timeout multiplier, relay-less CI mode |
|
||||||
|
| `(str, str)` unwrapped form collides with UDS in `wrap_address()` | guarded case ordered first + explicit regression test (§3.2) |
|
||||||
|
| scope creep into iroh's docs/blobs/gossip crates | this backend is `Endpoint`+`Connection`+bi-streams only; anything else is a separate issue |
|
||||||
|
|
||||||
|
## 9. Follow-up issue seeds
|
||||||
|
|
||||||
|
- `tractor.discovery` delegating to iroh discovery (DNS/pkarr/mdns)
|
||||||
|
- per-`Context` QUIC sub-streams: today one `Channel` == one
|
||||||
|
stream; QUIC would let each `tractor.Context` own its own
|
||||||
|
stream with independent flow-control and cancellation — this
|
||||||
|
is the genuinely novel win #353 gestures at, and it's a
|
||||||
|
runtime-layer change, not a transport one
|
||||||
|
- unreliable QUIC datagrams for a lossy-ok broadcast transport
|
||||||
|
(pairs with plan 01's TIPC-multicast seed)
|
||||||
|
- node-id allowlist → a real `tractor` authz story
|
||||||
|
- `aioquic` sans-io backend reusing §3's adapters
|
||||||
|
|
@ -0,0 +1,464 @@
|
||||||
|
# Plan 03 — WireGuard (and other tunnels) as a *nested bindspace* via `pyroute2`
|
||||||
|
|
||||||
|
Tracks gh [#482] + the tunnelled-maddr item of [#443].
|
||||||
|
Prereq reading:
|
||||||
|
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
|
||||||
|
|
||||||
|
**Thesis**: WireGuard is **not** a `MsgTransport`. It is an
|
||||||
|
interface-layer tunnel that is transparent to `socket(2)`, so
|
||||||
|
the correct abstraction is a *bindspace* — a scoped,
|
||||||
|
`@acm`-managed network context that an existing L4 transport
|
||||||
|
(`tcp`, and later `quic`/`tipc`-over-UDP-bearer) binds *inside*.
|
||||||
|
This plan implements `Address.namespace` (spec'd but unused
|
||||||
|
since day one) and the composed/tunnelled maddr grammar, with
|
||||||
|
`pyroute2` as the netlink codec and as much of the I/O moved
|
||||||
|
onto `trio` as the library's sans-io layer allows.
|
||||||
|
|
||||||
|
[#482]: https://github.com/goodboy/tractor/issues/482
|
||||||
|
[#443]: https://github.com/goodboy/tractor/issues/443
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. What exists today (verified, per #482)
|
||||||
|
|
||||||
|
- `wrap_address()` accepts maddr `str`s (leading-`/` dispatch,
|
||||||
|
`_addr.py:262`) but `parse_maddr()` only knows
|
||||||
|
`/ip4|ip6/<h>/tcp/<p>` and `/unix/<p>`; a `.../wg/u<key>`
|
||||||
|
maddr raises `ValueError('Unsupported multiaddr protocol
|
||||||
|
combo')`.
|
||||||
|
- there is no `wg` proto in the multiaddr *spec* yet, but
|
||||||
|
multiformats/py-multiaddr#108 (key form `u<base64url>`) is
|
||||||
|
**merged** as of 2026-07-28 (`f86519da`) — and unreleased, the
|
||||||
|
latest `0.2.0` predating it. Spec registration is still tracked
|
||||||
|
by multiformats/py-multiaddr#107 and gh #483.
|
||||||
|
- so **today's deployable story is declarative**: run `wg-quick`
|
||||||
|
out-of-band, parse the maddr, strip to the inner
|
||||||
|
`(host, port)`, verify the pubkey against the live tunnel,
|
||||||
|
hand the inner addr to `registry_addrs=`/`tpt_bind_addrs=`.
|
||||||
|
#482 already contains working example code for exactly this.
|
||||||
|
- `Address.namespace` exists in the Protocol
|
||||||
|
(`_addr.py:94-101`, "the if-available OS-specific network
|
||||||
|
namespace key") and **no backend implements it**. This plan is
|
||||||
|
its first consumer.
|
||||||
|
|
||||||
|
## 2. Three layers, three PRs
|
||||||
|
|
||||||
|
| layer | what | dep | ships |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| **A. declarative** | commit #482's examples; `parse_maddr()` learns `/wg/u<key>` → inner `Address` + verified pubkey | `multiaddr` (already), `wg(8)` CLI | first |
|
||||||
|
| **B. `pyroute2` read/verify** | replace the `subprocess.run(['sudo','wg','show'])` shelling with netlink queries | `pyroute2` extra | second |
|
||||||
|
| **C. `@acm` lifecycle** | create/configure/tear down wg ifaces + netns *from the runtime*, as nested bindspaces; implement `Address.namespace` | `pyroute2` + `CAP_NET_ADMIN` | third |
|
||||||
|
|
||||||
|
Each is independently valuable and independently reviewable.
|
||||||
|
**Do not attempt C first** — the interesting design (nested
|
||||||
|
bindspace `@acm`s) is only well-posed once A has pinned the
|
||||||
|
address grammar and B has proven the netlink path under trio.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Layer A — declarative `wg` maddrs
|
||||||
|
|
||||||
|
### 3.1 the address shape
|
||||||
|
|
||||||
|
The decision: **a wg segment annotates an existing address, it
|
||||||
|
does not create a new address type.** Two candidate encodings;
|
||||||
|
**pick (a)**:
|
||||||
|
|
||||||
|
- **(a) `TunnelledAddress` wrapper** (recommended):
|
||||||
|
```python
|
||||||
|
class TunnelledAddress(
|
||||||
|
msgspec.Struct,
|
||||||
|
frozen=True,
|
||||||
|
):
|
||||||
|
inner: Address # e.g. TCPAddress
|
||||||
|
tunnel: WGTunnelSpec # proto-specific, frozen
|
||||||
|
```
|
||||||
|
with `.proto_key` **delegating to `inner.proto_key`** so every
|
||||||
|
existing table lookup (`_addr_to_transport`,
|
||||||
|
`enable_transports` guard at `_root.py:391`,
|
||||||
|
`transport_from_addr()`) keeps working untouched, and
|
||||||
|
`.unwrap()` delegating to `inner.unwrap()` so **nothing new
|
||||||
|
crosses the wire**. `.namespace` and `.bindspace` come from
|
||||||
|
the tunnel spec. The wrapper is stripped (`→ .inner`) at the
|
||||||
|
moment of bind/connect.
|
||||||
|
- ⚠️ `is_wrapped_addr()` (`_addr.py:194`) tests
|
||||||
|
`type(addr) in _address_types.values()` — a `bidict` of
|
||||||
|
proto_key→type. `TunnelledAddress` isn't in it and must not
|
||||||
|
be (it's not 1:1 with a proto). So either add an explicit
|
||||||
|
`isinstance(addr, TunnelledAddress)` clause there, or give
|
||||||
|
the wrapper a marker and test structurally. Do the former;
|
||||||
|
it's two lines and honest.
|
||||||
|
- the reflection in `Endpoint.start_listener()`
|
||||||
|
(`inspect.getmodule(self.addr)`) would resolve to the
|
||||||
|
*wrapper's* module, not the transport's. **So the wrapper
|
||||||
|
must be unwrapped before it reaches `Endpoint`** — i.e. by
|
||||||
|
the bindspace `@acm` (layer C) or by `parse_maddr()`
|
||||||
|
(layer A). State this loudly in the docstring; it's the #1
|
||||||
|
way to get this wrong.
|
||||||
|
- (b) add fields to each existing `Address` type. Rejected:
|
||||||
|
duplicates tunnel logic per-backend and pollutes `.unwrap()`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
class WGTunnelSpec(
|
||||||
|
msgspec.Struct,
|
||||||
|
frozen=True,
|
||||||
|
):
|
||||||
|
peer_pubkey: str # std-base64 `wg(8)` form
|
||||||
|
iface: str = 'wg0'
|
||||||
|
netns: str|None = None
|
||||||
|
# layer-C-only fields, unset in layer A
|
||||||
|
maybe_endpoint: tuple[str, int]|None = None
|
||||||
|
maybe_allowed_ips: tuple[str, ...] = ()
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.2 `parse_maddr()`/`mk_maddr()`
|
||||||
|
|
||||||
|
Grammar — **verified** against py-multiaddr#108, first on the
|
||||||
|
`baudco/py-multiaddr@wg_support` branch and re-verified after it
|
||||||
|
merged upstream (`multiformats/py-multiaddr@f86519da`); all three
|
||||||
|
forms below parse *and* round-trip. Note the codec also validates
|
||||||
|
that the key decodes to exactly 32 bytes, so a truncated key is a
|
||||||
|
`StringParseError`, not a silently-mangled parse:
|
||||||
|
|
||||||
|
```
|
||||||
|
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
|
||||||
|
\_______ bearer __________/\__ key __/\______ overlay ______/
|
||||||
|
underlay, wg `ListenPort` the ONLY part we bind
|
||||||
|
```
|
||||||
|
|
||||||
|
The `/wg/` segment is **infix, not suffix** — the segments
|
||||||
|
*before* it are the wg **bearer** (the underlay `(ip, udp-port)`
|
||||||
|
that `wg(8)` itself listens on, per the codec docstring's own
|
||||||
|
`/ip4/1.2.3.4/udp/51820/wg/{key}` example), and the segments
|
||||||
|
*after* are the **overlay** endpoint that `tractor` binds.
|
||||||
|
|
||||||
|
⚠️ **CORRECTION** — an earlier revision of this plan (and the
|
||||||
|
examples in gh #482) used a *suffix* form
|
||||||
|
`/ip4/10.0.11.1/tcp/1616/wg/u<key>`. That parses, but it is
|
||||||
|
semantically inverted: it puts the overlay addr where the bearer
|
||||||
|
belongs, `tcp` where wg's `udp` `ListenPort` goes, and declares
|
||||||
|
no overlay endpoint at all. `parse_wg_maddr()` in
|
||||||
|
`examples/multihost/wg_lan/` now rejects it with an actionable
|
||||||
|
error.
|
||||||
|
Observed protocol-name lists, for writing the `match`:
|
||||||
|
|
||||||
|
| maddr | `[p.name for p in m.protocols()]` |
|
||||||
|
| --- | --- |
|
||||||
|
| `/ip4/1.2.3.4/udp/51820/wg/u<k>` | `['ip4','udp','wg']` |
|
||||||
|
| `/ip4/../udp/../wg/u<k>/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` |
|
||||||
|
|
||||||
|
- so the three parts have **three different owners**, and only the
|
||||||
|
third is an `Endpoint`:
|
||||||
|
|
||||||
|
| part | bound by | in the runtime? |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| bearer | kernel, via `wg-quick`/`pyroute2` | no |
|
||||||
|
| `/wg/u<key>` | nothing — it's an identity | no, verified out-of-band |
|
||||||
|
| overlay | `tractor`'s `IPCServer` | **yes**, as `.inner` |
|
||||||
|
|
||||||
|
This owner-split is the real axis of the design, *not* whether
|
||||||
|
the maddr stack is "composed" (it is).
|
||||||
|
- `parse_maddr()` gains a case on
|
||||||
|
`[('ip4'|'ip6'), 'udp', 'wg', ('ip4'|'ip6'), <inner-l4>]` →
|
||||||
|
build the inner `Address` from the trailing segments, decode
|
||||||
|
the multibase key to std-base64, and return
|
||||||
|
`TunnelledAddress(inner=..., tunnel=WGTunnelSpec(...))` with
|
||||||
|
the bearer recorded in the spec.
|
||||||
|
- keep the existing 2-proto cases byte-identical; add the new
|
||||||
|
case *after* them.
|
||||||
|
- generalize by **peeling at the tunnel segment**: split
|
||||||
|
`proto_names` at `'wg'`, hand the trailing list to the existing
|
||||||
|
inner-stack logic, and recurse for nested tunnels. Write it as
|
||||||
|
a small pure fn `_peel_tunnel_segs(proto_names) ->
|
||||||
|
(bearer_names, tunnel_specs, inner_names)`. This is also what
|
||||||
|
makes a wg-inside-wg stack fall out for free.
|
||||||
|
- `mk_maddr()` inverse for `TunnelledAddress`.
|
||||||
|
- **pending an upstream release**: py-multiaddr#108 is merged, so
|
||||||
|
`Multiaddr('/…/wg/u…')` parses — but off a `[tool.uv.sources]`
|
||||||
|
`rev` pin, since no release carries the codec. Gate the tests
|
||||||
|
on `_have_wg_maddr_proto()`, implemented as
|
||||||
|
`protocols.protocol_with_name('wg')` under
|
||||||
|
`except ProtocolNotFoundError`. Do **not** probe by parsing a
|
||||||
|
dummy like `Multiaddr('/wg/uAAAA')` — the codec enforces a
|
||||||
|
32-byte key, so that raises even when the proto *is* known. Do
|
||||||
|
**not** hand-roll a `wg` parser in `tractor` — the whole point
|
||||||
|
of #429 was dropping the NIH parser.
|
||||||
|
|
||||||
|
### 3.3 verification helper (pure, composable)
|
||||||
|
|
||||||
|
Port #482 §2's helpers into `tractor/discovery/_tunnel.py` as
|
||||||
|
*pure functions* + one impure probe, cleanly separated:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def parse_wg_maddr(maddr: str) -> TunnelledAddress: ... # pure
|
||||||
|
def wg8_pubkey(multibase_key: str) -> str: ... # pure
|
||||||
|
def verify_wg_peer(spec: WGTunnelSpec) -> bool: ... # impure probe
|
||||||
|
```
|
||||||
|
|
||||||
|
In layer A `verify_wg_peer()` may shell out (`wg show <if>
|
||||||
|
peers`), but it must be a *single* function so layer B swaps
|
||||||
|
only its body. Never call it implicitly from
|
||||||
|
`wrap_address()`/`parse_maddr()` — parsing must stay pure and
|
||||||
|
side-effect-free; verification is the *caller's* explicit step
|
||||||
|
(and later, the bindspace `@acm`'s).
|
||||||
|
|
||||||
|
### 3.4 deliverables
|
||||||
|
|
||||||
|
- `examples/` scripts distilled from #482 §§3-5 (this is the
|
||||||
|
unchecked "commit examples from ^" bullet in #443). They live
|
||||||
|
under `examples/multihost/` — `test_docs_examples.py` walks
|
||||||
|
`examples/` recursively and runs every collected file as a
|
||||||
|
subproc asserting `rc == 0` (it doesn't even filter by
|
||||||
|
extension, so a stray `README.md` would be `python`-run too),
|
||||||
|
and `'multihost' not in p[0]` is already in its exclusion
|
||||||
|
list. Anything needing a real second host or a live tunnel
|
||||||
|
belongs there.
|
||||||
|
- a `docs/` page: tunnel setup, the maddr form, the two-host
|
||||||
|
run. Keep prose in the docs; keep the examples runnable and
|
||||||
|
minimal.
|
||||||
|
- tests: maddr round-trip, `TunnelledAddress` delegation
|
||||||
|
(`proto_key`/`unwrap` identical to inner), `wrap_address()`
|
||||||
|
regression (a tunnelled maddr `str` → `TunnelledAddress`; a
|
||||||
|
plain one → unchanged), and **a real end-to-end over a
|
||||||
|
locally-created wg pair** gated on `CAP_NET_ADMIN` (see §5.3).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Layer B — `pyroute2` under `trio`
|
||||||
|
|
||||||
|
### 4.1 the library situation (verify at implementation time)
|
||||||
|
|
||||||
|
`pyroute2` ≥0.9 rewrote its core onto **asyncio**
|
||||||
|
(`AsyncIPRoute`; the sync `IPRoute` wraps it with its own loop).
|
||||||
|
It also ships a `WireGuard` netlink (generic-netlink) module
|
||||||
|
supporting `.set(iface, private_key=..., peer={...})` and
|
||||||
|
`.info(iface)`, plus `pyroute2.netns` / `NetNS` for namespaces,
|
||||||
|
and `IPRoute.link('add', kind='wireguard', ifname=...)`.
|
||||||
|
|
||||||
|
Three integration options, in increasing trio-nativeness:
|
||||||
|
|
||||||
|
- **(1) `trio.to_thread.run_sync()` around the sync API.**
|
||||||
|
Netlink ops here are one-shot, sub-millisecond, and happen at
|
||||||
|
bind/teardown time only — *not* in the msg hot path. This is
|
||||||
|
the **correct default**: it's ~10 lines, uses a battle-tested
|
||||||
|
API, and costs nothing where it's used.
|
||||||
|
- **(2) sans-io: `trio.socket` + pyroute2's message codecs.**
|
||||||
|
`pyroute2`'s message classes
|
||||||
|
(`pyroute2.netlink.rtnl.*`, `pyroute2.netlink.generic.wireguard.wgmsg`)
|
||||||
|
encode/decode independently of its I/O core. So a
|
||||||
|
`tractor/ipc/_netlink.py` with a small trio `NetlinkSocket`
|
||||||
|
(`trio.socket.socket(AF_NETLINK, SOCK_RAW|SOCK_DGRAM, proto)`,
|
||||||
|
`sendto`/`recv`, seq/pid matching, `NLMSG_DONE`/`NLMSG_ERROR`
|
||||||
|
handling) + pyroute2 codecs is very achievable and is the
|
||||||
|
honest reading of "as much trio wrapping as possible where any
|
||||||
|
other async support can be replaced".
|
||||||
|
**Do this for the paths we actually need** (link add/del,
|
||||||
|
addr add, wg get/set, netns bind) and *only* those — a
|
||||||
|
general netlink client is out of scope.
|
||||||
|
- (3) reimplement the codecs. Never.
|
||||||
|
|
||||||
|
**Recommended split**: ship (1) first so layer B is a small,
|
||||||
|
reviewable, behaviour-preserving swap of `verify_wg_peer()`'s
|
||||||
|
body; then land (2) as a follow-up commit for the read path
|
||||||
|
(`wg get`, `link get`) where the sans-io surface is smallest,
|
||||||
|
and keep (1) for the privileged mutating ops. Measure before
|
||||||
|
converting anything else — there is no perf argument here, only
|
||||||
|
a "no foreign event loop in a trio actor" argument, which (1)
|
||||||
|
already satisfies (a thread is not an event loop).
|
||||||
|
|
||||||
|
Explicitly **do not** pull in `trio-asyncio` for pyroute2: it
|
||||||
|
would be the one place in the runtime where an asyncio loop
|
||||||
|
exists for no reason.
|
||||||
|
|
||||||
|
### 4.2 API shape
|
||||||
|
|
||||||
|
Pure-ish, functional, `@acm` for anything with teardown:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def read_wg_peers(
|
||||||
|
iface: str = 'wg0',
|
||||||
|
netns: str|None = None,
|
||||||
|
) -> tuple[str, ...]: ... # base64 pubkeys
|
||||||
|
|
||||||
|
async def read_wg_pubkey(iface: str = 'wg0', ...) -> str: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
and `verify_wg_peer()` becomes a thin composition over the two.
|
||||||
|
Note the pure-getter rule: no `read_wg_peers(..., create=True)`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Layer C — nested bindspace `@acm`s + `Address.namespace`
|
||||||
|
|
||||||
|
This is the part #443 and `multiaddr_declare_eps.md` actually
|
||||||
|
ask for: *"for any tunneled maddr-`str`-entry we deliver a
|
||||||
|
data-structure which can easily be passed to nested `@acm`s
|
||||||
|
which consecutively setup nested net bindspaces for binding the
|
||||||
|
endpoint addrs"*.
|
||||||
|
|
||||||
|
### 5.1 the composition
|
||||||
|
|
||||||
|
```python
|
||||||
|
@acm
|
||||||
|
async def open_bindspace(
|
||||||
|
addr: TunnelledAddress,
|
||||||
|
) -> AsyncGenerator[Address, None]:
|
||||||
|
'''
|
||||||
|
Enter the net-bindspace implied by `addr`'s tunnel stack,
|
||||||
|
yielding the *inner* `Address` ready to bind/connect.
|
||||||
|
|
||||||
|
Nests: one `@acm` per tunnel segment, outermost-first, so
|
||||||
|
a 2-deep stack is just two nested `async with`s and the
|
||||||
|
teardown order is guaranteed by `trio`.
|
||||||
|
|
||||||
|
'''
|
||||||
|
```
|
||||||
|
|
||||||
|
with per-tunnel-kind implementations:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@acm
|
||||||
|
async def open_netns(name: str) -> AsyncGenerator[None, None]: ...
|
||||||
|
@acm
|
||||||
|
async def open_wg_iface(spec: WGTunnelSpec) -> AsyncGenerator[WGTunnelSpec, None]: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
and a driver that folds a list of specs into nested contexts
|
||||||
|
(`contextlib.AsyncExitStack` for the N-deep case). The
|
||||||
|
`parse_endpoints()` API (`_multiaddr.py:153`) is the front door:
|
||||||
|
it already returns `dict[name, list[Address]]` and the
|
||||||
|
`multiaddr_declare_eps.md` sketch anticipates the recursive
|
||||||
|
`dict[str, list[Address]]|dict[...]` return for tunnelled
|
||||||
|
entries. Extend it to carry the tunnel stack, not to *enter* it.
|
||||||
|
|
||||||
|
### 5.2 `Address.namespace`, at last
|
||||||
|
|
||||||
|
- `TunnelledAddress.namespace` → `(kind, id)` e.g.
|
||||||
|
`('netns', 'tractor-wg0')`.
|
||||||
|
- **and** the existing backends should implement it as `None`
|
||||||
|
explicitly (they currently just don't define it), so the
|
||||||
|
Protocol stops lying.
|
||||||
|
- consumers to audit: nothing reads `.namespace` today — so
|
||||||
|
adding it is safe, but the *point* is that
|
||||||
|
`Endpoint`/`Server.pformat()` should start showing it (there's
|
||||||
|
already a `# !TODO, always be ns aware!` +
|
||||||
|
`f'|_netns: {netns}\n'` placeholder sitting in
|
||||||
|
`Endpoint.pformat()`, `_server.py:645`). Fill that in; it's
|
||||||
|
the cheapest possible proof the layer is wired.
|
||||||
|
|
||||||
|
### 5.3 the netns/process reality — read this before designing
|
||||||
|
|
||||||
|
**The headline consequence, stated up front**: netns is a
|
||||||
|
**runtime-level config API, not an actor-app-code API.** It is
|
||||||
|
declared as part of how an actor process is *brought up* — a
|
||||||
|
spawn-time/boot-time input alongside `enable_transports` and
|
||||||
|
`tpt_bind_addrs` — and it is **not** dynamically re-enterable by
|
||||||
|
app code once the actor is live. There is deliberately no
|
||||||
|
`await actor.enter_netns(...)`. Two hard reasons, both below:
|
||||||
|
`setns(2)` doesn't retroactively move existing sockets, and it's
|
||||||
|
per-thread rather than per-process. Anything that *looks* like a
|
||||||
|
mid-life API here would be a footgun that silently leaves the IPC
|
||||||
|
server bound in the old namespace.
|
||||||
|
|
||||||
|
- `setns(2)` with `CLONE_NEWNET` affects **the calling thread
|
||||||
|
only**, and sockets already created keep their original netns.
|
||||||
|
A trio actor is effectively single-threaded for our purposes,
|
||||||
|
so "enter the netns, *then* bind" works — but any
|
||||||
|
`to_thread` worker (§4.1 option 1!) is in the **original**
|
||||||
|
netns unless it also `setns`. Concretely: a wg query issued
|
||||||
|
via `trio.to_thread` will hit the wrong namespace. Either
|
||||||
|
pass `netns=` down to `pyroute2` (which does the
|
||||||
|
fork/setns dance itself) or pin a dedicated worker. **This is
|
||||||
|
the single subtlest bug in this plan — write the test first.**
|
||||||
|
- entering a netns is *process-global-ish and irreversible-ish*
|
||||||
|
in practice. Therefore: **netns membership belongs to the
|
||||||
|
actor process, decided before the runtime binds**, not to a
|
||||||
|
mid-life `@acm`. Design:
|
||||||
|
- the root/parent decides the netns for a subactor and passes
|
||||||
|
it in the spawn spec (there's already
|
||||||
|
`enable_transports`/`accept_addrs` plumbing at
|
||||||
|
`_runtime.py:1595-1615` — the netns rides alongside).
|
||||||
|
- the child, in `_runtime.async_main()` **before**
|
||||||
|
`IPCServer.listen_on()`, enters it.
|
||||||
|
- the mid-life `@acm` form is then only for the *root* /
|
||||||
|
single-actor case, and for iface creation (which is
|
||||||
|
genuinely scoped).
|
||||||
|
- document the constraint rather than hiding it; a
|
||||||
|
`RuntimeError` if `open_netns()` is entered after any
|
||||||
|
listener exists.
|
||||||
|
- privileges: iface/netns creation needs `CAP_NET_ADMIN`.
|
||||||
|
Never `sudo` from inside the runtime. Two supported modes:
|
||||||
|
(i) pre-provisioned out-of-band (layers A/B — the default,
|
||||||
|
and what #482 documents), (ii) runtime-managed when the
|
||||||
|
process already holds the cap. Detect with a cheap
|
||||||
|
`os.geteuid()==0 or CAP_NET_ADMIN in /proc/self/status`
|
||||||
|
probe and *fail loudly with an actionable message* otherwise.
|
||||||
|
- teardown must be idempotent and tolerant: an iface/netns
|
||||||
|
already gone must not strand the rest of the teardown — the
|
||||||
|
exact lesson `_uds.close_listener()`'s `FileNotFoundError`
|
||||||
|
tolerance and `_serve_ipc_eps()`'s per-ep `try/except`
|
||||||
|
encode. Mirror both.
|
||||||
|
|
||||||
|
### 5.4 tests for layer C
|
||||||
|
|
||||||
|
- unit: fold-N-tunnel-specs-into-nested-`@acm`s, with fakes; assert
|
||||||
|
enter/exit ordering (outermost-last-out) via a trace list.
|
||||||
|
- integration, gated on `CAP_NET_ADMIN` (skip otherwise, and in
|
||||||
|
CI run it in a `--cap-add NET_ADMIN` container job): create two
|
||||||
|
netns + a wg pair entirely in-process, boot a `tractor` root in
|
||||||
|
one and a subactor in the other, `find_actor()` across the
|
||||||
|
tunnel. This is a *fantastic* test to have and is fully
|
||||||
|
self-contained — no second host, no `sudo` in the test body.
|
||||||
|
- the `to_thread`-netns-mismatch regression from §5.3, written
|
||||||
|
**first** (red), then the fix (green), per project convention.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. "Other shuttle-able tpts"
|
||||||
|
|
||||||
|
The generalization the #482 follow-up gestures at: once
|
||||||
|
`TunnelledAddress` + `open_bindspace()` exist, the same
|
||||||
|
machinery covers any iface-layer tunnel `pyroute2` can drive —
|
||||||
|
`ipip`/`gre`/`sit`/`vxlan`/`geneve`/`bridge`/`veth`. Keep
|
||||||
|
`WGTunnelSpec` as *one* frozen struct among a
|
||||||
|
`TunnelSpec = WGTunnelSpec|VxlanTunnelSpec|...` union with a
|
||||||
|
`kind: ClassVar[str]`, and dispatch `open_*` by `match` on it.
|
||||||
|
Design for it now (union + `match`), implement only `wg` +
|
||||||
|
`netns`. `veth`-pairs-in-netns is the natural second one because
|
||||||
|
it makes the §5.4 integration test possible without wg at all —
|
||||||
|
consider doing it *first* for exactly that reason.
|
||||||
|
|
||||||
|
## 7. Non-goals
|
||||||
|
|
||||||
|
- no wg userspace implementation, no key exchange, no
|
||||||
|
`wg-quick` reimplementation (config-file parsing is
|
||||||
|
out of scope; take structured input).
|
||||||
|
- no persistence of private keys beyond what layer C's iface
|
||||||
|
creation needs (and that stays in `get_rt_dir()`, 0600).
|
||||||
|
- macOS/Windows: layers B/C are Linux-only. Layer A (declarative)
|
||||||
|
works anywhere `wg` does. Gate accordingly and say so in the
|
||||||
|
docs — do not silently no-op.
|
||||||
|
|
||||||
|
## 8. Risks
|
||||||
|
|
||||||
|
| risk | mitigation |
|
||||||
|
| --- | --- |
|
||||||
|
| `to_thread` worker runs in the wrong netns | §5.3; pass `netns=` to pyroute2 or pin a worker; test-first |
|
||||||
|
| py-multiaddr#108 merged but unreleased | `[tool.uv.sources]` `rev` pin + `_have_wg_maddr_proto()` gate; layer A's inner-addr path works regardless |
|
||||||
|
| `TunnelledAddress` leaks into `Endpoint` and breaks `inspect.getmodule()` | unwrap at parse/bindspace boundary; assert `not isinstance(ep.addr, TunnelledAddress)` in `Endpoint.__post_init__` |
|
||||||
|
| privileged ops in a library | never `sudo`; explicit cap probe + actionable error; pre-provisioned is the default |
|
||||||
|
| pyroute2 0.9 asyncio core drags a loop into the actor | option (1) is a *thread*, not a loop; forbid `trio-asyncio` here (§4.1) |
|
||||||
|
| netns teardown strands actor teardown | idempotent/tolerant teardown mirroring `_uds.close_listener()` |
|
||||||
|
|
||||||
|
## 9. Follow-up issue seeds
|
||||||
|
|
||||||
|
- `veth`-in-netns bindspace (unblocks capless-ish integration
|
||||||
|
testing, and is a great local multi-"host" test rig)
|
||||||
|
- composed/tunnelled maddr grammar shared with plan 02's
|
||||||
|
`/…/quic-v1/…` stacks (gh #443)
|
||||||
|
- `wg` proto into the multiaddr **spec** (gh #483), then flip
|
||||||
|
`MsgTransport.maddr` to always return `Multiaddr` (the third
|
||||||
|
#443 bullet)
|
||||||
|
- runtime-managed wg key rotation / peer add-remove as a
|
||||||
|
`tractor` service actor — the natural "actor that owns the
|
||||||
|
network" demo
|
||||||
|
|
@ -0,0 +1,191 @@
|
||||||
|
# `/tipc` multiaddr protocol: upstream issue draft
|
||||||
|
|
||||||
|
Candidate issue for `multiformats/multiaddr`, to be submitted after
|
||||||
|
the encoding questions below have been reviewed locally.
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Linux TIPC (Transparent Inter-Process Communication) addresses a
|
||||||
|
service by a location-independent `(service type, instance)` name.
|
||||||
|
A server publishes that name into the kernel-maintained cluster name
|
||||||
|
table and a client connects by the same name; no host or transport
|
||||||
|
port forms part of the service identity.
|
||||||
|
|
||||||
|
TIPC is also known as **Cluster Domain Sockets**, a useful description
|
||||||
|
of its relationship to Unix-domain sockets. The registered protocol
|
||||||
|
name should nevertheless remain `tipc`: it matches Linux's
|
||||||
|
`AF_TIPC`, socket constants, kernel module and iproute2 frontend.
|
||||||
|
Registering `cds` would create an ecosystem-specific alias that is
|
||||||
|
harder to map back to the normative kernel API.
|
||||||
|
|
||||||
|
We would like to register a `tipc` multiaddr component so these
|
||||||
|
service endpoints can be represented directly and composed with a
|
||||||
|
bearer or tunnel description:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/tipc/1953628160:1616:2
|
||||||
|
|
||||||
|
/ip4/192.168.1.50/udp/51820/wg/u<key>/tipc/1953628160:1616:2
|
||||||
|
\____________ WireGuard bearer ____________/\____ TIPC service ____/
|
||||||
|
```
|
||||||
|
|
||||||
|
In the composed form, the components through `/wg/<key>` identify
|
||||||
|
the routed bearer and tunnel peer. The `/tipc/...` component is pure
|
||||||
|
service identity, resolved to a current publisher by the TIPC kernel
|
||||||
|
name table. Unlike a TCP endpoint, it deliberately has no inner IP
|
||||||
|
address or port.
|
||||||
|
|
||||||
|
This proposal does not imply that parsing the multiaddr configures a
|
||||||
|
TIPC bearer. In particular, TIPC over a WireGuard interface requires
|
||||||
|
a separately configured TIPC UDP bearer; WireGuard interfaces are L3
|
||||||
|
devices and cannot carry TIPC Ethernet media directly.
|
||||||
|
|
||||||
|
Today that bearer is configured through iproute2's `tipc` frontend,
|
||||||
|
which speaks the kernel's `TIPCv2` generic-netlink family. `pyroute2`
|
||||||
|
already provides WireGuard support and generic-netlink primitives but
|
||||||
|
has no TIPC codec/module; adding one is a complementary deployment
|
||||||
|
automation track, not part of this address-format proposal.
|
||||||
|
|
||||||
|
## Proposed protocol
|
||||||
|
|
||||||
|
- Name: `tipc`
|
||||||
|
- Code: TBD, allocated in `multiformats/multicodec` under the
|
||||||
|
`multiaddr` tag before implementations stabilize one
|
||||||
|
- Size: 72 bits
|
||||||
|
- Value: service type, service instance and publication scope
|
||||||
|
|
||||||
|
### Binary form
|
||||||
|
|
||||||
|
Exactly nine bytes with no value-length prefix:
|
||||||
|
|
||||||
|
| Offset | Size | Field | Encoding |
|
||||||
|
| ---: | ---: | --- | --- |
|
||||||
|
| 0 | 4 bytes | service type | unsigned 32-bit big-endian |
|
||||||
|
| 4 | 4 bytes | service instance | unsigned 32-bit big-endian |
|
||||||
|
| 8 | 1 byte | publication scope | unsigned enum byte |
|
||||||
|
|
||||||
|
```text
|
||||||
|
tipc-value = uint32be(type) || uint32be(instance) || uint8(scope)
|
||||||
|
```
|
||||||
|
|
||||||
|
For type `1953628160` (`0x74720000`), instance `1616` and
|
||||||
|
cluster scope `2`, the payload is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
74 72 00 00 00 00 06 50 02
|
||||||
|
```
|
||||||
|
|
||||||
|
### String form
|
||||||
|
|
||||||
|
Use one multiaddr value segment containing three canonical decimal
|
||||||
|
integers:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/tipc/<type>:<instance>:<scope>
|
||||||
|
```
|
||||||
|
|
||||||
|
Canonical values have no sign, whitespace, alternate radix or
|
||||||
|
leading zeroes, except that zero itself is `0`. `type` and `instance`
|
||||||
|
must fit unsigned 32-bit fields. Scope is one of:
|
||||||
|
|
||||||
|
- `2`: `TIPC_CLUSTER_SCOPE`
|
||||||
|
- `3`: `TIPC_NODE_SCOPE`
|
||||||
|
|
||||||
|
The existing experimental spelling
|
||||||
|
`/tipc/<type>/<instance>/<scope>` cannot be registered as one normal
|
||||||
|
multiaddr protocol: generic parsing treats each slash-delimited name
|
||||||
|
as another protocol component. A single structured value preserves
|
||||||
|
TIPC's atomic service-address semantics without registering three
|
||||||
|
artificial protocols.
|
||||||
|
|
||||||
|
## Why scope is included
|
||||||
|
|
||||||
|
TIPC scope controls where a bound service publication is visible.
|
||||||
|
The same address representation is used for listener configuration
|
||||||
|
and dialing, so retaining scope lets a multiaddr round-trip the full
|
||||||
|
socket address rather than silently turning a node-local bind into a
|
||||||
|
cluster publication.
|
||||||
|
|
||||||
|
Modern Linux UAPI defines cluster and node scopes. The deprecated
|
||||||
|
zone spelling should not receive a new wire value; implementations
|
||||||
|
may normalize legacy input to cluster scope before encoding.
|
||||||
|
|
||||||
|
## Composition
|
||||||
|
|
||||||
|
Standalone service:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/tipc/1953628160:1616:2
|
||||||
|
```
|
||||||
|
|
||||||
|
TIPC service reached through a WireGuard bearer:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/ip4/192.168.1.50/udp/51820/wg/u<key>/tipc/1953628160:1616:2
|
||||||
|
```
|
||||||
|
|
||||||
|
This differs intentionally from TCP over WireGuard:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/ip4/192.168.1.50/udp/51820/wg/u<key>/ip4/10.0.11.1/tcp/1616
|
||||||
|
```
|
||||||
|
|
||||||
|
TCP repeats an inner locative address. TIPC does not: its service
|
||||||
|
name is resolved and load-balanced in-kernel across current
|
||||||
|
publishers.
|
||||||
|
|
||||||
|
## Semantics and security
|
||||||
|
|
||||||
|
- A TIPC service name identifies a service, not a unique process.
|
||||||
|
Multiple publishers may bind the same name and connections can be
|
||||||
|
distributed among them.
|
||||||
|
- Publication scope is reachability metadata, not authentication.
|
||||||
|
- A composed `/wg` key authenticates the tunnel peer, not the TIPC
|
||||||
|
service publisher.
|
||||||
|
- TIPC's optional native AES-GCM link encryption is independent of
|
||||||
|
this address codec and of WireGuard.
|
||||||
|
- Codec implementations should validate field widths and canonical
|
||||||
|
text only; cluster membership and publisher authorization remain
|
||||||
|
deployment concerns.
|
||||||
|
|
||||||
|
## Implementation plan
|
||||||
|
|
||||||
|
1. Reserve a `multiaddr`-tagged code in
|
||||||
|
`multiformats/multicodec`.
|
||||||
|
2. Add the fixed-size protocol row and normative encoding text to
|
||||||
|
`multiformats/multiaddr`.
|
||||||
|
3. Add codecs and cross-language test vectors, beginning with
|
||||||
|
`multiformats/py-multiaddr`.
|
||||||
|
4. Verify standalone and composed `wg` + `tipc` string/binary
|
||||||
|
round-trips.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
1. Is a fixed 72-bit value preferred over a self-describing or
|
||||||
|
variable-width tuple for this kernel-defined address?
|
||||||
|
2. Should node scope be representable in a generally shareable
|
||||||
|
multiaddr, or should the registered form be cluster-only?
|
||||||
|
3. Does multiaddr have an existing convention for structured numeric
|
||||||
|
values that should replace the colon-separated text form?
|
||||||
|
4. Should the specification describe TIPC service *ranges*, or keep
|
||||||
|
this protocol limited to singleton service names used for
|
||||||
|
connection endpoints?
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Linux TIPC documentation:
|
||||||
|
https://docs.kernel.org/networking/tipc.html
|
||||||
|
- Cluster Domain Sockets terminology:
|
||||||
|
https://en.wikipedia.org/wiki/Transparent_Inter-process_Communication
|
||||||
|
- Linux socket UAPI:
|
||||||
|
https://github.com/torvalds/linux/blob/master/include/uapi/linux/tipc.h
|
||||||
|
- Linux TIPC generic-netlink UAPI:
|
||||||
|
https://github.com/torvalds/linux/blob/master/include/uapi/linux/tipc_netlink.h
|
||||||
|
- pyroute2 WireGuard and generic-netlink APIs:
|
||||||
|
https://docs.pyroute2.org/wireguard.html
|
||||||
|
- WireGuard multiaddr implementation discussion:
|
||||||
|
https://github.com/multiformats/py-multiaddr/issues/107
|
||||||
|
- WireGuard codec implementation:
|
||||||
|
https://github.com/multiformats/py-multiaddr/pull/108
|
||||||
|
- Downstream tracking and prototype:
|
||||||
|
https://github.com/goodboy/tractor/issues/498
|
||||||
|
|
@ -0,0 +1,58 @@
|
||||||
|
# next-gen `tractor.ipc` transport backend plans
|
||||||
|
|
||||||
|
Implementation specs for three prospective `.ipc` transport
|
||||||
|
backends, written so each can be worked independently (by a
|
||||||
|
different model/provider) without design or lib-selection drift.
|
||||||
|
|
||||||
|
**Read [`00_shared_backend_contract.md`](./00_shared_backend_contract.md)
|
||||||
|
first** — it is the normative description of what a `tractor`
|
||||||
|
transport backend *is* as of `main@83b34884` (the backend
|
||||||
|
duck-type, the 10-item registration checklist, the test-harness
|
||||||
|
plumbing, the code-style rules). The three plans assume it and
|
||||||
|
document only their own deltas.
|
||||||
|
|
||||||
|
| plan | issue | dep | size | lands |
|
||||||
|
| --- | --- | --- | --- | --- |
|
||||||
|
| [01 — TIPC](./01_tipc_backend.md) | [#378] | **none** (stdlib) | small | **landed**, PR [#493] — see the [handoff](./01_tipc_HANDOFF.md) |
|
||||||
|
| [02 — QUIC/`iroh`](./02_quic_iroh_backend.md) | [#353] | `iroh` (uniffi FFI) | large | needs a prep PR |
|
||||||
|
| [03 — `wg` bindspace](./03_wg_tunnel_bindspace.md) | [#482], [#443] | `pyroute2` | medium, 3 layers | layer A now |
|
||||||
|
| [04 — `/tipc` multiaddr](./04_tipc_multiaddr_upstream.md) | [#498] | upstream `multiformats` | issue draft | local review |
|
||||||
|
|
||||||
|
Headline conclusions:
|
||||||
|
|
||||||
|
- **TIPC is the cheap win.** Verified: `trio.SocketStream` and
|
||||||
|
`trio.SocketListener` are address-family agnostic (only
|
||||||
|
`SOCK_STREAM` + a trio socket), and CPython ships `AF_TIPC` +
|
||||||
|
23 `TIPC_*` constants. So the backend is ~one module of
|
||||||
|
contract boilerplate, zero new deps, and it buys
|
||||||
|
*kernel-native* service discovery: `bind()` publishes,
|
||||||
|
`connect()`-by-name resolves — no registrar in the loop.
|
||||||
|
(`modprobe tipc` is required; hard-gate everything.)
|
||||||
|
- **QUIC's cost is entirely in two adapters**, not in QUIC. The
|
||||||
|
`iroh` python bindings are `uniffi`-generated asyncio, but the
|
||||||
|
asyncio dependency is confined to *one* future-poll callback —
|
||||||
|
a ~40-line `trio` bridge (`TrioToken.run_sync_soon`) replaces
|
||||||
|
it. The second cost is that an iroh listener isn't a socket,
|
||||||
|
which needs a small, independently-reviewable prep PR to
|
||||||
|
`_server.py`/`_types.py`.
|
||||||
|
- **WireGuard is not a transport.** It's an iface-layer tunnel,
|
||||||
|
so it belongs as a *nested bindspace* (`TunnelledAddress` +
|
||||||
|
`open_bindspace()` `@acm`s) wrapping whatever L4 tpt is in
|
||||||
|
use — which is also what finally implements the long-spec'd
|
||||||
|
`Address.namespace`, and what generalizes to
|
||||||
|
`veth`/`vxlan`/`gre`.
|
||||||
|
|
||||||
|
Ordering rationale: plan 01 first as the cheap proof the
|
||||||
|
table-registration story generalizes to a genuinely new proto;
|
||||||
|
plan 03 layer A is already deployable-today doc/example work;
|
||||||
|
plan 02 last (and gated on its prep PR). Plans 01 and 02 both
|
||||||
|
want the same `Address.rebind_from_sockname` gate — whichever
|
||||||
|
lands first ships it.
|
||||||
|
|
||||||
|
[#378]: https://github.com/goodboy/tractor/issues/378
|
||||||
|
[#353]: https://github.com/goodboy/tractor/issues/353
|
||||||
|
[#482]: https://github.com/goodboy/tractor/issues/482
|
||||||
|
[#443]: https://github.com/goodboy/tractor/issues/443
|
||||||
|
[#498]: https://github.com/goodboy/tractor/issues/498
|
||||||
|
|
||||||
|
[#493]: https://github.com/goodboy/tractor/pull/493
|
||||||
|
|
@ -4,10 +4,12 @@ IPC and logging
|
||||||
Under every portal, context and stream sits a per-peer
|
Under every portal, context and stream sits a per-peer
|
||||||
:class:`~tractor.Channel`: a msgpack-typed messaging link wrapping
|
:class:`~tractor.Channel`: a msgpack-typed messaging link wrapping
|
||||||
one OS transport connection. Transports are pluggable per actor
|
one OS transport connection. Transports are pluggable per actor
|
||||||
via ``enable_transports=['tcp' | 'uds']`` — TCP is the default,
|
via ``enable_transports=['tcp' | 'uds' | 'tipc']`` — TCP is the
|
||||||
UDS (unix domain sockets) gives you port-less, same-host IPC with
|
default, UDS (unix domain sockets) gives you port-less, same-host
|
||||||
kernel-provided peer credentials for free — and exactly **one**
|
IPC with kernel-provided peer credentials for free, and TIPC is an
|
||||||
transport may currently be enabled per actor.
|
opt-in linux cluster protocol where the address *is* a
|
||||||
|
kernel-published service name (see :doc:`/guide/tipc`) — and
|
||||||
|
exactly **one** transport may currently be enabled per actor.
|
||||||
|
|
||||||
.. d2:: diagrams/runtime_stack.d2
|
.. d2:: diagrams/runtime_stack.d2
|
||||||
:caption: Where ``Channel`` sits in the runtime stack.
|
:caption: Where ``Channel`` sits in the runtime stack.
|
||||||
|
|
@ -15,7 +17,8 @@ transport may currently be enabled per actor.
|
||||||
:alt: layered runtime stack from app code down to transports
|
:alt: layered runtime stack from app code down to transports
|
||||||
|
|
||||||
Addresses are "unwrapped" tuples at the API edges:
|
Addresses are "unwrapped" tuples at the API edges:
|
||||||
``('host', port)`` for TCP, filesystem-path pairs for UDS. For
|
``('host', port)`` for TCP, filesystem-path pairs for UDS and the
|
||||||
|
proto-keyed ``('tipc', stype, instance, scope)`` for TIPC. For
|
||||||
the full layering story — transport protocols, the IPC server,
|
the full layering story — transport protocols, the IPC server,
|
||||||
address types and the msg loop — see
|
address types and the msg loop — see
|
||||||
:doc:`/explain/architecture`.
|
:doc:`/explain/architecture`.
|
||||||
|
|
|
||||||
|
|
@ -31,6 +31,8 @@ order,
|
||||||
SC-supervise ``asyncio`` tasks from ``trio``.
|
SC-supervise ``asyncio`` tasks from ``trio``.
|
||||||
- :doc:`msging` — typed IPC payloads, the wire
|
- :doc:`msging` — typed IPC payloads, the wire
|
||||||
msg-spec and custom codecs.
|
msg-spec and custom codecs.
|
||||||
|
- :doc:`tipc` — the ``AF_TIPC`` cluster backend,
|
||||||
|
where the kernel does discovery for you.
|
||||||
- :doc:`testing` — running + monitoring the
|
- :doc:`testing` — running + monitoring the
|
||||||
test suite (and testing your own actor apps).
|
test suite (and testing your own actor apps).
|
||||||
|
|
||||||
|
|
@ -49,4 +51,5 @@ order,
|
||||||
parallelism
|
parallelism
|
||||||
asyncio
|
asyncio
|
||||||
msging
|
msging
|
||||||
|
tipc
|
||||||
testing
|
testing
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,276 @@
|
||||||
|
TIPC: when the kernel does discovery
|
||||||
|
====================================
|
||||||
|
|
||||||
|
Every other ``tractor`` transport gives you a *pipe* and leaves
|
||||||
|
discovery to us: the registrar actor, the ``find_actor()``
|
||||||
|
round-trip, the whole :doc:`discovery` story. TIPC_
|
||||||
|
(Transparent Inter-Process Communication) is different — it's a
|
||||||
|
linux-kernel cluster protocol whose **service names live in a
|
||||||
|
cluster-wide name table the kernel itself maintains**.
|
||||||
|
|
||||||
|
Which flips the model:
|
||||||
|
|
||||||
|
- an actor's IPC address *is* a service name ``(stype,
|
||||||
|
instance)`` — no host, no port,
|
||||||
|
- ``.bind()``-ing that name **is** service registration,
|
||||||
|
- a peer's ``.connect()``-by-name **is** the lookup, resolved
|
||||||
|
and load-balanced in-kernel.
|
||||||
|
|
||||||
|
So for TIPC-capable deployments the registrar round-trip stops
|
||||||
|
being the only way peers find each other. Enable it per actor
|
||||||
|
like any other backend,
|
||||||
|
|
||||||
|
.. code:: python
|
||||||
|
|
||||||
|
async with tractor.open_nursery(
|
||||||
|
enable_transports=['tipc'],
|
||||||
|
) as an:
|
||||||
|
...
|
||||||
|
|
||||||
|
.. warning::
|
||||||
|
|
||||||
|
TIPC is **opt-in and linux-only**. The ``tipc`` kernel module
|
||||||
|
is not loaded on most boxes (``sudo modprobe tipc``), and the
|
||||||
|
address family doesn't exist off-linux at all. Check
|
||||||
|
:func:`tractor.ipc._tipc.is_tipc_available` before assuming;
|
||||||
|
``tractor`` never selects this backend for you.
|
||||||
|
|
||||||
|
.. _TIPC: https://en.wikipedia.org/wiki/Transparent_Inter-process_Communication
|
||||||
|
|
||||||
|
Your actor tree, in the kernel's name table
|
||||||
|
-------------------------------------------
|
||||||
|
|
||||||
|
The single best demo this backend has needs no ``tractor`` API
|
||||||
|
at all — boot a tree and ask ``tipc(8)`` what it sees:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
sudo modprobe tipc
|
||||||
|
python examples/multihost/tipc_cluster/single_host.py
|
||||||
|
|
||||||
|
.. code:: text
|
||||||
|
|
||||||
|
--- `tipc nametable show` :: root + 3 subactors ---
|
||||||
|
Type Lower Upper Scope Port
|
||||||
|
1953628160 1616 1616 cluster 3161982128
|
||||||
|
1953628160 1219427151 1219427151 cluster 1587358717
|
||||||
|
1953628160 2641339936 2641339936 cluster 1864021571
|
||||||
|
1953628160 3344505866 3344505866 cluster 3816483388
|
||||||
|
|
||||||
|
--- `tipc nametable show` :: after teardown (all withdrawn) ---
|
||||||
|
Type Lower Upper Scope Port
|
||||||
|
|
||||||
|
Reading the rows,
|
||||||
|
|
||||||
|
- ``1953628160`` is ``0x74720000``, ``tractor``'s reserved
|
||||||
|
service *type* — ascii ``tr`` in the high half, with the low
|
||||||
|
16 bits free so an app can partition its own service classes
|
||||||
|
via ``TIPCAddress._stype``,
|
||||||
|
- ``1616`` is the host-singleton registrar instance, the same
|
||||||
|
"1616 is tractor's registrar" idiom as the TCP port and the
|
||||||
|
``registry@1616.sock`` UDS filename,
|
||||||
|
- the other three are per-actor instances derived from a
|
||||||
|
``blake2b`` digest of the actor's identity (see
|
||||||
|
`Silent crosstalk`_),
|
||||||
|
- ``Scope`` is the address' :attr:`bindspace` — see `Scope is
|
||||||
|
the bindspace`_.
|
||||||
|
|
||||||
|
Push-based discovery
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
TIPC also exposes a *topology service*: subscribe and the kernel
|
||||||
|
pushes you name-table transitions as they happen.
|
||||||
|
:func:`tractor.ipc._tipc.open_topology_events` wraps it as an
|
||||||
|
``@acm`` yielding a ``trio`` receive-channel,
|
||||||
|
|
||||||
|
.. code:: python
|
||||||
|
|
||||||
|
from tractor.ipc._tipc import open_topology_events
|
||||||
|
|
||||||
|
async with open_topology_events() as events:
|
||||||
|
async for ev in events:
|
||||||
|
print(f'{ev.kind}: {ev.addr}')
|
||||||
|
|
||||||
|
.. code:: text
|
||||||
|
|
||||||
|
watching the TIPC name table..
|
||||||
|
[+] published instance=1616 port=0x00000000:2375440573
|
||||||
|
spawning subactors..
|
||||||
|
[+] published instance=186947472 port=0x00000000:3960753074
|
||||||
|
[+] published instance=2191362136 port=0x00000000:2263898853
|
||||||
|
tearing down..
|
||||||
|
[-] withdrawn instance=186947472 port=0x00000000:3960753074
|
||||||
|
|
||||||
|
No polling, no registrar round-trip — this is the groundwork for
|
||||||
|
a registrar that keeps a live view of the actor set without ever
|
||||||
|
calling ``find_actor()``.
|
||||||
|
|
||||||
|
``filt`` picks the granularity: ``TIPC_SUB_SERVICE`` gives one
|
||||||
|
event per *name* becoming (un)available, ``TIPC_SUB_PORTS`` one
|
||||||
|
per *publisher* — which is what makes the duplicate-name case
|
||||||
|
below externally observable.
|
||||||
|
|
||||||
|
Scope is the bindspace
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
Every ``tractor`` address type has a ``.bindspace`` — "the set
|
||||||
|
of hosts this bind is reachable from". For TCP that's the IP,
|
||||||
|
for UDS the socket-file directory. For TIPC it's the *scope*,
|
||||||
|
which is about as literal a reading of that docstring as exists:
|
||||||
|
|
||||||
|
.. list-table::
|
||||||
|
:header-rows: 1
|
||||||
|
:widths: 30 70
|
||||||
|
|
||||||
|
* - scope
|
||||||
|
- meaning
|
||||||
|
* - ``TIPC_NODE_SCOPE``
|
||||||
|
- same host only — the UDS analogue
|
||||||
|
* - ``TIPC_CLUSTER_SCOPE``
|
||||||
|
- cluster-visible (the default)
|
||||||
|
|
||||||
|
``TIPC_ZONE_SCOPE`` is deprecated and aliased to cluster-scope
|
||||||
|
by modern kernels; ``tractor`` accepts it on input, folds it to
|
||||||
|
cluster and logs at ``transport`` level.
|
||||||
|
|
||||||
|
Spanning hosts
|
||||||
|
--------------
|
||||||
|
|
||||||
|
Single-host TIPC needs only ``modprobe``. Crossing hosts needs a
|
||||||
|
**bearer** enabled on both — an ethernet (L2) or UDP underlay
|
||||||
|
the kernel routes service names over:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
# on BOTH hosts
|
||||||
|
sudo tipc bearer enable media eth device eth0
|
||||||
|
# ..or, when L2 isn't available:
|
||||||
|
sudo tipc bearer enable media udp name uc localip 10.0.11.1
|
||||||
|
|
||||||
|
tipc link list # must list the peer before you proceed
|
||||||
|
|
||||||
|
The two-host example pair then talks with **no IP, hostname or
|
||||||
|
port anywhere in either script** — both sides name the same
|
||||||
|
service and the kernel routes it. Move the server to a third
|
||||||
|
node and the client's dial keeps working, unchanged. See
|
||||||
|
``examples/multihost/tipc_cluster/`` for the full walkthrough
|
||||||
|
(that directory is excluded from CI precisely because it needs
|
||||||
|
real hardware).
|
||||||
|
|
||||||
|
Over a WireGuard mesh
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
TIPC over a `wg` mesh is the intended reference deployment for
|
||||||
|
multihost ``tractor`` (see gh #502), composing with the tunnel
|
||||||
|
examples in ``examples/multihost/wg_lan/``.
|
||||||
|
|
||||||
|
.. warning::
|
||||||
|
|
||||||
|
A wg interface is L3/``tun`` — ``POINTOPOINT,NOARP`` with
|
||||||
|
``link/none`` and no L2 address — so TIPC's ``eth`` media
|
||||||
|
**cannot** bind it. Over wg the udp bearer is *mandatory*,
|
||||||
|
not merely an alternative:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
# NOT possible over wg
|
||||||
|
sudo tipc bearer enable media eth device wg0
|
||||||
|
|
||||||
|
# required instead, bound to the wg overlay IP
|
||||||
|
sudo tipc bearer enable media udp name wgmesh \
|
||||||
|
localip 10.0.11.1
|
||||||
|
|
||||||
|
Mind the MTU too: wg links typically sit at 1420, under
|
||||||
|
ethernet's 1500.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
**wg is not about confidentiality here.** TIPC ships AES-GCM
|
||||||
|
crypto of its own (``tipc node set key``, linux 5.9+) with
|
||||||
|
cluster, master and per-node keys plus rekeying intervals, so
|
||||||
|
"wg adds the encryption TIPC lacks" is simply wrong.
|
||||||
|
|
||||||
|
The motivation is different but real: TIPC's keys are
|
||||||
|
*symmetric and pre-shared*, leaving distribution, rotation and
|
||||||
|
revocation to the operator, whereas wg brings public-key
|
||||||
|
identity and a handshake — plus NAT traversal, and one overlay
|
||||||
|
that *every* transport can share rather than a TIPC-only
|
||||||
|
mechanism. Which to prefer is worth benchmarking; native
|
||||||
|
crypto avoids a tunnel hop entirely.
|
||||||
|
|
||||||
|
Gotchas
|
||||||
|
-------
|
||||||
|
|
||||||
|
.. _Silent crosstalk:
|
||||||
|
|
||||||
|
**Silent crosstalk.** Unlike every other backend, a duplicate
|
||||||
|
bind does *not* raise ``EADDRINUSE``. TIPC accepts multiple
|
||||||
|
publishers of one name and **round-robins** connects between
|
||||||
|
them — verified: six dials alternated strictly between two
|
||||||
|
listeners. So an instance collision splits traffic silently
|
||||||
|
instead of erroring. That's why ``TIPCAddress.get_random()``
|
||||||
|
derives its instance from a ``blake2b`` digest of the actor
|
||||||
|
identity rather than a counter, and why two ``tractor`` trees
|
||||||
|
sharing both a cluster **and** an ``_stype`` share a namespace —
|
||||||
|
partition them with a distinct ``_stype``.
|
||||||
|
|
||||||
|
**Graceful close looks like a reset.** A peer closing cleanly
|
||||||
|
surfaces as ``BrokenResourceError``/``ECONNRESET`` rather than
|
||||||
|
the clean 0-byte EOF TCP and UDS give you. Benign — the
|
||||||
|
transport layer already classifies it as a normal disconnect —
|
||||||
|
but it does look alarming in ``transport``-level logs.
|
||||||
|
|
||||||
|
**Dialing an unpublished name** answers ``EHOSTUNREACH``
|
||||||
|
*instantly*, with no SYN-timeout wait. That's markedly better
|
||||||
|
discovery-ping behaviour than TCP; ``tractor`` normalizes it to
|
||||||
|
``ConnectionError`` so the usual lookup paths work unchanged.
|
||||||
|
|
||||||
|
**Multiaddrs are interim.** There's no registered ``/tipc``
|
||||||
|
protocol in the multiaddr table yet, so the grammar is
|
||||||
|
``str``-only:
|
||||||
|
|
||||||
|
.. code:: text
|
||||||
|
|
||||||
|
/tipc/<stype>/<instance>/<scope>
|
||||||
|
|
||||||
|
Composed with a wg bearer — the form that actually matters for
|
||||||
|
multihost — that becomes:
|
||||||
|
|
||||||
|
.. code:: text
|
||||||
|
|
||||||
|
/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>
|
||||||
|
|
||||||
|
Note the tipc segment carries **no** locative component, unlike
|
||||||
|
the ``/ip4/../tcp/..`` inner segment of the equivalent tcp maddr
|
||||||
|
— a TIPC service name is location-independent by design, so the
|
||||||
|
wg segments carry all the routing and the tipc segment is pure
|
||||||
|
identity.
|
||||||
|
|
||||||
|
Running the suite over TIPC
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
The backend is a first-class suite mode — the *entire* existing
|
||||||
|
test suite runs over it unmodified, which is the acceptance bar
|
||||||
|
for any ``tractor`` transport:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
sudo modprobe tipc
|
||||||
|
pytest --tpt-proto tipc
|
||||||
|
|
||||||
|
Without the module that fails loudly and immediately with an
|
||||||
|
actionable message rather than a few hundred confusing connect
|
||||||
|
timeouts. Backend-specific unit tests live in
|
||||||
|
``tests/ipc/test_tipc.py`` and self-skip when the module is
|
||||||
|
absent.
|
||||||
|
|
||||||
|
Normative references
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
The tipc.io documentation is stale in places. Treat the kernel
|
||||||
|
sources as the only authority:
|
||||||
|
|
||||||
|
- ``include/uapi/linux/tipc.h`` — address flavours, sockopts,
|
||||||
|
the topology ``struct``\s
|
||||||
|
- ``net/tipc/socket.c``, ``net/tipc/topsrv.c``
|
||||||
|
- ``man 8 tipc``
|
||||||
|
|
@ -0,0 +1,321 @@
|
||||||
|
# `tractor` over `AF_TIPC`, where the address *is* the service name
|
||||||
|
|
||||||
|
TIPC is a linux-kernel cluster IPC protocol whose service names
|
||||||
|
live in a **cluster-wide name table maintained by the kernel**.
|
||||||
|
It is also described as **Cluster Domain Sockets**: the Unix-domain
|
||||||
|
socket model extended from one kernel to a cluster. That name is a
|
||||||
|
useful explanation for new users, while the code keeps `tipc` as its
|
||||||
|
protocol key to match Linux's `AF_TIPC`, kernel module and tooling.
|
||||||
|
For `tractor` that means:
|
||||||
|
|
||||||
|
- an actor's IPC address is a service name `(stype, instance)`,
|
||||||
|
not a host/port,
|
||||||
|
- `.bind()`ing it **is** service registration,
|
||||||
|
- a peer's `.connect()`-by-name **is** the lookup.
|
||||||
|
|
||||||
|
So the discovery machinery `tractor.discovery` normally
|
||||||
|
implements with a registrar actor comes for free, in-kernel —
|
||||||
|
which is the ask in gh
|
||||||
|
[#378](https://github.com/goodboy/tractor/issues/378).
|
||||||
|
|
||||||
|
> **Why `examples/multihost/`?** `tests/test_docs_examples.py`
|
||||||
|
> walks `examples/` recursively and runs everything it collects
|
||||||
|
> as a subproc, asserting `rc == 0`. These need the `tipc`
|
||||||
|
> kernel module (and, for the two-host pair, a live bearer), so
|
||||||
|
> they can't satisfy that; `'multihost' not in p[0]` is already
|
||||||
|
> in the test's exclusion list, which is what keeps them out of
|
||||||
|
> CI. See "CI" below for the separate matrix-entry plan.
|
||||||
|
|
||||||
|
## the single best demo
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo modprobe tipc
|
||||||
|
python single_host.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Four actors boot, four service names appear in the kernel's
|
||||||
|
table, and all four are withdrawn on teardown — observed with
|
||||||
|
`tipc(8)`, entirely outside `tractor`:
|
||||||
|
|
||||||
|
```
|
||||||
|
--- `tipc nametable show` :: root + 3 subactors ---
|
||||||
|
Type Lower Upper Scope Port
|
||||||
|
1953628160 1616 1616 cluster 3161982128
|
||||||
|
1953628160 1219427151 1219427151 cluster 1587358717
|
||||||
|
1953628160 2641339936 2641339936 cluster 1864021571
|
||||||
|
1953628160 3344505866 3344505866 cluster 3816483388
|
||||||
|
|
||||||
|
--- `tipc nametable show` :: after teardown (all withdrawn) ---
|
||||||
|
Type Lower Upper Scope Port
|
||||||
|
```
|
||||||
|
|
||||||
|
`1953628160` is `0x74720000` — `tractor`'s reserved service
|
||||||
|
type, ascii `tr` in the high half. `1616` is the host-singleton
|
||||||
|
registrar, the same idiom as the TCP port and the
|
||||||
|
`registry@1616.sock` UDS filename. The other three instances are
|
||||||
|
per-actor digests (see "silent crosstalk" below).
|
||||||
|
|
||||||
|
## push-based discovery
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python watch_nametable.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Subscribes to the kernel's *topology service* and prints name
|
||||||
|
table transitions as they happen — no polling, no registrar
|
||||||
|
round-trip:
|
||||||
|
|
||||||
|
```
|
||||||
|
watching the TIPC name table..
|
||||||
|
[+] published instance=1616 port=0x00000000:2375440573
|
||||||
|
spawning subactors..
|
||||||
|
[+] published instance=186947472 port=0x00000000:3960753074
|
||||||
|
[+] published instance=2191362136 port=0x00000000:2263898853
|
||||||
|
[+] published instance=3484369663 port=0x00000000:2126817956
|
||||||
|
tearing down..
|
||||||
|
[-] withdrawn instance=186947472 port=0x00000000:3960753074
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
This is the groundwork for a push registry in
|
||||||
|
`tractor.discovery._registry` (gh
|
||||||
|
[#184](https://github.com/goodboy/tractor/issues/184),
|
||||||
|
[#216](https://github.com/goodboy/tractor/issues/216)) — a
|
||||||
|
registrar that *never polls* `find_actor()`.
|
||||||
|
|
||||||
|
## two hosts
|
||||||
|
|
||||||
|
Everything above is single-node (`modprobe` is enough). To span
|
||||||
|
hosts you need a **bearer** on both, which is the one thing that
|
||||||
|
can't be CI'd.
|
||||||
|
|
||||||
|
For the first physical test, use two wired Linux hosts on the same
|
||||||
|
L2 segment. Prefer a direct cable or uncomplicated switch; avoid
|
||||||
|
Wi-Fi, guest VLANs and port isolation until the basic link works.
|
||||||
|
Use the same checkout and Python environment on both hosts:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# on BOTH hosts
|
||||||
|
git rev-parse HEAD # must match on A and B
|
||||||
|
uv sync --all-extras --dev
|
||||||
|
sudo modprobe tipc
|
||||||
|
|
||||||
|
# choose the real wired iface; do not assume `eth0`
|
||||||
|
ip -br link
|
||||||
|
IFACE=enp3s0
|
||||||
|
|
||||||
|
# inspect existing cluster identity before changing anything
|
||||||
|
tipc node get address # must differ between hosts
|
||||||
|
tipc node get netid # must match between hosts
|
||||||
|
|
||||||
|
# use one private test netid on BOTH hosts, before enabling bearers
|
||||||
|
sudo tipc node set netid 37801
|
||||||
|
|
||||||
|
# ethernet is simplest when the hosts share an L2 segment
|
||||||
|
sudo tipc bearer enable media eth device "$IFACE"
|
||||||
|
|
||||||
|
# ..or over UDP when L2 isn't available — and MANDATORY over a
|
||||||
|
# `wg` mesh, see below
|
||||||
|
sudo tipc bearer enable media udp name uc localip 10.0.11.1
|
||||||
|
|
||||||
|
# verify BEFORE running anything: this must list the peer
|
||||||
|
tipc bearer list
|
||||||
|
tipc link list
|
||||||
|
tipc node list
|
||||||
|
```
|
||||||
|
|
||||||
|
If the link does not appear, first verify carrier, a common TIPC
|
||||||
|
network ID, distinct node addresses, a common VLAN and compatible
|
||||||
|
MTUs. Ethernet TIPC uses EtherType traffic rather than IP routing,
|
||||||
|
so a successful `ping` alone does not prove the bearer can work.
|
||||||
|
|
||||||
|
Run each command from `examples/multihost/tipc_cluster/`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# host A
|
||||||
|
uv run python host_a_srv.py
|
||||||
|
|
||||||
|
# host B
|
||||||
|
watch -n 0.5 tipc nametable show # optional second terminal
|
||||||
|
uv run python host_b_client.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Note what's absent from both scripts: any IP, hostname or port.
|
||||||
|
Both sides name the *same service*, and the kernel routes it.
|
||||||
|
Move `host_a_srv.py` to a third node and host B's dial keeps
|
||||||
|
working, unchanged.
|
||||||
|
|
||||||
|
For a first resilience pass, use a local console or separate
|
||||||
|
management link so the test does not cut off your own SSH session:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# host B: record the healthy baseline
|
||||||
|
tipc link list
|
||||||
|
tipc link statistics show
|
||||||
|
|
||||||
|
# either host: withdraw and recreate the bearer
|
||||||
|
sudo tipc bearer disable media eth device "$IFACE"
|
||||||
|
tipc link list
|
||||||
|
sudo tipc bearer enable media eth device "$IFACE"
|
||||||
|
tipc link list
|
||||||
|
|
||||||
|
# prove name withdrawal/republication and RPC recovery
|
||||||
|
tipc nametable show
|
||||||
|
uv run python host_b_client.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Capture `uname -a`, both node addresses, `tipc bearer list`,
|
||||||
|
`tipc link list`, `tipc link statistics show`, the name table and
|
||||||
|
both Python transcripts. Those artifacts distinguish an actor bug
|
||||||
|
from bearer discovery, cluster identity or switch configuration.
|
||||||
|
|
||||||
|
Clean up a disposable Ethernet test on both hosts with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo tipc bearer disable media eth device "$IFACE"
|
||||||
|
tipc link list
|
||||||
|
```
|
||||||
|
|
||||||
|
Restore any pre-existing network ID only after all bearers are
|
||||||
|
disabled. The first useful automation target is a two-node network
|
||||||
|
namespace fixture that asserts link-up, remote publication, RPC,
|
||||||
|
withdrawal and republication in that order; physical hardware then
|
||||||
|
remains the validation layer for real NIC and switch behaviour.
|
||||||
|
|
||||||
|
The commands above use iproute2's `tipc` frontend, which speaks the
|
||||||
|
kernel's `TIPCv2` generic-netlink family. The planned `pyroute2`
|
||||||
|
dependency already manages WireGuard, interfaces and namespaces and
|
||||||
|
provides generic-netlink primitives, but it does not currently ship a
|
||||||
|
TIPC message codec. Adding one upstream would let `tractor` replace
|
||||||
|
these manual commands with one Python netlink stack instead of
|
||||||
|
shelling out; until then, `tipc(8)` remains the canonical frontend.
|
||||||
|
|
||||||
|
### over a `wg` mesh
|
||||||
|
|
||||||
|
TIPC over WireGuard is the intended reference multihost
|
||||||
|
deployment (gh #502). One hard constraint: a wg interface is
|
||||||
|
L3/`tun` — `POINTOPOINT,NOARP`, `link/none`, no L2 address — so
|
||||||
|
TIPC's `eth` media **cannot** bind it. The udp bearer is
|
||||||
|
mandatory there, bound to the wg overlay IP, and wg's typical
|
||||||
|
1420 MTU sits under ethernet's 1500 so link MTU wants checking.
|
||||||
|
|
||||||
|
Composed, the deployment address is:
|
||||||
|
|
||||||
|
```
|
||||||
|
/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>
|
||||||
|
\____ wg bearer ________/\_key_/\______ tractor ep ________/
|
||||||
|
```
|
||||||
|
|
||||||
|
Note the tipc segment has no locative part, unlike tcp's inner
|
||||||
|
`/ip4/../tcp/..` — a service name is location-independent, so wg
|
||||||
|
carries the routing and tipc carries identity.
|
||||||
|
|
||||||
|
**wg here is not about confidentiality.** TIPC ships AES-GCM
|
||||||
|
crypto of its own (`tipc node set key`, linux 5.9+) with
|
||||||
|
cluster/master/per-node keys and rekeying, so "wg adds the
|
||||||
|
encryption TIPC lacks" is wrong.
|
||||||
|
|
||||||
|
The motivation is different but real: TIPC's keys are *symmetric
|
||||||
|
and pre-shared* — distribution, rotation and revocation are all
|
||||||
|
on you — whereas wg gives public-key identity and a handshake,
|
||||||
|
NAT traversal, and one overlay shared by every transport instead
|
||||||
|
of a TIPC-only mechanism. Worth benchmarking either way; native
|
||||||
|
crypto skips the tunnel hop.
|
||||||
|
|
||||||
|
Note too that the ethernet-bearer pairing #378 imagined does
|
||||||
|
**not** apply over wg: on a given link the L2 path and the wg
|
||||||
|
path are mutually exclusive.
|
||||||
|
|
||||||
|
### scope
|
||||||
|
|
||||||
|
`TIPCAddress._scope` is the backend's `.bindspace` — literally
|
||||||
|
"the set of hosts this published name is reachable from":
|
||||||
|
|
||||||
|
| scope | meaning |
|
||||||
|
| --- | --- |
|
||||||
|
| `TIPC_NODE_SCOPE` | same host only — the UDS analogue |
|
||||||
|
| `TIPC_CLUSTER_SCOPE` | cluster-visible (the default) |
|
||||||
|
|
||||||
|
`TIPC_ZONE_SCOPE` is deprecated and aliased to cluster by modern
|
||||||
|
kernels; `tractor` accepts it on input and folds it, logging at
|
||||||
|
`transport` level.
|
||||||
|
|
||||||
|
## gotchas worth knowing before you deploy
|
||||||
|
|
||||||
|
**Silent crosstalk.** Unlike every other backend, a duplicate
|
||||||
|
bind does **not** raise `EADDRINUSE` — TIPC happily accepts
|
||||||
|
multiple publishers of one name and *round-robins* connects
|
||||||
|
between them (verified: 6 dials alternated `b,a,b,a,b,a`). So an
|
||||||
|
instance collision is silent traffic-splitting, not an error.
|
||||||
|
That's why `TIPCAddress.get_random()` derives the instance from
|
||||||
|
a `blake2b` digest of the actor identity rather than a counter.
|
||||||
|
Two `tractor` trees sharing both a cluster **and** an `_stype`
|
||||||
|
share a name space; partition them by passing a distinct
|
||||||
|
`_stype`.
|
||||||
|
|
||||||
|
**Graceful close looks like a reset.** A peer closing cleanly
|
||||||
|
surfaces as `BrokenResourceError`/`ECONNRESET` rather than the
|
||||||
|
clean 0-byte EOF you get from TCP/UDS. It's benign — the
|
||||||
|
transport layer already classifies it as a normal disconnect —
|
||||||
|
but it does look alarming in `transport`-level logs.
|
||||||
|
|
||||||
|
**Dialing an unpublished name** answers `EHOSTUNREACH`
|
||||||
|
*instantly* (no SYN-timeout wait), which is much better
|
||||||
|
discovery-ping behaviour than TCP. `tractor` normalizes it to
|
||||||
|
`ConnectionError`.
|
||||||
|
|
||||||
|
**It's opt-in, never a default.** The module isn't loaded on
|
||||||
|
most boxes and doesn't exist off-linux, so
|
||||||
|
`enable_transports=['tipc']` is always explicit. Check
|
||||||
|
`tractor.ipc._tipc.is_tipc_available()` before assuming.
|
||||||
|
|
||||||
|
## maddr form
|
||||||
|
|
||||||
|
There is no registered `/tipc` protocol in the multiaddr table
|
||||||
|
yet (upstream track: gh
|
||||||
|
[#483](https://github.com/goodboy/tractor/issues/483) +
|
||||||
|
multiformats/py-multiaddr#107), so the grammar is interim and
|
||||||
|
`str`-only:
|
||||||
|
|
||||||
|
```
|
||||||
|
/tipc/<stype>/<instance>/<scope>
|
||||||
|
```
|
||||||
|
|
||||||
|
`parse_maddr()` special-cases this prefix *before* handing
|
||||||
|
anything to `Multiaddr()`, which would otherwise reject the
|
||||||
|
unregistered name outright. Registering it upstream is what
|
||||||
|
would unblock gh
|
||||||
|
[#443](https://github.com/goodboy/tractor/issues/443)'s
|
||||||
|
"return `Multiaddr` everywhere" item.
|
||||||
|
|
||||||
|
## running the suite over TIPC
|
||||||
|
|
||||||
|
The whole test suite runs under the backend:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo modprobe tipc
|
||||||
|
pytest --tpt-proto tipc
|
||||||
|
```
|
||||||
|
|
||||||
|
Without the module that fails loudly and immediately with an
|
||||||
|
actionable message rather than a few hundred connect timeouts.
|
||||||
|
Backend-specific unit tests live in `tests/ipc/test_tipc.py` and
|
||||||
|
self-skip when the module is absent.
|
||||||
|
|
||||||
|
## CI
|
||||||
|
|
||||||
|
Single-host TIPC *is* CI-able — the module ships with the
|
||||||
|
standard Ubuntu kernel package. CI loads it with `sudo modprobe
|
||||||
|
tipc` and runs the suite with `--tpt-proto tipc` as a blocking
|
||||||
|
matrix leg. Cross-node bearer testing stays manual — this README
|
||||||
|
is that smoke test.
|
||||||
|
|
||||||
|
## normative refs
|
||||||
|
|
||||||
|
The tipc.io docs are stale in places (gh #378 says as much).
|
||||||
|
Treat the kernel sources as the only normative reference:
|
||||||
|
|
||||||
|
- `include/uapi/linux/tipc.h` — address flavours, sockopts, the
|
||||||
|
topology `struct`s
|
||||||
|
- `net/tipc/socket.c`, `net/tipc/topsrv.c`
|
||||||
|
- `man 8 tipc`
|
||||||
|
|
@ -0,0 +1,78 @@
|
||||||
|
'''
|
||||||
|
HOST A — publish a `tractor` service on a cluster-scoped TIPC
|
||||||
|
service name.
|
||||||
|
|
||||||
|
Note what is NOT in this file: any IP address, hostname or port.
|
||||||
|
The actor's address IS the service name `(stype, instance)`, and
|
||||||
|
the kernel routes it over whatever bearer you enabled. Move this
|
||||||
|
process to another node and host B's dial keeps working,
|
||||||
|
unchanged.
|
||||||
|
|
||||||
|
Prereqs on BOTH hosts (see README.md),
|
||||||
|
|
||||||
|
sudo modprobe tipc
|
||||||
|
sudo tipc bearer enable media eth device <iface>
|
||||||
|
tipc link list # must show a link to the peer
|
||||||
|
|
||||||
|
Then here,
|
||||||
|
|
||||||
|
python host_a_srv.py
|
||||||
|
|
||||||
|
'''
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import trio
|
||||||
|
import tractor
|
||||||
|
from tractor.ipc._tipc import (
|
||||||
|
TIPCAddress,
|
||||||
|
is_tipc_available,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@tractor.context
|
||||||
|
async def echo(
|
||||||
|
ctx: tractor.Context,
|
||||||
|
) -> None:
|
||||||
|
await ctx.started()
|
||||||
|
async with ctx.open_stream() as stream:
|
||||||
|
async for msg in stream:
|
||||||
|
print(f'host-a <- {msg!r}')
|
||||||
|
await stream.send(f'{msg} (from host A)')
|
||||||
|
|
||||||
|
|
||||||
|
async def main() -> None:
|
||||||
|
# the host-singleton registrar name, `instance=1616` —
|
||||||
|
# the same "1616 is tractor's registrar" idiom as the tcp
|
||||||
|
# port and the `registry@1616.sock` UDS filename.
|
||||||
|
reg: TIPCAddress = TIPCAddress.get_root()
|
||||||
|
print(f'host A publishing {reg}')
|
||||||
|
|
||||||
|
# A registrar root is not an entry in its own
|
||||||
|
# `Registrar._registry`, so host B cannot discover that root by
|
||||||
|
# its actor name. `open_nursery()` keeps the root as registrar
|
||||||
|
# while the named child below registers as the dialable service.
|
||||||
|
async with tractor.open_nursery(
|
||||||
|
enable_transports=['tipc'],
|
||||||
|
registry_addrs=[reg.unwrap()],
|
||||||
|
) as an:
|
||||||
|
await an.start_actor(
|
||||||
|
'host_a',
|
||||||
|
# Host B imports this same module name to construct the
|
||||||
|
# RPC `NamespacePath`; direct script execution would
|
||||||
|
# otherwise expose it as `__main__` on host A.
|
||||||
|
enable_modules=['host_a_srv'],
|
||||||
|
)
|
||||||
|
print(
|
||||||
|
'host_a up — `tipc nametable show` on EITHER host\n'
|
||||||
|
'should now list this service. ctrl-c to stop.'
|
||||||
|
)
|
||||||
|
await trio.sleep_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if not is_tipc_available():
|
||||||
|
raise RuntimeError(
|
||||||
|
'The `tipc` kernel module is not loaded!\n'
|
||||||
|
' |_try: `sudo modprobe tipc`\n'
|
||||||
|
)
|
||||||
|
trio.run(main)
|
||||||
|
|
@ -0,0 +1,64 @@
|
||||||
|
'''
|
||||||
|
HOST B — dial host A's service *by name*, across the cluster.
|
||||||
|
|
||||||
|
The `.connect()` on a TIPC service name IS the discovery lookup:
|
||||||
|
the kernel resolves the published name to whichever node serves
|
||||||
|
it. So this client needs no IP, no port and no idea where host A
|
||||||
|
actually is.
|
||||||
|
|
||||||
|
Prereqs: same bearer setup as `host_a_srv.py`, and that script
|
||||||
|
already running on the other node.
|
||||||
|
|
||||||
|
python host_b_client.py
|
||||||
|
|
||||||
|
'''
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import trio
|
||||||
|
import tractor
|
||||||
|
from host_a_srv import echo
|
||||||
|
from tractor.ipc._tipc import (
|
||||||
|
TIPCAddress,
|
||||||
|
is_tipc_available,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def main() -> None:
|
||||||
|
reg: TIPCAddress = TIPCAddress.get_root()
|
||||||
|
print(f'host B dialling {reg} (by NAME, not address)')
|
||||||
|
|
||||||
|
async with tractor.open_root_actor(
|
||||||
|
name='host_b',
|
||||||
|
enable_transports=['tipc'],
|
||||||
|
registry_addrs=[reg.unwrap()],
|
||||||
|
):
|
||||||
|
async with tractor.find_actor('host_a') as ptl:
|
||||||
|
if ptl is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
'No `host_a` in the cluster name table!\n'
|
||||||
|
' |_is `host_a_srv.py` running?\n'
|
||||||
|
' |_does `tipc link list` show the peer?\n'
|
||||||
|
)
|
||||||
|
|
||||||
|
# `.open_context()` derives a `NamespacePath` from a
|
||||||
|
# callable. Importing `echo` also loads its module path
|
||||||
|
# locally, while host A's `enable_modules` authorizes the
|
||||||
|
# corresponding remote callable.
|
||||||
|
async with (
|
||||||
|
ptl.open_context(
|
||||||
|
echo,
|
||||||
|
) as (ctx, _),
|
||||||
|
ctx.open_stream() as stream,
|
||||||
|
):
|
||||||
|
for msg in ('hello', 'from', 'the other node'):
|
||||||
|
await stream.send(msg)
|
||||||
|
print(f'host-b <- {await stream.receive()!r}')
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if not is_tipc_available():
|
||||||
|
raise RuntimeError(
|
||||||
|
'The `tipc` kernel module is not loaded!\n'
|
||||||
|
' |_try: `sudo modprobe tipc`\n'
|
||||||
|
)
|
||||||
|
trio.run(main)
|
||||||
|
|
@ -0,0 +1,104 @@
|
||||||
|
'''
|
||||||
|
`tractor` over `AF_TIPC` on a single host.
|
||||||
|
|
||||||
|
Every actor's IPC address is a TIPC *service name*, and binding
|
||||||
|
one publishes it into the kernel's cluster-wide name table. So
|
||||||
|
`tipc nametable show` lists your live actor tree — no registrar
|
||||||
|
query, no `tractor` API, just the kernel telling you what's up.
|
||||||
|
|
||||||
|
Run,
|
||||||
|
|
||||||
|
sudo modprobe tipc
|
||||||
|
python single_host.py
|
||||||
|
|
||||||
|
'''
|
||||||
|
from __future__ import annotations
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
import trio
|
||||||
|
import tractor
|
||||||
|
from tractor.ipc._tipc import (
|
||||||
|
TRACTOR_STYPE,
|
||||||
|
is_tipc_available,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def show_nametable(tag: str) -> None:
|
||||||
|
'''
|
||||||
|
Dump the kernel name-table rows belonging to `tractor`.
|
||||||
|
|
||||||
|
'''
|
||||||
|
print(f'\n--- `tipc nametable show` :: {tag} ---')
|
||||||
|
out = subprocess.run(
|
||||||
|
['tipc', 'nametable', 'show'],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
)
|
||||||
|
for line in out.stdout.splitlines():
|
||||||
|
# header, or one of *our* service-type rows
|
||||||
|
if (
|
||||||
|
line.startswith('Type')
|
||||||
|
or
|
||||||
|
line.startswith(str(TRACTOR_STYPE))
|
||||||
|
):
|
||||||
|
print(f' {line}')
|
||||||
|
|
||||||
|
|
||||||
|
@tractor.context
|
||||||
|
async def wait_until_cancelled(
|
||||||
|
ctx: tractor.Context,
|
||||||
|
) -> None:
|
||||||
|
await ctx.started()
|
||||||
|
await trio.sleep_forever()
|
||||||
|
|
||||||
|
|
||||||
|
async def main() -> None:
|
||||||
|
async with tractor.open_nursery(
|
||||||
|
enable_transports=['tipc'],
|
||||||
|
) as an:
|
||||||
|
|
||||||
|
show_nametable('root only')
|
||||||
|
|
||||||
|
portals: list[tractor.Portal] = []
|
||||||
|
for name in ('donny', 'walter', 'dude'):
|
||||||
|
portals.append(
|
||||||
|
await an.start_actor(
|
||||||
|
name,
|
||||||
|
enable_modules=[__name__],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
async with trio.open_nursery() as tn:
|
||||||
|
for ptl in portals:
|
||||||
|
tn.start_soon(
|
||||||
|
_hold_open,
|
||||||
|
ptl,
|
||||||
|
)
|
||||||
|
await trio.sleep(0.5)
|
||||||
|
|
||||||
|
# XXX the money shot: 4 actors, 4 published names
|
||||||
|
show_nametable('root + 3 subactors')
|
||||||
|
|
||||||
|
tn.cancel_scope.cancel()
|
||||||
|
|
||||||
|
await an.cancel()
|
||||||
|
|
||||||
|
show_nametable('after teardown (all withdrawn)')
|
||||||
|
|
||||||
|
|
||||||
|
async def _hold_open(
|
||||||
|
ptl: tractor.Portal,
|
||||||
|
) -> None:
|
||||||
|
async with ptl.open_context(
|
||||||
|
wait_until_cancelled,
|
||||||
|
) as (ctx, _):
|
||||||
|
await trio.sleep_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if not is_tipc_available():
|
||||||
|
raise RuntimeError(
|
||||||
|
'The `tipc` kernel module is not loaded!\n'
|
||||||
|
' |_try: `sudo modprobe tipc`\n'
|
||||||
|
)
|
||||||
|
trio.run(main)
|
||||||
|
|
@ -0,0 +1,93 @@
|
||||||
|
'''
|
||||||
|
Watch `tractor` actors (de)register themselves, live, via TIPC's
|
||||||
|
topology service.
|
||||||
|
|
||||||
|
`open_topology_events()` subscribes to the kernel's name table
|
||||||
|
and yields a `trio` receive-channel of `publish`/`withdraw`
|
||||||
|
events. That's **push-based** service discovery: no registrar
|
||||||
|
round-trip, no polling — the kernel tells you the instant any
|
||||||
|
actor anywhere in the cluster comes or goes.
|
||||||
|
|
||||||
|
Run,
|
||||||
|
|
||||||
|
sudo modprobe tipc
|
||||||
|
python watch_nametable.py
|
||||||
|
|
||||||
|
'''
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import trio
|
||||||
|
import tractor
|
||||||
|
from tractor.ipc._tipc import (
|
||||||
|
TIPCNameEvent,
|
||||||
|
is_tipc_available,
|
||||||
|
open_topology_events,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@tractor.context
|
||||||
|
async def wait_until_cancelled(
|
||||||
|
ctx: tractor.Context,
|
||||||
|
) -> None:
|
||||||
|
await ctx.started()
|
||||||
|
await trio.sleep_forever()
|
||||||
|
|
||||||
|
|
||||||
|
async def print_events(
|
||||||
|
events: trio.MemoryReceiveChannel[TIPCNameEvent],
|
||||||
|
) -> None:
|
||||||
|
glyphs: dict[str, str] = {
|
||||||
|
'published': '[+]',
|
||||||
|
'withdrawn': '[-]',
|
||||||
|
'timeout': '[!]',
|
||||||
|
}
|
||||||
|
async for ev in events:
|
||||||
|
print(
|
||||||
|
f' {glyphs.get(ev.kind, "[?]")} {ev.kind:<10} '
|
||||||
|
f'instance={ev.addr._instance:<12} '
|
||||||
|
f'port=0x{ev.node:08x}:{ev.ref}'
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def main() -> None:
|
||||||
|
# NOTE, subscribe BEFORE booting the runtime so we catch the
|
||||||
|
# root actor's own publication too.
|
||||||
|
async with open_topology_events() as events:
|
||||||
|
async with trio.open_nursery() as tn:
|
||||||
|
tn.start_soon(print_events, events)
|
||||||
|
|
||||||
|
print('watching the TIPC name table..\n')
|
||||||
|
async with tractor.open_nursery(
|
||||||
|
enable_transports=['tipc'],
|
||||||
|
) as an:
|
||||||
|
await trio.sleep(0.3)
|
||||||
|
|
||||||
|
print('\nspawning subactors..')
|
||||||
|
portals: list[tractor.Portal] = []
|
||||||
|
for name in ('donny', 'walter', 'dude'):
|
||||||
|
portals.append(
|
||||||
|
await an.start_actor(
|
||||||
|
name,
|
||||||
|
enable_modules=[__name__],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await trio.sleep(0.2)
|
||||||
|
|
||||||
|
print('\ntearing down..')
|
||||||
|
for ptl in portals:
|
||||||
|
await ptl.cancel_actor()
|
||||||
|
await trio.sleep(0.2)
|
||||||
|
|
||||||
|
await an.cancel()
|
||||||
|
|
||||||
|
await trio.sleep(0.5)
|
||||||
|
tn.cancel_scope.cancel()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
if not is_tipc_available():
|
||||||
|
raise RuntimeError(
|
||||||
|
'The `tipc` kernel module is not loaded!\n'
|
||||||
|
' |_try: `sudo modprobe tipc`\n'
|
||||||
|
)
|
||||||
|
trio.run(main)
|
||||||
|
|
@ -0,0 +1,171 @@
|
||||||
|
# `tractor` over a WireGuard tunnel, declared as one maddr
|
||||||
|
|
||||||
|
A two-host LAN setup: a `tractor` actor tree on host A, dialed
|
||||||
|
from host B, with the endpoint declared as a single `wg`
|
||||||
|
multiaddr.
|
||||||
|
|
||||||
|
Supersedes the example set in gh
|
||||||
|
[#482](https://github.com/goodboy/tractor/issues/482) — see
|
||||||
|
[what changed](#what-changed-vs-482).
|
||||||
|
|
||||||
|
> **Why `examples/multihost/`?** `tests/test_docs_examples.py`
|
||||||
|
> walks `examples/` recursively and runs everything it collects
|
||||||
|
> as a subproc, asserting `rc == 0`. These need a real second
|
||||||
|
> host and a live `wg` tunnel, so they can't satisfy that;
|
||||||
|
> `'multihost' not in p[0]` is already in the test's exclusion
|
||||||
|
> list, which is what keeps them out of CI.
|
||||||
|
|
||||||
|
## the maddr form
|
||||||
|
|
||||||
|
```
|
||||||
|
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
|
||||||
|
\____ wg bearer ___________/\__ key __/\____ tractor ep _____/
|
||||||
|
underlay, wg `ListenPort` overlay, on the wg iface
|
||||||
|
(kernel/`wg(8)` owns it) (the ONLY part tractor binds)
|
||||||
|
```
|
||||||
|
|
||||||
|
Three parts, three different owners:
|
||||||
|
|
||||||
|
| part | who binds it | in the runtime? |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `/ip4/../udp/51820` bearer | kernel via `wg-quick`/`pyroute2` | no |
|
||||||
|
| `/wg/u<key>` | nothing — it's an identity | no, verified out-of-band |
|
||||||
|
| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | **yes**, as `.inner` |
|
||||||
|
|
||||||
|
Verified against py-multiaddr
|
||||||
|
[#108](https://github.com/multiformats/py-multiaddr/pull/108):
|
||||||
|
this composed form parses and round-trips
|
||||||
|
(`['ip4','udp','wg','ip4','tcp']`).
|
||||||
|
|
||||||
|
## requirements
|
||||||
|
|
||||||
|
py-multiaddr #108 is **merged** (2026-07-28) but ships in no
|
||||||
|
release yet — the latest `0.2.0` (2026-03-17) predates it and has
|
||||||
|
no `wg` codec. So `pyproject.toml` carries a temporary
|
||||||
|
`[tool.uv.sources]` `rev` pin at the merge commit, and a plain
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv sync
|
||||||
|
```
|
||||||
|
|
||||||
|
gets you a `wg`-aware `multiaddr`. That pin goes away once a
|
||||||
|
release carries the codec. You also need `multibase`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv pip install multibase
|
||||||
|
```
|
||||||
|
|
||||||
|
Without the codec `wg_maddr.py` degrades to a plain segment split
|
||||||
|
— the examples still run, but you lose per-segment validation
|
||||||
|
(incl. the 32-byte key-length check), so a malformed key reaches
|
||||||
|
the returned struct instead of raising. `_have_wg_maddr_proto()`
|
||||||
|
is the gate. It deliberately does **not** hand-roll a `wg` codec
|
||||||
|
(gh #429 was about *dropping* our NIH parser).
|
||||||
|
|
||||||
|
## 0. tunnel setup (out-of-band, both hosts)
|
||||||
|
|
||||||
|
Host A is the service host (underlay e.g. `192.168.1.50`), host B
|
||||||
|
your workstation. Overlay net `10.0.11.0/24`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
umask 077
|
||||||
|
wg genkey | tee wg_priv.key | wg pubkey > wg_pub.key
|
||||||
|
```
|
||||||
|
|
||||||
|
`/etc/wireguard/wg0.conf` on **host A**:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Interface]
|
||||||
|
PrivateKey = <A_priv>
|
||||||
|
Address = 10.0.11.1/24
|
||||||
|
ListenPort = 51820
|
||||||
|
```
|
||||||
|
```ini
|
||||||
|
[Peer]
|
||||||
|
PublicKey = <B_pub>
|
||||||
|
AllowedIPs = 10.0.11.2/32
|
||||||
|
```
|
||||||
|
|
||||||
|
on **host B**:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Interface]
|
||||||
|
PrivateKey = <B_priv>
|
||||||
|
Address = 10.0.11.2/24
|
||||||
|
```
|
||||||
|
```ini
|
||||||
|
[Peer]
|
||||||
|
PublicKey = <A_pub>
|
||||||
|
Endpoint = 192.168.1.50:51820
|
||||||
|
AllowedIPs = 10.0.11.1/32
|
||||||
|
PersistentKeepalive = 25
|
||||||
|
```
|
||||||
|
|
||||||
|
Note how `ListenPort` and `Endpoint` are exactly the maddr's
|
||||||
|
bearer segment, and `[Interface] Address` is its overlay host.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo wg-quick up wg0 # both hosts
|
||||||
|
ping -c1 10.0.11.1 # from B
|
||||||
|
```
|
||||||
|
|
||||||
|
## 1. get your pubkey into the maddr
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -c "
|
||||||
|
import base64, multibase
|
||||||
|
key = open('wg_pub.key').read().strip()
|
||||||
|
print(multibase.encode('base64url', base64.b64decode(key)).decode())
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
Paste the `u...` output into `WG_MADDR` in both scripts (they use
|
||||||
|
the same string — A's bearer, A's key, A's overlay ep).
|
||||||
|
|
||||||
|
## 2. run
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# host A
|
||||||
|
python host_a_srv.py
|
||||||
|
|
||||||
|
# host B
|
||||||
|
python host_b_client.py
|
||||||
|
```
|
||||||
|
|
||||||
|
`host_a_srv.py` must be importable on host B too, since
|
||||||
|
`portal.run()` refs the fn by module path — standard `tractor`
|
||||||
|
RPC semantics.
|
||||||
|
|
||||||
|
## what changed vs #482
|
||||||
|
|
||||||
|
Four corrections, all from
|
||||||
|
`ai/tpt-backends/03_wg_tunnel_bindspace.md`:
|
||||||
|
|
||||||
|
1. **the maddr semantics were inverted.** #482 used
|
||||||
|
`/ip4/10.0.11.1/tcp/1616/wg/u<key>` — that parses, but it puts
|
||||||
|
the *overlay* addr where the bearer belongs and `tcp` where
|
||||||
|
wg's `udp` `ListenPort` goes, and it declares no overlay ep at
|
||||||
|
all. `parse_wg_maddr()` now rejects it with an actionable
|
||||||
|
error.
|
||||||
|
2. **parsing is pure.** #482's helper had the key-check adjacent
|
||||||
|
to the parse; `verify_wg_peer()` is now a separate, explicitly
|
||||||
|
composed step that the caller invokes. A parser that shells
|
||||||
|
out is a nasty surprise.
|
||||||
|
3. **no `sudo`.** #482 ran `sudo wg show`; a library/example must
|
||||||
|
never escalate. `wg show` works unprivileged for read on most
|
||||||
|
setups; if yours needs root, run the script as root rather
|
||||||
|
than embedding `sudo`.
|
||||||
|
4. **no new `Address` proto-type.** The tunnel rides *beside* the
|
||||||
|
inner addr in a frozen `WGTunnelledAddr`, and only `.inner`
|
||||||
|
crosses into `open_nursery()`. #482 §6 floated a `WGAddress`
|
||||||
|
registered in `_address_types` — that table is a `bidict`
|
||||||
|
(1:1 proto-key↔type) and `_addr_to_transport` wants a
|
||||||
|
`MsgTransport` per addr-type, which `wg` doesn't have.
|
||||||
|
|
||||||
|
## next
|
||||||
|
|
||||||
|
`WGTunnelledAddr` is deliberately example-local. Promoting it to
|
||||||
|
`tractor.discovery` as a `TunnelledAddress` whose
|
||||||
|
`.proto_key`/`.unwrap()` delegate to `.inner`, plus
|
||||||
|
`open_bindspace()` `@acm`s that create/tear down the iface +
|
||||||
|
netns via `pyroute2`, is layers A→C of the plan doc.
|
||||||
|
|
@ -0,0 +1,61 @@
|
||||||
|
# tractor: distributed structured concurrency.
|
||||||
|
'''
|
||||||
|
Host A: the service host, reachable over a `wg` tunnel.
|
||||||
|
|
||||||
|
Binds `tractor`'s registrar + an `echo_srv` sub-actor on the
|
||||||
|
tunnel's *overlay* addr, declared as a single `wg` maddr.
|
||||||
|
|
||||||
|
'''
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import tractor
|
||||||
|
import trio
|
||||||
|
|
||||||
|
from wg_maddr import (
|
||||||
|
parse_wg_maddr,
|
||||||
|
verify_wg_peer,
|
||||||
|
WGTunnelledAddr,
|
||||||
|
)
|
||||||
|
|
||||||
|
# bearer = host A's underlay `(ip, wg ListenPort)`
|
||||||
|
# key = host A's OWN tunnel pubkey
|
||||||
|
# overlay = the ep `tractor` binds, on the wg iface's addr
|
||||||
|
WG_MADDR: str = (
|
||||||
|
'/ip4/192.168.1.50/udp/51820'
|
||||||
|
'/wg/u<A_pub_b64url>'
|
||||||
|
'/ip4/10.0.11.1/tcp/1616'
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def echo(msg: str) -> str:
|
||||||
|
actor = tractor.current_actor()
|
||||||
|
return f'{actor.aid.name!r} echoes: {msg}'
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR)
|
||||||
|
assert verify_wg_peer(addr), (
|
||||||
|
f'wg pubkey from maddr not active on wg0 !\n'
|
||||||
|
f'maddr: {WG_MADDR}\n'
|
||||||
|
f'key: {addr.peer_pubkey}\n'
|
||||||
|
)
|
||||||
|
print(
|
||||||
|
f'wg bearer (kernel-owned): {addr.bearer}\n'
|
||||||
|
f'tractor overlay ep: {addr.inner}\n'
|
||||||
|
)
|
||||||
|
async with tractor.open_nursery(
|
||||||
|
# XXX only `.inner` crosses into the runtime; the bearer
|
||||||
|
# + key are iface-layer concerns `tractor` never binds.
|
||||||
|
registry_addrs=[addr.inner],
|
||||||
|
enable_transports=[addr.inner_proto],
|
||||||
|
) as an:
|
||||||
|
await an.start_actor(
|
||||||
|
'echo_srv',
|
||||||
|
enable_modules=[__name__],
|
||||||
|
)
|
||||||
|
print(f'echo_srv up on\n {addr.maddr}\n')
|
||||||
|
await trio.sleep_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
trio.run(main)
|
||||||
|
|
@ -0,0 +1,52 @@
|
||||||
|
# tractor: distributed structured concurrency.
|
||||||
|
'''
|
||||||
|
Host B: workstation dialing host A's actor tree through the
|
||||||
|
`wg` tunnel.
|
||||||
|
|
||||||
|
'''
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import tractor
|
||||||
|
import trio
|
||||||
|
|
||||||
|
from host_a_srv import echo # noqa: F401 (RPC refs it by mod path)
|
||||||
|
from wg_maddr import (
|
||||||
|
parse_wg_maddr,
|
||||||
|
verify_wg_peer,
|
||||||
|
WGTunnelledAddr,
|
||||||
|
)
|
||||||
|
|
||||||
|
# same maddr as host A: A's bearer, A's key, A's overlay ep
|
||||||
|
WG_MADDR: str = (
|
||||||
|
'/ip4/192.168.1.50/udp/51820'
|
||||||
|
'/wg/u<A_pub_b64url>'
|
||||||
|
'/ip4/10.0.11.1/tcp/1616'
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR)
|
||||||
|
assert verify_wg_peer(addr), (
|
||||||
|
f'wg pubkey from maddr not a peer on wg0 !\n'
|
||||||
|
f'maddr: {WG_MADDR}\n'
|
||||||
|
)
|
||||||
|
async with (
|
||||||
|
tractor.open_root_actor(
|
||||||
|
name='wg_client',
|
||||||
|
registry_addrs=[addr.inner],
|
||||||
|
enable_transports=[addr.inner_proto],
|
||||||
|
),
|
||||||
|
tractor.find_actor(
|
||||||
|
'echo_srv',
|
||||||
|
registry_addrs=[addr.inner],
|
||||||
|
) as portal,
|
||||||
|
):
|
||||||
|
res: str = await portal.run(
|
||||||
|
echo,
|
||||||
|
msg='hello over wg!',
|
||||||
|
)
|
||||||
|
print(res)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
trio.run(main)
|
||||||
|
|
@ -0,0 +1,248 @@
|
||||||
|
# tractor: distributed structured concurrency.
|
||||||
|
r'''
|
||||||
|
Parse `wg`-tunnelled multiaddrs into `tractor`-ready addrs.
|
||||||
|
|
||||||
|
The canonical form (per py-multiaddr #108, verified to parse +
|
||||||
|
round-trip against its upstream merge) nests the *overlay*
|
||||||
|
endpoint **after** the `/wg/` segment:
|
||||||
|
|
||||||
|
/ip4/10.0.0.1/udp/51820/wg/u<key>/ip4/10.0.11.1/tcp/1616
|
||||||
|
\_______ wg bearer ______/\_ key _/\____ tractor ep _____/
|
||||||
|
(underlay, wg
|
||||||
|
`ListenPort`)
|
||||||
|
|
||||||
|
- the segments *before* `/wg/` are the **bearer**: the underlay
|
||||||
|
`(ip, udp-port)` that `wg(8)` itself listens on. Nothing in
|
||||||
|
`tractor` ever binds this — the kernel/`wg` iface owns it.
|
||||||
|
- `/wg/u<key>` carries the tunnel peer's Curve25519 pubkey as
|
||||||
|
multibase base64url (std base64 from `wg(8)` contains `/` and
|
||||||
|
can't go in a `/`-delimited maddr).
|
||||||
|
- the segments *after* are the **overlay** endpoint, i.e. the
|
||||||
|
addr `tractor` actually binds/dials. This is the only part the
|
||||||
|
runtime sees.
|
||||||
|
|
||||||
|
XXX NOTE, `tractor`'s own `parse_maddr()` can't parse this yet
|
||||||
|
(`ValueError('Unsupported multiaddr protocol combo')`), which is
|
||||||
|
why this module exists: parse here, hand `.inner` to the runtime.
|
||||||
|
|
||||||
|
Design rules this module follows (see
|
||||||
|
`ai/tpt-backends/03_wg_tunnel_bindspace.md`):
|
||||||
|
|
||||||
|
- **parsing is pure**. `parse_wg_maddr()` does no I/O, no
|
||||||
|
`subprocess`, no netlink. A parser that shells out is a nasty
|
||||||
|
surprise.
|
||||||
|
- **verification is an explicit, separate step**. The caller
|
||||||
|
composes `verify_wg_peer()` when it wants it; nothing implicit.
|
||||||
|
- **no new `Address` proto-type**. `wg` gets no entry in
|
||||||
|
`tractor.discovery._addr._address_types` (a `bidict`, so 1:1
|
||||||
|
proto-key<->type) bc it has no `MsgTransport` of its own. The
|
||||||
|
tunnel is a *bindspace*, so we carry it beside the inner addr
|
||||||
|
and strip to `.inner` at bind/dial time.
|
||||||
|
|
||||||
|
'''
|
||||||
|
from __future__ import annotations
|
||||||
|
import base64
|
||||||
|
import subprocess
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
import msgspec
|
||||||
|
|
||||||
|
|
||||||
|
class WGTunnelledAddr(
|
||||||
|
msgspec.Struct,
|
||||||
|
frozen=True,
|
||||||
|
):
|
||||||
|
'''
|
||||||
|
A `wg`-tunnelled endpoint: the underlay bearer, the tunnel
|
||||||
|
peer key, and the overlay addr `tractor` binds/dials.
|
||||||
|
|
||||||
|
'''
|
||||||
|
# underlay, owned by `wg(8)`/the kernel — NEVER bound by us
|
||||||
|
bearer: tuple[str, int]
|
||||||
|
|
||||||
|
# tunnel peer pubkey in the std-base64 `wg(8)` form, i.e.
|
||||||
|
# directly comparable to `wg show <if> peers` output
|
||||||
|
peer_pubkey: str
|
||||||
|
|
||||||
|
# overlay ep: an `UnwrappedAddress` as accepted by
|
||||||
|
# `tractor.discovery.wrap_address()`
|
||||||
|
inner: tuple[str, int]
|
||||||
|
inner_proto: Literal['tcp'] = 'tcp'
|
||||||
|
|
||||||
|
@property
|
||||||
|
def maddr(self) -> str:
|
||||||
|
'''
|
||||||
|
Re-render the canonical maddr `str` form.
|
||||||
|
|
||||||
|
'''
|
||||||
|
b_host, b_port = self.bearer
|
||||||
|
i_host, i_port = self.inner
|
||||||
|
return (
|
||||||
|
f'/ip4/{b_host}/udp/{b_port}'
|
||||||
|
f'/wg/{mb_pubkey(self.peer_pubkey)}'
|
||||||
|
f'/ip4/{i_host}/{self.inner_proto}/{i_port}'
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def mb_pubkey(wg8_key: str) -> str:
|
||||||
|
'''
|
||||||
|
`wg(8)` std-base64 pubkey -> multibase base64url (`u`-prefixed).
|
||||||
|
|
||||||
|
'''
|
||||||
|
import multibase
|
||||||
|
raw: bytes = base64.b64decode(wg8_key)
|
||||||
|
return multibase.encode('base64url', raw).decode('ascii')
|
||||||
|
|
||||||
|
|
||||||
|
def wg8_pubkey(mb_key: str) -> str:
|
||||||
|
'''
|
||||||
|
Inverse of `mb_pubkey()`: multibase -> `wg(8)` std-base64.
|
||||||
|
|
||||||
|
'''
|
||||||
|
import multibase
|
||||||
|
raw: bytes = multibase.decode(mb_key)
|
||||||
|
return base64.b64encode(raw).decode('ascii')
|
||||||
|
|
||||||
|
|
||||||
|
def parse_wg_maddr(
|
||||||
|
maddr: str,
|
||||||
|
) -> WGTunnelledAddr:
|
||||||
|
'''
|
||||||
|
Split a `wg`-tunnelled maddr into its bearer/key/overlay
|
||||||
|
parts. Pure — no I/O.
|
||||||
|
|
||||||
|
Total-or-raises: with a `wg`-aware `py-multiaddr` (#108) an
|
||||||
|
unparseable maddr raises instead of yielding a struct built
|
||||||
|
from garbage segments. See `_segments()` for the degraded
|
||||||
|
pre-#108 path.
|
||||||
|
|
||||||
|
'''
|
||||||
|
segs: list[str] = _segments(maddr)
|
||||||
|
try:
|
||||||
|
wg_at: int = segs.index('wg')
|
||||||
|
except ValueError:
|
||||||
|
raise ValueError(
|
||||||
|
f'Not a `wg`-tunnelled maddr, no `/wg/` segment ??\n'
|
||||||
|
f'maddr: {maddr!r}\n'
|
||||||
|
)
|
||||||
|
|
||||||
|
bearer_segs: list[str] = segs[:wg_at]
|
||||||
|
mb_key: str = segs[wg_at + 1]
|
||||||
|
inner_segs: list[str] = segs[wg_at + 2:]
|
||||||
|
|
||||||
|
match bearer_segs:
|
||||||
|
case ['ip4'|'ip6', str() as b_host, 'udp', str() as b_port]:
|
||||||
|
bearer = (b_host, int(b_port))
|
||||||
|
case _:
|
||||||
|
raise ValueError(
|
||||||
|
f'Bad `wg` bearer, expected `/ip4|ip6/<h>/udp/<p>`\n'
|
||||||
|
f'got: {"/".join(bearer_segs)!r}\n'
|
||||||
|
f'from maddr: {maddr!r}\n'
|
||||||
|
)
|
||||||
|
|
||||||
|
match inner_segs:
|
||||||
|
case ['ip4'|'ip6', str() as i_host, 'tcp', str() as i_port]:
|
||||||
|
inner = (i_host, int(i_port))
|
||||||
|
inner_proto = 'tcp'
|
||||||
|
case []:
|
||||||
|
raise ValueError(
|
||||||
|
f'`wg` maddr declares no overlay endpoint!\n'
|
||||||
|
f'A bare `/…/wg/<key>` names only the tunnel; '
|
||||||
|
f'append the ep `tractor` should bind, e.g.\n'
|
||||||
|
f' {maddr}/ip4/10.0.11.1/tcp/1616\n'
|
||||||
|
)
|
||||||
|
case _:
|
||||||
|
raise ValueError(
|
||||||
|
f'Unsupported `wg` overlay proto combo\n'
|
||||||
|
f'got: {"/".join(inner_segs)!r}\n'
|
||||||
|
f'from maddr: {maddr!r}\n'
|
||||||
|
)
|
||||||
|
|
||||||
|
return WGTunnelledAddr(
|
||||||
|
bearer=bearer,
|
||||||
|
peer_pubkey=wg8_pubkey(mb_key),
|
||||||
|
inner=inner,
|
||||||
|
inner_proto=inner_proto,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
_wg_proto_known: bool|None = None
|
||||||
|
|
||||||
|
|
||||||
|
def _have_wg_maddr_proto() -> bool:
|
||||||
|
'''
|
||||||
|
True iff the installed `py-multiaddr` knows the `/wg/` proto,
|
||||||
|
i.e. carries py-multiaddr#108.
|
||||||
|
|
||||||
|
Merged upstream 2026-07-28 but in no release as of `0.2.0`,
|
||||||
|
hence the `[tool.uv.sources]` `rev` pin.
|
||||||
|
|
||||||
|
Pure predicate; result cached since it can't change without a
|
||||||
|
reinstall.
|
||||||
|
|
||||||
|
'''
|
||||||
|
global _wg_proto_known
|
||||||
|
if _wg_proto_known is None:
|
||||||
|
from multiaddr.protocols import protocol_with_name
|
||||||
|
from multiaddr.exceptions import ProtocolNotFoundError
|
||||||
|
try:
|
||||||
|
protocol_with_name('wg')
|
||||||
|
_wg_proto_known = True
|
||||||
|
except ProtocolNotFoundError:
|
||||||
|
_wg_proto_known = False
|
||||||
|
|
||||||
|
return _wg_proto_known
|
||||||
|
|
||||||
|
|
||||||
|
def _segments(maddr: str) -> list[str]:
|
||||||
|
'''
|
||||||
|
Deliver a maddr's `/`-split segments, validating via the real
|
||||||
|
parser whenever it knows `wg`.
|
||||||
|
|
||||||
|
'''
|
||||||
|
if _have_wg_maddr_proto():
|
||||||
|
from multiaddr import Multiaddr
|
||||||
|
# the real thing: validates every proto + value, incl.
|
||||||
|
# that the `wg` key decodes to exactly 32 bytes. Let it
|
||||||
|
# raise — a maddr that doesn't parse must NOT reach
|
||||||
|
# `wg8_pubkey()`, which would happily emit a corrupt key.
|
||||||
|
Multiaddr(maddr)
|
||||||
|
|
||||||
|
# XXX, degraded path for a pre-#108 `py-multiaddr` ONLY: no
|
||||||
|
# per-segment validation, so a malformed key survives to the
|
||||||
|
# returned struct. We deliberately DON'T hand-roll a `wg`
|
||||||
|
# codec (the whole point of gh #429 was dropping the NIH
|
||||||
|
# parser) — install the pinned rev to get validation back.
|
||||||
|
return [s for s in maddr.split('/') if s]
|
||||||
|
|
||||||
|
|
||||||
|
def verify_wg_peer(
|
||||||
|
addr: WGTunnelledAddr,
|
||||||
|
iface: str = 'wg0',
|
||||||
|
) -> bool:
|
||||||
|
'''
|
||||||
|
True iff `addr.peer_pubkey` is a configured peer (or our own
|
||||||
|
pubkey) on `iface`.
|
||||||
|
|
||||||
|
IMPURE + explicit by design: never called from
|
||||||
|
`parse_wg_maddr()`.
|
||||||
|
|
||||||
|
?TODO, per plan-03 layer B, swap this body for `pyroute2`
|
||||||
|
(keeping the signature) — and note `setns(2)` is *per-thread*,
|
||||||
|
so a query issued via `trio.to_thread` lands in the ORIGINAL
|
||||||
|
netns unless `netns=` is passed down.
|
||||||
|
|
||||||
|
'''
|
||||||
|
def _wg(*args: str) -> str:
|
||||||
|
return subprocess.run(
|
||||||
|
['wg', 'show', iface, *args],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
check=True,
|
||||||
|
).stdout
|
||||||
|
|
||||||
|
return (
|
||||||
|
addr.peer_pubkey in _wg('peers').split()
|
||||||
|
or
|
||||||
|
addr.peer_pubkey == _wg('public-key').strip()
|
||||||
|
)
|
||||||
|
|
@ -166,6 +166,17 @@ sync_pause = {requires-python = ">=3.13, <3.14"}
|
||||||
# linux kernel networking
|
# linux kernel networking
|
||||||
# 'pyroute2
|
# 'pyroute2
|
||||||
|
|
||||||
|
# XXX TEMP, the `/wg/u<key>` maddr proto is MERGED upstream (in
|
||||||
|
# py-multiaddr#108, 2026-07-28) but is in NO release yet; the
|
||||||
|
# latest `0.2.0` (2026-03-17) predates the merge by ~4 months.
|
||||||
|
# Pinned by `rev` (not `branch`) so CI stays reproducible.
|
||||||
|
#
|
||||||
|
# Drop this pin (and bump the `multiaddr` dep floor above) the
|
||||||
|
# moment a release carries the `wg` codec; the only consumer is
|
||||||
|
# `examples/multihost/wg_lan/`.
|
||||||
|
# |_https://github.com/multiformats/py-multiaddr/pull/108
|
||||||
|
multiaddr = { git = 'https://github.com/multiformats/py-multiaddr.git', rev = 'f86519daaa21699023d0037c58cdff600313dd09' }
|
||||||
|
|
||||||
# ------ tool.uv.sources ------
|
# ------ tool.uv.sources ------
|
||||||
|
|
||||||
[tool.uv]
|
[tool.uv]
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,52 @@
|
||||||
|
'''
|
||||||
|
Unit tests for the `tractor.devx.pformat` render helpers.
|
||||||
|
|
||||||
|
'''
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from tractor.devx.pformat import (
|
||||||
|
pformat_boxed_tb,
|
||||||
|
pformat_caller_frame,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
'box_tb',
|
||||||
|
[True, False],
|
||||||
|
ids=['boxed', 'bare'],
|
||||||
|
)
|
||||||
|
def test_pformat_caller_frame_renders(box_tb: bool):
|
||||||
|
'''
|
||||||
|
`pformat_caller_frame()` must render, not raise.
|
||||||
|
|
||||||
|
XXX the `box_tb=True` branch was passing an `indent=''` kwarg
|
||||||
|
that `pformat_boxed_tb()` never accepted, so it blew up with
|
||||||
|
a `TypeError`. Nothing in the test suite covered it, and the
|
||||||
|
only caller is `_mk_send_mte()` — i.e. EVERY send-side
|
||||||
|
`MsgTypeError` died while formatting itself, masking the real
|
||||||
|
msg-spec violation behind a bogus `TypeError`.
|
||||||
|
|
||||||
|
'''
|
||||||
|
report: str = pformat_caller_frame(
|
||||||
|
stack_limit=3,
|
||||||
|
box_tb=box_tb,
|
||||||
|
)
|
||||||
|
assert isinstance(report, str)
|
||||||
|
assert 'test_pformat_caller_frame_renders' in report
|
||||||
|
|
||||||
|
|
||||||
|
def test_pformat_boxed_tb_rejects_unknown_kwargs():
|
||||||
|
'''
|
||||||
|
Pin the signature so a future typo'd kwarg fails loudly at the
|
||||||
|
call site rather than only when some rare error path runs.
|
||||||
|
|
||||||
|
'''
|
||||||
|
assert pformat_boxed_tb(tb_str='doggy\n')
|
||||||
|
|
||||||
|
with pytest.raises(TypeError):
|
||||||
|
pformat_boxed_tb(
|
||||||
|
tb_str='doggy\n',
|
||||||
|
indent='',
|
||||||
|
)
|
||||||
|
|
@ -84,6 +84,35 @@ def _wait_for_daemon_ready(
|
||||||
timeout=poll_interval,
|
timeout=poll_interval,
|
||||||
):
|
):
|
||||||
return
|
return
|
||||||
|
|
||||||
|
elif tpt_proto == 'tipc':
|
||||||
|
# TIPC — `reg_addr` is the proto-keyed
|
||||||
|
# `('tipc', stype, instance, scope)` per
|
||||||
|
# `tractor.ipc._tipc.TIPCAddress.unwrap()`.
|
||||||
|
#
|
||||||
|
# NOTE, connecting *by name* IS the readiness
|
||||||
|
# probe: until the daemon `.bind()`s (i.e.
|
||||||
|
# publishes) the name, the kernel answers
|
||||||
|
# `EHOSTUNREACH` immediately — no timeout wait.
|
||||||
|
from tractor.ipc._tipc import (
|
||||||
|
AF_TIPC,
|
||||||
|
TIPC_ADDR_NAME,
|
||||||
|
)
|
||||||
|
_, stype, instance, scope = reg_addr
|
||||||
|
sock = socket.socket(AF_TIPC, socket.SOCK_STREAM)
|
||||||
|
try:
|
||||||
|
sock.settimeout(poll_interval)
|
||||||
|
sock.connect((
|
||||||
|
TIPC_ADDR_NAME,
|
||||||
|
stype,
|
||||||
|
instance,
|
||||||
|
0, # domain: 0 == "anywhere in scope"
|
||||||
|
scope,
|
||||||
|
))
|
||||||
|
return
|
||||||
|
finally:
|
||||||
|
sock.close()
|
||||||
|
|
||||||
else:
|
else:
|
||||||
# UDS — `reg_addr` is a `(filedir, sockname)`
|
# UDS — `reg_addr` is a `(filedir, sockname)`
|
||||||
# tuple per `tractor.ipc._uds.UDSAddress.unwrap`.
|
# tuple per `tractor.ipc._uds.UDSAddress.unwrap`.
|
||||||
|
|
|
||||||
|
|
@ -19,7 +19,10 @@ from tractor.discovery._multiaddr import (
|
||||||
_tpt_proto_to_maddr,
|
_tpt_proto_to_maddr,
|
||||||
_maddr_to_tpt_proto,
|
_maddr_to_tpt_proto,
|
||||||
)
|
)
|
||||||
from tractor.discovery._addr import wrap_address
|
from tractor.discovery._addr import (
|
||||||
|
wrap_address,
|
||||||
|
_address_types,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_tpt_proto_to_maddr_mapping():
|
def test_tpt_proto_to_maddr_mapping():
|
||||||
|
|
@ -30,7 +33,12 @@ def test_tpt_proto_to_maddr_mapping():
|
||||||
'''
|
'''
|
||||||
assert _tpt_proto_to_maddr['tcp'] == 'tcp'
|
assert _tpt_proto_to_maddr['tcp'] == 'tcp'
|
||||||
assert _tpt_proto_to_maddr['uds'] == 'unix'
|
assert _tpt_proto_to_maddr['uds'] == 'unix'
|
||||||
assert len(_tpt_proto_to_maddr) == 2
|
assert _tpt_proto_to_maddr['tipc'] == 'tipc'
|
||||||
|
|
||||||
|
# NOTE, drive the expected set off the registration table
|
||||||
|
# (per the "drive-the-set-from-the-`Literal`" pattern) so
|
||||||
|
# adding a backend can't fail this for the wrong reason.
|
||||||
|
assert set(_tpt_proto_to_maddr) == set(_address_types)
|
||||||
|
|
||||||
|
|
||||||
def test_mk_maddr_tcp_ipv4():
|
def test_mk_maddr_tcp_ipv4():
|
||||||
|
|
@ -153,9 +161,12 @@ def test_maddr_to_tpt_proto_mapping():
|
||||||
|
|
||||||
'''
|
'''
|
||||||
assert _maddr_to_tpt_proto == {
|
assert _maddr_to_tpt_proto == {
|
||||||
'tcp': 'tcp',
|
maddr_proto: proto_key
|
||||||
'unix': 'uds',
|
for proto_key, maddr_proto in _tpt_proto_to_maddr.items()
|
||||||
}
|
}
|
||||||
|
assert _maddr_to_tpt_proto['tcp'] == 'tcp'
|
||||||
|
assert _maddr_to_tpt_proto['unix'] == 'uds'
|
||||||
|
assert _maddr_to_tpt_proto['tipc'] == 'tipc'
|
||||||
|
|
||||||
|
|
||||||
def test_parse_maddr_tcp_ipv4():
|
def test_parse_maddr_tcp_ipv4():
|
||||||
|
|
|
||||||
|
|
@ -14,6 +14,7 @@ from tractor import (
|
||||||
from tractor._testing.addr import (
|
from tractor._testing.addr import (
|
||||||
get_rando_addr,
|
get_rando_addr,
|
||||||
)
|
)
|
||||||
|
from tractor.ipc._tcp import TCPAddress
|
||||||
# TODO, use/check-roundtripping with some of these wrapper types?
|
# TODO, use/check-roundtripping with some of these wrapper types?
|
||||||
#
|
#
|
||||||
# from .._addr import Address
|
# from .._addr import Address
|
||||||
|
|
@ -70,3 +71,71 @@ def test_basic_ipc_server(
|
||||||
pdb=debug_mode,
|
pdb=debug_mode,
|
||||||
):
|
):
|
||||||
trio.run(main)
|
trio.run(main)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
'_tpt_proto',
|
||||||
|
['uds', 'tcp']
|
||||||
|
)
|
||||||
|
def test_ep_addr_reconciled_from_sockname(
|
||||||
|
_tpt_proto: str,
|
||||||
|
debug_mode: bool,
|
||||||
|
):
|
||||||
|
'''
|
||||||
|
Guard `Endpoint.start_listener()`'s post-bind reconciliation of
|
||||||
|
`.addr` against the listener's `socket.getsockname()`.
|
||||||
|
|
||||||
|
For `tcp` that reconciliation is the ONLY way a kernel-assigned
|
||||||
|
port (from a `port=0` bind) is ever learned, so it must keep
|
||||||
|
firing; for `uds` the sock-file path must survive the
|
||||||
|
round-trip through `.from_addr()` unchanged.
|
||||||
|
|
||||||
|
Both are pinned here *before* the reconciliation gets gated on
|
||||||
|
an `Address.rebind_from_sockname` opt-out (for backends whose
|
||||||
|
`getsockname()` reports something other than what was bound).
|
||||||
|
|
||||||
|
'''
|
||||||
|
async def main():
|
||||||
|
async with ipc._server.open_ipc_server() as server:
|
||||||
|
|
||||||
|
accept_addr: tuple[str, int|str]
|
||||||
|
match _tpt_proto:
|
||||||
|
# XXX the whole point: ask the kernel to pick.
|
||||||
|
case 'tcp':
|
||||||
|
accept_addr = (
|
||||||
|
TCPAddress.def_bindspace,
|
||||||
|
0,
|
||||||
|
)
|
||||||
|
case 'uds':
|
||||||
|
accept_addr = get_rando_addr(
|
||||||
|
tpt_proto=_tpt_proto,
|
||||||
|
)
|
||||||
|
|
||||||
|
eps: list[ipc._server.Endpoint] = await server.listen_on(
|
||||||
|
accept_addrs=[accept_addr],
|
||||||
|
stream_handler_nursery=None,
|
||||||
|
)
|
||||||
|
assert len(eps) == 1
|
||||||
|
ep: ipc._server.Endpoint = eps[0]
|
||||||
|
sockname = ep._listener.socket.getsockname()
|
||||||
|
|
||||||
|
match _tpt_proto:
|
||||||
|
case 'tcp':
|
||||||
|
# the bind req was for "any port"..
|
||||||
|
assert accept_addr[1] == 0
|
||||||
|
# ..and the ep learned the real one.
|
||||||
|
assert ep.addr._port != 0
|
||||||
|
assert ep.addr.unwrap() == tuple(sockname[:2])
|
||||||
|
|
||||||
|
case 'uds':
|
||||||
|
# sock-file path is stable across the
|
||||||
|
# `.from_addr()` round-trip.
|
||||||
|
assert ep.addr.unwrap() == accept_addr
|
||||||
|
assert str(ep.addr.sockpath) == sockname
|
||||||
|
|
||||||
|
server._parent_tn.cancel_scope.cancel()
|
||||||
|
|
||||||
|
with devx.maybe_open_crash_handler(
|
||||||
|
pdb=debug_mode,
|
||||||
|
):
|
||||||
|
trio.run(main)
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load Diff
|
|
@ -99,6 +99,20 @@ def get_rando_addr(
|
||||||
assert addr.sockpath.resolve()
|
assert addr.sockpath.resolve()
|
||||||
testrun_reg_addr = addr.unwrap()
|
testrun_reg_addr = addr.unwrap()
|
||||||
|
|
||||||
|
# NOTE, `.get_random()` already derives the service
|
||||||
|
# *instance* from a `uuid4`+pid-salted seed, so both the
|
||||||
|
# within- and cross-proc isolation the other 2 protos
|
||||||
|
# hand-roll above comes for free.
|
||||||
|
#
|
||||||
|
# XXX matters MORE here than for tcp/uds: a TIPC name
|
||||||
|
# clash doesn't raise `EADDRINUSE`, it silently
|
||||||
|
# round-robins connects between both publishers.
|
||||||
|
case 'tipc':
|
||||||
|
from tractor.ipc._tipc import TIPCAddress
|
||||||
|
addr: TIPCAddress = addr_type.get_random()
|
||||||
|
assert addr.is_valid
|
||||||
|
testrun_reg_addr = addr.unwrap()
|
||||||
|
|
||||||
# XXX, as sanity it should never the same as the default for the
|
# XXX, as sanity it should never the same as the default for the
|
||||||
# host-singleton registry actor.
|
# host-singleton registry actor.
|
||||||
assert def_reg_addr != testrun_reg_addr
|
assert def_reg_addr != testrun_reg_addr
|
||||||
|
|
|
||||||
|
|
@ -501,6 +501,12 @@ def pytest_configure(
|
||||||
'trio: legacy mark for tests meant to run under the `trio` '
|
'trio: legacy mark for tests meant to run under the `trio` '
|
||||||
'spawn backend (e.g. `test_local.py`).'
|
'spawn backend (e.g. `test_local.py`).'
|
||||||
)
|
)
|
||||||
|
config.addinivalue_line(
|
||||||
|
'markers',
|
||||||
|
'tipc: test targets the `AF_TIPC` tpt backend; the kernel- '
|
||||||
|
'touching cases self-skip unless the `tipc` module is loaded '
|
||||||
|
'(`sudo modprobe tipc`).'
|
||||||
|
)
|
||||||
|
|
||||||
# `--enable-stackscope`: install SIGUSR1 → trio task-tree
|
# `--enable-stackscope`: install SIGUSR1 → trio task-tree
|
||||||
# dump in pytest itself + propagate to every subactor via
|
# dump in pytest itself + propagate to every subactor via
|
||||||
|
|
@ -797,6 +803,27 @@ def tpt_protos(
|
||||||
addr_type = _addr._address_types[proto_key]
|
addr_type = _addr._address_types[proto_key]
|
||||||
assert addr_type.proto_key == proto_key
|
assert addr_type.proto_key == proto_key
|
||||||
|
|
||||||
|
# XXX, generic capability gate: an env-dependent tpt
|
||||||
|
# whose backing kernel-mod/lib/netns isn't present here
|
||||||
|
# must fail LOUDLY and EARLY rather than as a few hundred
|
||||||
|
# confusing connect-timeouts downstream.
|
||||||
|
#
|
||||||
|
# Any `Address` type MAY expose `.is_available()`
|
||||||
|
# returning `(ok, why_not)`; absence means "always
|
||||||
|
# available" (i.e. tcp/uds).
|
||||||
|
is_avail = getattr(
|
||||||
|
addr_type,
|
||||||
|
'is_available',
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
if is_avail:
|
||||||
|
avail, why_not = is_avail()
|
||||||
|
if not avail:
|
||||||
|
pytest.fail(
|
||||||
|
f'--tpt-proto={proto_key!r} is NOT usable here!\n'
|
||||||
|
f'{why_not}\n'
|
||||||
|
)
|
||||||
|
|
||||||
yield proto_keys
|
yield proto_keys
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -215,7 +215,6 @@ def pformat_caller_frame(
|
||||||
tb_str: str = pformat_boxed_tb(
|
tb_str: str = pformat_boxed_tb(
|
||||||
tb_str=tb_str,
|
tb_str=tb_str,
|
||||||
field_prefix=' ',
|
field_prefix=' ',
|
||||||
indent='',
|
|
||||||
)
|
)
|
||||||
return tb_str
|
return tb_str
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,7 @@ from ..runtime._state import (
|
||||||
)
|
)
|
||||||
from ..ipc._tcp import TCPAddress
|
from ..ipc._tcp import TCPAddress
|
||||||
from ..ipc._uds import UDSAddress
|
from ..ipc._uds import UDSAddress
|
||||||
|
from ..ipc._tipc import TIPCAddress
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from ..runtime._runtime import Actor
|
from ..runtime._runtime import Actor
|
||||||
|
|
@ -65,9 +66,22 @@ log = get_logger()
|
||||||
#
|
#
|
||||||
UnwrappedAddress = (
|
UnwrappedAddress = (
|
||||||
# tcp/udp/uds
|
# tcp/udp/uds
|
||||||
|
# ('127.0.0.1', 1616)
|
||||||
|
# ('/run/user/1000/tractor', 'registry@1616.sock')
|
||||||
|
#
|
||||||
|
# ..and the explicitly proto-keyed (`multiaddr`-spelled)
|
||||||
|
# form, which is where ALL backends should eventually land
|
||||||
|
# per the note below,
|
||||||
|
# ('tipc', 1953628160, 1616, 2)
|
||||||
|
#
|
||||||
|
# XXX VARIADIC bc `msgspec` refuses a union of >1 array-like
|
||||||
|
# type, so the two shapes can't be spelled as a union. Keep
|
||||||
|
# in sync with `.msg.types.UnwrappedAddress` which
|
||||||
|
# re-declares this to dodge a circular import AND is what
|
||||||
|
# actually validates the `SpawnSpec` wire msg!
|
||||||
tuple[
|
tuple[
|
||||||
str, # host/domain(tcp), filesys-dir(uds)
|
str|int,
|
||||||
int|str, # port/path(uds)
|
...,
|
||||||
]
|
]
|
||||||
# ?TODO? should we also include another 2 fields from
|
# ?TODO? should we also include another 2 fields from
|
||||||
# our `Aid` msg such that we include the runtime `Actor.uid`
|
# our `Aid` msg such that we include the runtime `Actor.uid`
|
||||||
|
|
@ -83,6 +97,17 @@ class Address(Protocol):
|
||||||
proto_key: ClassVar[str]
|
proto_key: ClassVar[str]
|
||||||
unwrapped_type: ClassVar[UnwrappedAddress]
|
unwrapped_type: ClassVar[UnwrappedAddress]
|
||||||
|
|
||||||
|
# whether `.ipc._server.Endpoint.start_listener()` should
|
||||||
|
# reconcile a bound `.addr` against its listener's
|
||||||
|
# `socket.getsockname()`.
|
||||||
|
#
|
||||||
|
# XXX NOTE, that reconciliation exists ONLY to learn the
|
||||||
|
# kernel-*assigned* port from a `port=0` tcp bind; a backend
|
||||||
|
# whose `getsockname()` reports a categorically different thing
|
||||||
|
# than what was `.bind()`ed must opt out with `False`, else the
|
||||||
|
# ep's addr gets clobbered by an un-dialable one.
|
||||||
|
rebind_from_sockname: ClassVar[bool]
|
||||||
|
|
||||||
# TODO, i feel like an `.is_bound()` is a better thing to
|
# TODO, i feel like an `.is_bound()` is a better thing to
|
||||||
# support?
|
# support?
|
||||||
# Lke, what use does this have besides a noop and if it's not
|
# Lke, what use does this have besides a noop and if it's not
|
||||||
|
|
@ -172,7 +197,8 @@ class Address(Protocol):
|
||||||
|
|
||||||
_address_types: bidict[str, Type[Address]] = {
|
_address_types: bidict[str, Type[Address]] = {
|
||||||
'tcp': TCPAddress,
|
'tcp': TCPAddress,
|
||||||
'uds': UDSAddress
|
'uds': UDSAddress,
|
||||||
|
'tipc': TIPCAddress,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -184,6 +210,9 @@ _default_lo_addrs: dict[
|
||||||
] = {
|
] = {
|
||||||
'tcp': TCPAddress.get_root().unwrap(),
|
'tcp': TCPAddress.get_root().unwrap(),
|
||||||
'uds': UDSAddress.get_root().unwrap(),
|
'uds': UDSAddress.get_root().unwrap(),
|
||||||
|
# NOTE, pure/cheap: a service-name pair, no kernel module
|
||||||
|
# nor I/O required at import time.
|
||||||
|
'tipc': TIPCAddress.get_root().unwrap(),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -229,6 +258,20 @@ def wrap_address(
|
||||||
# import pdbp; pdbp.set_trace()
|
# import pdbp; pdbp.set_trace()
|
||||||
match addr:
|
match addr:
|
||||||
|
|
||||||
|
# XXX, the explicitly proto-keyed form (spelled with the
|
||||||
|
# `multiaddr` proto name) which is where ALL backends
|
||||||
|
# should eventually land per the `UnwrappedAddress`
|
||||||
|
# migration note above.
|
||||||
|
#
|
||||||
|
# NOTE, a bare seq-pattern matches `list` too, which is
|
||||||
|
# what `msgpack` decodes our tuples back to.
|
||||||
|
case (
|
||||||
|
('tipc', int(), int())
|
||||||
|
|
|
||||||
|
('tipc', int(), int(), int())
|
||||||
|
):
|
||||||
|
cls = TIPCAddress
|
||||||
|
|
||||||
# classic network socket-address as tuple/list
|
# classic network socket-address as tuple/list
|
||||||
case (
|
case (
|
||||||
(str(), int())
|
(str(), int())
|
||||||
|
|
|
||||||
|
|
@ -38,8 +38,23 @@ if TYPE_CHECKING:
|
||||||
_tpt_proto_to_maddr: dict[str, str] = {
|
_tpt_proto_to_maddr: dict[str, str] = {
|
||||||
'tcp': 'tcp',
|
'tcp': 'tcp',
|
||||||
'uds': 'unix',
|
'uds': 'unix',
|
||||||
|
'tipc': 'tipc',
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# XXX, there is NO `/tipc` in the multiaddr protocol table yet
|
||||||
|
# (upstream track: gh #483 + multiformats/py-multiaddr#107), and
|
||||||
|
# `Multiaddr()` rejects an unregistered proto name outright.
|
||||||
|
#
|
||||||
|
# So until that lands `tipc` maddrs stay **`str`**-only — which
|
||||||
|
# `MsgTransport.maddr`s `Multiaddr|str` return type already
|
||||||
|
# allows and `MsgpackUDSStream.maddr` already exercises — and
|
||||||
|
# `parse_maddr()` special-cases the prefix BEFORE handing
|
||||||
|
# anything to `Multiaddr()`.
|
||||||
|
#
|
||||||
|
# This is also why gh #443's "always return `Multiaddr`" item
|
||||||
|
# stays blocked.
|
||||||
|
_tipc_maddr_prefix: str = '/tipc/'
|
||||||
|
|
||||||
# reverse mapping: multiaddr protocol name -> tractor proto_key
|
# reverse mapping: multiaddr protocol name -> tractor proto_key
|
||||||
_maddr_to_tpt_proto: dict[str, str] = {
|
_maddr_to_tpt_proto: dict[str, str] = {
|
||||||
v: k for k, v in _tpt_proto_to_maddr.items()
|
v: k for k, v in _tpt_proto_to_maddr.items()
|
||||||
|
|
@ -49,12 +64,15 @@ _maddr_to_tpt_proto: dict[str, str] = {
|
||||||
|
|
||||||
def mk_maddr(
|
def mk_maddr(
|
||||||
addr: 'Address',
|
addr: 'Address',
|
||||||
) -> Multiaddr:
|
) -> Multiaddr|str:
|
||||||
'''
|
'''
|
||||||
Construct a `Multiaddr` from a tractor `Address` instance,
|
Construct a `Multiaddr` from a tractor `Address` instance,
|
||||||
dispatching on the `.proto_key` to build the correct
|
dispatching on the `.proto_key` to build the correct
|
||||||
multiaddr-spec-compliant protocol path.
|
multiaddr-spec-compliant protocol path.
|
||||||
|
|
||||||
|
Return a `Multiaddr` for registered protocols. TIPC remains
|
||||||
|
an interim `str` until its upstream multiaddr protocol lands.
|
||||||
|
|
||||||
'''
|
'''
|
||||||
proto_key: str = addr.proto_key
|
proto_key: str = addr.proto_key
|
||||||
maddr_proto: str|None = _tpt_proto_to_maddr.get(proto_key)
|
maddr_proto: str|None = _tpt_proto_to_maddr.get(proto_key)
|
||||||
|
|
@ -75,6 +93,18 @@ def mk_maddr(
|
||||||
f'/{net_proto}/{host}/{maddr_proto}/{port}'
|
f'/{net_proto}/{host}/{maddr_proto}/{port}'
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# NOTE, interim `str`-only grammar (see the
|
||||||
|
# `_tipc_maddr_prefix` note above),
|
||||||
|
#
|
||||||
|
# /tipc/<stype>/<instance>/<scope>
|
||||||
|
#
|
||||||
|
# mirroring how `uds` maps onto the spec-legal `/unix`.
|
||||||
|
case 'tipc':
|
||||||
|
_, stype, instance, scope = addr.unwrap()
|
||||||
|
return (
|
||||||
|
f'/{maddr_proto}/{stype}/{instance}/{scope}'
|
||||||
|
)
|
||||||
|
|
||||||
case 'uds':
|
case 'uds':
|
||||||
filedir, filename = addr.unwrap()
|
filedir, filename = addr.unwrap()
|
||||||
filepath = Path(filedir) / filename
|
filepath = Path(filedir) / filename
|
||||||
|
|
@ -100,6 +130,23 @@ def parse_maddr(
|
||||||
# lazy imports to avoid circular deps
|
# lazy imports to avoid circular deps
|
||||||
from tractor.ipc._tcp import TCPAddress
|
from tractor.ipc._tcp import TCPAddress
|
||||||
from tractor.ipc._uds import UDSAddress
|
from tractor.ipc._uds import UDSAddress
|
||||||
|
from tractor.ipc._tipc import TIPCAddress
|
||||||
|
|
||||||
|
# XXX MUST come before `Multiaddr()` which rejects the
|
||||||
|
# not-yet-registered `/tipc` proto name outright.
|
||||||
|
if maddr_str.startswith(_tipc_maddr_prefix):
|
||||||
|
try:
|
||||||
|
_, _, stype, instance, scope = maddr_str.split('/')
|
||||||
|
return TIPCAddress.from_addr((
|
||||||
|
'tipc',
|
||||||
|
int(stype),
|
||||||
|
int(instance),
|
||||||
|
int(scope),
|
||||||
|
))
|
||||||
|
except (TypeError, ValueError) as src_err:
|
||||||
|
raise ValueError(
|
||||||
|
f'Invalid TIPC multiaddr: {maddr_str!r}'
|
||||||
|
) from src_err
|
||||||
|
|
||||||
maddr = Multiaddr(maddr_str)
|
maddr = Multiaddr(maddr_str)
|
||||||
proto_names: list[str] = [
|
proto_names: list[str] = [
|
||||||
|
|
|
||||||
|
|
@ -661,7 +661,16 @@ class Endpoint(Struct):
|
||||||
|
|
||||||
# NOTE, for handling the resolved non-0 port for
|
# NOTE, for handling the resolved non-0 port for
|
||||||
# TCP/UDP network sockets.
|
# TCP/UDP network sockets.
|
||||||
|
#
|
||||||
|
# XXX, gated on the addr-type's opt-in since for some
|
||||||
|
# backends `getsockname()` does NOT answer "the addr you
|
||||||
|
# bound"; `tipc` reports a `TIPC_ADDR_ID` port-id instead
|
||||||
|
# of the published name-seq, so rebinding from it would
|
||||||
|
# replace a dialable service-name with an un-dialable
|
||||||
|
# (and un-reconstructable) port-id.
|
||||||
if (
|
if (
|
||||||
|
self.addr.rebind_from_sockname
|
||||||
|
and
|
||||||
(unwrapped := lstnr.socket.getsockname())
|
(unwrapped := lstnr.socket.getsockname())
|
||||||
!=
|
!=
|
||||||
self.addr.unwrap()
|
self.addr.unwrap()
|
||||||
|
|
|
||||||
|
|
@ -65,6 +65,10 @@ class TCPAddress(
|
||||||
unwrapped_type: ClassVar[type] = tuple[str, int]
|
unwrapped_type: ClassVar[type] = tuple[str, int]
|
||||||
def_bindspace: ClassVar[str] = '127.0.0.1'
|
def_bindspace: ClassVar[str] = '127.0.0.1'
|
||||||
|
|
||||||
|
# XXX, REQUIRED here since a `port=0` bind means the kernel
|
||||||
|
# picks and `getsockname()` is the only way we learn it.
|
||||||
|
rebind_from_sockname: ClassVar[bool] = True
|
||||||
|
|
||||||
# ?TODO, actually validate ipv4/6 with stdlib's `ipaddress`
|
# ?TODO, actually validate ipv4/6 with stdlib's `ipaddress`
|
||||||
@property
|
@property
|
||||||
def is_valid(self) -> bool:
|
def is_valid(self) -> bool:
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load Diff
|
|
@ -38,17 +38,23 @@ from tractor.ipc._uds import (
|
||||||
UDSAddress,
|
UDSAddress,
|
||||||
MsgpackUDSStream,
|
MsgpackUDSStream,
|
||||||
)
|
)
|
||||||
|
from tractor.ipc._tipc import (
|
||||||
|
AF_TIPC,
|
||||||
|
TIPCAddress,
|
||||||
|
MsgpackTIPCStream,
|
||||||
|
)
|
||||||
|
|
||||||
# if TYPE_CHECKING:
|
# if TYPE_CHECKING:
|
||||||
# from tractor._addr import Address
|
# from tractor._addr import Address
|
||||||
|
|
||||||
|
|
||||||
Address = TCPAddress|UDSAddress
|
Address = TCPAddress|UDSAddress|TIPCAddress
|
||||||
|
|
||||||
# manually updated list of all supported msg transport types
|
# manually updated list of all supported msg transport types
|
||||||
_msg_transports = [
|
_msg_transports = [
|
||||||
MsgpackTCPStream,
|
MsgpackTCPStream,
|
||||||
MsgpackUDSStream
|
MsgpackUDSStream,
|
||||||
|
MsgpackTIPCStream,
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -59,15 +65,17 @@ _key_to_transport: dict[
|
||||||
] = {
|
] = {
|
||||||
('msgpack', 'tcp'): MsgpackTCPStream,
|
('msgpack', 'tcp'): MsgpackTCPStream,
|
||||||
('msgpack', 'uds'): MsgpackUDSStream,
|
('msgpack', 'uds'): MsgpackUDSStream,
|
||||||
|
('msgpack', 'tipc'): MsgpackTIPCStream,
|
||||||
}
|
}
|
||||||
|
|
||||||
# convert an Address wrapper to its corresponding transport type
|
# convert an Address wrapper to its corresponding transport type
|
||||||
_addr_to_transport: dict[
|
_addr_to_transport: dict[
|
||||||
Type[TCPAddress|UDSAddress],
|
Type[TCPAddress|UDSAddress|TIPCAddress],
|
||||||
Type[MsgTransport]
|
Type[MsgTransport]
|
||||||
] = {
|
] = {
|
||||||
TCPAddress: MsgpackTCPStream,
|
TCPAddress: MsgpackTCPStream,
|
||||||
UDSAddress: MsgpackUDSStream,
|
UDSAddress: MsgpackUDSStream,
|
||||||
|
TIPCAddress: MsgpackTIPCStream,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -108,6 +116,12 @@ def transport_from_stream(
|
||||||
case socket.AF_UNIX:
|
case socket.AF_UNIX:
|
||||||
transport = 'uds'
|
transport = 'uds'
|
||||||
|
|
||||||
|
# NOTE, `AF_TIPC` is linux-only in CPython so we
|
||||||
|
# match the `._tipc` constant (which carries a uapi
|
||||||
|
# fallback) rather than `socket.AF_TIPC`.
|
||||||
|
case _ if sock.family == AF_TIPC:
|
||||||
|
transport = 'tipc'
|
||||||
|
|
||||||
case _:
|
case _:
|
||||||
raise NotImplementedError(
|
raise NotImplementedError(
|
||||||
f'Unsupported socket family: {sock.family}'
|
f'Unsupported socket family: {sock.family}'
|
||||||
|
|
|
||||||
|
|
@ -114,9 +114,15 @@ class UDSAddress(
|
||||||
# -[ ] need to check what other mult-transport frameworks do
|
# -[ ] need to check what other mult-transport frameworks do
|
||||||
# like zmq, nng, uri-spec et al!
|
# like zmq, nng, uri-spec et al!
|
||||||
proto_key: ClassVar[str] = 'uds'
|
proto_key: ClassVar[str] = 'uds'
|
||||||
unwrapped_type: ClassVar[type] = tuple[str, int]
|
unwrapped_type: ClassVar[type] = tuple[str, str]
|
||||||
def_bindspace: ClassVar[Path] = get_rt_dir()
|
def_bindspace: ClassVar[Path] = get_rt_dir()
|
||||||
|
|
||||||
|
# NOTE, `getsockname()` answers the sock-file path as a `str`
|
||||||
|
# which never `==` our 2-tuple `.unwrap()`, so the round-trip
|
||||||
|
# always fires; it's a no-op modulo `.maybe_pid` and is kept
|
||||||
|
# `True` to preserve pre-existing behaviour exactly.
|
||||||
|
rebind_from_sockname: ClassVar[bool] = True
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def bindspace(self) -> Path:
|
def bindspace(self) -> Path:
|
||||||
'''
|
'''
|
||||||
|
|
@ -170,7 +176,7 @@ class UDSAddress(
|
||||||
f'{addr!r}\n'
|
f'{addr!r}\n'
|
||||||
)
|
)
|
||||||
|
|
||||||
def unwrap(self) -> tuple[str, int]:
|
def unwrap(self) -> tuple[str, str]:
|
||||||
# XXX NOTE, since this gets passed DIRECTLY to
|
# XXX NOTE, since this gets passed DIRECTLY to
|
||||||
# `.ipc._uds.open_unix_socket_w_passcred()`
|
# `.ipc._uds.open_unix_socket_w_passcred()`
|
||||||
return (
|
return (
|
||||||
|
|
|
||||||
|
|
@ -188,6 +188,31 @@ class Aid(
|
||||||
__repr__ = pretty_struct.Struct.__repr__
|
__repr__ = pretty_struct.Struct.__repr__
|
||||||
|
|
||||||
|
|
||||||
|
# NOTE, mirrors `.discovery._addr.UnwrappedAddress` but is
|
||||||
|
# re-declared here to dodge the circular import
|
||||||
|
# (`._addr` -> `.ipc._tcp` -> `.msg`).
|
||||||
|
#
|
||||||
|
# XXX this is the **wire** shape, so widening it is a wire-format
|
||||||
|
# change; keep the two decls in sync.
|
||||||
|
# XXX VARIADIC on purpose! `msgspec` rejects a union holding
|
||||||
|
# more than one array-like type ("Type unions may not contain
|
||||||
|
# more than one array-like (list, set, tuple) type"), so the two
|
||||||
|
# concrete shapes,
|
||||||
|
#
|
||||||
|
# ('127.0.0.1', 1616) # tcp
|
||||||
|
# ('/run/user/1000/tractor', 'x.sock') # uds
|
||||||
|
# ('tipc', 1953628160, 1616, 2) # proto-keyed (tipc)
|
||||||
|
#
|
||||||
|
# can't be spelled as `tuple[str, str|int]|tuple[str, int, int,
|
||||||
|
# int]`. Widen to one homogeneous variadic tuple instead.
|
||||||
|
#
|
||||||
|
# ?TODO, the real fix is the `UnwrappedAddress` proto-key
|
||||||
|
# migration (see `.discovery._addr`) after which this becomes a
|
||||||
|
# tagged union keyed off elem 0 and full per-proto validation
|
||||||
|
# comes back.
|
||||||
|
UnwrappedAddress = tuple[str|int, ...]
|
||||||
|
|
||||||
|
|
||||||
class SpawnSpec(
|
class SpawnSpec(
|
||||||
pretty_struct.Struct,
|
pretty_struct.Struct,
|
||||||
tag=True,
|
tag=True,
|
||||||
|
|
@ -213,8 +238,8 @@ class SpawnSpec(
|
||||||
|
|
||||||
# TODO: not just sockaddr pairs?
|
# TODO: not just sockaddr pairs?
|
||||||
# -[ ] abstract into a `TransportAddr` type?
|
# -[ ] abstract into a `TransportAddr` type?
|
||||||
reg_addrs: list[tuple[str, str|int]]
|
reg_addrs: list[UnwrappedAddress]
|
||||||
bind_addrs: list[tuple[str, str|int]]|None
|
bind_addrs: list[UnwrappedAddress]|None
|
||||||
|
|
||||||
|
|
||||||
# TODO: caps based RPC support in the payload?
|
# TODO: caps based RPC support in the payload?
|
||||||
|
|
|
||||||
|
|
@ -47,6 +47,7 @@ if TYPE_CHECKING:
|
||||||
TransportProtocolKey = Literal[
|
TransportProtocolKey = Literal[
|
||||||
'tcp',
|
'tcp',
|
||||||
'uds',
|
'uds',
|
||||||
|
'tipc',
|
||||||
]
|
]
|
||||||
_def_tpt_proto: TransportProtocolKey = 'tcp'
|
_def_tpt_proto: TransportProtocolKey = 'tcp'
|
||||||
|
|
||||||
|
|
|
||||||
8
uv.lock
8
uv.lock
|
|
@ -518,7 +518,7 @@ wheels = [
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "multiaddr"
|
name = "multiaddr"
|
||||||
version = "0.2.0"
|
version = "0.2.0"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { git = "https://github.com/multiformats/py-multiaddr.git?rev=f86519daaa21699023d0037c58cdff600313dd09#f86519daaa21699023d0037c58cdff600313dd09" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "base58" },
|
{ name = "base58" },
|
||||||
{ name = "dnspython" },
|
{ name = "dnspython" },
|
||||||
|
|
@ -533,10 +533,6 @@ dependencies = [
|
||||||
{ name = "trio-typing" },
|
{ name = "trio-typing" },
|
||||||
{ name = "varint" },
|
{ name = "varint" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/c7/10/4e26a8577cfce1c0febc8d83087e1373e93c695c6e73ad010546fb67e229/multiaddr-0.2.0.tar.gz", hash = "sha256:acb6b25c332ec1b2f1f8fef8d03a8c63385d34a87d690df0f4bba43cdf6efe8d", size = 58356, upload-time = "2026-03-17T21:51:00.274Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/b5/13/56e503d01218d1ca27ea9fda862045a4b400cae5e756f47315f5aaba0eee/multiaddr-0.2.0-py3-none-any.whl", hash = "sha256:bcff7bf3d7de3d6da0b865b25423bcb411de1d20d70cc6abfacf75170d17866c", size = 40424, upload-time = "2026-03-17T21:50:58.833Z" },
|
|
||||||
]
|
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "mypy-extensions"
|
name = "mypy-extensions"
|
||||||
|
|
@ -1178,7 +1174,7 @@ requires-dist = [
|
||||||
{ name = "bidict", specifier = ">=0.23.1" },
|
{ name = "bidict", specifier = ">=0.23.1" },
|
||||||
{ name = "colorlog", specifier = ">=6.8.2,<7" },
|
{ name = "colorlog", specifier = ">=6.8.2,<7" },
|
||||||
{ name = "msgspec", specifier = ">=0.20.0" },
|
{ name = "msgspec", specifier = ">=0.20.0" },
|
||||||
{ name = "multiaddr", specifier = ">=0.2.0" },
|
{ name = "multiaddr", git = "https://github.com/multiformats/py-multiaddr.git?rev=f86519daaa21699023d0037c58cdff600313dd09" },
|
||||||
{ name = "pdbp", specifier = ">=1.8.2,<2" },
|
{ name = "pdbp", specifier = ">=1.8.2,<2" },
|
||||||
{ name = "platformdirs", specifier = ">=4.4.0" },
|
{ name = "platformdirs", specifier = ">=4.4.0" },
|
||||||
{ name = "setproctitle", specifier = ">=1.3,<2" },
|
{ name = "setproctitle", specifier = ">=1.3,<2" },
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue