Compare commits
3 Commits
b1f6ade840
...
d80dcbe60f
| Author | SHA1 | Date |
|---|---|---|
|
|
d80dcbe60f | |
|
|
793ba9f94c | |
|
|
3e5a699df6 |
|
|
@ -0,0 +1,47 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
service: opencode
|
||||
session: tractor-net-api-extraction
|
||||
timestamp: 2026-08-30T02:52:01Z
|
||||
git_ref: b1f6ade8
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260830T025201Z_b1f6ade8_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
After rebasing the WireGuard Layer C stack, the human questioned the
|
||||
placement of bindspace APIs under `tractor.discovery`, considered new
|
||||
public package names, selected `tractor.net`, and authorized the
|
||||
implementation.
|
||||
|
||||
## Response summary
|
||||
|
||||
Extracted bindspace and tunnel implementations into a lazy
|
||||
`tractor.net` package. Moved public multiaddr, bindspace, tunnel, and
|
||||
WireGuard symbols out of `tractor.discovery`, updated internal imports,
|
||||
tests, examples, and documentation, and added cold-import regressions
|
||||
that keep optional networking dependencies off the root import path.
|
||||
Current-netns attachment pins `/proc/thread-self/ns/net` so calls from
|
||||
non-leader threads retain the caller's namespace rather than the process
|
||||
leader's.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/net/` - lazy public network API and implementation modules.
|
||||
- `tractor/discovery/` - retain actor discovery and internal address parsing.
|
||||
- `tractor/__init__.py` - expose `tractor.net` lazily.
|
||||
- `tests/net/` - network declaration and lifecycle coverage.
|
||||
- `tests/test_lazy_imports.py` - enforce the lazy package boundary.
|
||||
- `examples/multihost/wg_lan/` - use the public network package.
|
||||
- `docs/` - document network APIs separately from actor discovery.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human rejected `tractor.discovery` as the long-term public home,
|
||||
considered tunnel- and namespace-specific alternatives, and selected
|
||||
the broader `tractor.net` boundary because bindspaces may include plain
|
||||
netns, WireGuard, VRF, veth, and later network resources. The agent
|
||||
applied the resulting source changes; no direct manual edits were
|
||||
observed.
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-30T02:52:01Z
|
||||
git_ref: b1f6ade8
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
# Raw output - extract public network APIs
|
||||
|
||||
The human questioned whether bindspace and WireGuard lifecycle APIs
|
||||
belonged under actor discovery, selected the proposed `tractor.net`
|
||||
boundary, and authorized implementation.
|
||||
|
||||
## Generated code
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tractor/net tractor/discovery tractor/__init__.py`
|
||||
|
||||
Move bindspace and tunnel implementations into a lazy public network
|
||||
package. Keep actor discovery focused on registry and lookup behavior,
|
||||
while exposing bindspace, tunnel, WireGuard, and multiaddr declarations
|
||||
through `tractor.net` without loading optional networking dependencies
|
||||
during `import tractor`.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/net tests/test_lazy_imports.py tests/ipc examples/multihost/wg_lan docs`
|
||||
|
||||
Move network-focused tests to `tests/net`, update internal and public
|
||||
imports, verify lazy symbol resolution and removed discovery exports,
|
||||
and document the new package boundary.
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
service: opencode
|
||||
session: root-bindspace-bootstrap
|
||||
timestamp: 2026-08-30T02:52:02Z
|
||||
git_ref: b1f6ade8
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260830T025202Z_b1f6ade8_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The human chose to defer multiprocessing bindspace FD transport because
|
||||
that backend may be removed, selected root/single-actor namespace
|
||||
bootstrap as the next stage, approved caller-thread restoration and
|
||||
public `tractor.net.open_wg_bindspace()` composition, and authorized
|
||||
implementation after the stack rebase.
|
||||
|
||||
## Response summary
|
||||
|
||||
Added `bindspace=` to `tractor.open_root_actor()`. Root startup now
|
||||
duplicates and validates the live namespace FD, enters before any
|
||||
debugger, registry, IPC, or actor-runtime work, and synchronously
|
||||
restores the caller thread's original namespace after complete root
|
||||
teardown. Added deterministic failure/cancellation coverage, a real UDS
|
||||
root actor E2E across two network namespaces, and an executable public
|
||||
WireGuard-bindspace/root composition regression. Bound roots reject the
|
||||
persistent `mp_forkserver` backend because a helper from an earlier
|
||||
runtime may retain a stale namespace; the public annotation also remains
|
||||
runtime-resolvable without importing `tractor.net` eagerly.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/_root.py` - public root bindspace lifecycle integration.
|
||||
- `tractor/spawn/_netns.py` - temporary thread-local netns enter/restore.
|
||||
- `tests/test_netns_spawn.py` - deterministic and real root regressions.
|
||||
- `tests/net/test_wg_iface_lifecycle.py` - public root composition test.
|
||||
- `docs/api/net.rst` - root ownership and restoration contract.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human explicitly deferred multiprocessing support, selected root
|
||||
bootstrap as the next implementation stage, approved the requirement to
|
||||
restore the caller's original namespace so owned teardown can complete,
|
||||
and approved composition through public `tractor.net` contexts rather
|
||||
than a new one-shot convenience API. The human later rewrote
|
||||
`_enter_netns_temporarily()` to separate restoration from FD cleanup and
|
||||
make the transition flow easier to inspect. The agent retained that
|
||||
intent with nested lexical FD ownership while fixing setup-error masking,
|
||||
conditional restoration, and duplicate-FD cleanup found during the
|
||||
requested audit.
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-30T02:52:02Z
|
||||
git_ref: b1f6ade8
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
# Raw output - bootstrap the root actor in a bindspace
|
||||
|
||||
The human deferred multiprocessing descriptor transport, selected root
|
||||
namespace bootstrap as the next Layer C stage, approved enter/restore
|
||||
semantics and public `tractor.net` composition, and authorized
|
||||
implementation.
|
||||
|
||||
## Generated code
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tractor/_root.py tractor/spawn/_netns.py`
|
||||
|
||||
Add an optional realized `Bindspace` to `open_root_actor()`. Duplicate
|
||||
its namespace FD, enter before debugger, registry, IPC, or runtime
|
||||
startup, and restore the caller thread's original namespace after root
|
||||
teardown. Preserve caller FD ownership and primary errors across
|
||||
restoration and close failures.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_netns_spawn.py tests/net/test_wg_iface_lifecycle.py docs/api/net.rst`
|
||||
|
||||
Exercise same-netns behavior, body errors, cancellation, restoration
|
||||
failures, missing and closed FDs, real root entry/restoration under an
|
||||
unprivileged user namespace, and public WireGuard-bindspace/root
|
||||
composition.
|
||||
|
|
@ -47,7 +47,7 @@ Here is a small example from piker,
|
|||
We should take whatever common API is needed to support this and
|
||||
distill it into a
|
||||
```python
|
||||
tractor.discovery.parse_endpoints(
|
||||
tractor.net.parse_endpoints(
|
||||
) -> dict[
|
||||
str,
|
||||
list[Address]
|
||||
|
|
|
|||
|
|
@ -209,7 +209,7 @@ Observed protocol-name lists, for writing the `match`:
|
|||
### 3.3 pure codecs + explicit verification
|
||||
|
||||
Port #482 §2's pure helpers into
|
||||
`tractor/discovery/_tunnel.py`, keeping the impure probe cleanly
|
||||
`tractor/net/_tunnel.py`, keeping the impure probe cleanly
|
||||
separated until layer B:
|
||||
|
||||
```python
|
||||
|
|
|
|||
|
|
@ -53,6 +53,7 @@ Most-used names at a glance:
|
|||
core
|
||||
context
|
||||
discovery
|
||||
net
|
||||
errors
|
||||
msg
|
||||
trionics
|
||||
|
|
|
|||
|
|
@ -84,6 +84,7 @@ already distributed-system aware.
|
|||
.. seealso::
|
||||
|
||||
:doc:`/explain/architecture` for the transport/server
|
||||
internals, :doc:`/api/discovery` for how channel addresses
|
||||
get registered and found, and :doc:`/api/msg` for the codec
|
||||
layer every channel speaks.
|
||||
internals, :doc:`/api/net` for network declarations,
|
||||
:doc:`/api/discovery` for how channel addresses get registered
|
||||
and found, and :doc:`/api/msg` for the codec layer every channel
|
||||
speaks.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,95 @@
|
|||
Network declarations and lifecycles
|
||||
===================================
|
||||
|
||||
``tractor.net`` provides address composition, bindspace declarations,
|
||||
and tunnel configuration. The package is lazy: importing
|
||||
``tractor`` or ``tractor.net`` does not load multiaddr, WireGuard, or
|
||||
pyroute2 implementation modules until a public symbol is used.
|
||||
|
||||
Multiaddr helpers
|
||||
-----------------
|
||||
|
||||
.. currentmodule:: tractor.net
|
||||
|
||||
.. autofunction:: mk_maddr
|
||||
|
||||
.. autofunction:: parse_maddr
|
||||
|
||||
.. autofunction:: parse_endpoints
|
||||
|
||||
Bindspaces
|
||||
----------
|
||||
|
||||
.. autoclass:: BindspaceSpec
|
||||
|
||||
.. autoclass:: BindspaceRef
|
||||
|
||||
.. autoclass:: Bindspace
|
||||
|
||||
.. autofunction:: attach_netns
|
||||
|
||||
.. autofunction:: open_netns
|
||||
|
||||
.. autofunction:: open_bindspace
|
||||
|
||||
Root actor composition
|
||||
----------------------
|
||||
|
||||
A live :class:`Bindspace` can scope the root actor itself. Compose the
|
||||
bindspace manager outside :func:`tractor.open_root_actor` so its network
|
||||
namespace remains pinned for the complete actor runtime::
|
||||
|
||||
async with tractor.net.open_wg_bindspace(
|
||||
bindspace_spec=bindspace_spec,
|
||||
layers=layers,
|
||||
role='listen',
|
||||
) as bindspace:
|
||||
async with tractor.open_root_actor(
|
||||
bindspace=bindspace,
|
||||
enable_transports=['uds'],
|
||||
) as root_actor:
|
||||
...
|
||||
|
||||
Root entry happens before registry probes, IPC listeners, runtime sockets,
|
||||
or actor startup. On every exit, including cancellation or a body error,
|
||||
the calling thread is restored to its original network namespace before
|
||||
``open_root_actor()`` returns. The root context duplicates the live
|
||||
``Bindspace.namespace_fd`` and never consumes or closes the descriptor
|
||||
owned by ``open_wg_bindspace()``. Default child processes inherit the root
|
||||
namespace naturally; passing an explicit alternate child ``bindspace``
|
||||
continues to use that spawn backend's existing behavior. Bound roots reject
|
||||
the persistent ``mp_forkserver`` backend because a server started by an
|
||||
earlier runtime may retain that runtime's network namespace.
|
||||
|
||||
This is the current low-level composition API. A future convenience API
|
||||
may accept a tunnel-bearing multiaddr, realize its WireGuard bindspace
|
||||
internally, and supply that live capability to root startup.
|
||||
|
||||
Tunnels and WireGuard
|
||||
---------------------
|
||||
|
||||
.. autoclass:: TunnelledAddress
|
||||
|
||||
.. autoclass:: WGTunnelSpec
|
||||
|
||||
.. autoclass:: WGInterfaceConfig
|
||||
|
||||
.. autoclass:: WGPeerConfig
|
||||
|
||||
.. autofunction:: parse_wg_maddr
|
||||
|
||||
.. autofunction:: mk_wg_maddr
|
||||
|
||||
.. autofunction:: strip_tunnels
|
||||
|
||||
.. autofunction:: tunnels_of
|
||||
|
||||
.. autofunction:: open_wg_iface
|
||||
|
||||
.. autofunction:: open_wg_bindspace
|
||||
|
||||
.. autofunction:: read_wg_pubkey
|
||||
|
||||
.. autofunction:: read_wg_peers
|
||||
|
||||
.. autofunction:: verify_wg_peer
|
||||
|
|
@ -264,7 +264,7 @@ terminology is retired: it's *registrar*/*registry* everywhere now
|
|||
substitute "registrar" and you're up to date.
|
||||
|
||||
.. note::
|
||||
Multihoming nerds: ``tractor.discovery`` also ships
|
||||
Multihoming nerds: ``tractor.net`` ships
|
||||
libp2p-style *multiaddr* helpers — ``mk_maddr()`` and
|
||||
``parse_maddr()`` — for describing transport endpoints as
|
||||
structured strings.
|
||||
|
|
|
|||
|
|
@ -116,7 +116,7 @@ ping -c1 10.0.11.1 # from B
|
|||
|
||||
```bash
|
||||
python -c "
|
||||
from tractor.discovery import mb_pubkey
|
||||
from tractor.net import mb_pubkey
|
||||
key = open('wg_pub.key').read().strip()
|
||||
print(mb_pubkey(key))
|
||||
"
|
||||
|
|
@ -164,9 +164,23 @@ Four corrections, all from
|
|||
(1:1 proto-key↔type) and `_addr_to_transport` wants a
|
||||
`MsgTransport` per addr-type, which `wg` doesn't have.
|
||||
|
||||
## next
|
||||
## root composition
|
||||
|
||||
Layer A's `TunnelledAddress` and native maddr parser plus Layer B's
|
||||
explicit pyroute2 verification now live in `tractor.discovery`. Next,
|
||||
add `open_bindspace()` `@acm`s which create/tear down the iface and
|
||||
netns.
|
||||
The `TunnelledAddress`, native maddr parser, bindspace lifecycle, and
|
||||
explicit pyroute2 verification APIs live in `tractor.net`. Keep the
|
||||
owning bindspace context outside the root actor so its namespace FD
|
||||
remains live through complete actor teardown:
|
||||
|
||||
```python
|
||||
async with tractor.net.open_wg_bindspace(
|
||||
bindspace_spec,
|
||||
layers,
|
||||
role='listen',
|
||||
) as bindspace:
|
||||
async with tractor.open_root_actor(bindspace=bindspace):
|
||||
...
|
||||
```
|
||||
|
||||
The root actor enters before registry or IPC setup and restores the
|
||||
calling thread's original namespace before the outer bindspace context
|
||||
removes owned WireGuard and netns resources.
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ from __future__ import annotations
|
|||
|
||||
import tractor
|
||||
import trio
|
||||
from tractor.discovery import (
|
||||
from tractor.net import (
|
||||
TunnelledAddress,
|
||||
mk_maddr,
|
||||
parse_wg_maddr,
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ from __future__ import annotations
|
|||
|
||||
import tractor
|
||||
import trio
|
||||
from tractor.discovery import (
|
||||
from tractor.net import (
|
||||
TunnelledAddress,
|
||||
parse_wg_maddr,
|
||||
verify_wg_peer,
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ from tractor.devx import dump_on_hang
|
|||
from tractor.trionics import collapse_eg
|
||||
from tractor._testing import tractor_test
|
||||
from tractor.discovery._addr import wrap_address
|
||||
from tractor.discovery._multiaddr import mk_maddr
|
||||
from tractor.net import mk_maddr
|
||||
import trio
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ from tractor.discovery import _api
|
|||
from tractor.discovery._addr import (
|
||||
wrap_address,
|
||||
)
|
||||
from tractor.discovery._multiaddr import mk_maddr
|
||||
from tractor.net import mk_maddr
|
||||
from tractor.ipc import _connect_chan
|
||||
from tractor._testing.addr import get_rando_addr
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ from __future__ import annotations
|
|||
import pytest
|
||||
import trio
|
||||
|
||||
from tractor.discovery import (
|
||||
from tractor.net import (
|
||||
TunnelledAddress,
|
||||
WGTunnelSpec,
|
||||
tunnels_of,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ from __future__ import annotations
|
|||
|
||||
import trio
|
||||
|
||||
from tractor.discovery import (
|
||||
from tractor.net import (
|
||||
BindspaceRef,
|
||||
TunnelledAddress,
|
||||
WGTunnelSpec,
|
||||
|
|
|
|||
|
|
@ -0,0 +1 @@
|
|||
'''Network declaration and lifecycle tests.'''
|
||||
|
|
@ -13,7 +13,7 @@ import msgspec
|
|||
import pytest
|
||||
import trio
|
||||
|
||||
from tractor.discovery import (
|
||||
from tractor.net import (
|
||||
Bindspace,
|
||||
BindspaceOwnership,
|
||||
BindspaceRef,
|
||||
|
|
@ -23,7 +23,7 @@ from tractor.discovery import (
|
|||
open_bindspace,
|
||||
open_netns,
|
||||
)
|
||||
from tractor.discovery import _bindspace
|
||||
from tractor.net import _bindspace
|
||||
from tractor.msg import ProcessLocal
|
||||
|
||||
|
||||
|
|
@ -261,10 +261,11 @@ def test_open_bindspace_attaches_current_netns() -> None:
|
|||
'''
|
||||
The unnamed spec must borrow and pin the caller's current netns.
|
||||
|
||||
Open `/proc/self/ns/net`, prove the yielded bindspace records its
|
||||
stable inode and borrowed ownership, then exit the context and
|
||||
prove the exact descriptor was closed without altering the
|
||||
namespace itself.
|
||||
Opening `/proc/self/ns/net` could pin the thread-group leader's
|
||||
namespace when this context runs from another thread. Prove the
|
||||
implementation selects `/proc/thread-self/ns/net`, records the
|
||||
calling thread's stable inode and borrowed ownership, then closes
|
||||
the exact descriptor without altering the namespace itself.
|
||||
|
||||
'''
|
||||
async def main() -> int:
|
||||
|
|
@ -276,6 +277,9 @@ def test_open_bindspace_attaches_current_netns() -> None:
|
|||
kind='netns',
|
||||
)
|
||||
assert spec.key is CURRENT_NETNS
|
||||
assert _bindspace._THREAD_NETNS == Path(
|
||||
'/proc/thread-self/ns/net'
|
||||
)
|
||||
async with open_bindspace(spec) as bindspace:
|
||||
namespace_fd: int|None = bindspace.namespace_fd
|
||||
assert namespace_fd is not None
|
||||
|
|
@ -1,7 +1,6 @@
|
|||
'''
|
||||
Multiaddr construction, parsing, and round-trip tests for
|
||||
`tractor.discovery._multiaddr.mk_maddr()` and
|
||||
`tractor.discovery._multiaddr.parse_maddr()`.
|
||||
`tractor.net.mk_maddr()` and `tractor.net.parse_maddr()`.
|
||||
|
||||
'''
|
||||
from pathlib import Path
|
||||
|
|
@ -10,20 +9,20 @@ from types import SimpleNamespace
|
|||
import pytest
|
||||
from multiaddr import Multiaddr
|
||||
|
||||
from tractor.discovery import (
|
||||
from tractor.net import (
|
||||
TunnelledAddress,
|
||||
WGTunnelSpec,
|
||||
mb_pubkey,
|
||||
mk_wg_maddr,
|
||||
mk_maddr,
|
||||
parse_endpoints,
|
||||
parse_maddr,
|
||||
parse_wg_maddr,
|
||||
tunnels_of,
|
||||
)
|
||||
from tractor.ipc._tcp import TCPAddress
|
||||
from tractor.ipc._uds import UDSAddress
|
||||
from tractor.discovery._multiaddr import (
|
||||
mk_maddr,
|
||||
parse_maddr,
|
||||
parse_endpoints,
|
||||
_tpt_proto_to_maddr,
|
||||
_maddr_to_tpt_proto,
|
||||
)
|
||||
|
|
@ -14,7 +14,7 @@ from __future__ import annotations
|
|||
import msgspec
|
||||
import pytest
|
||||
|
||||
from tractor.discovery import (
|
||||
from tractor.net import (
|
||||
BindspaceRef,
|
||||
TunnelledAddress,
|
||||
WGTunnelSpec,
|
||||
|
|
@ -7,7 +7,7 @@ from __future__ import annotations
|
|||
import msgspec
|
||||
import pytest
|
||||
|
||||
from tractor.discovery import (
|
||||
from tractor.net import (
|
||||
WGInterfaceConfig,
|
||||
WGPeerConfig,
|
||||
)
|
||||
|
|
@ -8,12 +8,14 @@ from collections.abc import AsyncIterator
|
|||
from contextlib import asynccontextmanager as acm
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
from typing import BinaryIO
|
||||
|
||||
import pytest
|
||||
import trio
|
||||
|
||||
from tractor.discovery import (
|
||||
import tractor
|
||||
from tractor.net import (
|
||||
Bindspace,
|
||||
BindspaceRef,
|
||||
BindspaceSpec,
|
||||
|
|
@ -23,7 +25,7 @@ from tractor.discovery import (
|
|||
open_wg_bindspace,
|
||||
open_wg_iface,
|
||||
)
|
||||
from tractor.discovery import _tunnel
|
||||
from tractor.net import _tunnel
|
||||
|
||||
|
||||
_LOCAL_KEY: str = 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
|
||||
|
|
@ -376,3 +378,99 @@ def test_open_wg_bindspace_nests_resource_lifetimes(
|
|||
'wg-outer-exit',
|
||||
'bindspace-exit',
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
sys.platform != 'linux',
|
||||
reason='network namespaces are Linux-only',
|
||||
)
|
||||
def test_public_wg_bindspace_scopes_root_actor(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
tpt_proto: str,
|
||||
) -> None:
|
||||
'''
|
||||
Public network contexts must fully enclose the root runtime.
|
||||
|
||||
Attach the real current netns through `tractor.net`, fake only WG
|
||||
interface provisioning, and open a real root actor with the yielded
|
||||
`Bindspace`. The trace and inode checks prove interface setup wraps
|
||||
actor startup, the runtime occupies the realized bindspace, and root
|
||||
restoration finishes before network-resource teardown.
|
||||
|
||||
'''
|
||||
events: list[str] = []
|
||||
bindspace_spec: BindspaceSpec = BindspaceSpec(
|
||||
kind='netns',
|
||||
lifecycle='attach',
|
||||
)
|
||||
tunnel_spec: WGTunnelSpec = WGTunnelSpec(
|
||||
peer_pubkey=_PEER_KEY,
|
||||
iface='wg-root',
|
||||
)
|
||||
config: WGInterfaceConfig = WGInterfaceConfig(
|
||||
private_key=_LOCAL_KEY,
|
||||
)
|
||||
|
||||
@acm
|
||||
async def fake_open_wg_iface(
|
||||
spec: WGTunnelSpec,
|
||||
iface_config: WGInterfaceConfig,
|
||||
bindspace: Bindspace,
|
||||
role: _tunnel.WGRole,
|
||||
) -> AsyncIterator[WGTunnelSpec]:
|
||||
'''
|
||||
Trace one WG layer around the real root actor lifetime.
|
||||
|
||||
'''
|
||||
assert spec is tunnel_spec
|
||||
assert iface_config is config
|
||||
assert role == 'listen'
|
||||
assert bindspace.namespace_fd is not None
|
||||
events.append('wg-enter')
|
||||
try:
|
||||
yield spec
|
||||
finally:
|
||||
events.append('wg-exit')
|
||||
|
||||
monkeypatch.setattr(
|
||||
_tunnel,
|
||||
'open_wg_iface',
|
||||
fake_open_wg_iface,
|
||||
)
|
||||
|
||||
async def main() -> None:
|
||||
'''
|
||||
Compose the public network and root actor context managers.
|
||||
|
||||
'''
|
||||
async with tractor.net.open_wg_bindspace(
|
||||
bindspace_spec=bindspace_spec,
|
||||
layers=((tunnel_spec, config),),
|
||||
role='listen',
|
||||
) as bindspace:
|
||||
events.append('bindspace-open')
|
||||
async with tractor.open_root_actor(
|
||||
bindspace=bindspace,
|
||||
enable_transports=[tpt_proto],
|
||||
):
|
||||
events.append('root-open')
|
||||
assert bindspace.namespace_fd is not None
|
||||
assert os.fstat(
|
||||
bindspace.namespace_fd,
|
||||
).st_ino == bindspace.ref.inode
|
||||
assert Path(
|
||||
'/proc/thread-self/ns/net'
|
||||
).stat().st_ino == bindspace.ref.inode
|
||||
events.append('root-closed')
|
||||
|
||||
events.append('bindspace-closed')
|
||||
|
||||
trio.run(main)
|
||||
assert events == [
|
||||
'wg-enter',
|
||||
'bindspace-open',
|
||||
'root-open',
|
||||
'root-closed',
|
||||
'wg-exit',
|
||||
'bindspace-closed',
|
||||
]
|
||||
|
|
@ -13,13 +13,13 @@ from typing import (
|
|||
import pytest
|
||||
import trio
|
||||
|
||||
from tractor.discovery import (
|
||||
from tractor.net import (
|
||||
read_wg_peers,
|
||||
read_wg_pubkey,
|
||||
verify_wg_peer,
|
||||
WGTunnelSpec,
|
||||
)
|
||||
from tractor.discovery import _tunnel
|
||||
from tractor.net import _tunnel
|
||||
|
||||
pyroute2: Any = pytest.importorskip('pyroute2')
|
||||
|
||||
|
|
@ -12,6 +12,7 @@ from typing import (
|
|||
get_type_hints,
|
||||
)
|
||||
|
||||
import tractor
|
||||
from tractor.discovery import (
|
||||
_addr,
|
||||
_multiaddr,
|
||||
|
|
@ -44,8 +45,8 @@ def test_lazy_to_asyncio_package_api():
|
|||
Before the lazy conversion, package import side effects exposed
|
||||
`to_asyncio` to `dir()` and wildcard imports. Exercise those APIs
|
||||
in cold interpreters so this test proves normal `import tractor`
|
||||
leaves `asyncio` unloaded, while discovery and wildcard access
|
||||
still advertise and resolve the public submodule.
|
||||
leaves `asyncio` unloaded, while introspection and wildcard
|
||||
access still advertise and resolve the public submodule.
|
||||
|
||||
'''
|
||||
cold = run_cold_import(
|
||||
|
|
@ -102,6 +103,12 @@ def test_cold_import_budget():
|
|||
'bidict',
|
||||
'colorlog',
|
||||
'multiaddr',
|
||||
'multibase',
|
||||
'pyroute2',
|
||||
'tractor.discovery._multiaddr',
|
||||
'tractor.net',
|
||||
'tractor.net._bindspace',
|
||||
'tractor.net._tunnel',
|
||||
'wrapt',
|
||||
)
|
||||
code = (
|
||||
|
|
@ -136,6 +143,123 @@ def test_cold_import_budget():
|
|||
)
|
||||
|
||||
|
||||
def test_lazy_net_package_api():
|
||||
'''
|
||||
Keep the public network package cold until symbol access.
|
||||
|
||||
The old discovery re-exports imported bindspace, tunnel,
|
||||
multiaddr and optional dependencies while initializing a package.
|
||||
Import `tractor.net` in a clean interpreter, inspect its public
|
||||
surface, and prove no implementation or optional dependency was
|
||||
loaded. Then resolve one symbol from each backing module and
|
||||
prove the facade caches each value while preserving boundaries.
|
||||
|
||||
'''
|
||||
modules: tuple[str, ...] = (
|
||||
'tractor.net._bindspace',
|
||||
'tractor.net._tunnel',
|
||||
'tractor.discovery._multiaddr',
|
||||
'multiaddr',
|
||||
'multibase',
|
||||
'pyroute2',
|
||||
)
|
||||
cold: dict[str, object] = run_cold_import(
|
||||
'import json, sys; import tractor.net as net; '
|
||||
f'names = {modules!r}; '
|
||||
'print(json.dumps({'
|
||||
'"public": all(name in dir(net) for name in net.__all__), '
|
||||
'"loaded": [name for name in names if name in sys.modules]'
|
||||
'}))'
|
||||
)
|
||||
assert cold == {
|
||||
'public': True,
|
||||
'loaded': [],
|
||||
}
|
||||
|
||||
resolved: dict[str, object] = run_cold_import(
|
||||
'import json, sys; import tractor.net as net; '
|
||||
'bindspace = net.BindspaceSpec; '
|
||||
'bindspace_cached = net.BindspaceSpec is bindspace; '
|
||||
'maddr = net.mk_maddr; '
|
||||
'maddr_cached = net.mk_maddr is maddr; '
|
||||
'tunnel = net.WGTunnelSpec; '
|
||||
'tunnel_cached = net.WGTunnelSpec is tunnel; '
|
||||
'print(json.dumps({'
|
||||
'"bindspace_cached": bindspace_cached, '
|
||||
'"maddr_cached": maddr_cached, '
|
||||
'"tunnel_cached": tunnel_cached, '
|
||||
'"bindspace_module": bindspace.__module__, '
|
||||
'"maddr_module": maddr.__module__, '
|
||||
'"tunnel_module": tunnel.__module__, '
|
||||
'"multiaddr_loaded": "multiaddr" in sys.modules, '
|
||||
'"pyroute2_loaded": "pyroute2" in sys.modules'
|
||||
'}))'
|
||||
)
|
||||
assert resolved == {
|
||||
'bindspace_cached': True,
|
||||
'maddr_cached': True,
|
||||
'tunnel_cached': True,
|
||||
'bindspace_module': 'tractor.net._bindspace',
|
||||
'maddr_module': 'tractor.discovery._multiaddr',
|
||||
'tunnel_module': 'tractor.net._tunnel',
|
||||
'multiaddr_loaded': False,
|
||||
'pyroute2_loaded': False,
|
||||
}
|
||||
|
||||
|
||||
def test_net_root_export_and_old_discovery_surface():
|
||||
'''
|
||||
Publish networking only from its approved namespace.
|
||||
|
||||
Before extraction, unshipped network names and implementation
|
||||
modules lived under `tractor.discovery`. Exercise root attribute
|
||||
and wildcard access in clean interpreters, proving `tractor.net`
|
||||
is discoverable and cached without loading implementations. Also
|
||||
prove the old exports are absent and their modules no longer
|
||||
resolve, preventing accidental compatibility aliases.
|
||||
|
||||
'''
|
||||
root: dict[str, object] = run_cold_import(
|
||||
'import json, sys, tractor; '
|
||||
'advertised = "net" in dir(tractor); '
|
||||
'net = tractor.net; '
|
||||
'print(json.dumps({'
|
||||
'"advertised": advertised, '
|
||||
'"cached": tractor.net is net, '
|
||||
'"module": net.__name__, '
|
||||
'"bindspace_loaded": '
|
||||
'"tractor.net._bindspace" in sys.modules, '
|
||||
'"tunnel_loaded": "tractor.net._tunnel" in sys.modules'
|
||||
'}))'
|
||||
)
|
||||
assert root == {
|
||||
'advertised': True,
|
||||
'cached': True,
|
||||
'module': 'tractor.net',
|
||||
'bindspace_loaded': False,
|
||||
'tunnel_loaded': False,
|
||||
}
|
||||
|
||||
old: dict[str, object] = run_cold_import(
|
||||
'import importlib.util, json; '
|
||||
'import tractor.discovery as discovery; '
|
||||
'old_names = ("Bindspace", "TunnelledAddress", '
|
||||
'"mk_maddr", "parse_maddr", "parse_endpoints"); '
|
||||
'old_modules = ("tractor.discovery._bindspace", '
|
||||
'"tractor.discovery._tunnel"); '
|
||||
'print(json.dumps({'
|
||||
'"exports": [name for name in old_names '
|
||||
'if hasattr(discovery, name)], '
|
||||
'"modules": [name for name in old_modules '
|
||||
'if importlib.util.find_spec(name) is not None]'
|
||||
'}))'
|
||||
)
|
||||
assert old == {
|
||||
'exports': [],
|
||||
'modules': [],
|
||||
}
|
||||
|
||||
|
||||
def test_lazy_annotation_names_resolve():
|
||||
'''
|
||||
Resolve annotations without importing optional dependencies.
|
||||
|
|
@ -157,4 +281,7 @@ def test_lazy_annotation_names_resolve():
|
|||
assert get_type_hints(_addr.Address.get_random)[
|
||||
'current_actor'
|
||||
] is Any
|
||||
assert get_type_hints(tractor.open_root_actor)[
|
||||
'bindspace'
|
||||
] == Any|None
|
||||
assert _addr.__annotations__['_address_types'].startswith('dict')
|
||||
|
|
|
|||
|
|
@ -22,9 +22,12 @@ import pytest
|
|||
import trio
|
||||
|
||||
import tractor
|
||||
from tractor import _child
|
||||
from tractor import (
|
||||
_child,
|
||||
_root,
|
||||
)
|
||||
from tractor.devx import _proctitle
|
||||
from tractor.discovery._bindspace import (
|
||||
from tractor.net._bindspace import (
|
||||
Bindspace,
|
||||
BindspaceRef,
|
||||
BindspaceSpec,
|
||||
|
|
@ -40,8 +43,12 @@ from tractor.spawn import (
|
|||
from tractor.trionics import patches
|
||||
|
||||
|
||||
_SELF_NETNS_PATH = Path('/proc/self/ns/net')
|
||||
_SELF_NETNS_PATH = Path('/proc/thread-self/ns/net')
|
||||
_SELF_FD_DIR = Path('/proc/self/fd')
|
||||
_linux_netns_only = pytest.mark.skipif(
|
||||
sys.platform != 'linux',
|
||||
reason='Linux network namespace API',
|
||||
)
|
||||
|
||||
|
||||
def _assert_fd_closed(namespace_fd: int) -> None:
|
||||
|
|
@ -108,6 +115,87 @@ def _bindspace_for_fd(namespace_fd: int) -> Bindspace:
|
|||
)
|
||||
|
||||
|
||||
def _run_in_unshared_netns(
|
||||
test_name: str,
|
||||
reexec_var: str,
|
||||
) -> bool:
|
||||
'''
|
||||
Re-exec one E2E test with disposable user and net namespaces.
|
||||
|
||||
Return `True` in the outer pytest process after nested pytest
|
||||
succeeds. Return `False` inside that nested process so the caller
|
||||
performs the privileged namespace transitions itself.
|
||||
'''
|
||||
if os.environ.get(reexec_var) == '1':
|
||||
return False
|
||||
|
||||
unshare_path: str|None = shutil.which('unshare')
|
||||
if unshare_path is None:
|
||||
pytest.skip('`unshare` is unavailable')
|
||||
|
||||
# Give nested pytest `CAP_SYS_ADMIN` only inside a disposable
|
||||
# user namespace. Probe separately so hosts disabling
|
||||
# unprivileged user namespaces skip cleanly.
|
||||
probe = subprocess.run(
|
||||
[
|
||||
unshare_path,
|
||||
'--user',
|
||||
'--map-root-user',
|
||||
'--net',
|
||||
'true',
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if probe.returncode:
|
||||
reason: str = probe.stderr.strip()
|
||||
pytest.skip(
|
||||
f'unprivileged user/net namespaces unavailable: '
|
||||
f'{reason}'
|
||||
)
|
||||
|
||||
nested_env: dict[str, str] = dict(os.environ)
|
||||
nested_env[reexec_var] = '1'
|
||||
nested_env['VIRTUAL_ENV'] = sys.prefix
|
||||
nested_rt_dir: Path = Path(
|
||||
tempfile.mkdtemp(prefix='tne-')
|
||||
)
|
||||
nested_env['XDG_RUNTIME_DIR'] = str(nested_rt_dir)
|
||||
python_bin: str = str(Path(sys.executable).parent)
|
||||
nested_env['PATH'] = (
|
||||
python_bin
|
||||
+ os.pathsep
|
||||
+ nested_env['PATH']
|
||||
)
|
||||
test_id: str = f'tests/test_netns_spawn.py::{test_name}'
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
unshare_path,
|
||||
'--user',
|
||||
'--map-root-user',
|
||||
'--net',
|
||||
sys.executable,
|
||||
'-m',
|
||||
'pytest',
|
||||
test_id,
|
||||
'--spawn-backend=trio',
|
||||
'--tpt-proto=uds',
|
||||
'-x',
|
||||
'--tb=short',
|
||||
'--no-header',
|
||||
'--timeout=30',
|
||||
],
|
||||
env=nested_env,
|
||||
check=True,
|
||||
)
|
||||
finally:
|
||||
shutil.rmtree(nested_rt_dir)
|
||||
|
||||
return True
|
||||
|
||||
|
||||
class _MockIpcServer:
|
||||
'''
|
||||
Provide the peer-event state used by `trio_proc()` tests.
|
||||
|
|
@ -211,6 +299,347 @@ async def _report_child_netns(
|
|||
os.close(child_netns_fd)
|
||||
|
||||
|
||||
@_linux_netns_only
|
||||
def test_root_netns_same_namespace_skips_setns(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
'''
|
||||
Root entry into the current netns must not need privilege.
|
||||
|
||||
Pin the real current namespace and arm `enter_netns()` as a
|
||||
failure sentinel. The context validates through a duplicate,
|
||||
yield the current inode without calling `setns()`, preserve the
|
||||
source capability, and close the duplicates on normal exit.
|
||||
'''
|
||||
namespace_fd: int = os.open(
|
||||
_SELF_NETNS_PATH,
|
||||
os.O_RDONLY,
|
||||
)
|
||||
bindspace: Bindspace = _bindspace_for_fd(namespace_fd)
|
||||
initial_fds: set[int] = _fds_referencing(namespace_fd)
|
||||
|
||||
def fail_enter_netns(
|
||||
inherited_fd: int,
|
||||
inode: int,
|
||||
) -> int:
|
||||
'''
|
||||
Reject a privileged transition for the already-current netns.
|
||||
|
||||
'''
|
||||
raise AssertionError('same-netns entry called `setns()`')
|
||||
|
||||
monkeypatch.setattr(_netns, 'enter_netns', fail_enter_netns)
|
||||
try:
|
||||
with _netns._enter_netns_temporarily(
|
||||
bindspace,
|
||||
) as entered_inode:
|
||||
assert entered_inode == bindspace.ref.inode
|
||||
assert os.fstat(namespace_fd).st_ino == entered_inode
|
||||
# The target duplicate and original-netns snapshot both
|
||||
# reference the already-current namespace.
|
||||
assert len(_fds_referencing(namespace_fd)) == (
|
||||
len(initial_fds) + 2
|
||||
)
|
||||
|
||||
assert _fds_referencing(namespace_fd) == initial_fds
|
||||
finally:
|
||||
os.close(namespace_fd)
|
||||
|
||||
|
||||
@_linux_netns_only
|
||||
def test_root_netns_restores_after_body_error(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
'''
|
||||
A root-body failure must restore netns before it escapes.
|
||||
|
||||
Use distinct regular files as deterministic namespace stand-ins,
|
||||
replace only `enter_netns()`, and raise a unique error from the
|
||||
context body. The recorded transitions prove target entry then
|
||||
original restoration. FD snapshots prove neither temporary handle
|
||||
leaks, while the caller-owned target FD remains live.
|
||||
'''
|
||||
original_path: Path = tmp_path / 'original-netns'
|
||||
target_path: Path = tmp_path / 'target-netns'
|
||||
original_path.touch()
|
||||
target_path.touch()
|
||||
namespace_fd: int = os.open(target_path, os.O_RDONLY)
|
||||
bindspace: Bindspace = _bindspace_for_fd(namespace_fd)
|
||||
initial_fds: set[int] = _fds_referencing(namespace_fd)
|
||||
transitions: list[int] = []
|
||||
body_error = RuntimeError('root body failed')
|
||||
|
||||
def fake_enter_netns(
|
||||
inherited_fd: int,
|
||||
inode: int,
|
||||
) -> int:
|
||||
'''
|
||||
Record each verified target or restoration descriptor.
|
||||
|
||||
'''
|
||||
assert os.fstat(inherited_fd).st_ino == inode
|
||||
transitions.append(inode)
|
||||
return inode
|
||||
|
||||
monkeypatch.setattr(_netns, '_SELF_NETNS', original_path)
|
||||
monkeypatch.setattr(_netns, 'enter_netns', fake_enter_netns)
|
||||
try:
|
||||
with pytest.raises(RuntimeError) as exc_info:
|
||||
with _netns._enter_netns_temporarily(bindspace):
|
||||
raise body_error
|
||||
|
||||
assert exc_info.value is body_error
|
||||
# The fake records target entry before the body, then original
|
||||
# restoration during context exit.
|
||||
assert transitions == [
|
||||
bindspace.ref.inode,
|
||||
original_path.stat().st_ino,
|
||||
]
|
||||
assert _fds_referencing(namespace_fd) == initial_fds
|
||||
assert os.fstat(namespace_fd).st_ino == bindspace.ref.inode
|
||||
finally:
|
||||
os.close(namespace_fd)
|
||||
|
||||
|
||||
@_linux_netns_only
|
||||
def test_root_netns_restores_on_trio_cancellation(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
'''
|
||||
Trio cancellation must not interrupt root-netns restoration.
|
||||
|
||||
Use deterministic namespace stand-ins and cancel the task inside
|
||||
`_enter_root_bindspace()` immediately before an explicit Trio
|
||||
checkpoint. The enclosing `CancelScope` catches cancellation only
|
||||
after async-context exit. Two recorded sync transitions and exact
|
||||
FD state then prove restoration and close completed first.
|
||||
'''
|
||||
original_path: Path = tmp_path / 'cancel-original-netns'
|
||||
target_path: Path = tmp_path / 'cancel-target-netns'
|
||||
original_path.touch()
|
||||
target_path.touch()
|
||||
namespace_fd: int = os.open(target_path, os.O_RDONLY)
|
||||
bindspace: Bindspace = _bindspace_for_fd(namespace_fd)
|
||||
initial_fds: set[int] = _fds_referencing(namespace_fd)
|
||||
transitions: list[int] = []
|
||||
|
||||
def fake_enter_netns(
|
||||
inherited_fd: int,
|
||||
inode: int,
|
||||
) -> int:
|
||||
'''
|
||||
Record target entry and original-netns restoration.
|
||||
|
||||
'''
|
||||
assert os.fstat(inherited_fd).st_ino == inode
|
||||
transitions.append(inode)
|
||||
return inode
|
||||
|
||||
monkeypatch.setattr(_netns, '_SELF_NETNS', original_path)
|
||||
monkeypatch.setattr(_netns, 'enter_netns', fake_enter_netns)
|
||||
|
||||
async def main() -> None:
|
||||
'''
|
||||
Deliver cancellation at a checkpoint inside the netns scope.
|
||||
|
||||
'''
|
||||
with trio.CancelScope() as cancel_scope:
|
||||
async with _root._enter_root_bindspace(bindspace):
|
||||
cancel_scope.cancel()
|
||||
await trio.lowlevel.checkpoint()
|
||||
|
||||
assert cancel_scope.cancelled_caught
|
||||
|
||||
try:
|
||||
trio.run(main)
|
||||
# Target entry is recorded first; original-netns restoration
|
||||
# is recorded when `_enter_root_bindspace()` exits.
|
||||
assert transitions == [
|
||||
bindspace.ref.inode,
|
||||
original_path.stat().st_ino,
|
||||
]
|
||||
assert _fds_referencing(namespace_fd) == initial_fds
|
||||
assert os.fstat(namespace_fd).st_ino == bindspace.ref.inode
|
||||
finally:
|
||||
os.close(namespace_fd)
|
||||
|
||||
|
||||
@_linux_netns_only
|
||||
@pytest.mark.parametrize('body_fails', (False, True))
|
||||
def test_root_netns_restore_error_precedence(
|
||||
body_fails: bool,
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
'''
|
||||
Netns restoration failure must not hide a root-body failure.
|
||||
|
||||
Model target entry as successful and fail the second transition,
|
||||
which is restoration. The normal-body case must propagate that
|
||||
restoration error. The failing-body case must instead preserve
|
||||
its unique error and attach restoration failure as a note. In both
|
||||
schedules an exact target-FD snapshot proves cleanup still closes
|
||||
the context's duplicates.
|
||||
'''
|
||||
original_path: Path = tmp_path / 'failed-restore-original'
|
||||
target_path: Path = tmp_path / 'failed-restore-target'
|
||||
original_path.touch()
|
||||
target_path.touch()
|
||||
namespace_fd: int = os.open(target_path, os.O_RDONLY)
|
||||
bindspace: Bindspace = _bindspace_for_fd(namespace_fd)
|
||||
initial_fds: set[int] = _fds_referencing(namespace_fd)
|
||||
body_error = RuntimeError('root body failed first')
|
||||
restore_error = RuntimeError('root netns restore failed')
|
||||
transitions: int = 0
|
||||
|
||||
def fail_restore(
|
||||
inherited_fd: int,
|
||||
inode: int,
|
||||
) -> int:
|
||||
'''
|
||||
Enter the target once, then fail original-netns restoration.
|
||||
|
||||
'''
|
||||
nonlocal transitions
|
||||
assert os.fstat(inherited_fd).st_ino == inode
|
||||
transitions += 1
|
||||
if transitions == 2:
|
||||
raise restore_error
|
||||
return inode
|
||||
|
||||
monkeypatch.setattr(_netns, '_SELF_NETNS', original_path)
|
||||
monkeypatch.setattr(_netns, 'enter_netns', fail_restore)
|
||||
expected_error: RuntimeError = (
|
||||
body_error
|
||||
if body_fails
|
||||
else restore_error
|
||||
)
|
||||
try:
|
||||
with pytest.raises(RuntimeError) as exc_info:
|
||||
with _netns._enter_netns_temporarily(bindspace):
|
||||
if body_fails:
|
||||
raise body_error
|
||||
|
||||
assert exc_info.value is expected_error
|
||||
assert transitions == 2
|
||||
if body_fails:
|
||||
assert body_error.__notes__
|
||||
assert 'restore the original' in body_error.__notes__[0]
|
||||
assert repr(restore_error) in body_error.__notes__[0]
|
||||
assert _fds_referencing(namespace_fd) == initial_fds
|
||||
finally:
|
||||
os.close(namespace_fd)
|
||||
|
||||
|
||||
@_linux_netns_only
|
||||
def test_root_netns_requires_live_bindspace_fd() -> None:
|
||||
'''
|
||||
Root entry cannot use `BindspaceRef.inode` without a live FD.
|
||||
|
||||
Construct a valid ref-only `Bindspace` and enter the real root
|
||||
namespace scope directly. The concrete live-FD error must occur
|
||||
before namespace capture, probes, sockets, or actor runtime work.
|
||||
'''
|
||||
key: str = 'missing-root-netns'
|
||||
bindspace = Bindspace(
|
||||
spec=BindspaceSpec(
|
||||
kind='netns',
|
||||
key=key,
|
||||
),
|
||||
ref=BindspaceRef(
|
||||
kind='netns',
|
||||
key=key,
|
||||
inode=1,
|
||||
),
|
||||
# A stored inode cannot authorize namespace entry.
|
||||
namespace_fd=None,
|
||||
ownership='borrowed',
|
||||
)
|
||||
|
||||
with pytest.raises(
|
||||
ValueError,
|
||||
match='bindspace.namespace_fd.*live netns FD',
|
||||
):
|
||||
# Scope entry must reject the missing live handle.
|
||||
with _netns._enter_netns_temporarily(bindspace):
|
||||
pytest.fail('root scope accepted a ref-only bindspace')
|
||||
|
||||
|
||||
@_linux_netns_only
|
||||
def test_root_netns_rejects_closed_bindspace_fd() -> None:
|
||||
'''
|
||||
A stale integer is not a live root-netns capability.
|
||||
|
||||
Construct a `Bindspace` while its real current-netns FD is open,
|
||||
close that caller-owned descriptor, then attempt root entry. The
|
||||
concrete live-FD error proves `os.dup()` validates the descriptor
|
||||
at entry time instead of trusting construction-time metadata.
|
||||
'''
|
||||
namespace_fd: int = os.open(
|
||||
_SELF_NETNS_PATH,
|
||||
os.O_RDONLY,
|
||||
)
|
||||
bindspace: Bindspace = _bindspace_for_fd(namespace_fd)
|
||||
os.close(namespace_fd)
|
||||
|
||||
with pytest.raises(
|
||||
ValueError,
|
||||
match='bindspace.namespace_fd.*live FD',
|
||||
):
|
||||
with _netns._enter_netns_temporarily(bindspace):
|
||||
pytest.fail('root scope accepted a closed bindspace FD')
|
||||
|
||||
|
||||
@_linux_netns_only
|
||||
def test_bound_root_rejects_persistent_forkserver(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
'''
|
||||
A persistent multiprocessing forkserver process can retain a
|
||||
previous root's netns.
|
||||
|
||||
Select `mp_forkserver` before opening a later bound root, modeling
|
||||
reuse of the forkserver process which `multiprocessing` creates
|
||||
once and uses for later child starts. The root API must reject that
|
||||
backend before namespace entry or runtime startup, preventing
|
||||
default children from silently inheriting its stale namespace.
|
||||
|
||||
'''
|
||||
namespace_fd: int = os.open(
|
||||
_SELF_NETNS_PATH,
|
||||
os.O_RDONLY,
|
||||
)
|
||||
bindspace: Bindspace = _bindspace_for_fd(namespace_fd)
|
||||
monkeypatch.setattr(
|
||||
_spawn,
|
||||
'_spawn_method',
|
||||
'mp_forkserver',
|
||||
)
|
||||
|
||||
async def main() -> None:
|
||||
'''
|
||||
Reject the unsafe backend at root-context entry.
|
||||
|
||||
'''
|
||||
with pytest.raises(
|
||||
NotImplementedError,
|
||||
match='persistent forkserver',
|
||||
):
|
||||
async with tractor.open_root_actor(
|
||||
bindspace=bindspace,
|
||||
):
|
||||
pytest.fail('bound root started under mp_forkserver')
|
||||
|
||||
try:
|
||||
trio.run(main)
|
||||
assert os.fstat(namespace_fd).st_ino == bindspace.ref.inode
|
||||
finally:
|
||||
os.close(namespace_fd)
|
||||
|
||||
|
||||
@_linux_netns_only
|
||||
def test_enter_netns_rejects_mismatched_inherited_fd(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
|
|
@ -244,6 +673,7 @@ def test_enter_netns_rejects_mismatched_inherited_fd(
|
|||
)
|
||||
|
||||
|
||||
@_linux_netns_only
|
||||
def test_enter_netns_verifies_post_entry_inode(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
|
|
@ -252,7 +682,7 @@ def test_enter_netns_verifies_post_entry_inode(
|
|||
Successful `setns()` is insufficient without post-entry proof.
|
||||
|
||||
Use a real inherited FD and fake only the privileged syscall and
|
||||
`/proc/self/ns/net` observation. The recorded calls prove both
|
||||
`/proc/thread-self/ns/net` observation. The calls prove both
|
||||
hooks execute and `CLONE_NEWNET` constrains the namespace type;
|
||||
the returned inode proves bootstrap observed the expected netns.
|
||||
|
||||
|
|
@ -292,6 +722,7 @@ def test_enter_netns_verifies_post_entry_inode(
|
|||
assert entered_inode == inode
|
||||
|
||||
|
||||
@_linux_netns_only
|
||||
def test_enter_netns_rejects_wrong_post_entry_namespace(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
|
|
@ -300,7 +731,7 @@ def test_enter_netns_rejects_wrong_post_entry_namespace(
|
|||
Bootstrap must stop when the process lands in an unexpected netns.
|
||||
|
||||
Let the inherited FD check and fake syscall succeed, then report a
|
||||
different `/proc/self/ns/net` inode. The post-entry guard must raise
|
||||
different `/proc/thread-self/ns/net` inode. The guard must raise
|
||||
instead of allowing actor runtime sockets to start in the wrong
|
||||
namespace.
|
||||
|
||||
|
|
@ -562,76 +993,12 @@ def test_trio_spawn_relays_bindspace_to_child_actor(
|
|||
if start_method != 'trio':
|
||||
pytest.skip('bindspace FD relay is implemented by Trio spawn')
|
||||
|
||||
reexec_var: str = 'TRACTOR_TEST_NETNS_E2E_REEXEC'
|
||||
if os.environ.get(reexec_var) != '1':
|
||||
unshare_path: str|None = shutil.which('unshare')
|
||||
if unshare_path is None:
|
||||
pytest.skip('`unshare` is unavailable')
|
||||
|
||||
# Give nested pytest `CAP_SYS_ADMIN` only inside a disposable
|
||||
# user namespace. The first `--net` creates the target netns;
|
||||
# the nested test retains its FD before creating a second netns
|
||||
# for the parent and child to inherit at spawn. Probe separately
|
||||
# so hosts disabling unprivileged user namespaces skip cleanly.
|
||||
probe = subprocess.run(
|
||||
[
|
||||
unshare_path,
|
||||
'--user',
|
||||
'--map-root-user',
|
||||
'--net',
|
||||
'true',
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if probe.returncode:
|
||||
reason: str = probe.stderr.strip()
|
||||
pytest.skip(
|
||||
f'unprivileged user/net namespaces unavailable: '
|
||||
f'{reason}'
|
||||
)
|
||||
|
||||
nested_env: dict[str, str] = dict(os.environ)
|
||||
nested_env[reexec_var] = '1'
|
||||
nested_env['VIRTUAL_ENV'] = sys.prefix
|
||||
nested_rt_dir: Path = Path(
|
||||
tempfile.mkdtemp(prefix='tne-')
|
||||
)
|
||||
nested_env['XDG_RUNTIME_DIR'] = str(nested_rt_dir)
|
||||
python_bin: str = str(Path(sys.executable).parent)
|
||||
nested_env['PATH'] = (
|
||||
python_bin
|
||||
+ os.pathsep
|
||||
+ nested_env['PATH']
|
||||
)
|
||||
test_id: str = (
|
||||
'tests/test_netns_spawn.py::'
|
||||
if _run_in_unshared_netns(
|
||||
test_name=(
|
||||
'test_trio_spawn_relays_bindspace_to_child_actor'
|
||||
)
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
unshare_path,
|
||||
'--user',
|
||||
'--map-root-user',
|
||||
'--net',
|
||||
sys.executable,
|
||||
'-m',
|
||||
'pytest',
|
||||
test_id,
|
||||
'--spawn-backend=trio',
|
||||
'--tpt-proto=uds',
|
||||
'-x',
|
||||
'--tb=short',
|
||||
'--no-header',
|
||||
'--timeout=30',
|
||||
],
|
||||
env=nested_env,
|
||||
check=True,
|
||||
)
|
||||
finally:
|
||||
shutil.rmtree(nested_rt_dir)
|
||||
),
|
||||
reexec_var='TRACTOR_TEST_NETNS_E2E_REEXEC',
|
||||
):
|
||||
return
|
||||
|
||||
assert start_method == 'trio'
|
||||
|
|
@ -708,6 +1075,74 @@ def test_trio_spawn_relays_bindspace_to_child_actor(
|
|||
os.close(inherited_fd)
|
||||
|
||||
|
||||
def test_root_actor_enters_and_restores_bindspace(
|
||||
tpt_proto: str,
|
||||
) -> None:
|
||||
'''
|
||||
`open_root_actor()` must enter its supplied networking bindspace.
|
||||
|
||||
Re-exec under an unprivileged user/net namespace, retain that
|
||||
first netns as the target, then move nested pytest into a second.
|
||||
A real UDS root actor must run its body in the target inode and
|
||||
keep the source capability open. After full actor teardown, exact
|
||||
FD and inode assertions prove its duplicate did not leak and the
|
||||
caller thread returned to the second/original netns.
|
||||
'''
|
||||
if _run_in_unshared_netns(
|
||||
test_name='test_root_actor_enters_and_restores_bindspace',
|
||||
reexec_var='TRACTOR_TEST_ROOT_NETNS_E2E_REEXEC',
|
||||
):
|
||||
return
|
||||
|
||||
assert tpt_proto == 'uds'
|
||||
target_netns_fd: int = os.open(
|
||||
_SELF_NETNS_PATH,
|
||||
os.O_RDONLY,
|
||||
)
|
||||
target_netns_inode: int = os.fstat(target_netns_fd).st_ino
|
||||
reffed_tgt_fds: set[int] = _fds_referencing(
|
||||
target_netns_fd,
|
||||
)
|
||||
bindspace: Bindspace = _bindspace_for_fd(target_netns_fd)
|
||||
|
||||
# Pin the first disposable netns through `target_netns_fd`, then
|
||||
# move the caller into a distinct second netns. This gives the root
|
||||
# one real target to enter and one real caller netns to restore.
|
||||
os.unshare(os.CLONE_NEWNET)
|
||||
original_netns_fd: int = os.open(
|
||||
_SELF_NETNS_PATH,
|
||||
os.O_RDONLY,
|
||||
)
|
||||
original_netns_inode: int = os.fstat(original_netns_fd).st_ino
|
||||
assert original_netns_inode != target_netns_inode
|
||||
|
||||
async def main() -> None:
|
||||
'''
|
||||
Inspect the real root runtime inside the target netns.
|
||||
|
||||
'''
|
||||
async with tractor.open_root_actor(
|
||||
bindspace=bindspace,
|
||||
enable_transports=['uds'],
|
||||
):
|
||||
body_netns_inode: int = _SELF_NETNS_PATH.stat().st_ino
|
||||
assert body_netns_inode == target_netns_inode
|
||||
assert os.fstat(target_netns_fd).st_ino == (
|
||||
target_netns_inode
|
||||
)
|
||||
|
||||
try:
|
||||
trio.run(main)
|
||||
assert _SELF_NETNS_PATH.stat().st_ino == original_netns_inode
|
||||
assert _fds_referencing(
|
||||
target_netns_fd,
|
||||
) == reffed_tgt_fds
|
||||
assert os.fstat(target_netns_fd).st_ino == target_netns_inode
|
||||
finally:
|
||||
os.close(original_netns_fd)
|
||||
os.close(target_netns_fd)
|
||||
|
||||
|
||||
def test_trio_spawn_failure_closes_child_netns_fd_in_parent(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
|
|
|
|||
|
|
@ -18,6 +18,7 @@
|
|||
tractor: structured concurrent ``trio``-"actors".
|
||||
|
||||
"""
|
||||
from types import ModuleType as _ModuleType
|
||||
|
||||
from ._clustering import (
|
||||
open_actor_cluster as open_actor_cluster,
|
||||
|
|
@ -82,6 +83,7 @@ __all__: tuple[str, ...] = tuple(
|
|||
for name in globals()
|
||||
if not name.startswith('_')
|
||||
) + (
|
||||
'net',
|
||||
'to_asyncio',
|
||||
)
|
||||
|
||||
|
|
@ -92,21 +94,18 @@ def __dir__() -> list[str]:
|
|||
|
||||
def __getattr__(name: str):
|
||||
'''
|
||||
PEP 562 lazy sub-module loading, presently only for
|
||||
`.to_asyncio` which (transitively) imports `asyncio`
|
||||
itself: a non-trivial multi-ms chunk of the eager
|
||||
`import tractor` cost (gh #470) unneeded by
|
||||
`trio`-only apps.
|
||||
PEP 562 lazy public sub-package loading.
|
||||
|
||||
Any `tractor.to_asyncio.<attr>` access (or a
|
||||
`from tractor import to_asyncio`) still works, the
|
||||
sub-mod is simply imported on first-access instead
|
||||
of at pkg-import time.
|
||||
`tractor.to_asyncio` transitively imports `asyncio`, while
|
||||
`tractor.net` owns optional network dependencies. Neither is
|
||||
needed by most applications merely importing the root package.
|
||||
|
||||
'''
|
||||
if name == 'to_asyncio':
|
||||
if name in ('net', 'to_asyncio'):
|
||||
from importlib import import_module
|
||||
return import_module('.to_asyncio', __name__)
|
||||
module: _ModuleType = import_module(f'.{name}', __name__)
|
||||
globals()[name] = module
|
||||
return module
|
||||
|
||||
raise AttributeError(
|
||||
f'module {__name__!r} has no attribute {name!r}'
|
||||
|
|
|
|||
|
|
@ -18,6 +18,9 @@
|
|||
Root actor runtime ignition(s).
|
||||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import (
|
||||
asynccontextmanager as acm,
|
||||
)
|
||||
|
|
@ -31,6 +34,7 @@ import sys
|
|||
from typing import (
|
||||
Any,
|
||||
Callable,
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
import warnings
|
||||
|
||||
|
|
@ -55,7 +59,6 @@ from .discovery._addr import (
|
|||
mk_uuid,
|
||||
wrap_address,
|
||||
)
|
||||
from .discovery._tunnel import strip_tunnels
|
||||
from .trionics import (
|
||||
is_multi_cancelled,
|
||||
collapse_eg,
|
||||
|
|
@ -64,6 +67,11 @@ from ._exceptions import (
|
|||
RuntimeFailure,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .net._bindspace import Bindspace
|
||||
else:
|
||||
Bindspace = Any
|
||||
|
||||
|
||||
logger = log.get_logger('tractor')
|
||||
|
||||
|
|
@ -154,9 +162,30 @@ async def maybe_block_bp(
|
|||
os.environ.pop('PYTHONBREAKPOINT', None)
|
||||
|
||||
|
||||
@acm
|
||||
async def _enter_root_bindspace(
|
||||
bindspace: Bindspace|None,
|
||||
) -> AsyncIterator[None]:
|
||||
'''
|
||||
Adapt synchronous root netns entry to the outer async lifecycle.
|
||||
|
||||
The wrapped context has no checkpoints, so `trio` cancellation
|
||||
cannot interrupt thread-local namespace restoration.
|
||||
|
||||
'''
|
||||
from .spawn._netns import _enter_netns_temporarily
|
||||
|
||||
with _enter_netns_temporarily(bindspace):
|
||||
yield
|
||||
|
||||
|
||||
@acm
|
||||
async def open_root_actor(
|
||||
*,
|
||||
# Low-level realized scope. A future tunnelled-address bootstrap
|
||||
# may open and supply this capability internally.
|
||||
bindspace: Bindspace|None = None,
|
||||
|
||||
tpt_bind_addrs: list[
|
||||
Address # `Address.get_random()` case
|
||||
|UnwrappedAddress # registrar case `= uw_reg_addrs`
|
||||
|
|
@ -220,6 +249,10 @@ async def open_root_actor(
|
|||
All (disjoint) actor-process-trees-as-programs are created via
|
||||
this entrypoint.
|
||||
|
||||
When `bindspace` is provided, enter its network namespace before
|
||||
any registry or IPC activity and restore the calling thread's
|
||||
original namespace after complete actor teardown.
|
||||
|
||||
'''
|
||||
# XXX NEVER allow nested actor-trees!
|
||||
if already_actor := _state.current_actor(
|
||||
|
|
@ -240,10 +273,29 @@ async def open_root_actor(
|
|||
f'_registry_addrs: {registry_addrs!r}\n'
|
||||
)
|
||||
|
||||
effective_start_method: str = (
|
||||
os.environ.get('TRACTOR_SPAWN_METHOD')
|
||||
or start_method
|
||||
or _spawn._spawn_method
|
||||
)
|
||||
if (
|
||||
bindspace is not None
|
||||
and
|
||||
effective_start_method == 'mp_forkserver'
|
||||
):
|
||||
raise NotImplementedError(
|
||||
'Root actor bindspaces are not supported by the '
|
||||
'`mp_forkserver` spawn backend because a persistent '
|
||||
'forkserver may retain its original network namespace!'
|
||||
)
|
||||
|
||||
# debug.mk_pdb().set_trace()
|
||||
async with maybe_block_bp(
|
||||
debug_mode=debug_mode,
|
||||
maybe_enable_greenback=maybe_enable_greenback,
|
||||
async with (
|
||||
_enter_root_bindspace(bindspace),
|
||||
maybe_block_bp(
|
||||
debug_mode=debug_mode,
|
||||
maybe_enable_greenback=maybe_enable_greenback,
|
||||
),
|
||||
):
|
||||
if enable_transports is None:
|
||||
enable_transports: list[str] = _state.current_ipc_protos()
|
||||
|
|
@ -505,6 +557,8 @@ async def open_root_actor(
|
|||
# XXX INSTEAD, bind random addrs using the same tpt
|
||||
# proto if not already provided.
|
||||
if not tpt_bind_addrs:
|
||||
from .net._tunnel import strip_tunnels
|
||||
|
||||
for addr in ponged_addrs:
|
||||
bindable_addr: Address = strip_tunnels(addr)
|
||||
tpt_bind_addrs.append(
|
||||
|
|
|
|||
|
|
@ -15,48 +15,8 @@
|
|||
# along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
'''
|
||||
Discovery (protocols) API for automatic addressing
|
||||
and location management of (service) actors.
|
||||
Actor discovery and registrar implementation package.
|
||||
|
||||
NOTE: this ``__init__`` only eagerly imports the lightweight
|
||||
``._multiaddr`` and ``._tunnel`` submodules for public re-exports.
|
||||
Heavier submodules like ``._addr`` and ``._api`` are NOT imported
|
||||
here to avoid circular imports; use direct module paths for those.
|
||||
Network declarations and helpers are public from `tractor.net`.
|
||||
|
||||
'''
|
||||
from ._bindspace import (
|
||||
Bindspace as Bindspace,
|
||||
BindspaceKind as BindspaceKind,
|
||||
BindspaceLifecycle as BindspaceLifecycle,
|
||||
BindspaceOwnership as BindspaceOwnership,
|
||||
BindspaceRef as BindspaceRef,
|
||||
BindspaceSpec as BindspaceSpec,
|
||||
CURRENT_NETNS as CURRENT_NETNS,
|
||||
attach_netns as attach_netns,
|
||||
open_bindspace as open_bindspace,
|
||||
open_netns as open_netns,
|
||||
)
|
||||
from ._multiaddr import (
|
||||
parse_endpoints as parse_endpoints,
|
||||
parse_maddr as parse_maddr,
|
||||
mk_maddr as mk_maddr,
|
||||
)
|
||||
from ._tunnel import (
|
||||
TunnelledAddress as TunnelledAddress,
|
||||
TunnelSpec as TunnelSpec,
|
||||
WGTunnelSpec as WGTunnelSpec,
|
||||
WGInterfaceConfig as WGInterfaceConfig,
|
||||
WGPeerConfig as WGPeerConfig,
|
||||
WGRole as WGRole,
|
||||
mb_pubkey as mb_pubkey,
|
||||
mk_wg_maddr as mk_wg_maddr,
|
||||
open_wg_bindspace as open_wg_bindspace,
|
||||
open_wg_iface as open_wg_iface,
|
||||
parse_wg_maddr as parse_wg_maddr,
|
||||
read_wg_peers as read_wg_peers,
|
||||
read_wg_pubkey as read_wg_pubkey,
|
||||
strip_tunnels as strip_tunnels,
|
||||
tunnels_of as tunnels_of,
|
||||
verify_wg_peer as verify_wg_peer,
|
||||
wg8_pubkey as wg8_pubkey,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -42,7 +42,7 @@ from ..ipc._uds import (
|
|||
if TYPE_CHECKING:
|
||||
# ONLY type-annots, the eager import costs ~4.5ms
|
||||
# of `import tractor` wall-time (gh #470).
|
||||
from ._tunnel import (
|
||||
from tractor.net._tunnel import (
|
||||
TunnelledAddress,
|
||||
)
|
||||
from ..runtime._runtime import Actor
|
||||
|
|
@ -237,8 +237,9 @@ def is_wrapped_addr(addr: any) -> bool:
|
|||
# XXX NOTE, a `TunnelledAddress` is genuinely "wrapped" but is
|
||||
# deliberately NOT in `_address_types`: it has no
|
||||
# `MsgTransport` of its own (a tunnel is transparent to
|
||||
# `socket(2)`), so it gets no proto-key entry. See `._tunnel`.
|
||||
from ._tunnel import TunnelledAddress
|
||||
# `socket(2)`), so it gets no proto-key entry. See
|
||||
# `tractor.net._tunnel`.
|
||||
from tractor.net._tunnel import TunnelledAddress
|
||||
return (
|
||||
type(addr) in _address_types.values()
|
||||
or
|
||||
|
|
@ -333,7 +334,7 @@ def wrap_address(
|
|||
# multiaddr-format string, e.g.
|
||||
# '/ip4/127.0.0.1/tcp/1616'
|
||||
case str() if addr.startswith('/'):
|
||||
from tractor.discovery._multiaddr import (
|
||||
from tractor.net import (
|
||||
parse_maddr,
|
||||
)
|
||||
return parse_maddr(addr)
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@ if TYPE_CHECKING:
|
|||
# `import tractor` path (gh #470).
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.discovery._addr import Address
|
||||
from tractor.discovery._tunnel import (
|
||||
from tractor.net._tunnel import (
|
||||
TunnelledAddress,
|
||||
)
|
||||
else:
|
||||
|
|
@ -71,7 +71,7 @@ def mk_maddr(
|
|||
'''
|
||||
from multiaddr import Multiaddr
|
||||
|
||||
from ._tunnel import (
|
||||
from tractor.net._tunnel import (
|
||||
TunnelledAddress,
|
||||
mk_wg_maddr,
|
||||
)
|
||||
|
|
@ -130,7 +130,7 @@ def parse_maddr(
|
|||
# fails. Pre-checking the raw string would misclassify valid
|
||||
# values such as `/unix/tmp/wg/service.sock`.
|
||||
if '/wg/' in maddr_str:
|
||||
from ._tunnel import _wg_proto_code
|
||||
from tractor.net._tunnel import _wg_proto_code
|
||||
_wg_proto_code()
|
||||
raise
|
||||
proto_names: list[str] = [
|
||||
|
|
@ -156,7 +156,7 @@ def parse_maddr(
|
|||
)
|
||||
|
||||
case _ if 'wg' in proto_names:
|
||||
from ._tunnel import parse_wg_maddr
|
||||
from tractor.net._tunnel import parse_wg_maddr
|
||||
return parse_wg_maddr(maddr)
|
||||
|
||||
case _:
|
||||
|
|
|
|||
|
|
@ -46,10 +46,6 @@ from tractor.discovery._addr import (
|
|||
Address,
|
||||
UnwrappedAddress,
|
||||
)
|
||||
from tractor.discovery._tunnel import (
|
||||
TunnelledAddress,
|
||||
strip_tunnels,
|
||||
)
|
||||
from tractor.log import get_logger
|
||||
from tractor._exceptions import (
|
||||
MsgTypeError,
|
||||
|
|
@ -63,6 +59,9 @@ from tractor.msg import (
|
|||
|
||||
if TYPE_CHECKING:
|
||||
from ._transport import MsgTransport
|
||||
from tractor.net._tunnel import TunnelledAddress
|
||||
else:
|
||||
TunnelledAddress = Any
|
||||
|
||||
|
||||
log = get_logger()
|
||||
|
|
@ -190,6 +189,8 @@ class Channel:
|
|||
**kwargs
|
||||
) -> Channel:
|
||||
|
||||
from tractor.net._tunnel import strip_tunnels
|
||||
|
||||
if not is_wrapped_addr(addr):
|
||||
addr = wrap_address(addr)
|
||||
|
||||
|
|
|
|||
|
|
@ -68,7 +68,7 @@ from ._transport import MsgTransport
|
|||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ..discovery._tunnel import TunnelledAddress
|
||||
from ..net._tunnel import TunnelledAddress
|
||||
from ..runtime._runtime import Actor
|
||||
from ..runtime._supervise import ActorNursery
|
||||
|
||||
|
|
@ -1091,7 +1091,7 @@ async def _serve_ipc_eps(
|
|||
`.cancel_server()` is called.
|
||||
|
||||
'''
|
||||
from ..discovery._tunnel import strip_tunnels
|
||||
from ..net._tunnel import strip_tunnels
|
||||
|
||||
try:
|
||||
listen_tn: Nursery
|
||||
|
|
|
|||
|
|
@ -37,7 +37,6 @@ from trio import (
|
|||
|
||||
from tractor.msg import MsgCodec
|
||||
from tractor.log import get_logger
|
||||
from tractor.discovery._multiaddr import mk_maddr
|
||||
from tractor.ipc._transport import (
|
||||
MsgTransport,
|
||||
MsgpackTransport,
|
||||
|
|
@ -235,6 +234,8 @@ class MsgpackTCPStream(MsgpackTransport):
|
|||
|
||||
@property
|
||||
def maddr(self) -> Multiaddr:
|
||||
from tractor.net import mk_maddr
|
||||
|
||||
return mk_maddr(self.raddr)
|
||||
|
||||
def connected(self) -> bool:
|
||||
|
|
|
|||
|
|
@ -63,7 +63,6 @@ from trio._highlevel_open_unix_stream import (
|
|||
|
||||
from tractor.msg import MsgCodec
|
||||
from tractor.log import get_logger
|
||||
from tractor.discovery._multiaddr import mk_maddr
|
||||
from tractor.ipc._transport import (
|
||||
MsgpackTransport,
|
||||
)
|
||||
|
|
@ -611,6 +610,8 @@ class MsgpackUDSStream(MsgpackTransport):
|
|||
|
||||
@property
|
||||
def maddr(self) -> Multiaddr|str:
|
||||
from tractor.net import mk_maddr
|
||||
|
||||
if not self.raddr:
|
||||
return '<unknown-peer>'
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,75 @@
|
|||
# tractor: structured concurrent "actors".
|
||||
# Copyright 2018-eternity Tyler Goodlet.
|
||||
|
||||
'''
|
||||
Network declarations, bindspaces and tunnels.
|
||||
|
||||
Public symbols are imported and cached on first access so importing
|
||||
this package does not load optional network dependencies.
|
||||
|
||||
'''
|
||||
from importlib import import_module
|
||||
|
||||
|
||||
_SYMBOL_MODULES: dict[str, str] = {
|
||||
'Bindspace': '._bindspace',
|
||||
'BindspaceKind': '._bindspace',
|
||||
'BindspaceLifecycle': '._bindspace',
|
||||
'BindspaceOwnership': '._bindspace',
|
||||
'BindspaceRef': '._bindspace',
|
||||
'BindspaceSpec': '._bindspace',
|
||||
'CURRENT_NETNS': '._bindspace',
|
||||
'attach_netns': '._bindspace',
|
||||
'open_bindspace': '._bindspace',
|
||||
'open_netns': '._bindspace',
|
||||
'TunnelledAddress': '._tunnel',
|
||||
'TunnelSpec': '._tunnel',
|
||||
'WGTunnelSpec': '._tunnel',
|
||||
'WGInterfaceConfig': '._tunnel',
|
||||
'WGPeerConfig': '._tunnel',
|
||||
'WGRole': '._tunnel',
|
||||
'mb_pubkey': '._tunnel',
|
||||
'mk_wg_maddr': '._tunnel',
|
||||
'open_wg_bindspace': '._tunnel',
|
||||
'open_wg_iface': '._tunnel',
|
||||
'parse_wg_maddr': '._tunnel',
|
||||
'read_wg_peers': '._tunnel',
|
||||
'read_wg_pubkey': '._tunnel',
|
||||
'strip_tunnels': '._tunnel',
|
||||
'tunnels_of': '._tunnel',
|
||||
'verify_wg_peer': '._tunnel',
|
||||
'wg8_pubkey': '._tunnel',
|
||||
'mk_maddr': '..discovery._multiaddr',
|
||||
'parse_maddr': '..discovery._multiaddr',
|
||||
'parse_endpoints': '..discovery._multiaddr',
|
||||
}
|
||||
|
||||
__all__: tuple[str, ...] = tuple(_SYMBOL_MODULES)
|
||||
|
||||
|
||||
def __dir__() -> list[str]:
|
||||
'''
|
||||
Advertise the complete lazy public API.
|
||||
|
||||
'''
|
||||
return sorted(set(globals()) | set(__all__))
|
||||
|
||||
|
||||
def __getattr__(name: str) -> object:
|
||||
'''
|
||||
Import and cache one public network symbol on first access.
|
||||
|
||||
'''
|
||||
try:
|
||||
module_name: str = _SYMBOL_MODULES[name]
|
||||
except KeyError:
|
||||
raise AttributeError(
|
||||
f'module {__name__!r} has no attribute {name!r}'
|
||||
) from None
|
||||
|
||||
value: object = getattr(
|
||||
import_module(module_name, __name__),
|
||||
name,
|
||||
)
|
||||
globals()[name] = value
|
||||
return value
|
||||
|
|
@ -51,7 +51,7 @@ BindspaceOwnership: TypeAlias = Literal[
|
|||
]
|
||||
|
||||
_NETNS_RUN_DIR: Path = Path('/var/run/netns')
|
||||
_SELF_NETNS: Path = Path('/proc/self/ns/net')
|
||||
_THREAD_NETNS: Path = Path('/proc/thread-self/ns/net')
|
||||
|
||||
CURRENT_NETNS: Final[None] = None
|
||||
|
||||
|
|
@ -124,7 +124,7 @@ class BindspaceSpec(
|
|||
Serializable declaration of one requested bindspace.
|
||||
|
||||
For a netns spec, `.key = CURRENT_NETNS` selects the calling
|
||||
process's current namespace without a named-path lookup.
|
||||
thread's current namespace without a named-path lookup.
|
||||
|
||||
'''
|
||||
kind: BindspaceKind
|
||||
|
|
@ -277,7 +277,7 @@ async def _pin_netns(
|
|||
'''
|
||||
key: str|None = spec.key
|
||||
namespace_path: Path = (
|
||||
_SELF_NETNS
|
||||
_THREAD_NETNS
|
||||
if key is CURRENT_NETNS
|
||||
else _NETNS_RUN_DIR / key
|
||||
)
|
||||
|
|
@ -19,7 +19,8 @@ Tunnelled addresses: an `Address` that rides *inside* a tunnel.
|
|||
A tunnel (`wg`, and later plain ip-in-udp, `veth`-in-netns, ..) is
|
||||
**not** a `MsgTransport`. Its data plane is transparent to the
|
||||
application's `socket(2)`, so it never gets its own entry in
|
||||
`._addr._address_types` nor a `MsgpackTransport` impl. Instead it
|
||||
`tractor.discovery._addr._address_types` nor a `MsgpackTransport`
|
||||
impl. Instead it
|
||||
*annotates* an existing L4 addr, and this module carries that
|
||||
annotation beside it.
|
||||
|
||||
|
|
@ -102,7 +103,7 @@ from ._bindspace import (
|
|||
if TYPE_CHECKING:
|
||||
from multiaddr import Multiaddr
|
||||
|
||||
from ._addr import (
|
||||
from ..discovery._addr import (
|
||||
Address,
|
||||
UnwrappedAddress,
|
||||
)
|
||||
|
|
@ -1055,7 +1056,7 @@ def parse_wg_maddr(
|
|||
]
|
||||
match overlay_names:
|
||||
case [('ip4' | 'ip6'), 'tcp']:
|
||||
from ._multiaddr import parse_maddr
|
||||
from ..discovery._multiaddr import parse_maddr
|
||||
overlay: Address|TunnelledAddress = parse_maddr(
|
||||
str(overlay_ma)
|
||||
)
|
||||
|
|
@ -1161,7 +1162,7 @@ def mk_wg_maddr(
|
|||
f'/wg/{mb_pubkey(addr.tunnel.peer_pubkey)}'
|
||||
)
|
||||
|
||||
from ._multiaddr import mk_maddr
|
||||
from ..discovery._multiaddr import mk_maddr
|
||||
overlay_ma: Multiaddr = mk_maddr(addr.overlay)
|
||||
return (
|
||||
bearer_ma
|
||||
|
|
@ -65,7 +65,7 @@ from ..spawn import _spawn
|
|||
|
||||
if TYPE_CHECKING:
|
||||
import multiprocessing as mp
|
||||
from ..discovery._bindspace import Bindspace
|
||||
from ..net._bindspace import Bindspace
|
||||
# from ..ipc._server import IPCServer
|
||||
from ..ipc import IPCServer
|
||||
from ..spawn._spawn import ProcessType
|
||||
|
|
|
|||
|
|
@ -54,7 +54,7 @@ from ._spawn import (
|
|||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from tractor.discovery._bindspace import Bindspace
|
||||
from tractor.net._bindspace import Bindspace
|
||||
from tractor.ipc import (
|
||||
_server,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -20,13 +20,29 @@ Linux network-namespace actor-bootstrap primitives.
|
|||
'''
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from collections.abc import (
|
||||
Callable,
|
||||
Iterator,
|
||||
)
|
||||
from contextlib import contextmanager as cm
|
||||
import errno
|
||||
from pathlib import Path
|
||||
import os
|
||||
import sys
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
|
||||
_SELF_NETNS: Path = Path('/proc/self/ns/net')
|
||||
if TYPE_CHECKING:
|
||||
from ..net._bindspace import Bindspace
|
||||
|
||||
|
||||
# `setns(2)` mutates only the calling thread's namespace:
|
||||
# https://man7.org/linux/man-pages/man2/setns.2.html
|
||||
# `/proc/thread-self` addresses the caller's current task:
|
||||
# https://man7.org/linux/man-pages/man5/proc_pid_task.5.html
|
||||
# Do not use `/proc/self` because it resolves through the process
|
||||
# leader.
|
||||
_SELF_NETNS: Path = Path('/proc/thread-self/ns/net')
|
||||
|
||||
|
||||
def enter_netns(
|
||||
|
|
@ -36,7 +52,7 @@ def enter_netns(
|
|||
'''
|
||||
Enter and verify one inherited Linux network namespace.
|
||||
|
||||
The future spawn-bootstrap caller owns and closes `namespace_fd`.
|
||||
The caller owns and closes `namespace_fd`.
|
||||
|
||||
'''
|
||||
if sys.platform != 'linux':
|
||||
|
|
@ -102,3 +118,167 @@ def enter_netns(
|
|||
)
|
||||
|
||||
return entered_inode
|
||||
|
||||
|
||||
@cm
|
||||
def close_fd(
|
||||
owned_fd: int,
|
||||
fd_name: str,
|
||||
) -> Iterator[None]:
|
||||
'''
|
||||
Close one owned netns FD without masking a prior error.
|
||||
|
||||
'''
|
||||
operation: str = (
|
||||
f'close owned {fd_name} netns FD {owned_fd}'
|
||||
)
|
||||
try:
|
||||
yield
|
||||
except BaseException as primary_error:
|
||||
try:
|
||||
os.close(owned_fd)
|
||||
except BaseException as close_error:
|
||||
primary_error.add_note(
|
||||
f'Also failed to {operation}: {close_error!r}'
|
||||
)
|
||||
raise primary_error
|
||||
else:
|
||||
try:
|
||||
os.close(owned_fd)
|
||||
except BaseException as close_error:
|
||||
close_error.add_note(
|
||||
f'Failed to {operation} during root netns cleanup.'
|
||||
)
|
||||
raise close_error
|
||||
|
||||
|
||||
@cm
|
||||
def dup_fd(
|
||||
source_fd: int,
|
||||
) -> Iterator[int]:
|
||||
'''
|
||||
Duplicate and own the target netns FD for this context.
|
||||
|
||||
'''
|
||||
try:
|
||||
owned_fd: int = os.dup(source_fd)
|
||||
except OSError as dup_error:
|
||||
if dup_error.errno != errno.EBADF:
|
||||
raise dup_error
|
||||
raise ValueError(
|
||||
'`bindspace.namespace_fd` does not reference '
|
||||
'a live FD!'
|
||||
) from dup_error
|
||||
|
||||
with close_fd(owned_fd, 'target'):
|
||||
yield owned_fd
|
||||
|
||||
|
||||
@cm
|
||||
def _enter_netns_temporarily(
|
||||
bindspace: Bindspace|None,
|
||||
) -> Iterator[int|None]:
|
||||
'''
|
||||
Enter a root bindspace and restore the caller thread's netns.
|
||||
|
||||
`_root._enter_root_bindspace()` adapts this synchronous scope to
|
||||
the root actor's async lifecycle.
|
||||
|
||||
Only descriptors opened or duplicated by this context are used
|
||||
for validation, entry and restoration. Since `setns()` is
|
||||
thread-local, this synchronous context performs no checkpoints
|
||||
around either transition.
|
||||
|
||||
'''
|
||||
if bindspace is None:
|
||||
yield None
|
||||
return
|
||||
|
||||
if sys.platform != 'linux':
|
||||
raise RuntimeError(
|
||||
'Network namespace entry is Linux-only!'
|
||||
)
|
||||
|
||||
namespace_fd: int|None = bindspace.namespace_fd
|
||||
if namespace_fd is None:
|
||||
raise ValueError(
|
||||
'`bindspace.namespace_fd` must be a live netns FD for '
|
||||
'root actor entry!'
|
||||
)
|
||||
if (
|
||||
type(namespace_fd) is not int
|
||||
or
|
||||
namespace_fd < 0
|
||||
):
|
||||
raise ValueError(
|
||||
'`bindspace.namespace_fd` must be a live '
|
||||
'non-negative FD!'
|
||||
)
|
||||
|
||||
# Borrow `Bindspace.namespace_fd`; duplicate it so this context
|
||||
# owns target cleanup and cannot close the caller's capability.
|
||||
# Nested FD scopes aggregate later close failures as notes on the
|
||||
# first body, restoration or cleanup error.
|
||||
with dup_fd(namespace_fd) as tgt_fd:
|
||||
tgt_stat: os.stat_result = os.fstat(tgt_fd)
|
||||
tgt_inode: int = bindspace.ref.inode
|
||||
if tgt_stat.st_ino != tgt_inode:
|
||||
raise ValueError(
|
||||
f'Target namespace FD inode '
|
||||
f'{tgt_stat.st_ino} does '
|
||||
f'not match bindspace inode {tgt_inode}!'
|
||||
)
|
||||
|
||||
# Capture the calling thread's current netns before any
|
||||
# transition. It need not be the process's initial netns. This
|
||||
# context owns the snapshot FD even when no transition is
|
||||
# needed, so keep it live through restoration and always close
|
||||
# it afterward.
|
||||
orig_fd = os.open(
|
||||
_SELF_NETNS,
|
||||
os.O_RDONLY | os.O_CLOEXEC,
|
||||
)
|
||||
with close_fd(orig_fd, 'original'):
|
||||
orig_stat: os.stat_result = os.fstat(orig_fd)
|
||||
orig_inode: int = orig_stat.st_ino
|
||||
restore_needed: bool = (
|
||||
tgt_stat.st_dev != orig_stat.st_dev
|
||||
or
|
||||
tgt_inode != orig_inode
|
||||
)
|
||||
try:
|
||||
if restore_needed:
|
||||
enter_netns(
|
||||
tgt_fd,
|
||||
tgt_inode,
|
||||
)
|
||||
|
||||
yield tgt_inode
|
||||
|
||||
except BaseException as primary_error:
|
||||
if restore_needed:
|
||||
try:
|
||||
enter_netns(
|
||||
orig_fd,
|
||||
orig_inode,
|
||||
)
|
||||
except BaseException as restore_error:
|
||||
primary_error.add_note(
|
||||
'Also failed to restore the original '
|
||||
'network namespace: '
|
||||
f'{restore_error!r}'
|
||||
)
|
||||
raise primary_error
|
||||
|
||||
if restore_needed:
|
||||
try:
|
||||
enter_netns(
|
||||
orig_fd,
|
||||
orig_inode,
|
||||
)
|
||||
except BaseException as restore_error:
|
||||
restore_error.add_note(
|
||||
'Failed to restore the original network '
|
||||
'namespace during root netns cleanup.'
|
||||
)
|
||||
raise restore_error
|
||||
|
|
|
|||
|
|
@ -50,7 +50,7 @@ from tractor.msg import types as msgtypes
|
|||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from tractor.discovery._bindspace import Bindspace
|
||||
from tractor.net._bindspace import Bindspace
|
||||
from tractor.ipc import (
|
||||
_server,
|
||||
Channel,
|
||||
|
|
|
|||
|
|
@ -57,7 +57,7 @@ from ._spawn import (
|
|||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from tractor.discovery._bindspace import Bindspace
|
||||
from tractor.net._bindspace import Bindspace
|
||||
from tractor.ipc import (
|
||||
_server,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -61,7 +61,7 @@ from ..runtime._supervise import (
|
|||
|
||||
if TYPE_CHECKING:
|
||||
from ..discovery._addr import UnwrappedAddress
|
||||
from ..discovery._bindspace import Bindspace
|
||||
from ..net._bindspace import Bindspace
|
||||
from ..runtime._portal import Portal
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue