Compare commits

...

11 Commits

Author SHA1 Message Date
Tyler Goodlet 38f7212396 TOSQASH 22e62ed: with-stmt-ws-removal 2026-02-19 19:17:03 -05:00
Tyler Goodlet 34315b77d1 WIP, expand pldrx suite for tagged-multi-msgs
Nowhere near ready yet since this generates many test-body-logic
un-handled true-positives which currently fail, but it's a first draft
for the general case set. To start, includes a greater-than-one-`Msg` (a
strict top level tagged-union-of-structs) pld spec alongside
a `AnyFieldMsg`-workaround struct-msg for packing all other builtin
non-`Struct`/`Any` python types alongside the other explicit msgs.
2026-02-19 19:17:03 -05:00
Tyler Goodlet 20268ea345 Ext-types test suite clean out
Removing the now masked-for-a-while unit test remnants for
`test_limit_msgspec()` (and its helper `chk_pld_type()`) since these cases
are now covered in the `test_pldrx_limiting` suite at an e2e
IPC-system-spanning level.

Note that the contents of the `chk_pld_type()` might be useful in the
future once we start setting/allowing semantics for various "phases of
IPC with matching msgspecs", but that's a little ways off rn and this
commit can always be looked up, also iirc most of the details were
already somewhat out of date and causing suite failure.
2026-02-19 19:17:03 -05:00
Tyler Goodlet 0db6eaab5b Add todo-note for non-strict `msgspec` decode-mode? 2026-02-19 19:17:03 -05:00
Tyler Goodlet 57d48aca1e Set `hide_tb` at top of `.limit_plds()` body 2026-02-19 19:17:03 -05:00
Tyler Goodlet 3fe936e99a Always merge input `spec` with any `ext_types`
That is, in `.msg._codec.mk_dec()` to ensure we actually still respect
the provided `spec: Union[Type[Struct]]|Type|None` alongside any
"custom" extension-types expected to be `dec_hook()` pre-processed.

Notes,
- previously when `dec_hook()` was provided we were merging with
  a `msgspec.Raw` instead of `spec` which **is entirely wrong**; it was
  likely leftover code from the sloppy/naive first draft of extension
  types support.
- notice the `spec: Union[Type[Struct]]|Type|None` type annotation (and
  it appears as though a `test_ext_types_msgspec` suite actually passes
  the value `spec=None` fyi) with a value of `None` to imply merging as
  `Union[ext_types]|None` (or equivalently a `Optional[Union]`), due
  to the incorrect `Raw`-default usage this was actually being ignored..
  -> this case has now been clarified via comment in the fn-signature.
2026-02-19 19:17:03 -05:00
Tyler Goodlet dda8d7d01a Dynamically set `pld_spec` for `test_basic_payload_spec()
Such that we can parametrize the `@context(pld_spec)` endpoint setting
using `pytest` and of course enable testing more then just the lone
`maybe_msg_spec` case. The implementation was a bit tricky because
subactors import any `enable_modules` just after subproc spawn, so
there's no easy way to indicate from the parent should should be passed
to the `@context()` decorator since it's already resolved by the time an
IPC is established. Thus the bulk of this patch is implementing
a pre-ctx which monkey-patches the (test) `child()`-ep-defining-module
before running test logic.

Impl deats,
- drop `maybe_msg_spec` global instead providing the same value via
  a new `pld_spec: Union[Type]` parametrized input to the test suite.
- add a `decorate_child_ep()` helper which (re-)decorates the
  mod-defined `child()` IPC-context endpoint with the provided `pld_spec`.
- add a new "pre IPC context" endpoint: `set_chld_pldspec()` which can
  be opened (from another actor) just prior to opening the `child()` ep
  and it will decorate the latter (using `decorate_child_ep()`)
  presuming a `.msg._exts.enc_type_union()` generated  `pld_spec_strs`
  is provided.
- actually open the `set_chld_pldspec()` as a `deco_ctx` rom the
  root-actor and ensure we cancel it on block teardown in non-raising
  cases.
2026-02-19 19:17:03 -05:00
Tyler Goodlet 41d624e1ff Add `types`-mod to `.msg._exts.dec_type_union()`
Such that decoded output equivalent to `str|None` can actually be
unpacked from a `type_names = ['str', 'NoneType]` without just
ignoring the null-type entry.. Previously, the loop would fall through
silently ignoring the `None` -> `NoneType` string representation mapped
by `.enc_type_union()` and the output union would be incorrect.

Deats,
- include the stdlib's `types` in the lookup loop, obvi changing the
  output var's name to `_types` to not collide.
- add output checking versus input `type_names` such that we raise
  a value-error with a case specific `report: str` when either,
  * the output `_types: list[Type]` is empty,
  * the `len(_types) != len(type_names)`.
2026-02-19 19:17:03 -05:00
Tyler Goodlet ae2279b8f2 Extend `.to_asyncio.LinkedTaskChannel` for aio side
With methods to comms similar to those that exist for the `trio` side,
- `.get()` which proxies verbatim to the `._to_aio: asyncio.Queue`,
- `.send_nowait()` which thin-wraps to `._to_trio: trio.MemorySendChannel`.

Obviously the more correct design is to break up the channel type into
a pair of handle types, one for each "side's" task in each event-loop,
that's hopefully coming shortly in a follow up patch B)

Also,
- fill in some missing doc strings, tweak some explanation comments and
  update todos.
- adjust the `test_aio_errors_and_channel_propagates_and_closes()` suite
  to use the new `chan` fn-sig-API with `.open_channel_from()` including
  the new methods for msg comms; ensures everything added here works e2e.
2026-02-19 19:17:03 -05:00
Tyler Goodlet 94c0ff2703 Explain the `infect_asyncio: bool` param to pass in RTE msg 2026-02-19 19:17:03 -05:00
Tyler Goodlet 0092d4d645 Toss in masked `.set_trace()` for unshielded `.pause()` debug 2026-02-19 19:17:03 -05:00
8 changed files with 304 additions and 275 deletions

View File

@ -454,7 +454,7 @@ async def send_back_values(
with ( with (
maybe_apply_codec(nsp_codec) as codec, maybe_apply_codec(nsp_codec) as codec,
limit_plds( limit_plds(
rent_pld_spec, spec=rent_pld_spec,
dec_hook=dec_nsp if add_hooks else None, dec_hook=dec_nsp if add_hooks else None,
ext_types=[NamespacePath] if add_hooks else None, ext_types=[NamespacePath] if add_hooks else None,
) as pld_dec, ) as pld_dec,
@ -665,7 +665,9 @@ def test_ext_types_over_ipc(
expect_codec=nsp_codec, expect_codec=nsp_codec,
enter_value=codec, enter_value=codec,
) )
rent_pld_spec_type_strs: list[str] = _exts.enc_type_union(pld_spec) rent_pld_spec_type_strs: list[str] = _exts.enc_type_union(
pld_spec
)
# XXX should raise an mte (`MsgTypeError`) # XXX should raise an mte (`MsgTypeError`)
# when `add_hooks == False` bc the input # when `add_hooks == False` bc the input
@ -695,7 +697,7 @@ def test_ext_types_over_ipc(
limit_plds( limit_plds(
pld_spec, pld_spec,
dec_hook=dec_nsp if add_hooks else None, dec_hook=dec_nsp if add_hooks else None,
ext_types=[NamespacePath] if add_hooks else None, ext_types=[NamespacePath] if add_hooks else None,
) as pld_dec, ) as pld_dec,
): ):
ctx_pld_dec: MsgDec = ctx._pld_rx._pld_dec ctx_pld_dec: MsgDec = ctx._pld_rx._pld_dec
@ -704,7 +706,7 @@ def test_ext_types_over_ipc(
# if ( # if (
# not add_hooks # not add_hooks
# and # and
# NamespacePath in # NamespacePath in
# ): # ):
# pytest.fail('ctx should fail to open without custom enc_hook!?') # pytest.fail('ctx should fail to open without custom enc_hook!?')
@ -743,204 +745,10 @@ def test_ext_types_over_ipc(
assert exc.boxed_type is TypeError assert exc.boxed_type is TypeError
# def chk_pld_type( # TODO: further SC-msg-specific verification that the overridden
# payload_spec: Type[Struct]|Any, # subtypes DO NOT have modified type-annots from original!
# pld: Any, # 'Start', .pld: FuncSpec
# 'StartAck', .pld: IpcCtxSpec
# expect_roundtrip: bool|None = None, # 'Stop', .pld: UNSEt
# 'Error', .pld: ErrorData
# ) -> bool: # def test_per_msg_payload_spec_limits():
# pld_val_type: Type = type(pld)
# # TODO: verify that the overridden subtypes
# # DO NOT have modified type-annots from original!
# # 'Start', .pld: FuncSpec
# # 'StartAck', .pld: IpcCtxSpec
# # 'Stop', .pld: UNSEt
# # 'Error', .pld: ErrorData
# codec: MsgCodec = mk_codec(
# # NOTE: this ONLY accepts `PayloadMsg.pld` fields of a specified
# # type union.
# ipc_pld_spec=payload_spec,
# )
# # make a one-off dec to compare with our `MsgCodec` instance
# # which does the below `mk_msg_spec()` call internally
# ipc_msg_spec: Union[Type[Struct]]
# msg_types: list[PayloadMsg[payload_spec]]
# (
# ipc_msg_spec,
# msg_types,
# ) = mk_msg_spec(
# payload_type_union=payload_spec,
# )
# _enc = msgpack.Encoder()
# _dec = msgpack.Decoder(
# type=ipc_msg_spec or Any, # like `PayloadMsg[Any]`
# )
# assert (
# payload_spec
# ==
# codec.pld_spec
# )
# # assert codec.dec == dec
# #
# # ^-XXX-^ not sure why these aren't "equal" but when cast
# # to `str` they seem to match ?? .. kk
# assert (
# str(ipc_msg_spec)
# ==
# str(codec.msg_spec)
# ==
# str(_dec.type)
# ==
# str(codec.dec.type)
# )
# # verify the boxed-type for all variable payload-type msgs.
# if not msg_types:
# breakpoint()
# roundtrip: bool|None = None
# pld_spec_msg_names: list[str] = [
# td.__name__ for td in _payload_msgs
# ]
# for typedef in msg_types:
# skip_runtime_msg: bool = typedef.__name__ not in pld_spec_msg_names
# if skip_runtime_msg:
# continue
# pld_field = structs.fields(typedef)[1]
# assert pld_field.type is payload_spec # TODO-^ does this need to work to get all subtypes to adhere?
# kwargs: dict[str, Any] = {
# 'cid': '666',
# 'pld': pld,
# }
# enc_msg: PayloadMsg = typedef(**kwargs)
# _wire_bytes: bytes = _enc.encode(enc_msg)
# wire_bytes: bytes = codec.enc.encode(enc_msg)
# assert _wire_bytes == wire_bytes
# ve: ValidationError|None = None
# try:
# dec_msg = codec.dec.decode(wire_bytes)
# _dec_msg = _dec.decode(wire_bytes)
# # decoded msg and thus payload should be exactly same!
# assert (roundtrip := (
# _dec_msg
# ==
# dec_msg
# ==
# enc_msg
# ))
# if (
# expect_roundtrip is not None
# and expect_roundtrip != roundtrip
# ):
# breakpoint()
# assert (
# pld
# ==
# dec_msg.pld
# ==
# enc_msg.pld
# )
# # assert (roundtrip := (_dec_msg == enc_msg))
# except ValidationError as _ve:
# ve = _ve
# roundtrip: bool = False
# if pld_val_type is payload_spec:
# raise ValueError(
# 'Got `ValidationError` despite type-var match!?\n'
# f'pld_val_type: {pld_val_type}\n'
# f'payload_type: {payload_spec}\n'
# ) from ve
# else:
# # ow we good cuz the pld spec mismatched.
# print(
# 'Got expected `ValidationError` since,\n'
# f'{pld_val_type} is not {payload_spec}\n'
# )
# else:
# if (
# payload_spec is not Any
# and
# pld_val_type is not payload_spec
# ):
# raise ValueError(
# 'DID NOT `ValidationError` despite expected type match!?\n'
# f'pld_val_type: {pld_val_type}\n'
# f'payload_type: {payload_spec}\n'
# )
# # full code decode should always be attempted!
# if roundtrip is None:
# breakpoint()
# return roundtrip
# ?TODO? maybe remove since covered in the newer `test_pldrx_limiting`
# via end-2-end testing of all this?
# -[ ] IOW do we really NEED this lowlevel unit testing?
#
# def test_limit_msgspec(
# debug_mode: bool,
# ):
# '''
# Internals unit testing to verify that type-limiting an IPC ctx's
# msg spec with `Pldrx.limit_plds()` results in various
# encapsulated `msgspec` object settings and state.
# '''
# async def main():
# async with tractor.open_root_actor(
# debug_mode=debug_mode,
# ):
# # ensure we can round-trip a boxing `PayloadMsg`
# assert chk_pld_type(
# payload_spec=Any,
# pld=None,
# expect_roundtrip=True,
# )
# # verify that a mis-typed payload value won't decode
# assert not chk_pld_type(
# payload_spec=int,
# pld='doggy',
# )
# # parametrize the boxed `.pld` type as a custom-struct
# # and ensure that parametrization propagates
# # to all payload-msg-spec-able subtypes!
# class CustomPayload(Struct):
# name: str
# value: Any
# assert not chk_pld_type(
# payload_spec=CustomPayload,
# pld='doggy',
# )
# assert chk_pld_type(
# payload_spec=CustomPayload,
# pld=CustomPayload(name='doggy', value='urmom')
# )
# # yah, we can `.pause_from_sync()` now!
# # breakpoint()
# trio.run(main)

View File

@ -732,15 +732,21 @@ def test_aio_errors_and_channel_propagates_and_closes(
async def aio_echo_server( async def aio_echo_server(
to_trio: trio.MemorySendChannel, chan: to_asyncio.LinkedTaskChannel,
from_trio: asyncio.Queue,
) -> None: ) -> None:
'''
An IPC-msg "echo server" with msgs received and relayed by
a parent `trio.Task` into a child `asyncio.Task`
and then repeated back to that local parent (`trio.Task`)
and sent again back to the original calling remote actor.
to_trio.send_nowait('start') '''
# same semantics as `trio.TaskStatus.started()`
chan.started_nowait('start')
while True: while True:
try: try:
msg = await from_trio.get() msg = await chan.get()
except to_asyncio.TrioTaskExited: except to_asyncio.TrioTaskExited:
print( print(
'breaking aio echo loop due to `trio` exit!' 'breaking aio echo loop due to `trio` exit!'
@ -748,7 +754,7 @@ async def aio_echo_server(
break break
# echo the msg back # echo the msg back
to_trio.send_nowait(msg) chan.send_nowait(msg)
# if we get the terminate sentinel # if we get the terminate sentinel
# break the echo loop # break the echo loop
@ -765,7 +771,10 @@ async def trio_to_aio_echo_server(
): ):
async with to_asyncio.open_channel_from( async with to_asyncio.open_channel_from(
aio_echo_server, aio_echo_server,
) as (first, chan): ) as (
first, # value from `chan.started_nowait()` above
chan,
):
assert first == 'start' assert first == 'start'
await ctx.started(first) await ctx.started(first)
@ -776,7 +785,8 @@ async def trio_to_aio_echo_server(
await chan.send(msg) await chan.send(msg)
out = await chan.receive() out = await chan.receive()
# echo back to parent actor-task
# echo back to parent-actor's remote parent-ctx-task!
await stream.send(out) await stream.send(out)
if out is None: if out is None:
@ -1090,14 +1100,12 @@ def test_sigint_closes_lifetime_stack(
# ?TODO asyncio.Task fn-deco? # ?TODO asyncio.Task fn-deco?
# -[ ] do sig checkingat import time like @context?
# -[ ] maybe name it @aio_task ??
# -[ ] chan: to_asyncio.InterloopChannel ?? # -[ ] chan: to_asyncio.InterloopChannel ??
# -[ ] do fn-sig checking at import time like @context?
# |_[ ] maybe name it @a(sync)io_task ??
# @asyncio_task <- not bad ??
async def raise_before_started( async def raise_before_started(
# from_trio: asyncio.Queue,
# to_trio: trio.abc.SendChannel,
chan: to_asyncio.LinkedTaskChannel, chan: to_asyncio.LinkedTaskChannel,
) -> None: ) -> None:
''' '''
`asyncio.Task` entry point which RTEs before calling `asyncio.Task` entry point which RTEs before calling

View File

@ -7,7 +7,15 @@ related settings around IPC contexts.
from contextlib import ( from contextlib import (
asynccontextmanager as acm, asynccontextmanager as acm,
) )
import sys
import types
from typing import (
Any,
Union,
Type,
)
import msgspec
from msgspec import ( from msgspec import (
Struct, Struct,
) )
@ -22,11 +30,10 @@ from tractor import (
Portal, Portal,
) )
from tractor.msg import ( from tractor.msg import (
_codec,
_ops as msgops, _ops as msgops,
Return, Return,
) _exts,
from tractor.msg import (
_codec,
) )
from tractor.msg.types import ( from tractor.msg.types import (
log, log,
@ -41,13 +48,22 @@ class PldMsg(
# case of these details? # case of these details?
# #
# https://jcristharif.com/msgspec/structs.html#tagged-unions # https://jcristharif.com/msgspec/structs.html#tagged-unions
# tag=True, tag=True,
# tag_field='msg_type', tag_field='msg_type',
): ):
field: str field: str
maybe_msg_spec = PldMsg|None class Msg1(PldMsg):
field: str
class Msg2(PldMsg):
field: int
class AnyFieldMsg(PldMsg):
field: Any
@acm @acm
@ -104,9 +120,15 @@ async def maybe_expect_raises(
) )
@tractor.context( # NOTE, this decorator is applied dynamically by both the root and
pld_spec=maybe_msg_spec, # 'sub' actor such that we can dynamically apply various cases from
) # a parametrized test.
#
# maybe_msg_spec = PldMsg|None
#
# @tractor.context(
# pld_spec=maybe_msg_spec,
# )
async def child( async def child(
ctx: Context, ctx: Context,
started_value: int|PldMsg|None, started_value: int|PldMsg|None,
@ -114,13 +136,13 @@ async def child(
validate_pld_spec: bool, validate_pld_spec: bool,
raise_on_started_mte: bool = True, raise_on_started_mte: bool = True,
pack_any_field: bool = False,
) -> None: ) -> None:
''' '''
Call ``Context.started()`` more then once (an error). Call ``Context.started()`` more then once (an error).
''' '''
expect_started_mte: bool = started_value == 10
# sanaity check that child RPC context is the current one # sanaity check that child RPC context is the current one
curr_ctx: Context = current_ipc_ctx() curr_ctx: Context = current_ipc_ctx()
assert ctx is curr_ctx assert ctx is curr_ctx
@ -128,6 +150,7 @@ async def child(
rx: msgops.PldRx = ctx._pld_rx rx: msgops.PldRx = ctx._pld_rx
curr_pldec: _codec.MsgDec = rx.pld_dec curr_pldec: _codec.MsgDec = rx.pld_dec
ctx_meta: dict = getattr( ctx_meta: dict = getattr(
child, child,
'_tractor_context_meta', '_tractor_context_meta',
@ -136,10 +159,28 @@ async def child(
if ctx_meta: if ctx_meta:
assert ( assert (
ctx_meta['pld_spec'] ctx_meta['pld_spec']
is curr_pldec.spec is
is curr_pldec.pld_spec curr_pldec.spec
is
curr_pldec.pld_spec
) )
pld_types: set[Type] = _codec.unpack_spec_types(
curr_pldec.pld_spec,
)
if (
AnyFieldMsg in pld_types
and
pack_any_field
):
started_value = AnyFieldMsg(field=started_value)
expect_started_mte: bool = (
started_value == 10
and
not pack_any_field
)
# 2 cases: hdndle send-side and recv-only validation # 2 cases: hdndle send-side and recv-only validation
# - when `raise_on_started_mte == True`, send validate # - when `raise_on_started_mte == True`, send validate
# - else, parent-recv-side only validation # - else, parent-recv-side only validation
@ -219,16 +260,65 @@ async def child(
# msg-type-error from this RPC task ;) # msg-type-error from this RPC task ;)
return return_value return return_value
def decorate_child_ep(
pld_spec: Union[Type],
) -> types.ModuleType:
'''
Apply parametrized pld_spec to ctx ep like,
@tractor.context(
pld_spec=maybe_msg_spec,
)(child)
'''
this_mod = sys.modules[__name__]
global child # a mod-fn defined above
assert this_mod.child is child
this_mod.child = tractor.context(
pld_spec=pld_spec,
)(child)
return this_mod
@tractor.context
async def set_chld_pldspec(
ctx: tractor.Context,
pld_spec_strs: list[str],
):
'''
Dynamically apply the `@context(pld_spec=pld_spec)` deco to the
current actor's in-mem instance of this test module.
Allows dynamically applying the "payload-spec" in both a parent
and child actor after spawn.
'''
this_mod = sys.modules[__name__]
pld_spec: list[str] = _exts.dec_type_union(
pld_spec_strs,
mods=[
this_mod,
msgspec.inspect,
],
)
decorate_child_ep(pld_spec)
await ctx.started()
await trio.sleep_forever()
@pytest.mark.parametrize( @pytest.mark.parametrize(
'return_value', 'return_value',
[ [
'yo', 'yo',
None, None,
Msg2(field=10),
AnyFieldMsg(field='yo'),
], ],
ids=[ ids=[
'return[invalid-"yo"]', 'return[invalid-"yo"]',
'return[valid-None]', 'return[maybe-valid-None]',
'return[maybe-valid-Msg2]',
'return[maybe-valid-any-packed-yo]',
], ],
) )
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -236,10 +326,14 @@ async def child(
[ [
10, 10,
PldMsg(field='yo'), PldMsg(field='yo'),
Msg1(field='yo'),
AnyFieldMsg(field=10),
], ],
ids=[ ids=[
'Started[invalid-10]', 'Started[invalid-10]',
'Started[valid-PldMsg]', 'Started[maybe-valid-PldMsg]',
'Started[maybe-valid-Msg1]',
'Started[maybe-valid-any-packed-10]',
], ],
) )
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -253,12 +347,31 @@ async def child(
'no-started-pld-validate', 'no-started-pld-validate',
], ],
) )
@pytest.mark.parametrize(
'pld_spec',
[
PldMsg|None,
# demo how to have strict msgs alongside all other supported
# py-types by embedding the any-types inside a shuttle msg.
Msg1|Msg2|AnyFieldMsg,
# XXX, will never work since Struct overrides dict.
# https://jcristharif.com/msgspec/usage.html#typed-decoding
# Msg1|Msg2|msgspec.inspect.AnyType,
],
ids=[
'maybe_PldMsg_spec',
'Msg1_or_Msg2_or_AnyFieldMsg_spec',
]
)
def test_basic_payload_spec( def test_basic_payload_spec(
debug_mode: bool, debug_mode: bool,
loglevel: str, loglevel: str,
return_value: str|None, return_value: str|None,
started_value: int|PldMsg, started_value: int|PldMsg,
pld_check_started_value: bool, pld_check_started_value: bool,
pld_spec: Union[Type],
): ):
''' '''
Validate the most basic `PldRx` msg-type-spec semantics around Validate the most basic `PldRx` msg-type-spec semantics around
@ -267,16 +380,33 @@ def test_basic_payload_spec(
pld-spec. pld-spec.
''' '''
invalid_return: bool = return_value == 'yo' pld_types: set[Type] = _codec.unpack_spec_types(pld_spec)
invalid_started: bool = started_value == 10 invalid_return: bool = (
return_value == 'yo'
)
invalid_started: bool = (
started_value == 10
)
# dynamically apply ep's pld-spec in 'root'.
decorate_child_ep(pld_spec)
assert (
child._tractor_context_meta['pld_spec'] == pld_spec
)
pld_spec_strs: list[str] = _exts.enc_type_union(
pld_spec,
)
assert len(pld_types) > 1
async def main(): async def main():
nonlocal pld_spec
async with tractor.open_nursery( async with tractor.open_nursery(
debug_mode=debug_mode, debug_mode=debug_mode,
loglevel=loglevel, loglevel=loglevel,
) as an: ) as an:
p: Portal = await an.start_actor( p: Portal = await an.start_actor(
'child', 'sub',
enable_modules=[__name__], enable_modules=[__name__],
) )
@ -286,9 +416,11 @@ def test_basic_payload_spec(
if invalid_started: if invalid_started:
msg_type_str: str = 'Started' msg_type_str: str = 'Started'
bad_value: int = 10 bad_value: int = 10
elif invalid_return: elif invalid_return:
msg_type_str: str = 'Return' msg_type_str: str = 'Return'
bad_value: str = 'yo' bad_value: str = 'yo'
else: else:
# XXX but should never be used below then.. # XXX but should never be used below then..
msg_type_str: str = '' msg_type_str: str = ''
@ -302,6 +434,7 @@ def test_basic_payload_spec(
invalid_started invalid_started
) else None ) else None
) )
async with ( async with (
maybe_expect_raises( maybe_expect_raises(
raises=should_raise, raises=should_raise,
@ -315,6 +448,11 @@ def test_basic_payload_spec(
# only for debug # only for debug
# post_mortem=True, # post_mortem=True,
), ),
p.open_context(
set_chld_pldspec,
pld_spec_strs=pld_spec_strs,
) as (deco_ctx, _),
p.open_context( p.open_context(
child, child,
return_value=return_value, return_value=return_value,
@ -325,12 +463,18 @@ def test_basic_payload_spec(
# now opened with 'child' sub # now opened with 'child' sub
assert current_ipc_ctx() is ctx assert current_ipc_ctx() is ctx
assert type(first) is PldMsg # assert type(first) is PldMsg
assert isinstance(first, PldMsg)
assert first.field == 'yo' assert first.field == 'yo'
try: try:
res: None|PldMsg = await ctx.result(hide_tb=False) res: None|PldMsg = await ctx.result(hide_tb=False)
assert res is None assert res == return_value
if res is None:
await tractor.pause()
if isinstance(res, PldMsg):
assert res.field == 10
except MsgTypeError as mte: except MsgTypeError as mte:
maybe_mte = mte maybe_mte = mte
if not invalid_return: if not invalid_return:
@ -356,6 +500,9 @@ def test_basic_payload_spec(
ctx.outcome ctx.outcome
) )
if should_raise is None:
await deco_ctx.cancel()
if should_raise is None: if should_raise is None:
assert maybe_mte is None assert maybe_mte is None

View File

@ -561,6 +561,9 @@ async def _pause(
return return
elif isinstance(pause_err, trio.Cancelled): elif isinstance(pause_err, trio.Cancelled):
__tracebackhide__: bool = False
# XXX, unmask to REPL it.
# mk_pdb().set_trace(frame=inspect.currentframe())
_repl_fail_report += ( _repl_fail_report += (
'You called `tractor.pause()` from an already cancelled scope!\n\n' 'You called `tractor.pause()` from an already cancelled scope!\n\n'
'Consider `await tractor.pause(shield=True)` to make it work B)\n' 'Consider `await tractor.pause(shield=True)` to make it work B)\n'

View File

@ -181,7 +181,11 @@ class MsgDec(Struct):
def mk_dec( def mk_dec(
spec: Union[Type[Struct]]|Type|None, spec: (
Union[Type[Struct]]
|Type # lone type
|None # implying `Union[*ext_types]|None`
),
# NOTE, required for ad-hoc type extensions to the underlying # NOTE, required for ad-hoc type extensions to the underlying
# serialization proto (which is default `msgpack`), # serialization proto (which is default `msgpack`),
@ -194,16 +198,18 @@ def mk_dec(
Create an IPC msg decoder, a slightly higher level wrapper around Create an IPC msg decoder, a slightly higher level wrapper around
a `msgspec.msgpack.Decoder` which provides, a `msgspec.msgpack.Decoder` which provides,
- easier introspection of the underlying type spec via - easier introspection of the underlying type spec via the
the `.spec` and `.spec_str` attrs, `.spec` and `.spec_str` attrs,
- `.hook` access to the `Decoder.dec_hook()`, - `.hook` access to the `Decoder.dec_hook()`,
- automatic custom extension-types decode support when - automatic custom extension-types decode support when
`dec_hook()` is provided such that any `PayloadMsg.pld` tagged `dec_hook()` is provided such that any `PayloadMsg.pld` tagged
as a type from from `ext_types` (presuming the `MsgCodec.encode()` also used as a type from from `ext_types` (presuming the
a `.enc_hook()`) is processed and constructed by a `PldRx` implicitily. `MsgCodec.encode()` also used a `.enc_hook()`) is processed and
constructed by a `PldRx` implicitily.
NOTE, as mentioned a `MsgDec` is normally used for `PayloadMsg.pld: PayloadT` field NOTE, as mentioned a `MsgDec` is normally used for
decoding inside an IPC-ctx-oriented `PldRx`. `PayloadMsg.pld: PayloadT` field decoding inside an
IPC-ctx-oriented `PldRx`.
''' '''
if ( if (
@ -248,12 +254,16 @@ def mk_dec(
# will work? kk B) # will work? kk B)
# #
# maybe_box_struct = mk_boxed_ext_struct(ext_types) # maybe_box_struct = mk_boxed_ext_struct(ext_types)
spec = Raw | Union[*ext_types]
spec = spec | Union[*ext_types]
return MsgDec( return MsgDec(
_dec=msgpack.Decoder( _dec=msgpack.Decoder(
type=spec, # like `MsgType[Any]` type=spec,
dec_hook=dec_hook, dec_hook=dec_hook,
# ?TODO, support it?
# https://jcristharif.com/msgspec/usage.html#strict-vs-lax-mode
# strict=False,
), ),
) )

View File

@ -33,9 +33,7 @@ converters,
|_ https://jcristharif.com/msgspec/changelog.html |_ https://jcristharif.com/msgspec/changelog.html
''' '''
from types import ( import types
ModuleType,
)
import typing import typing
from typing import ( from typing import (
Type, Type,
@ -44,35 +42,51 @@ from typing import (
def dec_type_union( def dec_type_union(
type_names: list[str], type_names: list[str],
mods: list[ModuleType] = [] mods: list[types.ModuleType] = []
) -> Type|Union[Type]: ) -> Type|Union[Type]:
''' '''
Look up types by name, compile into a list and then create and Look up types by name, compile into a list and then create and
return a `typing.Union` from the full set. return a `typing.Union` from the full set.
''' '''
# import importlib _types: list[Type] = []
types: list[Type] = []
for type_name in type_names: for type_name in type_names:
for mod in [ for mod in [
typing, typing,
# importlib.import_module(__name__), types,
] + mods: ] + mods:
if type_ref := getattr( if type_ref := getattr(
mod, mod,
type_name, type_name,
False, False,
): ):
types.append(type_ref) _types.append(type_ref)
break
# special case handling only.. report: str = ''
# ipc_pld_spec: Union[Type] = eval( if not _types:
# pld_spec_str, report: str = 'No type-instances could be resolved from `type_names` ??\n'
# {}, # globals
# {'typing': typing}, # locals
# )
return Union[*types] elif len(type_names) != len(_types):
report: str = (
f'Some type-instances could not be resolved from `type_names` ??\n'
f'_types: {_types!r}\n'
)
if report:
raise ValueError(
report
+
f'type_names: {type_names!r}\n'
)
if not _types:
raise ValueError(
f'No type-instance could be resolved from `type_names` ??\n'
f'type_names: {type_names!r}\n'
)
return Union[*_types]
def enc_type_union( def enc_type_union(

View File

@ -119,7 +119,7 @@ class PldRx(Struct):
def limit_plds( def limit_plds(
self, self,
spec: Union[Type[Struct]], spec: Union[Type[Struct]],
**dec_kwargs, **mk_dec_kwargs,
) -> MsgDec: ) -> MsgDec:
''' '''
@ -135,7 +135,7 @@ class PldRx(Struct):
orig_dec: MsgDec = self._pld_dec orig_dec: MsgDec = self._pld_dec
limit_dec: MsgDec = mk_dec( limit_dec: MsgDec = mk_dec(
spec=spec, spec=spec,
**dec_kwargs, **mk_dec_kwargs,
) )
try: try:
self._pld_dec = limit_dec self._pld_dec = limit_dec
@ -582,6 +582,7 @@ async def drain_to_final_msg(
even after ctx closure and the `.open_context()` block exit. even after ctx closure and the `.open_context()` block exit.
''' '''
__tracebackhide__: bool = hide_tb
raise_overrun: bool = not ctx._allow_overruns raise_overrun: bool = not ctx._allow_overruns
parent_never_opened_stream: bool = ctx._stream is None parent_never_opened_stream: bool = ctx._stream is None
@ -834,7 +835,8 @@ async def drain_to_final_msg(
f'{ctx.outcome}\n' f'{ctx.outcome}\n'
) )
__tracebackhide__: bool = hide_tb # ?TODO? why was this here and not above?
# __tracebackhide__: bool = hide_tb
return ( return (
result_msg, result_msg,
pre_result_drained, pre_result_drained,

View File

@ -94,10 +94,14 @@ else:
QueueShutDown = False QueueShutDown = False
# TODO, generally speaking we can generalize this abstraction, a "SC linked # TODO, generally speaking we can generalize this abstraction as,
# parent->child task pair", as the same "supervision scope primitive" #
# **that is** our `._context.Context` with the only difference being # > A "SC linked, inter-event-loop" channel for comms between
# in how the tasks conduct msg-passing comms. # > a `parent: trio.Task` -> `child: asyncio.Task` pair.
#
# It is **very similar** in terms of its operation as a "supervision
# scope primitive" to that of our `._context.Context` with the only
# difference being in how the tasks conduct msg-passing comms.
# #
# For `LinkedTaskChannel` we are passing the equivalent of (once you # For `LinkedTaskChannel` we are passing the equivalent of (once you
# include all the recently added `._trio/aio_to_raise` # include all the recently added `._trio/aio_to_raise`
@ -122,6 +126,7 @@ class LinkedTaskChannel(
task scheduled in the host loop. task scheduled in the host loop.
''' '''
# ?TODO, rename as `._aio_q` since it's 2-way?
_to_aio: asyncio.Queue _to_aio: asyncio.Queue
_from_aio: trio.MemoryReceiveChannel _from_aio: trio.MemoryReceiveChannel
@ -235,9 +240,11 @@ class LinkedTaskChannel(
# #
async def receive(self) -> Any: async def receive(self) -> Any:
''' '''
Receive a value from the paired `asyncio.Task` with Receive a value `trio.Task` <- `asyncio.Task`.
Note the tasks in each loop are "SC linked" as a pair with
exception/cancel handling to teardown both sides on any exception/cancel handling to teardown both sides on any
unexpected error. unexpected error or cancellation.
''' '''
try: try:
@ -261,15 +268,42 @@ class LinkedTaskChannel(
): ):
raise err raise err
async def get(self) -> Any:
'''
Receive a value `asyncio.Task` <- `trio.Task`.
This is equiv to `await self._from_trio.get()`.
'''
return await self._to_aio.get()
async def send(self, item: Any) -> None: async def send(self, item: Any) -> None:
''' '''
Send a value through to the asyncio task presuming Send a value through `trio.Task` -> `asyncio.Task`
it defines a ``from_trio`` argument, if it does not presuming
it defines a `from_trio` argument or makes calls
to `chan.get()` , if it does not
this method will raise an error. this method will raise an error.
''' '''
self._to_aio.put_nowait(item) self._to_aio.put_nowait(item)
# TODO? could we only compile-in this method on an instance
# handed to the `asyncio`-side, i.e. the fn invoked with
# `.open_channel_from()`.
def send_nowait(
self,
item: Any,
) -> None:
'''
Send a value through FROM the `asyncio.Task` to
the `trio.Task` NON-BLOCKING.
This is equiv to `self._to_trio.send_nowait()`.
'''
self._to_trio.send_nowait(item)
# TODO? needed? # TODO? needed?
# async def wait_aio_complete(self) -> None: # async def wait_aio_complete(self) -> None:
# await self._aio_task_complete.wait() # await self._aio_task_complete.wait()
@ -337,9 +371,12 @@ def _run_asyncio_task(
''' '''
__tracebackhide__: bool = hide_tb __tracebackhide__: bool = hide_tb
if not tractor.current_actor().is_infected_aio(): if not (actor := tractor.current_actor()).is_infected_aio():
raise RuntimeError( raise RuntimeError(
"`infect_asyncio` mode is not enabled!?" f'`infect_asyncio: bool` mode is not enabled ??\n'
f'Ensure you pass `ActorNursery.start_actor(infect_asyncio=True)`\n'
f'\n'
f'{actor}\n'
) )
# ITC (inter task comms), these channel/queue names are mostly from # ITC (inter task comms), these channel/queue names are mostly from
@ -1666,7 +1703,7 @@ def run_as_asyncio_guest(
# asyncio.CancelledError, # asyncio.CancelledError,
# ^^XXX `.shield()` call above prevents this?? # ^^XXX `.shield()` call above prevents this??
)as state_err: ) as state_err:
# XXX be super dupere noisy about abandonment issues! # XXX be super dupere noisy about abandonment issues!
aio_task: asyncio.Task = asyncio.current_task() aio_task: asyncio.Task = asyncio.current_task()