The topology flip: all data-feed consumers now route to the
new `datad.<broker>` sibling daemon; `brokerd` becomes
trading-only and is ONLY ever booted lazily by `emsd`'s
`open_brokerd_dialog()` (see prior commit). Chart-only and
paper sessions run with zero (live, credentialed) `brokerd`
procs B)
Deats,
- `open_feed()` -> `maybe_spawn_datad()` (NB: imported
relative-direct from `._daemon` to dodge a partial-init
cycle via `piker.service`); flip the `open_feed_bus()`
actor-name assert to `'datad'`; comment sweep.
- slim `_setup_persistent_brokerd()` to a trading-only
fixture: console logging + pinned-open ctx; the feed-bus
alloc moves to `_setup_persistent_datad()` and backend
`open_trade_dialog()` ctxs own their own task trees.
(the `piker ledger` ad-hoc actor enters this same slimmed
fixture - exactly what it needs.)
- repoint data-flavoured spawn sites to `maybe_spawn_datad`:
`.ui._app` symbol-search (+ rename
`install_brokerd_search` -> `install_datad_search`),
`.brokers.core.symbol_search()`, `.brokers.cli`
`brokercheck`/`record`, legacy kivy `.ui.cli` +
`option_chain`'s `wait_for_actor()`.
- invert `tests.test_services` expectations: feed/EMS-paper
flows must spawn `datad.kraken` and `paperboi.kraken`
with an explicit negative assert that NO `brokerd.kraken`
service task exists.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Prompt-IO: ai/prompt-io/claude/20260610T171259Z_59d5d9a6_prompt_io.md
Change most sub-modules to use `get_logger(name=__name__)` for
per-leaf-module `log` instances vs previous subpkg-level/shared refs.
Primary changes,
- import `get_[console_]logger()` from top-level `piker.log` across leaf
mods.
- change any `<subsys>._util.log` logger-instances as well (though this
approach should no longer be used since it masks the endpoint module's
emissions.
Also,
- add a defaulted `loglevel: str` param to all `open_trade_dialog()`
endpoints, anticipating it being passed in by `.clearing`-engine.
- call `get_console_log(level=loglevel, name=__name__)` in each trade
dialog ep to enable per-`brokerd`-backend console writing.
- drop `get_logger` from `.brokers.__all__` exports
- fix type annotations: `str|None` vs `str | None`
- add TODOs for,
* comments in `._util` about multi-subsys logging
* `.accounting.__init__` about console log setup
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
As part of bringing the brokerd agnostic APIs up to date and modernizing
wrapping CLIs, this adds a new sub-cmd to allow more or less directly
calling the `.get_mkt_info()` broker mod endpoint and dumping the both
the backend specific `Pair`-ish and `.accounting.MktPair` normalized
version to console.
Deatz:
- make the click config's `brokermods` entry a `dict`
- make `.brokers.core.mkt_info()` strip the broker name part from the
input fqme before calling the backend.
For easy testing of questrade historical data from cli.
Re-org the common cli components into a new package to avoid having all
commands defined in a top-level module.
Makes it easy to request all the option contracts for a particular symbol.
Also, let `option_chain()` accept a `date` arg which can be used to only
retrieve quotes for a single expiry date (much faster then getting all
of them).
Drop all channel/connection handling from the core and break up all the
start up steps into compact and useful functions. The main difference is
the daemon now only needs to worry about spawning per broker streaming
tasks and handling symbol list subscription requests.
When a client loses a connection it will currently need to re-subscribe
for symbols and receive a symbol data summary as a first quote response.
Only run the provided coroutine on reconnect and call the kwarg
`on_reconnect`. The client consuming code is entirely expected at this
point to know how the symbol registration protocol works.
Event if a broker client is already spawned new clients should still
receive a detailed symbol data packet as the first response. Avoid
exposing the new client's queue to the broker (i.e. subscribing it for
quotes) until after first pushing this packet with all bad symbols
filtered out.
Oh boy where to start.
- Handle broken streams in the `StreamQueue` gracefully; terminate the
async generator.
- When a stream queue connection is unwritable discard its subscriptions
inside the quoter task
- If all subscriptions are discarded for a broker then tear down its
quoter task
- Use listener parent nursery for spawning quoter tasks
- Make broker subs data structures global/shared between conn
handler tasks
- Register the `tickers2qs` entry *after* instantiating broker client(s)
(avoids race condition when mulitple client connections are coming
online simultaneously)
- Push smoke quotes to every client not just the first that connects
- Track quoter tasks in a cross-task set
- Handle unsubscriptions more correctly
In order to start working toward a HA distributed
architecture make apps use a `Client` type to talk to daemons.
The `Client` provides fault-tolerance for connection failures such
that the app will continue running until a connection to the original
service can be made or the process is killed. This will make it easier
to simply spawn up new daemon child processes when faults are detected.
Filter out bad symbols by processing an initial batch quote and
pushing to the subscribing client before spawning a quoter task.
This also avoids exposing the quoter task to anything but the
broker module and a `get_quotes()` routine.
Allow client connections to subscribe for quote streams from specific
brokers and spawn broker-client quoter tasks on-demand according
to client connection demands. Support multiple subscribers to a
single daemon process.
Async generators are faster and less code. Handle segmented packets
which can happen during periods of high quote volume. Move per-broker
rate limit logic into daemon task.
Gud Boi
Tyler Goodlet
Tyler Goodlet