ib_2025_updates: to make it not suck despite edwault's epic exit #59
Loading…
Reference in New Issue
There is no content yet.
Delete Branch "ib_2025_updates"
Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?
Mostly fixes from having not run this backend for a while (over the summer of 2025) when a bit of my infra “died” and I had to re-hack various deps, error handling, and order ctl jigging after #44, #45 and #52.
Summary of “improvements”
stopped relaying “warning” API errors-msgs through the EMS since most are benign and shouldn’t cause order-entry-session-cancellation (which can leave actual stale live orders with ib’s EMS and
piker.clearingthinking they’re rejected/cancelled).5342f16adds legit venue “hours checking”,type-annots and code styling modernizations.
43e505e8solves an inter-API-connector issue with solvingasyncio-sidepush()exc propagation,porting to modern
tractor.to_asyncioAPI updates, namely:007e7e3switches to the new VNC client lib,pyvnc, sinceasyncvncappears to be un-maintained and i already could never get the password submission to work with our container, the new version of which enforces it,try_xdo_manual()to work around this in8283a1d4.4cdef4dadds support for per-ib.vnc_addrsvnc passwords in thebrokers.tomlconfig.Set.bs_mktidon all IB position-msg emissions so that order-tracking in.ui.order_modenever is erroneous..BrokerPositionfirst!Bump
docker/ib/README.rstafter moving to the new GH project/image.a63c7b1also does for container docs and compose file.db34999forbrokers.toml.rando
.brokers.coredoc str updates.That is to use the new `tractor.msg.types.Aid` struct to pull the `brokerd` info from the `tractor.Channel.aid: Aid` attr as well as more generally handling the new `Channel.raddr.proto_key: str` and no longer assuming a TCP IPC transport; this per the recent `tractor.ipc` subsys which adds multi-IPC-transports! Downstream tweaks to match, - use an "opt-in" field set to display in the `brokerd` info pane in `.ui._feedstatus.mk_feed_label()`. |_ also add some todos and drop some seemingly unneeded form sizing calcs? - tweak `.ui._label` to allow not using markdown, though ended up not doing that since it looked too plain..Using `tractor.trionics.collapse_eg()` as needed and doing some renames, in similar style as elsewhere: - `pcs` -> `rent_cs`, - `n` -> `tn` for nursery handles, Also, - tweak the `._reconnect_forever()` while loop to use the (also) `trio`-internal `mc_state: trio._channel.MemoryChannelState = snd._state` instead of `snd._close` to poll for open send/receive consumer task counts since, 1. it seems more reliable then using the `snd._closed`, 2. there's no other way to access the info.. afaik? - handle `ConnectionRejected` explicitly alongside handshake-errs as a retry case. - add a base-exc handler which `.exception()` reports the reconnect attempt failure explicitly. - drop some lingering `Optional` usage.Such that we can avoid other (pretty unreliable) "alternative" checks to determine whether a real-time quote should be waited on or (when venue is closed) we should just signal that historical backfilling can commence immediately. This has been a todo for a very long time and it turned out to be much easier to accomplish than anticipated.. Deats, - add a new `is_current_time_in_range()` dt range checker to predicate whether an input range contains `datetime.now(start_dt.tzinfo)`. - in `.ib.feed.stream_quotes()` add a `venue_is_open: bool` which uses all of the new ^^ to determine whether to branch for the short-circuit-and-do-history-now-case or the std real-time-quotes should-be-awaited-since-venue-is-open, case; drop all the old hacks trying to workaround not figuring that venue state stuff.. Other, - also add a gpt5 composed parser to `._util` for the `ib_insync.ContractDetails.tradingHours: str` for before i realized there was a `.tradingSessions` property XD - in `.ib_feed`, * add various EG-collapsings per recent tractor/trio updates. * better logging / exc-handling around ticker quote pushes. * stop clearing `Ticker.ticks` each quote iteration; not sure if this is needed/correct tho? * add masked `Ticker.ticks` poll loop that logs. - fix some `str.format()` usage in `._util.try_xdo_manual()` NOTE, resolved conflicts on `piker/brokers/ib/feed.py` due to rebasing onto up stream `brokers_refinery` commit,d809c797`.brokers.ib.feed`: better `tractor.to_asyncio` typing and var naming throughoutSuch that if/when the `push()` ticker callback (closure) errors internally, we actually eventually bubble the error out-and-up from the `asyncio.Task` and from there out the `.to_asyncio.open_channel_from()` to the parent `trio.Task`.. It ended up being much more subtle to solve then i would have liked thanks to, - whatever `Ticker.updateEvent.connect()` does behind the scenes in terms of (clearly) swallowing with only log reporting any exc raised in the registered callback (in our case `push()`), - `asyncio.Task.set_excepion()` never working and instead needing to resort to `Task.cancel()`, catching `CancelledError` and re-raising the stashed `maybe_exc` from `push()` when set.. Further this ports `.to_asyncio.open_channel_from()` usage to use the new `chan: tractor.to_asyncio.LinkedTaskChannel` fn-sig API, namely for `_setup_quote_stream()` task. Requires the latest `tractor` updates to the inter-eventloop-chan iface providing a `.set_nowait()` and `.get()` for the `asyncio`-side. Impl deats within `_setup_quote_stream()`, - implement `push()` error-bubbling by adding a `maybe_exc` which can be set by that callback itself or by its registering task; when set it is both, * reported on by the `teardown()` cb, * re-raised by the terminated (via `.cancel()`) `asyncio.Task` after woken from its sleep, aka "cancelled" (since that's apparently one of the only options.. see big rant further todo comments). - add explicit error-tolerance-tuning via a `handler_tries: int` counter and `tries_before_raise: int` limit such that we only bubble a `push()` raised exc once enough tries have consecutively failed. - as mentioned, use the new `chan` fn-sig support and thus the new method API for `asyncio` -> `trio` comms. - a big TODO XXX around the need to use a better sys for terminating `asyncio.Task`s whether it's by delegating to some `.to_asyncio` internals after a factor-out OR by potentially going full bore `anyio` throughout `.to_asyncio`'s impl in general.. - mk `teardown()` use appropriate `log.<level>()`s based on outcome. Surroundingly, - add a ton of doc-strings to mod fns previously missing them. - improved / added-new comments to `wait_on_data_reset()` internals and anything changed per ^above. NOTE, resolved conflicts on `piker/brokers/ib/feed.py` due to `brokers_refinery` commit:d809c797`.brokers.ib.feed`: better `tractor.to_asyncio` typing and var naming throughout!Such that the `brokers.toml` can contain any of the following <port> = dict|tuple styles, ```toml [ib.vnc_addrs] 4002 = {host = 'localhost', port = 5900, pw = 'doggy'} # host, port, pw 4002 = {host = 'localhost', port = 5900} # host, port, pw 4002 = ['localhost', 5900] # host, port, pw ``` With the first line demonstrating a vnc-server password (as normally set via a `.env` file in the `dockering/ib/` subdir) with the `pw =` field. This obviously removes the hardcoded `'doggy'` password from prior. Impl details in `.brokers.ib._util`: - pass the `ib.api.Client` down into `vnc_click_hack()` doing all config reading within and removing host, port unpacking in the callingn `data_reset_hack()`. - also pass the client `try_xdo_manual()` and comment (with plans to remove) the recently added localhost-only fallback section since we now have a fully working py vnc client again with `pyvnc` B) - in `vnc_click_hack()` match for all the possible config line styles and, * pass any `pw` field to `pyvncVNCConfig`, * continue matching host, port without password, * fallthrough to raising a val-err when neither ^ match.1674f99214todb34999553db34999553to4cb240adbe4cb240adbeto55116eea01