Extend the `col_sym_key` asset-type check in `start_backfill()`
to also exclude crypto-denominated futures (where `src` is
`'crypto_currency'` and `dst` is `'future'`) from the
`without_src=True` fqme path.
Also in `.brokers.binance` backend (it being the guilty culprit in the
discovery of this bug; and why i touched styling this code),
- reformat `make_sub()` fn sig to multiline style in
`.binance.feed`.
- add backtick around `dict` in `make_sub()` docstring.
- reformat `or` conditionals to multiline style in
`.binance.feed.get_mkt_info()`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Replace the debug breakpoint with a warning-log when a single-bar
null-segment is detected in `get_null_segs()`. This lets the gap
analysis continue while still alerting about the anomaly.
Deats,
- extract the 3-bar window (before, null, after) and calculate
a `gap: pendulum.Interval` for the warning msg.
- comment-out the old breakpoint block for optional debugging as needed.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Introduce `AnnotCtl.add_batch()` and `serve_rc_annots()` batch
handler to submit 1000s of gaps in single IPC msg instead of
per-annot round-trips. Server builds `GapAnnotations` from specs
and handles vectorized timestamp-to-index lookups.
Deats,
- add `'cmd': 'batch'` handler in `serve_rc_annots()`
- vectorized timestamp lookup via `np.searchsorted()` + masking
- build `gap_specs: list[dict]` from rect+arrow specs client-side
- create single `GapAnnotations` item for all gaps server-side
- handle `GapAnnotations.reposition()` in redraw handler
- add profiling to batch path for perf measurement
- support optional individual arrows for A/B comparison
Also,
- refactor `markup_gaps()` to collect specs + single batch call
- add `no_qt_updates()` context mgr for batch render ops
- add profiling to annotation teardown path
- add `GapAnnotations` case to `rm_annot()` match block
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
For a ~1000x perf gain says ol' claudy, our boi who wrote this entire
patch! Bo
Introduce `GapAnnotations` in `.ui._annotate` for batch-rendering gap
rects/arrows instead of individual `QGraphicsItem` instances. Uses
upstream's `pyqtgraph.Qt.internals.PrimitiveArray` for rects and
a `QPainterPath` for arrows. This API-replicates our prior annotator's
in view shape-graphics but now using (what we're dubbing)
"single-array-multiple-graphics" tech much like our `.ui._curve`
extensions to `pg` B)
Impl deats,
- batch draw ~1000 gaps in single paint call vs 1000 items
- arrows render in scene coords to maintain pixel size on zoom
- add vectorized timestamp-to-index lookup for repositioning
- cache bounding rect, rebuild on `reposition()` calls
- match `SelectRect` + `ArrowItem` visual style/colors
- skip reposition when timeframe doesn't match gap's period
Other,
- fix typo in `LevelMarker` docstring: "graphich" -> "graphic"
- reflow docstring in `qgo_draw_markers()` to 67 char limit
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Log shm file name and detected period before null segment
processing to aid debugging.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Replace hand-rolled `click`-based platform branching using
the much saner `platformdirs.user_config_path()`.
Deats,
- remove Windows/macOS/Unix `if/elif` platform dispatch
(~25 lines) in favour of single `user_config_path()` call.
- move `_posixify()` inside `force_posix` branch since it's
only used there.
- add `log.info()` reporting platform name and resolved dirs.
Also,
- drop now unneeded `sys` import.
- reformat `assert` in `repodir()` to multiline style.
- convert docstring from `r"""..."""` to `'''...'''` style.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add TAB + ENTER key presses after the `Ctrl+Alt+<key>` hotkey
combo to auto-confirm the "simulate a reset?" dialog that IB
gateway sometimes shows.
Deats,
- press `ISO_Enter` before click to dismiss any prior active
dialog window.
- add post-hotkey loop sending `Tab` then `KP_Enter` with
`asyncio.sleep()` delays to handle the confirmation dialog.
- add `asyncio` import.
Also,
- capture VNC connect error as `vnc_err` and log it instead of
falling through to `try_xdo_manual()`.
- comment-out `try_xdo_manual()` fallback in VNC error path.
- reformat `client.press()` call to multiline style.
- reformat `RuntimeError` raise to multiline style with `!r`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Refactor `open_symbol_search()` to use `partial()` for nursery task
spawning and add detailed query->results logging via `ppfmt()`.
Deats,
- change `extend_results()` to accept `target` callable +
`pattern` + `**kwargs` and invoke inside, instead of receiving
a pre-called awaitable; use `partial()` to pass args.
- add `ppfmt()` formatted logging of search query params and
results including client class + method repr.
- change `print()` -> `log.exception()` for `Lagged` overrun.
- bump `upto=5` -> `upto=10` for `search_symbols()` call.
Also for styling,
- add type some missing type annots.
- add multiline style to `or` conditionals in pattern check.
- reformat log msgs to multiline style throughout.
- use `ppfmt()` for fuzzy match debug log.
- rename nursery `sn` -> `tn`.
- add TODO comment about `assert 0` hang.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add `isinstance()` dispatch for the `'error'` event case in
`deliver_trade_events()` to handle `ib_async` sometimes emitting plain
`str` error items instead of the previously expected `dict`.
Deats,
- add `isinstance(err, dict)` branch for the standard case with
`error_code`, `reason`, and `reqid` fields.
- add `isinstance(err, str)` branch to parse error strings of the
form `'[code 104] connection failed'` into `code` and `reason`.
- set `reqid: str = '<unknown>'` for string-form errors since
there's no request ID available.
- update `err` type annot to `dict|str`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add guards for empty-array and zero-bar-diff cases in the TSP backfill
loops to avoid crashes and allow graceful loop termination.
In `maybe_fill_null_segments()`,
- add `array.size == 0` guard in `maybe_fill_null_segments()` to detect
valid (venue closure) gaps from the backend; add a warning + bp
+ break for this case.
- add TODO that we should likely be filling nulls with the close price
for the gap's duration.
In `start_backfill()`,
- expand the "0 bars after diff" warning msg with
`backfill_until_dt` and `end_dt_param` context.
- mask the `await tractor.pause()` and add a `break` to avoid blocking
the backfill loop.
(this commit msg was generated in some part by
[`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Downgrade the `start_dt`-trimming check in `open_history_client()`
from a `RuntimeError` raise to a warning log, allowing the caller
to still receive a (shorter) frame of bars (though we may need to still
specially handle such cases in the backfiller's biz logic layer).
Deats,
- add `trimmed_bars.size` guard to skip check on empty results.
- change condition to `>=` and log a warning with the short-frame
size instead of raising.
- comment-out `raise RuntimeError` and breakpoint for future
removal once confident.
- add docstring-style comment on `start_dt=` kwarg noting that
`Client.bars()` doesn't truly support it (uses duration-style
queries internally).
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Use (the only available in `ib_async`) `returnAll=True` in
`qualifyContractsAsync()` calls within `get_fute()` and handle the case
where IB returns a list of ambiguous contract matches instead of
a single result.
Deats,
- add `returnAll=True` to both `ContFuture` and `Future`
qualification calls.
- add `isinstance(con, list)` check after unpacking first result
to detect ambiguous contract sets.
- log warning with input params and matched contracts when
ambiguous.
- update return type annot to `Contract|list[Contract]`.
Also,
- handle list-of-contracts case in `find_contracts()` by unpacking
`*contracts` into the `qualifyContractsAsync()` call.
- reformat `qualifyContractsAsync()` calls to multiline style.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Use `con.exchange` as fallback when `con.primaryExchange` is empty
in `has_holiday()` to handle contracts like futures that don't
always set a `primaryExchange`.
Deats,
- extract `con: Contract` from `con_deats.contract` for reuse.
- use `con.primaryExchange or con.exchange` to ensure a valid
exchange code is always passed to the calendar lookup.
- add `Contract` to `TYPE_CHECKING` imports.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add exchange name translation in `.ib.venues.has_holiday()` to handle
non-standard exchange codes when looking up holiday gaps..
Deats,
- add an ad-hoc lookup dict to remap an IB `Contract.primaryExchange` val
which doesn't exist in the `exchange_calendars`'s alias set.
- use `.get()` with fallback to map `exch` to new `std_exch` and pass
that to `xcals.get_calendar()`.
- add the case i just caught, `'ARCA'` -> `'ARCX'` to the table when i loaded
the `gld.arca.ib` market..
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add fallback handling for unknown IB order status strings to
avoid crashes when IB returns unexpected status values.
Deats,
- add `'ValidationError': 'error'` mapping to `_statuses` dict.
- use `.get()` with `'error'` default instead of direct dict
lookup for `status.status`.
- add `elif status_str == 'error'` block to log unknown status
values.
- add type annots to `event_name` and `item` in
`deliver_trade_events()` loop.
Also,
- reformat log msg in `deliver_trade_events()` to multiline.
- drop extra conditional in `if status_str == 'filled'` check.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Migrate the IB broker backend to use `ib_async` (the actively maintained
fork) instead of the now stale, original `ib_insync` lib.
Deats,
- update `pyproject.toml` dep: drop `ib-insync` pin, add
`ib-async>=2.1.0`.
- update lock file with `ib-async` and its new `aeventkit` dep (which
i guess replaces `eventkit`).
- obvi, change all `ib_insync` imports to `ib_async` across `.ib.*`.
- update docs and select internal comments referencing the original lib.
Also,
- drop unused `ledger_dict` init in `_flex_reports.load_flex_trades()`.
- fix union type annot style: `dict | None` -> `dict|None`.
- strip `.tzinfo` from `lastTimeStamp` in `normalize()` to avoid
IPC codec issues with `ib_async`'s `timezone.utc` injection.
- pop `'defaults'` from ticker data dict in `normalize()` to avoid
non-serializable `timezone` objects and warning-log in such
cases.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
That is to be able to (eventually) introspect "ambiguous" contract sets
once we move to `ib_async` and its `returnAll: bool` now offered by
`IB.qualifyContractsAsync()`,
https://github.com/ib-api-reloaded/ib_async/blob/main/ib_async/ib.py#L2115
Also, tweak some type type annots to multline style in sibling mods.
Seems to be required to get certain widgets to size correctly on macos
like the order-mode "health bar" among other types of labels?
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Originally from @dnks with a weird(er) commit msg XP
Shorten SHM buffer key names to fit Darwin's 31
char filename limit by hashing the `fqme`/content
portion with `md5` and truncating to 8 hex chars.
Deats,
- `.fsp._api`: replace `piker.{actor}[{uuid}].{sym}`
format with `{uuid[:8]}_{hash}.fsp`
- `.tsp._history`: add `platform.system()` check to
conditionally shorten `.hist`/`.rt` keys on Darwin
while keeping the full key format on Linux
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
NOTE that this patch was orig by @dnks and broken out from a larger
commit which added unnecessary/out-of-scope changes we didn't end up
requiring.
That is, in-mem history buffer files should have a `.hist` suffix (which
was added back instead of the weird `.h` change from the rebased
8352d89e, initially providing the macos support fix).
I'm pretty sure we can afford the 3 extra chars if we're already
truncating to a max 16 char `key: str = f'{uuid[:8]}_{fqme_hash}'`, per
@goodboy's edit from the original key-truncation approach in the
original original @dnks patch: 75d8582b).
Refactor `SKILL.md` to adhere to claude-code skills docs and
eliminate content duplication with the supporting file.
Deats,
- fix `allowed-tools` from YAML list to comma-separated string
per frontmatter spec.
* https://code.claude.com/docs/en/skills#frontmatter-reference
- drop ~220 lines of inlined style-guide content that duplicated
`style-guide-reference.md`; replace with compact "Quick
Reference" section + markdown link to the full guide.
- fix supporting file ref from backtick-code to proper markdown
link syntax: `[style-guide-reference.md](./...)`.
- inline the file-writing instructions (timestamp + hash filename
format) directly, replacing the now-broken `CLAUDE.md` ref.
Also,
- add missing "commit msg" footer variant to
`style-guide-reference.md` (previously only showed "patch").
- move `.claude/CLAUDE.md` -> `style-guide-reference.md` as
proper skill supporting file.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Reorganize all 5 skills from loose `.md` files (and one
partially-formatted `commit_msg/`) into the documented
`subdirectory/SKILL.md` format with YAML frontmatter.
Deats,
- `commit_msg/` -> `commit-msg/` w/ enhanced frontmatter:
`argument-hint`, `disable-model-invocation`,
`allowed-tools`, dynamic `!` context injection for
staged diff + recent log, `$ARGUMENTS` support
- `piker_profiling.md` -> `piker-profiling/SKILL.md` +
`patterns.md` for detailed profiling patterns
- `piker_slang_and_communication_style.md` ->
`piker-slang/SKILL.md` + `dictionary.md` +
`examples.md`
- `pyqtgraph_rendering_optimization.md` ->
`pyqtgraph-optimization/SKILL.md` + `examples.md`
- `timeseries_numpy_polars_optimization.md` ->
`timeseries-optimization/SKILL.md` +
`numpy-patterns.md` + `polars-patterns.md`
Also,
- all background skills use `user-invocable: false`
for auto-application when relevant.
- use a hyphen convention across all dir names.
- content is now split into supporting files linked from each
`SKILL.md`.
(this patch was generated in some part by
[`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
We may eventually want to actually track these in git itself so we can
check/sync state with the corresponding git hosting service however? I'm
not sure how feasible it'll be but def worth thinking about Bp
- any `claude` commit-msg gen tmp files used for my `claude.commit`
thingie.
- any `nix develop --profile .nixdev` profile cache file.
- an `Session.vim` state-file used by `:Obsession .`.
Since apparently my commit-msg generator thingie stores the "training"
prompt content in this file by default.. REALLY this should be put into
a `SKILL.md` or similar later so that only truly global ctx content is
put here.
Integrate `exchange_calendars` lib to detect market holidays in
gap-checking logic via new `.ib.venues.has_holiday()` helper!
The `.ib.venues` impl deats,
- add a new `has_holiday()` using `xcals.get_calendar()` and friends
for sanity checking a venue's holiday closure-gaps.
* final holiday detection-check is basically,
`(cash_gap := (next_open - prev_close)) > period`
- include `time_step_s` param to `is_venue_closure()` for boundary
tolerance checks.
* let's us expand closure-time checks to include `+/-time_step_s`
"off-by-one-`timeframe`-sample" edge case ranges.
- add real docstring to `has_weekend()`.
In `.ib.api` refine usage for ^ changes,
- move `is_venue_open()` call + tz-convert outside gap check
- use a walrus to capture `has_closure_gap` from `is_venue_closure()`
- add a `not has_closure_gap` condition to the
mismatched-duration/short-frame warning block to avoid needless warns.
- keep duration-based "short-frame" log as `.error()` but toss in a bp
so (somone can) umask to figure out wtf is going on..
* we should **never** really hit this path unless there's a valid bug
or data issue with IB/GFIS!
* keep recursion path masked-out just leave a `breakpoint()` for now.
Also some logger updates,
- import `get_logger()` from top-level `piker.log` vs `.ib._util` which
was always kinda wrong..
- change `NonShittyIB._logger` to use `__name__` vs literal.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Gud Boi