Add Piker `/run-tests` harness
Extract current test env, scope, fixture, and runtime guidance into a repo-owned override for the canonical Run base. Deats, - require `xonsh` plugin disable and approved Nix provisioning - separate deterministic checks from live, GUI, and container cases - map source areas and document actor timeout retry behavior Prompt-IO: ai/prompt-io/opencode/20260724T220156Z_43d7a3ed_prompt_io.md (this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))backfiller_deep_fixes
parent
43d7a3edad
commit
e766c158de
|
|
@ -0,0 +1,264 @@
|
||||||
|
# Test Harness Reference: piker
|
||||||
|
|
||||||
|
This repository-local file supplements the canonical `/run-tests` skill.
|
||||||
|
Keep shared execution, worktree, failure-inspection, and cleanup policy in the
|
||||||
|
canonical `SKILL.md`; keep Piker commands, paths, fixtures, and known outcomes
|
||||||
|
here.
|
||||||
|
|
||||||
|
## Project And Environment
|
||||||
|
|
||||||
|
- Project/import: `piker`
|
||||||
|
- Test root: `tests/`
|
||||||
|
- Supported Python: `>=3.12,<3.14`
|
||||||
|
- Preferred complete environment: worktree-local `py313` inside the current
|
||||||
|
`nix develop` shell
|
||||||
|
- The flake shell pins CPython 3.13 and sets
|
||||||
|
`UV_PROJECT_ENVIRONMENT=py313`.
|
||||||
|
- Verify the interpreter, package resolution, and dependency import before
|
||||||
|
running tests. A bare `py313` may lack the Qt binding supplied by Nix.
|
||||||
|
|
||||||
|
Use an already-provisioned `py313` only when `import piker` succeeds:
|
||||||
|
|
||||||
|
```text
|
||||||
|
py313/bin/python -m pytest -p no:xonsh
|
||||||
|
```
|
||||||
|
|
||||||
|
If direct environment paths are unavailable, an existing uv environment can
|
||||||
|
be used without changing it, subject to the same import check:
|
||||||
|
|
||||||
|
```text
|
||||||
|
UV_PROJECT_ENVIRONMENT=py313 uv run --frozen --no-sync python -m pytest -p no:xonsh
|
||||||
|
```
|
||||||
|
|
||||||
|
Ask before running provisioning commands such as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
UV_PROJECT_ENVIRONMENT=py313 uv sync --dev --all-extras --no-group lint
|
||||||
|
nix develop
|
||||||
|
nix-shell default.nix
|
||||||
|
```
|
||||||
|
|
||||||
|
`nix develop` is the current Wayland/Qt 6 shell. `default.nix` is the current
|
||||||
|
X11 shell. Do not use `develop.nix` for current testing; it retains the old
|
||||||
|
Python 3.11, Poetry, and Qt 5 stack.
|
||||||
|
|
||||||
|
The current root-checkout `py313` resolves `piker` locally but fails
|
||||||
|
`import piker` outside Nix because PyQtGraph cannot import PyQt or PySide. Do
|
||||||
|
not treat that environment as test-ready and do not enter `nix develop`
|
||||||
|
without approval: its shell hook may recreate and sync `py313`.
|
||||||
|
|
||||||
|
Plain `uv sync` does not include the testing group. The `dbs` dependency group
|
||||||
|
is also absent from normal dev-shell provisioning.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
Base command in the preferred environment:
|
||||||
|
|
||||||
|
```text
|
||||||
|
py313/bin/python -m pytest -p no:xonsh
|
||||||
|
```
|
||||||
|
|
||||||
|
The explicit `-p no:xonsh` is required. The tracked comments-only
|
||||||
|
`pytest.ini` takes precedence over `pyproject.toml`, so the intended
|
||||||
|
`addopts = "-p no:xonsh"` and `testpaths = ["tests"]` are inactive.
|
||||||
|
Always pass a test path or node ID explicitly.
|
||||||
|
|
||||||
|
Package-resolution check that does not import Piker's dependencies:
|
||||||
|
|
||||||
|
```text
|
||||||
|
py313/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||||||
|
```
|
||||||
|
|
||||||
|
Dependency import check, required before collection or execution:
|
||||||
|
|
||||||
|
```text
|
||||||
|
py313/bin/python -c 'import pathlib, piker, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||||||
|
```
|
||||||
|
|
||||||
|
Safe core collection check:
|
||||||
|
|
||||||
|
```text
|
||||||
|
py313/bin/python -m pytest -p no:xonsh -q --collect-only tests/test_watchlists.py tests/test_accounting.py tests/test_services.py tests/test_ems.py tests/test_feeds.py tests/test_cli.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Default first-pass flags are `-q -x --tb=short --no-header` unless the user
|
||||||
|
requests otherwise. For actor-heavy tests, use one file or node per process
|
||||||
|
with an outer timeout:
|
||||||
|
|
||||||
|
```text
|
||||||
|
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q <one-file-or-node>
|
||||||
|
```
|
||||||
|
|
||||||
|
If an actor-heavy command exits `124` or `143`, retry that exact command once
|
||||||
|
and report both attempts. Never convert a retry pass into an unconditional
|
||||||
|
pass, and do not retry assertion, import, collection, or configuration
|
||||||
|
failures.
|
||||||
|
|
||||||
|
## Scope And Path Resolution
|
||||||
|
|
||||||
|
Resolve bare test filenames beneath `tests/`. Preserve complete node IDs and
|
||||||
|
apply `-k` only within explicitly selected paths. There is no marker-based
|
||||||
|
offline/full-suite split, so never use `pytest tests` as a deterministic
|
||||||
|
default.
|
||||||
|
|
||||||
|
Deterministic or local first-pass targets:
|
||||||
|
|
||||||
|
- `tests/test_watchlists.py`
|
||||||
|
- `tests/test_accounting.py::test_account_file_default_empty`
|
||||||
|
- `tests/test_services.py::test_runtime_boot`
|
||||||
|
- `tests/test_services.py::test_datad_spawn`
|
||||||
|
- `tests/test_ems.py::test_ems_err_on_bad_broker`
|
||||||
|
|
||||||
|
Require explicit authorization before running:
|
||||||
|
|
||||||
|
- `tests/test_feeds.py` - live Binance/Kraken feeds;
|
||||||
|
- `tests/test_services.py::test_ensure_datafeed_actors` - live Kraken feed;
|
||||||
|
- `tests/test_services.py::test_ensure_ems_in_paper_actors` - paper EMS with
|
||||||
|
live Kraken symbology/feed access;
|
||||||
|
- `tests/test_ems.py::test_multi_fill_positions` - live backend setup and
|
||||||
|
persisted state;
|
||||||
|
- `tests/test_accounting.py::test_paper_ledger_position_calcs` - tracked
|
||||||
|
fixtures plus possible live symcache generation;
|
||||||
|
- `tests/test_accounting.py::test_ib_account_with_duplicated_mktids` - active
|
||||||
|
broker/account configuration and state writes;
|
||||||
|
- `tests/test_dpi_font.py` - Qt/UI import and user-config side effects;
|
||||||
|
- `tests/test_docker_services.py` - optional dependencies and containers;
|
||||||
|
- `tests/test_questrade.py` - obsolete credentialed imports.
|
||||||
|
|
||||||
|
`tests/test_cli.py` is currently hard-skipped. It is not an active CLI
|
||||||
|
regression gate.
|
||||||
|
|
||||||
|
Never execute `piker store anal` or `piker store ldshm` as tests. They are
|
||||||
|
mutating or interactive operational commands.
|
||||||
|
|
||||||
|
## Project-Specific Flags And Backend Matrix
|
||||||
|
|
||||||
|
| Flag | Purpose |
|
||||||
|
|---|---|
|
||||||
|
| `--ll LEVEL` | Piker log level |
|
||||||
|
| `--confdir PATH` | Override `piker.config._config_dir` |
|
||||||
|
| `--spawn-backend trio|mp_spawn|mp_forkserver` | Tractor process backend |
|
||||||
|
| `--tpt-proto PROTO` | Tractor transport; one protocol per session |
|
||||||
|
| `--tpdb` / `--debug-mode` | Tractor plugin debug mode |
|
||||||
|
| `--pdb` | Standard pytest debugger |
|
||||||
|
| `-s` | No capture; required with `--pdb` and `open_test_pikerd` |
|
||||||
|
|
||||||
|
Do not invent `network`, `offline`, `docker`, `gui`, or `broker` markers; none
|
||||||
|
currently exists. `CI=1` is not an offline selector: feed tests still leave a
|
||||||
|
live Kraken case enabled. Current Piker tests exercise TCP; do not assume the
|
||||||
|
whole suite supports UDS merely because the Tractor plugin exposes it.
|
||||||
|
|
||||||
|
## Fixture Invariants
|
||||||
|
|
||||||
|
- The session `confdir` fixture does nothing unless `--confdir` is passed.
|
||||||
|
Its claimed `tests/data` fallback is not implemented and that directory is
|
||||||
|
absent.
|
||||||
|
- Function-scoped `tmpconfdir` changes process-global config state and does
|
||||||
|
not restore the previous path. Use separate pytest processes when
|
||||||
|
diagnosing state leakage.
|
||||||
|
- `open_test_pikerd` passes the temporary config path to child actors through
|
||||||
|
`tractor_runtime_overrides`.
|
||||||
|
- `tests/_inputs/trades_binance_paper.toml` and
|
||||||
|
`tests/_inputs/account.binance.paper.toml` are used in place. Accounting
|
||||||
|
contexts can write them on exit. Inspect `git diff -- tests/_inputs` after
|
||||||
|
any selected accounting case.
|
||||||
|
- Importing `tests/test_dpi_font.py` constructs module-global font objects
|
||||||
|
before fixtures can isolate config. If explicitly requested, isolate
|
||||||
|
`XDG_CONFIG_HOME` before Python starts and use the proper Qt/Nix shell.
|
||||||
|
- Piker and the installed Tractor pytest plugin do not provide a
|
||||||
|
repository-local process, shared-memory, or socket reaper. Never apply
|
||||||
|
historical broad `pkill -f tractor._child` guidance automatically.
|
||||||
|
|
||||||
|
## Test Layout
|
||||||
|
|
||||||
|
```text
|
||||||
|
tests/
|
||||||
|
conftest.py Piker options, config fixtures, Tractor plugin
|
||||||
|
_inputs/ tracked ledger/account fixtures
|
||||||
|
test_accounting.py config, ledgers, accounts, and position math
|
||||||
|
test_cli.py legacy CLI suite; hard-skipped
|
||||||
|
test_docker_services.py container integrations; optional deps, skipped
|
||||||
|
test_dpi_font.py Qt DPI/font behavior
|
||||||
|
test_ems.py actor, EMS, and paper-position behavior
|
||||||
|
test_feeds.py live Binance/Kraken feeds and shared memory
|
||||||
|
test_questrade.py obsolete credentialed tests; skipped
|
||||||
|
test_services.py pikerd/datad/feed/EMS actor lifecycle
|
||||||
|
test_watchlists.py deterministic watchlist JSON operations
|
||||||
|
```
|
||||||
|
|
||||||
|
## Change-To-Test Mapping
|
||||||
|
|
||||||
|
| Changed area | Run first | Caveat |
|
||||||
|
|---|---|---|
|
||||||
|
| `piker/watchlists/` | `tests/test_watchlists.py` | CLI suite is skipped |
|
||||||
|
| `piker/config.py` | `test_account_file_default_empty` | root-network test has a known mismatch |
|
||||||
|
| `piker/accounting/` | targeted accounting node | some cases use live/configured state |
|
||||||
|
| `piker/ui/_style.py`, `piker/ui/qt.py` | `tests/test_dpi_font.py` | GUI/config-isolated opt-in |
|
||||||
|
| `piker/service/_actor_runtime.py`, `_registry.py`, `_mngr.py` | `test_runtime_boot` | then `test_datad_spawn` |
|
||||||
|
| `piker/service/`, `piker/data/_daemon.py` | `test_datad_spawn` | feed lifecycle cases are live |
|
||||||
|
| `piker/data/feed.py`, `flows.py`, `_sharedmem.py`, `_sampling.py` | collect first | feed execution needs live permission |
|
||||||
|
| `piker/clearing/` | `test_ems_err_on_bad_broker` | multi-fill case is live/persisted |
|
||||||
|
| `piker/brokers/binance/`, `kraken/` | selected feed/accounting node | live network |
|
||||||
|
| `piker/brokers/ib/` | duplicated-market-ID node | controlled account config required |
|
||||||
|
| Docker/service adapters | `tests/test_docker_services.py` | optional deps and containers |
|
||||||
|
| project, lock, or Nix files | import check and safe collection | full collection is not safe by default |
|
||||||
|
|
||||||
|
Prefer deterministic filesystem/config tests, then local actor-runtime nodes,
|
||||||
|
then explicitly approved live broker, GUI, or container coverage.
|
||||||
|
|
||||||
|
## Quick Checks
|
||||||
|
|
||||||
|
```text
|
||||||
|
py313/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||||||
|
py313/bin/python -c 'import pathlib, piker, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||||||
|
py313/bin/python -m pytest -p no:xonsh -q tests/test_watchlists.py
|
||||||
|
py313/bin/python -m pytest -p no:xonsh -q tests/test_accounting.py::test_account_file_default_empty
|
||||||
|
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_services.py::test_runtime_boot
|
||||||
|
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_services.py::test_datad_spawn
|
||||||
|
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_ems.py::test_ems_err_on_bad_broker
|
||||||
|
```
|
||||||
|
|
||||||
|
## Known Outcomes
|
||||||
|
|
||||||
|
- The current root-checkout `py313` fails `import piker` outside the Nix shell
|
||||||
|
with `ImportError: PyQtGraph requires one of PyQt5, PyQt6, PySide2 or
|
||||||
|
PySide6`. This is an incomplete environment, not an application regression.
|
||||||
|
- `tests/test_accounting.py::test_root_conf_networking_section` currently
|
||||||
|
expects `network.tsdb`, which is absent from the tracked config template.
|
||||||
|
Match the current `KeyError: 'tsdb'` before classifying it as the known
|
||||||
|
repository mismatch.
|
||||||
|
- `tests/test_docker_services.py` is marked skipped but imports
|
||||||
|
`elasticsearch` first. Without the `dbs` group it fails collection rather
|
||||||
|
than skipping.
|
||||||
|
- `tests/test_questrade.py` is marked skipped but imports undeclared `asks`
|
||||||
|
through the legacy broker module before marks apply.
|
||||||
|
- `test_open_orders_reloaded` and `test_dark_order_clearing` in
|
||||||
|
`tests/test_ems.py` contain only ellipsis bodies. A pass does not verify the
|
||||||
|
named behavior.
|
||||||
|
- Treat `.pytest_cache` feed IDs using old FQME forms as stale historical
|
||||||
|
cache, not current known failures.
|
||||||
|
|
||||||
|
Do not classify every `TooSlowError`, timeout, or child survivor as the known
|
||||||
|
Tractor teardown wedge. Match the selected node and the documented
|
||||||
|
second-runtime/lingering-child signature.
|
||||||
|
|
||||||
|
## Tractor Runtime Notes
|
||||||
|
|
||||||
|
The suite loads `tractor._testing.pytest`. Defaults are the `trio` spawn
|
||||||
|
backend and TCP transport. Registry addresses are normally session-unique,
|
||||||
|
except `tests/test_services.py::test_runtime_boot`, which binds
|
||||||
|
`127.0.0.1:6666`. Check that fixed port only for that node; ordinary tests do
|
||||||
|
not require Piker's production `127.0.0.1:6116` registry address.
|
||||||
|
|
||||||
|
Current repository concurrency notes document an intermittent second
|
||||||
|
in-process `pikerd` boot wedge with a lingering broker child and unread parent
|
||||||
|
IPC bytes. Use the outer timeout for actor-heavy nodes. Retry an exact command
|
||||||
|
once only after status `124` or `143`, and report both attempts.
|
||||||
|
|
||||||
|
Ordinary actor tests support normal capture. Diagnose a capture-dependent hang
|
||||||
|
by retrying only the exact node with `-s`. Standard `--pdb` with
|
||||||
|
`open_test_pikerd` requires `-s`; Tractor's `--tpdb` is a separate option.
|
||||||
|
|
||||||
|
After abnormal exit, inspect only descendants, sockets, and shared-memory
|
||||||
|
objects attributable to the exact pytest session. Ask before signaling or
|
||||||
|
unlinking anything.
|
||||||
|
|
@ -0,0 +1,32 @@
|
||||||
|
---
|
||||||
|
model: openai/gpt-5.6-sol
|
||||||
|
service: opencode
|
||||||
|
session: ses_092a25886ffeT0nrIrdkMEcgJK
|
||||||
|
timestamp: 2026-07-24T22:01:56Z
|
||||||
|
git_ref: flake_update
|
||||||
|
scope: config
|
||||||
|
substantive: true
|
||||||
|
raw_file: 20260724T220156Z_43d7a3ed_prompt_io.raw.md
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prompt
|
||||||
|
|
||||||
|
After committing Piker's provider-neutral deployment, the user asked to
|
||||||
|
extract Piker-specialized `/run-tests` content from the historical
|
||||||
|
`wkt/run_tests_skill` implementation into the new deployment.
|
||||||
|
|
||||||
|
## Response summary
|
||||||
|
|
||||||
|
Create a repository-owned `test-harness-reference.md` for the canonical Run
|
||||||
|
base. Revalidate historical commands and assumptions against current pytest,
|
||||||
|
uv/Nix environments, tests, fixtures, and Tractor runtime behavior, retaining
|
||||||
|
only current Piker-specific guidance.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `.claude/skills/run-tests/test-harness-reference.md` - define Piker's safe
|
||||||
|
commands, opt-in boundaries, mappings, known outcomes, and runtime notes.
|
||||||
|
|
||||||
|
## Human edits
|
||||||
|
|
||||||
|
None at generation time; pending human review.
|
||||||
|
|
@ -0,0 +1,32 @@
|
||||||
|
---
|
||||||
|
model: openai/gpt-5.6-sol
|
||||||
|
service: opencode
|
||||||
|
timestamp: 2026-07-24T22:01:56Z
|
||||||
|
git_ref: flake_update
|
||||||
|
diff_cmd: git diff HEAD~1..HEAD
|
||||||
|
---
|
||||||
|
|
||||||
|
The user asked to extract useful Piker-specific `/run-tests` behavior from
|
||||||
|
`wkt/run_tests_skill` after committing the provider-neutral deployment.
|
||||||
|
|
||||||
|
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
||||||
|
|
||||||
|
Generate a repository-owned harness reference rather than restoring the old
|
||||||
|
standalone `SKILL.md`. Preserve verified Piker-specific environment, command,
|
||||||
|
scope, fixture, test-layout, change-mapping, known-outcome, and Tractor-runtime
|
||||||
|
details while leaving shared execution and cleanup policy in the canonical
|
||||||
|
`ai.skillz` base.
|
||||||
|
|
||||||
|
Repository inspection found that the historical skill needs correction:
|
||||||
|
|
||||||
|
- the comments-only `pytest.ini` shadows `pyproject.toml` pytest options, so
|
||||||
|
`-p no:xonsh` must be explicit;
|
||||||
|
- no marker-based offline suite exists, and several live/GUI/container tests
|
||||||
|
are unsafe defaults;
|
||||||
|
- skipped Docker and Questrade modules can fail during import before marks;
|
||||||
|
- actor timeout retry remains relevant only for the exact timed-out command;
|
||||||
|
- `py313` and frozen no-sync `uv run` are the current local environment paths.
|
||||||
|
|
||||||
|
Validate with checkout import resolution, safe collection, and deterministic
|
||||||
|
watchlist/accounting tests. Do not run live broker, GUI, container,
|
||||||
|
credentialed, or user-config-dependent cases during harness verification.
|
||||||
Loading…
Reference in New Issue