Add Piker `/run-tests` harness
Extract current test env, scope, fixture, and runtime guidance into a repo-owned override for the canonical Run base. Deats, - require `xonsh` plugin disable and approved Nix provisioning - separate deterministic checks from live, GUI, and container cases - map source areas and document actor timeout retry behavior Prompt-IO: ai/prompt-io/opencode/20260724T220156Z_43d7a3ed_prompt_io.md (this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))backfiller_deep_fixes
parent
43d7a3edad
commit
e766c158de
|
|
@ -0,0 +1,264 @@
|
|||
# Test Harness Reference: piker
|
||||
|
||||
This repository-local file supplements the canonical `/run-tests` skill.
|
||||
Keep shared execution, worktree, failure-inspection, and cleanup policy in the
|
||||
canonical `SKILL.md`; keep Piker commands, paths, fixtures, and known outcomes
|
||||
here.
|
||||
|
||||
## Project And Environment
|
||||
|
||||
- Project/import: `piker`
|
||||
- Test root: `tests/`
|
||||
- Supported Python: `>=3.12,<3.14`
|
||||
- Preferred complete environment: worktree-local `py313` inside the current
|
||||
`nix develop` shell
|
||||
- The flake shell pins CPython 3.13 and sets
|
||||
`UV_PROJECT_ENVIRONMENT=py313`.
|
||||
- Verify the interpreter, package resolution, and dependency import before
|
||||
running tests. A bare `py313` may lack the Qt binding supplied by Nix.
|
||||
|
||||
Use an already-provisioned `py313` only when `import piker` succeeds:
|
||||
|
||||
```text
|
||||
py313/bin/python -m pytest -p no:xonsh
|
||||
```
|
||||
|
||||
If direct environment paths are unavailable, an existing uv environment can
|
||||
be used without changing it, subject to the same import check:
|
||||
|
||||
```text
|
||||
UV_PROJECT_ENVIRONMENT=py313 uv run --frozen --no-sync python -m pytest -p no:xonsh
|
||||
```
|
||||
|
||||
Ask before running provisioning commands such as:
|
||||
|
||||
```text
|
||||
UV_PROJECT_ENVIRONMENT=py313 uv sync --dev --all-extras --no-group lint
|
||||
nix develop
|
||||
nix-shell default.nix
|
||||
```
|
||||
|
||||
`nix develop` is the current Wayland/Qt 6 shell. `default.nix` is the current
|
||||
X11 shell. Do not use `develop.nix` for current testing; it retains the old
|
||||
Python 3.11, Poetry, and Qt 5 stack.
|
||||
|
||||
The current root-checkout `py313` resolves `piker` locally but fails
|
||||
`import piker` outside Nix because PyQtGraph cannot import PyQt or PySide. Do
|
||||
not treat that environment as test-ready and do not enter `nix develop`
|
||||
without approval: its shell hook may recreate and sync `py313`.
|
||||
|
||||
Plain `uv sync` does not include the testing group. The `dbs` dependency group
|
||||
is also absent from normal dev-shell provisioning.
|
||||
|
||||
## Commands
|
||||
|
||||
Base command in the preferred environment:
|
||||
|
||||
```text
|
||||
py313/bin/python -m pytest -p no:xonsh
|
||||
```
|
||||
|
||||
The explicit `-p no:xonsh` is required. The tracked comments-only
|
||||
`pytest.ini` takes precedence over `pyproject.toml`, so the intended
|
||||
`addopts = "-p no:xonsh"` and `testpaths = ["tests"]` are inactive.
|
||||
Always pass a test path or node ID explicitly.
|
||||
|
||||
Package-resolution check that does not import Piker's dependencies:
|
||||
|
||||
```text
|
||||
py313/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||||
```
|
||||
|
||||
Dependency import check, required before collection or execution:
|
||||
|
||||
```text
|
||||
py313/bin/python -c 'import pathlib, piker, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||||
```
|
||||
|
||||
Safe core collection check:
|
||||
|
||||
```text
|
||||
py313/bin/python -m pytest -p no:xonsh -q --collect-only tests/test_watchlists.py tests/test_accounting.py tests/test_services.py tests/test_ems.py tests/test_feeds.py tests/test_cli.py
|
||||
```
|
||||
|
||||
Default first-pass flags are `-q -x --tb=short --no-header` unless the user
|
||||
requests otherwise. For actor-heavy tests, use one file or node per process
|
||||
with an outer timeout:
|
||||
|
||||
```text
|
||||
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q <one-file-or-node>
|
||||
```
|
||||
|
||||
If an actor-heavy command exits `124` or `143`, retry that exact command once
|
||||
and report both attempts. Never convert a retry pass into an unconditional
|
||||
pass, and do not retry assertion, import, collection, or configuration
|
||||
failures.
|
||||
|
||||
## Scope And Path Resolution
|
||||
|
||||
Resolve bare test filenames beneath `tests/`. Preserve complete node IDs and
|
||||
apply `-k` only within explicitly selected paths. There is no marker-based
|
||||
offline/full-suite split, so never use `pytest tests` as a deterministic
|
||||
default.
|
||||
|
||||
Deterministic or local first-pass targets:
|
||||
|
||||
- `tests/test_watchlists.py`
|
||||
- `tests/test_accounting.py::test_account_file_default_empty`
|
||||
- `tests/test_services.py::test_runtime_boot`
|
||||
- `tests/test_services.py::test_datad_spawn`
|
||||
- `tests/test_ems.py::test_ems_err_on_bad_broker`
|
||||
|
||||
Require explicit authorization before running:
|
||||
|
||||
- `tests/test_feeds.py` - live Binance/Kraken feeds;
|
||||
- `tests/test_services.py::test_ensure_datafeed_actors` - live Kraken feed;
|
||||
- `tests/test_services.py::test_ensure_ems_in_paper_actors` - paper EMS with
|
||||
live Kraken symbology/feed access;
|
||||
- `tests/test_ems.py::test_multi_fill_positions` - live backend setup and
|
||||
persisted state;
|
||||
- `tests/test_accounting.py::test_paper_ledger_position_calcs` - tracked
|
||||
fixtures plus possible live symcache generation;
|
||||
- `tests/test_accounting.py::test_ib_account_with_duplicated_mktids` - active
|
||||
broker/account configuration and state writes;
|
||||
- `tests/test_dpi_font.py` - Qt/UI import and user-config side effects;
|
||||
- `tests/test_docker_services.py` - optional dependencies and containers;
|
||||
- `tests/test_questrade.py` - obsolete credentialed imports.
|
||||
|
||||
`tests/test_cli.py` is currently hard-skipped. It is not an active CLI
|
||||
regression gate.
|
||||
|
||||
Never execute `piker store anal` or `piker store ldshm` as tests. They are
|
||||
mutating or interactive operational commands.
|
||||
|
||||
## Project-Specific Flags And Backend Matrix
|
||||
|
||||
| Flag | Purpose |
|
||||
|---|---|
|
||||
| `--ll LEVEL` | Piker log level |
|
||||
| `--confdir PATH` | Override `piker.config._config_dir` |
|
||||
| `--spawn-backend trio|mp_spawn|mp_forkserver` | Tractor process backend |
|
||||
| `--tpt-proto PROTO` | Tractor transport; one protocol per session |
|
||||
| `--tpdb` / `--debug-mode` | Tractor plugin debug mode |
|
||||
| `--pdb` | Standard pytest debugger |
|
||||
| `-s` | No capture; required with `--pdb` and `open_test_pikerd` |
|
||||
|
||||
Do not invent `network`, `offline`, `docker`, `gui`, or `broker` markers; none
|
||||
currently exists. `CI=1` is not an offline selector: feed tests still leave a
|
||||
live Kraken case enabled. Current Piker tests exercise TCP; do not assume the
|
||||
whole suite supports UDS merely because the Tractor plugin exposes it.
|
||||
|
||||
## Fixture Invariants
|
||||
|
||||
- The session `confdir` fixture does nothing unless `--confdir` is passed.
|
||||
Its claimed `tests/data` fallback is not implemented and that directory is
|
||||
absent.
|
||||
- Function-scoped `tmpconfdir` changes process-global config state and does
|
||||
not restore the previous path. Use separate pytest processes when
|
||||
diagnosing state leakage.
|
||||
- `open_test_pikerd` passes the temporary config path to child actors through
|
||||
`tractor_runtime_overrides`.
|
||||
- `tests/_inputs/trades_binance_paper.toml` and
|
||||
`tests/_inputs/account.binance.paper.toml` are used in place. Accounting
|
||||
contexts can write them on exit. Inspect `git diff -- tests/_inputs` after
|
||||
any selected accounting case.
|
||||
- Importing `tests/test_dpi_font.py` constructs module-global font objects
|
||||
before fixtures can isolate config. If explicitly requested, isolate
|
||||
`XDG_CONFIG_HOME` before Python starts and use the proper Qt/Nix shell.
|
||||
- Piker and the installed Tractor pytest plugin do not provide a
|
||||
repository-local process, shared-memory, or socket reaper. Never apply
|
||||
historical broad `pkill -f tractor._child` guidance automatically.
|
||||
|
||||
## Test Layout
|
||||
|
||||
```text
|
||||
tests/
|
||||
conftest.py Piker options, config fixtures, Tractor plugin
|
||||
_inputs/ tracked ledger/account fixtures
|
||||
test_accounting.py config, ledgers, accounts, and position math
|
||||
test_cli.py legacy CLI suite; hard-skipped
|
||||
test_docker_services.py container integrations; optional deps, skipped
|
||||
test_dpi_font.py Qt DPI/font behavior
|
||||
test_ems.py actor, EMS, and paper-position behavior
|
||||
test_feeds.py live Binance/Kraken feeds and shared memory
|
||||
test_questrade.py obsolete credentialed tests; skipped
|
||||
test_services.py pikerd/datad/feed/EMS actor lifecycle
|
||||
test_watchlists.py deterministic watchlist JSON operations
|
||||
```
|
||||
|
||||
## Change-To-Test Mapping
|
||||
|
||||
| Changed area | Run first | Caveat |
|
||||
|---|---|---|
|
||||
| `piker/watchlists/` | `tests/test_watchlists.py` | CLI suite is skipped |
|
||||
| `piker/config.py` | `test_account_file_default_empty` | root-network test has a known mismatch |
|
||||
| `piker/accounting/` | targeted accounting node | some cases use live/configured state |
|
||||
| `piker/ui/_style.py`, `piker/ui/qt.py` | `tests/test_dpi_font.py` | GUI/config-isolated opt-in |
|
||||
| `piker/service/_actor_runtime.py`, `_registry.py`, `_mngr.py` | `test_runtime_boot` | then `test_datad_spawn` |
|
||||
| `piker/service/`, `piker/data/_daemon.py` | `test_datad_spawn` | feed lifecycle cases are live |
|
||||
| `piker/data/feed.py`, `flows.py`, `_sharedmem.py`, `_sampling.py` | collect first | feed execution needs live permission |
|
||||
| `piker/clearing/` | `test_ems_err_on_bad_broker` | multi-fill case is live/persisted |
|
||||
| `piker/brokers/binance/`, `kraken/` | selected feed/accounting node | live network |
|
||||
| `piker/brokers/ib/` | duplicated-market-ID node | controlled account config required |
|
||||
| Docker/service adapters | `tests/test_docker_services.py` | optional deps and containers |
|
||||
| project, lock, or Nix files | import check and safe collection | full collection is not safe by default |
|
||||
|
||||
Prefer deterministic filesystem/config tests, then local actor-runtime nodes,
|
||||
then explicitly approved live broker, GUI, or container coverage.
|
||||
|
||||
## Quick Checks
|
||||
|
||||
```text
|
||||
py313/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||||
py313/bin/python -c 'import pathlib, piker, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||||
py313/bin/python -m pytest -p no:xonsh -q tests/test_watchlists.py
|
||||
py313/bin/python -m pytest -p no:xonsh -q tests/test_accounting.py::test_account_file_default_empty
|
||||
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_services.py::test_runtime_boot
|
||||
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_services.py::test_datad_spawn
|
||||
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_ems.py::test_ems_err_on_bad_broker
|
||||
```
|
||||
|
||||
## Known Outcomes
|
||||
|
||||
- The current root-checkout `py313` fails `import piker` outside the Nix shell
|
||||
with `ImportError: PyQtGraph requires one of PyQt5, PyQt6, PySide2 or
|
||||
PySide6`. This is an incomplete environment, not an application regression.
|
||||
- `tests/test_accounting.py::test_root_conf_networking_section` currently
|
||||
expects `network.tsdb`, which is absent from the tracked config template.
|
||||
Match the current `KeyError: 'tsdb'` before classifying it as the known
|
||||
repository mismatch.
|
||||
- `tests/test_docker_services.py` is marked skipped but imports
|
||||
`elasticsearch` first. Without the `dbs` group it fails collection rather
|
||||
than skipping.
|
||||
- `tests/test_questrade.py` is marked skipped but imports undeclared `asks`
|
||||
through the legacy broker module before marks apply.
|
||||
- `test_open_orders_reloaded` and `test_dark_order_clearing` in
|
||||
`tests/test_ems.py` contain only ellipsis bodies. A pass does not verify the
|
||||
named behavior.
|
||||
- Treat `.pytest_cache` feed IDs using old FQME forms as stale historical
|
||||
cache, not current known failures.
|
||||
|
||||
Do not classify every `TooSlowError`, timeout, or child survivor as the known
|
||||
Tractor teardown wedge. Match the selected node and the documented
|
||||
second-runtime/lingering-child signature.
|
||||
|
||||
## Tractor Runtime Notes
|
||||
|
||||
The suite loads `tractor._testing.pytest`. Defaults are the `trio` spawn
|
||||
backend and TCP transport. Registry addresses are normally session-unique,
|
||||
except `tests/test_services.py::test_runtime_boot`, which binds
|
||||
`127.0.0.1:6666`. Check that fixed port only for that node; ordinary tests do
|
||||
not require Piker's production `127.0.0.1:6116` registry address.
|
||||
|
||||
Current repository concurrency notes document an intermittent second
|
||||
in-process `pikerd` boot wedge with a lingering broker child and unread parent
|
||||
IPC bytes. Use the outer timeout for actor-heavy nodes. Retry an exact command
|
||||
once only after status `124` or `143`, and report both attempts.
|
||||
|
||||
Ordinary actor tests support normal capture. Diagnose a capture-dependent hang
|
||||
by retrying only the exact node with `-s`. Standard `--pdb` with
|
||||
`open_test_pikerd` requires `-s`; Tractor's `--tpdb` is a separate option.
|
||||
|
||||
After abnormal exit, inspect only descendants, sockets, and shared-memory
|
||||
objects attributable to the exact pytest session. Ask before signaling or
|
||||
unlinking anything.
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: ses_092a25886ffeT0nrIrdkMEcgJK
|
||||
timestamp: 2026-07-24T22:01:56Z
|
||||
git_ref: flake_update
|
||||
scope: config
|
||||
substantive: true
|
||||
raw_file: 20260724T220156Z_43d7a3ed_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
After committing Piker's provider-neutral deployment, the user asked to
|
||||
extract Piker-specialized `/run-tests` content from the historical
|
||||
`wkt/run_tests_skill` implementation into the new deployment.
|
||||
|
||||
## Response summary
|
||||
|
||||
Create a repository-owned `test-harness-reference.md` for the canonical Run
|
||||
base. Revalidate historical commands and assumptions against current pytest,
|
||||
uv/Nix environments, tests, fixtures, and Tractor runtime behavior, retaining
|
||||
only current Piker-specific guidance.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `.claude/skills/run-tests/test-harness-reference.md` - define Piker's safe
|
||||
commands, opt-in boundaries, mappings, known outcomes, and runtime notes.
|
||||
|
||||
## Human edits
|
||||
|
||||
None at generation time; pending human review.
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-07-24T22:01:56Z
|
||||
git_ref: flake_update
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
The user asked to extract useful Piker-specific `/run-tests` behavior from
|
||||
`wkt/run_tests_skill` after committing the provider-neutral deployment.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
||||
|
||||
Generate a repository-owned harness reference rather than restoring the old
|
||||
standalone `SKILL.md`. Preserve verified Piker-specific environment, command,
|
||||
scope, fixture, test-layout, change-mapping, known-outcome, and Tractor-runtime
|
||||
details while leaving shared execution and cleanup policy in the canonical
|
||||
`ai.skillz` base.
|
||||
|
||||
Repository inspection found that the historical skill needs correction:
|
||||
|
||||
- the comments-only `pytest.ini` shadows `pyproject.toml` pytest options, so
|
||||
`-p no:xonsh` must be explicit;
|
||||
- no marker-based offline suite exists, and several live/GUI/container tests
|
||||
are unsafe defaults;
|
||||
- skipped Docker and Questrade modules can fail during import before marks;
|
||||
- actor timeout retry remains relevant only for the exact timed-out command;
|
||||
- `py313` and frozen no-sync `uv run` are the current local environment paths.
|
||||
|
||||
Validate with checkout import resolution, safe collection, and deterministic
|
||||
watchlist/accounting tests. Do not run live broker, GUI, container,
|
||||
credentialed, or user-config-dependent cases during harness verification.
|
||||
Loading…
Reference in New Issue