265 lines
12 KiB
Markdown
265 lines
12 KiB
Markdown
|
# Test Harness Reference: piker
|
||
|
|
|
||
|
|
This repository-local file supplements the canonical `/run-tests` skill.
|
||
|
|
Keep shared execution, worktree, failure-inspection, and cleanup policy in the
|
||
|
|
canonical `SKILL.md`; keep Piker commands, paths, fixtures, and known outcomes
|
||
|
|
here.
|
||
|
|
|
||
|
|
## Project And Environment
|
||
|
|
|
||
|
|
- Project/import: `piker`
|
||
|
|
- Test root: `tests/`
|
||
|
|
- Supported Python: `>=3.12,<3.14`
|
||
|
|
- Preferred complete environment: worktree-local `py313` inside the current
|
||
|
|
`nix develop` shell
|
||
|
|
- The flake shell pins CPython 3.13 and sets
|
||
|
|
`UV_PROJECT_ENVIRONMENT=py313`.
|
||
|
|
- Verify the interpreter, package resolution, and dependency import before
|
||
|
|
running tests. A bare `py313` may lack the Qt binding supplied by Nix.
|
||
|
|
|
||
|
|
Use an already-provisioned `py313` only when `import piker` succeeds:
|
||
|
|
|
||
|
|
```text
|
||
|
|
py313/bin/python -m pytest -p no:xonsh
|
||
|
|
```
|
||
|
|
|
||
|
|
If direct environment paths are unavailable, an existing uv environment can
|
||
|
|
be used without changing it, subject to the same import check:
|
||
|
|
|
||
|
|
```text
|
||
|
|
UV_PROJECT_ENVIRONMENT=py313 uv run --frozen --no-sync python -m pytest -p no:xonsh
|
||
|
|
```
|
||
|
|
|
||
|
|
Ask before running provisioning commands such as:
|
||
|
|
|
||
|
|
```text
|
||
|
|
UV_PROJECT_ENVIRONMENT=py313 uv sync --dev --all-extras --no-group lint
|
||
|
|
nix develop
|
||
|
|
nix-shell default.nix
|
||
|
|
```
|
||
|
|
|
||
|
|
`nix develop` is the current Wayland/Qt 6 shell. `default.nix` is the current
|
||
|
|
X11 shell. Do not use `develop.nix` for current testing; it retains the old
|
||
|
|
Python 3.11, Poetry, and Qt 5 stack.
|
||
|
|
|
||
|
|
The current root-checkout `py313` resolves `piker` locally but fails
|
||
|
|
`import piker` outside Nix because PyQtGraph cannot import PyQt or PySide. Do
|
||
|
|
not treat that environment as test-ready and do not enter `nix develop`
|
||
|
|
without approval: its shell hook may recreate and sync `py313`.
|
||
|
|
|
||
|
|
Plain `uv sync` does not include the testing group. The `dbs` dependency group
|
||
|
|
is also absent from normal dev-shell provisioning.
|
||
|
|
|
||
|
|
## Commands
|
||
|
|
|
||
|
|
Base command in the preferred environment:
|
||
|
|
|
||
|
|
```text
|
||
|
|
py313/bin/python -m pytest -p no:xonsh
|
||
|
|
```
|
||
|
|
|
||
|
|
The explicit `-p no:xonsh` is required. The tracked comments-only
|
||
|
|
`pytest.ini` takes precedence over `pyproject.toml`, so the intended
|
||
|
|
`addopts = "-p no:xonsh"` and `testpaths = ["tests"]` are inactive.
|
||
|
|
Always pass a test path or node ID explicitly.
|
||
|
|
|
||
|
|
Package-resolution check that does not import Piker's dependencies:
|
||
|
|
|
||
|
|
```text
|
||
|
|
py313/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||
|
|
```
|
||
|
|
|
||
|
|
Dependency import check, required before collection or execution:
|
||
|
|
|
||
|
|
```text
|
||
|
|
py313/bin/python -c 'import pathlib, piker, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||
|
|
```
|
||
|
|
|
||
|
|
Safe core collection check:
|
||
|
|
|
||
|
|
```text
|
||
|
|
py313/bin/python -m pytest -p no:xonsh -q --collect-only tests/test_watchlists.py tests/test_accounting.py tests/test_services.py tests/test_ems.py tests/test_feeds.py tests/test_cli.py
|
||
|
|
```
|
||
|
|
|
||
|
|
Default first-pass flags are `-q -x --tb=short --no-header` unless the user
|
||
|
|
requests otherwise. For actor-heavy tests, use one file or node per process
|
||
|
|
with an outer timeout:
|
||
|
|
|
||
|
|
```text
|
||
|
|
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q <one-file-or-node>
|
||
|
|
```
|
||
|
|
|
||
|
|
If an actor-heavy command exits `124` or `143`, retry that exact command once
|
||
|
|
and report both attempts. Never convert a retry pass into an unconditional
|
||
|
|
pass, and do not retry assertion, import, collection, or configuration
|
||
|
|
failures.
|
||
|
|
|
||
|
|
## Scope And Path Resolution
|
||
|
|
|
||
|
|
Resolve bare test filenames beneath `tests/`. Preserve complete node IDs and
|
||
|
|
apply `-k` only within explicitly selected paths. There is no marker-based
|
||
|
|
offline/full-suite split, so never use `pytest tests` as a deterministic
|
||
|
|
default.
|
||
|
|
|
||
|
|
Deterministic or local first-pass targets:
|
||
|
|
|
||
|
|
- `tests/test_watchlists.py`
|
||
|
|
- `tests/test_accounting.py::test_account_file_default_empty`
|
||
|
|
- `tests/test_services.py::test_runtime_boot`
|
||
|
|
- `tests/test_services.py::test_datad_spawn`
|
||
|
|
- `tests/test_ems.py::test_ems_err_on_bad_broker`
|
||
|
|
|
||
|
|
Require explicit authorization before running:
|
||
|
|
|
||
|
|
- `tests/test_feeds.py` - live Binance/Kraken feeds;
|
||
|
|
- `tests/test_services.py::test_ensure_datafeed_actors` - live Kraken feed;
|
||
|
|
- `tests/test_services.py::test_ensure_ems_in_paper_actors` - paper EMS with
|
||
|
|
live Kraken symbology/feed access;
|
||
|
|
- `tests/test_ems.py::test_multi_fill_positions` - live backend setup and
|
||
|
|
persisted state;
|
||
|
|
- `tests/test_accounting.py::test_paper_ledger_position_calcs` - tracked
|
||
|
|
fixtures plus possible live symcache generation;
|
||
|
|
- `tests/test_accounting.py::test_ib_account_with_duplicated_mktids` - active
|
||
|
|
broker/account configuration and state writes;
|
||
|
|
- `tests/test_dpi_font.py` - Qt/UI import and user-config side effects;
|
||
|
|
- `tests/test_docker_services.py` - optional dependencies and containers;
|
||
|
|
- `tests/test_questrade.py` - obsolete credentialed imports.
|
||
|
|
|
||
|
|
`tests/test_cli.py` is currently hard-skipped. It is not an active CLI
|
||
|
|
regression gate.
|
||
|
|
|
||
|
|
Never execute `piker store anal` or `piker store ldshm` as tests. They are
|
||
|
|
mutating or interactive operational commands.
|
||
|
|
|
||
|
|
## Project-Specific Flags And Backend Matrix
|
||
|
|
|
||
|
|
| Flag | Purpose |
|
||
|
|
|---|---|
|
||
|
|
| `--ll LEVEL` | Piker log level |
|
||
|
|
| `--confdir PATH` | Override `piker.config._config_dir` |
|
||
|
|
| `--spawn-backend trio|mp_spawn|mp_forkserver` | Tractor process backend |
|
||
|
|
| `--tpt-proto PROTO` | Tractor transport; one protocol per session |
|
||
|
|
| `--tpdb` / `--debug-mode` | Tractor plugin debug mode |
|
||
|
|
| `--pdb` | Standard pytest debugger |
|
||
|
|
| `-s` | No capture; required with `--pdb` and `open_test_pikerd` |
|
||
|
|
|
||
|
|
Do not invent `network`, `offline`, `docker`, `gui`, or `broker` markers; none
|
||
|
|
currently exists. `CI=1` is not an offline selector: feed tests still leave a
|
||
|
|
live Kraken case enabled. Current Piker tests exercise TCP; do not assume the
|
||
|
|
whole suite supports UDS merely because the Tractor plugin exposes it.
|
||
|
|
|
||
|
|
## Fixture Invariants
|
||
|
|
|
||
|
|
- The session `confdir` fixture does nothing unless `--confdir` is passed.
|
||
|
|
Its claimed `tests/data` fallback is not implemented and that directory is
|
||
|
|
absent.
|
||
|
|
- Function-scoped `tmpconfdir` changes process-global config state and does
|
||
|
|
not restore the previous path. Use separate pytest processes when
|
||
|
|
diagnosing state leakage.
|
||
|
|
- `open_test_pikerd` passes the temporary config path to child actors through
|
||
|
|
`tractor_runtime_overrides`.
|
||
|
|
- `tests/_inputs/trades_binance_paper.toml` and
|
||
|
|
`tests/_inputs/account.binance.paper.toml` are used in place. Accounting
|
||
|
|
contexts can write them on exit. Inspect `git diff -- tests/_inputs` after
|
||
|
|
any selected accounting case.
|
||
|
|
- Importing `tests/test_dpi_font.py` constructs module-global font objects
|
||
|
|
before fixtures can isolate config. If explicitly requested, isolate
|
||
|
|
`XDG_CONFIG_HOME` before Python starts and use the proper Qt/Nix shell.
|
||
|
|
- Piker and the installed Tractor pytest plugin do not provide a
|
||
|
|
repository-local process, shared-memory, or socket reaper. Never apply
|
||
|
|
historical broad `pkill -f tractor._child` guidance automatically.
|
||
|
|
|
||
|
|
## Test Layout
|
||
|
|
|
||
|
|
```text
|
||
|
|
tests/
|
||
|
|
conftest.py Piker options, config fixtures, Tractor plugin
|
||
|
|
_inputs/ tracked ledger/account fixtures
|
||
|
|
test_accounting.py config, ledgers, accounts, and position math
|
||
|
|
test_cli.py legacy CLI suite; hard-skipped
|
||
|
|
test_docker_services.py container integrations; optional deps, skipped
|
||
|
|
test_dpi_font.py Qt DPI/font behavior
|
||
|
|
test_ems.py actor, EMS, and paper-position behavior
|
||
|
|
test_feeds.py live Binance/Kraken feeds and shared memory
|
||
|
|
test_questrade.py obsolete credentialed tests; skipped
|
||
|
|
test_services.py pikerd/datad/feed/EMS actor lifecycle
|
||
|
|
test_watchlists.py deterministic watchlist JSON operations
|
||
|
|
```
|
||
|
|
|
||
|
|
## Change-To-Test Mapping
|
||
|
|
|
||
|
|
| Changed area | Run first | Caveat |
|
||
|
|
|---|---|---|
|
||
|
|
| `piker/watchlists/` | `tests/test_watchlists.py` | CLI suite is skipped |
|
||
|
|
| `piker/config.py` | `test_account_file_default_empty` | root-network test has a known mismatch |
|
||
|
|
| `piker/accounting/` | targeted accounting node | some cases use live/configured state |
|
||
|
|
| `piker/ui/_style.py`, `piker/ui/qt.py` | `tests/test_dpi_font.py` | GUI/config-isolated opt-in |
|
||
|
|
| `piker/service/_actor_runtime.py`, `_registry.py`, `_mngr.py` | `test_runtime_boot` | then `test_datad_spawn` |
|
||
|
|
| `piker/service/`, `piker/data/_daemon.py` | `test_datad_spawn` | feed lifecycle cases are live |
|
||
|
|
| `piker/data/feed.py`, `flows.py`, `_sharedmem.py`, `_sampling.py` | collect first | feed execution needs live permission |
|
||
|
|
| `piker/clearing/` | `test_ems_err_on_bad_broker` | multi-fill case is live/persisted |
|
||
|
|
| `piker/brokers/binance/`, `kraken/` | selected feed/accounting node | live network |
|
||
|
|
| `piker/brokers/ib/` | duplicated-market-ID node | controlled account config required |
|
||
|
|
| Docker/service adapters | `tests/test_docker_services.py` | optional deps and containers |
|
||
|
|
| project, lock, or Nix files | import check and safe collection | full collection is not safe by default |
|
||
|
|
|
||
|
|
Prefer deterministic filesystem/config tests, then local actor-runtime nodes,
|
||
|
|
then explicitly approved live broker, GUI, or container coverage.
|
||
|
|
|
||
|
|
## Quick Checks
|
||
|
|
|
||
|
|
```text
|
||
|
|
py313/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||
|
|
py313/bin/python -c 'import pathlib, piker, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
|
||
|
|
py313/bin/python -m pytest -p no:xonsh -q tests/test_watchlists.py
|
||
|
|
py313/bin/python -m pytest -p no:xonsh -q tests/test_accounting.py::test_account_file_default_empty
|
||
|
|
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_services.py::test_runtime_boot
|
||
|
|
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_services.py::test_datad_spawn
|
||
|
|
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_ems.py::test_ems_err_on_bad_broker
|
||
|
|
```
|
||
|
|
|
||
|
|
## Known Outcomes
|
||
|
|
|
||
|
|
- The current root-checkout `py313` fails `import piker` outside the Nix shell
|
||
|
|
with `ImportError: PyQtGraph requires one of PyQt5, PyQt6, PySide2 or
|
||
|
|
PySide6`. This is an incomplete environment, not an application regression.
|
||
|
|
- `tests/test_accounting.py::test_root_conf_networking_section` currently
|
||
|
|
expects `network.tsdb`, which is absent from the tracked config template.
|
||
|
|
Match the current `KeyError: 'tsdb'` before classifying it as the known
|
||
|
|
repository mismatch.
|
||
|
|
- `tests/test_docker_services.py` is marked skipped but imports
|
||
|
|
`elasticsearch` first. Without the `dbs` group it fails collection rather
|
||
|
|
than skipping.
|
||
|
|
- `tests/test_questrade.py` is marked skipped but imports undeclared `asks`
|
||
|
|
through the legacy broker module before marks apply.
|
||
|
|
- `test_open_orders_reloaded` and `test_dark_order_clearing` in
|
||
|
|
`tests/test_ems.py` contain only ellipsis bodies. A pass does not verify the
|
||
|
|
named behavior.
|
||
|
|
- Treat `.pytest_cache` feed IDs using old FQME forms as stale historical
|
||
|
|
cache, not current known failures.
|
||
|
|
|
||
|
|
Do not classify every `TooSlowError`, timeout, or child survivor as the known
|
||
|
|
Tractor teardown wedge. Match the selected node and the documented
|
||
|
|
second-runtime/lingering-child signature.
|
||
|
|
|
||
|
|
## Tractor Runtime Notes
|
||
|
|
|
||
|
|
The suite loads `tractor._testing.pytest`. Defaults are the `trio` spawn
|
||
|
|
backend and TCP transport. Registry addresses are normally session-unique,
|
||
|
|
except `tests/test_services.py::test_runtime_boot`, which binds
|
||
|
|
`127.0.0.1:6666`. Check that fixed port only for that node; ordinary tests do
|
||
|
|
not require Piker's production `127.0.0.1:6116` registry address.
|
||
|
|
|
||
|
|
Current repository concurrency notes document an intermittent second
|
||
|
|
in-process `pikerd` boot wedge with a lingering broker child and unread parent
|
||
|
|
IPC bytes. Use the outer timeout for actor-heavy nodes. Retry an exact command
|
||
|
|
once only after status `124` or `143`, and report both attempts.
|
||
|
|
|
||
|
|
Ordinary actor tests support normal capture. Diagnose a capture-dependent hang
|
||
|
|
by retrying only the exact node with `-s`. Standard `--pdb` with
|
||
|
|
`open_test_pikerd` requires `-s`; Tractor's `--tpdb` is a separate option.
|
||
|
|
|
||
|
|
After abnormal exit, inspect only descendants, sockets, and shared-memory
|
||
|
|
objects attributable to the exact pytest session. Ask before signaling or
|
||
|
|
unlinking anything.
|