pikers
/
piker
4
1
Fork 0
Code Issues 20 Pull Requests 11 Packages Projects Releases Wiki Activity
piker/.claude/skills/run-tests/test-harness-reference.md

12 KiB
Raw Blame History

Test Harness Reference: piker

This repository-local file supplements the canonical /run-tests skill. Keep shared execution, worktree, failure-inspection, and cleanup policy in the canonical SKILL.md; keep Piker commands, paths, fixtures, and known outcomes here.

Project And Environment

  • Project/import: piker
  • Test root: tests/
  • Supported Python: >=3.12,<3.14
  • Preferred complete environment: worktree-local py313 inside the current nix develop shell
  • The flake shell pins CPython 3.13 and sets UV_PROJECT_ENVIRONMENT=py313.
  • Verify the interpreter, package resolution, and dependency import before running tests. A bare py313 may lack the Qt binding supplied by Nix.

Use an already-provisioned py313 only when import piker succeeds:

py313/bin/python -m pytest -p no:xonsh

If direct environment paths are unavailable, an existing uv environment can be used without changing it, subject to the same import check:

UV_PROJECT_ENVIRONMENT=py313 uv run --frozen --no-sync python -m pytest -p no:xonsh

Ask before running provisioning commands such as:

UV_PROJECT_ENVIRONMENT=py313 uv sync --dev --all-extras --no-group lint
nix develop
nix-shell default.nix

nix develop is the current Wayland/Qt 6 shell. default.nix is the current X11 shell. Do not use develop.nix for current testing; it retains the old Python 3.11, Poetry, and Qt 5 stack.

The current root-checkout py313 resolves piker locally but fails import piker outside Nix because PyQtGraph cannot import PyQt or PySide. Do not treat that environment as test-ready and do not enter nix develop without approval: its shell hook may recreate and sync py313.

Plain uv sync does not include the testing group. The dbs dependency group is also absent from normal dev-shell provisioning.

Commands

Base command in the preferred environment:

py313/bin/python -m pytest -p no:xonsh

The explicit -p no:xonsh is required. The tracked comments-only pytest.ini takes precedence over pyproject.toml, so the intended addopts = "-p no:xonsh" and testpaths = ["tests"] are inactive. Always pass a test path or node ID explicitly.

Package-resolution check that does not import Piker’s dependencies:

py313/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'

Dependency import check, required before collection or execution:

py313/bin/python -c 'import pathlib, piker, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'

Safe core collection check:

py313/bin/python -m pytest -p no:xonsh -q --collect-only tests/test_watchlists.py tests/test_accounting.py tests/test_services.py tests/test_ems.py tests/test_feeds.py tests/test_cli.py

Default first-pass flags are -q -x --tb=short --no-header unless the user requests otherwise. For actor-heavy tests, use one file or node per process with an outer timeout:

timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q <one-file-or-node>

If an actor-heavy command exits 124 or 143, retry that exact command once and report both attempts. Never convert a retry pass into an unconditional pass, and do not retry assertion, import, collection, or configuration failures.

Scope And Path Resolution

Resolve bare test filenames beneath tests/. Preserve complete node IDs and apply -k only within explicitly selected paths. There is no marker-based offline/full-suite split, so never use pytest tests as a deterministic default.

Deterministic or local first-pass targets:

  • tests/test_watchlists.py
  • tests/test_accounting.py::test_account_file_default_empty
  • tests/test_services.py::test_runtime_boot
  • tests/test_services.py::test_datad_spawn
  • tests/test_ems.py::test_ems_err_on_bad_broker

Require explicit authorization before running:

  • tests/test_feeds.py - live Binance/Kraken feeds;
  • tests/test_services.py::test_ensure_datafeed_actors - live Kraken feed;
  • tests/test_services.py::test_ensure_ems_in_paper_actors - paper EMS with live Kraken symbology/feed access;
  • tests/test_ems.py::test_multi_fill_positions - live backend setup and persisted state;
  • tests/test_accounting.py::test_paper_ledger_position_calcs - tracked fixtures plus possible live symcache generation;
  • tests/test_accounting.py::test_ib_account_with_duplicated_mktids - active broker/account configuration and state writes;
  • tests/test_dpi_font.py - Qt/UI import and user-config side effects;
  • tests/test_docker_services.py - optional dependencies and containers;
  • tests/test_questrade.py - obsolete credentialed imports.

tests/test_cli.py is currently hard-skipped. It is not an active CLI regression gate.

Never execute piker store anal or piker store ldshm as tests. They are mutating or interactive operational commands.

Project-Specific Flags And Backend Matrix

Flag Purpose
--ll LEVEL Piker log level
--confdir PATH Override piker.config._config_dir
--spawn-backend trio|mp_spawn|mp_forkserver Tractor process backend
--tpt-proto PROTO Tractor transport; one protocol per session
--tpdb / --debug-mode Tractor plugin debug mode
--pdb Standard pytest debugger
-s No capture; required with --pdb and open_test_pikerd

Do not invent network, offline, docker, gui, or broker markers; none currently exists. CI=1 is not an offline selector: feed tests still leave a live Kraken case enabled. Current Piker tests exercise TCP; do not assume the whole suite supports UDS merely because the Tractor plugin exposes it.

Fixture Invariants

  • The session confdir fixture does nothing unless --confdir is passed. Its claimed tests/data fallback is not implemented and that directory is absent.
  • Function-scoped tmpconfdir changes process-global config state and does not restore the previous path. Use separate pytest processes when diagnosing state leakage.
  • open_test_pikerd passes the temporary config path to child actors through tractor_runtime_overrides.
  • tests/_inputs/trades_binance_paper.toml and tests/_inputs/account.binance.paper.toml are used in place. Accounting contexts can write them on exit. Inspect git diff -- tests/_inputs after any selected accounting case.
  • Importing tests/test_dpi_font.py constructs module-global font objects before fixtures can isolate config. If explicitly requested, isolate XDG_CONFIG_HOME before Python starts and use the proper Qt/Nix shell.
  • Piker and the installed Tractor pytest plugin do not provide a repository-local process, shared-memory, or socket reaper. Never apply historical broad pkill -f tractor._child guidance automatically.

Test Layout

tests/
  conftest.py               Piker options, config fixtures, Tractor plugin
  _inputs/                  tracked ledger/account fixtures
  test_accounting.py        config, ledgers, accounts, and position math
  test_cli.py               legacy CLI suite; hard-skipped
  test_docker_services.py   container integrations; optional deps, skipped
  test_dpi_font.py          Qt DPI/font behavior
  test_ems.py               actor, EMS, and paper-position behavior
  test_feeds.py             live Binance/Kraken feeds and shared memory
  test_questrade.py         obsolete credentialed tests; skipped
  test_services.py          pikerd/datad/feed/EMS actor lifecycle
  test_watchlists.py        deterministic watchlist JSON operations

Change-To-Test Mapping

Changed area Run first Caveat
piker/watchlists/ tests/test_watchlists.py CLI suite is skipped
piker/config.py test_account_file_default_empty root-network test has a known mismatch
piker/accounting/ targeted accounting node some cases use live/configured state
piker/ui/_style.py, piker/ui/qt.py tests/test_dpi_font.py GUI/config-isolated opt-in
piker/service/_actor_runtime.py, _registry.py, _mngr.py test_runtime_boot then test_datad_spawn
piker/service/, piker/data/_daemon.py test_datad_spawn feed lifecycle cases are live
piker/data/feed.py, flows.py, _sharedmem.py, _sampling.py collect first feed execution needs live permission
piker/clearing/ test_ems_err_on_bad_broker multi-fill case is live/persisted
piker/brokers/binance/, kraken/ selected feed/accounting node live network
piker/brokers/ib/ duplicated-market-ID node controlled account config required
Docker/service adapters tests/test_docker_services.py optional deps and containers
project, lock, or Nix files import check and safe collection full collection is not safe by default

Prefer deterministic filesystem/config tests, then local actor-runtime nodes, then explicitly approved live broker, GUI, or container coverage.

Quick Checks

py313/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
py313/bin/python -c 'import pathlib, piker, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
py313/bin/python -m pytest -p no:xonsh -q tests/test_watchlists.py
py313/bin/python -m pytest -p no:xonsh -q tests/test_accounting.py::test_account_file_default_empty
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_services.py::test_runtime_boot
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_services.py::test_datad_spawn
timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_ems.py::test_ems_err_on_bad_broker

Known Outcomes

  • The current root-checkout py313 fails import piker outside the Nix shell with ImportError: PyQtGraph requires one of PyQt5, PyQt6, PySide2 or PySide6. This is an incomplete environment, not an application regression.
  • tests/test_accounting.py::test_root_conf_networking_section currently expects network.tsdb, which is absent from the tracked config template. Match the current KeyError: 'tsdb' before classifying it as the known repository mismatch.
  • tests/test_docker_services.py is marked skipped but imports elasticsearch first. Without the dbs group it fails collection rather than skipping.
  • tests/test_questrade.py is marked skipped but imports undeclared asks through the legacy broker module before marks apply.
  • test_open_orders_reloaded and test_dark_order_clearing in tests/test_ems.py contain only ellipsis bodies. A pass does not verify the named behavior.
  • Treat .pytest_cache feed IDs using old FQME forms as stale historical cache, not current known failures.

Do not classify every TooSlowError, timeout, or child survivor as the known Tractor teardown wedge. Match the selected node and the documented second-runtime/lingering-child signature.

Tractor Runtime Notes

The suite loads tractor._testing.pytest. Defaults are the trio spawn backend and TCP transport. Registry addresses are normally session-unique, except tests/test_services.py::test_runtime_boot, which binds 127.0.0.1:6666. Check that fixed port only for that node; ordinary tests do not require Piker’s production 127.0.0.1:6116 registry address.

Current repository concurrency notes document an intermittent second in-process pikerd boot wedge with a lingering broker child and unread parent IPC bytes. Use the outer timeout for actor-heavy nodes. Retry an exact command once only after status 124 or 143, and report both attempts.

Ordinary actor tests support normal capture. Diagnose a capture-dependent hang by retrying only the exact node with -s. Standard --pdb with open_test_pikerd requires -s; Tractor’s --tpdb is a separate option.

After abnormal exit, inspect only descendants, sockets, and shared-memory objects attributable to the exact pytest session. Ask before signaling or unlinking anything.