Commit Graph

13 Commits (513646c802788acc11649a8f1ac20c855843ecc7)

Author SHA1 Message Date
Tyler Goodlet 02b7d6cd19 Add prepend support to shm system 2020-12-19 16:49:35 -05:00
Tyler Goodlet 247b5fa2ec Tidy up doc string 2020-12-19 16:06:28 -05:00
Tyler Goodlet 8aede3cbcb Add field diffing on failed push 2020-12-19 16:05:22 -05:00
Tyler Goodlet c625dc90f1 Use new global var stack from tractor 2020-12-19 15:05:29 -05:00
Tyler Goodlet acf8aeb33e Allocate space for 2d worth of 5s bars 2020-12-19 15:05:03 -05:00
Tyler Goodlet c1109ee3fb Add license headers to pertinent files 2020-11-06 12:23:14 -05:00
Tyler Goodlet cb72662350 Add warnings for shm cache misses 2020-10-15 15:07:56 -04:00
Tyler Goodlet 8a4528c006 Always ask backend for ohlc dtype 2020-10-02 12:13:50 -04:00
Tyler Goodlet e3e219aa4b Add multi-symbol-buffer increment support 2020-10-02 12:13:50 -04:00
Tyler Goodlet 38469bd6ef Slight rework: shm API
Add an internal `_Token` to do interchange (un)packing for passing
"references" to shm blocks between actors.  Part of the token involves
providing the `numpy.dtype` in a cross-actor format.  Add a module
variable for caching "known tokens" per actor.  Drop use of context
managers since they tear down shm blocks too soon in debug mode and
there seems to be no reason to unlink/close shm before the process has
terminated; if code needs it torn down explicitly, it can.
2020-10-02 12:13:50 -04:00
Tyler Goodlet cd540fd07e Cleanups 2020-10-02 12:13:50 -04:00
Tyler Goodlet 17491ba819 Disconnect stdlib's resource_tracker, fix .push()
Logic in `SharedArray.push()` was totally wrong.
Remove all the `multiprocessing.resource_tracker` crap such that we
aren't loading an extra process at every layer and we don't get tons of
errors when cleaning on in an SC way.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 712e36b9d5 First draft of a shared numpy array sub-system
This adds a shared memory "incrementing array" sub-sys interface
for single writer, multi-reader style data passing. The main motivation
is to avoid multiple copies of the same `numpy` array across actors
(plus now we can start being fancy like ray).

There still seems to be some odd issues with the "resource tracker"
complaining at teardown (likely partially to do with SIGINT stuff) so
some further digging in the stdlib code is likely coming.

Pertains to #107 and #98
2020-10-02 12:13:50 -04:00