Pipe block operators proposal

Block-template operator

• Excellent parallel
  • map
  • filter
• Good parallel (local accumulators)
  • reduce (makes sure that size >= 1)
  • reduce0 (also takes zero element. Can accept zero-sized stream)
  • group_by (in fact it's reduce by google whitepaper)
  • partition_by (??? должен разделять по условию... формат???)
• Bad parallel
  • sort
  • unique (alias disctinct)
• Shrinkable
  • skip
  • limit
  • any
  • all
• Debug
  • peek(dst)
  • print (==peek(stdout))

Frame operators

• Pack N frames into 1.
• Unpack 1 frame into N. (in fact combos with pack-blocking)

Block operators

• Pack each block to make it operational over N-subblock-frame
  • unroll
  • vectorize

Alignment

• Array of objects (default)
• Object of arrays
  • Object of mixed arrays
    • Fusing same type into single array
    • Partial fusing
    • Rest trash into object
  • How to
    • align<unroll,separate> # Default=object of arrays, may be overriden
      • align_fuse 'field_a', 'field_b' # Force fuse N fields. Transitive. Must be same type in first implementations.
      • align_fuse_type 'type_name' # Force fuse for all fields of specified type
      • align_rest 'field_a' # Force field be not unrolled, but putted in "rest" object
    • align<unroll,type_fuse> # Default=object of arrays, all fields of same type fused
    • align<unroll,manual> # Default=all in "rest" object
      • align_unroll 'field' # Remove from rest. Multiple field same as align_fuse. So use 1-liners for separate unroll.

Black magic ???

• Loop Stream Detector size/format tuning
  • some info
  • intel info
• CPU cache line tuning
  • e.g. frame_size=1MB # Causes autotuner pack block-operations in blob that will precess 1MB data frame per tick. Round down, but at last 1 (will provide warning if constrain not met).
• CPU time quant granularity rounding/tuning (enough work for thread)
  • e.g. time_frame=1ms # similar to frame_size