The following query hangs forever.
$ printf '{a:1}\n' > n1.sup &&
printf '{a:2}\n' > n2.sup &&
printf '{a:1}\n{a:2}\n' > outer.sup &&
super -version &&
super -s -c "from './outer.sup' | values (from n*.sup | count())"
Version: v0.3.0-357-g0c71fac6a
Details
Repro is with super commit 0c71fac.
The repro above is a simplification of a hang I experienced when trying to follow the "logical equivalent" of the blend operator as described in the docs to get around the lack of spill-to-disk in the current vectorized implementation of blend. Attempting the following on the full set of GitHub Archive JSON input files triggers the same deadlock symptom:
$ super -dynamic -f parquet -o gha.parquet -c "from ./2023\-02\-08\-*.json.gz | values upcast(this, (from ./2023\-02\-08\-*.json.gz | aggregate blend(this)))"
While getting Claude's assistance with creating the simplified repro, it gave its take on the root cause and possible approach to a fix, which is written up in a Gist.
The following query hangs forever.
Details
Repro is with super commit 0c71fac.
The repro above is a simplification of a hang I experienced when trying to follow the "logical equivalent" of the
blendoperator as described in the docs to get around the lack of spill-to-disk in the current vectorized implementation ofblend. Attempting the following on the full set of GitHub Archive JSON input files triggers the same deadlock symptom:While getting Claude's assistance with creating the simplified repro, it gave its take on the root cause and possible approach to a fix, which is written up in a Gist.