Invalid Array Buffer Length #396
Comments
Downgraded to 4.4.5 and was able to reproduce the error. |
I want to suggest the Storj team to always do one or two 100GB uploads before each core release because if a bug is present, it will 100% guaranteed surface with these very large files. I have been running core for at least a month now and have never managed to upload a 100GB file successfully, a 100GB upload test at the Storj office should become a routine task. |
We can't test everything with our own little farmer network. The last 1GB upload problem had to do with tunnel nodes not sending data channel close. We are not able to predict issues like that. That is the reason we are running a test group. The cummunity has a better farmer network than we could ever setup. That way we find all issues. |
same error: command
error
|
@MeijeSibbel your concurrency is at 20. You're adding too much data into memory which is causing this error. Shawn is doing the same thing. We're gonna warn about high concurrency values in the logs now. See this issue |
Don't believe that is correct. I set both file and shard concurrency to 1 On Sep 8, 2016 11:20 AM, "Alexander Leitner" notifications@github.com
|
Gordon says prove it. Include the command next time |
How much we betting on this? I'm uploading a 100 GB file, which would give I can try it myself but it's going to take me bit to SSH a VM with my On Sep 8, 2016 12:00 PM, "Gordon Hall" notifications@github.com wrote:
|
How much ram is on your computer? |
Trying to remember what computer I tried this on. I believe it was the On Sep 8, 2016 12:17 PM, "Alexander Leitner" notifications@github.com
|
If it's 8gb then 2 shards would cause this issue |
Command
Error
|
Replicated by @MeijeSibbel. Here are my suggestions:
|
I can confirm that it is indeed a lack of memory issue, once the memory reaches 100% it crashes. |
Package Versions
Replace the values below using the output from
npm list storj
. Usenpm list -g storj
if installed globally.Replace the values below using the output from
node --version
.I'm using the wrong version number. But I think this is reproducible across versions. Will downgrade and try again.
Expected Behavior
Upload proceeds until completion.
Actual Behavior
Please describe the program's actual behavior. Please include any stack traces
or log output in the back ticks below.
Steps to Reproduce
Please include the steps the reproduce the issue, numbered below. Include as
much detail as possible.
The text was updated successfully, but these errors were encountered: