[Bug] stdout maxBuffer exceeded

[KnisterPeter]

Hi,

while preparing my project to build with lerna I'll encountered this error.
Any hint on what is going on here? I've done lerna boostrap before.
It may be relevant to note that the build script in my packages are compiled using typescript and I'm using debian linux with node 4.x.

~/source/paeckchen (multiple-packages)$ ./node_modules/.bin/lerna run build
Lerna v2.0.0-beta.23
Independent Versioning Mode
Errored while running npm script 'build' in 'paeckchen-core'
Error: stdout maxBuffer exceeded
    at Socket.<anonymous> (child_process.js:266:14)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at readableAddChunk (_stream_readable.js:153:18)
    at Socket.Readable.push (_stream_readable.js:111:10)
    at Pipe.onread (net.js:531:20)
Errored while running RunCommand.execute
Error: stdout maxBuffer exceeded
    at Socket.<anonymous> (child_process.js:266:14)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at readableAddChunk (_stream_readable.js:153:18)
    at Socket.Readable.push (_stream_readable.js:111:10)
    at Pipe.onread (net.js:531:20)

[jamiebuilds]

@gigabo didn't you fix something with this?

[KnisterPeter]

I've tried to build 'paeckchen-core' inside the packages folder with npm run build and it does fail but there is at least some output.
The problem here was that with lerna boostrap an npm install was done but the postinstall script did not execute. After fixing this the build was fine and now lerna run build does work as well.

There might be two different problems here, one with lerna boostrap and one with stdout and lerna run task.

[jamiebuilds]

I think the issue is that we're using an api that stores the buffer until the script is completed rather than streaming it.

[gigabo]

The fix in #217 was specifically for output from npm during bootstrap (which was being ignored anyway).

The output from npm run scripts is intentionally captured so that the output from different packages doesn't get interleaved.

There are two potential solutions here:

1. Increase maxBuffer.
2. Allow output from npm run scripts to be interleaved.

[jamiebuilds]

What would people prefer of those two options?

[KnisterPeter]

I would prefer increasing the maxBuffer.
Interleaved output in the run scripts may be completly unreadable.

[doug-wade]

I would prefer the output of npm run scripts to not be interleaved -- it would make it prohibitively difficult to read the output if you used the script to inform (for instance, when running a task that runs david for every dependency, you wouldn't be able to tell where deps were updated)

[jamiebuilds]

What would be a sane choice for the maxBuffer then?

[doug-wade]

What would be a sane choice for the maxBuffer then?

Aye, there's the rub 😉 -- npm silly logs can get pretty large; I've seen them be in excess of a mb.

[KnisterPeter]

I think a sane default would be some megabyte but there should be a configuration option in the lerna.json to define it.
Lerna by itself could never know how much log output will be generated.

[jamiebuilds]

If we were just creating a string out of it would we have to mess with maxBuffer at all?

[jamiebuilds]

I think a sane default would be some megabyte but there should be a configuration option in the lerna.json to define it.

I would like to avoid the default solution to things always being another config option. I want to keep things simple and not end up with users having hundreds of options to configure

[rtsao]

What about using https://github.com/feross/multistream (in conjunction with child_process.spawn)? Then output stream order could be preserved.

[jamiebuilds]

Maybe, never used it before.

[glortho]

+1. we generally can't use lerna bootstrap or lerna run <some build> because of this.

[screendriver]

Any news about that? I'm running into the same issue here.

[gigabo]

@rygine increased maxBuffer in Asini in asini/asini#16.

I've ported that over here in #442.

[gigabo]

maxBuffer has been increased in lerna@2.0.0-beta.31. Hope that does the trick!

[marcelkottmann]

The problem still persists. If anyone else is running into this: lerna cli option --stream fixes this for me.

[vjpr]

@PePe79 lerna bootstrap --stream did not work for me with 2.5.1.

---

Seems it was only supported in run command. @hzoo Can we have --stream for bootstrap?

[vjpr]

@gigabo This is still an issue for me. Its blocking my use of lerna entirely. What is the quickest fix? Maybe add a maxBuffer flag?

Or is there a way to add a flag to my package manager tool?

https://github.com/lerna/lerna#--npm-client-client

E.g. lerna bootstrap --npm-client-flags --logger=off

---

Tried lerna bootstrap --concurrency=1 --max-buffer=99999999999999 and it still gives me exceeded error.

[lock]

This thread has been automatically locked because there has not been any recent activity after it was closed. Please open a new issue for related bugs.