Inconsistent Build Behavior & Errors While Building

[antoinne85]

🐛 bug report

I'm getting inconsistent build behavior. I get errors every time. But if I start making modifications to files, sometimes the build will succeed. Once it succeeds once, subsequent builds succeed. However, when debugging in Chrome, the debugger's position is out of sync with the file contents (I can tell that it's executing a different line than the UI indicates).

Once I get a successful build, I can quit and then attempt the build again and it will succeed, albeit with the above oddities. Clearing the output folder and cache takes me back to a non-buildable state (with the same source files).

I really hate bringing this to your team without a reproducible sample in-hand, but I've been searching and working on this issue for hours now and haven't made any meaningful progress towards understanding the issue or narrowing it down. I intend to ask some other engineers to build it in the morning to try to narrow it down further—maybe it's an environmental issue, though it seems somewhat unlikely.

🎛 Configuration (.babelrc, package.json, cli command)

Running yarn parcel test-index.html

//tsconfig.json
{
    "compilerOptions": {
        "target": "es5",
        "module": "commonjs",
        "outDir": "out",
        "sourceMap": true
    }
}

//.parcelrc
{
  "extends": "@parcel/config-default",
  "transformers": {
    "*.{ts,tsx}": ["@parcel/transformer-typescript-tsc"]
  }
}

🤔 Expected Behavior

Expected it to build/launch.

😯 Current Behavior

I'm getting the following error...

Server running at http://localhost:1234
⠼ Bundling...
node:internal/streams/buffer_list:97
    return this.head.data;
                     ^

TypeError: Cannot read properties of null (reading 'data')
    at BufferList.first (node:internal/streams/buffer_list:97:22)
    at howMuchToRead (node:internal/streams/readable:385:27)
    at TapStream.Readable.read (node:internal/streams/readable:430:7)
    at flow (node:internal/streams/readable:1012:34)
    at resume_ (node:internal/streams/readable:993:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

The precise text before the error varies (e.g. bundling, bundling runtime, packaging test-index.html, etc.).

💁 Possible Solution

I'm at a complete loss. I've never experienced this with other parcel-based typescript projects.

🔦 Context

I'm working on porting and TypeScript-ifying a large JavaScript library intended for the browser. I reached a point where I felt like it was ready to put through it's paces, but then hit this hurdle.

💻 Code Sample

I tried to pare this down to something that I could share or even narrow down the culprit. I tried deleting large hunks of code and building. Eventually I did get it to work. But then when clearing .parcel-cache and dist subsequent builds didn't work. So there seems to be some state at play other than just "what do the files look like on disk" that I've been unable to pin down.

Since I can't provide the environment, I want to at least describe the stateful behavior I'm seeing...

• Delete .parcel-cache and dist
• yarn parcel test-index.html

Server running at http://localhost:1234
⠴ Optimizing test-index.[hash].js...
node:internal/streams/buffer_list:97
    return this.head.data;
                     ^

TypeError: Cannot read properties of null (reading 'data')
    at BufferList.first (node:internal/streams/buffer_list:97:22)
    at howMuchToRead (node:internal/streams/readable:385:27)
    at TapStream.Readable.read (node:internal/streams/readable:430:7)
    at flow (node:internal/streams/readable:1012:34)
    at resume_ (node:internal/streams/readable:993:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)
error Command failed with exit code 1.

• Comment out an entire file (even though it's referenced elsewhere and so it creates an error in the code)
• yarn parcel test-index.html

Server running at http://localhost:1234
√ Built in 2.13s

• Uncomment that file and save it

Server running at http://localhost:1234
√ Built in 586ms

• CTRL+C
• yarn parcel test-index.html

Server running at http://localhost:1234
√ Built in 22ms

• CTRL+C
• Delete .parcel-cache and dist
• yarn parcel test-index.html

Server running at http://localhost:1234
⠼ Building runtime-6bd5baf8476e8c86.js...
node:internal/streams/buffer_list:97
    return this.head.data;
                     ^

TypeError: Cannot read properties of null (reading 'data')
    at BufferList.first (node:internal/streams/buffer_list:97:22)
    at howMuchToRead (node:internal/streams/readable:385:27)
    at TapStream.Readable.read (node:internal/streams/readable:430:7)
    at flow (node:internal/streams/readable:1012:34)
    at resume_ (node:internal/streams/readable:993:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)
error Command failed with exit code 1.

That's probably much more info than is truly useful...

🌍 Your Environment

Software	Version(s)
Parcel	2.0.1
Node	16.13.2
yarn	1.22.15
Operating System	Windows 10
TypeScript	4.5.4

[antoinne85]

I've continued chipping away at this, trying to get to a minimal reproducible project. I've been chipping away at the code. There's still quite a lot left, but I've reached a state where it doesn't matter what I do, the code moves from broken to functional. For example, some states that I've seen...

• Two lines of comments back to back, if I delete either one of them, it builds.
• If I change a variable name, it builds.
• If I delete function body, it builds.

It doesn't matter what file I'm in. There seems to be something I can do to make it switch from "it doesn't work" to "it works." And there's no rhyme or reason why. It almost feels like there's something going on with content length that's causing it to go sideways.

As an example...
I've got a section of code where I replaced a series of single-line comments that fails with:

    //1-------10--------20--------30--------40--------50--------60--------70------78
    //1-------10--------20--------30--------40--------50--------60--------70-----77
    //1-------10--------20--------30--------40--------50--------60--------70-----77

🔼 This also fails.
🔽 This succeeds.

    //1-------10--------20--------30--------40--------50--------60--------70-----77
    //1-------10--------20--------30--------40--------50--------60--------70-----77
    //1-------10--------20--------30--------40--------50--------60--------70-----77

So, 78+77+77=232 fails and 231 succeeds.

    //1-------10--------20--------30--------40--------50--------60--------70--------80
    //1-------10--------20--------30--------40--------50--------60--------70--------80
    //1-------10--------20--------30--------40--------50--------60--------7072

🔼 Fails.

    //1-------10--------20--------30--------40--------50--------60--------70--------80--------90-------100
    //1-------10--------20--------30--------40--------50--------60--------70--------80--------90-------100
    //1-------10--------20--------30--

🔼 Fails.
🔽 Succeeds.

    //1-------10--------20--------30--------40--------50--------60--------70--------80--------90-------100
    //1-------10--------20--------30--------40--------50--------60--------70--------80--------90-------100
    //1-------10--------20--------30-

This length thing would explain why I've had such a hard time pinning it down. I've been focused on "what's wrong with the code," deleting and changing sections. But it seems that the problem may actually be related to the amount of code in the file I'm modifying (they're not overly long—the file this experimentation was done on is only ~105 lines).

[JennaSys]

I am getting this same exact error:

⠏ Optimizing index.[hash].js...
node:internal/streams/buffer_list:97
    return this.head.data;
                     ^

TypeError: Cannot read properties of null (reading 'data')
    at BufferList.first (node:internal/streams/buffer_list:97:22)
    at howMuchToRead (node:internal/streams/readable:385:27)
    at TapStream.Readable.read (node:internal/streams/readable:430:7)
    at flow (node:internal/streams/readable:1012:34)
    at resume_ (node:internal/streams/readable:993:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

Software	Version(s)
Parcel	2.2.1
Node	16.13.2
npm	8.1.2
Operating System	Linux Mint 19.3

It works with serve but not build.

This build command fails:

"build": "NODE_ENV=production parcel build --no-source-maps src/index.html"

If I remove the --no-source-maps option from the build command it works:

"build": "NODE_ENV=production parcel build src/index.html"

When I tried it on an Ubuntu 18.04 OS with Node 14.17.1 it worked with the --no-source-maps option.

Edit: I just tested it on Windows 10 with Node 15.14.0 and it also worked there with the --no-source-maps option.
Edit2: It fails on MacOS with Node 16.13.1

[antoinne85]

Pulling the thread that @JennaSys found...
If I add the --no-source-maps flag, suddenly it builds successfully. And this is repeatable and predictable on the same source files.

So yarn parcel test-index.html will fail and yarn parcel test-index.html --no-source-maps will succeed. The behavior remains the same even if I leave .parcel-cache intact while switching back-and-forth between commands.

[antoinne85]

I already have to retract that previous statement...

That was the behavior in the pared down demo I was trying to pull together. When I took that technique back to the full project, it didn't change anything.

[JennaSys]

It did seem to be somewhat random at first for me too, but what I outlined was consistent after trying different combinations repeatedly (6-10 times).

I will say that I could have sworn this was working fine just a week ago with the same source and top level dependency versions. I first noticed this behavior yesterday after a full build from scratch.

I will continue testing different variations, but so far the common denominator in these 2 instances seems to be Node 16.13.2

[JennaSys]

OK, I think I may have found the culprit. By comparing the package-lock.json files between a recent build and an older working build of the same project, it's the lmdb dependency that is apparently causing the issue. It was updated to version 2.2.0 in the last few days.

Pinning this dependency to 2.1.7 makes my Parcel build work again.

  "devDependencies": {
    "lmdb": "2.1.7",
    "parcel": "^2.2.1"
  }

It appears to be a dependency of @parcel/cache which has it as "lmdb": "^2.0.2"

@antoinne85 can you see if this fixes it for you as well?

[garth]

I can confirm that reverting lmdb fixes the build.

  "resolutions": {
    "lmdb": "2.1.7"
  }

[JennaSys]

The problem does also appear to be specific to Node 16 as well, since it still seems to work with lmdb==2.2.0 on Node 14 & Node 15

Edit: Nevermind, I checked again and it breaks with Node 15 too.

[antoinne85]

@JennaSys I slapped lmdb@2.1.7 in my resolutions as @garth suggested and that seems to have put things back on the rails.

[antoinne85]

Thanks to the both of you, @garth and @JennaSys!

[devongovett]

cc. @kriszyp

[kriszyp]

I have not been able to reproduce this, and I'm not sure if anyone has come up with a reliable set of steps for reproducing this yet. If they do, I would be glad to investigate further. In the meantime, there a couple of performance optimizations that I recently made to lmdb-js in v2.2.0, that I had actually hoped would provide performance benefits to parcel, but possibly one of the could be the source of this regression. If these are only occurring in a closed environment, perhaps maybe someone would be graciously willing to try flipping some of these switches:

• v2.2 uses an alternate method of retrieving binary data if the entry is larger 64KB, that reduces memory copying. This seems like the most likely culprit because I believe parcel is a bit unique in using direct binary data (and no other lmdb-js users have noticed any issues), and I think there was an indication that this might be file size dependent. If someone is willing to test this theory, you can adjust this threshold; it is currently hard-coded on line 1336 & 1338 in /node_modules/lmdb/dist/index.cjs, and if you significantly increased that threshold (maybe over a gigabyte), could check if the error still occurs. The larger sizes certainly have been tested with unit tests, but perhaps some case was missed.
• v2.2 uses an async/delayed flushing of data to disk by default now, for better write performance. It is possible that this could cause some writes to be uncommitted if the process/thread was abruptly ended (with process.exit/worker.terminate). However this change was specifically not applied for Windows, so this should not be causing issues for @antoinne85 , at least. I am working on making this more foolproof, but again, don't think this is the issue.

[JennaSys]

@kriszyp I can confirm that increasing the threshold value in /node_modules/lmdb/dist/index.cjs does fix the issue in my case. I changed it from 0x10000 to 0x100000 and the build worked where it was failing before. Changing it back made it fail again, so it was repeatable.

[kriszyp]

@JennaSys Thank you for checking on this for me. I actually still have no idea what is going wrong, I can't see how this would be failing, but I will release a new version with this optimization disabled, until I can figure it out.

[kriszyp]

Ok, the main optimization that I believe was affecting parcel should be reverted in lmdb@v2.2.1. Again, if anyone is able to offer any more clues on how to reproduce this, would love to know, as I haven't been able to find any issues with how it was working.

[JennaSys]

@kriszyp, I was just using a simple React demo app I had laying around to test out a parcel transformer for Python that I developed when I encountered this issue. It's a bit of an odd example since the original source is Python, but since the issue happens after the transpilation process, it shouldn't really matter. You are more than welcome to use it to try and reproduce the issue yourself if it helps. (I did put an override in package.json for lmdb so you'll need to remove that.)

[kriszyp]

@JennaSys Thank you! I was able to reproduce this with your directions, and I believe I was able to implement a proper fix for the optimization (will release in v2.2.2, but v2.2.1 should be safe for now).

[JennaSys]

That's fantastic - and thanks for the quick turnaround @kriszyp!

[valkum]

We seem to encounter this bug in a monorepo case.
We use yarn and have two yarn workspaces for packages (A,B) that use parcel for building libraries.
A depends on B.

build step in package.json of both packages:

...
"build": "parcel build"
...

When running yarn build for a single package everything is fine. But when running yarn workspaces foreach -vptR --from 'A run build with an empty .parcel-cache dir we get a process exited (exit code 129) from yarn.
This sometimes happens for the first build (B) but more often for the second one (B).

We also get a coredump for /home/rf/.nvm/versions/node/v16.14.0/bin/node /home/rf/git/project/node_modules/parcel/lib/bin.js build . I can provide that coredump but I am not sure if that helps.

Attach strace -f to the whole process could not show any hints, as in this case the process does not segfault. Might be related to everything taking longer, which hints to the async flushing.

Using lmdb 2.1.7 seems to fix this issue for us as well.

[kriszyp]

@valkum Are you saying that you see this problem in lmdb 2.2.1 or 2.2.0? It should be fixed in 2.2.1, but let me know if that is not the case.

[valkum]

We see that with 2.2.1

[kriszyp]

@valkum Do you have any available repo/package I could try to reproduce this with? It sounds like this may be a different issue since I believe #7645 was fixed with lmdb@2.2.1?

[valkum]

As you think this might not be related to the possible fixed issues raised here, I opened a new issue for this. Feel free to close that new one as a dup if this resolves to being caused by the same possible issues raised here.

[mischnic]

Looks like this has been fixed