Error when adding --strategy=TypeScriptCompile=worker

[mattem]

The following error is thrown when using the worker strategy with the Typescript Compiler

bazel.rc

build --strategy=TypeScriptCompile=worker
build --symlink_prefix=dist/

output

INFO: Analysed target //app:devserver (2 packages loaded).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_matt/3fed23a2ef51f4f0c7bcf4b6ca1a97a8/external/rxjs/BUILD.bazel:7:1: Compiling TypeScript (devmode) @rxjs//:lib failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at /private/var/tmp/_bazel_matt/3fed23a2ef51f4f0c7bcf4b6ca1a97a8/bazel-workers/worker-0-TypeScriptCompile.log ---8<---8<---
Starting TypeScript compiler persistent worker...

/private/var/tmp/_bazel_matt/3fed23a2ef51f4f0c7bcf4b6ca1a97a8/execroot/fabrik/external/build_bazel_rules_typescript/internal/tsc_wrapped/worker.ts:117
      process.stdout.write(new workerpb.WorkResponse()
                           ^
TypeError: workerpb.WorkResponse is not a constructor
    at Socket.<anonymous> (/private/var/tmp/_bazel_matt/3fed23a2ef51f4f0c7bcf4b6ca1a97a8/execroot/fabrik/external/build_bazel_rules_typescript/internal/tsc_wrapped/worker.ts:117:28)
    at emitNone (events.js:106:13)
    at Socket.emit (events.js:208:7)
    at emitReadable_ (_stream_readable.js:513:10)
    at emitReadable (_stream_readable.js:507:7)
    at addChunk (_stream_readable.js:274:7)
    at readableAddChunk (_stream_readable.js:250:11)
    at Socket.Readable.push (_stream_readable.js:208:10)
    at Pipe.onread (net.js:594:20)
---8<---8<--- End of log ---8<---8<---
Target //app:devserver failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 25.220s, Critical Path: 8.61s
INFO: 14 processes: 6 darwin-sandbox, 8 local.
FAILED: Build did NOT complete successfully

bazel version:

release: release 0.14.1-homebrew

rules_typescript version:

http_archive(
    name = "build_bazel_rules_typescript",
    url = "https://github.com/bazelbuild/rules_typescript/archive/0.15.0.zip",
    strip_prefix = "rules_typescript-0.15.0",
    sha256 = "1aa75917330b820cb239b3c10a936a10f0a46fe215063d4492dd76dc6e1616f4",
)

I tried hacking around to change worker.ts to the protobuf js syntax, but I hit errors about invalid buffers and out of range indexes, and couldn't find any docs on the API between Bazel server and the workers. If someone can provide some in site I can put a PR together if this is the right direction.

const out = workerpb.WorkResponse
          .encodeDelimited({ exit_code: exitCode, output: consoleOutput })
          .finish();
process.stdout.write(out);

[alexeagle]

Looks like version skew to me.

We ask node to resolve the 'protobufjs' library, and I suspect we end up getting the version installed in your project, if you have a direct dependency on it. That's because rules_typescript is downloaded by bazel rather than npm, so the usual npm rules of getting the right version of a transitive dependency don't apply.

The right long-term fix is for the bazel tooling to be fetched by npm - you can already do this with yarn add -D @bazel/typescript@0.15.0 and replace the http_archive with local_repository(name="build_bazel_rules_typescript", path="node_modules/@bazel/typescript") but that may introduce other issues (trying to load the ts_setup_workspace before it's fetched)

[mattem]

Yes, a project in our repo has a dependency on protobufjs@6.8.6.
I added in the @bazel/typescript@0.15.0 dep, but this doesn't seem to have changed things.

$ ibazel run //app:devserver
WARNING: no ibazel version found in your node_modules.
        We recommend installing a devDependency on ibazel so you use the same
        version as other engineers on this project.
        Using the globally installed version at /Users/matt/Documents/workspace/fabrik/node_modules/@bazel/ibazel
Querying for files to watch...
INFO: SHA256 (https://github.com/bazelbuild/skydoc/archive/0.1.4.zip) = 2885adbe581576b2b3ec906eeebc3692e45386651502411284c50979add0fa89
Error watching file /Users/matt/Documents/workspace/fabrik/tools/defaults/BUILD
Error: lstat /Users/matt/Documents/workspace/fabrik/tools/defaults/BUILD: no such file or directory
Runing //app:devserver
Launching with notifications
listening on :35729
INFO: Analysed target //app:devserver (2 packages loaded).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_matt/3fed23a2ef51f4f0c7bcf4b6ca1a97a8/external/rxjs/BUILD.bazel:7:1: Compiling TypeScript (devmode) @rxjs//:lib failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at /private/var/tmp/_bazel_matt/3fed23a2ef51f4f0c7bcf4b6ca1a97a8/bazel-workers/worker-0-TypeScriptCompile.log ---8<---8<---
Starting TypeScript compiler persistent worker...

/private/var/tmp/_bazel_matt/3fed23a2ef51f4f0c7bcf4b6ca1a97a8/execroot/fabrik/external/build_bazel_rules_typescript/internal/tsc_wrapped/worker.ts:117
      process.stdout.write(new workerpb.WorkResponse()
                           ^
TypeError: workerpb.WorkResponse is not a constructor
    at Socket.<anonymous> (/private/var/tmp/_bazel_matt/3fed23a2ef51f4f0c7bcf4b6ca1a97a8/execroot/fabrik/external/build_bazel_rules_typescript/internal/tsc_wrapped/worker.ts:117:28)
    at emitNone (events.js:106:13)
    at Socket.emit (events.js:208:7)
    at emitReadable_ (_stream_readable.js:513:10)
    at emitReadable (_stream_readable.js:507:7)
    at addChunk (_stream_readable.js:274:7)
    at readableAddChunk (_stream_readable.js:250:11)
    at Socket.Readable.push (_stream_readable.js:208:10)
    at Pipe.onread (net.js:594:20)
---8<---8<--- End of log ---8<---8<---
Target //app:devserver failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 24.573s, Critical Path: 10.34s
INFO: 15 processes: 7 darwin-sandbox, 8 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
Error starting process: fork/exec /var/folders/l1/01kc79ss1tv3g4kntxhhvpm80000gn/T/bazel_script_path268784147: permission denied
Run start failed fork/exec /var/folders/l1/

Did a bazel clean --expunge before running

[alexeagle]

We are digging into module loading and dependency management, it's a big change with a lot of implications and dependencies.

In the meantime, the workaround is to install the protobufjs@5.0.0 as a devDependency of your project so that it appears in the top-level node_modules folder.

[gregmagolan]

The principled fix for this will be to pull rules_typescript from npm (@bazel/typescript package) and install it as a local_repository path = "node_modules/@bazel/typescript". npm/yarn will install the appropriate protobufjs dependency for @bazel/typescript and if your project depends on a newer version of protobufjs, the rules_typescript version will be hoisted into node_modules/@bazel/typescript/node_modules and the latest rules_nodejs will ensure it will be resolved from there when running tsc_wrapped.

As mentioned above, the principled fix depends on the latest rules_nodejs which in turn depends on the latest angular bazel rules. The latest angular code is currently broken when building with Bazel (angular/angular#24521). I'm currently working on landing the last few pieces so that user's can build the latest angular with bazel.

Closing this as there is a current work-around and a planned principled fix.

[gregmagolan]

@mattem If your project is not using angular then you could use the latest @bazel/typescript npm package combined with the latest rules_nodejs and have this working with your project depending on a newer version of protobufjs. Is this your use case?

If so, you'll also have to add the @build_bazel_rules_typescript//:node_modules label to the list of srcs in your //:node_modules filegroup for the version of protobufjs that tsc_wrapped requires to be resolved.

[mattem]

@gregmagolan This project also uses Angular, so it looks like we will also be waiting for angular/angular#24521 to land.

[gregmagolan]

@mattem angular/angular-bazel-example#160 was merged in and https://github.com/alexeagle/angular-bazel-example is now building Angular 6.1.2 from source with Bazel.

It also pulls in rules_typescript from its npm package @bazel/typescript which will allow the main package.json to have a different protobufjs version that what rules_typescript requires. The rules_typescript protobufjs dep (v5.0.0) would end up in node_modules/@bazel/typescript/node_modules/protobufjs and it will be resolved from in there if @build_bazel_rules_typescript//:node_modules is included in your //:node_modules filegroup (See https://github.com/alexeagle/angular-bazel-example/blob/master/BUILD.bazel#L51).