relay-compiler 13.0.1 fails when schema is in __generated__ directory

[hanford]

Hello, I have a project structured like this:

root/
├─ apps/
│  ├─ frontend/
│  │  ├─ pages/
│  │  ├─ components/
│  │
│  ├─ backend/
│  │  ├─ schemas/
│  │  │  ├─ __generated__/
│  │  │  │  ├─ main.graphql
|
├─ package.json (where relay-compiler is)

I've updated the application to work with Relay 13, and everything works locally, but on CircleCI relay-compiler began failing with a pretty cryptic error:

pnpm run relay
child_process.js:383
      ex = new Error('Command failed: ' + cmd + '\n' + stderr);
           ^

Error: Command failed: pnpm run relay
[default] compiling...
thread 'main' panicked at 'no entry found for key', crates/relay-compiler/src/build_project/build_schema.rs:37:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

    at ChildProcess.exithandler (child_process.js:383:12)
    at ChildProcess.emit (events.js:400:28)
    at maybeClose (internal/child_process.js:1058:16)
    at Socket.<anonymous> (internal/child_process.js:443:11)
    at Socket.emit (events.js:400:28)
    at Pipe.<anonymous> (net.js:686:12) {
  killed: false,
  code: 1,
  signal: null,
  cmd: 'pnpm run relay',
  stdout: '\n' +
    '> *******@0.0.0 relay /home/circleci/project\n' +
    '> relay-compiler\n' +
    '\n' +
    ' ELIFECYCLE  Command failed with exit code 101.\n',
  stderr: '[default] compiling...\n' +
    "thread 'main' panicked at 'no entry found for key', crates/relay-compiler/src/build_project/build_schema.rs:37:17\n" +
    'note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\n'

I was able to SSH into the machine and after some trial and error was able to get relay-compiler to work once I copied the schema out of the __generated__ directory.

I wonder if there is something going on with the __generated__ namespace.

We could change our code so this works without moving stuff around, but this might trip others up too

My relay configuration looks like this:

  "relay": {
    "src": "./",
    "artifactDirectory": "./relay-types",
    "schema": "./apps/backend/schemas/__generated__/main.graphql",
    "exclude": [
      "**/node_modules/**/*",
       ...
    ],
    "language": "typescript",
    "noFutureProofEnums": true,
    "customScalars": {
      "NonEmptyString": "string",
      "EntityId": "EntityId",
      "UnixTimestamp": "Int",
      "DateTime": "DateString",
      "Date": "DateString",
      "Time": "string"
    }
  }

[hanford]

It seems that beyond the __generated__ issue, the new compiler isn't always able to find the configuration for my app.

I've landed on having a relay.config.json that looks like this:

{
  "src": "./",
  "artifactDirectory": "./relay-types",
  "schema": "./apps/backend/schemas/main.graphql",
  "exclude": [
    "**/node_modules/**/*",
     ...
  ],
  "language": "typescript",
  "noFutureProofEnums": true,
  "customScalars": {
    "NonEmptyString": "string",
    "EntityId": "EntityId",
    "UnixTimestamp": "Int",
    "DateTime": "DateString",
    "Date": "DateString",
    "Time": "string"
  }
}

and invoking the compiler through a script in my package.json

relay-compiler --src . --schema ./apps/weaver/schemas/main.graphql relay.config.json

which works locally on OS X, but I'm consistently getting the following on CI

relay-compiler --src . --schema ./apps/weaver/schemas/main.graphql relay.config.json
[default] compiling...
thread 'main' panicked at 'no entry found for key', crates/relay-compiler/src/build_project/build_schema.rs:37:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[alunyov]

Is it possible to create a simple repro for this error somehow with github actions (or otherwise)?

and invoking the compiler through a script in my package.json
relay-compiler --src . --schema ./apps/weaver/schemas/main.graphql relay.config.json

I think we always use relay.config.json and ignore CLI arguments, when both are passed. We should probably warn about this. Can you try removing --src and --schema?

[hanford]

@alunyov I have a repro which is structured nearly the same. It's how I was originally trying to structure the project, happy to move anything around though if you have ideas.

https://github.com/hanford/relay-compiler-ci

In the repro the type files generate fine locally, but error in the GH action.

I've tried a lot of different approaches for loading my config (both in my private app and in the repro), The bug seems to have to do with non CLI config options and (or) your schema is in a __generated__ directory.

1. [FAIL] First attempt was putting all of my relay config in <rootDir>/package.json#relay: https://github.com/hanford/relay-compiler-ci/runs/4778711616?check_suite_focus=true

2. [FAIL] Second attempt was all of my relay config in <rootDir>/package.json#relay and schema moved out of generated: https://github.com/hanford/relay-compiler-ci/runs/4778790426?check_suite_focus=true

3. [FAIL] Third attempt was invoking relay compiler with CLI args (relay-compiler --src . --schema ...__generated__: https://github.com/hanford/relay-compiler-ci/runs/4778819704?check_suite_focus=true

4. [PASS -ish] Fourth attempt was the same approach as the third, but moving the schema out of __generated__: https://github.com/hanford/relay-compiler-ci/actions/runs/1683718692

With the fourth attempt there is a new problem though, not all of the relay config is used. i.e. The subset of CLI args work, but I'm not able to use exclude/customScalars etc, as they're not supported by the CLI and it doesn't seem relay-compiler looks for the config and reliably finds it.

5. [FAIL] Fifth attempt was invoking relay compiler with a config file, similar to the first 2, this works locally but fails on CI: https://github.com/hanford/relay-compiler-ci/runs/4779004424?check_suite_focus=true

--

There are some branches you can click around in the repro and see the different attempts. Feel free to fork the repo or I can give you push access

--

The new compiler is a gamer changer for us, I'm incredibly excited to roll it out @ our company.

With the JS compiler it takes ~10 seconds to generate 2.6k TS type files, with the new rust based compiler it takes ~1 second.

[alunyov]

A quick update. The bug here is related to exclude and the way we handle them in glob_file_source. Or maybe we do not handle excludes in watchman (I need to double-check).

It looks like locally you're using watchman to find files, and everything works "correctly" (it finds the schema files). But on CI the watchman is not installed, and the compiler is using glob file source, and the schema files are in excludes and we filter them out.

To reproduce this locally run:

FORCE_NO_WATCHMAN=1 pnpm relay

I'll take a look at the possible fix. To unblock yourself: 1) do not exclude backend files (temporary), or 2) copy schema file to a different place, outside of excludes.

[hanford]

1. do not exclude backend files (temporary)

Seems to work in my private repo, thanks for the quick unblock @alunyov!

[RichardLindhout]

This was also a problem in previous versions if I'm not mistaken. I have a different directory generated_schema