Schema JSON validation fails with ESLint 4.2.0

[jseminck]

Tell us about your environment

• ESLint Version: 4.2.0
• Node Version: 6.9.2
• npm Version: 3.10.2

What parser (default, Babel-ESLint, etc.) are you using?
default

Please show your full configuration:
N/A

What did you do? Please include the actual source code causing the issue.

It seems that ESLint has upgraded its JSON validator for 4.2.0: #8852

Note: It's only a guess that the above change has caused the problems, but it's not fully confirmed.

This is causing issues for a rule in eslint-plugin-react:. It seems that the following schema validation is causing problems: https://github.com/yannickcr/eslint-plugin-react/blob/master/lib/rules/jsx-curly-spacing.js#L34

The produced error:

can't resolve reference #/definitions/basicConfig from id #
can't resolve reference #/definitions/basicConfigOrBoolean from id #
Warning: [object Object]:
        Configuration for rule "react/jsx-curly-spacing" is invalid:
refVal[2] is not a function Use --force to continue.
 at validateRuleOptions (node_modules\eslint\lib\config\config-validator.js:112:15)
      at Object.keys.forEach.id (node_modules\eslint\lib\config\config-validator.js:152:9)
      at Array.forEach (native)
      at validateRules (node_modules\eslint\lib\config\config-validator.js:151:30)
      at Object.validate (node_modules\eslint\lib\config\config-validator.js:206:5)
      at runRuleForItem (node_modules\eslint\lib\testers\rule-tester.js:331:23)
      at testValidTemplate (node_modules\eslint\lib\testers\rule-tester.js:413:28)
      at Context.RuleTester.it (node_modules\eslint\lib\testers\rule-tester.js:546:25)

The issue can be reproduced by cloning the eslint-plugin-react repo and running npm run test.

What did you expect to happen?

No errors.

What actually happened? Please include the actual, raw output from ESLint.

See above.

[gyandeeps]

@gajus @epoberezkin would love u guys feedback on this. thanks

[epoberezkin]

It seems related to #8864

Eslint currently does not correctly support references when schema is defined as array of schemas (see docs change in the referenced PR)

I will double check how it was working before, but I believe it was not working correctly anyway. The schema in the rule should be updated to be a schema object (to be able to use references).

[epoberezkin]

@jseminck are you passing some options to the rule or is it just enabled?

[jseminck]

I am running the eslint-plugin-react test suite which covers many scenarios.

I did a bit of digging and saw that this doesn't crash:

  valid: [{
    code: [
      '<App foo={42} {...bar} baz={ { 4: 2 } }>',
      '{foo} { { bar: baz } }',
      '</App>'
    ].join('\n'),
    options: [{
      when: 'never',
      spacing: {objectLiterals: 'always'},
      attributes: true
    }]

But this does;

  valid: [{
    code: [
      '<App foo={42} {...bar} baz={ { 4: 2 } }>',
      '{foo} { { bar: baz } }',
      '</App>'
    ].join('\n'),
    options: [{
      when: 'never',
      spacing: {objectLiterals: 'always'},
      attributes: true
      children: {when: 'never'} // <-- This was added
    }]

Edit: But the following warning is always displayed. It's only displayed once even for all of the test cases, so I assume it's being logged when the rule is evaluated:

can't resolve reference #/definitions/basicConfig from id #
can't resolve reference #/definitions/basicConfigOrBoolean from id #

[mysticatea]

As far as I know, we cannot use $ref if the schema is an array.
The invalid $refs have been ignored before, but it's errored since #8852.
I think it was a breaking change but released with a minor version for some reason.

[epoberezkin]

@jseminck here is the example showing that with is-my-json-valid it was never working correctly: https://runkit.com/esp/596268082d308c00122b7d86. Both options should fail validation ("whatever" is not an allowed value) yet they pass. It seems that when is-my-json-valid cannot resolve $ref it quietly passes validation.

This is the same example with Ajv: https://runkit.com/esp/59626adc6f924f00124f5fed
Ajv is configured to log warnings in case of missing references and pass validation, which it does in the first case but fails to do in the second case. I am going to make a fix so it correctly ignores missing $ref in the schema as was agreed: #8852 (comment)

But in any case, the schema in eslint-plugin-react should be either changed to use the full JSON Schema format or to be refactored to not use $ref, otherwise it will not work correctly (as the examples show).

[epoberezkin]

I think it was a breaking change but released with a minor version for some reason.

@mysticatea Ajv is supposed to ignore missing $ref, will release a fix.

[jseminck]

I'll make a PR to eslint-plugin-react as well to use full json schema object instead of an array. Thanks for the quick response and good explanations. TIL alot 😄

The only thing I still fail to understand completely is why the $ref doesn't work for the schema when it is an array. As I understood from the PR comments, the schema array is being transformed into:

{
  type: 'array',
  items: [
    obj
  ],
}

But why don't the definitions inside obj not work with this schema?

[epoberezkin]

The only thing I still fail to understand completely is why the $ref doesn't work for the schema when it is an array. As I understood from the PR comments, the schema array is being transformed into:

@jseminck That is correct, as the result of this transformation all the JSON pointers to the same sub-schemas change. So instead of #/definitions/basicConfig you would have to use #/items/0/definitions/basicConfig. When you refactor to object, definitions will be on the top level.