When running with Spectral rule not all non spectral warnings are returned

[ChrisPhillips-cminion]

I appreciate this is beta functionality.

When running the config in the zip against the api with out the spectral config I get the following

cminion@random:~/src/tmp$ !l
lint-openapi api-with-examples.yaml -s

warnings

  Message :   OpenAPI `host` must be present and non-empty string.
  Path    :
  Line    :   0

  Message :   OpenAPI host `schemes` must be present and non-empty array.
  Path    :
  Line    :   0

  Message :   OpenAPI object should have non-empty `tags` array.
  Path    :
  Line    :   0

  Message :   Operation `description` must be present and non-empty string.
  Path    :   paths./.get
  Line    :   7

  Message :   Operation should have non-empty `tags` array.
  Path    :   paths./.get
  Line    :   7

  Message :   Operation `description` must be present and non-empty string.
  Path    :   paths./v2.get
  Line    :   76

  Message :   Operation should have non-empty `tags` array.
  Path    :   paths./v2.get
  Line    :   76

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./.get.operationId
  Line    :   8

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./v2.get.operationId
  Line    :   77

statistics

  Total number of errors   : 0
  Total number of warnings : 9

  warnings
   1  (11%) : OpenAPI `host` must be present and non-empty string.
   1  (11%) : OpenAPI host `schemes` must be present and non-empty array.
   1  (11%) : OpenAPI object should have non-empty `tags` array.
   2  (22%) : Operation `description` must be present and non-empty string.
   2  (22%) : Operation should have non-empty `tags` array.
   2  (22%) : operationIds must follow case convention

When I run it with the spectral in the zip I get the following

cminion@random:~/src/tmp$ !li
lint-openapi api-with-examples.yaml -s
Function 'oasResponseHasExample' could not be loaded: Not Found

errors

  Message :   'produces' object should contain the value 'application'
  Path    :   paths./.get.produces.0
  Line    :   11

  Message :   'produces' object should contain the value 'application'
  Path    :   paths./v2.get.produces.0
  Line    :   80

warnings

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./.get.operationId
  Line    :   8

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./v2.get.operationId
  Line    :   77

statistics

  Total number of errors   : 2
  Total number of warnings : 2

  errors
   2 (100%) : 'produces' object should contain the value 'application'

  warnings
   2 (100%) : operationIds must follow case convention

cminion@random:~/src/tmp$ ls -lart
total 28
-rwxrwxrwx 1 cminion cminion 2868 Sep 28 15:55 .validaterc
drwxrwxrwx 1 cminion cminion 4096 Sep 29 20:07 ..
-rwxrwxrwx 1 cminion cminion 6103 Sep 30 20:53 api-with-examples.yaml.old
-rwxrwxrwx 1 cminion cminion 1464 Sep 30 20:54 .spectral.json
-rwxrwxrwx 1 cminion cminion 6101 Sep 30 20:54 api-with-examples.yaml
-rwxrwxrwx 1 cminion cminion 3352 Sep 30 20:57 demo.zip
drwxrwxrwx 1 cminion cminion 4096 Sep 30 20:58 .
cminion@random:~/src/tmp$

I would expect all the warnings in the first run to be present in the secondrun
lint.zip

[jorge-ibm]

@ChrisPhillips-cminion - This is an unintended effect of adding your own ruleset file. What we currently do is "override" our spectral ruleset with the custom one, and since your ruleset doesn't include the rules we defined https://github.com/IBM/openapi-validator/blob/master/src/spectral/rulesets/.defaultsForSpectral.yaml, they are not included in the call to spectral. A temporary fix is to just copy the rules from https://github.com/IBM/openapi-validator/blob/master/src/spectral/rulesets/.defaultsForSpectral.yaml into your ruleset file, but we are currently working on a fix to make sure any ruleset you define is "appended" to the default spectral ruleset we run in our validator.

[rmkeezer]

With the new changes in this PR: #202 The .spectral.json custom rules will be properly appended to the spectral ruleset and will show up as expected.

[jorge-ibm]

This should now be fixed in v0.31.0 of the validator. Closing as fixed.
@ChrisPhillips-cminion - please let us know if you are still having issues