stac-check confusing results

[m-mohr]

I'm trying to validate the following catalog: https://github.com/EOEPCA/open-science-catalog-metadata-testing

If I run stac-check catalog.json -l -a without recursion it says the catalog.json is valid.

If I run stac-check catalog.json -l -a -r with recursion is reports the following:

stac-check catalog.json -l -a -r

 ____  ____  __    ___       ___  _  _  ____  ___  __ _
/ ___)(_  _)/ _\  / __)___  / __)/ )( \(  __)/ __)(  / )
\___ \  )( /    \( (__(___)( (__ ) __ ( ) _)( (__  )  (
(____/ (__)\_/\_/ \___)     \___)\_)(_/(____)\___)(__\_)

stac-check: STAC spec validaton and linting tool

Thanks for using STAC version 1.0.0!

Validator: stac-validator 3.3.1


Recursive: Validate all assets in a collection or catalog
Max-depth = None
-------------------------
Asset 1 Validated: catalog.json

Valid CATALOG: True

Schemas validated:
    https://schemas.stacspec.org/v1.0.0/catalog-spec/json-schema/catalog.json

STAC Best Practices:
    A link to 'self' in links is strongly recommended

This object has 7 links

-------------------------
Asset 2 Validated: catalog.json

Valid: False
Schemas validated:
    https://schemas.stacspec.org/v1.0.0/catalog-spec/json-schema/catalog.json
Error Type: Exception
Error Message: list index out of range
-------------------------

What does this mean? Which file does it refer to? It's twice catalog.json, maybe report a full file path or so?
If it's the root catalog, there are no assets in there?!
Should "Asset 2 Validated" be: "File 2 Validated" to not confuse it with STAC assets?
The "list index out of range" should probably not occur?

[m-mohr]

Looks like this is partially originating from stac-validator, I see similar results there.

[jonhealy1]

Hi @m-mohr. Does this have a url I can test? This may be fixed now. We are displaying better information and we also have a --verbose mode. Sometimes the error message makes no sense and verbose can help give more information. Off the top of my head I don't know why you would get a list index out of range error but I would try again with --verbose or maybe even --pydantic.

stac-check: STAC spec validation and linting tool

Please upgrade from version 1.0.0 to version 1.1.0!

 Validator: stac-validator 3.9.1

 Validation method: JSONSchema


Recursive: Validate all assets in a collection or catalog
Max-depth = None
-------------------------

 Asset 1: sample_files/1.0.0/./bad-item.json

ITEM Passed: False

Schemas checked: 
    https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json

Failed Schema: 
    https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json

Recommendation: 
    For more accurate error information, rerun with --verbose.


 Validation Errors: 

Validation error type: 
    JSONSchemaValidationError

Validation error message: 
    'id' is a required property

[jonhealy1]

I cloned the repo. I am getting the same error even with verbose in stac-validator. Maybe the number of collections is causing an error with recursion? We have --item-collection and --collections options now too fyi. I can dig into this and see if I can figure it out.

[jonhealy1]

@m-mohr If I move your whole catalog out of the repo, it will pass with --recursion. For some reason the other code in the repo is causing the error, which shouldn't happen obviously.

[jonhealy1]

Fixed via stac-validator v3.9.3.

 Validation Summary

✅ Passed: 1339/1339

⚠️  Best Practice Warnings (1339 assets)
  (Use --verbose to see details)

🔍 All 1339 Assets Checked
  (Use --verbose to see all assets)

[m-mohr]

Thank you very much!

[jonhealy1]

Thanks for posting the issue - any time