Running a pipeline from the codebase

[camelia-tfds]

Hi everyone, I am new to mlcroissant and have been trying to use it from colab. However, for development and next steps I would like to understand how to build the code, run tests and so on in my workspace. Is there specific documentation which I could use which I might have overseen? For the moment I want to run the pipeline at "/workspaces/croissant/python/mlcroissant/mlcroissant/scripts/load.py".
Thanks in advance!

[marcenacp]

Hi Camelia, what are you trying to achieve with load.py?

load.py is a helper script to yield data from a Croissant JSON-LD. It takes in input a jsonld or a file, and it will print the result in the CLI. You can also use it to generate our test sets for a new dataset by using the flag --update_output. You can see example of usage in our CI/CD.