How are data differences (files used, etc) recorded?

[jaymefosa]

For example if we are continuously adding files and increase our train or test set, are we able to see these differences across experiments as well? Or if we simply change all of the train files to a different version (synthetically generated files for example), is there a way to view this between experiments?

[bfirsh]

Replicate can store arbitrary files and directories with the path option to replicate.init(), but doesn't diff the files it stores yet. That's certainly something we want to add: https://github.com/replicate/replicate/issues/303

You can bodge it by checking out the two experiments into two directories:

replicate checkout [first experiment id] -o exp1/
replicate checkout [second experiment id] -o exp2/
diff -qr exp1/ exp2/

You might also want to store your training data in an external system. There's more information about that here: https://replicate.ai/docs/guides/training-data