BUG: GC fails to run when files other than metarange and ranges are found in _lakefs folder

[N-o-Z]

When non range files exist in the path we get the following error:

Job failed, please check complete logs in configured logging destination. ExitCode: 1. Last few exceptions: Caused by: io.treeverse.jpebble.BadFileFormatException: Bad magic 39 38 64 31 39 22 0a 7d: wrong bytes Caused by: org.apache.spark.SparkException: Job aborted due to stage failure:

This issue becomes even more acute since our ref dump documentation and procedure saves the ref dumps file under the _lakefs path

[N-o-Z]

@itaiad200 I think we need to prioritize this, whether we need to change GC or ref dumps flow

[idanovo]

I've tried running GC on a repo that contains a random json file under the _lakefs path, and I couldn't reproduce this issue.

[N-o-Z]

I think we need to try to reproduce it with the same scenario - perform ref dumps and then run the GC

[idanovo]

I also tried to test this scenario, and the bug wasn't reproduced

[N-o-Z]

That is really strange - I'd probably check the following:

1. See which GC version is running on the region (ireland) and see if maybe it's related
2. Ask for the exact command when they run refs dump
3. If possible - obtain the files from their _lakefs path (all of them, or at least the refs dump file) to try and reproduce it locally