the normal success rate and downloading speed?

[Tycho-Xue]

Hi, somehow the success rate starts at 0.8 and then dropped to 0.2 after downloading for a while, what success rate should we expect to be healthy? if an average success rate is 0.5 does it mean after the downloading I have only downloaded 50% of the total dataset?
Also, will the factors like the image resolution downloaded have a great impact on the success rate?
And do you know if it's possible to download the datacomp-1B using a 96 CPU pods in a week?
Thanks

[rom1504]

Are you using knot resolver yet ? Can you share a wandb of your img2dataset download?

…

On Thu, Oct 5, 2023, 6:21 AM Tycho-Xue ***@***.***> wrote: Hi, somehow the success rate starts at 0.8 and then dropped to 0.2 after downloading for a while, what success rate should we expect to be healthy? if an average success rate is 0.5 does it mean after the downloading I have only downloaded 50% of the total dataset? Also, will the factors like the image resolution downloaded have a great impact on the success rate? And do you know if it's possible to download the datacomp-1B using a 96 CPU pods in a week? Thanks — Reply to this email directly, view it on GitHub <#68>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAR437R4WJ6M7FEMVLBBWBTX5XONXAVCNFSM6AAAAAA5TJ3FWWVHI2DSMVQWIX3LMV43ASLTON2WKOZRHEZDOMBWGU3TCNI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>