-
-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dataset upload structuring #577
Comments
NOTE eventually the |
@Burhan-Q We have error handling in place to manage multiple different formats, but we only suggest the correct one. I am not clear if you are stating that you are not able to upload a dataset format like the example or only that you can upload a dataset formatted differently? The timeout error suggest that there was an issue connected with the server, we allow a retry option from the dropdown in those cases. |
@kalenmike I was only able to upload a dataset with the structure mentioned in the opening comment. It is not possible to upload a dataset using the shown layout, not only did I have this issue it's been experienced by other users (how it was brought to my attention). With respect to the timeout error, I did attempt a retry and when I did it immediately failed again, but I may not have waited enough time to try again. The timeout error seemingly "resolved itself" as it showed as correctly uploaded some time after uploading with no preventing. |
One thing that was frustrating about the dataset uploading errors is that there is no indication as to what the error is or what the problem might be. This means that if an upload fails, as a user I have no clue why or what to change/fix. Having some kind of report of what errors occurred would be helpful. |
@Burhan-Q There is error reporting, it sounds like you just had the same issue every time. Timeout is no response from the server. We also have:
I may need to run through your issue with you tomorrow. |
@kalenmike that's the crazy part, the YAML with I decided to do some testing and I'm wondering if something was strange in particular in the last few days because all of the iterations I tested below worked without error. I tested changing the directory structure by varying the presence of a subdirectory in the Retesting 2024-02-20Test 1
Details
VisDrone20.yaml path: ../VisDrone20
train: images/train
val: images/val
test: null
names:
0: pedestrian
1: people
2: bicycle
3: car
4: van
5: truck
6: tricycle
7: awning-tricycle
8: bus
9: motor VisDrone20.zip structure
Test 2
Details
VisDrone20.yaml path: ../VisDrone20
train: images/train
val: images/val
test: null
names:
0: pedestrian
1: people
2: bicycle
3: car
4: van
5: truck
6: tricycle
7: awning-tricycle
8: bus
9: motor VisDrone20.zip structure
Test 3
Details
VisDrone20.yaml path: ../VisDrone20
train: images/train
val: images/val
test: null
names:
0: pedestrian
1: people
2: bicycle
3: car
4: van
5: truck
6: tricycle
7: awning-tricycle
8: bus
9: motor VisDrone20.zip structure
Test 4
Details
VisDrone20.yaml path: VisDrone20
train: images/train
val: images/val
test: null
names:
0: pedestrian
1: people
2: bicycle
3: car
4: van
5: truck
6: tricycle
7: awning-tricycle
8: bus
9: motor VisDrone20.zip structure
Test 5
Details
VisDrone20.yaml path: VisDrone20
train: train/images
val: val/images
test: null
names:
0: pedestrian
1: people
2: bicycle
3: car
4: van
5: truck
6: tricycle
7: awning-tricycle
8: bus
9: motor VisDrone20.zip structure
Test 6
Details
VisDrone20.yaml path: ../VisDrone20
train: train/images
val: val/images
test: null
names:
0: pedestrian
1: people
2: bicycle
3: car
4: van
5: truck
6: tricycle
7: awning-tricycle
8: bus
9: motor VisDrone20.zip structure
Test 7
Details
VisDrone20.yaml path: ../VisDrone20
train: train/images
val: val/images
test: null
names:
0: pedestrian
1: people
2: bicycle
3: car
4: van
5: truck
6: tricycle
7: awning-tricycle
8: bus
9: motor VisDrone20.zip structure
Test 8
Details
VisDrone20.yaml path: VisDrone_20
train: train/images
val: val/images
test: null
names:
0: pedestrian
1: people
2: bicycle
3: car
4: van
5: truck
6: tricycle
7: awning-tricycle
8: bus
9: motor VisDrone20.zip structure
|
@Burhan-Q To confirm you are no longer seeing any errors? We have an example of what a dataset should look like, but we also fix datasets with very common and obvious mistakes. The dataset processing happens after it is requested so sometimes it can fail without any reason or crash due to excess memory usage. We are constantly optimizing this. |
Yeah I was unable to get an error in testing any of the examples above. I failed to document as thoroughly the attempts I made from yesterday, so it makes it more difficult to pin down the issue. I think these tests cover most variations and all were successful. @kalenmike is it possible for you to enable verbose logging to my HUB account? Something like "log every action for |
@Burhan-Q No, that's not possible. |
Search before asking
HUB Component
Datasets
Bug
Dataset structure shown on HUB
Tested working dataset structure and YAML file
Also mentioned on #569 (comment)
Environment
Minimal Reproducible Example
No response
Additional
I attempted again with the tiger pose dataset which uploaded with out issue, but failed due to a Timeout error. Retrying immediately raised a timeout error.
The text was updated successfully, but these errors were encountered: