Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-uploading the same file will not trigger a hook event. #348

Closed
zxl777 opened this issue Feb 2, 2020 · 12 comments
Closed

Re-uploading the same file will not trigger a hook event. #348

zxl777 opened this issue Feb 2, 2020 · 12 comments
Labels

Comments

@zxl777
Copy link

zxl777 commented Feb 2, 2020

I uploaded the file for the first time and received the json data of the hook event correctly.
After I refreshed the page and uploaded the same file again, no hook event occurred.

@zxl777 zxl777 added the bug label Feb 2, 2020
@zxl777
Copy link
Author

zxl777 commented Feb 4, 2020

I found a solution, which is to delete the file after processing it.

@tuscland
Copy link

tuscland commented Feb 6, 2020

Hello,

We have the same problem. Given that deleting the file may not succeed (e.g. backend failure), it is necessary to re-emit this event when the file has been fully downloaded.

@tuscland
Copy link

tuscland commented Feb 6, 2020

I would also like to add that it is not possible to delete the file before uploading it because it would defeat the resumable nature of’ the upload...

@Acconut
Copy link
Member

Acconut commented Feb 6, 2020

This is not a problem with the tus server but your client, which likely has a setting enabled that it will not reupload a file that it has already uploaded. For example, if you are using tus-js-client look at the resume option. Removing the file on the server works, because the client can then not find the upload anymore and will reupload it.

@tuscland
Copy link

tuscland commented Feb 6, 2020

Hi Marius,

Here are the settings we use:

{
    endpoint: this.endpoint,
    retryDelays: [0, 3000, 5000, 10000, 20000],
    metadata: {
        ...
    },
    onProgress: (bytesUploaded, bytesTotal) => {
        ...
    },
    onSuccess: () => {
        ...
    },
    onError: (error) => {
        ...
    }
}

The use case is the following:

  • A user uploads completely the file.
  • Tusd notifies the backend with post-finish hook
  • The backend fails for some reason
  • ...
  • Later, the same user tries to upload again
  • The upload is resumed
  • No event is sent, because the upload is still present, and the backend has no way to know.
  • The user can never complete the upload.

Thank you for your attention.
Camille

@Acconut
Copy link
Member

Acconut commented Feb 6, 2020

@tuscland Please look into the removeFingerprintOnSuccess: true option for tus-js-client. It should allow you to reupload the file again.

@tuscland
Copy link

tuscland commented Feb 6, 2020

Sounds great.. I will, thank you!

@Pontohi
Copy link

Pontohi commented Nov 21, 2022

While removeFingerprintOnSuccess does work, it would be great to have the ability to re-trigger the completion hooks in one way or another. When uploading 100gb files, it can be a touch bit demoralizing if an API failure forces a full restart of the upload.

@Acconut
Copy link
Member

Acconut commented Nov 22, 2022

API failure forces a full restart of the upload.

How do you mean that? API failure in the sense that the API which consumes the uploaded file fails and needs to be retried?

@Pontohi
Copy link

Pontohi commented Nov 22, 2022

API failure forces a full restart of the upload.

How do you mean that? API failure in the sense that the API which consumes the uploaded file fails and needs to be retried?

Yep, that would be the case. TUS will have the full file with all bytes uploaded, but if the API fails to consume the hook we have no choice but to have the user restart since we can't get the hook to refire. Of course the API should be more reliable for consuming the hooks, but for files that take hours to days to upload, seems like it could stand to be more fail-tolerant

@Acconut
Copy link
Member

Acconut commented Feb 11, 2023

TUS will have the full file with all bytes uploaded, but if the API fails to consume the hook we have no choice but to have the user restart since we can't get the hook to refire.

tusd will retry hooks if they fail. At least for HTTP and gRPC hooks, I think. So if you bubble up the error from inside your hooks, then it should be rare that an uploads fails due to this.

@Acconut
Copy link
Member

Acconut commented Sep 18, 2024

Closing this since the question has been answered.

@Acconut Acconut closed this as completed Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants