Maximum attachment size for Dropbox sync

[ferrouswheel]

I have some PDF files of around 150MB as attachments to Joplin notes that won't sync. This is an import from Evernote and 40k other notes and attachments have synced without issue.

These PDFs repeatedly and consistently fail to sync via Dropbox.

I have encryption enabled, so as I understand it, the actual size of the encrypted files is larger.

The error on the sync page is:

eye_opener_tog_paper_highres.pdf (d71a39a688ba48fc94c8d265376a0ab3) could not be uploaded: network timeout at: https://content.dropboxapi.com/2/files/upload

This Dropbox forum post suggests that 150MB is the official limit for the endpoint being called:

https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/API-V2-Upload-File-size-above-300MB/td-p/389687

and suggests using the upload sessions API instead.

I have not looked at Joplin source code, but initial thoughts are:

• Does Joplin use upload sessions API for large files?
• If so, is it choosing which upload API to use based on the attachment size before encryption instead of after?

Environment

Joplin version: Joplin 2.12.18 (prod, win32)
Platform: Windows

Steps to reproduce

0. Enable encryption
1. Download the paper PDF from https://www3.cs.stonybrook.edu/~cvl/content/eyeopener/eyeopener.html
2. Attach to note
3. Sync to Dropbox and you should get an error like:

eye_opener_tog_paper_highres.pdf (d71a39a688ba48fc94c8d265376a0ab3) could not be uploaded: network timeout at: https://content.dropboxapi.com/2/files/upload

Describe what you expected to happen

Successful sync without error

[personalizedrefrigerator]

Thank you for reporting this (and for linking to a forum post with a possible solution)!

I don't think that Joplin does use the upload sessions API.

These seem to be the relevant parts of Joplin's source code:

joplin/packages/lib/DropboxApi.js

Line 115 in cc19b2b

const endPointFormat = ['files/upload', 'files/download'].indexOf(path) >= 0 ? 'content' : 'api';

joplin/packages/lib/file-api-driver-dropbox.js

Line 176 in cc19b2b

'files/upload',

[github-actions]

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? If you require support or are requesting an enhancement or feature then please create a topic on the Joplin forum. This issue may be closed if no further activity occurs. You may comment on the issue and I will leave it open. Thank you for your contributions.

[personalizedrefrigerator]

Still an issue!

[github-actions]

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? If you require support or are requesting an enhancement or feature then please create a topic on the Joplin forum. This issue may be closed if no further activity occurs. You may comment on the issue and I will leave it open. Thank you for your contributions.

[7adidaz]

I can work on this, I need some verification on my approach to this before I miss things up:)

1. calculate how many "150 MB" in the file.
2. create an /upload_session/start and get a session_id.
3. for that many ("150 MB" -1) in a file, use /upload_session/append with the session_id.
4. for the last "150 MB", call /upload_session/finish

and voila!

[ferrouswheel]

@7adidaz I'm not a contributor to Joplin, so someone else will have to provide implementation guidance.

However, to avoid a potential bug, I imagine you'll need to ensure you check the file size after encryption has taken place (if the user has encryption enabled).

[7adidaz]

ah yes, there will be a hash comparison to ensure data integrity.