Some of the org files in dropbox can't be opened.

[tru2dagame]

Some of the org files in my dropbox would show "File not found".
But the backup file named xxx.org.organice-bak would be always created.

[munen]

Can you try reloading the page? I suppose this is a Dropbox error. If the problem persists, it would be helpful to have the output of your browsers console.

[Linuus]

Same here. Some files work if I reload. Some files never work. :(

[munen]

@Linuus Do you have any errors in the console when it happens? If so, please relay them here.

[Linuus]

I’ve only tried it on my iPhone and I don’t think I can get a web console here? I’ll check on my laptop tomorrow.

[munen]

No, Apple doesn’t make that kind of info available, unfortunately.

If you can get errors posted here from the laptop, it’ll be a great help! Otherwise I am stuck saying “it works for me” which I don’t want to, really(;

[Linuus]

Here's the error I get on my laptop when I open the file and then reload:

[munen]

Thank you for sending the screenshot with an error! From reading it, my best guess is that the Dropbox API sometimes doesn't successfully retrieve a file and sends this error. Maybe when they are under too much load. When they return "path not found" sometimes and sometimes they return a file, it seems like an error on their part.

I just spend 15min logging in and reloading organice in three different browsers on two different machines. It works for me every time, unfortunately.

There's multiple safe-guards in the code to check if a path is set. For example here for the dropbox sync backend or here where the loading of the file for the specific strategy is triggered.

It would be best if somebody can debug/confirm this who actually gets the error. The question is whether something changes in the request from organice between "it fails" and "it works". If so, there's a clean bug to fix. If not, we would need to write a work around for Dropbox that probably goes along with "keep retrying automatically until Dropbox finally finds the file"(;

Can either of you do that @Linuus or @tru2dagame?

[Linuus]

I’ll try to do it when I get some time during the day! :)

[munen]

@Linuus Awesome!

[Linuus]

I just checked the requests and it's quite weird. It is doing two separate fetch requests to https://content.dropboxapi.com/2/files/download. One is a json and one is octet-stream.
The json request always fails with 409.

If I open a file that doesn't work, json response that returns 409 always happens first. If I open a file that works, the octet-stream that returns 200 and the base64 encoded data always comes first and it works.

So, I think it's just a matter of luck if the requests come in the right order or something :-/

UPDATE:

The requests are not the same. The one that fails passes this header: Dropbox-API-Arg: {"path":"/.organice-config.json"}.

What's that?

UPDATE2:
Ah, so organice can store settings in Dropbox or whatever. So, I think you need to guard against if that file does not exist.

[munen]

Thanks for checking it out @Linuus! 🙏

Yes, organice can save settings on your sync back-end. But it will also only try to read it when this setting is enabled.

Did you really get a 409 for when the header was correctly set to Dropbox-API-Arg: {"path":"/.organice-config.json"}? Because that would be weird.. I finally got the error once after trying many times and saw an error where the path was not set.

Here's screenshots from the one time I got the error. Honestly, it looks like an error in the Dropbox JavaScript SDK to me, but I have an idea on how to build a workaround which I'll get to, soon.

[munen]

Before building the workaround, another good option would also be to upgrade the Dropbox SDK. We're running 4.0.9, the newest version is 4.0.30 which indicates a couple potential bugfixes on their side(;

[Linuus]

So, this is an issue in any case I guess. It doesn't seem to be the cause of the "file not found" as I thought though.

I opted to store the settings file so now I don't get that error anymore. However, the file doesn't load anyway. I can see that the request succeeds and returns the Base64 encoded content of the file though so, to me, it looks like an issue with Organice?

[munen]

@Linuus Thanks for checking, again! After having seen your debug messages, I made the following changes now:

• Upgraded the DB SDK
• Catching the two types of Dropbox errors explicitly
• Improved checking for empty paths when downloading files

I did the deploy this to organice.200ok.ch already. Can you make sure to clear your cache, load the new version and check if the problem persists, please? I apologize that I have to ask, again, but it's hard for me to test. I could find the error only once and cannot reproduce it whilst I keep trying hard^^

[tru2dagame]

@munen

Sorry for the late reply. I still got the error.

Error in call to API function "files/download": HTTP header "Dropbox-API-Arg": missing required field 'path'

[Linuus]

I'm not getting any errors at all. I did figure out why my files aren't working though. It's a parse issue.

My org file has comments in the logbook like this:

* Foo
:LOGBOOK:
CLOCK: [2019-11-21 Thu 19:40]--[2019-11-21 Thu 19:51] =>  0:11
- Something something...
:END:

It seems organice doesn't support that? It crashes, but we are catching all errors here: https://github.com/200ok-ch/organice/blob/master/src/actions/sync_backend.js#L123 and just assume that the error is that the file is not found.

So... :) Some more error handling is probably needed in this case.

[tru2dagame]

@Linuus Good point. I will definitely check mine too.
All the files which can't be open are all very large org files.

[munen]

@Linuus Just to confirm: You're not getting the discussed error for files not having clocked time with comments in drawers as before? Because if so, then potentially the fix I pushed worked(;

With regards to the comments in the logbooks: The ability to clock time in drawers is a rather new feature in organice. And you're right, at this point it does not support comments within the drawer. I'll open another issue for that right away.

Thanks @tru2dagame for double checking!

[munen]

The new issue regarding parsing of :LOGBOOK: drawers is here: #111

[Linuus]

Just to confirm: You're not getting the discussed error for files not having clocked time with comments in drawers as before?

No I don’t think so 👍

[munen]

@Linuus Sweet! 🚀

@tru2dagame How about you? When you run the newest version of organice (cleared cache), do you have issues opening files, still? That is apart from the newly found bug for parsing :LOGBOOK: drawers.

[tru2dagame]

@munen
I tried to delete all my LOGBOOK blocks, it works.
I do have a backup. Will try to test more details for this part.

[munen]

@Linuus Parsing of the :LOGBOOK: drawer has been addressed in a PR #112 by @jamesnvc.

Could you please give it a go one more time and confirm that the issue is gone for you, too? Please make sure to empty the cache and reload organice beforehand.

[Linuus]

Looks good to me. Thank you both 👍🎉

[munen]

Nice! Since you both confirmed that opening files works, now, I'm closing this issue.

However, if this (or another) problem arises at any point, please do not hesitate to open another issue, again, so we can get ahead of it as quickly as this time.

Thank you, everyone, for reporting, testing and fixing! 🙏 🙇‍♂️