Pictures never upload

[tomchiverton]

v1.9.0 from Google Play on Android 4.4.4

I have tried clearing the app data and deleting the folder in my Camera Upload library, and still newly taken photos do not appear.

[Logan676]

Here are some clues for trouble shooting.

1. turn on the camera backup settings, go to Settings - Turn on Camera Upload.
2. check if newly taken photos were added to the uploading queue, you can find uploading tasks (if have some) in Transfer List.
3. if the two steps above doesn`t solve your problem. Please tell us some detailed info in order to reproduce the bug, e.g. phone brand, counts of SD cards.

[tomchiverton]

1. Yes, this is on. I ran it's setup wizard after clearing app data.
2. Nothing in upload list

3. Sony Z1, one SD card

Tom
Sent from my phone.

[msutas]

I have the same issue.

V1.9.1 from Google Play on Android 5.0 Samsung Galaxy S5

[jhadenfeldt]

Same here:

v1.9.1 from Google Play, Android 5.1 (Cyanogenmod), OnePlus One

[damianorenfer]

Same here:

v1.9.1 from Google Play, Android 5.1.1, Sony Xperia Z1 Compact

[tomchiverton]

My Sony Z1 is still on Android 4.4 and it still doesn't work, even though it's now picked up the update to v1.9.1 as well.
Bit of a shame it's been broken for months now, as this is a killer feature for leaving Dropbox behind. It used to work.

3 pictures have appeared in my library, all from '9 days ago'. But I have taken many more photos than that.
From the file names, these are generated by an app (probably Janetter, a Twitter client), not the main camera app, because the file names in the SeaFile library don't start with "DSC".

[forouher]

I've looked at the code and found a couple of issues with media upload. Since the problems are somewhat fundamental, I've started on refactoring CameraUploadService/TransferService and converting them into SyncAdapters. Along the way I'll hopefully be able to address most of the issues reported regarding camera upload.

If someone has already started doing another refactoring of that code (especially one of the core developers), please give me a hint so our work won't collide.

[lins05]

I've started on refactoring CameraUploadService/TransferService and converting them into SyncAdapters.

👍

If someone has already started doing another refactoring of that code

@forouher No one is working on that afaik.

[forouher]

Could you try and find out the full file path of one of the photos that wasn't uploaded?

You'd need a file manager app to see the directory structure (e.g. "File manager" on Cyanogen or some file manager app like "ES Explorer" from the app store).

E.g. on my phone it's [/storage/emulated/0] /DCIM/Camera/IMG_20151020_101015.jpg
But it might be some other path on your phones.

[lupa18]

Same here.

Details:
Samsung gt-i8190
Android 4.1.2
Camera store their photos in a folder I choose on /mnt/extSdCard/DCIM/Camera

[forouher]

@lupa18 Thanks! Did you manually select that folder in Seadroid for camera upload? Or did you leave Seadroid camera upload on its default settings?

[lupa18]

Yes, I mannualy selected.

[forouher]

@lupa18

And the first theory is down the toilet...
Could you please verify the following:

1. Do the photos in question show up in the standard Android Gallery?
2. Please reboot your phone and start the Seafile app. Are the photos uploaded then?

[tomchiverton]

For me, yes they do show in the phone gallery, and restarting the phone has no effect.

Why isn't the code just (even as a double check) scanning for files in $dir newer than the last time it looked?

If you want to make a build with extra logging in available I'm happy to install this btw.

Tom
Sent from my phone.

On 19 November 2015 06:03:22 GMT+00:00, Dariush Forouher notifications@github.com wrote:

@lupa18

And the first theory is down the toilet...
Could you please verify the following:

1. Do the photos in question show up in the standard Android Gallery?
2. Please reboot your phone and start the Seafile app. Are the photos
   uploaded then?

---

Reply to this email directly or view it on GitHub:
#423 (comment)

[lupa18]

Here the same: stock gallery works fine. Phone restarted and nothing. :-/

[forouher]

@lupa18 @tomchiverton thanks!

I've opened PR #444 a couple of days ago, where I attempt to fix some of the camera upload issues. It would be very interesting to see whether PR #444 would also fix your issue.

I've packaged that PR into an .apk, which can be downloaded here: https://github.com/forouher/seadroid/releases/tag/account_and_camera_upload_tag

Warning: PR #444 is experimental and has seen no code review by the Seadroid core developers yet. You will probably have to configure Seadroid again once you downgrade from that apk back to the official release.

[tomchiverton]

This immediately uploaded the backlog of old photos (default accepted for the upload set up, to an existing library).

A newly taken photo did not upload (even though was on wifi). Nothing in transfer list. A reboot didn't nudge it into working.

Progress :-)

[forouher]

@tomchiverton Thanks for taking the time to test this patch set!

Could you please check a couple more things:

1. Take a new picture (with wifi on).
2. Wait 1-2 minutes for upload to happen (I guess it won't happen :)
3. Go to Android Settings -> Accounts -> Seafile -> [the account you use for camera upload]. There you should see the sync status of the camera upload:
   
4. Is the syncing enabled? (like in the screenshot above?)
5. What does the timestamp of the last successful sync say? Is it recent (1-2 minutes old)?
6. If you select Dots -> "Sync now": Are the missing photos synced then? Is the time stamp updated correctly?

[tomchiverton]

OK.
Wifi on
Took picture.
Waited a few minutes.
No upload.
Settings, accounts has 'auto-sync' enabled.
Last success for the seafile account is recent, yes, only a few minutes
I selected 'sync now' and it rapidly changed to 'syncing now' and then to the current time
I waited another few minutes and then re-checked the web view of the library. No new photo.

[forouher]

@tomchiverton Thanks, that's very useful!

So, the first full sync does work correctly. Subsequent incrementals fail because Seadroid isn't seeing any new photos. Maybe there is an issue with the image timestamp comparison...

I've uploaded a new apk (https://github.com/forouher/seadroid/releases/download/account_and_camera_upload_tag/seadroid_PR444_64f3274.apk) that contains a possible workaround. Would you mind trying that one?

Which camera app are you using, btw?

[tomchiverton]

Didn't seem to make any difference; new photos do not upload, the sync data is recent in the account, and manualy do a sync from there just updates the time stamp.
It's the stock Android 4.x camera app.

[forouher]

@tomchiverton

Hmm, that I didn't expect. I must have taken a wrong turn somewhere. Okay, sorry. It seems like I have to look at the log messages after all.

Would it be possible for youto record the log messages of the Seafile app?

I've uploaded a new .apk that pushes the loglevel up to 11 (https://github.com/forouher/seadroid/releases/download/account_and_camera_upload_tag/seadroid_PR444_ddef5da.apk).

Unfortunately, fetching logs from an Android app is not that simple.

Method A: If you know your way around Linux, the easiest way would be

1. to enable USB Debugging on your phone (https://developer.android.com/tools/help/adb.html#Enabling)
2. install "android-tools-adb" on your computer (on Debian/Ubuntu, maybe something else on other distros)
3. connect it via USB to your computer.
4. run "adb logcat CameraSyncAdapter:D *:E" on the console (your phone may ask you to authenticate adb)
5. take a new picture
6. force manually a new camera sync
7. save the output of "adb logcat" starting at step 5.

Method B: Alternatively you can use one of the "logcat" apps from the AppStore to record the logs. The require a rooted phone, however.

If neither method is an option for you, no problem. Just give me a note and I'll think up something else.

[tomchiverton]

Installed. Rebooted. Took photo. Log from there is
save to internal storage http://pastebin.com/XuFTRuUS
save to sd card http://pastebin.com/C0Tz3SrK
manual sync http://pastebin.com/kTf8bamA

[forouher]

@tomchiverton thank you very much!

Okay, I think I know what's going on. Android sorts images into "albums" (basically folders). Seadroid by default only uploads images from the folder "Camera" (a hard coded string). I'd wager that the pictures you've taken are not saved into "Camera".

Could you please check your image gallery and find out which album your images are saved into?

If my theory is correct, you can enable upload of that album by

1. going to Seadroid settings
2. go to "Camera Photo Upload" -> "Advanced Options"
3. select "Custom Upload Albums"
4. select "Pick photo albums"
5. select the album(s) you identified above (which contain the photos you just took)
6. select "Confirm"
7. photo upload should start immediately

[tomchiverton]

So, on this Android 4.4 Sony, I open'ed Albums, then from the menu pic 'folders' (never knew this was here).
"camera" does indeed include 37 random images from some time ago.
"100andro" has 500 odd, including the ones I just took tonight.

Enter that in Seafile (which lists it twice, so I picked both, as well as 'cemera') , and, with wifi on, suddenly whole new folders appear in the web view of the library !

Cracked it. Serious skillz.

Is the folder setting available in the released Seafile client i.e. I can just go back to it ?

So maybe Seafile needs to have a longer set of default folders, as it seems there are many devices that don't use 'camera' as the default ?

[forouher]

Excellent!

Yeah, I'll add "100ANDRO" to the default list. Unfortunately that seems arbitrary and who knows what folder names other camera apps or phones might use...

Is the folder setting available in the released Seafile client i.e. I can just go back to it ?

1.9.2 also has a (different) folder setting, yes. Let me know it works for you.

Thanks again for your help!

[tomchiverton]

I uninstalled Seafile, and installed 1.9.2 from the Play Store.
I removed my existing 'camera upload' library.

In the new version I started the camera upload wizard, and made sure to navigate the (confusing) series of folders to /storage/sdcard1/DCIM and picked '100ANDRO' there (listed as several hundred items.
The chooser in your builds was a lot better.

Nothing uploads. Nothing in the app's transfer list. Not even listed in Settings, accounts.
Nothing in the client transfer list

I uninstalled the Play version and went to your
https://github.com/forouher/seadroid/releases/download/account_and_camera_upload_tag/seadroid_PR444_64f3274.apk
the last one before the extra logging and that's fine (with picking the folders at the extra step in the upload set up).
That upload (some) of the back log of photos already on the device. I assume it sends them 20 at a time or something and will eventually get up to date and stay there.

Will report back.

[tomchiverton]

Hmm. No uploads of the remaining back log, or new phots after 15 minutes. I'll try the latest build you posted.

[forouher]

Thanks, so, 1.9.2 folder selection is (partially) broken. That's good to know.

But I'd like to make sure that the refactoring really fixes that issue.

seadroid_PR444_64f3274.apk should work just as well as seadroid_PR444_ddef5da.apk does.
(It just adds log messages and some null checks, which in theory shouldn't matter).

Would you mind going back to seadroid_PR444_64f3274.apk one last time and record a logcat?

1. start the logcat
2. take a picture (syncing should be triggered automatically if on wifi)
3. go to settings->accounts->seafile and wait until the sync timestamp displayed is more recent than the photo)
4. stop the logcat

I've never done tests with more than 30 images, so you may have stumbled on another issue there. I'd just like to be sure.

[tomchiverton]

Sure, I can test that in the next few days.

Although rather than adding another default folder, why not just use all of them as listed by the advanced config GUI? This matches what people expect.

Tom
Sent from my phone.

On 26 November 2015 06:26:07 GMT+00:00, Dariush Forouher notifications@github.com wrote:

Thanks, so, 1.9.2 folder selection is (partially) broken. That's good
to know.

But I'd like to make sure that the refactoring really fixes that issue.

seadroid_PR444_64f3274.apk should work just as well as
seadroid_PR444_ddef5da.apk does.
(It just adds log messages and some null checks, which in theory
shouldn't matter).

Would you mind going back to seadroid_PR444_64f3274.apk one last time
and record a logcat?

1. start the logcat
2. take a picture (syncing should be triggered automatically if on
   wifi)
3. go to settings->accounts->seafile and wait until the sync timestamp
   displayed is more recent than the photo)
4. stop the logcat

I've never done tests with more than 30 images, so you may have
stumbled on another issue there. I'd just like to be sure.

---

Reply to this email directly or view it on GitHub:
#423 (comment)

[forouher]

Sure, I can test that in the next few days.

Only if you experience problems. If the upload with 64f3274 works fine, just give me a note and I'll stop worrying.

Although rather than adding another default folder, why not just use all of them as listed by the
advanced config GUI? This matches what people expect.

Well, that would also upload Screenshots/WhatsApp/Facebook/Twitter/Instagram pix. Some people won't care, but others will. So, having an option is important. But the UI of the advanced config page should probably be improved. So users are more aware of what Seadroid will do. Maybe remove that switch and always show the grid of albums and just preselect a default list of albums...

[tomchiverton]

Installed 64f3274, took photo, log cat shows no evidence of upload immediately,
http://pastebin.com/DtW7Scbj

But a few minutes later I saw a bunch of "File XXX in bucket" messages, then there was a notice it was getting a 500 error.
This corresponded with a message from mod_security in my server logs, which I removed with SecRuleRemoveById for the Seafile virtual host.

But now even triggering a manual sync produces no output in logcat. I am rebooting to see if it makes a difference.

[tomchiverton]

After a reboot, uploads resume, so it's back filling the old photos now.

When it finished I turned on video upload as well. This consistently logged errors because I have a 1gig video that far exceeds the upload size setting in Apache. New issue opened : #451

Anyway once it had caught up (~500 photos) I tried taking a new one, this was uploaded right away (at least while the wifi was on). Photos taken while wifi is off upload a few minutes after wifi returns.

This means build 64f3274 should be good to go.
I agree it might be better to preselect a (larger) range of folders, rather than all. Personally I've just selected them all now. It's nice to have photos taken by other apps (like Twitter or Untappd clients) backed up.

[forouher]

Installed 64f3274, took photo, log cat shows no evidence of upload immediately,

Yeah, I can't see anything either, which sounds dubious.
[5min later]. Heh, found a typo in the AndroidManifest. The Media Observer Service wasn't restarted on apk upgrade. Will fix that.

But now even triggering a manual sync produces no output in logcat. I am rebooting to see if it makes a difference.

That might be by design. A 500 error is a server side error and handled the same as a network/IO error. That means Android will do automatic retries with an exponential back off (i.e.: retry after 1 min, 2 min, 4 min, 8min,...). I'm not sure that manually hitting the "refresh" button can override that back off.

After a reboot, uploads resume, so it's back filling the old photos now.

I guess a reboot will reset that back off, so this makes sense.

The account shows 'sync OK' even though it clearly wasn't.
The Seafile client should make these 500 errors more obvious, for a start.

I was thinking about that, too. But I decided against it. It is a server side issue and nothing a typical user can do anything about. If our server fucks up, our monitoring tools have to notify us. We shouldn't rely on users to call us. ;)

Anyway once it had caught up (~500 photos) I tried taking a new one, this was uploaded right
away (at least while the wifi was on). Photos taken while wifi is off upload a few minutes after wifi
returns.

Excellent, as it should. In general, a sync will be performed at least every 24h, but it might happen sooner if there is a pending sync request.

Again, thank you very much for testing that patch! If you find any further issues over time, please bring them forward. :)

[tomchiverton]

This is awesome news. Any idea when the store version will be updated?

[Logan676]

This week according to our agenda