process_and_upload: import_path: support read-only file system

[ProBackup-nl]

Basic information

• Release version: 0.1.2
• System: Mac OS X
• Capture Device: Action Camera | Sony 200V

Steps to reproduce behavior

1. $ mapillary_tools process_and_upload --import_path /Volumes/Untitled/DCIM --user_name "xyz" --verbose --rerun --flag_duplicates --duplicate_distance 0.5 --duplicate_angle 1 --keep_original --advanced

Expected behavior

No error, just do the magic.

Actual behavior

Traceback (most recent call last):
  File "/usr/local/bin/mapillary_tools", line 60, in <module>
    command.run(args)
  File "/usr/local/lib/python2.7/site-packages/mapillary_tools/commands/process_and_upload.py", line 91, in run
    process_user_properties(**({k: v for k, v in vars_args.iteritems()
  File "/usr/local/lib/python2.7/site-packages/mapillary_tools/process_user_properties.py", line 70, in process_user_properties
    user_properties)
  File "/usr/local/lib/python2.7/site-packages/mapillary_tools/processing.py", line 624, in create_and_log_process_in_list
    verbose)
  File "/usr/local/lib/python2.7/site-packages/mapillary_tools/processing.py", line 632, in create_and_log_process
    os.makedirs(log_root)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/Volumes/Untitled/DCIM/100MSDCF/.mapillary'

Additional information

Sony AS200V only mounts as read only file system.
Don't write anything in any location of the import_path.
Please, use a default temporary location for writing stuff.

Mapillary upload also fails

$ mapillary_tools upload --import_path /Volumes/Untitled/DCIM --verbose --advanced            
No images to upload.
Please check if all images contain the required Mapillary metadata. If not, you can use "mapillary_tools process" to add them

PS#1 For --verbose there is very little output in the upload command
PS#2 After already having uploaded 253.9 K images through the web interface, it would be nice that the command line tools finally start working.

[jernejaMislej]

hi, sorry for the late reply.
We are planning to improve the logging procedure in the next releases. Unfortunately it is not so simple, as we need to make sure we keep exact logs of the upload state for each image, otherwise we would quickly end up with many duplicate uploads. Currently we are handling this by creating log files where ever the image is located, since otherwise, in case of using a default temporary location, we would have to be relying that the absolute path doesnt change at all, which in our experience does not work out well.
Nevertheless, we do have plans to improve the logging and the solution will work for read only file systems as well.
We will keep you informed about the progress and also improve the output of --verbose in the upload command.

[andrewharvey]

on linux ~/.config/mapillary_tools or ~/.mapillary_tools/ would work. It's not so temporary so would survive machine restarts and avoids writing to the sdcard or other device which the images are stored on.

[ProBackup-nl]

@andrewharvey I don't understand what you are trying to say with your comment.

[ptpt]

It's fixed in v0.8.0 which doesn't touch images during process and upload. You can:

mapillary_tools process_and_upload path/to/images/ --desc_path=/tmp/description.json