New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unicode filenames break server #104

Closed
kevinlondon opened this Issue Aug 15, 2014 · 5 comments

Comments

3 participants
@kevinlondon

kevinlondon commented Aug 15, 2014

If you use Unicode characters for filenames, it causes the server not to run. Locally, I added filename = filename.decode("utf-8") at the start of the __init__ method for Media in the gallery.py file and that resolved the issue.

Traceback (most recent call last):
  File "/usr/home/kevin/.virtualenvs/thumbtest/bin/sigal", line 11, in <module>
    sys.exit(main())
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/site-packages/sigal/__init__.py", line 154, in main
    parser.dispatch()
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/site-packages/argh/helpers.py", line 53, in dispatch
    return dispatch(self, *args, **kwargs)
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/site-packages/argh/dispatching.py", line 125, in dispatch
    for line in lines:
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/site-packages/argh/dispatching.py", line 202, in _execute_command
    for line in result:
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/site-packages/argh/dispatching.py", line 185, in _call
    result = args.function(*positional, **keywords)
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/site-packages/sigal/__init__.py", line 108, in build
    gal = Gallery(settings, theme=theme, ncpu=ncpu)
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/site-packages/sigal/gallery.py", line 452, in __init__
    album = Album(relpath, settings, dirs, files, self)
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/site-packages/sigal/gallery.py", line 205, in __init__
    media = Image(f, self.path, settings)
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/site-packages/sigal/gallery.py", line 129, in __init__
    super(Image, self).__init__(filename, path, settings)
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/site-packages/sigal/gallery.py", line 74, in __init__
    self.dst_path = join(settings['destination'], path, filename)
  File "/usr/home/kevin/.virtualenvs/thumbtest/lib/python2.7/posixpath.py", line 80, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 6: ordinal not in range(128)
@saimn

This comment has been minimized.

Owner

saimn commented Aug 18, 2014

Hmm, everything here should already be unicode ... could you run in debug mode (sigal build -d) and paste the output, and maybe put a print in the __init__ to print settings['destination'], path, filename ?

edit: what OS and python version are you using ?

@braoult

This comment has been minimized.

braoult commented Nov 3, 2014

Title does not seem to accept UTF-8, anyway (but filenames do).
Ubuntu 14.04, sigal 0.8.1, Python 2.7.6.

With
title = "à"
I get:

br@lorien:~/www/w/sigal$ sigal build
-> Sample Album 2 : 9 images
-> Album 1 àvec UTF-8 : 9 images

Traceback (most recent call last):
File "/usr/local/bin/sigal", line 9, in
load_entry_point('sigal==0.8.1', 'console_scripts', 'sigal')()
File "/usr/local/lib/python2.7/dist-packages/click-3.3-py2.7.egg/click/core.py", line 610, in call
return self.main(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/click-3.3-py2.7.egg/click/core.py", line 590, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click-3.3-py2.7.egg/click/core.py", line 936, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python2.7/dist-packages/click-3.3-py2.7.egg/click/core.py", line 782, in invoke
return ctx.invoke(self.callback, *_ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click-3.3-py2.7.egg/click/core.py", line 416, in invoke
return callback(_args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/sigal-0.8.1-py2.7.egg/sigal/init.py", line 133, in build
gal.build(force=force)
File "/usr/local/lib/python2.7/dist-packages/sigal-0.8.1-py2.7.egg/sigal/gallery.py", line 549, in build
writer.write(album)
File "/usr/local/lib/python2.7/dist-packages/sigal-0.8.1-py2.7.egg/sigal/writer.py", line 110, in write
page = self.template.render(**self.generate_context(album))
File "/usr/local/lib/python2.7/dist-packages/Jinja2-2.7.3-py2.7.egg/jinja2/environment.py", line 969, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/local/lib/python2.7/dist-packages/Jinja2-2.7.3-py2.7.egg/jinja2/environment.py", line 742, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/sigal-0.8.1-py2.7.egg/sigal/themes/galleria/templates/index.html", line 22, in top-level template code
[ removed html code here ]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

@saimn

This comment has been minimized.

Owner

saimn commented Nov 8, 2014

Could you run again sigal with sigal build -fd -n 1 to get a useful traceback ? Thanks.

@braoult

This comment has been minimized.

braoult commented Nov 10, 2014

Here is the log with fd -n 1. I did escape 2 lines at the bottom (did not find a way to attach the file directly).

INFO: Reading settings ...
DEBUG: Settings file: sigal.conf.py
DEBUG: Rewrite destination : forPatrick -> /export/home/br/www/w/sigal/forPatrick
DEBUG: Settings:
{'albums_sort_reverse': False,
'autorotate_images': True,
'colorbox_column_size': 4,
'copy_exif_data': False,
'destination': u'/export/home/br/www/w/sigal/forPatrick',
'files_to_copy': (),
'google_analytics': '',
'ignore_directories': [],
'ignore_files': [],
'img_processor': 'ResizeToFit',
'img_size': (800, 600),
'index_in_url': False,
'jpg_options': {'optimize': True, 'progressive': True, 'quality': 85},
'keep_orig': False,
'links': '',
'locale': '',
'make_thumbs': True,
'medias_sort_attr': 'filename',
'medias_sort_reverse': False,
'orig_dir': 'original',
'orig_link': False,
'plugin_paths': [],
'plugins': [],
'source': u'/home/br/www/t/pictures',
'theme': 'galleria',
'thumb_dir': 'thumbnails',
'thumb_fit': True,
'thumb_prefix': '',
'thumb_size': (280, 210),
'thumb_suffix': '',
'title': '\xc3\xa0',
'video_size': (480, 360),
'webm_options': ['-crf', '10', '-b:v', '1.6M', '-qmin', '4', '-qmax', '63'],
'write_html': True,
'zip_gallery': False,
'zip_media_format': 'resized'}
INFO: Source : /home/br/www/t/pictures
INFO: Destination : /export/home/br/www/w/sigal/forPatrick
INFO: Theme : galleria
DEBUG: Plugin paths: []
INFO: Using 1 cores
DEBUG: Albums:
[(path=u'Sample Album 2', title=u'Sample Album 2'), (path=u'Album 1 \xe0vec UTF-8', title=u'Album 1 \xe0vec UTF-8'), (path='.', title=u'pictures')]
-> Sample Album 2 : 9 images
INFO: Processing /home/br/www/t/pictures/Sample Album 2/Angelfish.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Sample Album 2/Angelfish.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Sample Album 2/thumbnails/Angelfish.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Sample Album 2/Big-Eye Squirrel Fish.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Sample Album 2/Big-Eye Squirrel Fish.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Sample Album 2/thumbnails/Big-Eye Squirrel Fish.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Sample Album 2/Blue-Banded Sea Perch.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Sample Album 2/Blue-Banded Sea Perch.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Sample Album 2/thumbnails/Blue-Banded Sea Perch.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Sample Album 2/Breaching, Humpback Whale.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Sample Album 2/Breaching, Humpback Whale.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Sample Album 2/thumbnails/Breaching, Humpback Whale.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Sample Album 2/Bright Idea, Bottlenose Dolphin.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Sample Album 2/Bright Idea, Bottlenose Dolphin.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Sample Album 2/thumbnails/Bright Idea, Bottlenose Dolphin.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Sample Album 2/Caribbean Blue Angelfish, Gulf Of Mexico.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Sample Album 2/Caribbean Blue Angelfish, Gulf Of Mexico.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Sample Album 2/thumbnails/Caribbean Blue Angelfish, Gulf Of Mexico.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Sample Album 2/False Clown Anemonefish, Bali, Indonesia.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Sample Album 2/False Clown Anemonefish, Bali, Indonesia.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Sample Album 2/thumbnails/False Clown Anemonefish, Bali, Indonesia.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Sample Album 2/Feeling Crabby_.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Sample Album 2/Feeling Crabby_.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Sample Album 2/thumbnails/Feeling Crabby_.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Sample Album 2/Flame Angelfish.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Sample Album 2/Flame Angelfish.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Sample Album 2/thumbnails/Flame Angelfish.jpg (JPEG)
-> Album 1 àvec UTF-8 : 9 images
INFO: Processing /home/br/www/t/pictures/Album 1 àvec UTF-8/Snow - 01.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/Snow - 01.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/thumbnails/Snow - 01.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Album 1 àvec UTF-8/Snow - 02.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/Snow - 02.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/thumbnails/Snow - 02.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Album 1 àvec UTF-8/Snow - 03.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/Snow - 03.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/thumbnails/Snow - 03.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Album 1 àvec UTF-8/Snow - 04.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/Snow - 04.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/thumbnails/Snow - 04.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Album 1 àvec UTF-8/Snow - 05.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/Snow - 05.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/thumbnails/Snow - 05.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Album 1 àvec UTF-8/Snow - 06.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/Snow - 06.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/thumbnails/Snow - 06.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Album 1 àvec UTF-8/Snow - 07.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/Snow - 07.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/thumbnails/Snow - 07.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Album 1 àvec UTF-8/Snow - 08.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/Snow - 08.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/thumbnails/Snow - 08.jpg (JPEG)
INFO: Processing /home/br/www/t/pictures/Album 1 àvec UTF-8/Snow - 09.jpg
DEBUG: Processor: ResizeToFit
DEBUG: Save resized image to /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/Snow - 09.jpg (JPEG)
DEBUG: Save thumnail image: /export/home/br/www/w/sigal/forPatrick/Album 1 àvec UTF-8/thumbnails/Snow - 09.jpg (JPEG)
INFO: Album (path='.', title=u'pictures') is empty
INFO: Theme : /usr/local/lib/python2.7/dist-packages/sigal-0.8.1-py2.7.egg/sigal/themes/galleria
INFO: Output album : (path=u'Sample Album 2', title=u'Sample Album 2')
Traceback (most recent call last):
File "/usr/local/bin/sigal", line 9, in
load_entry_point('sigal==0.8.1', 'console_scripts', 'sigal')()
File "/usr/local/lib/python2.7/dist-packages/click-3.3-py2.7.egg/click/core.py", line 610, in call
return self.main(_args, *_kwargs)
File "/usr/local/lib/python2.7/dist-packages/click-3.3-py2.7.egg/click/core.py", line 590, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click-3.3-py2.7.egg/click/core.py", line 936, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python2.7/dist-packages/click-3.3-py2.7.egg/click/core.py", line 782, in invoke
return ctx.invoke(self.callback, *_ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click-3.3-py2.7.egg/click/core.py", line 416, in invoke
return callback(_args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/sigal-0.8.1-py2.7.egg/sigal/init.py", line 133, in build
gal.build(force=force)
File "/usr/local/lib/python2.7/dist-packages/sigal-0.8.1-py2.7.egg/sigal/gallery.py", line 549, in build
writer.write(album)
File "/usr/local/lib/python2.7/dist-packages/sigal-0.8.1-py2.7.egg/sigal/writer.py", line 110, in write
page = self.template.render(**self.generate_context(album))
File "/usr/local/lib/python2.7/dist-packages/Jinja2-2.7.3-py2.7.egg/jinja2/environment.py", line 969, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/local/lib/python2.7/dist-packages/Jinja2-2.7.3-py2.7.egg/jinja2/environment.py", line 742, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/sigal-0.8.1-py2.7.egg/sigal/themes/galleria/templates/index.html", line 22, in top-level template code
<h1><a href="{{ album.index_url }}"> {{ index_title }}</a></h1>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

@saimn saimn closed this in d42915a Nov 23, 2014

@saimn

This comment has been minimized.

Owner

saimn commented Nov 23, 2014

@braoult It should be ok now, but if I understand correctly, you set the title (with special characters) in the settings file ? If this is the case you should use the unicode notation (u" ... ") because the settings file is a python file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment