Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

unicode encoding error in sqlalchemy causing app exit #240

Open
redshodan opened this issue Nov 26, 2017 · 1 comment
Open

unicode encoding error in sqlalchemy causing app exit #240

redshodan opened this issue Nov 26, 2017 · 1 comment

Comments

@redshodan
Copy link
Collaborator

Syncing with unsonic/mishmash got this error which caused an application exit. It doesn't say what file it was working on when this happened, but syncing this one directory does it.

baron@dib ~/src/unsonic $ ./bin/unsonic -c foo.ini sync '/data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/'
2017-11-24 09:51:44,791 DEBUG [mishmash.database][MainThread] Checking for database 'sqlite:////home/baron/src/unsonic/build/development.sqlite'
2017-11-24 09:51:44,795 DEBUG [mishmash.database][MainThread] Connecting to database 'sqlite:////home/baron/src/unsonic/build/development.sqlite'
Syncing library 'Music': paths=['/data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/']
2017-11-24 09:51:44,802 INFO  [mishmash.commands.sync.sync][MainThread] Syncing library 'Music': paths=['/data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/']
Adding artist: Franz Ferdinand
Adding album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/01 Jacqueline.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/02 Tell Her Tonight.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/03 Take Me Out.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/05 Auf Achse.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/06 Cheating on You.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/07 This Fire.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/08 Darts of Pleasure.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/09 Michael.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/10 Come On Home.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/11 40'.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/Franz Ferdinand - 01 - Jacqueline.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/Franz Ferdinand - 02 - Tell Her Tonight.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/Franz Ferdinand - 03 - Take Me Out.mp3
2017-11-24 09:51:45,041 ERROR [mishmash][MainThread] 'utf-8' codec can't encode character '\udce9' in position 104: surrogates not allowed
Traceback (most recent call last):
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/__main__.py", line 53, in main
    retval = args.command_func(args, args.config) or 0
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/core.py", line 20, in run
    retval = super().run(args)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/nicfit/command.py", line 62, in run
    return self._run()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 458, in _run
    result = _syncLib(lib)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 451, in _syncLib
    return eyed3_main(args, None)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/eyed3/main.py", line 50, in main
    fs_encoding=args.fs_encoding)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/eyed3/utils/__init__.py", line 112, in walk
    handler.handleDirectory(root, files)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 324, in handleDirectory
    session)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 161, in _syncAudioFile
    .filter_by(path=path, lib_id=self._lib.id).one()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2814, in one
    ret = self.one_or_none()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2784, in one_or_none
    ret = list(self)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__
    return self._execute_and_instances(context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2878, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1405, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce9' in position 104: surrogates not allowed
General error:
Traceback (most recent call last):
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/__main__.py", line 53, in main
    retval = args.command_func(args, args.config) or 0
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/core.py", line 20, in run
    retval = super().run(args)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/nicfit/command.py", line 62, in run
    return self._run()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 458, in _run
    result = _syncLib(lib)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 451, in _syncLib
    return eyed3_main(args, None)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/eyed3/main.py", line 50, in main
    fs_encoding=args.fs_encoding)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/eyed3/utils/__init__.py", line 112, in walk
    handler.handleDirectory(root, files)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 324, in handleDirectory
    session)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 161, in _syncAudioFile
    .filter_by(path=path, lib_id=self._lib.id).one()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2814, in one
    ret = self.one_or_none()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2784, in one_or_none
    ret = list(self)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__
    return self._execute_and_instances(context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2878, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1405, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce9' in position 104: surrogates not allowed
2017-11-24 09:51:45,044 DEBUG [unsonic][MainThread] Application._atexit
baron@dib ~/src/unsonic $
@redshodan
Copy link
Collaborator Author

It has to do, somehow that isn't immediately obvious, with the path variable in the sync._syncAudioFile() function. audo_file.path is fine, and path is set equal to that but its bad.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@redshodan