Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception on unencodable filename #891

Closed
zooba opened this issue Dec 17, 2019 · 4 comments
Closed

Exception on unencodable filename #891

zooba opened this issue Dec 17, 2019 · 4 comments
Labels
Milestone

Comments

@zooba
Copy link

@zooba zooba commented Dec 17, 2019

Apologies for the quick report - I'm only seeing this in the CPython GitHub Actions CI against Python 3.9 right now. I'll come back with more details when I get a chance, but wanted to provide a heads-up in case this is an obvious enough issue.

See the build logs at https://github.com/python/cpython/commit/9707e8e22d80ca97bf7a9812816701cecde6d226/checks?check_suite_id=363085844 - to navigate there, "Coverage" / "Ubuntu (Coverage)" / "Tests with coverage". The build step is marked as successful because we suppress the failure (and then the upload fails because the file was never written).

  File "/home/runner/work/cpython/cpython/.venv/lib/python3.9/site-packages/coverage/cmdline.py", line 555, in command_line
    return self.do_run(options, args)
  File "/home/runner/work/cpython/cpython/.venv/lib/python3.9/site-packages/coverage/cmdline.py", line 710, in do_run
    self.coverage.save()
  File "/home/runner/work/cpython/cpython/.venv/lib/python3.9/site-packages/coverage/control.py", line 613, in save
    data = self.get_data()
  File "/home/runner/work/cpython/cpython/.venv/lib/python3.9/site-packages/coverage/control.py", line 667, in get_data
    if self._collector and self._collector.flush_data():
  File "/home/runner/work/cpython/cpython/.venv/lib/python3.9/site-packages/coverage/collector.py", line 427, in flush_data
    self.covdata.add_arcs(self.mapped_file_dict(self.data))
  File "/home/runner/work/cpython/cpython/.venv/lib/python3.9/site-packages/coverage/sqldata.py", line 474, in add_arcs
    file_id = self._file_id(filename, add=True)
  File "/home/runner/work/cpython/cpython/.venv/lib/python3.9/site-packages/coverage/sqldata.py", line 367, in _file_id
    cur = con.execute("insert or replace into file (path) values (?)", (filename,))
  File "/home/runner/work/cpython/cpython/.venv/lib/python3.9/site-packages/coverage/sqldata.py", line 1025, in execute
    return self.con.execute(sql, parameters)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position 74: surrogates not allowed

I suspect the 0xDCFF character comes from one of our deliberately-failing tests, but it seems to be coming from a point that we couldn't possibly override in coverage without changing the test itself.

As I said, when I get a few more minutes, I'll try and turn this into a minimal repro. But wanted to at least provide the easily-accessible info now rather than delaying any report by my availability.

@zooba zooba added the bug label Dec 17, 2019
@nedbat nedbat added this to the 5.0.1 milestone Dec 18, 2019
@zooba

This comment has been minimized.

Copy link
Author

@zooba zooba commented Dec 18, 2019

Here is a minimal repro script:

exec(compile("pass", "\udcff.py", "exec"))

Running coverage run test-file.py produces the same traceback as above.

@nedbat nedbat modified the milestones: 5.0.1, 5.0.2 Dec 22, 2019
@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Jan 1, 2020

@zooba @gpshead - I'm not sure what to do here. SQLite insists on encoding the string, and all of its choices for encodings reject the surrogate. I suppose I could simply refuse to record data whose filename is not encodable. Would that be enough?

nedbat added a commit that referenced this issue Jan 1, 2020
nedbat added a commit that referenced this issue Jan 1, 2020
@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Jan 1, 2020

I've gone ahead and changed coverage to not measure files whose names can't be recorded, in 016af5f.

@nedbat nedbat closed this Jan 1, 2020
@nedbat nedbat added the fixed label Jan 1, 2020
@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Jan 5, 2020

This is now available in coverage 5.0.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.