New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find in file failing - UnicodeDecodeError: 'utf8' codec can't decode byte #1602

Closed
jolyonralph opened this Issue May 25, 2016 · 6 comments

Comments

Projects
None yet
3 participants
@jolyonralph

jolyonralph commented May 25, 2016

Short Summary

Find in files is always failing in my directory tree (quite large, mounted over NFS from a linux server to my Mac, handled via filesystem within Komodo).

Steps to Reproduce

Search for any phrase in all files.

Expected results

List of files containing matching phrase!

Actual results

A small number of files (100+) are scanned correctly, then one is clearly causing it to fail

Error: UnicodeDecodeError: 'utf8' codec can't decode byte 0xfd in position 3: invalid start byte

IMPORTANT: Despite Error the Log file is useless in that it doesn't pinpoint which file has the problem. So if the problem is due to one of my files having a corrupt UTF8 BOM (quite possible) then it isn't telling me which file. With 600+ files in root (don't ask) of project this is not easy to track down otherwise.

Platform Information

Komodo IDE
Komodo Version v9 (registered) and v10 (21-day trial) BOTH have example same issue.
Mac OS X 10.11.4

Additional Information

[2016-05-25 01:25:44,055] [INFO] Startup: Welcome to Komodo IDE 10.0.0 build 89159 (platform macosx, running on Darwin 15.4.0 version Darwin Kernel Version 15.4.0: Fri Feb 26 22:08:05 PST 2016; root:xnu-3248.40.184~3/RELEASE_X86_64)
[2016-05-25 01:25:44,055] [INFO] Startup: /Applications/Komodo IDE 10.app/Contents/MacOS/komodo built on Wed May 11 15:42:24 2016
[2016-05-25 01:25:44,111] [WARNING] koInitService: Unable to determine the current locale settings, defaulting to mac-roman
[2016-05-25 01:25:44,431] [WARNING] console-logger: mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create (1) in resource://gre/modules/Preferences.jsm:381
[2016-05-25 01:25:45,042] [ERROR] console-logger: ReferenceError: require is not defined (2) in chrome://komodo/content/bindings/windows.xml:39
Traceback (most recent call last):
File "chrome://komodo/content/bindings/windows.xml", line 39, in

[2016-05-25 01:25:45,061] [WARNING] root: ko.logging has been converted to a CommonJS module; use require("ko/logging") instead (since Komodo 9.0.0a1).
@chrome://komodo/content/komodo.js:15:1

[2016-05-25 01:25:45,283] [ERROR] console-logger: TypeError: redeclaration of variable val (2) in chrome://analytics/content/analytics.js:204
Traceback (most recent call last):
File "chrome://analytics/content/analytics.js", line 204, in

[2016-05-25 01:25:46,697] [INFO] koInitService: Adding pre startup service for 'koFileStatusService': '@activestate.com/koFileStatusService;1'
[2016-05-25 01:25:46,774] [INFO] komodospellchecker: Spell checker loaded.
[2016-05-25 01:25:46,775] [INFO] komodospellchecker: Using dictionary en-US
[2016-05-25 01:25:46,829] [INFO] elastic_tabstops: Elastic tabstops loaded.
[2016-05-25 01:25:47,283] [WARNING] console-logger: mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create (1) in file:///Applications/Komodo%20IDE%2010.app/Contents/Resources/components/koamAddonManager.js:138
[2016-05-25 01:25:47,762] [ERROR] console-logger: Exception while registering Focus Mode (0) in resource://gre/modules/addons/XPIProvider.jsm -> file:///Applications/Komodo%20IDE%2010.app/Contents/Resources/extensions/focusmode@activestate.com/bootstrap.js:11
Traceback (most recent call last):
File "resource://gre/modules/addons/XPIProvider.jsm -> file:///Applications/Komodo%20IDE%2010.app/Contents/Resources/extensions/focusmode@activestate.com/bootstrap.js", line 11, in

[2016-05-25 01:25:47,765] [ERROR] console-logger: object Object in resource://gre/modules/addons/XPIProvider.jsm -> file:///Applications/Komodo%20IDE%2010.app/Contents/Resources/extensions/focusmode@activestate.com/bootstrap.js:12
Traceback (most recent call last):
File "resource://gre/modules/addons/XPIProvider.jsm -> file:///Applications/Komodo%20IDE%2010.app/Contents/Resources/extensions/focusmode@activestate.com/bootstrap.js", line 12, in

[2016-05-25 01:25:48,809] [WARNING] console-logger: Unknown property '-moz-border-radius'. Declaration dropped. (1) in chrome://docksearch/skin/styles.css:6
[2016-05-25 01:25:50,444] [INFO] koInitService: Adding pre startup service for 'KoMemoryReporter': '@activestate.com/koMemoryReporter;1'
[2016-05-25 01:25:50,446] [INFO] koInitService: Adding pre startup service for 'koCommandmentService': '@activestate.com/koCommandmentService;1'
[2016-05-25 01:25:50,449] [INFO] koInitService: Adding pre startup service for 'koDBGPManager': '@activestate.com/koDBGPManager;1'
[2016-05-25 01:26:07,383] [ERROR] koScopeDocs-Py: Could not find cix for ext
[2016-05-25 01:26:07,385] [ERROR] koScopeDocs-Py: Could not find cix for html5
Exception in thread Find/Replace Thread 1:
Traceback (most recent call last):
File "/Applications/Komodo IDE 10.app/Contents/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 808, in __bootstrap_inner
self.run()
File "/Applications/Komodo IDE 10.app/Contents/Resources/components/koFinder.py", line 192, in run
self._find_in_paths(self.regex, self.paths)
File "/Applications/Komodo IDE 10.app/Contents/Resources/components/koFinder.py", line 254, in _find_in_paths
textInfoFactory=self.loadedFileTextFactory):
File "/Applications/Komodo IDE 10.app/Contents/Resources/python/komodo/findlib2.py", line 169, in grep
for path in paths:
File "/Applications/Komodo IDE 10.app/Contents/Resources/python/komodo/findlib2.py", line 1602, in paths_from_path_patterns
filepath = os.path.join(path,filename)
File "/Applications/Komodo IDE 10.app/Contents/MacOS/./../Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py", line 80, in join
path += '/' + b
UnicodeDecodeError: 'utf8' codec can't decode byte 0xfd in position 3: invalid start byte

@mitchell-as mitchell-as added this to the 10.1 milestone May 25, 2016

@mitchell-as mitchell-as self-assigned this May 25, 2016

@mitchell-as

This comment has been minimized.

Member

mitchell-as commented May 25, 2016

Thanks for the report. I'll make sure the filename is reported in the error message.

@jolyonralph

This comment has been minimized.

jolyonralph commented May 25, 2016

Is there any way to figure out how to find the culprit on my system now?
I'm really stuck without a useable find in file on my project. It's getting
to the point where I'm having to investigate alternative IDEs :( I'd prefer
not to but this is causing me big issues.

Jolyon

On 25 May 2016 at 15:11, mitchell notifications@github.com wrote:

Thanks for the report. I'll make sure the filename is reported in the
error message.


You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#1602 (comment)

@jolyonralph

This comment has been minimized.

jolyonralph commented May 25, 2016

Also, would it not be better for find in file to report that a file
couldn't be read and continue rather than just die?

On 25 May 2016 at 15:22, Jolyon Ralph jolyon@mindat.org wrote:

Is there any way to figure out how to find the culprit on my system now?
I'm really stuck without a useable find in file on my project. It's getting
to the point where I'm having to investigate alternative IDEs :( I'd prefer
not to but this is causing me big issues.

Jolyon

On 25 May 2016 at 15:11, mitchell notifications@github.com wrote:

Thanks for the report. I'll make sure the filename is reported in the
error message.


You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#1602 (comment)

@mitchell-as mitchell-as modified the milestones: 10.0.1, 10.1 May 25, 2016

@mitchell-as

This comment has been minimized.

Member

mitchell-as commented May 25, 2016

I'll bump this up and try to get it in our next nightly build.

@Naatan

This comment has been minimized.

Member

Naatan commented May 25, 2016

@mitchell-as please make it fail gracefully while you're there. ie. as @jolyonralph suggested - make it skip files that throw exceptions (but still log them, of course).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment