Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Found bug, fixed it, here is my solution.... #1876

Closed
deeredman1991 opened this issue Jan 23, 2018 · 8 comments
Closed

Found bug, fixed it, here is my solution.... #1876

deeredman1991 opened this issue Jan 23, 2018 · 8 comments

Comments

@deeredman1991
Copy link

deeredman1991 commented Jan 23, 2018

I already fixed the issue, so I don't need help, but I popped in to let you guys know there is an issue and how I solved it.

So I configured my vimrc to use python3

let g:powerline_pycmd = 'py3'

after that I started getting an exception in my vim console, so I output it to a logfile

This was the file:

2018-01-23 01:40:12,716:DEBUG:vim:Using libuv-based watcher
2018-01-23 01:40:16,876:ERROR:vim:branch:Exception while computing segment: _getvolumepathname() argument 1 must be str, not bytes
Traceback (most recent call last):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segment.py", line 173, in process_segment
    contents = segment['contents_func'](pl, segment_info)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segment.py", line 412, in <lambda>
    contents_func = lambda pl, segment_info: _contents_func(pl=pl, segment_info=segment_info, **args)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segments\common\vcs.py", line 21, in __call__
    repo = guess(path=name, create_watcher=create_watcher)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\lib\vcs\__init__.py", line 236, in guess
    for directory in generate_directories(path):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\lib\vcs\__init__.py", line 23, in generate_directories
    if os.path.ismount(path):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\ntpath.py", line 294, in ismount
    return path.rstrip(seps) == _getvolumepathname(path).rstrip(seps)
TypeError: _getvolumepathname() argument 1 must be str, not bytes
2018-01-23 01:40:16,877:ERROR:vim:file_vcs_status:Exception while computing segment: _getvolumepathname() argument 1 must be str, not bytes
Traceback (most recent call last):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segment.py", line 173, in process_segment
    contents = segment['contents_func'](pl, segment_info)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segment.py", line 412, in <lambda>
    contents_func = lambda pl, segment_info: _contents_func(pl=pl, segment_info=segment_info, **args)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segments\vim\__init__.py", line 555, in file_vcs_status
    repo = guess(path=name, create_watcher=create_watcher)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\lib\vcs\__init__.py", line 236, in guess
    for directory in generate_directories(path):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\lib\vcs\__init__.py", line 23, in generate_directories
    if os.path.ismount(path):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\ntpath.py", line 294, in ismount
    return path.rstrip(seps) == _getvolumepathname(path).rstrip(seps)
TypeError: _getvolumepathname() argument 1 must be str, not bytes

turns out it was an issue with python3 using bytes instead of strings for some things now...
So tl;dr I changed these lines of code and it fixed the issue on my end (blocks/lines I added are commented)

IN: .\powerline\lib\vcs\__init__.py:

def generate_directories(path):
	#Converts path from 'bytes' to 'str' object
	path = path.decode('utf-8')
	if os.path.isdir(path):
		#Converts path back into bytes and sends it out to wherever it needs to go.
		yield bytes(path, 'utf-8')
	while True:
		if os.path.ismount(path):
			break
		old_path = path
		path = os.path.dirname(path)
		if path == old_path or not path:
			break
		#Converts path back into bytes and sends it out to wherever it needs to go.
		yield bytes(path, 'utf-8')

IN: .\powerline\lib\shell.py:

def readlines(cmd, cwd):
	'''Run command and read its output, line by line

	:param list cmd:
		Command which will be run.
	:param str cwd:
		Working directory of the command which will be run.
	'''
	
	#Converts every 'bytes' object in cmd into a str object
	cmd = list(cmd)
	for i in range(len(cmd)):
		if type(cmd[i]) == type(bytes('', 'utf-8')):
			cmd[i] = cmd[i].decode()
	cmd = tuple(cmd)
	
	#converts cwd to 'str' object
	cwd = cwd.decode()
	
	p = Popen(cmd, shell=False, stdout=PIPE, stderr=PIPE, cwd=cwd)
	encoding = get_preferred_input_encoding()
	p.stderr.close()
	with p.stdout:
		for line in p.stdout:
			yield line[:-1].decode(encoding)

I'm sure there are more elegant ways of doing this (and that is why I am not issuing a pull request) but feel free to make any modifications to and/or use this solution in your codebase if you want. 👍

@ZyX-I
Copy link
Contributor

ZyX-I commented Jan 24, 2018

It is not really valid to assume specific encoding of the paths: in linux file name is absolutely any sequence of bytes which does not include / (for it being a path separator) and NUL (for it terminating a C string). Command arguments are only not allowed to have NUL, but otherwise lack any encoding requirements. According to Python documentation, os.path.ismount must accept any path-like object, and “path-like” does include bytes. Same goes for Popen’s cwd. And though I don’t find any details of what kind of strings it accepts as an arguments, I do not know any code which will call that with non-unicode (non-str in Python-3*) strings.

@ZyX-I ZyX-I closed this as completed Jan 24, 2018
@deeredman1991
Copy link
Author

deeredman1991 commented Jan 25, 2018

@ZyX-I While it is true that "os.path.ismount must accept any path-like object, and "path-like" does include bytes". If you notice; I modified the yield statements in generate_directories to cast path from a string to a byte. The original code, when I first opened the file, was:

def generate_directories(path):
	if os.path.isdir(path):
		yield path
	while True:
		if os.path.ismount(path):
			break
		old_path = path
		path = os.path.dirname(path)
		if path == old_path or not path:
			break
		yield path

^Copied from Github repository.

The issue was that path was a string and os.path.ismount was expecting a byte(or other "path-like" object I assume).

Also, (and I could be wrong on this) but I think I tried casting cmd[0:-1] and cwd to bytes first and that didn't work so I casted it all to strings and that fixed the issue. :)

@ZyX-I
Copy link
Contributor

ZyX-I commented Jan 25, 2018

“Path-like object” both applies to str() and bytes(), so no matter what generate_directories actually accepted of these two it should actually work, and the code should be fine as long as the caller knows how to deal with generate_directories() returning the same type of object it was called with.

@deeredman1991
Copy link
Author

@ZyX-I Oh wow, yeah, I just tested it, you are absolutely right. os.path.ismount() takes bytes AND strings...My question is then; why did python throw an error? and more importantly...how did my code fix it? (⊙ヘ⊙ ; )

@deeredman1991
Copy link
Author

deeredman1991 commented Jan 26, 2018

Yeah...I just reverted the code back to the way it was and threw a print statement in ntpath.py

try:
    from nt import _getvolumepathname
except ImportError:
    _getvolumepathname = None
def ismount(path):
    """Test whether a path is a mount point (a drive root, the root of a
    share, or a mounted volume)"""
    path = os.fspath(path)
    seps = _get_bothseps(path)
    path = abspath(path)
    root, rest = splitdrive(path)
    
    #I added this line...
    print('{0} is of type {1}'.format( path, type(path) ))
    
    if root and root[0] in seps:
        return (not rest) or (rest in seps)
    if rest in seps:
        return True
        
    if _getvolumepathname:
        return path.rstrip(seps) == _getvolumepathname(path).rstrip(seps)
    else:
        return False

This is what I got out:

b'J:\\PCTools\\cmder_mini\\test.py' is of type <class 'bytes'>
2018-01-26 11:33:20,291:ERROR:vim:branch:Exception while computing segment: _getvolumepathname() argument 1 must be str, not bytes
Traceback (most recent call last):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segment.py", line 173, in process_segment
    contents = segment['contents_func'](pl, segment_info)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segment.py", line 412, in <lambda>
    contents_func = lambda pl, segment_info: _contents_func(pl=pl, segment_info=segment_info, **args)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segments\common\vcs.py", line 21, in __call__
    repo = guess(path=name, create_watcher=create_watcher)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\lib\vcs\__init__.py", line 236, in guess
    for directory in generate_directories(path):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\lib\vcs\__init__.py", line 24, in generate_directories
    if os.path.ismount(path):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\ntpath.py", line 297, in ismount
    return path.rstrip(seps) == _getvolumepathname(path).rstrip(seps)
TypeError: _getvolumepathname() argument 1 must be str, not bytes
b'J:\\PCTools\\cmder_mini\\test.py' is of type <class 'bytes'>
2018-01-26 11:33:20,294:ERROR:vim:file_vcs_status:Exception while computing segment: _getvolumepathname() argument 1 must be str, not bytes
Traceback (most recent call last):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segment.py", line 173, in process_segment
    contents = segment['contents_func'](pl, segment_info)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segment.py", line 412, in <lambda>
    contents_func = lambda pl, segment_info: _contents_func(pl=pl, segment_info=segment_info, **args)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\segments\vim\__init__.py", line 555, in file_vcs_status
    repo = guess(path=name, create_watcher=create_watcher)
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\lib\vcs\__init__.py", line 236, in guess
    for directory in generate_directories(path):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\site-packages\powerline\lib\vcs\__init__.py", line 24, in generate_directories
    if os.path.ismount(path):
  File "J:\ProgrammingStuff\ProgrammingTools\Python36\lib\ntpath.py", line 297, in ismount
    return path.rstrip(seps) == _getvolumepathname(path).rstrip(seps)
TypeError: _getvolumepathname() argument 1 must be str, not bytes
b'J:\\PCTools\\cmder_mini\\test.py' is of type <class 'bytes'>
b'J:\\PCTools\\cmder_mini\\test.py' is of type <class 'bytes'>
b'J:\\PCTools\\cmder_mini\\test.py' is of type <class 'bytes'>
b'J:\\PCTools\\cmder_mini\\test.py' is of type <class 'bytes'>

so as you can see it does confirm that b'J:\\PCTools\\cmder_mini\\test.py' is of type <class 'bytes'> before it throws the error. So I have no idea...it might be possible that we just stumbled upon a bug in python...that...or there is a problem somewhere on my end... (⊙_⊙ ; )

I tried to find the nt module and add some print statements to _getvolumepathname() but it just keeps telling me that nt is a built in module...I guess that probably means it's a binary somewhere in my Python36 directory...There isn't a whole lot of info on it on Google...if you know how I can take a look at it; let me know and I'll go try some more print debugging.

I guess for now I'll just uncomment out my code and forget about it...at least until my PC explodes...lol</nervous laughter>

@ZyX-I
Copy link
Contributor

ZyX-I commented Jan 28, 2018

“built-in” here means that it is in C code, possible, but not necessary, in the Python library/executable itself. Based on what I know on writing C extensions it will just have some call like PyArg_ParseTuple and it is what throws. Note a note near the s format.

@ZyX-I
Copy link
Contributor

ZyX-I commented Jan 28, 2018

And it appears that Python has placed its nt module into Modules/posixmodule.c, and I forgot about another variant of parsing: you can declare function as accepting a specific set of arguments. What throws should be some kind of generated code because function itself uses PyUnicode_AsUnicodeAndSize, which calls PyErr_BadArgument on non-unicode arguments and that may only throw error with less helpful message.

@ZyX-I
Copy link
Contributor

ZyX-I commented Jan 28, 2018

https://bugs.python.org/issue32693

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants