Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New user: Plugin doesn't work #127

Closed
traskrogers opened this issue Mar 16, 2016 · 22 comments
Closed

New user: Plugin doesn't work #127

traskrogers opened this issue Mar 16, 2016 · 22 comments

Comments

@traskrogers
Copy link

I randomly had the idea that Evernote would be much better if I could use it through Sublime and a few searches led me here. Wonderful!

A fresh setup on Sublime 3 (build 3103) doesn't work for me. I've setup the dev key.

If I create a new note and then try to "send as new note" I get the following errors in the console:

Evernote plugin error: unpack requires a bytes object of length 1
    Last command: insert {'characters': '\nblah'}
    BEFORE SUBMITTING AN ISSUE (https://github.com/bordaigorl/sublime-evernote/issues):
      1. Enable the `debug` setting in your Evernote.sublime-settings file and try again. If the problem persists take a note of the output in the console.
         Make sure you delete personal information (e.g. Developer Token) from the output before posting it in an issue.
      2. Check the wiki at https://github.com/bordaigorl/sublime-evernote/wiki
      3. Search for similar issues at https://github.com/bordaigorl/sublime-evernote/issues?q=is%3Aissue
    (Evernote plugin v2.7.0, ST 3103, Python 3.3.3, linux x64, debug)
error: Evernote plugin error, please see the console for more details.
Then contact developer at
https://github.com/bordaigorl/sublime-evernote/issues
Evernote: unpack requires a bytes object of length 1
Evernote: <p>blah</p>

Evernote: []
Evernote plugin error: unpack requires a bytes object of length 1
    Last command: insert {'characters': '\nblah'}
    BEFORE SUBMITTING AN ISSUE (https://github.com/bordaigorl/sublime-evernote/issues):
      1. Enable the `debug` setting in your Evernote.sublime-settings file and try again. If the problem persists take a note of the output in the console.
         Make sure you delete personal information (e.g. Developer Token) from the output before posting it in an issue.
      2. Check the wiki at https://github.com/bordaigorl/sublime-evernote/wiki
      3. Search for similar issues at https://github.com/bordaigorl/sublime-evernote/issues?q=is%3Aissue
    (Evernote plugin v2.7.0, ST 3103, Python 3.3.3, linux x64, debug)
error: Evernote plugin error, please see the console for more details.
Then contact developer at
https://github.com/bordaigorl/sublime-evernote/issues
Evernote: unpack requires a bytes object of length 1
@bordaigorl
Copy link
Owner

Hi and welcome!
I am very sorry but you got bitten by one of the few bugs I have no clue how to fix.
It has been reported before (see #40 #59 #89) but I cannot reproduce it so I cannot even start debugging it properly.
From a preliminary study it would seem it is a bug of the Evernote API bindings but I am not sure.
Could you check if you are behind a proxy or a corporate firewall?

@traskrogers
Copy link
Author

I'm not behind a proxy nor does my machine have a software firewall running. We have a corporate firewall but it does very little to outbound traffic. I just tried disabling that temporarily and the behavior is unchanged.

I found those other issues you mentioned when trying to diagnose the problem. I'm happy to dig deeper if you've got pointers. I'm on Ubuntu 14.04.

@bordaigorl
Copy link
Owner

@traskrogers Thanks for offering to help.
My current guess is that the problem arises from an incompatibility between the Evernote SDK and python 3.
In fact the SDK for python 3 is declared of "beta" quality.
To make everything work they used 2to3 on the libraries, including the Thrift support libraries.
I am not an expert but the Thrift libraries seem to be the ones responsible for the low-level handling of the communication protocol with the server.
I think some feature related to unpack is subtly different between python 2 (officially supported) and python 3 (only experimental).
Since in Sublime Text 3 we are stuck with python 3, we need to patch things up.
I tried to prompt the Evernote team before but without success.

I found the following on StackOverflow:
http://stackoverflow.com/a/33956718/2753846
It may be worth trying to patch the Thrift library as they describe and see if it works.
Let me know if you need assistance with this.

@traskrogers
Copy link
Author

I have done a "pip3 install thrift" and made the modifications per the StackOverflow thread. It changes the error message (shown below). Following another SO thread I changed line 122 in TBinaryProtocol.py from buff = bytes(str, 'utf-8') to buff = str(str, 'utf-8') also without success.

I'm not sure how the Python module on disk interacts with the Sublime plugin. There was no "thrift" module installed on my system prior to me installing it manually but the Sublime plugin still seemed to work. Is it embedded within the sublime plugin? How do I ensure my changes on disk have an effect on the sublime plugin? I got differing behaviors when I uninstalled/reinstalled the sublime plugin and when I just did a restart of sublime or retried certain Evernote plugin commands.

Error after the changes outline in the SO post you linked:

TypeError: encoding or errors without a string argument

Traceback (most recent call last):
File "/opt/sublime_text/sublime_plugin.py", line 377, in on_query_completions
res = callback.on_query_completions(v, prefix, locations)
File "sublime_evernote in /home/user/.config/sublime-text-3/Installed Packages/Evernote.sublime-package", line 1459, in on_query_completions
File "sublime_evernote in /home/user/.config/sublime-text-3/Installed Packages/Evernote.sublime-package", line 467, in cache_all_tags
File "/home/user/.config/sublime-text-3/Installed Packages/Evernote.sublime-package/lib/evernote/edam/notestore/NoteStore.py", line 3129, in listTags
self.send_listTags(authenticationToken)
File "/home/user/.config/sublime-text-3/Installed Packages/Evernote.sublime-package/lib/evernote/edam/notestore/NoteStore.py", line 3136, in send_listTags
args.write(self._oprot)
File "/home/user/.config/sublime-text-3/Installed Packages/Evernote.sublime-package/lib/evernote/edam/notestore/NoteStore.py", line 10578, in write
oprot.writeString(self.authenticationToken.encode('utf-8'))
File "/usr/local/lib/python3.4/dist-packages/thrift/protocol/TBinaryProtocol.py", line 122, in writeString
buff = bytes(str, 'utf-8')
TypeError: encoding or errors without a string argument
Evernote plugin error: encoding or errors without a string argument
Last command: {}
BEFORE SUBMITTING AN ISSUE (https://github.com/bordaigorl/sublime-evernote/issues):
1. Enable the `debug` setting in your Evernote.sublime-settings file and try again. If the problem persists take a note of the output in the console.
Make sure you delete personal information (e.g. Developer Token) from the output before posting it in an issue.
2. Check the wiki at https://github.com/bordaigorl/sublime-evernote/wiki
3. Search for similar issues at https://github.com/bordaigorl/sublime-evernote/issues?q=is%3Aissue
(Evernote plugin v2.7.0, ST 3103, Python 3.3.3, linux x64, debug)
error: Evernote plugin error, please see the console for more details.
Then contact developer at
https://github.com/bordaigorl/sublime-evernote/issues
Evernote: encoding or errors without a string argument
Evernote: <p>aadfa</p>

Evernote: []
Evernote plugin error: encoding or errors without a string argument
Last command: {}
BEFORE SUBMITTING AN ISSUE (https://github.com/bordaigorl/sublime-evernote/issues):
1. Enable the `debug` setting in your Evernote.sublime-settings file and try again. If the problem persists take a note of the output in the console.
Make sure you delete personal information (e.g. Developer Token) from the output before posting it in an issue.
2. Check the wiki at https://github.com/bordaigorl/sublime-evernote/wiki
3. Search for similar issues at https://github.com/bordaigorl/sublime-evernote/issues?q=is%3Aissue
(Evernote plugin v2.7.0, ST 3103, Python 3.3.3, linux x64, debug)
error: Evernote plugin error, please see the console for more details.
Then contact developer at
https://github.com/bordaigorl/sublime-evernote/issues
Evernote: encoding or errors without a string argument

@bordaigorl
Copy link
Owner

Hi, thanks for trying this.
One thing before we continue: there are two separate python installations in your system. One is the system wide one that runs directly on your OS. The other is the one embedded within Sublime Text.
With pip you only install stuff for the system-wide python, which is completely disjoint from the ST embedded one. Indeed, to add libraries to ST-python for use in plugins one needs to include the sourcecode of the library in the plugin's directory so that it is available to ST-python. This is why the markdown parsing and evernote api libraries are included as sourcecode in the evernote plugin. This means that if you want to make changes to the evernote api that the plugin uses you need to modify the library included in the plugin's directory. ST3 does not store the plugin's directory directly but it packages them in .sublime-package files that are just zipped versions of the directory.
One way to contribute to a plugin is to fork the repo on github, clone it in your Sublime folder (for this plugin you have to make sure to rename the cloned folder to "Evernote" instead of the name of the repo).
Then you create a new branch off the devel one and start making changes/testing. When you obtain results you can open a Pull Request so that I can inspect the changes, review and merge.

The changes suggested in the SO thread I linked should be done on the files under Evernote/lib/thrift.
The error trace you reported is most likely due to the change to buff = str(str, 'utf-8') which seems not to be helping here.

@traskrogers
Copy link
Author

Thanks for the tips. My changes are now being reflecting as I expect they would.

Initial error with fresh clone: Evernote: unpack requires a bytes object of length 1

Following the SO post I get the error below. Note that it seems like TBinaryProtocol.py has been modified within the Evernote plugin already (the line 122-123 changes). I simply overwrote that function to be as the SO post expects it to be (and like a fresh download of Thrift 0.9.3 is).

Traceback (most recent call last):
  File "/opt/sublime_text/sublime_plugin.py", line 377, in on_query_completions
    res = callback.on_query_completions(v, prefix, locations)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/sublime_evernote.py", line 1459, in on_query_completions
    self.cache_all_tags()
  File "/home/user/.config/sublime-text-3/Packages/Evernote/sublime_evernote.py", line 467, in cache_all_tags
    tags = self.get_note_store().listTags(self.token())
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 3129, in listTags
    self.send_listTags(authenticationToken)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 3136, in send_listTags
    args.write(self._oprot)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 10578, in write
    oprot.writeString(self.authenticationToken.encode('utf-8'))
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/thrift/protocol/TBinaryProtocol.py", line 122, in writeString
    buff = bytes(str, 'utf-8')
TypeError: encoding or errors without a string argument

If I leave that section of code alone (lines 122-123) in TBinaryProtocol.py and do make the changes on line 223 I get this:

Traceback (most recent call last):
  File "/opt/sublime_text/sublime_plugin.py", line 377, in on_query_completions
    res = callback.on_query_completions(v, prefix, locations)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/sublime_evernote.py", line 1459, in on_query_completions
    self.cache_all_tags()
  File "/home/user/.config/sublime-text-3/Packages/Evernote/sublime_evernote.py", line 467, in cache_all_tags
    tags = self.get_note_store().listTags(self.token())
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 3130, in listTags
    return self.recv_listTags()
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 3148, in recv_listTags
    result.read(self._iprot)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 10632, in read
    _elem205.read(iprot)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/type/ttypes.py", line 2215, in read
    self.guid = iprot.readString().decode('utf-8')
AttributeError: 'str' object has no attribute 'decode'

If I change nothing in TBinaryProtocol.py, but do leave the changes in TTransport.py I get the original error back:
Evernote: unpack requires a bytes object of length 1

@bordaigorl
Copy link
Owner

Ok. It is not clear which edits you are making.
I would try with the following:

  • line 20

    from io import BytesIO as StringIO
    
  • line 55

    buff = b''
    
  • line 142

    self.__rbuf = StringIO("")
    

If all this keeps failing I would revert back and try to replace every pack and unpack call to the ones below (from the SO post)

from io import pack, unpack

def packStr(fmt, v):
  return "".join(["\\x{:02x}".format(i) for i in list(pack(fmt, v))])

def unpackStr(fmt, v):
  return unpack(fmt, bytes([int(i, 16) for i in v.split("\\x")[1:]]))

making sure the definitions are included in the module.

If this fails I would try to also test this in conjuction with the changes above.
If this all fails again then we need the help of somebody who knows this Thrift stuff.

@traskrogers
Copy link
Author

So I tested a few combinations of what you described (implementing pack/unpackStr). Didn't work. Here's an error:

Traceback (most recent call last):
File "/opt/sublime_text/sublime_plugin.py", line 377, in on_query_completions
res = callback.on_query_completions(v, prefix, locations)
File "/home/user/.config/sublime-text-3/Packages/Evernote/sublime_evernote.py", line 1459, in on_query_completions
self.cache_all_tags()
File "/home/user/.config/sublime-text-3/Packages/Evernote/sublime_evernote.py", line 467, in cache_all_tags
tags = self.get_note_store().listTags(self.token())
File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 3129, in listTags
self.send_listTags(authenticationToken)
File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 3133, in send_listTags
self._oprot.writeMessageBegin('listTags', TMessageType.CALL, self._seqid)
File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/thrift/protocol/TBinaryProtocol.py", line 52, in writeMessageBegin
self.writeI32(TBinaryProtocol.VERSION_1 | type)
File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/thrift/protocol/TBinaryProtocol.py", line 117, in writeI32
self.trans.write(buff)
File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/thrift/transport/THttpClient.py", line 102, in write
self.__wbuf.write(buf)
TypeError: 'str' does not support the buffer interface

While I was fooling with it I decided to grab the latest commit from the thrift git project and give it a try. It looks like they've gone a long way to working on compatibility issues with py3 but I still get problems. I had to find and drop the "six.py" module in the plugin "lib" dir to get it to run as well.

Traceback (most recent call last):
  File "/opt/sublime_text/sublime_plugin.py", line 377, in on_query_completions
    res = callback.on_query_completions(v, prefix, locations)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/sublime_evernote.py", line 1459, in on_query_completions
    self.cache_all_tags()
  File "/home/user/.config/sublime-text-3/Packages/Evernote/sublime_evernote.py", line 467, in cache_all_tags
    tags = self.get_note_store().listTags(self.token())
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 3129, in listTags
    self.send_listTags(authenticationToken)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 3136, in send_listTags
    args.write(self._oprot)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 10578, in write
    oprot.writeString(self.authenticationToken.encode('utf-8'))
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/thrift/protocol/TProtocol.py", line 121, in writeString
    self.writeBinary(str_to_binary(str_val))
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/thrift/compat.py", line 40, in str_to_binary
    return bytes(str_val, 'utf8')
TypeError: encoding or errors without a string argument

I think I'm done messing with this. Let me know if I can try anything else otherwise I'm going to drop it.

@bordaigorl
Copy link
Owner

Thanks a lot for trying.
It is difficult to solve this because I cannot reproduce the prolem, which is very strange as I have been using the plugin with various versions of ST3 on linux and OSX without any problem.
This makes it even more difficult to understand the source of the issue.

@bordaigorl
Copy link
Owner

@traskrogers could you please do one more test for me?
Just run in ST3's console

sublime.find_resources("Thrift.py")

and check that there is only one such module in ST's paths
and just to double check

import Evernote.lib.thrift as t; t.__file__

to make sure no other plugin is injecting a different version of Thrift.

Just a wild guess though 😞

@traskrogers
Copy link
Author

Your gut was right. I had made a ".original" backup of the thrift library that was also getting pulled in. I re-did the tests and I end up with this traceback:

Traceback (most recent call last):
  File "/opt/sublime_text/sublime_plugin.py", line 377, in on_query_completions
    res = callback.on_query_completions(v, prefix, locations)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/sublime_evernote.py", line 1459, in on_query_completions
    self.cache_all_tags()
  File "/home/user/.config/sublime-text-3/Packages/Evernote/sublime_evernote.py", line 467, in cache_all_tags
    tags = self.get_note_store().listTags(self.token())
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 3130, in listTags
    return self.recv_listTags()
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 3148, in recv_listTags
    result.read(self._iprot)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/notestore/NoteStore.py", line 10632, in read
    _elem205.read(iprot)
  File "/home/user/.config/sublime-text-3/Packages/Evernote/lib/evernote/edam/type/ttypes.py", line 2215, in read
    self.guid = iprot.readString().decode('utf-8')
AttributeError: 'str' object has no attribute 'decode'

I fiddled for 20 minutes trying to blindly adjust things here and there in the evernote/edam library area. I didn't get much further. I feel like you've got the issue pretty narrowed down but I'm not sure where to best address it. I used some print statements to see that some strings come through TBinaryProtocol.py as "bytes" and some as type == str. It seems like perhaps the evernote library is expecting them to all be bytes.

@xdutaotao
Copy link

@bordaigorl same problem happened on my ST3 when i want to save evernote notes after some editing, but one thing must be noticed, not all notes could trigger such error. It seems related to content of evernote note. maybe it's useful and just FYI. BTW, is there any debugging method to investigate it deeply?

Evernote plugin error: need more than 1 value to unpack
    Last command:  {}
    BEFORE SUBMITTING AN ISSUE (https://github.com/bordaigorl/sublime-evernote/issues):
      1. Enable the `debug` setting in your Evernote.sublime-settings file and try again. If the problem persists take a note of the output in the console.
         Make sure you delete personal information (e.g. Developer Token) from the output before posting it in an issue.
      2. Check the wiki at https://github.com/bordaigorl/sublime-evernote/wiki
      3. Search for similar issues at https://github.com/bordaigorl/sublime-evernote/issues?q=is%3Aissue
    (Evernote plugin v2.7.0, ST 3083, Python 3.3.3, linux x64)
error: Evernote error:
Evernote plugin error, please see the console for more details.
Then contact developer at
https://github.com/bordaigorl/sublime-evernote/issues

@xdutaotao
Copy link

@bordaigorl @traskrogers I noticed that there're 2 headers in such note with some similar fields link below, it can be saved after 2 headers combined to 1 header. so my issue disappeared after making some changes to notes' header. In my ST3, 2 headers will cause update/save issue cetertainly. :-)

---
title: 
layout: post
description:
category: Tools
notebook: 
tags: 
---


---
title: 
layout: post
description:
category: Tools
tags: 
---

@bordaigorl
Copy link
Owner

@xdutaotao Wow Thank you so so so much for tracking this one down!!!
I'll check and issue a bugfix release as soon as possible!
I cannot believe this might be yet another bug due to the markdown library...time to get rid of that...

@bordaigorl bordaigorl added the bug label Apr 11, 2016
@bordaigorl bordaigorl added this to the Next Release milestone Apr 11, 2016
@jastuccio
Copy link

Same issue for me on both my laptops. I'll try reinstalling the package after the next update.

Evernote plugin error: unpack requires a bytes object of length 1
Last command: {}
BEFORE SUBMITTING AN ISSUE...
(Evernote plugin v2.7.0, ST 3103, Python 3.3.3, osx x64)

If it is helpful in any way here is the output from the last two tests @bordaigorl suggested.

sublime.find_resources("Thrift.py")
['Packages/Evernote/lib/thrift/Thrift.py']

import Evernote.lib.thrift as t; t.file
'/Users/joe/Library/Application Support/Sublime Text 3/Installed Packages/Evernote.sublime-package/Evernote.lib.thrift'

@bordaigorl
Copy link
Owner

@jastuccio can I ask you if you are using multiple metadata headers or --- in the body similarly as @xdutaotao was mentioning?
I could reproduce the error using @xdutaotao's examples and I already fixed it in my local branch.
Before I release a new version I wanted to test it a little...

@jastuccio
Copy link

I don't think so. My new note header is just:

---
title: 
notebook: 
tags:
---

@bordaigorl
Copy link
Owner

@jastuccio ok I think I understand what is going on now.
There are two unrelated bugs, both with an error message mentioning "unpack":

  1. unpack requires a bytes object of length 1 (experienced by @traskrogers and @jastuccio )
  2. need more than 1 value to unpack (experienced by @xdutaotao )

Now, bug number 2 is a bug of the markdown library and can be easily fixed.

Bug number 1 however is buried deep in the Evernote SDK for Python 3 (so it would seem) and I do not know how to fix it because:

  • I do not have expertise about Thrift, nor py2 to py3 conversions
  • I cannot reproduce the error

I am very puzzled also because I am running the plugin on exactly the same version and platform as @jastuccio and I do not have any issues.
@jastuccio : do you experience the same problem if you connect with different networks (i.e. not behind a firewall or proxy or corporate netword)?

@jastuccio
Copy link

I am connecting from home. The OSX built in firewall was off at the time.

You mentioned markdown library. I have a different error in my console a few messages back. I cant remember what sublime-evernote action triggered it.

Error loading scope:text.html.markdown.gfm: Unable to find syntax file for scope text.html.markdown.gfm in Packages/Evernote/Evernote.tmLanguage
Evernote plugin error: unpack requires a bytes object of length 1
    Last command:  {}
    BEFORE SUBMITTING AN ISSUE ....

    (Evernote plugin v2.7.0, ST 3103, Python 3.3.3, osx x64)

@ghost
Copy link

ghost commented Nov 21, 2018

I had this error after trying to reinstall the plugin on a new OS - it turns out I was supplying the notestore as the developer token (since I had previously generated a token, Evernote didn't show the old token on the page). The solution was to revoke the old token and generate a new one. In case this helps anyone.

@bordaigorl
Copy link
Owner

@jastuccio @traskrogers coul you try an see if what @ktleary is suggesting applies to you?

@justiceamoh
Copy link

worked for me as well! thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants