lpvs fails without error

[avallen]

Hi Aaron,

many thanks for your great work, I look forward to making extensive use of it in the future, saving me some minutes per day! :-)

Now everything seems to work fine save the most important thing: lpvs <term> does just not respond, instead it just displays "Searching vault...". Nothing is output in alfred's debug mode (set to output all information).

The data that returned from lpass export does contain non-ASCII characters, so this might be related to issue #2, testing with the 3.2 pre-release that you offer in that issue does not fix the issue however (after re-executing lpdd).

I'd be glad to provide additional debugging help if you'd like to investigate this further.

Thanks,
Andreas

[bachya]

Good morning Andreas – sorry to hear you're having trouble.

It sounds like you are able to execute `lpass export from Terminal? If so, the problem definitely lies with my side.

Could you run lpsettings within Alfred, select the 4th option ("Edit Config File"), and paste those results here?

[avallen]

Thanks Aaron,

yes lpass export runs fine. Here is the output from "Edit Config File":

{
  "general": {
    "browser": 3, 
    "cache_bust": 300
  }, 
  "lastpass": {
    "path": "/usr/local/bin/lpass", 
    "username": "andreas.vallen@gmail.com"
  }, 
  "passwords": {
    "length": 14, 
    "number": 10
  }
}

[bachya]

Thank you. That all looks good.

Let's go a level deeper:

1. Open Alfred Preferences, right click on "LastPass Vault Manager", and select "Show in Finder".
2. Open Terminal and type python – do not hit ENTER yet.
3. From Finder, drag lpvm.py into Terminal; this should effectively leave something like python /some/path/to/lpvm.py.
4. At the end of that string, type search-vault "<QUERY>" – replace <QUERY> with a valid query.
   • Your Terminal should now show python /some/path/to/lpvm.py search-vault "<QUERY>".
5. Hit ENTER.

Let me know what gets output and we'll go from there.

[avallen]

Great instructions, thanks. The resulting output is the following:

andreas:~ $ python /Users/andreas/Dropbox/Applications/Alfred/Alfred.alfredpreferences/workflows/user.workflow.A33E58DD-49C4-46E2-8F58-846AAFA9009B/lpvm.py  search-vault yahoo
22:11:56 workflow.py:1598 DEBUG    Loading cached data from : /Users/andreas/Library/Caches/com.runningwithcrayons.Alfred-2/Workflow Data/com.bachya.lpvm/vault_items.cpickle
<?xml version="1.0" encoding="utf-8"?>
<items><item autocomplete="yahoo.com" uid="yahoo.com" valid="yes"><title>yahoo.com</title><subtitle>Click to launch; &#8984;-Click to copy password; Shift-Click to copy username.</subtitle><subtitle mod="cmd">&#8984;-Click to copy password</subtitle><subtitle mod="shift">Shift-Click to copy username</subtitle><arg>yahoo.com***https://login.yahoo.com/config/login_verify2?.intl=us&amp;.src=ygrp&amp;.done=http%3a//health.groups.yahoo.com%2Fgroup%2Fwombtwin%2Flinks%2F%3Fprop%3Deupdate</arg></item></items>22:11:56 workflow.py:2113 WARNING  Can't save last version: workflow has no version
22:11:56 workflow.py:1969 DEBUG    Workflow finished in 0.061 seconds.

Prettified:

andreas:~ $ python /Users/andreas/Dropbox/Applications/Alfred/Alfred.alfredpreferences/workflows/user.workflow.A33E58DD-49C4-46E2-8F58-846AAFA9009B/lpvm.py  search-vault yahoo
22:11:56 workflow.py:1598 DEBUG    Loading cached data from : /Users/andreas/Library/Caches/com.runningwithcrayons.Alfred-2/Workflow Data/com.bachya.lpvm/vault_items.cpickle

<?xml version="1.0" encoding="utf-8" ?>
<items>
    <item autocomplete="yahoo.com" uid="yahoo.com" valid="yes">
        <title>yahoo.com</title>
        <subtitle>Click to launch; &#8984;-Click to copy password; Shift-Click to copy username.</subtitle>
        <subtitle mod="cmd">&#8984;-Click to copy password</subtitle>
        <subtitle mod="shift">Shift-Click to copy username</subtitle>
        <arg>yahoo.com***https://login.yahoo.com/config/login_verify2?.intl=us&amp;.src=ygrp&amp;.done=http%3a//health.groups.yahoo.com%2Fgroup%2Fwombtwin%2Flinks%2F%3Fprop%3Deupdate</arg>
    </item>
</items>

22:11:56 workflow.py:2113 WARNING  Can't save last version: workflow has no version
22:11:56 workflow.py:1969 DEBUG    Workflow finished in 0.061 seconds.```

[bachya]

Well that is very odd, indeed – that's exactly the XML I would expect. In fact, if I hardcode it into my lpvs Script Filter, I get what you'd expect:

Let's try one other thing: inside the lpvs Script Filter, change:

/usr/bin/env python

...to:

python (so that the entire script reads python lpvm.py search-vault "{query}")

Any difference?

[avallen]

Yes, after this change and a restart it now shows the following error - this time searching for google which should exist too:

[avallen]

Here's some debug output that goes with it:

[ERROR: alfred.workflow.input.scriptfilter] Code 0: 00:13:40 workflow.py:1598 DEBUG    Loading cached data from : /Users/andreas/Library/Caches/com.runningwithcrayons.Alfred-2/Workflow Data/com.bachya.lpvm/vault_items.cpickle
00:13:40 workflow.py:2113 WARNING  Can't save last version: workflow has no version
00:13:40 workflow.py:1969 DEBUG    Workflow finished in 0.037 seconds.

[bachya]

Would you be willing to share your cash to data with me? You can get the path from that Alfred debug info. It only contains URLs and host names, so there shouldn't be anything damaging to you there. Feel free to email it to me at bachya1208@gmail.com.

[ghost]

I'm a bit of an amateur when it comes to Terminal, but I can't seem to get past the python stage. I just get:

-bash: python/Users/mattkneale/Library/Application Support/Alfred 2/Alfred.alfredpreferences/workflows/user.workflow.9E41EC43-4013-41AA-9C2C-A21A4B5B88D7/lpvm.py: No such file or directory

EDIT: Scrap that, no space after python before dragging. Whoops! Here's what I get now:

16:04:24 workflow.py:2113 WARNING Can't save last version: workflow has no version
16:04:24 workflow.py:1969 DEBUG Workflow finished in 0.673 seconds.

[bachya]

Thanks, Matt! Looks like I could use some better debugging info. :) Hang tight while I look into some things; I may have both you and Andreas try a pre-release build with more debugging.

[ghost]

No problem :) Thanks for your work. FWIW I love the look of the workflow, and it's still pretty cool for generating passwords quickly on the fly so still useful even if we can't get it resolved!

[bachya]

Appreciate that! We'll get this squashed, don't worry – I want you to experience the full benefits of this workflow. :)

Could you confirm whether your lpvs Script Filter starts with python instead of /usr/bin/env python (and whether that makes a difference)?

[ghost]

Let's hope so!

/usr/bin/env python, and can confirm it makes no difference if I remove the /usr/bin/env.

[bachya]

@avallen – I loaded the data you sent my into my cache and was able to query it just fine:

So, this tells me that the issue might be environmental. Still experimenting, but wanted to let you know.

[bachya]

@avallen and @MattKneale – I've updated the 3.2 pre-release with more debugging; would you mind:

1. installing the updated workflow (https://github.com/bachya/lp-vault-manager/releases/tag/v3.2).
2. Re-running lpdd and posting the debug output from the Run Script action connected to it.
3. Re-running lpvs <QUERY> and posting the debug output from the Script Filter itself.

Thanks!

[avallen]

I tried but did not receive any debug output from lpvs and only the following line when doing lpdd - which had been there before already.

[INFO: alfred.workflow.input.keyword] Processing output 'alfred.workflow.action.script' with arg ''

Could it be that the pre-release version that is offered under the above link still points to the previous version - it says:

bachya released this 2 days ago.

[ghost]

I'll try this later tonight/early tomorrow :) Just out at the moment.
Thanks for your efforts Aaron.

On 6 January 2015 at 20:12, Andreas Vallen notifications@github.com wrote:

I tried but did not receive any debug output from lpvs and only the
following line when doing lpdd - which had been there before already.

[INFO: alfred.workflow.input.keyword] Processing output 'alfred.workflow.action.script' with arg ''

Could it be that the pre-release version that is offered under the above
link still points to the previous version - it says:

bachya released this 2 days ago.

—
Reply to this email directly or view it on GitHub
#3 (comment)
.

[bachya]

@avallen It's the same release; I just added a different .alfredworkflow file.

That's interesting that you don't see any debug output from either; you should definitely be seeing XML and other results.

So, we've figured out that:

1. Running the script by itself gives the correct results.
2. No debug output or results appear when running the same script within Alfred.

Is there anything special about your environment? What version of OS X and Python are you using?

[ghost]

No idea RE: python (can't even remember installing it; how do I find the
version?)

I'm on the beta channel for Yosemite... probably should have mentioned that
sooner!

On Tue, 6 Jan 2015 22:06 Aaron Bach notifications@github.com wrote:

@avallen https://github.com/avallen It's the same release; I just added
a different .alfredworkflow file.

That's interesting that you don't see any debug output from either; you
should definitely be seeing XML and other results.

So, we've figured out that:

1. Running the script by itself gives the correct results.
2. No debug output or results appear when running the same script
   within Alfred.

Is there anything special about your environment? What version of OS X and
Python are you using?

—
Reply to this email directly or view it on GitHub
#3 (comment)
.

[bachya]

No idea RE: python (can't even remember installing it; how do I find the version?)

In Terminal, run: python --version

I'm on the beta channel for Yosemite... probably should have mentioned that sooner!

Interesting... @avallen, what about you? Are you on a Yosemite beta, as well? I'm running full-fledged Yosemite... I wonder if that's a problem.

[ghost]

Version is 2.7.6.

Yosemite is 10.10.2

On Tue, 6 Jan 2015 23:51 Aaron Bach notifications@github.com wrote:

No idea RE: python (can't even remember installing it; how do I find the
version?)

In Terminal, run: python --version

I'm on the beta channel for Yosemite... probably should have mentioned
that sooner!

Interesting... @avallen https://github.com/avallen, what about you? Are
you on a Yosemite beta, as well? I'm running full-fledged Yosemite... I
wonder if that's a problem.

—
Reply to this email directly or view it on GitHub
#3 (comment)
.

[avallen]

No, I'm not on the Yosemite beta channel, instead running the regular current Yosemite version (10.10.1).

Python version in Terminal (with PATH having brew's /usr/local/bin prepended) is:

andreas:~/Downloads $ which python
/usr/local/bin/python
andreas:~/Downloads $ python --version
Python 2.7.9
andreas:~/Downloads $ brew info python
python: stable 2.7.9 (bottled), HEAD
https://www.python.org
/usr/local/Cellar/python/2.7.6 (4663 files, 73M)
  Built from source
/usr/local/Cellar/python/2.7.9 (4860 files, 79M) *
  Poured from bottle
From: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/python.rb

Not sure about the python version in a launchd environment however, if alfred uses that, it might pick up the system python under /usr/bin instead.

[ghost]

Hi Aaron,

I've installed 3.2 with no success regretfully. Could you clarify how to access the debug outputs of the scripts? Still a novice to this!

[shkm]

@MattKneale This what you're after?

[ghost]

Thanks @shkm

@bachya lpdd output:

[INFO: alfred.workflow.input.keyword] Processing output 'alfred.workflow.action.script' with arg ''

Nothing comes up when running lpvs

[bachya]

Well guys, this is stumping me. Interestingly enough, I'm starting to notice it happen on my side, too: I'll type lpvs <QUERY> and nothing will show up. If I close and re-open my Alfred search, things work as expected. It's a stumper.

I'm currently working on a complete rewrite that makes it much cleaner to see where things are going on. Once it's ready, I'll report back here and would love to have you test. Assuming it continues, I'm going to exercise the nuclear option and ask the Alfred forum participants for help. :)

Stay tuned!

[ghost]

No problem, sorry it didn't work as planned! Look forward to seeing the
next version.

Best,

Matt
On 9 Jan 2015 18:44, "Aaron Bach" notifications@github.com wrote:

Well guys, this is stumping me. Interestingly enough, I'm starting to
notice it happen on my side, too: I'll type lpvs and nothing will
show up. If I close and re-open my Alfred search, things work as expected.
It's a stumper.

I'm currently working on a complete rewrite that makes it much cleaner to
see where things are going on. Once it's ready, I'll report back here and
would love to have you test. Assuming it continues, I'm going to exercise
the nuclear option and ask the Alfred forum participants for help. :)

Stay tuned!

—
Reply to this email directly or view it on GitHub
#3 (comment)
.

[giulios123]

@bachya that's a problem, if you do lpass --version it should answer 0.3.0... but try also brew info lastpass-cli it should give you 0.4.1. Knowing the fact that both can't be installed... I just think is a bug of the brew package.
Then, the request for the master password happens only in the terminal, so doing it in background (via python) is just hiding that request ( and spawning a ton of lpass process in my system).
The request for the masterpassword, for the export happens ALSO when I'm logged in.

[bdsivak]

For what it's worth, I have exactly the same behavior as @markerx's summary.

[bachya]

@bdsivak: That seems to indicate something going wrong with Alfred, then. :-\ Have you tried the new 2.6 version?

[markerx]

@bachya I am using 2.6 since its release yesterday. No help.

[giulios123]

@bachya I've found the solution 👍 (at least for the update/no result error)
The problem is that when the workflow tries to export from lpass (for the first time or update) the list of accounts, if one (or more) of them requires password to be seen, lpass, automatically, ask the user for the master password. This, happening in background, is not seen by the user, and keep going in background, spawing each lpdd a new process which never ends...

So, the only solution, is to enter the vault from the website and check if any of your websites repromt for master password.

Another solution is to run the download-data process not in background, but directly in the terminal, in order to let the user insert the password

Hope that is a good solution which helps lot of us...

ref: lastpass/lastpass-cli#70

[bachya]

@giulios123: WOW – great detective work! It makes sense that I would not be able to see this, as I don't have any sites for which I require a master password re-entry.

@avallen @MattKneale @shkm @AnxiousGummi @nicatronTg @markerx: What about you all? Do you have items in your vault that require/force a master password re-entry?

[markerx]

@bachya - I have over 1200 passwords... but I don't recall using the "Require Password Reprompt".

@giulios123 - Do you know of a way to filter/identify any passwords using "Require Password Reprompt"?

[giulios123]

@markerx the lock is the signal of master password reprompt.... you can easily showing them on top, by clicking on the top of the list (you see the blue triangle)

[avallen]

HI Aaron, yes I certainly have some!

Am 28.01.2015 um 23:40 schrieb Aaron Bach notifications@github.com:

@giulios123 https://github.com/giulios123: WOW – great detective work! It makes sense that I would not be able to see this, as I don't have any sites for which I require a master password re-entry.

@avallen https://github.com/avallen @MattKneale https://github.com/MattKneale @shkm https://github.com/shkm @AnxiousGummi https://github.com/AnxiousGummi @nicatronTg https://github.com/nicatronTg @markerx https://github.com/markerx: What about you all? Do you have items in your vault that require/force a master password re-entry?

—
Reply to this email directly or view it on GitHub #3 (comment).

[markerx]

@giulios123 What screen is that?

I don't see a padlock anywhere.

[markerx]

@giulios123 - nevermind. I found it. The Google LP Vault does not have the "Require Password Reprompt" column available. But FireFox does.

@bachya - I found 2 entries that required "Require Password Reprompt". I removed that option from both.

I did a:

• lpdd_exec.py download-data
• update.py
• lpvs_query.py search-vault-for-query 'google'

All of those work.

LPVS does not :-(

[bachya]

@markerx: What happens if you open a Terminal and run lpass export?

[markerx]

@bachya

What happens if you open a Terminal and run lpass export?

A thousand sites scroll up the terminal window.

[markerx]

@bachya - For what it is worth, lpbrowser works.

[bachya]

@markerx: Well that's scary: lpvs and lpbrowser are just different Script Filters into the same overall mechanism...

[ghost]

@bachya I definitely have a few like that, including form fills.

On 29 January 2015 at 00:52, markerx notifications@github.com wrote:

@giulios123 https://github.com/giulios123 - nevermind. I found it. The
Google LP Vault does not have the "Require Password Reprompt" column
available. But FireFox does.

@bachya https://github.com/bachya - I found 2 entries that required
"Require Password Reprompt". I removed that option from both.

I did a:

• lpdd_exec.py download-data
• update.py
• lpvs_query.py search-vault-for-query 'google'

All of those work.

LPVS does not :-(

—
Reply to this email directly or view it on GitHub
#3 (comment)
.

[giulios123]

Ok, I've tried some more stuff...
Please for whom have problems do follow this procedure.

1. delete the workflow
2. open terminal
3. lpass logout (y and enter)
   * open system monitor and kill all the lpass processes
4. login on lastpass web vault and unlock the sites that require password reprompt
5. download again the workflow and install it
6. configure it and try

Let me know if this works... ( on mine it worked perfectly).

Or, if you want to keep the reprompt active you have to run *python path_to_workflow/lpdd_exec.py download-data each time you want to update the vault.

@bachya one question, can you switch the process of lpass export from subprocess to a terminal window(it should workout fully with whom has the pass reprompt)?

[markerx]

Arg... so close...

Here's where I am at:

• Deleted the workflow from Alfred.
• Opened terminal & lpass logout (y and enter)
• Opened system monitor and killed 4 instances of lpass
• Logged into lastpass web vault and confirmed there are no sites requiring password reprompt
• Downloaded workflow Version 4.2
• Installed Workflow
• Logged in via lpsettings lastpass-login & then terminal
• Downloaded data via LPDD, which worked for the first time I have seen:

Starting debug for 'LastPass Vault Manager'
[ERROR: alfred.workflow.action.script] 08:27:35 workflow.py:1386 DEBUG Reading settings from /Users/Brandon/Library/Application Support/Alfred 2/Workflow Data/com.bachya.lpvm/settings.json ...
08:27:35 workflow.py:1960 DEBUG Workflow version : 4.2.0
08:27:35 lpdd_exec.py:22 DEBUG Exec arguments: [u'download-data']
08:27:35 lpdd_exec.py:34 DEBUG Parsed command: download-data
08:27:35 lpdd_exec.py:35 DEBUG Parsed argument: None
08:27:35 lpdd_exec.py:36 DEBUG Parsed delimiter: >
08:27:35 lpdd_exec.py:43 DEBUG Executing command: download-data
08:27:42 utilities.py:66 DEBUG 1270 LastPass vault items cached.
08:27:42 workflow.py:1645 DEBUG Cached data saved at : /Users/Brandon/Library/Caches/com.runningwithcrayons.Alfred-2/Workflow Data/com.bachya.lpvm/vault_items.cpickle
08:27:42 workflow.py:1996 DEBUG Workflow finished in 6.769 seconds.
[INFO: alfred.workflow.action.script] Processing output 'alfred.workflow.output.notification' with arg 'LastPass metadata successfully downloaded.'

• Attempted LPVS

Nothing output to Alfred Debug window.

Fail :-(

lpbrowser still works

[giulios123]

@markerx try in terminal python path_to_workflow/lpdd_exec.py download-data

N: path_to_workflow is where lpdd_exec is

[markerx]

/usr/bin/python /Users/Brandon/Library/Application\ Support/Alfred\ 2/Alfred.alfredpreferences/workflows/user.workflow.3791B45A-EA77-477F-9386-5C7A5A56F5E7/lpdd_exec.py download-data

08:46:31 workflow.py:1386 DEBUG Reading settings from /Users/Brandon/Library/Application Support/Alfred 2/Workflow Data/com.bachya.lpvm/settings.json ...
08:46:31 workflow.py:1960 DEBUG Workflow version : 4.2.0
08:46:31 lpdd_exec.py:22 DEBUG Exec arguments: [u'download-data']
08:46:31 lpdd_exec.py:34 DEBUG Parsed command: download-data
08:46:31 lpdd_exec.py:35 DEBUG Parsed argument: None
08:46:31 lpdd_exec.py:36 DEBUG Parsed delimiter: >
08:46:31 lpdd_exec.py:43 DEBUG Executing command: download-data
08:46:33 utilities.py:66 DEBUG 1270 LastPass vault items cached.
08:46:33 workflow.py:1645 DEBUG Cached data saved at : /Users/Brandon/Library/Caches/com.runningwithcrayons.Alfred-2/Workflow Data/com.bachya.lpvm/vault_items.cpickle
LastPass metadata successfully downloaded.
08:46:33 workflow.py:1996 DEBUG Workflow finished in 2.186 seconds.

/usr/bin/python /Users/Brandon/Library/Application\ Support/Alfred\ 2/Alfred.alfredpreferences/workflows/user.workflow.3791B45A-EA77-477F-9386-5C7A5A56F5E7/lpvs_query.py search-vault-for-query 'github'

08:45:17 workflow.py:1386 DEBUG Reading settings from /Users/Brandon/Library/Application Support/Alfred 2/Workflow Data/com.bachya.lpvm/settings.json ...
08:45:18 workflow.py:1960 DEBUG Workflow version : 4.2.0
08:45:18 lpvs_query.py:30 DEBUG Query arguments: [u'search-vault-for-query', u'github']
08:45:18 lpvs_query.py:38 DEBUG Parsed command: search-vault-for-query
08:45:18 lpvs_query.py:39 DEBUG Parsed argument: github
08:45:18 lpvs_query.py:40 DEBUG Parsed delimiter: >
08:45:18 lpvs_query.py:41 DEBUG Parsed query: search-vault-for-query github
08:45:18 lpvs_query.py:53 DEBUG Executing command: search-vault-for-query
08:45:18 workflow.py:1609 DEBUG Loading cached data from : /Users/Brandon/Library/Caches/com.runningwithcrayons.Alfred-2/Workflow Data/com.bachya.lpvm/vault_items.cpickle
08:45:18 utilities.py:221 DEBUG Search results: [{'url': u'https://github.com/login?return_to=%2Fnatew%2Fobtvse', 'hostname': u'github.com'}]

<title>github.com</title>TAB to explore; ENTER to copy password; ⌘-Click to copy username; Shift-Click to open URL⌘-Click to copy username.Shift-Click to open the URL.get-password github.com***https://github.com/login?return_to=%2Fnatew%2Fobtvse08:45:18 workflow.py:1996 DEBUG Workflow finished in 0.049 seconds.

[giulios123]

so it's working?

because it seems so

eventually... log out again from terminal and kill all the lpass processes

[markerx]

@giulios123

It works from a terminal stand point, but there seems to be a disconnect in the Alfred LPVS function.

[bachya]

@giulios123 From your feedback, I've seen something interesting.

I used to use lpass ls as a way to check whether the user was logged in or not; if they weren't, lpass would return an error immediately and I would catch it. Now, as you note, it goes straight to the master login prompt – that's not good. :)

At any rate, it appears that that's where the hang is. From what you've reported, I think I need a smarter way to check whether a user is already logged in or not.

Researching...

[giulios123]

I would suggest lpass sync, which syncronize the local cache with the server one... but usually no change happens (due to the fact that we can't save from the workflow), so it just give no output if logged in and error if not logged :)

[bachya]

@giulios123 The only problem with that (or any lpass subcommand): if the agent somehow gets killed, but the user isn't logged out, it's still going to prompt you for your master password (or hang if launched via Alfred).

I think I've landed on checking for the existence of ~/.lpass/agent.sock – in all my testing, when that exists, the user is logged in.

Here's the next problem: if I detect that the user isn't logged in, what would be the preferred action? If I notify the user (via Alfred) that they aren't logged in, that's great, but then what?

[giulios123]

I would rather automatically open the terminal for the insertion of the master pass ( and second factor if required). What do you think?

[warthurton]

It would be, I was going to play around with unix "expect" this weekend to
see if I could detect a request for re-auth when running a pass command.
Then have it spawn up a terminal session running the command again and
allowing you to enter in the additional credentials.

Just got busy this week after spending too much time researching earlier.

Wayne A Arthurton http://warthurton.com

On Thu, Jan 29, 2015 at 5:25 PM, Giulio notifications@github.com wrote:

I would rather automatically open the terminal for the insertion of the
master pass ( and second factor if required). What do you think?

—
Reply to this email directly or view it on GitHub
#3 (comment)
.