Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Events are not firing again for several people #142

Closed
cymplecy opened this issue Aug 30, 2023 · 129 comments
Closed

Events are not firing again for several people #142

cymplecy opened this issue Aug 30, 2023 · 129 comments

Comments

@cymplecy
Copy link

https://discourse.nodered.org/t/alexa-on-device-activity-not-working/80901/1

@cymplecy cymplecy changed the title Events are not firing again for me and at least one other person Events are not firing again for several people Aug 30, 2023
@Apollon77
Copy link

Hm … I‘m not at home till the weekend. Son-in-law fact debug logs and check what comes in via wsmqtt would be the only helping to get an idea. But it might be that Amazon now also closes this way :-(

@claudiopj
Copy link

i have same problem!

@mactwin
Copy link

mactwin commented Aug 31, 2023

same here, all other nodes are working....events no, it's grayed out and not receiving anything.

@DanPatten
Copy link

@Apollon77 Looks like events are no longer occurring via wsmqtt, I connected directly to that WS and not seeing anything. Is there an alternative here? It does appear the API supports returning recent events although probably not a great solution for real time as compared to a websocket connection.

@Apollon77
Copy link

What api you talk about?

@sreevo
Copy link

sreevo commented Sep 2, 2023

Apollon, pls help us :( we need you

@Apollon77
Copy link

Apollon77 commented Sep 2, 2023

If someone is firm with sniffing ios/android network traffic including tls decoding using wireshark or such ... contact me. So you know what we do currently ... this is not that funny doing as it sounds reading.

lets see if we find a way

@DanPatten
Copy link

@Apollon77
Copy link

Ahh you mean that api. Yes this is not the topic here because we need to know when to get these data

@Chris380
Copy link

Chris380 commented Sep 8, 2023

Same for me. Other nodes work, "Alexa Event" says "listening" but does not emit anything.
Thanks for everybody involved in fixing this issue.

@questuk
Copy link

questuk commented Sep 8, 2023

Same for me. Other nodes work, "Alexa Event" says "listening" but does not emit anything.
Thanks for everybody involved in fixing this issue.

Yes same here for me as well, hope it gets fixed as it was very very useful

@Apollon77
Copy link

I'm in communication with @bbindreiter ... stay tuned ... and prepare your system: It will require Node.js 16+ !

If it works and you like my work ... I'm happy for any Donations: https://github.com/sponsors/Apollon77 / https://paypal.me/Apollon77

@bbindreiter
Copy link
Owner

bbindreiter commented Sep 8, 2023

There is a new version available (5.0.40), please update and make sure Events is "On" in Account settings. Everything should work then again.

Please donate to Apollon77 using the link above if you rely on this functionality.

@claudiopj
Copy link

still not working for me... :(

@Chris380
Copy link

Chris380 commented Sep 8, 2023

@Apollon77 I can confirm that the "Alexa Event" is working now again (layla.amazon.de is my service host). Great work!
As @bbindreiter mentioned it's crucial to set Events to "On".
P.S. Ein kleiner Dank ist unterwegs

@cymplecy
Copy link
Author

cymplecy commented Sep 9, 2023

For some reason, V5.0.40 wasn't showing as being available in the palette but installing from command worked and back in the listening business once I'd configured Events = On

@Apollon77 (and @bbindreiter ) for your efforts

Also to everyone who relies on this

#142 (comment)

@Bismar
Copy link

Bismar commented Sep 9, 2023

Unsure what's happening with this new update, but it's causing my nodered to continuously crash.

9 Sep 19:59:55 - [info] [server:Home Assistant] Connecting to http://supervisor/core
9 Sep 19:59:55 - [warn] [alexa-remote-account:acd14ce107b43b33] ENOENT: no such file or directory, open '/home/authFile'
9 Sep 19:59:55 - [info] [alexa-remote-account:acd14ce107b43b33] intialising with the PROXY method and NO saved data...
9 Sep 19:59:55 - [warn] [alexa-remote-account:acd14ce107b43b33] open 192.168.1.133:3456 in your browser
9 Sep 19:59:55 - [info] [server:Home Assistant] Connected to http://supervisor/core
9 Sep 19:59:55 - [error] [ha-sensor:Projector Control] InputError: Integration not loaded
9 Sep 19:59:55 - [error] [alexa-remote-routine:3b1dad892822b537] Account not initialised!
(node:307) TimeoutOverflowWarning: 2592000000 does not fit into a 32-bit signed integer.
Timeout duration was set to 1.
(Use node --trace-warnings ... to show where the warning was created)
9 Sep 20:00:23 - [info] [alexa-remote-account:acd14ce107b43b33] auto refreshing cookie...
9 Sep 20:00:23 - [info] [alexa-remote-account:acd14ce107b43b33] intialising with the PROXY method and saved data...
(node:307) TimeoutOverflowWarning: 2592000000 does not fit into a 32-bit signed integer.
Timeout duration was set to 1.
(node:307) TimeoutOverflowWarning: 2592000000 does not fit into a 32-bit signed integer.
Timeout duration was set to 1.
9 Sep 20:00:27 - [info] [alexa-remote-account:acd14ce107b43b33] auto refreshing cookie...
9 Sep 20:00:27 - [info] [alexa-remote-account:acd14ce107b43b33] intialising with the PROXY method and saved data...
(node:307) TimeoutOverflowWarning: 2592000000 does not fit into a 32-bit signed integer.
Timeout duration was set to 1.
(node:307) TimeoutOverflowWarning: 2592000000 does not fit into a 32-bit signed integer.
Timeout duration was set to 1.
9 Sep 20:00:31 - [info] [alexa-remote-account:acd14ce107b43b33] auto refreshing cookie...
9 Sep 20:00:31 - [info] [alexa-remote-account:acd14ce107b43b33] intialising with the PROXY method and saved data...
9 Sep 20:00:31 - [red] Uncaught Exception:
9 Sep 20:00:31 - [error] Error: getaddrinfo ENOTFOUND bob-dispatch-prod-fe.amazon.com.au
at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)
[20:00:31] INFO: Service Node-RED exited with code 1 (by signal 0)
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service nginx: stopping

@Apollon77
Copy link

Apollon77 commented Sep 9, 2023

Hm. One thing is this error about timeout value. Please use the mentioned

node --trace-warnings parameter for modeled start to trace the codeplace and report here. Something tries to set a timeout if 720h or 30d which is too long. Find it because it gives a lot of issue! Maybe also check config if the node?

Other then that the getaddr error means that there is a dns lookup issue.

And btw: show your config of the Alexa node in any case. Wh it tries for you to connect to far east server? Japanese? Australia account?

@elgerg
Copy link

elgerg commented Sep 9, 2023

Hi all,

Thanks for the work you have all done on this to get events working again.

I updated my installation, set events to on, restarted node-red and the events reset itself to blank so had to set it to on again. After another restart the setting was retained.

However after a couple of minutes the events stopped firing again.

I'm using NodeJS 18 if that is of any help.

Does anyone know how I can get the events to fire all of the time like they used to or what I can do to help debug the issue?

Thanks in advance!

@Apollon77
Copy link

Apollon77 commented Sep 9, 2023

@Bismar locate that file and remove the .au at the end of the Hostname.

https://github.com/Apollon77/alexa-remote/blob/c29aec3fba6acd2404d16b9bc5878d2e74283123/alexa-http2push.js#L41

Does it work then ?

@Apollon77
Copy link

@elgerg show your node config. Especially the domains names in there.

@elgerg
Copy link

elgerg commented Sep 9, 2023

@elgerg show your node config. Especially the domains names in there.

Here you go:
image

@Apollon77
Copy link

Should work. Other users reported it working. Try a restart. Else need debug logs

@elgerg
Copy link

elgerg commented Sep 9, 2023

Ok, so it worked for a few minutes then stopped again.

How do I get the debug logs? Sorry for the noob question but I've not had to do this before..

Thanks

@spants
Copy link

spants commented Sep 9, 2023

also in the uk but debugging on a flight.....
Events do not seem to be working for me

@Apollon77
Copy link

In fact I mainly assuming is not so much about that domain but where your Amazon account is located. So even if you use .co.uk but have an us account then we simply assume wrong.

@Apollon77
Copy link

@bbindreiter i think I add a new config variable to set/override that settings. Most likely easier on the long run.

@sccsltd
Copy link

sccsltd commented Oct 1, 2023

@cymplecy is your simon says just sending text to speech to the device?

to clarify on mine all my voice prompts triggered by HA all continue to work. (doors opening/closing, doorbell ...) all fire in HA and alexa speaks the commands fine, looks like i can also see the volume changes on the drvices ok as well.

the only thing that fails for me is when i initiate the speech commands, these dont get picked up.

@cymplecy
Copy link
Author

cymplecy commented Oct 1, 2023

"to clarify on mine all my voice prompts triggered by HA all continue to work. (doors opening/closing, doorbell ...) all fire in HA and alexa speaks the commands fine, looks like i can also see the volume changes on the drvices ok as well."

Yes - its just the listening for spoken events that is problematic - all else works

" is your simon says just sending text to speech to the device?" - I've got another Pi connected to a speaker that's next to an Echo and that runs my "simon says" counting up

When everthing working - it can count up to 40 (4 per minute) and all are picked up by the NR On Device Activity node

When it goes into failure mode its approximately one minute of working - one minute of not working

So, at this present time, I'm saying that if I restart Node-RED and it only initialises once, then it seems to work reliably.

But if re-initialises (either by direct inject into the node or due to it being redeployed - then it goes into the intermittant mode

So, now I've got all the Alexa nodes on it's own Pi with only one flow tab and I've locked it so I don't accidently change it.

I'll see what happens - I'll run my test routine every now and again just to make sure it works (just done another 10 minute run - all fine)

@sccsltd
Copy link

sccsltd commented Oct 1, 2023

That's what I thought just wanted to make sure I was explaining my self tidy.

I have restarted my node red and am no longer getting the http2-push close errors as yet, will see how long it takes to break.

@GianZambo
Copy link

@cymplecy

It could be a link between this two issues?

node-red/node-red#4370

@cymplecy
Copy link
Author

cymplecy commented Oct 1, 2023

I wouldn't know but suspect not as that's a crash report as opposed to the intermittent failures that we are seeing

@sccsltd
Copy link

sccsltd commented Oct 1, 2023

Still working ok here, restarted node-red and haven't touched it since, just periodically checking voice responses are picked up.
ill give it another 15 then if its still running ill redeploy and see if it breaks

also i do have a flow to init and auth that runs once an hour,

[{"id":"443b92d6.132dfc","type":"inject","z":"e8bd56f8f70ccb15","g":"b6c11feb.c1f0a","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"3600","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"Started!","payloadType":"str","x":1700,"y":140,"wires":[["6313cbe4.e7fc64"]]},{"id":"6313cbe4.e7fc64","type":"change","z":"e8bd56f8f70ccb15","g":"b6c11feb.c1f0a","name":"set","rules":[{"t":"set","p":"payload","pt":"msg","to":"alexa","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":1870,"y":140,"wires":[["6c43f2b4.15104c"]]},{"id":"6c43f2b4.15104c","type":"alexa-remote-init","z":"e8bd56f8f70ccb15","g":"b6c11feb.c1f0a","name":"","account":"20c7ba7a0decab30","option":"initialise","x":2060,"y":140,"wires":[["9439aeae7a79b3fd"]]},{"id":"20c7ba7a0decab30","type":"alexa-remote-account","name":"Main","authMethod":"proxy","proxyOwnIp":"192.168.0.156","proxyPort":"34567","cookieFile":"Authnew","refreshInterval":"1","alexaServiceHost":"alexa.amazon.co.uk","pushDispatchHost":"","amazonPage":"amazon.co.uk","acceptLanguage":"en-UK","onKeywordInLanguage":"","userAgent":"","usePushConnection":"on","autoInit":"off"}]

i found it somewhere as a solution to having to reauth manually a while ago, don't know of this is the best way or the right way but its been working fine so don't think its impacting but just incase.

just had a failure..... but then it started working again..

1 Oct 13:36:52 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Close: undefined: undefined
1 Oct 13:36:52 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Retry Connection in 20s
1 Oct 13:37:06 - [debug] [alexa-remote-account:Main] Alexa-Remote: No authentication check needed (time elapsed 567062)
1 Oct 13:37:12 - [debug] [alexa-remote-account:Main] Alexa-Remote: Update access token ...
1 Oct 13:37:12 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Use host bob-dispatch-prod-eu.amazon.com
1 Oct 13:37:13 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Initialization completed
1 Oct 13:38:38 - [debug] [alexa-remote-account:Main] Alexa-Remote: Update access token ...
1 Oct 13:38:38 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Use host bob-dispatch-prod-eu.amazon.com
1 Oct 13:38:38 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Initialization completed
1 Oct 13:41:33 - [debug] [alexa-remote-account:Main] Alexa-Remote: Update access token ...
1 Oct 13:41:33 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Use host bob-dispatch-prod-eu.amazon.com
1 Oct 13:41:33 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Initialization completed

(edit again)
I think we are getting closer.. had an undefined close immediately after i redeployed, which stopped it working, then it updated the token and we are back..

1 Oct 13:51:10 - [debug] [server:Hassio] HA State: running
1 Oct 13:51:10 - [debug] [server:Home Assistant] HA State: running
1 Oct 13:51:21 - [debug] Flushing localfilesystem context scope e8bd56f8f70ccb15
1 Oct 13:51:21 - [debug] Flushing localfilesystem context scope global
1 Oct 13:51:22 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Close: undefined: undefined
1 Oct 13:51:22 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Retry Connection in 1s
1 Oct 13:51:23 - [debug] [alexa-remote-account:Main] Alexa-Remote: Update access token ...
1 Oct 13:51:23 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Use host bob-dispatch-prod-eu.amazon.com
1 Oct 13:51:23 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Initialization completed
1 Oct 13:52:24 - [debug] [alexa-remote-account:Main] Alexa-Remote: Update access token ...
1 Oct 13:52:24 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Use host bob-dispatch-prod-eu.amazon.com
1 Oct 13:52:24 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Initialization completed```




@GianZambo
Copy link

I read in the previous post ..

1 Oct 13:51:22 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Close: undefined: undefined

and @knolleary write "we don't support HTTP2 "

Excuse me, but I don't understand .....

@spants
Copy link

spants commented Oct 1, 2023

Nick is saying its not a node red issue, that's all.

@DanPatten
Copy link

I think the problem may be not shutting down previously open Alexa-remote2 http2 connections which is why it works initially until you deploy the node again.

@sccsltd
Copy link

sccsltd commented Oct 1, 2023

ok so its been over an hour now and i still have functionality, im still seeing the undefined close but less often and it seems to be self recovering from them.

1 Oct 15:20:10 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Close: undefined: undefined
1 Oct 15:20:10 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Retry Connection in 60s
1 Oct 15:20:52 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Close: undefined: undefined
1 Oct 15:20:52 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Retry Connection in 60s
1 Oct 15:21:10 - [debug] [alexa-remote-account:Main] Alexa-Remote: Update access token ...
1 Oct 15:21:10 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Use host bob-dispatch-prod-eu.amazon.com
1 Oct 15:21:11 - [debug] [alexa-remote-account:Main] Alexa-Remote HTTP2-PUSH: Initialization completed

It looks as if retry isnt working correctly but the update token is fixing it?
i have added some extra logging where i think its getting stuck will see what the logs say

@sccsltd
Copy link

sccsltd commented Oct 1, 2023

still working fine with the last changed made by @DanPatten,

i removed my auto init/authenticate flow as that was triggering the initialize on each boot / redeploy and every hour, dont know if it had any effect but i see no ill effect in the changes my side.,

@cymplecy
Copy link
Author

cymplecy commented Oct 1, 2023

Updated to v5.0.43 - no change as far as I can tell

If the Init node is re-initilised, then within two minutes, the events stop being received (and then cycle on/off every few minutes)

Now - if we stop re-initialising, then we won't invoke the poor behaviour and we'd be alright

unfortunately, exactly the same behaviour is triggered if you do a Full deploy :(

That can be avoided as well but a lot of NR users will leave their installation in default Full deployment setting and will get hit by the issue

This shows my test logs showing the missing gaps

21:04:34.176_alexa_PI1881_activity = simon says one
21:04:46.763_alexa_PI1881_activity = simon says two
21:05:02.239_alexa_PI1881_activity = simon says three
21:05:16.887_alexa_PI1881_activity = simon says four
21:05:31.910_alexa_PI1881_activity = simon says five
21:05:47.039_alexa_PI1881_activity = simon says six
21:06:01.938_alexa_PI1881_activity = simon says seven
21:06:16.953_alexa_PI1881_activity = simon says eight
21:06:32.043_alexa_PI1881_activity = simon says nine
21:06:46.762_alexa_PI1881_activity = simon says ten
21:07:02.040_alexa_PI1881_activity = simon says eleven
missed 12,13,14 &15
21:08:17.079_alexa_PI1881_activity = simon says sixteen
21:08:32.745_alexa_PI1881_activity = simon says seventeen
21:08:47.028_alexa_PI1881_activity = simon says eighteen
21:09:02.173_alexa_PI1881_activity = simon says nineteen
21:09:17.292_alexa_PI1881_activity = simon says twenty
then missed 21,22,23 and 24
21:10:32.667_alexa_PI1881_activity = simon says twenty five
21:10:47.449_alexa_PI1881_activity = simon says twenty six
21:11:02.603_alexa_PI1881_activity = simon says twenty seven

So, AFAICT to avoid the issue, you have to:

  1. make sure you set the initialise node to auto init and never re-initialise it during a session
  2. Set deployment mode to modified nodes only and not edit the initialise node during a session

@DanPatten
Copy link

@cymplecy Those two steps is exactly how I have mine setup so it makes sense why mine works. Basically need to shutdown any existing connections when calling init and that does not appear to be happening. The intermittently comes from having 2 open connections to Amazon open which causes events to not get received because the event is fired by the wrong listener.

@obaldius
Copy link

obaldius commented Oct 4, 2023

I did manually initialize and I'm getting this:

image

Can anybody help please?

@cymplecy
Copy link
Author

cymplecy commented Oct 4, 2023 via email

@tomy-d
Copy link

tomy-d commented Oct 13, 2023

@obaldius: try to turn "Events" to "on" in the configuration note > "off" causes the error message....

@cymplecy: 2 questions:

if the problem occurs - how can I "reset" to working mode - any idea/hint?
Auto init means that I only turn the aut init "on in the configuration node right?!

Thank you!

@cymplecy
Copy link
Author

cymplecy commented Oct 13, 2023

@tomy-d
The answer used to be to restart Node-RED. (And yes to the auto-init question)
But a new problem has arisen yesterday (12Oct2023) that cannot be cured by simply doing this
#152

@obaldius
Copy link

@tomy-d thanks I did that and it worked or a couple of days but as @cymplecy pointed out, there's something else going on sine yesterday... I guess we'll have to wait.

@Apollon77
Copy link

Amazon stopped sending "Activity" push notifications ... but also the alexa app is not getting them ... sooo lets see if it is an error or wanted

@brunialti
Copy link

+1

@Apollon77
Copy link

Ok guys, @bbindreiter released 5.0.45 of the node with a new version of the library: It should bring back the activity event like before ... I added some workarounds that try to detect activities also without the proper events.

Please make sure to DISABLE all manual activity/history triggering you added as woraround yourself

I hope it works.

Ingo

PS: Donations always welcome ;-))

@tomy-d
Copy link

tomy-d commented Oct 27, 2023

@Apollon77
It works again! Thank you man! Good Job :)

@stephennutt
Copy link

I updated the pallete, restarted Node Red, deleted old Listening Node and added a new one, and even rebooted the RPi hosting Node Red and nothing shows in the Debug Node attached to Listening Node. I never did any of the workarounds; just patiently waited for the fix. Events is On in the Config node. Any other suggestions to try?

@cymplecy
Copy link
Author

Was your palette showing 5.0.45 - mine wasn't at first
Check installed version number

@PaytonPeterson
Copy link

Ok guys, @bbindreiter released 5.0.45 of the node with a new version of the library: It should bring back the activity event like before ... I added some workarounds that try to detect activities also without the proper events.

Please make sure to DISABLE all manual activity/history triggering you added as woraround yourself

This fixed things for me! Just updated to 5.0.45 in NodeRed - rebooted, and all my events started working again! Wow, big thank you @Apollon77 + @bbindreiter!

@Apollon77
Copy link

I released an update to my lib with improvements and optimizations. That should fix cases where it stayed empty.

@bbindreiter
Copy link
Owner

I released an update to my lib with improvements and optimizations. That should fix cases where it stayed empty.

Nice, 5.0.46 released.

@bbindreiter
Copy link
Owner

I will close this issue now, let's continue here: #152

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests