Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mercure Hub not automatically pushing updates #105

Closed
football2801 opened this issue Jun 19, 2019 · 24 comments · Fixed by #167

Comments

@football2801
Copy link

@football2801 football2801 commented Jun 19, 2019

I have a quick test Symfony application running to try to integrate the Mercure protocol.

I have a test Mercure hub running with the following:

JWT_KEY='aVerySecretKey' ADDR='localhost:3000' ALLOW_ANONYMOUS=1 CORS_ALLOWED_ORIGINS=* ./mercure

I have a form that submits the data to the following endpoint:

     /**
     * @Route("/resources", name="resourcesPost", methods={"POST"})
     */
    public function resourcesPost(Publisher $publisher, Request $request){

        $user = $this->getUser();

        $update = new Update(
            'http://example.com/books/1',
            json_encode([
                'from' => $user->getFullName(),
                'sent at' => (new DateTime())->format('H:i:s'),
                'message' => $request->request->get('message'),
            ])
        );

        $publisher($update);

        return $this->redirectToRoute('resources');
    }

Once I hit the form I get the following in the debugger:

127.0.0.1 - - [19/Jun/2019:13:27:40 -0400] "GET /hub?topic=http%3A%2F%2Fexample.com%2Fbooks%2F1&topic=http%3A%2F%2Fexample.com%2Fbooks%2F2&topic=http%3A%2F%2Fexample.com%2Freviews%2F%7Bid%7D HTTP/1.1" 200 31 "http://localhost:8888/resources" "Mozilla/5.0 (Windows N
T 10.0; Win64; x64; rv:67.0) Gecko/20100101 Firefox/67.0"
time="2019-06-19T13:32:50-04:00" level=info msg="Subscriber disconnected" remote_addr="127.0.0.1:65391"
time="2019-06-19T13:32:53-04:00" level=info msg="Update published" event_id=cd3acff1-7e9b-4e28-b6ba-e79aeb4c1791 remote_addr="127.0.0.1:49546"
127.0.0.1 - - [19/Jun/2019:13:32:53 -0400] "POST /hub HTTP/1.0" 200 36 "" ""
time="2019-06-19T13:32:53-04:00" level=info msg="Event sent" event_id=cd3acff1-7e9b-4e28-b6ba-e79aeb4c1791 remote_addr="127.0.0.1:65384"

However, the javascript on the second browser that is listening, does not put the response in the console log :

  const u = new URL('http://localhost:3000/hub');
  u.searchParams.append('topic', 'http://example.com/books/1');
  // Subscribe to updates of several Book resources
  u.searchParams.append('topic', 'http://example.com/books/2');
  // All Review resources will match this pattern
  u.searchParams.append('topic', 'http://example.com/reviews/{id}');
   
  const es = new EventSource(u);
  let lastEventId = null;
  es.onmessage = e => {
       let data = JSON.parse(e.data);
       lastEventId = e.lastEventId;
       console.log(data);
  }

If I go into the back into the terminal and run ctrl+c (to kill the Mercure server) the data will then get pushed into the javascript. My question is how do I get it to automatically push the data to the javascript without having to kill the Mercure server (running on localhost:3000) and then restart it?

@dunglas

This comment has been minimized.

Copy link
Owner

@dunglas dunglas commented Jun 19, 2019

It's super weird. What OS do you use?

@football2801

This comment has been minimized.

Copy link
Author

@football2801 football2801 commented Jun 19, 2019

Windows 10 running the client in a Git Bash environment

@football2801

This comment has been minimized.

Copy link
Author

@football2801 football2801 commented Jun 20, 2019

I just ran some more tests in my windows 10 environment.
If I build a batch file that was recommended in issue #76 and run it in Command Prompt, I get the exact same functionality as I get from Git Bash. Same with Windows PowerShell. I have to close the server connection for the clients to receive the data.

Edit: I've also now run the docker instance and was unable to get the updates pushed into the browser

@dunglas

This comment has been minimized.

Copy link
Owner

@dunglas dunglas commented Jun 20, 2019

I tested on Mac on Linux and isn't able to reproduce. Unfortunately I don't have a Windows computer available right now. I'll try to get one and to test next week.

@txptr

This comment has been minimized.

Copy link
Contributor

@txptr txptr commented Jun 27, 2019

Same problem here. Windows 10 with PowerShell running the mercure Server.

In Chrome Dev Tools the connection to "hub?topic=..." is marked as pending and I have no "EventStream" tab under the connection details. After shutting down the mercure server, the "EventStream" tab is visible and the script tries to reconnect to the hub.

@txptr

This comment has been minimized.

Copy link
Contributor

@txptr txptr commented Jun 27, 2019

I found the solution for me and maybe you:

I've set the wrong value for the env variable --> ADDR="127.0.0.1:3000"
After setting it to ADDR=":3000" everything workes fine and windows asked me for giving "mercure.exe" firewall access. Hope this helps @football2801

@dunglas

This comment has been minimized.

Copy link
Owner

@dunglas dunglas commented Jun 27, 2019

Wow! Windows :) Do you think you can add this tip to a new "Windows" section in the troubleshooting guide? I'm sure it will be very valuable for most Windows users!

@football2801

This comment has been minimized.

Copy link
Author

@football2801 football2801 commented Jun 27, 2019

@txptr I owe you a drink! This has me so much closer. I'm still having some issues, but chrome is accepting the response from the hub instantly.
Unfortunately firefox, IE, and Edge are both ignoring the event's from the hub still.

@football2801

This comment has been minimized.

Copy link
Author

@football2801 football2801 commented Jun 27, 2019

Can somebody give detailed instructions on how to deploy this to a CENTOS7 linux server? I'll give up on the local development environment and push to a linode, but I am completely unsure how to do so.

@mantis

This comment has been minimized.

Copy link
Contributor

@mantis mantis commented Jul 6, 2019

@dunglas i'm also getting same behaviour as @football2801. Chrome works, Firefox only updates when Ctrl+C the mercure service

@falconsmilie

This comment has been minimized.

Copy link

@falconsmilie falconsmilie commented Jul 28, 2019

@dunglas Same issue here.

Event is not received until i Ctrl+C the Mercure server in Git bash.

Running Windows 10 with Mercure allowed in Windows Firewall.

Laravel to serve the subscriber page using; idplus/laravel-mercure-publisher and php artisan server.

Calling the subscriber through Chrome; http://127.0.0.1:8000/subscribe

Using curl (or new browser tab) to call the publisher; curl http://127.0.0.1:8000/publish

Running Mercure with;
JWT_KEY='realBadKey' ADDR=':3000' DEMO=1 ALLOW_ANONYMOUS=1 CORS_ALLOWED_ORIGINS=* PUBLISH_ALLOWED_ORIGINS='http://localhost:3000' ./mercure

Here is the log;

time="2019-07-28T07:45:20+02:00" level=info msg="Mercure started" protocol=http
time="2019-07-28T07:45:21+02:00" level=info msg="New subscriber" remote_addr="127.0.0.1:58542"
127.0.0.1 - - [28/Jul/2019:07:45:21 +0200] "GET /hub?topic=mytopicname HTTP/1.1" 200 31 "http://127.0.0.1:8000/subscribe" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36"
time="2019-07-28T07:45:34+02:00" level=info msg="Subscriber disconnected" remote_addr="127.0.0.1:58542"
time="2019-07-28T07:45:34+02:00" level=info msg="New subscriber" remote_addr="127.0.0.1:58548"
time="2019-07-28T07:45:41+02:00" level=info msg="Update published" event_id=fa86a1a6-65ef-41d6-8d3f-bd4ee9f9320d remote_addr="127.0.0.1:58552"
127.0.0.1 - - [28/Jul/2019:07:45:41 +0200] "POST /hub HTTP/1.0" 200 36 "" ""
time="2019-07-28T07:45:41+02:00" level=info msg="Event sent" event_id=fa86a1a6-65ef-41d6-8d3f-bd4ee9f9320d remote_addr="127.0.0.1:58548"

// This gap here is where i press Ctrl + C, then the following happens;

127.0.0.1 - - [28/Jul/2019:07:45:34 +0200] "GET /hub?topic=mytopicname HTTP/1.1" 200 108 "http://127.0.0.1:8000/subscribe" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36"
time="2019-07-28T07:45:58+02:00" level=info msg="My Baby Shot Me Down"
@dunglas

This comment has been minimized.

Copy link
Owner

@dunglas dunglas commented Jul 29, 2019

Honestly, I've no clue of what's going on. I'm abroad for some weeks, but when I'll be back at the office I'll try to find a Windows computer to debug this. Any more information you can gather will be very appreciated.

In the meantime, maybe can you use Docker or any other Unix-based solution (a VM, Vagrant, Laravel Homestead etc) to run the Mercure hub?

@mantis

This comment has been minimized.

Copy link
Contributor

@mantis mantis commented Aug 10, 2019

@falconsmilie @dunglas - I spent a while looking at this today

I think that the issue here seems to be that the solution only works with HTTP/2.

In my initial setup, I was using localhost - I then tried on a linux server remotely with HTTP (not https) and Compress = 0. My initial thoughts was whether it was the linefeeds being generated. The sendheader's routine adds a dummy :\n message to push the headers out from GO, however I believe the protocol normally expects to see \n\n ?

In any case, with HTTP - I was finding that chrome/firefox etc do not work either locally or remotely.

I've then used the command line to generate a dummy openssl certificate

openssl req -newkey rsa:2048 -nodes -keyout domain.key -x509 -days 365 -out domain.crt

And specifieid this as extra paramater to the mercure application:

"CERT_FILE": "domain.crt", "KEY_FILE": "domain.key"

At first glance, the browsers now seem to be processing the responses correctly.

In terms of debugging this:

  1. onopen hook

If adding a onopen hook to the javascript e.g.:

es.onopen = e => { console.log("open"); }

The word 'open' does not appear in the console until pressing ctrl+c in mercure.

  1. tcp dump -wireshark with npcap for localhost debugging

A wireshark trace showed the mercure client sending the message to the mercure server and the mercure server sending the message to the browser.

Until Ctrl+C is pressed in mercure, nothing is processed.

I was hitting the same issue with responses not being processed when trying some online sample code for SSE (e.g. https://github.com/mchaov/simple-sse-nodejs-setup ) - so I'm not sure if the browser vendors have changed something such that https is required or similar for this to work.

@noubicastel4

This comment has been minimized.

Copy link

@noubicastel4 noubicastel4 commented Sep 6, 2019

Hello I have a problem to subscribe. When i pubish throuth mercure server after subscribing i get the data respone in my console but when i subscribe after publishing, i don't have anything as response. Please can i have a solution for it ?

Please excuse me for my bad english.

@football2801

This comment has been minimized.

Copy link
Author

@football2801 football2801 commented Sep 20, 2019

Any progress? We are at a point in my project where we either need to get this running or switch to another service.

@mantis

This comment has been minimized.

Copy link
Contributor

@mantis mantis commented Sep 20, 2019

@football2801 did you move from local development ?

@football2801

This comment has been minimized.

Copy link
Author

@football2801 football2801 commented Sep 20, 2019

We are attempting to deploy and test on a Linux machine running CentOS. We have so far been unable to get our localhost project to connect to the server to test. Typically getting 401 errors. We have yet to get it to connect as of right now.

@mantis

This comment has been minimized.

Copy link
Contributor

@mantis mantis commented Sep 20, 2019

when I tried to debug this - i came to conclusion that it might be better to try non-localhost

@football2801

This comment has been minimized.

Copy link
Author

@football2801 football2801 commented Sep 24, 2019

We have the code now running and posting to the CentOS machine. We get the Event Sent notification after POSTing to the server, but the server does not actually send the response through until I ctrl-c the connection.

@dunglas

This comment has been minimized.

Copy link
Owner

@dunglas dunglas commented Sep 24, 2019

Is there some kind of firewall on CentOS?

@dunglas

This comment has been minimized.

Copy link
Owner

@dunglas dunglas commented Sep 24, 2019

Could you try to add a breakpoint here to see if it's called or not? https://github.com/dunglas/mercure/blob/master/hub/subscribe.go#L186

@dunglas

This comment has been minimized.

Copy link
Owner

@dunglas dunglas commented Sep 24, 2019

On Windows, you should disable any firewall, antivirus, and check for malwares. It's most likely the cause of the issue.

@football2801

This comment has been minimized.

Copy link
Author

@football2801 football2801 commented Sep 25, 2019

I can't debug that line, I don't have the ability to add breakpoints on the Linux server. The CentOS machine is a remote server hosted by Linode. We have set it up with a temporary URL to be able to POST traffic from localhost to an actual remote environment to best simulate what the production environment will be.

The command we are running in Linux is:

JWT_KEY='!ChangeMe!' ADDR=':3000' ALLOW_ANONYMOUS=1 CORS_ALLOWED_ORIGINS=* ./mercure

Is there any chance we could do a screen share session to get some face to face live tests run?

@football2801

This comment has been minimized.

Copy link
Author

@football2801 football2801 commented Sep 25, 2019

On Windows, you should disable any firewall, antivirus, and check for malwares. It's most likely the cause of the issue.

This seems to be the issue. I have the exact same code running on a mac and everything works in all browsers that I've tested.

The major issue remains that I work in a large corporation who wants to use this software, we have managed firewalls and antivirus on our machines. If we want to use this service, we need to know what to allow through the security. So far we've tried a few things such as the CentOS server's ip, but the browser on windows still reports that it cannot make a connection.

The CentOS machine shows the event subscription, but the browser says that it cannot connect, almost as if the handshake is being made on the server, but the response is not making it back to the local machine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.