Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is port 9001 needed for Mosquitto? #67

Closed
Paraphraser opened this issue Jun 14, 2020 · 4 comments
Closed

Is port 9001 needed for Mosquitto? #67

Paraphraser opened this issue Jun 14, 2020 · 4 comments

Comments

@Paraphraser
Copy link

Is port 9001 needed for Mosquitto?

Summary:

  • Out-of-the box, the eclipse-mosquitto image does not listen to port 9001.
  • The evidence suggests that the docker-compose port mapping for 9001 is redundant.
  • Does anyone know any different?
  • If there are no objections, I'm proposing removal.

Chain of evidence

What eclipse-mosquitto image is on my RPi4?

$ docker images eclipse-mosquitto
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
eclipse-mosquitto   latest              a2b55301913b        11 days ago         5.86MB

What does Docker know about this image?

$ docker inspect eclipse-mosquitto
[
    {
        … [snip] …
        "RepoTags": [
            "eclipse-mosquitto:latest"
        ],
        … [snip] …
        "Created": "2020-06-02T21:51:19.217586203Z",
        … [snip] …
        "ContainerConfig": {
            … [snip] …
            "ExposedPorts": {
                "1883/tcp": {}
            },
            … [snip] …
            "Env": [
                … [snip] …
                "VERSION=1.6.10",
                … [snip] …
            ],
        },
    }
]

Observation: as far as docker inspect is concerned, only port 1883 is exposed.

Assume IOTstack is not running.

What does netstat have to say about the baseline situation on the RPi4:

netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp6       0      0 :::5900                 :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:5353            0.0.0.0:*                          
udp        0      0 0.0.0.0:41959           0.0.0.0:*                          
udp6       0      0 :::5353                 :::*                               
udp6       0      0 :::37708                :::*

Observation: no process is listening to either port 1883 or 9001.

Run-up mosquitto all by itself, explicitly specifying ports 1883 & 9001 (ie how docker-compose.yml does it at the moment):

$ docker run -d --name mosquitto -p 1883:1883 -p 9001:9001 eclipse-mosquitto
38589d808156e385aa58ebce2e25121b500c009c731995e610dd29c0e41fdb56

What does netstat think outside the container?

$ netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp6       0      0 :::9001                 :::*                    LISTEN     
tcp6       0      0 :::5900                 :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
tcp6       0      0 :::1883                 :::*                    LISTEN     
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:5353            0.0.0.0:*                          
udp        0      0 0.0.0.0:41959           0.0.0.0:*                          
udp6       0      0 :::5353                 :::*                               
udp6       0      0 :::37708                :::*

Observation: a process (Docker) is listening to ports 1883 & 9001.

What does netstat think inside the container?

pi@sec-dev:~/IOTstack $ docker exec mosquitto netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      
tcp        0      0 :::1883                 :::*                    LISTEN

Observation: a process (mosquitto) is listening to port 1883 but nothing is listening to port 9001.

Any complaints in the log?

$ docker logs mosquitto
1591924206: mosquitto version 1.6.10 starting
1591924206: Config loaded from /mosquitto/config/mosquitto.conf.
1591924206: Opening ipv4 listen socket on port 1883.
1591924206: Opening ipv6 listen socket on port 1883.

Observation: no complaints.

OK. Tear that down.

$ docker stop mosquitto
$ docker rm mosquitto

Now, repeat the test but this time without port 9001:

$ docker run -d --name mosquitto -p 1883:1883 eclipse-mosquitto
248a2763bf72f4d9c704a590083ce2170130a3cb82768d381c6e0e509fdb9ba7

The netstat view from outside:

$ netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp6       0      0 :::5900                 :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
tcp6       0      0 :::1883                 :::*                    LISTEN     
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:5353            0.0.0.0:*                          
udp        0      0 0.0.0.0:41959           0.0.0.0:*                          
udp6       0      0 :::5353                 :::*                               
udp6       0      0 :::37708                :::*

Observation: no listener for port 9001 (as expected).

The netstat view from inside:

$ docker exec mosquitto netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      
tcp        0      0 :::1883                 :::*                    LISTEN

Observation: no change – still only port 1883 (also as expected).

Any complaints in the log?

$ docker logs mosquitto
1591924553: mosquitto version 1.6.10 starting
1591924553: Config loaded from /mosquitto/config/mosquitto.conf.
1591924553: Opening ipv4 listen socket on port 1883.
1591924553: Opening ipv6 listen socket on port 1883.

Observation: mosquitto is still happy.

OK. Tear that down.

Now for the acid test. Comment-out the line in docker-compose.yml:

  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto
    restart: unless-stopped
    user: "1883"
    ports:
      - 1883:1883
#      - 9001:9001
    volumes:
      - ./volumes/mosquitto/data:/mosquitto/data
      - ./volumes/mosquitto/log:/mosquitto/log
      - ./services/mosquitto/mosquitto.conf:/mosquitto/config/mosquitto.conf
      - ./services/mosquitto/filter.acl:/mosquitto/config/filter.acl

Bring up the stack:

$ cd ~/IOTstack
$ docker-compose up -d

Look at netstat both inside and outside:

$ netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp6       0      0 :::9000                 :::*                    LISTEN     
tcp6       0      0 :::5900                 :::*                    LISTEN     
tcp6       0      0 :::2222                 :::*                    LISTEN     
tcp6       0      0 :::7920                 :::*                    LISTEN     
tcp6       0      0 :::2003                 :::*                    LISTEN     
tcp6       0      0 :::8083                 :::*                    LISTEN     
tcp6       0      0 :::53                   :::*                    LISTEN     
tcp6       0      0 :::8086                 :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
tcp6       0      0 :::1880                 :::*                    LISTEN     
tcp6       0      0 :::5432                 :::*                    LISTEN     
tcp6       0      0 :::3000                 :::*                    LISTEN     
tcp6       0      0 :::8089                 :::*                    LISTEN     
tcp6       0      0 :::1883                 :::*                    LISTEN     
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:68              0.0.0.0:*                          
udp        0      0 0.0.0.0:5353            0.0.0.0:*                          
udp        0      0 0.0.0.0:41959           0.0.0.0:*                          
udp6       0      0 :::53                   :::*                               
udp6       0      0 :::67                   :::*                               
udp6       0      0 :::5353                 :::*                               
udp6       0      0 :::37708                :::*                               

$ docker exec mosquitto netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 127.0.0.11:38641        0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      
tcp        0      0 :::1883                 :::*                    LISTEN      
udp        0      0 127.0.0.11:37575        0.0.0.0:*

Observation: no sign of port 9001.

Anything in the log?

$ tail volumes/mosquitto/log/mosquitto.log 
2020-06-12T00:59:27: mosquitto version 1.6.10 terminating
2020-06-12T00:59:27: Saving in-memory database to /mosquitto/data/mosquitto.db.
2020-06-12T01:19:18: mosquitto version 1.6.10 starting
2020-06-12T01:19:18: Config loaded from /mosquitto/config/mosquitto.conf.
2020-06-12T01:19:18: Opening ipv4 listen socket on port 1883.
2020-06-12T01:19:18: Opening ipv6 listen socket on port 1883.
2020-06-12T01:19:27: New connection from 172.27.0.6 on port 1883.

Observation: No complaints.

Conclusion: Port 9001 is not used and is probably not needed.

Googling

There are some references to port 9001:

  • fiware-tutorials

    Port 9001 is the standard port for HTTP/Websocket communications

  • steves-internet-guide

    MQTT over Websockets usually uses port 9001 but it isn’t fixed.

  • A restricted Google search:

     "9001" site:.mosquitto.org

    returns:

     Your search - "9001" site:.mosquitto.org - did not match any documents

    which suggests that, if there ever was a case for Mosquitto needing port 9001, it's disappeared into the bit-bucket of history.

Off-hand, I can't think of a use-case for Mosquitto to be involved in HTTP/Websocket comms. Can anyone else?

I do have a couple of use-cases for IoT devices communicating over HTTP but they do it with Node-Red, not Mosquitto:

  • TrigBoards searching for firmware updates (a quick check over HTTP is better for battery life than firing up OTA); and
  • A PurpleAir air-quality monitor (which doesn't support user-logging via MQTT).

Some brute force

Assuming there was a use-case for Mosquitto to chat over port 9001, it needs a bit more than just the port mapping in docker-compose.yml to get it going. In particular, these lines also need to be added to mosquitto.conf:

listener 1883
listener 9001

Both lines are needed - if you only add 9001 then 1883 stops working.

That definitely causes Mosquitto to listen on the port:

$ docker exec mosquitto netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      
tcp        0      0 0.0.0.0:9001            0.0.0.0:*               LISTEN      
tcp        0      0 127.0.0.11:40117        0.0.0.0:*               LISTEN      
tcp        0      0 :::1883                 :::*                    LISTEN      
tcp        0      0 :::9001                 :::*                    LISTEN      
udp        0      0 127.0.0.11:55876        0.0.0.0:*

but it's an open question as to what you can actually do with it.....

Bottom line

My view is that port 9001 should be removed from the Mosquitto template, with the two-step how-to of enabling port 9001 put in the documentation for anyone who subsequently discovers a use-case.

If nobody objects to that plan, I'll prepare a pull request.
@Willem-Dekker
Copy link
Collaborator

I think its a good thing to remove it. since it's not needed for 99% of the usecases. I think you can make the pr.

@Paraphraser
Copy link
Author

I think I will hold off until PR #69 has been applied. Just removing 9001 from the template probably wouldn't cause a conflict but I also intended to document the how-to of adding it back and that probably would create a conflict somewhere. Granted, I might be being a bit over-cautious in my estimation of what would and would not be likely to cause a conflict...

@Slyke
Copy link
Collaborator

Slyke commented Jun 17, 2020

I just merged #69, feel free!

Paraphraser added a commit to Paraphraser/IOTstack that referenced this issue Jun 21, 2020
@Paraphraser
Mosquitto - remove port 9001 from template
29d329d 
Following on from issue SensorsIot#67, this PR:

1. Removes the definition of port 9001 from the Mosquitto template;

2. Adds discussion to the Mosquitto documentation explaining how to
support port 9001 *properly,* should the need arise.
@Paraphraser Paraphraser mentioned this issue Jun 21, 2020
Merged
@thelostinthought
Copy link

you have to add a

listener 9001
protocol websockets

to mosquitto.conf for mosquitto to use it. This would allow web browsers to publish/subscibe to topics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Slyke @Willem-Dekker @Paraphraser @thelostinthought