New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No data sent/recv when using mosquitto_connect_async with my own loop #990

Closed
doragasu opened this Issue Oct 17, 2018 · 6 comments

Comments

Projects
None yet
2 participants
@doragasu

doragasu commented Oct 17, 2018

I have an application that uses asynchronous operation without the threaded interface. I need to work like this because I have a library managing file descriptors using epoll, and I need to use it to manage the mosquitto socket.

This was working flawlessly with mosquitto 1.4.8 and 1.4.10. But since I updated to version 1.5.1, it stopped working. And the same happens with 1.5.3. I have coded the following minimum [non] working example:

#include <mosquitto.h>
#include <stdio.h>
#include <sys/epoll.h>

// Set the following parameters as needed
#define USERNAME	"my_username"
#define PASSWORD	"my_password"
#define HOST		"my_broker.com"
#define CLIENT_ID 	USERNAME
#define PORT		8883
#define KEEPALIVE	60

#define EVENTS 		(EPOLLIN | EPOLLOUT | EPOLLERR | EPOLLHUP | EPOLLRDHUP | EPOLLET)

static void connect_cb(struct mosquitto *mosq, void *priv, int rc)
{
	printf("%s: %i received (%s)\n", __func__, rc, mosquitto_connack_string(rc));
}

static void disconnect_cb(struct mosquitto *mosq, void *priv, int rc)
{
	printf("%s: %i received (%s)\n", __func__, rc, mosquitto_connack_string(rc));
}

static void log_cb(struct mosquitto *mosq, void *priv, int level, const char *msg)
{
	printf("%s: %i %s\n", __func__, level, msg);
}

static struct mosquitto* mosq_init(void)
{
	struct mosquitto *mosq = NULL;
	const char *client_id = CLIENT_ID;
	int err = 0;

	err = mosquitto_lib_init();
	if (!err) {
		mosq = mosquitto_new(client_id, true, NULL);
		err = mosq == NULL?-2:0;
	}
	if (!err) err = mosquitto_username_pw_set(mosq, USERNAME, PASSWORD);
	if (!err) err = mosquitto_tls_set(mosq, NULL, "/etc/ssl/certs", NULL, NULL, NULL);
	if (!err) {
		mosquitto_connect_callback_set(mosq, connect_cb);
		mosquitto_disconnect_callback_set(mosq, disconnect_cb);
		mosquitto_log_callback_set(mosq, log_cb);
	}
	printf("%s: code %d\n", __func__, err);

	if (err && mosq) {
		mosquitto_destroy(mosq);
		mosq = NULL;
	}

	return mosq;
}

int main(void)
{
	struct mosquitto *mosq = NULL;
	int mosq_socket = -1;
	int err = -1;
	int epfd = -1;
	struct epoll_event ev;

	// Init and connect
	mosq = mosq_init();
	if (mosq) err = mosquitto_connect_async(mosq, HOST, PORT, KEEPALIVE);

	if (!err) {
		epfd = epoll_create1(0);
		err = epfd < 0?-3:0;
	}
	if (!err) {
		mosq_socket = mosquitto_socket(mosq);
		ev.data.fd = mosq_socket;
		ev.events = EVENTS;
		err = epoll_ctl(epfd, EPOLL_CTL_ADD, mosq_socket, &ev);
	}

	// Loop
	while (!err) {
		err = epoll_wait(epfd, &ev, 1, 250);
		if (err < 0) err = -4;
		else if (err > 0) {
			// Event
			err = -5;
			if (ev.events & EPOLLIN) {
				printf("read triggered\n");
				mosquitto_loop_read(mosq, 1);
				err = 0;
			}
			if (ev.events & EPOLLOUT) {
				if (mosquitto_want_write(mosq)) {
					printf("write triggered\n");
					mosquitto_loop_write(mosq, 1);
				}
				err = 0;
			}
		}
		mosquitto_loop_misc(mosq);
	}

	printf("%s: ERROR %d\n", __func__, err);

	return err;
}

If I compile and run this code using mosquitto 1.4.10, I get the following output:

mosq_init: code 0
log_cb: 16 Client my_username sending CONNECT
write triggered
read triggered
write triggered
read triggered
write triggered
read triggered
write triggered
read triggered
log_cb: 16 Client my_username received CONNACK
connect_cb: 0 received (Connection Accepted.)

Everything works, and if I capture the packets, I can see the cert and key exchange prior to the CONNACK. But when I run the program using mosquitto 1.5.3, the "write_triggered" message is printed once and the connection never completes. If I monitor the network using tcpdump, I can see that the TCP handshake is completed, but the TLS negotiation never starts:

TCP      76     35026 → 8883 [SYN, ECN, CWR] Seq=0 Win=26280 Len=0 MSS=1314 SACK_PERM=1 TSval=1714539719 TSecr=0 WS=128
TCP      76     8883 → 35026 [SYN, ACK, ECN] Seq=0 Ack=1 Win=28960 Len=0 MSS=1460 SACK_PERM=1 TSval=44011310 TSecr=1714539719 WS=256
TCP      68     35026 → 8883 [ACK] Seq=1 Ack=1 Win=26368 Len=0 TSval=1714539778 TSecr=44011310

And that's all. If I remove the EPOLLET from EVENTS, the "write triggered" message is continuously printed, but the connection is never completed, the cert/key exchange never happens. So it looks like mosquitto_loop_write() is not working as expected.

Am I doing anything wrong, or is there a regression?

@doragasu

This comment has been minimized.

doragasu commented Oct 22, 2018

Is my approach correct? There are some things that I am not sure about:

  1. Should I use mosquitto_connect() or mosquitto_connect_async()?
  2. Should I call mosquitto_loop_start()? With version 1.4.10 it worked without calling it...

I have been experimenting doing variations of the program above, and it looks like it works if I call mosquitto_loop_start() independently of using connect or connect_async, but I would like to know which one is the correct approach, and why it worked with version 1.4.10 and it does not work now.

@ralight

This comment has been minimized.

Contributor

ralight commented Nov 4, 2018

I've found the commit that introduces the problem, it's not you at fault :) I still need to diagnose the underlying problem, that is a task for next week.

For reference - if you're using mosquitto_loop_start(), you should not use any of the other loop functions, the library is doing the work for you.

@ralight ralight added this to the 1.5.4 milestone Nov 4, 2018

@doragasu

This comment has been minimized.

doragasu commented Nov 4, 2018

Thanks for the feedback. I was currently working around the bug by calling mosquitto_loop() during the connection stage, and then mosquitto_loop_{read,write,misc}() once the connection is completed. It's ugly but works until this gets fixed.

@ralight

This comment has been minimized.

Contributor

ralight commented Nov 7, 2018

I think this is fixed on the fixes branch, could you give it a try?

@doragasu

This comment has been minimized.

doragasu commented Nov 8, 2018

Thanks, I will give it a try as soon as I have a slot.

@ralight ralight closed this in 71b8c4d Nov 8, 2018

@doragasu

This comment has been minimized.

doragasu commented Nov 12, 2018

Finally I got time to test this. I have tried with version 1.5.4 (that I can see the fix has been merged into). The example above works if I connect using mosquitto_connect(). But if I leave the example as is (using mosquitto_connect_async()), it still does not work. Maybe this is the correct behavior, and I should use moquitto_connect().

Thank you very much for the fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment