Cannot compile with new (1.7.0) libmongoc

[mochrul]

syslog-ng

Version of syslog-ng

Probably all, tested with 3.11.1

Platform

Probably all, tested on amd64

Issue

Failure

The test script fails, probably because the accepted mongodb URI scheme is changed.

FAIL: modules/afmongodb/tests/test-mongodb-config
=================================================

2017/08/27 15:54:26.0627: [31686]:  WARNING:       mongoc: Empty hostname in URI
2017/08/27 15:54:26.0628: [31686]:  WARNING:       mongoc: Error parsing URI: 'Invalid host string in URI'

  ###########################################################################
  #
  # FAIL: ASSERTION FAILED: mismatch; no grabbed message contains the pattern=Initializing MongoDB destination; uri='mongodb:///tmp/mongo.sock', db='tmp/mongo.sock', collection='messages'
  #
  # Test case: test-mongodb-config.c/_execute()
  #
  #     _execute(socket, Initializing MongoDB destination; uri='mongodb:///tmp/mongo.sock', db='tmp/mongo.sock', collection='messages')
  #
  ###########################################################################

  # Grabbed internal messages follow:
  #	Error parsing MongoDB URI; uri='mongodb:///tmp/mongo.sock', driver='(null)'
error: expected the subject to succeed, but it failed
2017/08/27 15:54:26.0628: [31686]:  WARNING:       mongoc: Error parsing URI: 'Invalid URI Schema, expecting 'mongodb://''
2017/08/27 15:54:26.0628: [31686]:  WARNING:       mongoc: Empty hostname in URI
2017/08/27 15:54:26.0628: [31686]:  WARNING:       mongoc: Error parsing URI: 'Invalid host string in URI'

  ###########################################################################
  #
  # FAIL: ASSERTION FAILED: mismatch; no grabbed message contains the pattern=Initializing MongoDB destination; uri='mongodb:///tmp/mongo.sock?wtimeoutMS=60000&socketTimeoutMS=60000&connectTimeoutMS=60000', db='tmp/mongo.sock', collection='messages'
  #
  # Test case: test-mongodb-config.c/_execute()
  #
  #     _execute(path, Initializing MongoDB destination; uri='mongodb:///tmp/mongo.sock?wtimeoutMS=60000&socketTimeoutMS=60000&connectTimeoutMS=60000', db='tmp/mongo.sock', collection='messages')
  #
  ###########################################################################

  # Grabbed internal messages follow:
  #	WARNING: Using path() option is deprecated in mongodb driver, please use uri() instead;
  #	Error parsing MongoDB URI; uri='mongodb:///tmp/mongo.sock?wtimeoutMS=60000&socketTimeoutMS=60000&connectTimeoutMS=60000', driver='(null)'
error: expected the subject to succeed, but it failed
2017/08/27 15:54:26.0629: [31686]:  WARNING:       mongoc: Unsupported URI option "dont-care"
2017/08/27 15:54:26.0629: [31686]:  WARNING:       mongoc: A '/' is required between the host list and any options.
2017/08/27 15:54:26.0629: [31686]:  WARNING:       mongoc: Error parsing URI: 'Invalid host string in URI'
2017/08/27 15:54:26.0629: [31686]:  WARNING:       mongoc: Empty hostname in URI
2017/08/27 15:54:26.0629: [31686]:  WARNING:       mongoc: Error parsing URI: 'Invalid host string in URI'
2017/08/27 15:54:26.0629: [31686]:  WARNING:       mongoc: Empty hostname in URI
2017/08/27 15:54:26.0629: [31686]:  WARNING:       mongoc: Error parsing URI: 'Invalid host string in URI'
2017/08/27 15:54:26.0629: [31686]:  WARNING:       mongoc: Empty hostname in URI
2017/08/27 15:54:26.0629: [31686]:  WARNING:       mongoc: Error parsing URI: 'Invalid host string in URI'
FAIL modules/afmongodb/tests/test-mongodb-config (exit status: 1)

Steps to reproduce

1. Download and install the new libmnogoc version 1.7.0

2. Try to compile syslog-ng with system libmongoc

[mochrul]

I did some research and now I'm quite confident that the issue is really caused by the changed uri parsing method in Mongo db. The key problem here is that the mongodb decided to move the Unix domain sockets to the host part of the uri and therefore cannot contains forward slash. They decided that the Unix domain socket path therfore should be uri escaped and the slash should be replaced by %2F. Unfortunately syslog-ng also parse the uri and did not prepared for this change. I'm still investigating where the code fails exactly.

[bazsi]

Hi, Thanks for diagnosing this. How often do you think people use mongodb over unix domain sockets? Does anyone know?

…

-- Bazsi

On Mon, Aug 28, 2017 at 1:02 PM, Attila Szalay ***@***.***> wrote: I did some research and now I'm quite confident that the issue is really caused by the changed uri parsing method in Mongo db. The key problem here is that the mongodb decided to move the Unix domain sockets to the host part of the uri and therefore cannot contains forward slash. They decided that the Unix domain socket path therfore should be uri escaped and the slash should be replaced by %2F. Unfortunately syslog-ng also parse the uri and did not prepared for this change. I'm still investigating where the code fails exactly. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1653 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AArldqdeScP47hbzTgC7oE4z4l76287Tks5scp4ogaJpZM4PD2z2> .

[mochrul]

With more digging, I can tell, that the problem is not that big I thought.

Only the test case should be fixed and then the syslog-ng can be compiled. I removed the legacy test case because that was easier. I also created a patch, but this patch is definitively unfinished. It just compiles. So I just attached the patch here.

0002-Adapt-test-case-to-the-new-afmongodb-version.patch.txt

with the new version, I think.

[bazsi]

\o/

…

On Aug 28, 2017 22:56, "Attila Szalay" ***@***.***> wrote: With more digging, I can tell, that the problem is not that big I thought. Only the test case should be fixed and then the syslog-ng can be compiled. I removed the legacy test case because that was easier. I also created a patch, but this patch is definitively unfinished. It just compiles. So I just attached the patch here. 0002-Adapt-test-case-to-the-new-afmongodb-version.patch.txt <https://github.com/balabit/syslog-ng/files/1258352/0002-Adapt-test-case-to-the-new-afmongodb-version.patch.txt> with the new version, I think. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1653 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AArldoLc26Ke_LNOv48DNSZMEB_jfASzks5scymZgaJpZM4PD2z2> .

[lbudai]

@mochrul : for the upcoming 3.13 release I'll check the patch you sent and if works fine, we will integrate it.