DBZ-92, DBZ-97 Makes logging more verbose and changes the snapshot reader to produce separate events for each DDL change

[hchiorean]

No description provided.

[dchentech]

@hchiorean Great! Our team needs it.

[dchentech]

Oh, sorry for that, I did a duplicated thing https://github.com/DataPipelineInc/debezium/commit/3b9dddb51fd461d4e5b63fd5453d68cecf92cf91

[rhauch]

@hchiorean, I know this is what we talked about yesterday, but this splitting on ; concerns me because if any of the statements were to contain a ; character, then the split logic would break that one statement into several.

Perhaps an alternative is to not join but instead of one call to schema.applyDdl with a list on line 295 we call schema.applyDdl for each of the statements in the list. Or, @mvj3's suggestion doesn't even add the statements to the list but instead calls schema.applyDdl for each statement as we create them.

Thoughts?

[dchentech]

@rhauch Or I just create a PR about this?

This commit https://github.com/DataPipelineInc/debezium/commit/3b9dddb51fd461d4e5b63fd5453d68cecf92cf91 is tests passed in my local machine (via run mvn clean test with docker support)

[hchiorean]

amended the commit

[hchiorean]

/home/travis/build.sh: line 45: docker: command not found...not sure what that's about

[dchentech]

@hchiorean docker: unrecognized service
https://travis-ci.org/debezium/debezium/builds/150970664#L156

I also meet several docker issues on Travis, but them didn't happen on my local Mac. One of them is https://issues.jboss.org/browse/DBZ-93 [travis] Unable to create data directory /zookeeper/txns/version-2 . It suddenly fixed a few days ago, and I don't know why. But Travis still gives me other docker problems.

[rhauch]

@hchiorean, @mvj3: Travis are having issues at the moment, though at this time they've identified the problem. I'll restart the job when they fully resolve the problems.

[rhauch]

This pull request failed on Travis multiple times with errors like the following after the first build (recall that the build attempts to re-run maven upon an error, and the subsequent failures are different):

[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.15.12:start 
(start) on project debezium-connector-mongodb: I/O Error: Unable to start container id
[39fdb7dcd6cb]: driver failed programming external connectivity on endpoint mongo1
(ae0bf53b1e6f17f44e47c298998223fda6510667ee088f0eb1590d3bf4a839cc): 
Error starting userland proxy: listen tcp 0.0.0.0:27017: bind: address already in use 
(Internal Server Error: 500) -> [Help 1]

This error implies that the mongo container that we're trying to start is failing because the port 27017 is already being used. Either something is still messed up in the Travis environment, though late yesterday they reported everything is fine while today they've said people are still having problems and that people should contact them if that is the case. I've sent them an email asking whether this might be related to the recent problems or if port 27017 is now reserved in some way. I'll keep you posted.

In the meantime, @hchiorean perhaps you might try creating a pull request that changes the MongoDB port to something else (e.g., 28017). Unfortunately, it needs to be changed in two places; feel free to set a property to make this easier.

[rhauch]

I'm not sure what's going on, but this builds locally. Since I'm certain that nothing in the PR is broken but instead there's something wrong with the Travis environment, I'm going to merge this anyway.

[rhauch]

Travis-CI support responded, and they've upgraded their images to ensure MongoDB is disabled upon boot. Woot!