[005] - Fix problems in productionalized CDC code #2248

jbartok · 2020-05-12T12:11:03Z

Checklist

Tags Set
Milestone Set

ghost · 2020-05-12T12:39:33Z

The job jet-oss-pr-builder of your PR failed (log, artifacts).
Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log file

--------------------------
---------SUMMARY----------
--------------------------
[ERROR] COMPILATION ERROR : 
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project hazelcast-jet-cdc-debezium: Compilation failure: Compilation failure: 
--------------------------
---------ERRORS-----------
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[55,35] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[55,35] cannot find symbol
--------------------------

ghost · 2020-05-13T12:30:02Z

The job jet-oss-pr-builder of your PR failed (log, artifacts).
Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log file

--------------------------
---------SUMMARY----------
--------------------------
[ERROR] COMPILATION ERROR : 
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project hazelcast-jet-cdc-debezium: Compilation failure: Compilation failure: 
--------------------------
---------ERRORS-----------
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[55,35] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[55,35] cannot find symbol
--------------------------

ghost · 2020-05-14T11:48:35Z

The job jet-oss-pr-builder of your PR failed (log, artifacts).
Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log file

--------------------------
---------SUMMARY----------
--------------------------
[ERROR] COMPILATION ERROR : 
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project hazelcast-jet-cdc-debezium: Compilation failure: Compilation failure: 
--------------------------
---------ERRORS-----------
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[55,35] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[55,35] cannot find symbol
--------------------------

ghost · 2020-05-14T12:25:28Z

The job jet-oss-pr-builder of your PR failed (log, artifacts).
Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log file

--------------------------
---------SUMMARY----------
--------------------------
[ERROR] COMPILATION ERROR : 
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project hazelcast-jet-cdc-debezium: Compilation failure: Compilation failure: 
--------------------------
---------ERRORS-----------
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[55,35] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[44,28] cannot find symbol
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[55,35] cannot find symbol
--------------------------

ghost · 2020-05-18T07:18:42Z

The job jet-oss-pr-builder of your PR failed (log, artifacts).
Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log file

--------------------------
---------SUMMARY----------
--------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:license-maven-plugin:1.19:add-third-party (default) on project hazelcast-jet-distribution: There are some forbidden licenses used, please check your dependencies. -> [Help 1]
--------------------------

extensions/cdc-debezium/pom.xml

extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/ChangeRecordImpl.java

extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/CdcSource.java

extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java

...et-core/src/main/java/com/hazelcast/jet/impl/connector/UpdateMapWithMaterializedValuesP.java

hazelcast-jet-core/src/main/java/com/hazelcast/jet/impl/connector/AsyncHazelcastWriterP.java

cangencer · 2020-05-19T05:38:36Z

CdcSources seem to be missing on https://jet-start.sh/docs/next/api/sources-sinks#sources

cangencer · 2020-05-19T05:39:34Z

One more: https://jet-start.sh/docs/next/api/sources-sinks#debezium section should now be removed

jbartok · 2020-05-19T05:49:09Z

CdcSources seem to be missing on https://jet-start.sh/docs/next/api/sources-sinks#sources

One more: https://jet-start.sh/docs/next/api/sources-sinks#debezium section should now be removed

These are already fixed, but on this PR's branch, so will show up once we merge it in. I'll extract the sink part and address the other concerns, so that we can merge it in today, hopefully.

ghost · 2020-05-19T06:16:30Z

The job jet-oss-pr-builder of your PR failed (log, artifacts).
Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log file

--------------------------
---------SUMMARY----------
--------------------------
[ERROR] COMPILATION ERROR : 
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[43,27] method mapFrom in class com.hazelcast.jet.json.JsonUtil cannot be applied to given types;
--------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project hazelcast-jet-cdc-debezium: Compilation failure
--------------------------
---------ERRORS-----------
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[43,27] method mapFrom in class com.hazelcast.jet.json.JsonUtil cannot be applied to given types;
--------------------------
[ERROR] /home/jenkins/jenkins_slave/workspace/jet-oss-pr-builder/extensions/cdc-debezium/src/main/java/com/hazelcast/jet/cdc/impl/RecordPartImpl.java:[43,27] method mapFrom in class com.hazelcast.jet.json.JsonUtil cannot be applied to given types;
--------------------------

cangencer · 2020-05-19T06:27:35Z

CdcSources seem to be missing on https://jet-start.sh/docs/next/api/sources-sinks#sources

One more: https://jet-start.sh/docs/next/api/sources-sinks#debezium section should now be removed

These are already fixed, but on this PR's branch, so will show up once we merge it in. I'll extract the sink part and address the other concerns, so that we can merge it in today, hopefully.

you're right, missed it

jbartok · 2020-05-19T07:52:53Z

Moved all sink related code and website content to #2262

jbartok · 2020-05-19T07:53:11Z

This should be ready to merge in

Bartok Jozsef added 5 commits May 12, 2020 09:30

Set Jet version for website replacement script

60c326b

Change type of serialization used

4edb48e

Use JSON convenience instead of Jackson jr

a53fbbf

Fix jar assembly

73bd41c

Remove redundant cast

2e5db9f

jbartok self-assigned this May 12, 2020

jbartok added the cdc Change Data Capture label May 12, 2020

jbartok added this to the 4.2 milestone May 12, 2020

Fix some website content

3d0a87e

Re-implement CDC specific map sinks

fe83154

Bartok Jozsef added 2 commits May 14, 2020 14:57

Make jackson annotations part of the main CDC jar

413937b

Adjust tutorial to latest changes

16ad3b0

Bartok Jozsef added 3 commits May 14, 2020 16:17

Merge branch 'master' into cdc-fixes

a591068

Adapt to latest json convenience changes

701f42a

Add CDC map sink to sources & sinks page

9ef66ff

cangencer changed the title ~~Fix problems in productionalized CDC code~~ [005] - Fix problems in productionalized CDC code May 15, 2020

Bartok Jozsef added 2 commits May 15, 2020 09:54

Revert some changes in the tutorial

4bb9f77

Change licensing of the CDC modules

ce0aca5

jbartok marked this pull request as ready for review May 15, 2020 07:30

Bartok Jozsef added 4 commits May 15, 2020 11:13

Fix javadoc

df1d4e6

Extract some common processor code

6f1dd23

Force total parallelism of sink to 1

2574aad

Merge branch 'master' into cdc-fixes

e04c2b4

Bartok Jozsef added 2 commits May 18, 2020 10:40

Re-add dependency exclusion from master

dafdb7c

Proper fix for licenses

ecb7715

jbartok requested a review from cangencer May 18, 2020 08:04

cangencer reviewed May 18, 2020

View reviewed changes

Inline some methods used for serialization

e826cc5

Rename Apache license header file

fb6f9f8

Bartok Jozsef added 3 commits May 19, 2020 09:35

Merge branch 'master' into cdc-fixes

5f497ad

Adapt to latest changes

b49840f

Move CdcSinks to another PR

cb1ddb0

jbartok requested a review from cangencer May 19, 2020 11:34

cangencer approved these changes May 19, 2020

View reviewed changes

jbartok merged commit 5bc2781 into hazelcast:master May 20, 2020

jbartok deleted the cdc-fixes branch May 20, 2020 06:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[005] - Fix problems in productionalized CDC code #2248

[005] - Fix problems in productionalized CDC code #2248

jbartok commented May 12, 2020 •

edited

ghost commented May 12, 2020

ghost commented May 13, 2020

ghost commented May 14, 2020

ghost commented May 14, 2020

ghost commented May 18, 2020

cangencer commented May 19, 2020

cangencer commented May 19, 2020

jbartok commented May 19, 2020

ghost commented May 19, 2020

cangencer commented May 19, 2020

jbartok commented May 19, 2020

jbartok commented May 19, 2020

[005] - Fix problems in productionalized CDC code #2248

[005] - Fix problems in productionalized CDC code #2248

Conversation

jbartok commented May 12, 2020 • edited

ghost commented May 12, 2020

ghost commented May 13, 2020

ghost commented May 14, 2020

ghost commented May 14, 2020

ghost commented May 18, 2020

cangencer commented May 19, 2020

cangencer commented May 19, 2020

jbartok commented May 19, 2020

ghost commented May 19, 2020

cangencer commented May 19, 2020

jbartok commented May 19, 2020

jbartok commented May 19, 2020

jbartok commented May 12, 2020 •

edited