Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bump io.confluent.schema-registry.version to 7.6.0-0 after bumping ksql version to 7.6 #9969

Merged
merged 1 commit into from Jun 16, 2023
Merged

Bump io.confluent.schema-registry.version to 7.6.0-0 after bumping ksql version to 7.6 #9969

merged 1 commit into from Jun 16, 2023

Conversation

rkhachatryan
Copy link
Contributor

@rkhachatryan rkhachatryan commented Jun 15, 2023 •
edited

On master, io.confluent.ksql.version is 7.6.0-0, but io.confluent.schema-registry.version is 7.5.0-811
which supposedly causes packaging build failure:

 [ERROR] Failed to execute goal on project ksqldb-common: Could not resolve dependencies for project io.confluent.ksql:ksqldb-common:jar:7.6.0-0: Could not find artifact io.confluent:kafka-connect-avro-converter:jar:7.5.0-0 in confluent (https://packages.confluent.io/maven/) -> [Help 1]

The latest kafka-connect-avro-converter version is 7.6.0-3

@rkhachatryan rkhachatryan marked this pull request as ready for review June 15, 2023 08:45
@rkhachatryan rkhachatryan requested a review from a team as a code owner June 15, 2023 08:45
@rkhachatryan
Copy link
Contributor Author

rkhachatryan commented Jun 16, 2023 •
edited

The build failed with

  [2023-06-16T11:45:53.714Z] [ERROR]   Run 8451: Generated topology differs from that built by previous versions of KSQL - this likely means there is a non-backwards compatible change.
  [2023-06-16T11:45:53.714Z] THIS IS BAD!
  [2023-06-16T11:45:53.716Z] Expected: is "Topologies:
   Sub-topology: 0
    Source: KSTREAM-SOURCE-0000000001 (topics: [left_topic])
      --> KTABLE-SOURCE-0000000002
    Processor: KTABLE-SOURCE-0000000002 (stores: [])
      --> KTABLE-TRANSFORMVALUES-0000000003
      <-- KSTREAM-SOURCE-0000000001
    Processor: KTABLE-TRANSFORMVALUES-0000000003 (stores: [])
      --> KTABLE-TRANSFORMVALUES-0000000004
      <-- KTABLE-SOURCE-0000000002
    Processor: KTABLE-TRANSFORMVALUES-0000000004 (stores: [])
      --> KTABLE-TOSTREAM-0000000005
      <-- KTABLE-TRANSFORMVALUES-0000000003
    Processor: KTABLE-TOSTREAM-0000000005 (stores: [])
      --> LeftSourceKeyed-SelectKey-Mapper
      <-- KTABLE-TRANSFORMVALUES-0000000004
    Processor: LeftSourceKeyed-SelectKey-Mapper (stores: [])
      --> LeftSourceKeyed-SelectKey-repartition-filter
      <-- KTABLE-TOSTREAM-0000000005
    Processor: LeftSourceKeyed-SelectKey-repartition-filter (stores: []  )
      --> LeftSourceKeyed-SelectKey-repartition-sink
      <-- LeftSourceKeyed-SelectKey-Mapper
    Sink: LeftSourceKeyed-SelectKey-repartition-sink (topic: LeftSourceKeyed-SelectKey-repartition)
      <-- LeftSourceKeyed-SelectKey-repartition-filter

  Sub-topology: 1
    Source: LeftSourceKeyed-SelectKey-repartition-source (topics: [LeftSourceKeyed-SelectKey-repartition])
        --> LeftSourceKeyed-SelectKey
    Source: RightSourceKeyed-SelectKey-repartition-source (topics: [RightSourceKeyed-SelectKey-repartition])
      --> RightSourceKeyed-SelectKey
    Processor: LeftSourceKeyed-SelectKey (stores: [LeftSourceKeyed-Materialize])
      --> PrependAliasLeft
      <-- LeftSourceKeyed-SelectKey-repartition-source
    Processor: RightSourceKeyed-SelectKey (stores: [RightSourceKeyed-Materialize])
      --> PrependAliasRight
      <-- RightSourceKeyed-SelectKey-repartition-source
    Processor: PrependAliasLeft (stores: [])
      --> KTABLE-JOINTHIS-0000000025
      <-- LeftSourceKeyed-SelectKey
    Processor: PrependAliasRight (stores: [])
      --> KTABLE-JOINOTHER-0000000026
      <-- RightSourceKeyed-SelectKey
    Processor: KTABLE-JOINOTHER-0000000026 (stores: [LeftSourceKeyed-Materialize])
      --> KTABLE-MERGE-0000000024
      <-- PrependAliasRight
    Processor: KTABLE-JOINTHIS-0000000025 (stores: [RightSourceKeyed-Materialize])
      --> KTABLE-MERGE-0000000024
      <-- PrependAliasLeft
    Processor: KTABLE-MERGE-0000000024 (stores: [])
      --> KTABLE-TRANSFORMVALUES-0000000027
        <-- KTABLE-JOINTHIS-0000000025, KTABLE-JOINOTHER-0000000026
    Processor: KTABLE-TRANSFORMVALUES-0000000027 (stores: [Project-Last])
      --> KTABLE-TOSTREAM-0000000028
      <-- KTABLE-MERGE-0000000024
    Processor: KTABLE-TOSTREAM-0000000028 (stores: [])
      --> KSTREAM-SINK-0000000029
      <-- KTABLE-TRANSFORMVALUES-0000000027
    Sink: KSTREAM-SINK-0000000029 (topic: OUTPUT)
      <-- KTABLE-TOSTREAM-0000000028

  Sub-topology: 2
    Source: KSTREAM-SOURCE-0000000013 (topics: [right_topic])
      --> KTABLE-SOURCE-0000000014
    Processor: KTABLE-SOURCE-0000000014 (stores: [])
      --> KTABLE-TRANSFORMVALUES-0000000015
      <-- KSTREAM-SOURCE-0000000013
    Processor: KTABLE-TRANSFORMVALUES-0000000015 (stores: [])
      --> KTABLE-TRANSFORMVALUES-0000000016
      <-- KTABLE-SOURCE-0000000014
    Processor: KTABLE-TRANSFORMVALUES-0000000016 (stores: [])
      --> KTABLE-TOSTREAM-0000000017
      <-- KTABLE-TRANSFORMVALUES-0000000015
    Processor: KTABLE-TOSTREAM-0000000017 (stores: [])
      --> RightSourceKeyed-SelectKey-Mapper
      <-- KTABLE-TRANSFORMVALUES-0000000016
    Processor: RightSourceKeyed-SelectKey-Mapper (stores: [])
      --> RightSourceKeyed-SelectKey-repartition-filter
      <-- KTABLE-TOSTREAM-0000000017
    Processor: RightSourceKeyed-SelectKey-repartition-filter (stores: [])
      --> RightSourceKeyed-SelectKey-repartition-sink
      <-- RightSourceKeyed-SelectKey-Mapper
    Sink: RightSourceKeyed-SelectKey-repartition-sink (topic: RightSourceKeyed-SelectKey-repartition)
      <-- RightSourceKeyed-SelectKey-repartition-filter

"
  [2023-06-16T11:45:53.718Z]      but: was "Topologies:
   Sub-topology: 0
    Source: KSTREAM-SOURCE-0000000001 (topics: [left_topic])
      --> KTABLE-SOURCE-0000000002
    Processor: KTABLE-SOURCE-0000000002 (stores: [])
      --> KTABLE-TRANSFORMVALUES-0000000003
      <-- KSTREAM-SOURCE-0000000001
    Processor: KTABLE-TRANSFORMVALUES-0000000003 (stores: [])
      --> KTABLE-TRANSFORMVALUES-0000000004
      <-- KTABLE-SOURCE-0000000002
    Processor: KTABLE-TRANSFORMVALUES-0000000004 (stores: [])
      --> KTABLE-TOSTREAM-0000000005
      <-- KTABLE-TRANSFORMVALUES-0000000003
    Processor: KTABLE-TOSTREAM-0000000005 (stores: [])
      --> LeftSourceKeyed-SelectKey-Mapper
      <-- KTABLE-TRANSFORMVALUES-0000000004
    Processor: LeftSourceKeyed-SelectKey-Mapper (stores: [])
      --> LeftSourceKeyed-SelectKey-repartition-filter
      <-- KTABLE-TOSTREAM-0000000005
    Processor: LeftSourceKeyed-SelectKey-repartition-filter (stores: [])
      --> LeftSourceKeyed-SelectKey-repartition-sink
      <-- LeftSourceKeyed-SelectKey-Mapper
    Sink: LeftSourceKeyed-SelectKey-repartition-sink (topic: LeftSourceKeyed-SelectKey-repartition)
      <-- LeftSourceKeyed-SelectKey-repartition-filter

  Sub-topology: 1
    Source: LeftSourceKeyed-SelectKey-repartition-source (topics: [LeftSourceKeyed-SelectKey-repartition])
        --> LeftSourceKeyed-SelectKey
    Source: RightSourceKeyed-SelectKey-repartition-source (topics: [RightSourceKeyed-SelectKey-repartition])
      --> RightSourceKeyed-SelectKey
    Processor: LeftSourceKeyed-SelectKey (stores: [LeftSourceKeyed-Materialize])
      --> PrependAliasLeft
      <-- LeftSourceKeyed-SelectKey-repartition-source
    Processor: RightSourceKeyed-SelectKey (stores: [RightSourceKeyed-Materialize])
      --> PrependAliasRight
      <-- RightSourceKeyed-SelectKey-repartition-source
    Processor: PrependAliasLeft (stores: [])
      --> KTABLE-JOINTHIS-0000000025
      <-- LeftSourceKeyed-SelectKey
    Processor: PrependAliasRight (stores: [])
      --> KTABLE-JOINOTHER-0000000026
      <-- RightSourceKeyed-SelectKey
    Processor: KTABLE-JOINOTHER-0000000026 (stores: [RightSourceKeyed-Materialize, LeftSourceKeyed-Materialize])
      --> KTABLE-MERGE-0000000024
      <-- PrependAliasRight
    Processor: KTABLE-JOINTHIS-0000000025 (stores: [RightSourceKeyed-Materialize, LeftSourceKeyed-Materialize])
      --> KTABLE-MERGE-0000000024
      <-- PrependAliasLeft
    Processor: KTABLE-MERGE-0000000024 (stores: [])
      --> KTABLE-TRANSFORMVALUES-0000000027
      <-- KTABLE-JOINTHIS-0000000025, KTABLE-JOINOTHER-0000000026
    Processor: KTABLE-TRANSFORMVALUES-0000000027 (stores: [Project-Last])  
      --> KTABLE-TOSTREAM-0000000028
      <-- KTABLE-MERGE-0000000024
    Processor: KTABLE-TOSTREAM-0000000028 (stores: [])
      --> KSTREAM-SINK-0000000029
      <-- KTABLE-TRANSFORMVALUES-0000000027
    Sink: KSTREAM-SINK-0000000029 (topic: OUTPUT)
      <-- KTABLE-TOSTREAM-0000000028

  Sub-topology: 2
    Source: KSTREAM-SOURCE-0000000013 (topics: [right_topic])
        --> KTABLE-SOURCE-0000000014
    Processor: KTABLE-SOURCE-0000000014 (stores: [])
      --> KTABLE-TRANSFORMVALUES-0000000015
      <-- KSTREAM-SOURCE-0000000013
    Processor: KTABLE-TRANSFORMVALUES-0000000015 (stores: [])
      --> KTABLE-TRANSFORMVALUES-0000000016
      <-- KTABLE-SOURCE-0000000014
    Processor: KTABLE-TRANSFORMVALUES-0000000016 (stores: [])
      --> KTABLE-TOSTREAM-0000000017
      <-- KTABLE-TRANSFORMVALUES-0000000015
    Processor: KTABLE-TOSTREAM-0000000017 (stores: [])
      --> RightSourceKeyed-SelectKey-Mapper
      <-- KTABLE-TRANSFORMVALUES-0000000016
    Processor: RightSourceKeyed-SelectKey-Mapper (stores: [])
      --> RightSourceKeyed-SelectKey-repartition-filter
      <-- KTABLE-TOSTREAM-0000000017
    Processor: RightSourceKeyed-SelectKey-repartition-filter (stores: [])
      --> RightSourceKeyed-SelectKey-repartition-sink
      <-- RightSourceKeyed-SelectKey-Mapper
    Sink: RightSourceKeyed-SelectKey-repartition-sink (topic: RightSourceKeyed-SelectKey-repartition)
      <-- RightSourceKeyed-SelectKey-repartition-filter

"
  [2023-06-16T11:45:53.719Z] failed test: table-table-join - table-table key join with avro schema and schema id-7.4.0-1662536988504
  [2023-06-16T11:45:53.719Z] in file:///home/jenkins/workspace/lic_Repo_PR_builder_ksql_PR-9969/ksqldb-functional-tests/src/test/resources/historical_plans/table-table-join_-_table-table_key_join_with_avro_schema_and_schema_id/7.4.0_1662536988504

Which I think is unrelated to this PR.

@rkhachatryan rkhachatryan merged commit 7bce1ea into master Jun 16, 2023
@rkhachatryan rkhachatryan deleted the bump-schema-registry-version branch June 16, 2023 15:01
@vcrfxia vcrfxia mentioned this pull request Jun 22, 2023
Merged
3 tasks
@vcrfxia
Copy link
Contributor

vcrfxia commented Jun 22, 2023

Hey @rkhachatryan @lucasbru , thanks for noticing the outdated dependency version and fixing it! A note for the future, though: the SR dependency version should not be set to a -0 nanoversion. Setting it to a -0 nanoversion means that Jenkins no longer automatically bumps the version as new nanoversions are released (here's an example of where this previously happened), which means ksql development locally will not match what's seen in Jenkins, since Jenkins continually recompiles the latest schema-registry whereas ksql developers typically do not. I stumbled upon this while debugging the test failure in #9982 -- the tests were failing in Jenkins but not locally due to the use of the -0 nanoversion, which was very confusing.

The fix is to set the version to the latest available nanoversion instead, which I've done in #9982.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lucasbru lucasbru lucasbru approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@rkhachatryan @vcrfxia @lucasbru