Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store protocol version from spec #16416

Merged
merged 8 commits into from
Sep 9, 2022

Conversation

gosusnp
Copy link
Contributor

@gosusnp gosusnp commented Sep 7, 2022

What

Store the protocol version of the connector when creating/updating a specific connector.

Closes #15978

@github-actions github-actions bot added area/api Related to the api area/documentation Improvements or additions to documentation area/platform issues related to the platform area/server labels Sep 7, 2022
@gosusnp gosusnp temporarily deployed to more-secrets September 7, 2022 22:32 Inactive
@gosusnp
Copy link
Contributor Author

gosusnp commented Sep 7, 2022

The end to end tests are currently failing because we need to publish the test connectors.
Currently being addressed in #16404

@@ -2603,6 +2603,9 @@ components:
format: uri
icon:
type: string
protocolVersion:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what we have done in the past, but this has been named protocol_version in the protocol. Should we match the case with the field in the protocol or have we been using camel case traditionally here in the API definition even if the protocol uses lower underscore?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sticking camel case for consistency.
Our casing is a bit inconsistent at the moment, but a lot of it is in the protocol which would make fixing it a breaking change. We should plan that clean up at some point.

"2.1.1, 0.2.0",
"2.2.0, 0.2.1",
})
void testCreateSourceSpec(final String dockerImageTag, final String expectedProtocolVersion)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not very clear about what is going to be feed in the parameter without looking a the doc. Am I the only one to feel that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option would be using a different source. I am not a fan of the extra link to follow to get to the test arguments.

@MethodSource("stringIntAndListProvider")
void testWithMultiArgMethodSource(String str, int num, List<String> list) {
    assertEquals(5, str.length());
    assertTrue(num >=1 && num <=2);
    assertEquals(2, list.size());
}

static Stream<Arguments> stringIntAndListProvider() {
    return Stream.of(
        arguments("apple", 1, Arrays.asList("a", "b")),
        arguments("lemon", 2, Arrays.asList("x", "y"))
    );
}

@@ -197,6 +200,8 @@ private StandardDestinationDefinition destinationDefinitionFromCreate(final Dest
destinationDefCreate.getDockerRepository(),
destinationDefCreate.getDockerImageTag());

final AirbyteVersion airbyteProtocolVersion = AirbyteProtocolVersion.getWithDefault(spec.getProtocolVersion());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not directly related to this PR but AirbyteVersion is confusing. Should we rename it SemanticVersion because it might not be an AirbyteVersion but a protocolVersion instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, I was thinking of using a 3rd party Version implementation (I saw maven/jackson) until I found out that we had this one already. The slight difference is that our implementation allows the version to be dev.
Happy to submit a PR follow up PR to rename it.

Copy link
Contributor

@jdpgrailsdev jdpgrailsdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@gosusnp gosusnp temporarily deployed to more-secrets September 8, 2022 18:35 Inactive
@gosusnp gosusnp temporarily deployed to more-secrets September 8, 2022 21:06 Inactive
@gosusnp gosusnp temporarily deployed to more-secrets September 8, 2022 22:45 Inactive
@gosusnp gosusnp temporarily deployed to more-secrets September 8, 2022 23:42 Inactive
@gosusnp gosusnp merged commit 9ad847b into master Sep 9, 2022
@gosusnp gosusnp deleted the gosusnp/15978-store_protocol_version_from_spec branch September 9, 2022 01:21
robbinhan pushed a commit to robbinhan/airbyte that referenced this pull request Sep 29, 2022
* Update protocol version from actor defs API operations

* Implement default airbyte protocol version support

* Add version parsing

* Add acceptance tests

* Fix Acceptance Tests

* format

* Make test package private
jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022
* Update protocol version from actor defs API operations

* Implement default airbyte protocol version support

* Add version parsing

* Add acceptance tests

* Fix Acceptance Tests

* format

* Make test package private
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api Related to the api area/documentation Improvements or additions to documentation area/platform issues related to the platform area/server
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Store protocol_version from a SPEC command
3 participants