Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🚨🚨🎉Source GitLab: migrate to low-code #35989

Merged
merged 37 commits into from Apr 1, 2024
Merged

🚨🚨🎉Source GitLab: migrate to low-code #35989

merged 37 commits into from Apr 1, 2024
+1,194 −1,122

Conversation

askarpets
Copy link
Contributor

@askarpets askarpets commented Mar 12, 2024 •
edited

What

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/6340

How

Migrate to Low-Code

Recommended reading order

  1. source_gitlab/manifest.yaml
  2. source_gitlab/components/partition_routers.py
  3. source_gitlab/source.py

🚨 User Impact 🚨

In this release, a format of stream state has been changed due to migration to low-code.
Also, the primary key was changed for streams group_members, group_labels, project_members, project_labels, branches, and tags.
Users will need to refresh schemas and reset the affected streams after upgrading.

Pre-merge Actions

Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Unit & integration tests added

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • If new credentials are required for use in CI, add them to GSM. Instructions.

@askarpets askarpets self-assigned this Mar 12, 2024
@vercel Vercel
Copy link

vercel bot commented Mar 12, 2024 •
edited

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 1, 2024 2:51pm

@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Mar 12, 2024
askarpets and others added 17 commits March 13, 2024 12:48
@askarpets
Add error handler
bdc4d06
@askarpets
Add custom partition router for group based streams
e8f6b2b
@askarpets
Add project based streams
e71e5fb
@askarpets
Fix partition routing
f2ebaaa
@askarpets
Add transformations
15156a0
@artem1205
Source GitLab: fix partition_routers
c0513f4 
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
@artem1205
Source GitLab: migrate check method
ad4cae3 
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
@artem1205
Source GitLab: migrate check method [skip ci]
3a04bbf 
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
@artem1205
Source GitLab: remove extra auth method [skip ci]
554482c 
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
@askarpets
Update streams list
5272f81
@artem1205
Source GitLab: update oauth [skip ci]
f65a1e6 
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
@askarpets
Add incremental sync
eac52cf
@artem1205
Source GitLab: fix single use refresh token updater
d03f9e5 
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
@askarpets
Merge branch 'master' into source-gitlab-migrate-to-low-code
d22ec4b
@askarpets
Fix transformations, update abnormal_state.json
80f861f
@askarpets
Update version
55d8a4e
@askarpets
Merge branch 'master' into source-gitlab-migrate-to-low-code
1f9dccd
@roman-yermilov-gl
Copy link
Collaborator

LGTM but require unit tests for custom components

maxi297
maxi297 reviewed Mar 18, 2024
View reviewed changes
Copy link
Contributor

@maxi297 maxi297 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you review the comments and make sure those changes are fine with you? There are some comments not related to changes but changes that I would like to tag along as there are breaking and it would be less noisy for users to make them in one go

airbyte-integrations/connectors/source-gitlab/source_gitlab/manifest.yaml Show resolved Hide resolved
airbyte-integrations/connectors/source-gitlab/source_gitlab/manifest.yaml Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-gitlab/source_gitlab/manifest.yaml Show resolved Hide resolved
airbyte-integrations/connectors/source-gitlab/source_gitlab/manifest.yaml Show resolved Hide resolved
airbyte-integrations/connectors/source-gitlab/source_gitlab/manifest.yaml Show resolved Hide resolved
airbyte-integrations/connectors/source-gitlab/source_gitlab/components/partition_routers.py Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-gitlab/source_gitlab/manifest.yaml Show resolved Hide resolved
airbyte-integrations/connectors/source-gitlab/metadata.yaml Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-gitlab/metadata.yaml Outdated Show resolved Hide resolved
airbyte-integrations/connectors/source-gitlab/metadata.yaml Outdated Show resolved Hide resolved
@octavia-squidington-iv octavia-squidington-iv requested a review from a team March 19, 2024 09:15
maxi297
maxi297 approved these changes Mar 19, 2024
View reviewed changes
Copy link
Contributor

@maxi297 maxi297 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments! I'm good with this one

bazarnov
bazarnov approved these changes Mar 19, 2024
View reviewed changes
Copy link
Collaborator

@bazarnov bazarnov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

@askarpets askarpets added the low-code-migration This connector has been migrated to the low-code CDK label Mar 22, 2024
brianjlai
brianjlai reviewed Mar 28, 2024
View reviewed changes
Copy link
Contributor

@brianjlai brianjlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small reminder. Before merging, please re-run poetry lock to upgrade to the latest version of airbyte-cdk. Thanks!

@octavia-squidington-iv octavia-squidington-iv requested a review from a team April 1, 2024 14:34
@octavia-squidington-iv octavia-squidington-iv requested a review from a team April 1, 2024 14:38
@askarpets askarpets merged commit 5f2c3d8 into master Apr 1, 2024
33 of 35 checks passed
@askarpets askarpets deleted the source-gitlab-migrate-to-low-code branch April 1, 2024 16:53
nurikk pushed a commit to nurikk/airbyte that referenced this pull request Apr 4, 2024
@askarpets @artem1205 @nurikk-sa
🚨🚨🎉Source GitLab: migrate to low-code (airbytehq#35989)
889f495 
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
Co-authored-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
markcusack pushed a commit to markcusack/airbyte that referenced this pull request Apr 9, 2024
@askarpets @artem1205
🚨🚨🎉Source GitLab: migrate to low-code (airbytehq#35989)
fbb7ae8 
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
Co-authored-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
@hugovandernotte
Copy link

hugovandernotte commented Apr 19, 2024 •
edited

@askarpets @maxi297 @katmarkham I think this merge led to this issue: #37436

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brianjlai brianjlai brianjlai left review comments

@maxi297 maxi297 maxi297 approved these changes

@bazarnov bazarnov bazarnov approved these changes

@katmarkham katmarkham katmarkham approved these changes

@lazebnyi lazebnyi lazebnyi approved these changes

@roman-yermilov-gl roman-yermilov-gl roman-yermilov-gl approved these changes

@artem1205 artem1205 Awaiting requested review from artem1205

Assignees

@askarpets askarpets

Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/gitlab low-code-migration This connector has been migrated to the low-code CDK
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
10 participants
@askarpets @roman-yermilov-gl @hugovandernotte @maxi297 @bazarnov @katmarkham @brianjlai @lazebnyi @octavia-squidington-iii @artem1205