Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/kafka schema registry integration #52

Closed
wants to merge 13 commits into from
Closed

Feature/kafka schema registry integration #52

wants to merge 13 commits into from

Conversation

farbodahm
Copy link
Collaborator

@farbodahm farbodahm commented Aug 15, 2022 •
edited
Loading

Summary of Changes

Added an extractor for KafkaSchemaRegistry schemas.

Tests

Added databuilder/tests/unit/extractor/test_kafka_schema_registry_extractor.py for testing the new extractor.

Documentation

Updated README.md and databuilder/README.md to reflect the addition of the new extractor.

CheckList

Make sure you have checked all steps below to ensure a timely review.

  • PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"
  • PR includes a summary of changes.
  • PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does

@farbodahm farbodahm force-pushed the feat/kafka-schema-registry-integration branch from 00e20cb to 0ef889c Compare August 15, 2022 10:47
@farbodahm farbodahm marked this pull request as ready for review August 15, 2022 12:23
@farbodahm farbodahm changed the title Feat/kafka schema registry integration Feature/kafka schema registry integration Aug 15, 2022
@farbodahm farbodahm force-pushed the feat/kafka-schema-registry-integration branch 5 times, most recently from 0084d2b to 004cd6f Compare August 16, 2022 06:43
@farbodahm
feat: add base of integrating Kafka schema registry to Amundsen
625b9d4 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
fix: get schema registry password from config file and not from Secre…
f8bb0b1 
…tsManager

Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
fix: change staticmethod functions related to getting schemas to clas…
a14aa3b 
…s functions

Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
refactor: move exception handling to a higher layer
6c9ab61 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
feat: add create table functionality from given schema
d7bf00b 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
feat: add main extracting functionality
8c61c26 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
feat: add authentication to schema registry
46f7982 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
fix: pass columns to table
d8a3450 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
fix: cluster name and columns descriptions
d35e6ae 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
fix: return None instead of empty string for descriptions
37c7ab0 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
feat: add tests for KafkaSchemaRegistry databuilder
aa7a34d 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm
fix: Pep8
ab22098 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm farbodahm force-pushed the feat/kafka-schema-registry-integration branch from 004cd6f to ab22098 Compare August 17, 2022 06:23
@farbodahm farbodahm force-pushed the feat/kafka-schema-registry-integration branch 3 times, most recently from 313ca49 to c602b70 Compare August 17, 2022 14:00
@farbodahm
feat: use SchemaRegistryClient instead of requests
6021ffa 
Signed-off-by: Farbod Ahmadian <farbodahmadian2014@gmail.com>
@farbodahm farbodahm force-pushed the feat/kafka-schema-registry-integration branch from c602b70 to 6021ffa Compare August 18, 2022 06:02
@farbodahm
Copy link
Collaborator Author

I'm closing this because it got merged to upstream in this pull request.

@farbodahm farbodahm closed this Aug 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@farbodahm