New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make Transform an ExtensionPoint #9319
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the suggestion and contribution!
@@ -46,6 +46,7 @@ Druid's extensions leverage Guice in order to add things at runtime. Basically, | |||
1. Add new Jersey resources by calling `Jerseys.addResource(binder, clazz)`. | |||
1. Add new Jetty filters by extending `org.apache.druid.server.initialization.jetty.ServletFilterHolder`. | |||
1. Add new secret providers by extending `org.apache.druid.metadata.PasswordProvider`. | |||
1. Add new ingest transform by implementing the `org.apache.druid.segment.transform.Transform` interface from the `druid-processing` package. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a section in here describing how to register the custom transformation similar to what's documented for adding support for an Input Source (https://github.com/apache/druid/blob/118e89c1e2ae9ea0e1c923c355bff6f1ccacf756/docs/development/modules.md#adding-support-for-a-new-input-source) or a Password Provider (https://github.com/apache/druid/blob/118e89c1e2ae9ea0e1c923c355bff6f1ccacf756/docs/development/modules.md#adding-a-new-password-provider-implementation)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It makes sense to me to add a new item in the list of 15 items under "Writing your own extensions" for completeness sake. I'd be happy to write a dedicated section of these docs, although I think this page seems destined to be split into many pages: one describing the approach for each kind of extension.
Hi @mitchlloyd, I assume you are interested in writing docs what @suneet-s mentioned based on your comment above. Do you want to add them in this PR? I think that could be done in a follow-up PR if you want.
I’m happy to take at shot at more documentation in this PR. I should have a
chance to revisit this week.
…On Mon, Feb 10, 2020 at 2:40 PM Jihoon Son ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In docs/development/modules.md
<#9319 (comment)>:
> @@ -46,6 +46,7 @@ Druid's extensions leverage Guice in order to add things at runtime. Basically,
1. Add new Jersey resources by calling `Jerseys.addResource(binder, clazz)`.
1. Add new Jetty filters by extending `org.apache.druid.server.initialization.jetty.ServletFilterHolder`.
1. Add new secret providers by extending `org.apache.druid.metadata.PasswordProvider`.
+1. Add new ingest transform by implementing the `org.apache.druid.segment.transform.Transform` interface from the `druid-processing` package.
It makes sense to me to add a new item in the list of 15 items under
"Writing your own extensions" for completeness sake. I'd be happy to write
a dedicated section of these docs, although I think this page seems
destined to be split into many pages: one describing the approach for each
kind of extension.
Hi @mitchlloyd <https://github.com/mitchlloyd>, I assume you are
interested in writing docs what @suneet-s <https://github.com/suneet-s>
mentioned based on your comment above. Do you want to add them in this PR?
I think that could be done in a follow-up PR if you want.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#9319?email_source=notifications&email_token=AAADWQPAJMXRT47DZ4NL4XLRCHJUZA5CNFSM4KQXQN52YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCU6PYFQ#discussion_r377360313>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAADWQIRNDWSRSEOEWZEB7TRCHJUZANCNFSM4KQXQN5Q>
.
|
@mitchlloyd great, thanks! |
@suneet-s, @jihoonson I added an example implementation for a Transform extension. I realize this is a little more verbose than the other sections, but the full implementation with imports would have helped me out. Happy to trim it down by removing imports, class declarations or anything else. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thank you @mitchlloyd!
@suneet-s do you have more comments? |
lgtm ! |
Fixes #9311.
Description
Transform
is a useful place for 3rd party developers to add their own extensions. Informally this has been an available extension point since 2017. This PR proposes adding the compatibility guarantees of @ExtensionPoint.This PR has:
ToDo:
It makes sense to me to add a new item in the list of 15 items under "Writing your own extensions" for completeness sake. I'd be happy to write a dedicated section of these docs, although I think this page seems destined to be split into many pages: one describing the approach for each kind of extension.