proposal: rename plugins to components with sub-categories of connectors, tools, and resources #6561
Replies: 3 comments 2 replies
-
Components would essentially be a drop-in replacement for plugins. Airflow, dbt, etc are components of your data stack.
I'm partial to dropping the top-level
Maybe Component Extension SDK? Something along those lines. Not sold on Extension fully though
Funny enough, we already use components in https://docs.meltano.com/concepts/plugins I don't think it's too much to go from this:
to this
|
Beta Was this translation helpful? Give feedback.
-
@tayloramurphy I talked about this in office hours but wanted to note it here as well. I understand why we would want to bucket them for things like the hub, marketing, and generally for making it cleaner to talk about them. As a user though, having to re-organize my project and configurations for that reason is pretty annoying. When it comes to the actual meltano.yml and organizing a project I like the idea of calling them all one thing. As a user it doesnt matter to me if something is a service/tool/resources/connector/component/plugin/etc. I could see them all being under a single top level key, or no key at all. Then on the hub we can group them however we'd like and easily regroup them over time. If we are getting value out of bucketing them, for example if the backend uses its type as a way of determining what functionality to provide (like we do today with EL vs utility plugins) then I wonder if we could add something like
In 6 months we can decide to call them integrations but we dont require every user to re-organize their project and configurations again to support our new names, they just update the component type or leave it and we have backwards compatibility. It's sort of like the pattern we've been using with capabilities, just list what it can do, for a tap meltano should manage catalogs and state but for superset it shouldnt. Also I do have the control to organize them however I'd like using the include_paths so when it comes to the negative effects of a user having a giant blob of plugins/components sitting together, they can split them up like the terraform pattern. What do you think? I know its more of a marketing concern but I wanted to understand how it affects users and projects. |
Beta Was this translation helpful? Give feedback.
-
@aaronsteers wanted to ping you on this since you mentioned the tools vs utilities distinction. In poking around google and other sites, it seems like "tools" is a pretty accepted term for the different parts of a data stack: https://duckduckgo.com/?t=ffab&q=modern+data+stack&ia=web I like the framing of components as the high-level with the 2 main distinctions being connectors and tools. |
Beta Was this translation helpful? Give feedback.
-
Previously discussed in https://github.com/meltano/internal-general/issues/333 (private repo).
Proposal is to have a broad renaming of
plugins
-->components
. A sub-category from there could potentially beconnectors
,tools
andresources
.@aaronsteers proposed a few "frames" which we can use to evaluate how we feel about this change.
plugins:
entry. (Another top-level entry or the new subcategories.)--plugin-type
and in the CLI?Beta Was this translation helpful? Give feedback.
All reactions