inits Deployment contrib #2498

mhausenblas · 2023-03-13T18:29:49Z

Addresses #1696

Original draft PR (for context), see #2312

Preview: https://deploy-preview-2498--opentelemetry.netlify.app/docs/collector/deployment/

dmitryax

Are we introducing new deployment concepts or replacing Agent/Gateway with Decentralized/Centralized as 1:1? It doesn't seem to be 1:1 based on the docs:

Agent is supposed to represent an installation on one host so that instrumentation libraries can point to local endpoints like http://localhost:4318. The decentralized doc says collector.example.com:4318 instead. Also, the decentralized section mentions "Clear 1:1 mapping between application and collector" in "Pros" section, which is right for the agent term as well, but it confuses me when I read the first paragraph that seems to contradict:

The decentralized collector deployment pattern consists of applications—instrumented with an OpenTelemetry SDK using OpenTelemetry protocol (OTLP)—or other collectors (using the OTLP exporter) that send telemetry signals to one or more collectors.

Several Collector pull-based receivers are intended to run in the agent (decentralized?) mode, for example, hostmetrics receiver. If we are extending the documentation, I believe it's worth mentioning.

In general, I don't fully agree that decentralized/centralized terms are easier to understand than agent/gateway. I added this topic to the next Collector SIG meeting's agenda https://docs.google.com/document/d/1r2JC5MB7GupCE7N32EwGEXs9V_YIsPgoFiLP4VWVMkE/edit

smithclay · 2023-03-14T19:00:23Z

@dmitryax Whatever term is used, we should add/link/update the glossary and include the other terminology that is used to refer to the pattern.

TylerHelmuth

I love this content. I also vote not to change to Centralized and Decentralized. Personally I am good with Agent and Gateway, although during this SIG today it sounds like some would opt to change only Gateway to something else.

mhausenblas · 2023-03-15T18:30:40Z

Thanks for the feedback @TylerHelmuth and I will implement whatever the consensus is. It sounds to me there is no clear preference yet either way?

cartermp · 2023-03-15T18:42:04Z

I would be in favor of merging this as-is if there's no strong consensus. We can update it later when there is consensus. It's far more valuable to have these docs in now than to hold it up over two words :)

TylerHelmuth · 2023-03-15T20:08:51Z

It sounds to me there is no clear preference yet either way?

There was definitely consensus on Agent. @jpkrohling @dmitryax dissented on Gateway.

Seeing as Gateway is currently the term used on the site and seems to be the most prevalent term, if there is pressure to get this doc through soon, I would say using Agent and Gateway at the moment is proper. In the future, if Collector maintainers want to switch from Gateway to something else thats fine, but at least for this PR we would not be introducing any new terms. I would like to avoid switching from Agent and Gateway to Centralized and Decentralized only to switch again to Agent and some new term. Better in my mind to keep Agent and Gateway for now and switch Gateway later.

As always, nothing is every as exciting and as engaging as naming stuff lol

@open-telemetry/collector-approvers @open-telemetry/collector-contrib-approvers

mhausenblas · 2023-03-16T06:57:49Z

Thanks for the clarification @TylerHelmuth

In the future, if Collector maintainers want to switch from Gateway to something else thats fine

Are you suggesting that the collector maintainers are the only ones who can and should decide terminology?

codeboten

Are you suggesting that the collector maintainers are the only ones who can and should decide terminology?

I don't think maintainers should be the only ones who decide on terminology. I do think it's important that new terminology be introduced carefully, especially when there is already terminology defined to address those concepts.

I agree that this documentation is a significant improvement, and I'd prefer not to get stuck in naming limbo. Maybe a vote on this PR for each term would allow us to move forward with it?

content/en/docs/collector/deployment/_index.md

content/en/docs/collector/deployment/best-practices.md

content/en/docs/collector/deployment/no-collector.md

mhausenblas · 2023-03-16T16:28:32Z

Thanks for your feedback @codeboten!

Concerning:

I do think it's important that new terminology be introduced carefully, especially when there is already terminology defined to address those concepts.

Just to clarify, it's not that I'm pulling things out of the thin air, right? :)
I was following @jpkrohling input and suggestion from #1368 (comment). I just mention that here to make it clear: I don't have a dog in this race, I just want to make sure that we ship stuff, ASAP.

I agree that this documentation is a significant improvement, and I'd prefer not to get stuck in naming limbo.

+1

Maybe a vote on this PR for each term would allow us to move forward with it?

Might not be necessary. Unless I hear a good argument why we should introduce the new terminology from supportive folks such as @jpkrohling I will undo the changes and we keep the old terminology. Seems OpenTelemetry is getting slowly big enough for these kind of coordination issues … we really need a SIG Release like Kubernetes has ha ha

codeboten · 2023-03-16T20:45:45Z

I just mention that here to make it clear: I don't have a dog in this race, I just want to make sure that we ship stuff, ASAP

100%. My only concern is around not creating confusion in the process :D When this PR was discussed in the collector SIG yesterday there was a lot of confusion in the discussion :)

Seems OpenTelemetry is getting slowly big enough for these kind of coordination issues

Yes... I'd say at least for terminology, maybe the coordination point could be the glossary in the specification? Which I sadly just realized differs from the glossary in the documentation :(

dmitryax · 2023-03-16T21:02:02Z

@jpkrohling mentioned during the SIG that he didn't suggest Centralized/Decentralized. He doesn't agree with the Gateway term. I am up for improving the documentation but I don't think we should introduce new concepts that collector maintainers don't agree with. The new concepts should be used in the codebase, not only in the docs.

I agree with @TylerHelmuth. I think we can improve the documentation keeping the existing terminology, and decide on a Gateway replacement as a separate effort.

cartermp · 2023-03-16T21:07:29Z

Let's say we just change the names in this PR to Agent and Gateway (even if we're not 100% happy with that existing term) - is that sufficient to get this merged?

TylerHelmuth · 2023-03-16T21:13:36Z

Let's say we just change the names in this PR to Agent and Gateway (even if we're not 100% happy with that existing term) - is that sufficient to get this merged?

Yes, this is what I was proposing. 👍 from me.

cartermp · 2023-03-16T21:16:12Z

Alrighty, let's consider that the plan for now. @mhausenblas could you make the terminology change? Then I think this is pretty close to getting merged. Collector folks can orthogonally decide on what the "right" terminology is and the content can be updated accordingly. But for now, there's too much good stuff in here already to let it go unmerged 🙂

chalin

Hi. There should be no change to themes/docsy in this PR. You probably need to rebase then rebuild to fully sync with the latest Docsy.

content/en/docs/collector/deployment/best-practices.md

mhausenblas · 2023-03-17T10:31:42Z

Thanks for all the feedback, will address and wrap up on Mon Mar 20 (today's St Patrick's day here in Ireland and … well ;)

You probably need to rebase then rebuild to fully sync with the latest Docsy.

@chalin before I f0rk up again, what exactly does "rebuild" mean? Can I ping you on Monday?

chalin · 2023-03-17T11:44:41Z

@chalin before I f0rk up again, what exactly does "rebuild" mean? Can I ping you on Monday?

@mhausenblas - yes certainly, though I can clarify here (sorry, could have been clearer the first time 🤷🏼‍♂️): generally after a rebase, git submodules might not be in sync. By "rebuild" I meant re-run npm run build, which in turn runs npm run get:submodule to sync to submodules. I hope that helps, if not DM me (though I'll probably be OOO part of next week).

🍀🍀 Happy St. Paddy's Day! 🍀🍀

content/en/docs/collector/deployment/best-practices.md

svrnm · 2023-03-22T18:20:48Z

Just looking through that PR. I would love to have that in as soon as possible, I am just wondering if we can find a way to simplify the PR and have some follow-up PRs later down to get it closed a little bit quicker.

Just a few suggestions:

Pros & Cons go into their own PR
Best Practices go into their own PR

WDYT?

chalin · 2023-03-29T15:41:54Z

@mhausenblas - if you prefer, I can fix the Docsy issue. Let me know.

mhausenblas · 2023-03-29T15:57:09Z

@chalin thanks, that would be lovely! I plan to wrap up tomorrow …

Co-authored-by: Alex Boten <alex@boten.ca>

Co-authored-by: Patrice Chalin <chalin@users.noreply.github.com>

Co-authored-by: Severin Neumann <severin.neumann@altmuehlnet.de>

Co-authored-by: Patrice Chalin <chalin@users.noreply.github.com>

chalin · 2023-05-10T17:06:51Z

Rebased and checks are passing (even though I can see only one of the following two checks in the "checks summary" above):

jpkrohling

I'm fine merging this as is if you promise to expand on the cons for the agent pattern as part of a follow-up PR :-)

content/en/docs/collector/deployment/agent.md

jpkrohling · 2023-05-10T18:06:19Z

content/en/docs/collector/deployment/agent.md

+Cons:
+
+- Scalability (human and load-wise)
+- Inflexible


Why? Reading this myself, I can't tell exactly what you had in mind.

jpkrohling · 2023-05-10T18:07:41Z

content/en/docs/collector/deployment/agent.md

+
+Cons:
+
+- Scalability (human and load-wise)


Why? What are the gotchas regarding scalability on this pattern? You could say that sidecars could become an unnecessary overhead and that daemon sets are harder to scale than deployments. You need to expand a bit instead of just stating "Scalability" :-)

Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de>

mhausenblas · 2023-05-10T18:17:03Z

@jpkrohling deal! Added the ask to my list of things that need to be addressed for the follow-up PR.

Other reviews supersede, changes addressed

cartermp · 2023-05-10T18:24:52Z

And we're doin' it.

cartermp · 2023-05-10T18:25:17Z

@mhausenblas can you file an issue on the repo here with a link to @jpkrohling's feedback to address as a follow-up?

mhausenblas · 2023-05-10T18:31:07Z

Thanks @cartermp and see #2692

mhausenblas requested review from a team as code owners March 13, 2023 18:29

mhausenblas requested review from Aneurysm9 and removed request for a team March 13, 2023 18:29

mhausenblas force-pushed the col-deploy branch from be11fa0 to 3a189be Compare March 13, 2023 18:54

dmitryax previously requested changes Mar 14, 2023

View reviewed changes

TylerHelmuth reviewed Mar 15, 2023

View reviewed changes

codeboten reviewed Mar 16, 2023

View reviewed changes

chalin requested changes Mar 16, 2023

View reviewed changes

mx-psi reviewed Mar 17, 2023

View reviewed changes

content/en/docs/collector/deployment/best-practices.md Outdated Show resolved Hide resolved

chalin force-pushed the col-deploy branch from ed9dc3f to 134f8d7 Compare March 17, 2023 11:44

svrnm reviewed Mar 22, 2023

View reviewed changes

content/en/docs/collector/deployment/best-practices.md Outdated Show resolved Hide resolved

svrnm reviewed Mar 22, 2023

View reviewed changes

content/en/docs/collector/deployment/best-practices.md Outdated Show resolved Hide resolved

chalin force-pushed the col-deploy branch from c513dc0 to 3d7961e Compare March 29, 2023 15:39

mhausenblas and others added 12 commits May 10, 2023 12:57

updates deployment patterns

678f232

Sync Docsy

101ffb1

Simplification of tabpane arguments

9cdb6f7

changes

007a1d2

Co-authored-by: Alex Boten <alex@boten.ca>

fixes overview resources

9fe2121

Co-authored-by: Patrice Chalin <chalin@users.noreply.github.com>

addresses feedback

76b2992

addresses feedback

fd2aae7

fixes broken link from blog post to gateway

ecb3fde

applies pretier

3884f27

Update content/en/docs/collector/deployment/agent.md

47b29e5

Co-authored-by: Severin Neumann <severin.neumann@altmuehlnet.de>

Update content/en/docs/collector/deployment/gateway.md

af991ee

Co-authored-by: Patrice Chalin <chalin@users.noreply.github.com>

addresses feedback

199c979

chalin force-pushed the col-deploy branch from a12bda1 to 199c979 Compare May 10, 2023 16:58

mhausenblas and others added 2 commits May 10, 2023 18:24

Merge branch 'main' into col-deploy

04e6239

Merge branch 'main' into col-deploy

00d882c

jpkrohling approved these changes May 10, 2023

View reviewed changes

Update content/en/docs/collector/deployment/agent.md

d830b04

Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de>

fixes formatting

2e0f876

cartermp merged commit d98c705 into open-telemetry:main May 10, 2023
8 checks passed

mhausenblas mentioned this pull request May 10, 2023

Follow up on collector deployment #2692

Open

mhausenblas deleted the col-deploy branch May 11, 2023 07:45

svrnm mentioned this pull request Jan 18, 2024

Collector "Deployment" Rewrite #1696

Closed

snyk-io bot mentioned this pull request Mar 29, 2024

[Snyk] Security upgrade gulp from 4.0.2 to 5.0.0 Cachiman/opentelemetry.io#8

Open

Cachiman mentioned this pull request May 13, 2024

[Snyk] Security upgrade gulp from 4.0.2 to 5.0.0 Cachiman/opentelemetry.io#9

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

inits Deployment contrib #2498

inits Deployment contrib #2498

mhausenblas commented Mar 13, 2023 •

edited

dmitryax left a comment

smithclay commented Mar 14, 2023 •

edited

TylerHelmuth left a comment

mhausenblas commented Mar 15, 2023

cartermp commented Mar 15, 2023

TylerHelmuth commented Mar 15, 2023

mhausenblas commented Mar 16, 2023

codeboten left a comment

mhausenblas commented Mar 16, 2023

codeboten commented Mar 16, 2023

dmitryax commented Mar 16, 2023

cartermp commented Mar 16, 2023

TylerHelmuth commented Mar 16, 2023

cartermp commented Mar 16, 2023

chalin left a comment

mhausenblas commented Mar 17, 2023

chalin commented Mar 17, 2023

svrnm commented Mar 22, 2023

chalin commented Mar 29, 2023

mhausenblas commented Mar 29, 2023

chalin commented May 10, 2023 •

edited

jpkrohling left a comment

jpkrohling May 10, 2023

jpkrohling May 10, 2023

mhausenblas commented May 10, 2023

cartermp commented May 10, 2023

cartermp commented May 10, 2023

mhausenblas commented May 10, 2023

inits Deployment contrib #2498

inits Deployment contrib #2498

Conversation

mhausenblas commented Mar 13, 2023 • edited

dmitryax left a comment

Choose a reason for hiding this comment

smithclay commented Mar 14, 2023 • edited

TylerHelmuth left a comment

Choose a reason for hiding this comment

mhausenblas commented Mar 15, 2023

cartermp commented Mar 15, 2023

TylerHelmuth commented Mar 15, 2023

mhausenblas commented Mar 16, 2023

codeboten left a comment

Choose a reason for hiding this comment

mhausenblas commented Mar 16, 2023

codeboten commented Mar 16, 2023

dmitryax commented Mar 16, 2023

cartermp commented Mar 16, 2023

TylerHelmuth commented Mar 16, 2023

cartermp commented Mar 16, 2023

chalin left a comment

Choose a reason for hiding this comment

mhausenblas commented Mar 17, 2023

chalin commented Mar 17, 2023

svrnm commented Mar 22, 2023

chalin commented Mar 29, 2023

mhausenblas commented Mar 29, 2023

chalin commented May 10, 2023 • edited

jpkrohling left a comment

Choose a reason for hiding this comment

jpkrohling May 10, 2023

Choose a reason for hiding this comment

jpkrohling May 10, 2023

Choose a reason for hiding this comment

mhausenblas commented May 10, 2023

cartermp commented May 10, 2023

cartermp commented May 10, 2023

mhausenblas commented May 10, 2023

mhausenblas commented Mar 13, 2023 •

edited

smithclay commented Mar 14, 2023 •

edited

chalin commented May 10, 2023 •

edited