MINOR: Add draft threat model + SECURITY.md + AGENTS.md for security-model discoverability

[potiuk]

This is a draft proposal for the Kafka PMC to review — please correct,
reject, or discuss as needed. Nothing here is a requirement; the
maintainers are the decision-makers, and this describes Kafka as the
PMC says it is.

This PR adds THREAT_MODEL.md + SECURITY.md + AGENTS.md, wiring
AGENTS.md -> SECURITY.md -> THREAT_MODEL.md.

Framing: Kafka is a configurable platform — it provides mechanisms
(SASL/mTLS auth, an ACL authorizer, TLS, quotas) and the operator
chooses which listeners use them; a broker can run wide open
(PLAINTEXT, no authorizer) or fully locked down. The untrusted network
client is the adversary; the operator and trusted cluster peers /
metadata quorum are out of model.

Draft-first, mostly inferred (~16 documented / 0 maintainer / ~58
inferred); every *(inferred)* claim routes to a numbered §14
question. The wave-1 rulings decide VALID-vs-misconfiguration:

• Is running the default PLAINTEXT listener with no authorizer a
  supported posture (network-trust), so an "unauthenticated broker"
  report against defaults is by-design — or should it be VALID?
• Under StandardAuthorizer, is the default
  allow.everyone.if.no.acl.found "no ACL ⇒ deny"?
• Does the Connect REST API require authentication by default, and
  how should connector-config URL handling (SSRF) be treated?

Scope note: this covers the broker + Connect; Kafka Streams is treated
as a client library (in-app trust), and tools/shell/trogdor/tests are
out of the runtime model.

Context: the ASF Security team is preparing the project for an automated
agentic security scan we're piloting. Drafted via the
threat-model-producer
rubric. If you'd rather author it yourselves, close this PR and we'll
regroup.

Reviewers: Christo Lolov lolovc@amazon.com, Luke Chen
showuon@gmail.com, Mickael Maison mickael.maison@gmail.com

[clolov]

Thanks for opening this PR. I have tried providing answers to the questions I have answers to and pulled in others who I think are better-suited at answering the ones I don't.

[clolov]

My opinion is that a "PLAINTEXT listener and no authorizer" is valid only for development.

[clolov]

The default is DENY (source: https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/metadata/authorizer/StandardAuthorizer.java#L210-L214)

[clolov]

Idempotent producers and transactions are gated by ACLs (source: https://kafka.apache.org/43/security/authorization-and-acls/#operations-and-resources-on-protocols)

Delegation tokens are a mix of both ACLs and additional checks specifically for tokens i.e. something authenticated with a token cannot create another token. For the purposes of this the answer is yes.

[clolov]

SCRAM/GSSAPI/OAUTHBEARER are recommended.

As far as I am aware nothing enforces TLS for PLAIN (sources: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/security/auth/SecurityProtocol.java#L28, https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/security/auth/SecurityProtocol.java#L32). Everything over PLAINTEXT should be development-only.

For OAUTHBEARER we support client_credentials and client_assertion (source: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1258%3A+Add+Support+for+OAuth+Client+Assertion+to+client_credentials+Grant+Type)

[clolov]

Let's trust cluster peers and KRaft quorum for now. Let's remove references to Apache ZooKeeper. Technically versions 3.9.x still have ZooKeeper, but it isn't on trunk.

[clolov]

@mimaison maybe you know some examples? Or @showuon?

[clolov]

Ideally the THREAT_MODEL.md makes it in as a new page under docs/security, but I am happy to move it as a subsequent step if that formatting will somehow break things. I think it is fair to treat Streams as client library for now (@mjsax maybe you can weigh in here?). Given that we expose REST APIs from Connect I have a feeling we need to go in a bit more depth (@mimaison thoughts?).

[clolov]

@mimaison you may be the best-suited person to answer this question? If not maybe you know who might be?

[clolov]

The configurations which protect against a DoS and their default values are:
socket.request.max.bytes at 100 MiB
queued.max.requests at 500
connection.failed.authentication.delay.ms at 100 ms

We further have the following unset by default ones:
queued.max.request.bytes
max.connections{.per.ip}
max.connection.creation.rate

By default quotas are unset. They can be set on a produce/consume level, on a generic request level, or on the number of mutations processable by the controller.

[clolov]

What is addendum C?

[clolov]

What is the reason that an idempotent producer is grouped with the replication internals? Is the idea here that Kafka has some internal state (i.e. for idempotent producer or for replication) which lives on brokers and is not exposed?

[clolov]

I don't know whether you need an exhaustive list or just an list of examples, but let's also exclude committer-tools

[clolov]

And bin. bin contains the .sh files which we use to start a broker, for example, but if we are assuming that this is out of scope due to it being a responsibility of the operator I don't think we need to look into it either

[potiuk]

Thanks @clolov — pushed a revision addressing the review:

• Removed all ZooKeeper references (gone from trunk; legacy 3.9.x out of scope).
• Framed PLAINTEXT listener + no authorizer as development-only; authorizer default = DENY (StandardAuthorizer); idempotent producers / transactions / delegation tokens are ACL-gated (a token can't mint another).
• SASL guidance (SCRAM/GSSAPI/OAUTHBEARER recommended; nothing enforces TLS for PLAIN; OAUTHBEARER client_credentials + client_assertion).
• DoS defaults documented (socket.request.max.bytes=100MiB, queued.max.requests=500, connection.failed.authentication.delay.ms=100ms; queued.max.request.bytes/max.connections*/quotas unset by default).
• Kafka Streams treated as a client library; committer-tools + bin out of scope; defined "addendum C".

Left as open §14 questions where you pinged others: in-cluster-peer threat examples (@mimaison / @showuon), the Streams-as-client-library final call (@mjsax), Connect/REST depth (@mimaison), and whether the out-of-scope list should be exhaustive vs examples. WDYT?

[showuon]

Thanks for the PR. Had a review, but I don't know how this works. Could we try to run with this, and based on the results and then update and run again?

[showuon]

Suggested change

	at `private@kafka.apache.org`). Do **not** open public GitHub issues or pull requests for security reports.
	at `private@kafka.apache.org`). Do **not** open public JIRA issues or pull requests for security reports.

[showuon]

No, I don't remember there's any in-cluster-peer threats. I think it's safe we trust it.

[mimaison]

Thanks for getting the process started. I left a number of comments across the files.

Overall I found the markdown files really hard to read and review. There is a lot of duplication, odd formatting and inconsistent phrasing/grammar. Maybe it does not really matter as it's for LLMs (or maybe it's even by design?).

[mimaison]

Do we require the SECURITY.md and THREAD_MODEL.md files? I'd rather have these sections in our documentation directly, also in markdown in the repository. That way it would available on the website and to end users.

[mimaison]

I'm a bit confused by the sentences after
Agents that scan this repository should consult SECURITY.mdand the linkedTHREAT_MODEL.md before reporting issues..

Is it trying to be a TLDR of the other files?

[mimaison]

The adversary is an **untrusted network client** of a broker (or the Connect REST API);

I'm not sure I understand that sentence.

I agree that the adversary is a client, as in an attack cannot depend on a rogue administrator. Clients only access Kafka components via the network, via Kafka RPCs to brokers and controllers, and HTTP/REST to Connect workers. An attack cannot rely on local access to the brokers, controllers, and Connect workers hosts.

However, I think, an adversary could be a trusted client that breaches its boundaries. For example a client has READ access but manages to perform a WRITE (privilege escalation). I guess it depends what you meant exactly by untrusted.

[mimaison]

tools is pretty much the same as Clients library. These are just clients setup to perform specific tasks

[mimaison]

What do we mean by parses broker responses? Is that responses from the brokers could be malicious and that clients must handle them gracefully?

[mimaison]

Do we really need tho state that if you're allowed to perform an action, being able to perform it is not an issue? Or are we trying to say something else and I don't understand?

[mimaison]

I have no idea what this is referring to?

[mimaison]

I think it's out of scope, brokers and controllers are assumed to be trusted. If a user manages to make a malicious broker/controller join the cluster they must have already breached a number of other security measures.

[mimaison]

As said in https://github.com/apache/kafka/pull/22431/changes#r3356434930, Connect is in a pretty poor state if we don't consider it a client side application.

Regardless we should separate it from the brokers/controllers as it works differently.

[mimaison]

Known issues (not considered security issues) include:

• FileConfigProvider and DirectoryConfigProvider don't handle symlinks in their path allowlist validation logic. While we should fix this issue, I don't consider it a security issue as it requires local disk access and the ability to create arbitrary symlinks on Connect workers.
• https://issues.apache.org/jira/browse/KAFKA-20450: Over the years we got a lot of reports for this one. We tidied the code recently but it's not in a release yet. Again not a security issue as it requires disk write access on Connect workers for exploitation.
• Abuse of sasl.server.max.receive.size: The default value of 512kB could (should?) be reduced.
• Abuse of connections.max.idle.ms pre authentication: We same idle period is applied to both authed and unauthed connections. The config defaults to 10mins, it could be shorter pre-auth.

[mimaison]

I also want to note we, well mostly @clolov, had started work in another PR last week: #22398

Our draft is less comprehensive and much shorter but I find it way easier to read and well organized. So I'm considering whether it would be easier to update it, by lifting a few sections from here, to match what is needed for the scan. I understand it's an option you gave us:

If you'd rather author it yourselves, close this PR and we'll regroup.

But I'd be curious what other PMC members think?

[potiuk]

Thanks @mimaison, @showuon (and @clolov) — genuinely helpful, and the readability point is fair. Since the PMC already has its own draft in #22398 that you find easier to work with, the right move is to defer to that one — it's PMC-authored and PMC-owned, which is exactly where a threat model should live.

Rather than keep two competing PRs open, I'll close this one (#22431) in favor of #22398. Two pieces from here might be worth grafting across, since they're the bits most directly useful when triaging scan output (and easy to under-include in a first pass):

• §11a "known non-findings" — the recurring false-positive suppression list; highest-leverage section for keeping scan noise down.
• §13 triage dispositions — the closed set of outcomes (VALID / BY-DESIGN / OUT-OF-MODEL / …) so every finding routes somewhere.

I'm happy to open a small follow-up against #22398 porting just those two (trimmed to your format) — or leave it entirely to you. And @showuon, on how it's used: the model is the reference you classify each scan finding against (in-scope / out-of-scope / known-non-finding), and it's meant to be iterated as real results arrive — so "run with it and refine" is exactly right.

Closing this in favor of #22398; just say the word if you'd like the §11a/§13 graft.