core(config): implement llms.txt check for AI agents#16970
Merged
TravenReese merged 1 commit intomainfrom Apr 20, 2026
Merged
Conversation
TravenReese
force-pushed
the
agentic-browsing-llmtxt
branch
from
4f06232 to
d8f71e8
Compare
TravenReese
force-pushed
the
agentic-browsing-llmtxt
branch
from
d8f71e8 to
bb045ab
Compare
TravenReese
force-pushed
the
agentic-browsing-llmtxt
branch
from
bb045ab to
aad09c6
Compare
TravenReese
requested review from
lusayaa and
paulirish
and removed request for
lusayaa
paulirish
reviewed
Apr 17, 2026
paulirish
approved these changes
Apr 17, 2026
Member
paulirish
left a comment
paulirish
left a comment
There was a problem hiding this comment.
nice!
gatherer and audit all in one. looks good.
left a bunch of thoughts on this one. none technical. :)
TravenReese
force-pushed
the
agentic-browsing-llmtxt
branch
from
aad09c6 to
8ca9988
Compare
TravenReese
force-pushed
the
agentic-browsing-llmtxt
branch
from
8ca9988 to
ccc389e
Compare
TravenReese
force-pushed
the
agentic-browsing-llmtxt
branch
from
ccc389e to
bc02568
Compare
TravenReese
force-pushed
the
agentic-browsing-llmtxt
branch
from
bc02568 to
ec57075
Compare
TravenReese
force-pushed
the
agentic-browsing-llmtxt
branch
from
ec57075 to
a7a379a
Compare
lusayaa
reviewed
Apr 17, 2026
Collaborator
There was a problem hiding this comment.
I didn't realize I never sent this...whoops
also it looks like you're also getting a bunch of localization changes...idk where they come from tbh
edit: so it came from ME. my bad! created a revert. you probbaly need to pull in these two (hopefully now) net-zero change commits first before pushing.
TravenReese
force-pushed
the
agentic-browsing-llmtxt
branch
from
3d92942 to
6f35060
Compare
TravenReese
force-pushed
the
agentic-browsing-llmtxt
branch
from
6f35060 to
48bf2d9
Compare
lusayaa
approved these changes
Apr 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a new gatherer and audit to check for a valid
llms.txtMarkdown file on the site, and integrates it into the agentic browsing config.