Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Default robots.txt for media repo #6533

Closed
lub opened this issue Dec 12, 2019 · 2 comments
Closed

Default robots.txt for media repo #6533

lub opened this issue Dec 12, 2019 · 2 comments
Labels
A-Media-Repository Uploading, downloading images and video, thumbnailing z-p3 (Deprecated Label)

Comments

@lub
Copy link
Contributor

lub commented Dec 12, 2019

Hi,

maybe it makes sense to add a default robots.txt. Otherwise depending on your configuration you could use the SEO of a domain by getting media repository content indexed. For example on matrix.org:

image
@jryans jryans transferred this issue from matrix-org/synapse Dec 12, 2019
@jryans
Copy link
Contributor

jryans commented Dec 12, 2019

Oh hmm, I assumed this was about the matrix.org blog from your search.

@jryans jryans transferred this issue from matrix-org/matrix.org Dec 12, 2019
@jryans jryans changed the title default robots.txt Default robots.txt for media repo Dec 12, 2019
@neilisfragile neilisfragile added A-Media-Repository Uploading, downloading images and video, thumbnailing z-p3 (Deprecated Label) labels Dec 17, 2019
@aaronraimist aaronraimist mentioned this issue Dec 6, 2020
Merged
4 tasks
anoadragon453 pushed a commit that referenced this issue Dec 8, 2020
@aaronraimist
Add X-Robots-Tag header to stop crawlers from indexing media (#8887)
cd9e72b 
Fixes / related to: #6533

This should do essentially the same thing as a robots.txt file telling robots to not index the media repo. https://developers.google.com/search/reference/robots_meta_tag

Signed-off-by: Aaron Raimist <aaron@raim.ist>
@lub
Copy link
Contributor Author

lub commented Jan 28, 2021

Fixed by #8887

@lub lub closed this as completed Jan 28, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
A-Media-Repository Uploading, downloading images and video, thumbnailing z-p3 (Deprecated Label)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jryans @neilisfragile @lub