Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Fix wrong rules in robots.txt #9462
Until now, robots.txt was blocking not only directories at the root, but also directories named the same in any subdirectory.
This has been changed in order to avoid accidentally blocking assets like CSS and JS, for example.
Non existent directories like "/css" and "/web" have been removed, while missing directories like "/app", "/var" have been added.