Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

robots.txt is wrong #14836

Closed
MathiasReker opened this issue Jul 23, 2019 · 3 comments · Fixed by #26246
Closed

robots.txt is wrong #14836

MathiasReker opened this issue Jul 23, 2019 · 3 comments · Fixed by #26246
Labels
1.7.5.2 Affects versions 1.7.6.0 Affects versions Bug Type: Bug PR available Solution: issue is being addressed SEO & URLs Label: Which BO under menu is concerned Trivial Severity: trivial bug > https://build.prestashop.com/news/severity-classification
Milestone

Comments

@MathiasReker
Copy link
Contributor

MathiasReker commented Jul 23, 2019

Describe the bug
Robots.txt file is generated for wrong dirs.

To Reproduce
Steps to reproduce the behavior:
0. Install Danish language on a clean install (no second language). Try to install PrestaShop in a folder instead of in the root (I did not test in the root).

  1. Go to configure -> shop -> seo & url -> generate robots.txt
  2. See robots.txt Should be similair to below:
    Please note that there is generated some URLs with language code /da/ but I do only have one language installed, so I don't have a language code in my URL.
# robots.txt automatically generated by PrestaShop e-commerce open-source solution
# http://www.prestashop.com - http://www.prestashop.com/forums
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/robotstxt.html
User-agent: *
# Allow Directives
Allow: */modules/*.css
Allow: */modules/*.js
Allow: */modules/*.png
Allow: */modules/*.jpg
Allow: /js/jquery/*
# Private pages
Disallow: /*?order=
Disallow: /*?tag=
Disallow: /*?id_currency=
Disallow: /*?search_query=
Disallow: /*?back=
Disallow: /*?n=
Disallow: /*&order=
Disallow: /*&tag=
Disallow: /*&id_currency=
Disallow: /*&search_query=
Disallow: /*&back=
Disallow: /*&n=
Disallow: /*controller=addresses
Disallow: /*controller=address
Disallow: /*controller=authentication
Disallow: /*controller=cart
Disallow: /*controller=discount
Disallow: /*controller=footer
Disallow: /*controller=get-file
Disallow: /*controller=header
Disallow: /*controller=history
Disallow: /*controller=identity
Disallow: /*controller=images.inc
Disallow: /*controller=init
Disallow: /*controller=my-account
Disallow: /*controller=order
Disallow: /*controller=order-slip
Disallow: /*controller=order-detail
Disallow: /*controller=order-follow
Disallow: /*controller=order-return
Disallow: /*controller=order-confirmation
Disallow: /*controller=pagination
Disallow: /*controller=password
Disallow: /*controller=pdf-invoice
Disallow: /*controller=pdf-order-return
Disallow: /*controller=pdf-order-slip
Disallow: /*controller=product-sort
Disallow: /*controller=search
Disallow: /*controller=statistics
Disallow: /*controller=attachment
Disallow: /*controller=guest-tracking
# Directories for localhost
Disallow: /prestashop_WH/app/
Disallow: /prestashop_WH/cache/
Disallow: /prestashop_WH/classes/
Disallow: /prestashop_WH/config/
Disallow: /prestashop_WH/controllers/
Disallow: /prestashop_WH/download/
Disallow: /prestashop_WH/js/
Disallow: /prestashop_WH/localization/
Disallow: /prestashop_WH/log/
Disallow: /prestashop_WH/mails/
Disallow: /prestashop_WH/modules/
Disallow: /prestashop_WH/override/
Disallow: /prestashop_WH/pdf/
Disallow: /prestashop_WH/src/
Disallow: /prestashop_WH/tools/
Disallow: /prestashop_WH/translations/
Disallow: /prestashop_WH/upload/
Disallow: /prestashop_WH/var/
Disallow: /prestashop_WH/vendor/
Disallow: /prestashop_WH/webservice/
Disallow: /da/app/
Disallow: /da/cache/
Disallow: /da/classes/
Disallow: /da/config/
Disallow: /da/controllers/
Disallow: /da/download/
Disallow: /da/js/
Disallow: /da/localization/
Disallow: /da/log/
Disallow: /da/mails/
Disallow: /da/modules/
Disallow: /da/override/
Disallow: /da/pdf/
Disallow: /da/src/
Disallow: /da/tools/
Disallow: /da/translations/
Disallow: /da/upload/
Disallow: /da/var/
Disallow: /da/vendor/
Disallow: /da/webservice/
# Files
Disallow: /*da/gendan-adgangskode
Disallow: /*da/adresse
Disallow: /*da/adresser
Disallow: /*da/log-ind
Disallow: /*da/kurv
Disallow: /*da/rabat
Disallow: /*da/ordrehistorik
Disallow: /*da/mine-oplysninger
Disallow: /*da/min-konto
Disallow: /*da/foelg-din-bestilling
Disallow: /*da/kreditnota
Disallow: /*da/ordre
Disallow: /*da/soeg
Disallow: /*da/Gaestesporing
Disallow: /*da/ordrebekraeftelse

Additional information
PrestaShop version: 1.7.6.0
PHP version: 7.2.x
screen

@khouloudbelguith
Copy link
Contributor

Hi @MathiasReker,

I have the same issue with PS1.7.6.0.
image
I used a virtual host.
I have the same issue after installing PS1.7.6.0 with France language & also with PS1.7.5.2.
image

I’ll add this to the debug roadmap so that it’s fixed. If you have already fixed it on your end or if you think you can do it, please do send us a pull request!
Thanks!

@khouloudbelguith khouloudbelguith added 1.7.5.2 Affects versions 1.7.6.0 Affects versions Bug Type: Bug SEO & URLs Label: Which BO under menu is concerned Trivial Severity: trivial bug > https://build.prestashop.com/news/severity-classification Ready Status: Issue is ready to be worked on labels Jul 24, 2019
@khouloudbelguith
Copy link
Contributor

Related to #9657

@jonpontet
Copy link

Be great to get this fixed guys - we too have the same issue with our FR store (which has just the 1 language!)

I have to edit manually robots.txt - if I press the button to re-create, everything is lost !

@prestonBot prestonBot added the PR available Solution: issue is being addressed label Oct 7, 2021
@prestashop-issue-bot prestashop-issue-bot bot removed the Ready Status: Issue is ready to be worked on label Feb 8, 2022
@Progi1984 Progi1984 added this to the 8.0.0 milestone Feb 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1.7.5.2 Affects versions 1.7.6.0 Affects versions Bug Type: Bug PR available Solution: issue is being addressed SEO & URLs Label: Which BO under menu is concerned Trivial Severity: trivial bug > https://build.prestashop.com/news/severity-classification
Projects
None yet
5 participants