Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

DEVELOPER-828 enable robots.txt for prod #403

Merged
merged 1 commit into from

3 participants

Pete Muir jboss-developer Paul Robinson
Pete Muir
Owner

No description provided.

jboss-developer
Collaborator

Triggering build using a merge of 71a2e0e on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/

jboss-developer
Collaborator

Build 419 is now running using a merge of 71a2e0e on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/419

Paul Robinson paulrobinson commented on the diff
robots.txt
@@ -1,5 +0,0 @@
-# www.robotstxt.org/
-# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
-
-User-agent: *
-Disallow: /
Paul Robinson Owner

missing new-line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Paul Robinson

This is DEVELOPER-835, not DEVELOPER-828. Is it worth fixing the commit comment?

jboss-developer
Collaborator

Triggering build using a merge of b5c615d on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/

Pete Muir
Owner

Fixed issue number. I can't add a new line to a file I'm removing

jboss-developer
Collaborator

Build 428 is now running using a merge of b5c615d on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/428

Paul Robinson paulrobinson merged commit 7322c6d into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jul 22, 2014
  1. Pete Muir
This page is out of date. Refresh to see the latest.
Showing with 22 additions and 11 deletions.
  1. +0 −4 Rakefile
  2. +4 −2 _config/site.yml
  3. +0 −5 robots.txt
  4. +18 −0 robots.txt.slim
4 Rakefile
View
@@ -179,10 +179,6 @@ task :deploy, [:profile, :tag_name] => [:check, :tag, :push] do |task, args|
rsync(local_path: LOCAL_CDN_PATH, host: cdn_host, remote_path: cdn_path)
end
- if !$config['robots']
- $resources << 'robots.txt'
- end
-
# Deploy the site
# If we are running a non-site root build (e.g. Pull Request) we alter where the site is copied too, and we don't delete
if ENV['site_path_suffix']
6 _config/site.yml
View
@@ -26,7 +26,8 @@ profile_base: &profile
push_to_searchisko: true
minify: &minify true
metrics: true
- robots: true
+ robots:
+ disallow: true
deploy:
<<: *deploy
scss:
@@ -69,7 +70,8 @@ profiles:
dcp_base_protocol_relative_url: //dcp.jboss.org/
dcp_base_url: http://dcp.jboss.org/
require_tag: true
- robots: false
+ robots:
+ disallow: false
deploy:
<<: *deploy
path: /www_htdocs/www
5 robots.txt
View
@@ -1,5 +0,0 @@
-# www.robotstxt.org/
-# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
-
-User-agent: *
-Disallow: /
Paul Robinson Owner

missing new-line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
18 robots.txt.slim
View
@@ -0,0 +1,18 @@
+/ www.robotstxt.org/
+/ www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
+
+- if site.robots.nil? || site.robots.disallow
+ | User-agent: *
+ Disallow: /
+
+- else
+ | User-agent: *
+ Disallow: /jbossesb/docs/4.4.GA/javadoc/esb/index-all.html
+ Disallow: /jbossaop/docs/
+ Disallow: /jbossesb/docs/
+ Disallow: /jbossmessaging/docs/
+ Disallow: /jbossremoting/docs/
+ Disallow: /jbosstm/docs/
+ Disallow: /overview/mainColumnParagraphs/0..
+ Disallow: /schema/
+
Something went wrong with that request. Please try again.