Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

DEVELOPER-828 enable robots.txt for prod #403

Merged
merged 1 commit into from

3 participants

@pmuir
Owner

No description provided.

@jboss-developer-ci
Collaborator

Triggering build using a merge of 71a2e0e on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/

@jboss-developer-ci
Collaborator

Build 419 is now running using a merge of 71a2e0e on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/419

@paulrobinson paulrobinson commented on the diff
robots.txt
@@ -1,5 +0,0 @@
-# www.robotstxt.org/
-# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
-
-User-agent: *
-Disallow: /
@paulrobinson Owner

missing new-line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@paulrobinson

This is DEVELOPER-835, not DEVELOPER-828. Is it worth fixing the commit comment?

@jboss-developer-ci
Collaborator

Triggering build using a merge of b5c615d on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/

@pmuir
Owner

Fixed issue number. I can't add a new line to a file I'm removing

@jboss-developer-ci
Collaborator

Build 428 is now running using a merge of b5c615d on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/428

@paulrobinson paulrobinson merged commit 7322c6d into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jul 22, 2014
  1. @pmuir
This page is out of date. Refresh to see the latest.
Showing with 22 additions and 11 deletions.
  1. +0 −4 Rakefile
  2. +4 −2 _config/site.yml
  3. +0 −5 robots.txt
  4. +18 −0 robots.txt.slim
View
4 Rakefile
@@ -179,10 +179,6 @@ task :deploy, [:profile, :tag_name] => [:check, :tag, :push] do |task, args|
rsync(local_path: LOCAL_CDN_PATH, host: cdn_host, remote_path: cdn_path)
end
- if !$config['robots']
- $resources << 'robots.txt'
- end
-
# Deploy the site
# If we are running a non-site root build (e.g. Pull Request) we alter where the site is copied too, and we don't delete
if ENV['site_path_suffix']
View
6 _config/site.yml
@@ -26,7 +26,8 @@ profile_base: &profile
push_to_searchisko: true
minify: &minify true
metrics: true
- robots: true
+ robots:
+ disallow: true
deploy:
<<: *deploy
scss:
@@ -69,7 +70,8 @@ profiles:
dcp_base_protocol_relative_url: //dcp.jboss.org/
dcp_base_url: http://dcp.jboss.org/
require_tag: true
- robots: false
+ robots:
+ disallow: false
deploy:
<<: *deploy
path: /www_htdocs/www
View
5 robots.txt
@@ -1,5 +0,0 @@
-# www.robotstxt.org/
-# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
-
-User-agent: *
-Disallow: /
@paulrobinson Owner

missing new-line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
View
18 robots.txt.slim
@@ -0,0 +1,18 @@
+/ www.robotstxt.org/
+/ www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
+
+- if site.robots.nil? || site.robots.disallow
+ | User-agent: *
+ Disallow: /
+
+- else
+ | User-agent: *
+ Disallow: /jbossesb/docs/4.4.GA/javadoc/esb/index-all.html
+ Disallow: /jbossaop/docs/
+ Disallow: /jbossesb/docs/
+ Disallow: /jbossmessaging/docs/
+ Disallow: /jbossremoting/docs/
+ Disallow: /jbosstm/docs/
+ Disallow: /overview/mainColumnParagraphs/0..
+ Disallow: /schema/
+
Something went wrong with that request. Please try again.