Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

DEVELOPER-828 enable robots.txt for prod #403

Merged
merged 1 commit into from

3 participants

@pmuir
Owner

No description provided.

@jboss-developer-ci
Collaborator

Triggering build using a merge of 71a2e0e on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/

@jboss-developer-ci
Collaborator

Build 419 is now running using a merge of 71a2e0e on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/419

@paulrobinson paulrobinson commented on the diff
robots.txt
@@ -1,5 +0,0 @@
-# www.robotstxt.org/
-# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
-
-User-agent: *
-Disallow: /
@paulrobinson Owner

missing new-line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@paulrobinson

This is DEVELOPER-835, not DEVELOPER-828. Is it worth fixing the commit comment?

@jboss-developer-ci
Collaborator

Triggering build using a merge of b5c615d on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/

@pmuir
Owner

Fixed issue number. I can't add a new line to a file I'm removing

@jboss-developer-ci
Collaborator

Build 428 is now running using a merge of b5c615d on branch master:
Private: https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/www.jboss.org-pull-player-executor/428

@paulrobinson paulrobinson merged commit 7322c6d into jboss-developer:master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jul 22, 2014
  1. @pmuir
This page is out of date. Refresh to see the latest.
Showing with 22 additions and 11 deletions.
  1. +0 −4 Rakefile
  2. +4 −2 _config/site.yml
  3. +0 −5 robots.txt
  4. +18 −0 robots.txt.slim
View
4 Rakefile
@@ -179,10 +179,6 @@ task :deploy, [:profile, :tag_name] => [:check, :tag, :push] do |task, args|
rsync(local_path: LOCAL_CDN_PATH, host: cdn_host, remote_path: cdn_path)
end
- if !$config['robots']
- $resources << 'robots.txt'
- end
-
# Deploy the site
# If we are running a non-site root build (e.g. Pull Request) we alter where the site is copied too, and we don't delete
if ENV['site_path_suffix']
View
6 _config/site.yml
@@ -26,7 +26,8 @@ profile_base: &profile
push_to_searchisko: true
minify: &minify true
metrics: true
- robots: true
+ robots:
+ disallow: true
deploy:
<<: *deploy
scss:
@@ -69,7 +70,8 @@ profiles:
dcp_base_protocol_relative_url: //dcp.jboss.org/
dcp_base_url: http://dcp.jboss.org/
require_tag: true
- robots: false
+ robots:
+ disallow: false
deploy:
<<: *deploy
path: /www_htdocs/www
View
5 robots.txt
@@ -1,5 +0,0 @@
-# www.robotstxt.org/
-# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
-
-User-agent: *
-Disallow: /
@paulrobinson Owner

missing new-line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
View
18 robots.txt.slim
@@ -0,0 +1,18 @@
+/ www.robotstxt.org/
+/ www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
+
+- if site.robots.nil? || site.robots.disallow
+ | User-agent: *
+ Disallow: /
+
+- else
+ | User-agent: *
+ Disallow: /jbossesb/docs/4.4.GA/javadoc/esb/index-all.html
+ Disallow: /jbossaop/docs/
+ Disallow: /jbossesb/docs/
+ Disallow: /jbossmessaging/docs/
+ Disallow: /jbossremoting/docs/
+ Disallow: /jbosstm/docs/
+ Disallow: /overview/mainColumnParagraphs/0..
+ Disallow: /schema/
+
Something went wrong with that request. Please try again.