Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
added readme and a few other fies
  • Loading branch information
thomasdavis committed Oct 3, 2012
1 parent 62f218f commit a0bbc1b
Show file tree
Hide file tree
Showing 6 changed files with 63 additions and 9 deletions.
46 changes: 46 additions & 0 deletions README
@@ -0,0 +1,46 @@
<h3>Welcome!</h3>
<p>Seo Server is a command line tool that runs a server that allows GoogleBot(and any other crawlers) to crawl your heavily Javascript built websites. The tool works with very little changes to your server or client side code.</p>
<p><i>This entire site is driven by Javascript(view the source or see the <a href="https://github.com/apiengine/seoserver-site">code</a>). Click the `What does Google see?` button at the bottom of each page to see Seo Server in action.</i></p>

<h3>How it works</h3>
<img src="http://yuml.me/5b1b60bb" /><br /><br />
<p>Seo Server runs <a href="http://phantomjs.org/">PhantomJs</a>(headless webkit browser) which renders the page fully and returns the fully executed code to GoogleBot.</p>

<h3>Getting started</h3>
<p>1) you must install PhantomJs(<a href="http://phantomjs.org/">http://phantomjs.org/</a>) and link into your bin so that Seo Server can call it.</p>
<p>2) Seo Server is an NPM module so install via</p>
<code>sudo npm install -g seoserver</code>
<p>3) Now we have access to the Seo Server command line tool</p>
<code>seoserver start</code>
<p>Which starts an Express server on port 3000 or</p>
<code> seoserver -p 4000 start</code>
<p>Start it as a background process and log the output</p>
<code> seoserver -p 4000 start > seoserver.log &</code>

<h3>Telling GoogleBot to fetch from Seo Server</h3>
<p>To tell GoogleBot that we are using ajaxed content we simply add to our sites index.html file the Google specific <a href="https://developers.google.com/webmasters/ajax-crawling/docs/specification">meta tag</a>. If you view the source of this page you can see we have included the tag below. </p>
<code>&lt;meta name="fragment" content="!"&gt;</code>
<p>Now whenever GoogleBot visits any of our pages it will try to load <code>?_escaped_fragment_=pathname</code></p>
<p>So if we were using Apache with mod rewrite and mod proxy, we can include in our .htaccess</p>
<code>
RewriteCond %{QUERY_STRING} ^_escaped_fragment_=(.*)$<br />
RewriteRule (.*) http://address-of-seoserver:3000/%1? [P]
</code>
<p>Now all request from GoogleBot will be returned fully rendered. How GoogleBot sees the page can be tested with Google <a href="http://www.google.com/webmasters/">WebMasters</a>(they allow you to simulate Google crawls and see the result instantly).</p>

<h3>For other crawlers</h3>
<p>
Using mod rewrite, we can send other crawlers to Seo Server also
</p>
<code>
RewriteCond %{HTTP_USER_AGENT} ^DuckDuckBot/1.0;<br />
RewriteRule (.*) http://address-of-seoserver:3000/%1? [P]

</code>
<h3>FAQ</h3>
<p>Nothing here yet, but check out the examples on the left to see different types of ajaxed content. Also ask questions and give feedback on GitHub <a href="https://github.com/apiengine/seoserver/issues">issues</a>.
<hr />
<button class="btn btn-info showseo">What does Google see?</button>
<hr />
<textarea class="showseo-output" disabled="disabled"></textarea>

6 changes: 6 additions & 0 deletions css/theme.css
Expand Up @@ -3,6 +3,7 @@
width: 157px; width: 157px;
height: 34px; height: 34px;
background: url('../img/api-engine-logo_official.png'); background: url('../img/api-engine-logo_official.png');
display: inline-block;
} }


.showseo-output { .showseo-output {
Expand All @@ -17,4 +18,9 @@ code {


body { body {
overflow-y: scroll; overflow-y: scroll;
}

h1 {
margin-left: 17px;
margin-bottom: 16px;
} }
2 changes: 1 addition & 1 deletion js/main.js
Expand Up @@ -27,7 +27,7 @@ require([
'clicky' 'clicky'
], function(AppView, Vm, Router, norefclicky){ ], function(AppView, Vm, Router, norefclicky){
// Some hackery to include clicky in our app // Some hackery to include clicky in our app
try{ clicky.init(203165); }catch(e){} try{ clicky.init(100532051); }catch(e){}
var appView = Vm.create({}, 'AppView', AppView); var appView = Vm.create({}, 'AppView', AppView);
Router.initialize({appView: appView}); Router.initialize({appView: appView});
appView.render(); // render() calls Backbone.history when its ready to start appView.render(); // render() calls Backbone.history when its ready to start
Expand Down
2 changes: 1 addition & 1 deletion templates/examples/json.html
@@ -1,6 +1,6 @@
<div class="well"> <div class="well">
<h3>JSON data with Javascript template</h3> <h3>JSON data with Javascript template</h3>
<p>Pulling in data from stats.cdnjs.com (a CORS enabled API)</p> <p>Pulling in data from stats.cdnjs.com (a CORS enabled API) from <a href="http://cdnjs.com">cdnjs.com</a></p>
<table class="table"> <table class="table">
<thead><tr><th>Library</th><th>Wednesday's hits</th></tr></thead> <thead><tr><th>Library</th><th>Wednesday's hits</th></tr></thead>
<tbody> <tbody>
Expand Down
12 changes: 7 additions & 5 deletions templates/home/page.html
@@ -1,20 +1,22 @@
<div class="well"> <div class="well">
<h3>Welcome!</h3> <h3>Welcome!</h3>
<p>Seo Server is a command line tool that runs a server that allows GoogleBot(any any other crawlers) to crawl your heavily Javascript built websites. The tool works with very little changes to your server or client side code.</p> <p>Seo Server is a command line tool that runs a server that allows GoogleBot(and any other crawlers) to crawl your heavily Javascript built websites. The tool works with very little changes to your server or client side code.</p>
<p><i>This entire site is driven by Javascript(view the source). Click the `What does Google see?` button at the bottom of each page to see Seo Server in action.</i></p> <p><i>This entire site is driven by Javascript(view the source or see the <a href="https://github.com/apiengine/seoserver-site">code</a>). Click the `What does Google see?` button at the bottom of each page to see Seo Server in action.</i></p>


<h3>How it works</h3> <h3>How it works</h3>
<img src="http://yuml.me/5b1b60bb" /><br /><br /> <img src="http://yuml.me/5b1b60bb" /><br /><br />
<p>Seo Server runs <a href="http://phantomjs.org/">PhantomJs</a>(headless webkit browser) which renders the page fully and returns the fully executed code to GoogleBot.</p> <p>Seo Server runs <a href="http://phantomjs.org/">PhantomJs</a>(headless webkit browser) which renders the page fully and returns the fully executed code to GoogleBot.</p>


<h3>Getting started</h3> <h3>Getting started</h3>
<p><strong>First</strong> you must install PhantomJs(<a href="http://phantomjs.org/">http://phantomjs.org/</a>) and link into your bin so that Seo Server can call it.</p> <p>1) you must install PhantomJs(<a href="http://phantomjs.org/">http://phantomjs.org/</a>) and link into your bin so that Seo Server can call it.</p>
<p>Seo Server is an NPM module so install via</p> <p>2) Seo Server is an NPM module so install via</p>
<code>sudo npm install -g seoserver</code> <code>sudo npm install -g seoserver</code>
<p>Now we have access to the Seo Server command line tool</p> <p>3) Now we have access to the Seo Server command line tool</p>
<code>seoserver start</code> <code>seoserver start</code>
<p>Which starts an Express server on port 3000 or</p> <p>Which starts an Express server on port 3000 or</p>
<code> seoserver -p 4000 start</code> <code> seoserver -p 4000 start</code>
<p>Start it as a background process and log the output</p>
<code> seoserver -p 4000 start > seoserver.log &</code>


<h3>Telling GoogleBot to fetch from Seo Server</h3> <h3>Telling GoogleBot to fetch from Seo Server</h3>
<p>To tell GoogleBot that we are using ajaxed content we simply add to our sites index.html file the Google specific <a href="https://developers.google.com/webmasters/ajax-crawling/docs/specification">meta tag</a>. If you view the source of this page you can see we have included the tag below. </p> <p>To tell GoogleBot that we are using ajaxed content we simply add to our sites index.html file the Google specific <a href="https://developers.google.com/webmasters/ajax-crawling/docs/specification">meta tag</a>. If you view the source of this page you can see we have included the tag below. </p>
Expand Down
4 changes: 2 additions & 2 deletions templates/menu.html
Expand Up @@ -8,6 +8,6 @@
</ul> </ul>
</div> </div>
<div class="well sidebar-nav"> <div class="well sidebar-nav">
<p>Bought to you by</p> <p>Brought to you by</p>
<div class="logo"></div> <a class="logo" href="http://apiengine.io"></a>
</div> </div>

0 comments on commit a0bbc1b

Please sign in to comment.