Why is the crawlable feature removed in 1.6? Any further explanation would be awesome. ;)
I don't consider this crawlable mechanism as a good long term investment. The new history API removes the need of hashes completely and provides a much better base for search engine indexing. Here are sample queries that show how the content can be indexed:
Hi. I am not too sure as to what your explanation demonstrates…
As far as I can see, google is still recommending the #! method (https://developers.google.com/webmasters/ajax-crawling/docs/getting-started).
What is the alternative you are talking about ?
If, for example, an user that is visiting the site with IE9 (there are still many) and paste the url in facebook (for sharing), the site is crawled wrongly, because it doesn't have the hashbang for old browsers.
Specifically, what needs to be done to make this work? I'm sure if you tell people to just "use the History API", it won't work for everyone.
Why would Google suggest us to use a headless browser if Google is using a headless browser already? Something's up here. How is your linked example not just a special (accidental?) case?
Is Google really doing this???
Just curious. Food for thought. :)
Ok, I thought about this a little more:
Google will be able to index your content if you use the History API if and only if you send identical content from the server when that URL is visited in the browser, not if you rely on reading the URL then using AJAX to get and show content based on the URL.
Example: your site uses history.pushState and loads content via ajax. If a user visits the URL you saved to the history, the server will send back the page in the same form as it appeared when it was generated by ajax. In this case, History API works fine. But if your app relies solely on AJAX to generate content (as in no matter what URL you visit the app always retrieves data based on the URL then displays it with AJAX (so that the server always sends exactly the same HTML at first)) then Google will not be able to index the site.
So simply telling someone to use History API isn't the entire solution. The Ajax site still needs to be able to send an initial HTML snapshot of the page for each URL, which is a big task for someone who designed their site using JSON to retrieve data and not HTML. Those users will have to do heavy modification to make sure the app sends HTML on each initial request.
A solution for apps relying on JSON would be to use a headless browser on initial request in order to generate HTML, then send that, and from that moment on the app can continue to use JSON like normal.