ajaxify and google adsense integration suggestion

[wikiloops]

I am in the progress of testing a setup of ajaxify and google adsense, which has been causing headaches to quite a lot of people. I will document my testing here, hoping it might be of value to someone.
As you will see, my approach is rather on the safe side concerning googles TOS. I do not guarantee google will agree about this, so please dont blame me should this turn out to be not the right solution for you or them.

Observation

When ajaxifying my project, I noticed the google banners didn't work as they used to.
To be precise, they would be displayed on the first page I refreshed, and on the next couple of pages I would browse to by ajaxified links, but after about the fourth page change, the banners seemed to "run dry" - there was no new ads fetched, and my project was showing white spaces where the ads were supposed to be.
Even during the first pages in which the ad spots were still populated, it was noticeable that adsense was messing up if the ad size varied from page to page- it would display narrow ads in wide spaces if no wide spaces had been present on the initial page.

Conclusion

Without ever looking into any code, I'd conclude something within the adsense script (the external javascript) does register the format of ads on the page it is requested by, and obviously prepares a certain amount of suitable ads. Without re-fetching the google js within my ajaxifiy page calls, it would fit in those prepared ads from the initial page load into my ad-slots, until all "prepared" ads have been shown.

Any attempt to re-fetch the google js to trigger fresh ads fails if it is done by ajax.
I assume they have a very simple and rudimentary security lock built in that just states: If anything tries to fetch this script via ajax, proceed without complaining, but dont show any ads either.
Thats googles politics, so lets just respect that for now.

Side-note: Since the refetching doesnt work, I actually prevented the output of

<script type='text/javascript' src='http://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js' async></script>

on server side when pages are fetched via ajaxy, see issue #30 )

Solution I - count along

My very first working solution was quite a crude hack:
Since ads would appear for around four pages, I just built in some kind of counter which counts along any page changes done with ajaxify.js loading, and which prevents the ajax load on each fourth page, just as if all links were suddenly given the "no-ajaxy" class - so I had three nice ajaxified page transitions and one standard page load which would say "hoohoo" to google and fetch new ads. Given, that is crude, but it worked, and since the users first impression of a sites behavior is settled after three page loads, I felt the occasional "harsh load" in between was still acceptable - and this solution didnt require any changes to my page other than the added counter functionality in the ajaxify file I customized.
If you do not have

• different ad formats on diffrent pages
• any other reason than a slightly compromised visual effect on every fourth page load
  I'd say, use such a counter and be happy to see adsense and ajaxify get along.

However, I had to deal with both of above issues - different ad sizes and the fact that I wanted to rely on staying "ajaxified" all the time, mostly because I enjoy the presence of a chatbox outside of the #content which would disappear and need to be reopened on every fourth page load.

To say it straight away - I have not found any solution that would do without the eventual harsh load to refresh the google hookup, but I have come up with a solution that feels at least a bit less crude than the one mentioned above.

Lets go over things step by step:

Recognizing different layouts needing different ad sizes

A simple scenario: You have some pages with a big square ad, lets call those "layout A", and some pages with a narrow banner, "Layout B".
Using the count-along solution, you will always see the right ads as long as a user stays on pages of the same layout - if a user navigates from one layout to another by ajaxify, you might see portions of a square ad in a narrow spot and vice versa.
What we'll need on top of the count along to prevent that, is some kind of detection/comparison between the current pages layout and the layout of the page to come.
If that detects a layout change, it should again disable the ajax loading and cause a "harsh load" to make sure the diffrent ad layout requests a new set of ads.
How to do that?
on any page, add a line of script within the #content div to pass the current pages layout to javascript:

<script> window.layout = 'A'</script>

Next, we need to prepare the comparison between that variable and the layout of the page we are about to load.
I solved this by letting ajaxify check the URI to call for certain identifiers - for a simple example, lets assume all pages in the folders "products/" and "offers/" have layout A, and the page "home.php" has layout B, this looks like:

var target=0;
url = n.href; /* grabbing the URI from ajaxy function here, before it calles the ajax load */
if ((url.indexOf('products/') >= 0) || (url.indexOf('offers/')  >= 0) {
target='A';
} else  if ( url.indexOf('home.php') >= 0) {
target='B';
} 

if (window.layout==target) {
/* target page has identical layout, let ajaxify proceed */
} else {
/* target page has diffrent layout, stop ajaxify = let harsh load happen*/
}

Of course, you can just ad "no-ajaxy" classes to all links that lead to a different layout page instead, but that may be a lot more work to do, and besides that, we'll be needing the layout detection by JS some more later.

Now, you may believe I am just disabling ajaxify on more and more instances, and depending on your page structure, using it like this may already seem absurd (if you have layout changes all the time, no call would ever be ajaxified). I was not happy about this myself, even tho this solution still guarantees to show google ads all the time and without annoying ad size messups.

Placing ads outside of the content

So, if at this point we are still not entirely happy, lets take the final step into a crazy implementation.
This last one has some benefits which I'll demonstrate, but also some clear downsides which I want to share right ahead:

• it will absolutely reduce the number of ad impressions noticed by google, so if you earn by impression rather than per-click, this will not be good for you. I will share my own testing results as soon as I have some reliable data.
• again, it will require "harsh load" on ad layout change.

It all comes down to the idea to move your ads out of the #content div, so they are loaded on the initial page once and just stay right where they are as a user navigates the ajaxified site.
Try it, you will be surprised how awesomly quick your page suddenly feels, because it doesnt take the extra second untill the ads appear :) Lets not forget, our users dont distinguish between document.ready and "this page looks ready", so by having ads that are right there, they will say your site is much quicker than before.
If you are using a fade effect, you might also enjoy the way anything but the ads fades in and out, one might possibly even get better click thru rates because of this.

Positioning out-of-content ads on the content

Now, in my case my best performing ad block happens to be within the #content, so I really felt bad about letting go of that. So again, heres a workaround for this case:

I added another <div> and let it float:left of my #content div with position:relative .
within this container, I added a second div which holds the google ad.
By assigning position:absolute and top:123px;right:123px to this ad holding div, it can be placed to the spot inside the #content div where it belongs.

edit on oct 29th - If you are worried about positioning ads with position:absolute - that is actually something google promotes! Here: https://support.google.com/adsense/answer/187769?hl=en

summed up that looks like this:

<div id='content'>
/* your content */
</div>
<div id='adpositioner' style='position:relative;float:left'>
   <div id='centerad' style='position:absolute;top:123px;right:123px'>
        <ins /*google - ad*/>
        </ins>
        <script>
             (adsbygoogle = window.adsbygoogle || []).push({});
        </script>
   </div>
</div>

Of course you may move the 'centerad' div around by jQuery, since adsense bot might not do that I felt more on the safe side with this approach which places the ads at a given position on page load.
Since my page content is dynamic and ad spaces move down depending on the content text length,
I have added a slight placing correction via jQuery on top of that, but to make sure googlebot can tell if an ad is "above fold" or not, I believe the js-independent placement is crucial.

All that being said and looking really promising, I have still found no way to deal with the need for diffrently sized ads on diffrent page layouts than the "listen for layout changes and do a harsh load if necessary" move explained above.
I'll be sure to update you on the effect of loading fewer ads and showing them a longer time as used by this solution. My first impression when surfing around was a very positive user experience.

Testing results (updated on oct 28th)

Well, after about two weeks of testing, I believe its safe to say that I experienced no negative monetary effects from implementing adsense the way described in detail above.
The number of counted ad impressions gets reduced as expected, but since I am making most revenue by CPC, there has been no difference in earnings since the implementation.
have a look at http://www.wikiloops.com to see it in action.

[arvgta]

Hey :) Fantastic research, specification, design if not even implementation!
It sounds as if you had a working demo somewhere in the back-office ;)

Sorry for answering a bit late, but that other issue(30) was "bugging" me quite a bit.

I'll get straight to the point, just briefly recapping what I understood of what you posted:

1. We agree, that this issue pertains to "Ajax + Adsense" in general - i.e. not only "Ajaxify + Adsense".
2. Google refuses to serve more than 3 re-freshes of the same ad slot, if in an Ajax environment.
3. Google's TOS are very defensive and harsh, the worst thing possible being the deletion of the whole account.
4. Any too aggressive approach is therefore a no-go!
5. Differences in layout between pages may cause a headache and raise the need for quite a bit of "acrobatics" and low-level hacks...
6. It is much easier (and already feasible) to place the ads outside of the content div(s).
7. A sort of "illusion" can be created with CSS + maybe jQuery to position them in the content area nevertheless.
8. If one goes for the rather simple approach of placing the ads semantically outside of the content area, it is more lucrative to get paid for real clicks, rather than impressions.
9. Rendering the ads in the content area and enforcing a refresh is feasible, but disrupts Ajaxify every fourth page load.
10. If the site is heterogenous as far as the ad placement is concerned, it is proposed to detect the type first.
11. If successfully performed - the user experience generally is awesome :)

It reads as if the approach of placing the ads outside of the content div(s) like performed successfully on Dark Realm Gaming
already is feasible. You illustrated the possibility to manipulate the exact position of the ads. I would regard that a bit like a "decorator pattern" i.e. optional, the main issue being that the ads are visible instead of "whiteboxes".

Don't mean to be rude, but I am wary of your very first approach - causing a hard refresh every 4 pages.
The main downside of such coarse hacks is that they aren't very resilient to change (e.g. to changes at Google Adsense).

Issues of the "outside content div" approach(which I would favourise):

• It must be measured, what effect it has on the user to always see the same ads across clicks.
• It must be tested (maybe against Dark Realm Gaming), whether the Firebug issue is still
  a nuisance or has been solved in the meantime, or whether there is a workaround.

Thanks very much!!

[wikiloops]

There is a few things I'd like to add:

• the Firebug issue - yes, I have encountered that one, it didnt compromise any of my other JS script, so besides seeing the red alert in firebug, I had no reason to worry about that too much.
• there might be an option to work around the mentioned ad-size-change problem by using googles dynamic size ads (initially introduced for mobile pages). Those just fill the spot with the largest fitting ad, you must choose between horizontal and vertical orientation tho. Interestingly, the ad changes on orientation change on a mobile device (if that goes along with a resize of the ad holder), so obviously some changes are allowed and expected.
• after thinking about it some more, I believe what may go into the standard ajaxify version from the extras I implemented might be the option to let a custom function check if a click should be handled by ajaxify. The "no-ajaxy" class on the javascript side, so to speak.
  Like this:

 jQuery("#content").ajaxify({
jscheck: true 
/* replace true by external function name 
to let an external function decide 
if ajaxify shall load the click*/
})

So I could use:

 jQuery("#content").ajaxify({
jscheck: nojycheck 
})

function nojycheck() {
if (my.custom.requirements) {
       return true;
            } else {
       return false;
                     }
}

Quite similar approach to the "cb" parameter. It may even be possible to send some parameter(s) from within ajaxify into any custom function (remember I am using the url as registerred by ajaxify in my custom check)

• I hope I will be able to offer some "same site/same ad placement" comparison between the "crude" approach and the "static ads" approach.
  When switching from "no ajaxify" to "ajaxify+crude counter workaround", there was no change in my revenue, so much I know.
  I do understand that counter solution is too crude to implement in a plugin - it still is the only working solution without any changes to the pages code I have come across so far, and its definetly no malicious hack into googles code.
  Speaking of malicious hacks: Who knows, might be they also check for the xhtml request header to identify ajax loads ;)

[arvgta]

Hey :)

• So the Firebug issue is still a nuisance, but no reason to abandon the whole approach.
• Dynamic ads from Google could be a relief in case of differently sized ads / selecting the slot with the biggest free space automatically (does this work on the desktop as well?)
• Regarding the extra external function to check whether a click should be processed - I would reckon that overriding the existing selector with an external function would be more elegant? Or would you like a check "on top" of the selector?
• Just a note: There are events with data being routed back to the client of Ajaxify already.

(I am still wondering about the two bugs in #30 :( )

[wikiloops]

• dynamic ads do work on desktop, I'm a not sure about potential effects on the monetary side, so I'm a bit wary about those. Maybe I'll do some test at some point, since the fixed orientation of those dynamic ads is a con in my usecase, I cannot really solve my issue with this unless I change my ad design.
• Not sure if your "selector override" will work for me - don't forget, I am initiating ajaxify only once, and the custom parameters I need to check against may change as the user navigates around.
  I have considered using jQuery to add the "no-ajaxy"-class to all 'a'-anchors on a page (i understood thats what you are suggesting) if I needed to prevent processing, the check "on top" of the selector felt much less intrusive, especially because it does allow to evaluate custom parameters as the user clicks.
• had a quick look at the pronto events, thats pretty much what I was looking for. I assume
  eventInfo.target will return the uri, right? If the "no-ajaxy"-class can be applied to the clicked link after the pronto.request has fired, and before the internal check for the class happens, that might indeed be a very clean way to prevent processing by JS.
  I'd need some help setting this up, because I am not sure how to refference the object resembling the clicked link within ajaxify from an external function - I assume $(this) will not work.
  So, I am thinking

$(window).on('pronto.request', function(event, eventInfo){
    var target = eventInfo.target;
if (my.custom.requirements==target) {/* <- simplified example */
      $('whatgoeshere?').addClass('no-ajaxy');
            }   
})

Is that what you were suggesting? And how can one be sure ajaxify wont continue processing while I am still in the process of checking if I want it to? If that can be guaranteed, this would be a great implementation!

[arvgta]

• dynamic adds: I'll leave that up to you, as I have no conception of how they work
• "filter" function for links on top of selector- should not be a problem, is, of course, more powerful, just requiring an additional parameter
• Pronto events: exactly as you perceived it, however, the addClass() would only have effect "the next time around", I'm afraid...

[wikiloops]

the "filter" function it should be, then - "next time around" is too late if I have ads hovering over the wrong spots after missing the check for a layout change :)

Will monitor the adsense revenue effects for a few more days, adsense is always a bit shaky about their "estimated revenues", so there is not much use looking at short term data.
Doesn't look bad so far, even tho the number of pageviews registerred by adsense has dropped as expected.

[arvgta]

Hey :)

I have no idea, how the filter function could have immediate effect, if you're applying it in the Pronto events, sorry.

The Pronto algorithm is, that at the very beginning there is a check with on() only once.
After that, there are no more checks...

If you have an idea, then please shout :)

[wikiloops]

sorry, we just got confused about the "filter" term.
I was suggesting to get back to the variant I hinted at with the "jscheck" parameter earlier.
To give a structural overview what should be changed within ajaxify, let me draw a little logical lineup of events:

Current version does the following:
user clicks
-> ajaxify prevents default click response
-> aj. checks for "no-ajaxy"-presence
-> either handles the click as ajax, or fires standard load

I would wrap the check for "no-ajaxy" and further processing in another check, which may happen customly outside of ajaxify (thats what I demonstrated in the earlier post).
Within ajaxify, it would then have to look somewhat like this:
user clicks
->ajaxify prevents default click response, then:

if (jscheck=='none' || jscheck.(clickedlink) == 'true') {
                                /* run standard procedure, check for class, fire pronto etc etc */
                                                                      } else {
                                /* dont ajaxify, trigger harsh load */
                                                                                }

so, if the jscheck parameter is not set (none should be default), everything will work as it does now,
if jscheck (= the external check functions name) is defined and the external function returns true or false,
ajaxify responds accordingly.

I passed the clickedlink object to the external function in this example - this should ideally be the same info as returned by pronotos eventInfo, so the custom function is fed some data to be able to evaluate what to do.
Since pronto will fire later, we can not use it here, but it should be easy to spot the right var from ajaxify to use there.
This setup will require you to add the jscheck parameter and to do one additional if-wrap in ajaxify.js, it does add a new level of instant customization without breaking the out-of-the box concept.

This feature will be usefull to detect layout changes that require a harsh load to make sure ads are placed correctly.
Also, instead of adding the "no-ajaxy" class to all links leading off your site, one might as well let the external function find out if the clicked link contains a diffrent domain than the current one - this would actually be much less work than adding the class to all the links manually...

And - once we are talking customization options - I could imagine a third offered external function (you already had the cb parameter that works similar, now jscheck is introduced), lets call the third transcheck for now:
If you'd offer another instant check which (if enabled) will decide which transition to use, designers might be interested to use diffrent transitions depending on the sites structure... p.e. - use a squeeze transition when flipping from shop item to shop item, but using a fade when switching from items to "company history" or some other diffrently layouted chapter of a page.
As always, just offering some ideas, solved my own demands by working inside of ajaxify quite rudely :)

[arvgta]

Thanks for the splendid specification!

I totally agree, except for some small things:

• I would like to call the method filter
• default false - let's everything "through"
• if true - let's nothing through, a bit like pluginon: false
• if function specified then parse it

If the function returns:

• true -> do nothing (possibly raising another event, if the client needs to know)
• false -> let's Pronto continue

I'd like to modify the following line in the function _click() this line

if (_exoticKey(e) || _diffHost(link)) return;

...to test our new functionality:

if (_exoticKey(e) || _diffHost(link) || _checkFilter()) return;

_checkFilter() would perform our above logic, be passed the eventInfo (at least) and return true if filter is "set" on that link.
It might also raise another event, e.g. pronto.filter - in case of returning false (?), passing the eventInfo at least...

Once we've implemented this successfully, we can continue with the transcheck feature - sounds interesting! :)

Please let's also debug #32 first - thanks!

[arvgta]

Resume: There are no big problems, if the ads are placed outside of the content div(s).
Example

[arvgta]

Please re-open, if there is news...

[wikiloops]

Well, after about two weeks of testing, I believe its safe to say that I experienced no negative monetary effects from implementing adsense the way described in detail above.
The number of counted ad impressions gets reduced as expected, but since I am making most revenue by CPC, there has been no difference in earnings since the implementation.

However, I'd suggest to showcase another site for this implementation, because the ad placement on the example you linked will not convince people who like to place ads above the fold and within content.
Actually, there is talk the google panda update penaltizes sites who place ads at the top of pages (which would be the alternative to hiding them below), so the fact that I am placing ads within the content is of relevance for this demonstration.
It may be looked at on http://www.wikiloops.com

[arvgta]

Hi Wikiloops!

Great to hear from you! Thanks for tuning back in!

So, what is pending is to make a brief DIY-tutorial on 4nf.org.
There seem to be three cases:

• Ads at the tops of the page - not recommended because penalised by Google
• Ads below the content div(s) - easiest case technically, but placing above the fold may be more effective
• Ads in the main content div. Wikiloops could serve as a pioneer example with some explanations on 4nf.org in prosa

Do you agree?

[wikiloops]

few things to ad here:

• "ads on tops of the page..."
  mind - google is not equal google :) PANDA is a google algo that affects SearchEngineRanking and seems to penaltize ads above content, while Google adsense actually recommends placing ads there to make more revenue. One should be very, very carefull about statements regarding this, because google is likely to switch their segment algos anytime and without regard to the policies of their adsense branch. Theres lots of articles from SEO people on this issue, like p.e. this one here:
  http://www.shoutmeloud.com/page-layout-algo-ads-above-fold-bad-seo.html
• found another pro argument for my implementation: I wasn't all sure wether the position:absolute trick used to place the ads would possibly be considered malicious. As I learned yesterday, google adsense actually promotes this approach as a "best use" case, here:
  https://support.google.com/adsense/answer/187769?hl=en (will ad this link to the first post here, too)

As for the DIY, the most important aspect is to make sure people understand the difference between "visually at the top / within the visual #content area" and "at the top of the code / outside of the DOM element #content", which is the key to understanding what we are doing here.

I would steer clear of giving any advice on adsense placement, there is a ton of footage out there and people using adsense a lot will be familiar with that - the fact that an ad placed somewhere way below the pages content will not perform well compared to an ad the users are actually seeing at first sight is common sense rather than a bright expert trick ;)
I wouldn't risk people saying things like "well, Arvind told me that was good/bad for SEO and my adsense revenue, now that it doesnt work as expected, he's to blame!" - it all happens, thats why I kept repeating I'm only offering an approach suitable for my site that may not work for others.

Last, I'll make another test run after finding the adsense article linked above, now placing the ads code right at the top of the code (not the page!) as advised to see if that really affects the CPC bids - I had the code below the #content so far, so this might actually do good. will report.

[arvgta]

Just a quick note - I've created a skeleton of the page @4nf.org:

• http://4nf.org/dollars/

Thanks for summoning to have caution for liability. Hence, I've introduced the topic with a disclaimer :)