Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

For a visit via a campaign, if the campaign keyword is not set, detect keyword from Referrer URL #517

Closed
mattab opened this Issue · 17 comments

4 participants

@mattab
Owner

Bug report:

When a visitor comes and a campaign “name” is detected, there are sometimes no “keyword” set. Instead of leaving the value empty, we could try to fill up the gap and:

  • try detect the keyword from the Referrer URL if it is found there (ie. if a search engine was used)
  • if no keyword could be detected from the ref URL, set the “campaign keyword” as the domain hostname. This way it will give an easy breakdown of referrer website URL for campaigns.

This would improve the campaign tracking data quality.

Google Adwords tracking would also work better and give more accurate data.

  • See new [Tracking Campaigns](http://piwik.org/docs/tracking-campaigns/)
  • [URL Builder tool](http://piwik.org/docs/tracking-campaigns/url-builder/)
  • Official piwik campaign parameters names are now: pk_campaign and pk_kwd (consistent with cookie prefix, better than piwik_ prefix)
  • New FAQ: [“Does Piwik support Google Analytics campaign parameters (utm_campaign, utm_medium, utm_source, utm_term)?”](http://piwik.org/faq/general/#faq_119)
  • New FAQ: [“How do I customize the Piwik Campaign parameters pk_campaign and pk_kwd?” ](http://piwik.org/faq/how-to/#faq_120)
@anonymous-piwik-user

No worries I'll give this a shot with a small adwords account I am playing with. Though I also have some more advanced idea's for tracking adwords without auto-tagging and thus giving more functionality in other area's.

Will test this out firstly and go from there.

@mattab
Owner

marc, sounds good; let us know if you find any bug or issues, this has never been tested, but we want this to work perfectly and with an easy to use documentation; I think we can assume that we need adwords to be auto tagged using landing keyword, etc.

@anonymous-piwik-user

Hey Matt,

Not really. There are other less obvious to others, ways to track adwords that still keeps your data private.

See bots and others can crawl and track adwords adds and backwards engineer what you are bidding on. Do that every day over a period of time and you have a clear picture of what keywords are profitable for someone in what industry.

I'll test it out on auto-tagging and then discuss privately the other concepts and decide if you want to introduce those.

Of course there could be both options, auto-tagging and then the privacy option :D

Talk Soon
Marc

@mattab
Owner

example adwords url

http://www.google.co.uk/aclk?sa=L&ai=Ct4TzXV_9TL6QIYS9hAfg9oGfCuj09-QBwMGNhxi4hsLuVggAEAIoAlDA-7T6-P____8BYLueyoPcCsgBAakC1ql8VKZNuj6qBB5P0As2Ghmje5goPNu05sgS5wCcz4ejXah6M3nHdqg&sig=AGiWqtwqIsBBq929N9J0_AiRN6Mo0uHa4Q&adurl=http://www.itsaviours.com&rct=j&q=site%3Apcfixt.com%20computer%20repair%20falkirk 
@mattab
Owner

From forum post


Generally, Adwords links seem to look like this:

http://www.google.com/aclk?sa=L&ai=CiUrQZYQYTbfCIMrgswaj9KjVApnw_9gB69247wnD4P8JCAAQASgDUPrYzh5gldqIgpgHyAEBqgQhT9CSfMdbwbHaUGieetZZj76iHm8iOC3ULeIvG3mH4k5qugUTCIOhx9G1jKYCFYbhzAodME9x38oFAA&rct=j&q=flights&ei=ZYQYTcPaDobDswawnsX7DQ&sig=AGiWqtwu0APuZWGM9C-z3t8c6j8f6irfjA&adurl=http://pixel1590.everesttech.net/1590/rq/3/s_1b6ee0e1a0059564b80fdcb8e19f29f3_2558612431/url%3Dhttp%253A//www.cheaptickets.com%253Fgcid%253DS16036x129%2526keyword%253Dcheap%252Btickets%252Bcom_ge%2526WT.mc_id%253Dec105%2526WT.mc_ev%253Dclick%2526WT.srch%253D1%2526DCSext.mc_kw%253Dcheap%252Btickets%252Bcom&cad=rja

http://www.google.com/aclk?sa=l&ai=CMkaoZYQYTbfCIMrgswaj9KjVAuz2g8gBzNXciRbD4PMYCAAQAigDUMHX2Kb4_____wFgldqIgpgHoAH608P1A8gBAakCHLcuXY2yhz6qBBxP0MITvVvCudvoax-w1S8ZuQgOZQKZVQJ1_pDNugUTCIOhx9G1jKYCFYbhzAodME9x38oFAA&rct=j&q=flights&ei=ZYQYTcPaDobDswawnsX7DQ&sig=AGiWqtw2ETvZZJV8d4SQdSSHAJqf3ACuXw&adurl=http://en.momondo.com/Default.aspx%3Faw%3Dde&cad=rja 
@mattab
Owner

I made some test for a few days with an adwords account. Adwords reports 5 clicks and I see 5 clicks in the Piwik DB, good start. But 1 didnt have the referer set, 2 others were from google & Ask.com, 2 from Google display (adsense).

The only way to track Adwords properly is to have users enable auto tagging in their account:
http://adwords.google.com/support/aw/bin/answer.py?hl=en&answer=55590

Then landing page URLs have the parameter 'gclid' which is how we detect that the user is coming from adwords.

Alternatively (AND / OR), Users can 'manually' tag their landing pages, then they simply can use the standard Piwik campaign feature #604

Then we can look at the ref URL

  • If google (or Ask or other SE) then it is Google Search Network, with given keyword
  • If domain = googleads.g.doubleclick.net then it is Google Display Network In the "Display" case, the ref URL looks like this:
    http://googleads.g.doubleclick.net/pagead/ads?client=ca-amazon-alexa_js&format=336x280_as&output=html&h=280&w=336&lmt=1294593285&channel=g-channel-14&ad_type=text&color_bg=ffffff&color_border=ffffff&color_line=00ff00&color_link=0033cc&color_text=333333&color_url=339900&oe=utf8&flash=10.0.32&hints=dospuntocerovision.com&url=http%3A%2F%2Fwww.alexa.com%2Fsiteinfo%2Fdospuntocerovision.com&adsafe=high&dt=1294593287457&shv=r20101117&jsv=r201106&saldr=1&correlator=123287498&frm=0&adk=3179527&ga_vid=1772644835.1294598&ga_sid=1294588&ga_hid=489613643&ga_fc=0&u_tz=0&u_his=3&u_java=1&u_h=800&u_w=1280&u_ah=760&u_aw=1280&u_cd=32&u_nplug=25&u_nmime=151&biw=1280&bih=577&eid=30143102&ref=http%3A%2F%2Fwww.bing.com%2Fsearch%3Fq%3Ddospuntocerovision.com%26pc%3Dconduit%26form%3DCONMHP%26ptag%3DA5DC7BAE8890B42A4A3F%26conlogo%3DCT2769726&fu=0&ifi=1&dtd=71&xpc=cO7JaJYx7F&p=http%3A//www.alexa.com

The parameter 'url' gives the URL where the click comes from. This is important to report it in Piwik since adwords advertisers would want to ban domains that have poor conversion rates from their Display pool.

Maybe a good way to report Adwords in Piwik would be:

  • on each page view in Piwik, check if URL contains gclid
    • if it does contain, AND there is no standard Piwik campaign parameters (OR GA campaign parameters), check referer:
    • if referer not empty and is googleads.g.doubleclick.net OR is a known search engine (big search engines have deals to display adwords on their search result pages)
    • we have detected a click from Adwords (ie. that we are paying for)
    • if this is a new visit OR if an existing visit with empty referer, then we record adwords as referer type 'campaign'
    • Set campaign = 'Google Adwords - Display Network', keyword = URL parameter from referer
    • OR set campaign = 'Google Adwords - Search Network', keyword = KWD detected in the referer

See google search network: http://adwords.google.com/support/aw/bin/answer.py?hl=en&answer=90956

and google display network: http://adwords.google.com/support/aw/bin/answer.py?hl=en&answer=117120

interesting page: http://www.google.com/support/forum/p/Google%20Analytics/thread?tid=3b79c0131cf3148d&hl=en

Tasks

  • update the Tracker algorithm to properly detect adwords search/display
  • Link the 'Campaigns' Referer report from the Referer Overview report
  • Ensure that logos are displayed in the campaign report like normal organic search
  • Create a FAQ answer to explain that this is working, that auto tagging must be enabled, etc.
  • Create a FAQ answer to explain how to track Campaigns in Piwik #1042
  • Update the Goal Tracking documentation and clarify that Goal conversions will be reported also for each Adword referer (ie. 'Adwords / keyword 'web analytics' has convertion rate 5%). Put a screenshot in the doc.

Any other suggestion / idea, please comment

@mattab
Owner

on a related note...

@mattab
Owner
@mattab
Owner

Claryfying & Restricting the scope of this ticket

This was the previous ticket description:

Piwik currently support campaigns and keywords for campaigns, but this support hasn't been tested with Adwords use case. This ticket is mainly about testing the support with real data, making sure that adword campaigns, when url tagging is enabled (see in adwords), piwik will detect the campaign name, the keyword from the referer or pick it up from the tagged URL, etc.

'''Draft'''

You need to enable auto tagging in your adwords campaign. "Google Analytics makes it easy to track your AdWords referrals effortlessly using the auto-tagging feature. Auto-tagging is enabled by default when you link your AdWords and Analytics accounts. To turn auto-tagging on or off at any time, update the checkbox in your Account Preferences from your AdWords My Account tab. Using auto-tagging allows you to bypass the steps to tag the URLs of your AdWords advertisements."

In Piwik, you can tag your links with two variables:

    * Campaign name (identifier of the campaign). This parameter is by default called piwik_campaign
    * Campaign detail (for example, the banner name). This parameter is by default called piwik_kwd 

When you enable auto tagging in adwords specifying the parameters names piwik_campaign and piwik_kwd, your visits coming from Adwords will be tracked in Referers > Campaigns. You will see how many visits, bounce rate, page views for each campaign and keyword. Also, if you have setup goals, you can see conversion rate and revenue per goal, for all your Adwords campaigns.

'''Parameter names'''

You can change the parameter names by setting the following in the config/config.ini.php and replacing by the name you wish to use.

; variable name to track the campaign name
campaign_var_name = piwik_campaign
; variable name to track the campaign detail (eg. keyword)
campaign_keyword_var_name = piwik_kwd


'''Note: missing feature'''

"when i track campaigns via "piwik_campaign" from adwords i got only the data from the campaign query but not the real searched keyword (referer)." Explanation: currently when a campaign is detected, it doesn't try to load the keyword from the referer. This is a bug (or missing feature). What we can do is: when the campaign keyword is not specified in the landing URL, we can automatically load the keyword from the referer if found. 


'''Other'''

We also need a user guide on 
 * How to use campaigns in Piwik? (see [604 documentation example for tracking newsletter]) 

See also #476 for tracking adsense ads or yahoo content network.

@diosmosis
Collaborator

(In [7811]) Refs #517, search for campaign keywords in referrer URL when not supplied by query param and use referrer hostname when no keywords in referrer URL and nothing supplied in query param.

Note: also includes quick tweak to integration test base for more descriptive error when expected file cannot be found.

@mattab
Owner

(In [7812]) Fixes #517 if a campaign name is set, we will try to fill in the keyword with the publisher Url (when visitor came from adsense ad) or with user keyword (when coming from adwords)

@michalfapso

I have also problems with tracking AdWords campaigns. This issue is closed, but the problem seems to be not solved yet. I see lots of "googleads.g.doubleclick.net" referrers in my statistics. They are not detected as campaigns. The referrer URLs are of the form in #476. There are many parameters in the link, but parsing at least the "url=..." would be nice.

I would also like to help with coding, just tell me what would be the best way to implement this in Piwik.

Thanks a lot,
Michal

@michalfapso

I have found a workaround:

--- core/Tracker/Visit.php.orig 2013-10-14 10:42:00.000000000 +0200
+++ core/Tracker/Visit.php  2013-10-15 14:46:16.411174133 +0200
@@ -1703,6 +1703,12 @@
             }
         }

+        if (empty($campaignName) &&
+            strpos($this->refererHost, 'doubleclick') !== false
+        ) {
+            $campaignName = "AdWords";
+        }
+
         if (!empty($campaignName)) {
             $this->typeRefererAnalyzed = Piwik_Common::REFERER_TYPE_CAMPAIGN;
             $this->nameRefererAnalyzed = $campaignName;

The original code did not work because the pk_campaign parameter was not set for our AdWords campaigns. This way, every visit from doubleclick host is considered as the AdWords campaign and the referring sites are encoded as campaign keywords.

@mattab
Owner

thanks for the suggestion. Could you send me an example of a Adwords URL and an example of a Adsense URL ?

I think it would be best to set the campaign name to either Adwords or Adsense depending on the referrer URL.

@michalfapso

Thanks Matt, I sent you the URL examples to your email. Don't know if there is any secure info in the url parameters.

There is an url parameter "adsense_enabled", which is always true in our links, so I don't have any examples of Adwords (no Adsense) urls.

@mattab
Owner

In d542986: Fixes #4261 Refs #517 Properly attribute to Adwords those visitors that come from a click on adwords

@mattab
Owner

see also #4468 when &gclid= parameter found, mark visitor as Adwords

@mattab mattab added this to the 1.11 - Piwik 1.11 milestone
@mattab mattab self-assigned this
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@mattab mattab Fixes #4261 Refs #517 Properly attribute to Adwords those visitors th…
…at come from a click on adwords
d542986
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.