about how to estimate the intervention effection

[luoruisichuan]

for the R code, for the intervention of recommending algorithem, the effect on CTR is as following

naive_observational_estimate <- function(user_visits){

Naive observational estimate

Simply the fraction of visits that resulted in a recommendation click-through.

est =
summarise(user_visits, naive_estimate=sum(is_rec_visit)/length(is_rec_visit))
return(est)
}

in the beforehand code ,the estimate formulas is
"""
sum(is_rec_visit)/length(is_rec_visit)
"""
but how to derivative and get this formuls?
thanks

[amit-sharma]

If I understand correctly, you are asking how the naive effect formula is derived.

The naive formula is simply the observational estimate: if the question is what is the effect of recommendations on clicks-through to a page P, we can simply count all the visits to page P that came through recommendations (is_rec_visit) and then divide by the total number of visits to that page P.

More details: To derive the formula, note that the naive effect corresponds to P(Click|Recommendation=True). This can be approximated by simply estimating the fraction: when Recommendation is True (recommendations are shown), count the number of events where a page was accessed via a recommendation, and when it was accessed directly.

Total number of visits to a page (T): length(is_rec_visit) or [length of any other column grouped by the page]
Accessed via a recommendation (Clicked on recommendation) (C): sum(is_rec_visit==1)
Accessed directly (D): length(is_rec_visit) - sum(is_rec_visit==1)

Hence, P(Clicked on recommendation| Recommendation=True) = C/T

Hope this helps @luoruisichuan

[luoruisichuan]

thanks for your help very much!
if I assume that "Accessed directly (D)" is constant, may I use following formula to estimate the naive effect?
P(Clicked on recommendation| Recommendation=True) = C/D=C/(T-C)

thanks for your help very much!

[amit-sharma]

That may not work, since it is not necessary that accessed directly (D) is always higher than C. For some new products, it is possible that C>D and the probability will >1.

But I see what you are saying. An alternative way is to compute the fraction of shown recommendations that got clicked. So if you have the number of times a recommendation was shown (S), you can compute naive effect as C/S where C is the number of clicks on those recommendations.

[luoruisichuan]

thanks for your help very much.