The algorithm in use basically says that the weight given to a variant is equal to its share of past goals plus a declining constant. The declining constant is there to boost weights of all variants above zero but less as more data comes in. This is an ad-hoc algorithm with many alternatives, a true genetic algorithm being one of them. I plan on factoring out the algorithm part of genetify so people can swap in their algorithm of choice.
Yes, this is a feature of genetify that is turned on by default. The demo is perhaps confusing because it randomizes with every pageview. It wouldn’t be much fun otherwise :)
Genetify is designed to fail gracefully. Even if the server is completely absent, users will get a sensible default experience, because all the code for rendering is in the HTML document. The server only deals with results and optimization weights. All variants are hidden until genetify.vary is called. Until then, you just have a normal webpage.
As much as I love stats, you shouldn’t need to learn them to use genetify. It can be thought of simply as getting a computer to figure out what is best to achieve a goal. If the goal is achieved more often after optimization than before, you know it has worked.
It has been running for over a year on a couple of websites — meaningoflife.tv and knottyboy.com.
Genetify reports a statistical measure of confidence in observed differences between variants of a gene. You can use it to say things like “We are 86% sure there is a difference between these two buttons.” The confidence is calculated from the F distribution. It is essentially the same as a one-way ANOVA. See the source.
However, Genetify does not use this measure directly in its optimization decision. This is an area that is ripe for improvement by someone who knows more about statistical optimization than I. You can do your own analysis or optimization with the basic numbers contained in genetify.results.