/
atom.xml
211 lines (170 loc) · 19.8 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title><![CDATA[Category: javascript | valve's]]></title>
<link href="http://valve.github.io/blog/categories/javascript/atom.xml" rel="self"/>
<link href="http://valve.github.io/"/>
<updated>2014-02-23T18:53:13+04:00</updated>
<id>http://valve.github.io/</id>
<author>
<name><![CDATA[valve]]></name>
</author>
<generator uri="http://octopress.org/">Octopress</generator>
<entry>
<title type="html"><![CDATA[anonymous browser fingerprinting]]></title>
<link href="http://valve.github.io/blog/2013/07/14/anonymous-browser-fingerprinting/"/>
<updated>2013-07-14T12:29:00+04:00</updated>
<id>http://valve.github.io/blog/2013/07/14/anonymous-browser-fingerprinting</id>
<content type="html"><![CDATA[<h2>What is fingerprinting?</h2>
<p>Fingerprinting is a technique, outlined in the <a href="https://panopticlick.eff.org/browser-uniqueness.pdf">research by Electronic Frontier Foundation</a>, of anonymously identifying a web browser with accuracy of up to 94%.</p>
<p>Browser is queried its agent string, screen color depth, language, installed plugins with supported mime types, timezone offset and other capabilities, such as local storage and session storage. Then these values are passed through a hashing function to produce a fingerprint that gives weak guarantees of uniqueness.</p>
<p>No cookies are stored to identify a browser.</p>
<p>It’s worth noting that a mobile share of browsers is much more uniform, so fingerprinting should be used only as a supplementary identifying mechanism there.</p>
<p>In this post I’m going to explain how it works in detail and give you real-life statistics accumulated over the period of 4 months of production usage.</p>
<!--more-->
<p></p>
<h3>Why</h3>
<p>I was given an experimental task to implement the fingerprinting for both anonymous and logged-in users of one of our web sites. We wanted to see if it was possible at all to rely on identifying someone this way and not leave cookies. The idea was to accumulate the fingerprints and associated preferences and then pre-filter the information on front page based on what’s known about a user.</p>
<h3>Implementation</h3>
<p>So I got to work and started making a basic outline in my head. What is that identifies a browser? I gathered it would be:
<em>browser agent, browser language, screen color depth, installed plugins and their mime types, timezone offset, local storage, and session storage.</em></p>
<p>Initially I added the screen resolution as well, but a colleague adviced that one can use multiple monitors with a single laptop, for example connect an external monitor when working in office, so I removed it.</p>
<p>On my laptop browser the values are:</p>
<p><div class='bogus-wrapper'><notextile><figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
<span class='line-number'>16</span>
<span class='line-number'>17</span>
<span class='line-number'>18</span>
<span class='line-number'>19</span>
<span class='line-number'>20</span>
<span class='line-number'>21</span>
<span class='line-number'>22</span>
<span class='line-number'>23</span>
<span class='line-number'>24</span>
<span class='line-number'>25</span>
<span class='line-number'>26</span>
<span class='line-number'>27</span>
<span class='line-number'>28</span>
<span class='line-number'>29</span>
<span class='line-number'>30</span>
<span class='line-number'>31</span>
<span class='line-number'>32</span>
<span class='line-number'>33</span>
<span class='line-number'>34</span>
<span class='line-number'>35</span>
<span class='line-number'>36</span>
<span class='line-number'>37</span>
<span class='line-number'>38</span>
<span class='line-number'>39</span>
<span class='line-number'>40</span>
<span class='line-number'>41</span>
<span class='line-number'>42</span>
<span class='line-number'>43</span>
<span class='line-number'>44</span>
<span class='line-number'>45</span>
<span class='line-number'>46</span>
<span class='line-number'>47</span>
<span class='line-number'>48</span>
<span class='line-number'>49</span>
<span class='line-number'>50</span>
<span class='line-number'>51</span>
<span class='line-number'>52</span>
<span class='line-number'>53</span>
<span class='line-number'>54</span>
<span class='line-number'>55</span>
<span class='line-number'>56</span>
</pre></td><td class='code'><pre><code class='javascript'><span class='line'><span class="c1">// Assuming jQuery in scope</p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">></span><span class="nx">navigator</span><span class="p">.</span><span class="nx">userAgent</span>
</span><span class='line'><span class="c1">// &ldquo;Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36&rdquo;</p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">></span><span class="nx">navigator</span><span class="p">.</span><span class="nx">language</span>
</span><span class='line'><span class="c1">// &ldquo;en-US&rdquo;</p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">></span><span class="kd">var</span> <span class="nx">plugins</span> <span class="o">=</span> <span class="nx">$</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">navigator</span><span class="p">.</span><span class="nx">plugins</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">p</span><span class="p">){</span>
</span><span class='line'> <span class="kd">var</span> <span class="nx">mimeTypes</span> <span class="o">=</span> <span class="nx">$</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">p</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">mimeType</span><span class="p">){</span><span class="o"><</span><span class="err">/p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">pre</span><span class="o">><</span><span class="nx">code</span><span class="o">></span><span class="k">return</span> <span class="p">[</span><span class="nx">mimeType</span><span class="p">.</span><span class="nx">type</span><span class="p">,</span> <span class="nx">mimeType</span><span class="p">.</span><span class="nx">suffixes</span><span class="p">].</span><span class="nx">join</span><span class="p">(</span><span class="s1">'~'</span><span class="p">);</span>
</span><span class='line'><span class="o"><</span><span class="err">/code></pre></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">></span> <span class="p">}).</span><span class="nx">join</span><span class="p">(</span><span class="o">&</span><span class="nx">lsquo</span><span class="p">;,</span><span class="o">&</span><span class="nx">rsquo</span><span class="p">;);</span>
</span><span class='line'> <span class="k">return</span> <span class="p">[</span><span class="nx">p</span><span class="p">.</span><span class="nx">name</span><span class="p">,</span> <span class="nx">p</span><span class="p">.</span><span class="nx">description</span><span class="p">,</span> <span class="nx">mimeTypes</span><span class="p">].</span><span class="nx">join</span><span class="p">(</span><span class="o">&</span><span class="nx">lsquo</span><span class="p">;</span><span class="o">::&</span><span class="nx">rsquo</span><span class="p">;);</span>
</span><span class='line'><span class="p">});</span><span class="o"><</span><span class="err">/p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">></span><span class="nx">$</span><span class="p">.</span><span class="nx">each</span><span class="p">(</span><span class="nx">plugins</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">i</span><span class="p">,</span> <span class="nx">p</span><span class="p">){</span>
</span><span class='line'> <span class="c1">// truncate only for blog example</span>
</span><span class='line'> <span class="k">if</span><span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">length</span> <span class="o">></span> <span class="mi">80</span><span class="p">){</span><span class="o"><</span><span class="err">/p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">pre</span><span class="o">><</span><span class="nx">code</span><span class="o">></span><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">p</span><span class="p">.</span><span class="nx">substring</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">77</span><span class="p">)</span> <span class="o">+</span> <span class="s1">'...'</span><span class="p">);</span>
</span><span class='line'><span class="o"><</span><span class="err">/code></pre></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">></span> <span class="p">}</span> <span class="k">else</span><span class="p">{</span><span class="o"><</span><span class="err">/p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">pre</span><span class="o">><</span><span class="nx">code</span><span class="o">></span><span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="nx">p</span><span class="p">);</span>
</span><span class='line'><span class="o"><</span><span class="err">/code></pre></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">></span> <span class="p">}</span>
</span><span class='line'><span class="p">});</span><span class="o"><</span><span class="err">/p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">></span><span class="err">/<em></span>
</span><span class='line'><span class="nx">Shockwave</span> <span class="nx">Flash</span><span class="o">:</span><span class="nx">Shockwave</span> <span class="nx">Flash</span> <span class="mf">11.7</span> <span class="nx">r700</span><span class="o">:</span><span class="nx">application</span><span class="o">/</span><span class="nx">x</span><span class="o">-</span><span class="nx">shockwave</span><span class="o">-</span><span class="nx">flash</span><span class="o">~</span><span class="nx">swf</span><span class="p">,</span><span class="nx">a</span><span class="o">&</span><span class="nx">hellip</span><span class="p">;</span>
</span><span class='line'><span class="nx">Chrome</span> <span class="nx">Remote</span> <span class="nx">Desktop</span> <span class="nx">Viewer</span><span class="o">:</span><span class="nx">This</span> <span class="nx">plugin</span> <span class="nx">allows</span> <span class="nx">you</span> <span class="nx">to</span> <span class="nx">securely</span> <span class="nx">access</span> <span class="nx">other</span> <span class="o">&</span><span class="nx">hellip</span><span class="p">;</span>
</span><span class='line'><span class="nx">Widevine</span> <span class="nx">Content</span> <span class="nx">Decryption</span> <span class="nx">Module</span><span class="o">:</span><span class="nx">Enables</span> <span class="nx">Widevine</span> <span class="nx">licenses</span> <span class="k">for</span> <span class="nx">playback</span> <span class="nx">of</span> <span class="o">&</span><span class="nx">hellip</span><span class="p">;</span>
</span><span class='line'><span class="nx">Native</span> <span class="nx">Client</span><span class="o">::</span><span class="nx">application</span><span class="o">/</span><span class="nx">x</span><span class="o">-</span><span class="nx">nacl</span><span class="o">~</span><span class="nx">nexe</span>
</span><span class='line'><span class="nx">Chrome</span> <span class="nx">PDF</span> <span class="nx">Viewer</span><span class="o">::</span><span class="nx">application</span><span class="o">/</span><span class="nx">pdf</span><span class="o">~</span><span class="nx">pdf</span><span class="p">,</span><span class="nx">application</span><span class="o">/</span><span class="nx">x</span><span class="o">-</span><span class="nx">google</span><span class="o">-</span><span class="nx">chrome</span><span class="o">-</span><span class="nx">print</span><span class="o">-</span><span class="nx">prev</span><span class="o">&</span><span class="nx">hellip</span><span class="p">;</span>
</span><span class='line'><span class="nx">Google</span> <span class="nx">Talk</span> <span class="nx">Plugin</span> <span class="nx">Video</span> <span class="nx">Accelerator</span><span class="o">:</span><span class="nx">Google</span> <span class="nx">Talk</span> <span class="nx">Plugin</span> <span class="nx">Video</span> <span class="nx">Accelerator</span> <span class="nx">ver</span><span class="o">&</span><span class="nx">hellip</span><span class="p">;</span>
</span><span class='line'><span class="nx">Google</span> <span class="nx">Talk</span> <span class="nx">Plugin</span><span class="o">:</span><span class="nx">Version</span><span class="o">:</span> <span class="mf">4.0</span><span class="p">.</span><span class="mf">1.0</span><span class="o">:</span><span class="nx">application</span><span class="o">/</span><span class="nx">googletalk</span><span class="o">~</span><span class="nx">googletalk</span>
</span><span class='line'><span class="nx">Google</span> <span class="nx">Talk</span> <span class="nx">Plugin</span> <span class="nx">Video</span> <span class="nx">Renderer</span><span class="o">:</span><span class="nx">Version</span><span class="o">:</span> <span class="mf">4.0</span><span class="p">.</span><span class="mf">1.0</span><span class="o">:</span><span class="nx">application</span><span class="o">/</span><span class="nx">o1d</span><span class="o">~</span><span class="nx">o1d</span>
</span><span class='line'><span class="nx">Shockwave</span> <span class="nx">Flash</span><span class="o">:</span><span class="nx">Shockwave</span> <span class="nx">Flash</span> <span class="mf">11.2</span> <span class="nx">r202</span><span class="o">:</span><span class="nx">application</span><span class="o">/</span><span class="nx">x</span><span class="o">-</span><span class="nx">shockwave</span><span class="o">-</span><span class="nx">flash</span><span class="o">~</span><span class="nx">swf</span><span class="p">,</span><span class="nx">a</span><span class="o">&</span><span class="nx">hellip</span><span class="p">;</span>
</span><span class='line'><span class="o"><</span><span class="sr">/em>/</span><span class="o"><</span><span class="err">/p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">></span><span class="nx">screen</span><span class="p">.</span><span class="nx">colorDepth</span>
</span><span class='line'><span class="c1">// 24</p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">></span><span class="k">new</span> <span class="nb">Date</span><span class="p">().</span><span class="nx">getTimezoneOffset</span><span class="p">();</span>
</span><span class='line'><span class="c1">// -240</p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">>!!</span><span class="nb">window</span><span class="p">.</span><span class="nx">localStorage</span>
</span><span class='line'><span class="c1">// true</p></span>
</span><span class='line'>
</span><span class='line'><span class="o"><</span><span class="nx">p</span><span class="o">>!!</span><span class="nb">window</span><span class="p">.</span><span class="nx">sessionStorage</span>
</span><span class='line'><span class="c1">// true</span>
</span></code></pre></td></tr></table></div></figure></notextile></div></p>
<p>So I now knew all my browser had, and I needed to produce the fingerprint itself.
For that I wanted to use a fast, non-cryptographic hashing function, such as <a href="http://en.wikipedia.org/wiki/MurmurHash%E2%80%8E">murmur hashing</a>.</p>
<p>Murmur hashing produces 32-bit integer as a result and works really well. <a href="http://programmers.stackexchange.com/questions/49550/which-hashing-algorithm-is-best-for-uniqueness-and-speed">When compared to other popular hash functions, MurmurHash performed well in a random distribution of regular keys.</a></p>
<p>I picked <a href="http://github.com/garycourt/murmurhash-js">this implementation</a> and added it to the code.</p>
<p>The last step was to combine all browser’s capabilities into a long string and pass it through hashing.</p>
<p>The end result on my laptop was: <code>3723825959</code></p>
<p>As a finishing touch, I wanted to get rid of jQuery, so I implemented the <code>each</code> and <code>map</code> methods and got a no-dependencies script.</p>
<h4>How to improve accuracy?</h4>
<p>The above research states that the identification accuracy is surprisingly high. But to improve it even further, Flash or Java integration is required to get a list of installed fonts, thus making each browser even more unique.</p>
<h4>What about hash collisions?</h4>
<p>My tests show that for random strings Murmurh hashing indeed produces collisions, but their number is negligible for my purposes: 5-7 collisions per ~200K of capabilities strings.</p>
<h4>What about mobile browsers?</h4>
<p>It’s simple: browser fingerprinting is not good with mobile browsers, unless you want to distinguish Android users from iPhone ones.</p>
<h3>Results</h3>
<p>After having had the fingerprinting on production for 4 months, I have some data to analyze. First of all, let me say that I’m not at liberty to tell the exact number of visitors to the web site, but I can say it is several millions a month, so we have some data to play with. All numbers below represent our usage and do not represent what you might have.</p>
<p><strong>89%</strong> of fingerprints are unique</p>
<p><strong>20%</strong> of our users have more than one fingerprint, i.e. several browsers or devices.</p>
<p>Very few users have a staggering amount of fingerprints, for example 20-25. I don’t know if they have a lot of devices, use different browsers or something else.</p>
<p>After viewing the results we removed the fingerprinting because of poor identification, especially with mobile devices.
If your traffic mostly comes from desktops and you’re OK with 10-12% of false identifications you might want to try it.</p>
<h3>Show me the code</h3>
<h4><a href="https://github.com/Valve/fingerprintjs">code on github</a> – the version I had in production</h4>
<h4><a href="http://valve.github.io/fingerprintjs/">test your browser</a></h4>
]]></content>
</entry>
</feed>