-
Notifications
You must be signed in to change notification settings - Fork 1
/
snowflake.tex
3004 lines (2831 loc) · 166 KB
/
snowflake.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
\documentclass[letterpaper,twocolumn]{article}
\usepackage{bigdelim}
\usepackage[top=1in,bottom=1in,left=0.75in,right=0.75in]{geometry}
\usepackage{graphicx}
\usepackage{makecell}
\usepackage{multirow}
\usepackage{newtxtext}
\usepackage{newtxmath}
\usepackage{pifont} % For checkmarks in tab:nat-matching.
\usepackage{titling}
\usepackage[hyphens]{url}
\usepackage[table]{xcolor}
\usepackage[textsize=footnotesize]{todonotes}\setlength{\marginparwidth}{1.5cm}
% Better look for citations that include a section reference like \cite[\S 3]{foobar}.
\usepackage{cite}
\renewcommand{\citemid}{~}
% Load hyperref after other packages.
\usepackage[pdfa,hidelinks,pdfusetitle,pdfcreator={},pdfproducer={}]{hyperref}
\urlstyle{same}
\def\sectionautorefname{Section}
\def\subsectionautorefname{Section}
% Disable metadata for reproducible PDF.
% https://tex.stackexchange.com/a/313605
\usepackage{ifpdf}
\ifpdf
\pdfinfoomitdate=1
\pdftrailerid{}
\pdfsuppressptexinfo=-1
\fi
\hyphenation{Web-RTC}
\hyphenation{Web-Exten-sion}
\hyphenation{Java-Script}
\hyphenation{uProxy}
\hyphenation{off-line}
\hyphenation{mac-OS}
% Highlight the first usage or definition of a technical term.
% Like the firstterm element in DocBook: https://tdg.docbook.org/tdg/5.2/firstterm.html.
\newcommand{\firstterm}[1]{\textit{#1}}
\begin{document}
\date{}
\title{Snowflake, a censorship circumvention system\\using temporary WebRTC proxies\\\strut\large (Draft \today)}
\author{%
Cecylia Bocovich%
\and
Arlo Breault%
\and
David Fifield%
\and
Serene%
\and
Xiaokang Wang%
}
\renewcommand{\maketitlehookc}{\centering\normalsize Authors are listed alphabetically.}
\maketitle
\begin{abstract}
Snowflake is a system for circumventing Internet censorship.
Its blocking resistance comes from
the use of numerous, ultra-light, temporary proxies (``snowflakes''),
which accept traffic from censored clients using peer-to-peer WebRTC protocols
and forward it to a centralized bridge.
The temporary proxies are simple enough to be implemented in JavaScript,
in~a web page or browser extension,
making them vastly cheaper to run than
a traditional proxy or VPN server.
The large and constantly changing pool
of proxy addresses resists enumeration and blocking by a censor.
The system is built on the assumption
that proxies may appear or disappear at any time:
clients discover live proxies dynamically
using a secure rendezvous protocol;
when an \mbox{in-use} proxy goes offline,
its client switches to another on the fly,
invisibly to upper network layers.
Snowflake has been deployed with success
in Tor Browser and Orbot for several years.
It has been a significant circumvention tool
during high-profile network disruptions,
including in Russia in~2021 and Iran in~2022.
In this paper, we explain the composition of Snowflake's many parts,
give a history of deployment and blocking attempts,
and reflect on implications for circumvention generally.
\end{abstract}
% General references:
% https://keroserene.net/snowflake/technical/#history
% https://www.bamsoftware.com/papers/thesis/#chap:snowflake
\section{Introduction}
\label{sec:intro}
Censorship circumvention systems---systems
to enable network communication
despite interference by a censor---may
be characterized on multiple axes.
Some systems imitate a common network protocol;
others try
not to look like any protocol in particular.
Some distribute connections over numerous proxy servers;
others concentrate on a single proxy
that is, for one reason or another, difficult for a censor to block.
What all circumvention systems have in common
is that they strive to increase the \emph{cost}
to the censor of blocking them---whether that cost be in
research and development, human resources, and hardware;
or in the inevitable overblocking that results
when a censor tries to selectively block
some connections but not others.
Snowflake, the subject of this paper,
is a circumvention system that
uses thousands of temporary proxies
and makes switching between them easy and fast.
On~the spectrum of imitation to randomization,
Snowflake falls on the side of imitation;
on the scale of diffuse to concentrated, it is diffuse.
What characterizes Snowflake the most is that
it pushes the idea of distributed, disposable
proxies to an extreme:
its~proxies can run in a web browser,
and censored clients communicate with them using WebRTC.
WebRTC is a suite of protocols
intended for real-time communication applications
on the web~\cite{rfc8825}.
Video and voice chat are typical examples
of WebRTC applications.
Snowflake exchanges WebRTC data formats
in the course of establishing a connection,
and uses WebRTC protocols for traversal of NAT (network address translation)
and communication between clients and proxies.
Crucially for Snowflake, WebRTC APIs
are available to JavaScript code in web browsers,
meaning it is possible to implement a proxy
in a web page or browser extension.
WebRTC is also usable outside a browser,
which is how we implement the Snowflake client program
and alternative, command line--based proxies.
As~is usual in circumvention research,
we assume a threat model in which
\firstterm{clients} reside in a network
controlled by a \firstterm{censor}.
The censor has the power to inspect and interfere with
traffic that crosses the border of its network;
typical real-world censor behaviors include
inspecting IP addresses and hostnames,
checking packet contents for keywords,
blocking IP addresses, and injecting false DNS responses
or TCP RST packets.
The client wants to communicate with some
\firstterm{destination} outside the censor's network,
possibly with the aid of third-party \firstterm{proxies}.
The censor is motivated to block the contents
of the client's communication, or even the destination itself.
The censor is aware of the possibility of circumvention,
and therefore seeks to block not only direct communication,
but also indirect communication by way of a proxy or circumvention system.
Circumvention is accomplished when the client
can reliably reach any proxy,
because a proxy, being outside the censor's control,
can then forward the client's communication to any destination.
(In Snowflake, we separate the roles of temporary \firstterm{proxies}
and a stable long-term \firstterm{bridge}, but the idea is the same.)
The censor is presumed to derive benefit
from permitting some forms of network access:
the censor cannot trivially ``win''
by shutting down all communication,
but must be selective in its blocking decisions,
in order to optimize some objective of its own.
The art of censorship circumvention is
forcing the censor into a dilemma
of~overblocking or underblocking,
by making circumvention traffic difficult to distinguish
from traffic that the censor prefers not to block.
Snowflake originates in two earlier projects:
flash proxy and uProxy.
% https://lists.torproject.org/pipermail/tor-dev/2016-January/010310.html "Snowflake is a webrtc pluggable transport inspired by flashproxy."
% https://keroserene.net/snowflake/technical/#1-introduction "It is inspired by and builds upon the previous work of Flashproxy. Snowflake is much like a hybrid of previous Pluggable Transports..."
Flash proxy~\cite{Fifield2012a}, like Snowflake, used a model
of untrusted, temporary JavaScript proxies in web browsers,
but the link between client and proxy used WebSocket
rather than WebRTC.
(WebSocket still finds use in Snowflake,
but on the proxy--bridge link,
not the client--proxy link.)
Flash proxy was deployed in Tor Browser
% 2013: https://blog.torproject.org/combined-flash-proxy-pyobfsproxy-browser-bundles
% 2016: "Remove Flashproxy from Tor Browser" https://bugs.torproject.org/17428#note_2203210
from 2013 to~2016,
but never saw much use,
probably because the reliance on WebSocket,
which lacks the built-in NAT traversal of WebRTC,
required client users to do their own port forwarding.
% https://gitlab.torproject.org/legacy/trac/-/wikis/doc/PluggableTransports/FlashProxy/Howto
WebRTC was then an emerging technology, and while
% "Investigate WebRTC for flash proxy NAT punching" https://bugs.torproject.org/5578
% flashproxy.pdf §5.2: "New technologies like WebRTC [24] may fill this need in the future, if they become sufficiently popular that flash proxies' use of them does not stand out as unusual."
it had been considered as a transport protocol for flash proxy,
we decided to start Snowflake as an independent project.
% https://serene.cx/snowflake/#note-flashproxy "...one could say that uProxy and flashproxy are the ancestors of snowflake."
uProxy~\cite{uproxy}, in one of its early incarnations,
% "in one of its earlier incarnations": uProxy pivoted from friend proxies to cloud servers in 2016:
% https://web.archive.org/web/20161211194847/https://blog.uproxy.org/2016/02/get-access-24x7-through-your-own-uproxy.html
% and added support for proxying through Tor in 2016:
% https://lists.torproject.org/pipermail/tor-dev/2016-September/011489.html
pioneered the use of WebRTC proxies for circumvention.
uProxy's proxies were browser-based,
but its trust and deployment models were different
from flash proxy's and Snowflake's.
Each censored client would arrange, out of band,
for an acquaintance outside the censor's network
to run a proxy in their browser.
% uProxy v1.2.5 Design Doc: https://docs.google.com/document/d/1t_30vX7RcrEGuWwcg0Jub-HiNI0Ko3kBOyqXgrQN3Kw
% "uProxy depends on leveraging existing trust relationships to to find and use a proxy."
A~personal trust relationship was necessary to prevent misuse,
since browser proxies fetched destination content directly---meaning
the client's activity would be attributed to the proxy,
and the proxy could inspect the client's traffic.
Clients did not change proxies on the fly.
uProxy supported protocol obfuscation:
the communications protocol was fundamentally WebRTC,
but packets could be transformed to resemble something else.
% https://github.com/uProxy/uproxy-obfuscators
This obfuscation was possible because of
uProxy's implementation as a privileged browser extension,
with access to real sockets.
Because Snowflake uses ordinary unprivileged browser APIs,
its WebRTC can only look like WebRTC;
on the other hand, for the same reason,
Snowflake proxies are easier to deploy.
Like flash proxy, uProxy was active in the years
2013--2016.
% 2013: Serene's 30C3 lightning talk on uProxy https://events.ccc.de/congress/2013/wiki/Static:Lightning_Talks#Day_3
% 2013–2016 contributions in uProxy-p2p: https://github.com/UWNetworksLab/uProxy-p2p/graphs/contributors
Among existing circumvention systems,
the one that is most similar to Snowflake is MassBrowser~\cite{Nasr2020a},
% reading group summary of MassBrowser https://github.com/net4people/bbs/issues/32
which offers
proxying though volunteer proxies, called buddies.
MassBrowser's architecture is similar to Snowflake's:
there is a centralized component that coordinates
connections between clients and buddies,
corresponding to a piece in Snowflake called the broker;
buddies play the same role as our proxies.
The~trust model is intermediate between Snowflake's and uProxy's.
Buddies preferentially operate as one-hop proxies, as in uProxy,
but are not limited to proxying only for trusted friends.
To~deter misuse, buddies specify a policy of
what categories of content they are willing to proxy.
An~innovation in MassBrowser not present in Snowflake is client-to-client proxying:
clients may act as buddies for other clients,
the logic being that what is censored for one client may not be censored for another.
The buddy software is
not constrained by a web browser environment,
and can, like uProxy, use protocol obfuscation
on the client--buddy link.
% V-D: "We also implement traffic obfuscation to protect MassBrowser's traffic
% against traffic analysis attacks. Particularly, we have built a custom
% implementation of the obfsproxy Tor pluggable transport tailored to work with
% our MassBrowser implementation."
% VI-A: "MassBrowser uses a custom protocol over TCP/UDP for the communications
% between Clients and Buddies."
Protozoa~\cite{Barradas2020a}
% reading group summary of Protozoa https://github.com/net4people/bbs/issues/55
and Stegozoa~\cite{Figueira2022a}
show ways of building a point-to-point covert tunnel over WebRTC,
the former by directly replacing encoded media
with its own ciphertexts,
the latter using video steganography.
Designs like these might serve as alternatives
for the link between client and proxy in Snowflake.
% They would need to be made to run in an unmodified browser, which may be possible:
% "WebRTC Encoded Transform (or Insertable Streams) for media channels in Snowflake?" https://lists.torproject.org/pipermail/anti-censorship-team/2023-February/000284.html
% "Insertable streams" https://dl.acm.org/doi/pdf/10.1145/3488932.3517419#page=12
Significantly, where Snowflake now uses WebRTC data channels,
Protozoa and Stegozoa use WebRTC media streams,
which may be an advantage in blocking resistance.
We will say more on this point in \autoref{sec:fingerprinting}.
\todo{Cite TorKameleon~\cite{arxiv.2303.17544}.}
% Very early versions of Lantern (circa 2014) used social network–based trusted proxies:
% https://web.archive.org/web/20140326223853/http://techpresident.com/news/wegov/24455/why-remarkably-similar-circumvention-tools-uproxy-and-lantern-are-not-overkill
% https://lists.torproject.org/pipermail/tor-dev/2014-March/006356.html "HOWTO use Lantern as a pluggable transport"
% https://web.archive.org/web/20130831160152/https://www.youtube.com/watch?v=aiPkCugE-RY
% https://web.archive.org/web/2oe_/http://wayback-fakeurl.archive.org/yt/aiPkCugE-RY
% But it wasn't WebRTC, so was less like Snowflake than uProxy was.
% I'll draw the line here, since even Tor bridges are "volunteer-operated" in a sense.
Our goal in this paper
is not to exaggerate the advantages of Snowflake,
nor disproportionately emphasize the limitations
of other circumvention systems.
Circumvention research is a cooperative enterprise,
and we recognize and support our colleagues who
pursue and maintain their own designs.
While challenges remain,
today's circumvention systems by and large
accomplish their intended purpose,
and are a vital element of day-to-day Internet access for many people.
With Snowflake, we have explored a different point in the design space---a~fruitful
one to be sure---but one with its own tradeoffs.
We~acknowledge that Snowflake will be a better choice in some
censorship environments and
worse in others; indeed,
one of the ideas we hope to convey
is that blocking resistance
can be meaningfully understood only in relation to particular censor
and its resources, costs, and motivations.
In~this paper we present the design of Snowflake,
discuss various challenges and considerations,
and reflect on over three years of deployment.
As~of January 2024, Snowflake supports an estimated 40,000 average concurrent users
% > library("tidyverse")
% > WANTED_FINGERPRINTS <- c(
% "7659DA0F96B156C322FBFF3ACCC9B9DC01C27C73" = "snowman",
% "5481936581E23D2D178105D44DB6915AB06BFB7F" = "snowflake-01",
% "91DA221A149007D0FD9E5515F5786C3DD07E4BB0" = "snowflake-02"
% )
% > read_csv("figures/users/userstats-bridge-transport-multi.csv") %>%
% filter(transport == "snowflake" & fingerprint %in% names(WANTED_FINGERPRINTS)) %>%
% filter(date < "2024-01-01") %>%
% mutate(users = users / (coverage / pmax(num_instances, coverage))) %>%
% group_by(date, transport) %>% summarize(users = sum(users, na.rm = TRUE), .groups = "drop") %>%
% tail()
% # A tibble: 6 × 3
% date transport users
% <date> <chr> <dbl>
% 1 2023-12-26 snowflake 40513.
% 2 2023-12-27 snowflake 40797.
% 3 2023-12-28 snowflake 40068.
% 4 2023-12-29 snowflake 39154.
% 5 2023-12-30 snowflake 40665.
% 6 2023-12-31 snowflake 41086.
and transfers around 30~TB of circumvention traffic per day.
% > library("tidyverse")
% > WANTED_FINGERPRINTS <- c(
% "7659DA0F96B156C322FBFF3ACCC9B9DC01C27C73" = "snowman",
% "5481936581E23D2D178105D44DB6915AB06BFB7F" = "snowflake-01",
% "91DA221A149007D0FD9E5515F5786C3DD07E4BB0" = "snowflake-02"
% )
% > options(width = 200)
% > userstats <- read_csv("figures/users/userstats-bridge-transport-multi.csv") %>%
% filter(fingerprint %in% names(WANTED_FINGERPRINTS)) %>%
% mutate(users = users / (coverage / pmax(num_instances, coverage)))
% > bandwidth <- read_csv("figures/users/bandwidth-multi.csv") %>%
% filter(fingerprint %in% names(WANTED_FINGERPRINTS)) %>%
% filter(coverage > 0) %>%
% mutate(bytes = bytes / (coverage / pmax(num_instances, coverage))) %>%
% pivot_wider(id_cols = c(date, fingerprint), names_from = c(type), values_from = c(bytes)) %>%
% mutate(
% good_read = read - `dirreq-read`,
% good_write = write - `dirreq-write`,
% good_avg = (good_read + good_write) / 2
% )
% > left_join(userstats, bandwidth, by = c("date", "fingerprint")) %>%
% # Subtract out the pro-rated fraction of non-snowflake transports (basically negligible).
% group_by(date, fingerprint) %>%
% mutate(across(c(read, write, `dirreq-read`, `dirreq-write`, good_read, good_write, good_avg), ~ .x * users / sum(users))) %>%
% ungroup() %>%
% filter(transport == "snowflake") %>%
% filter(date < "2024-01-01") %>%
% group_by(date) %>%
% summarize(
% date = last(date),
% across(c(read, write, `dirreq-read`, `dirreq-write`, good_read, good_write, good_avg), sum, na.rm = TRUE)
% ) %>%
% mutate(across(c(read, write, `dirreq-read`, `dirreq-write`, good_read, good_write, good_avg), scales::label_bytes(units = "auto_si", accuracy = 0.01))) %>%
% arrange(date) %>% tail()
% # A tibble: 6 × 8
% date read write `dirreq-read` `dirreq-write` good_read good_write good_avg
% <date> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
% 1 2023-12-26 34.11 TB 33.98 TB 11.52 GB 228.92 GB 34.10 TB 33.76 TB 33.93 TB
% 2 2023-12-27 33.41 TB 33.29 TB 11.58 GB 237.66 GB 33.40 TB 33.05 TB 33.22 TB
% 3 2023-12-28 33.66 TB 33.54 TB 11.40 GB 239.30 GB 33.65 TB 33.30 TB 33.48 TB
% 4 2023-12-29 34.67 TB 34.54 TB 10.89 GB 223.63 GB 34.66 TB 34.31 TB 34.49 TB
% 5 2023-12-30 33.42 TB 33.28 TB 11.14 GB 228.21 GB 33.41 TB 33.05 TB 33.23 TB
% 6 2023-12-31 32.87 TB 32.74 TB 11.26 GB 231.99 GB 32.86 TB 32.50 TB 32.68 TB
\section{How it works}
\label{sec:mechanics}
\begin{figure*}[t]
\includegraphics{figures/architecture/architecture.jpg}
\caption{
Architecture of Snowflake.
The client contacts the broker through a special rendezvous channel with high blocking resistance.
The broker matches the client with one of the proxies that are currently polling.
The client and proxy connect to one another using WebRTC.
The proxy connects to the bridge,
then begins copying traffic in both directions.
If~the proxy disappears,
the client does another rendezvous
and resumes its session with a new proxy.
}
\label{fig:architecture}
\todo[inline]{Maybe this graphic should show STUN servers?}
\end{figure*}
A~Snowflake proxy connection proceeds in three phases.
First, there is rendezvous, in which a client
indicates its need for circumvention service
and is matched with a temporary proxy.
Rendezvous is facilitated by a central server called the broker.
Then, there is connection establishment,
where the client and its proxy connect to each other
with WebRTC, using information exchanged during rendezvous.
Finally, there is data transfer,
where the proxy transports data
between the client and the bridge.
The bridge is responsible for directing the client's traffic
to its eventual destination
(in~our case, by feeding it into the Tor network).
\autoref{fig:architecture} illustrates the process.
These phases repeat as needed, as temporary proxies come and~go.
Proxy failure is not an abnormal condition---it~happens whenever
a proxy is running in a browser that is closed, for example.
A~client builds a circumvention session over
a sequence of proxies, switching to a new one
whenever the current one stops working.
State variables stored at the client and the bridge
let the session can pick up where it left off.
The change of proxies is invisible to the applications using Snowflake
(except for a brief delay while rendezvous happens):
the Snowflake client presents an abstraction of a single, uninterrupted connection.
It~does not avail a censor to block the broker or bridge,
because Snowflake clients never contact either one directly.
Clients reach the broker over an indirect rendezvous channel.
Access to the bridge is always mediated by a temporary proxy.
\subsection{Rendezvous}
\label{sec:rendezvous}
A~session begins with a client sending a rendezvous message to the broker.
There is an ambient population of proxies
constantly polling the broker to check for clients in need of service.
The broker matches the client with an available proxy,
taking into consideration factors like NAT compatibility.
% NAT type is currently the only constraint:
% matchSnowflake https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/9edaee65470a1483bbdbe984e5e15a885f1e95d2/broker/ipc.go#L236
% But protocol versions may become a consideration in the future:
% "Analysis of speed deficiency of Snowflake in China, 2023 Q1" https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/40251#note_2903271
The client's rendezvous message
% ClientPollRequest https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/9edaee65470a1483bbdbe984e5e15a885f1e95d2/common/messages/client.go#L64
is a bundle of data that the broker will need to match the client with a proxy,
and the proxy will need to connect to the client.
The primary element is a
Session Description Protocol (SDP) \firstterm{offer}~\cite{rfc8839},
which contains the information necessary for a WebRTC connection,
including the client's external IP addresses
and cryptographic data to secure a later key exchange.
% Specifically, a certificate fingerprint: https://www.rfc-editor.org/rfc/rfc8122.html#section-5
The broker forwards the client's SDP offer to the proxy,
and the proxy sends back an SDP \firstterm{answer}
with its share of connection details.
The broker forwards the proxy's SDP answer to the client.
The client and proxy then connect to each other directly.
In WebRTC terms, this offer/\allowbreak answer exchange is called
``signaling,'' and here the broker acts as a signaling server.
To~gather the information for an SDP offer or answer,
clients and proxies communicate with third-party servers,
called STUN servers,
before contacting the broker.
We~will say more about how this information is used in \autoref{sec:connection}.
Communication with STUN servers is a normal and expected part of WebRTC,
though there are fingerprinting considerations
that we discuss in \autoref{sec:fingerprinting}.
Interaction with the broker uses a ``long-polling'' model.
An~example is shown in \autoref{fig:rendezvous}.
Proxies poll the broker periodically,
making an HTTPS request to a designated URL path.
The broker does not respond immediately to a proxy poll,
but instead holds the connection idle for a few seconds
to await the possible arrival of a client rendezvous message.
If~none arrives, the broker sends a response saying ``no clients''
and the proxy goes to sleep until its next poll.
When a client does arrive,
the broker sends the SDP offer in response
to the proxy's poll request.
The proxy sends its SDP answer to the broker
in a separate HTTPS request.
The broker responds to the client's pending request
with the proxy's SDP answer,
at~the same time sending an acknowledgement to the proxy.
At~this point rendezvous is finished,
and the client and the proxy may connect to one another.
\begin{figure}
\includegraphics{figures/rendezvous/rendezvous}
\caption{
The long-polling communication model of Snowflake rendezvous.
Proxies poll periodically to check for new clients.
When the broker makes a match,
the proxy gets the client's SDP offer,
then immediately re-connects to send back its SDP answer.
It~all happens during one round trip from the client's perspective.
Not shown here is the indirect channel
used by the client to access the broker through the censor's zone of control
(shaded background).
}
\label{fig:rendezvous}
\end{figure}
The client must use an indirect,
blocking-resistant channel
when communicating with the broker.
What is needed, essentially,
is a miniature circumvention system
to bootstrap the full system.
What makes rendezvous
different from general circumvention
are its different (generally more lenient) requirements,
which permit a larger solution space.
Because rendezvous is only a small fraction
of total communication volume,
and it happens relatively infrequently,
it~may use techniques that would be
too slow, expensive, or complicated
for real-time or bulk data transfer.
Rendezvous is separable and modular:
more than one method can be used,
and the methods do not necessarily need to bear any relation
to the circumvention techniques of the main system.
While the assumption of WebRTC permeates Snowflake's design,
its rendezvous modules are independent.
We~currently support two rendezvous methods in Snowflake:
\begin{description}
\item[Domain fronting]
In~this method, the client does an HTTPS exchange with the broker
through an intermediary web service such as a content delivery network (CDN),
setting the externally visible hostname
(the TLS Server Name Indication, or SNI~\cite[\S 3]{rfc6066})
to a ``front domain'' different from the broker's.
The CDN routes the HTTPS request to the broker not according to the TLS SNI
but rather the HTTP Host header, which, under TLS encryption,
reflects the broker's true hostname~\cite{Fifield2015a}.
A~censor cannot easily block domain-fronted rendezvous
without also blocking unrelated connections to the front domain,
which should be selected to have high value to the censor.
(But see \autoref{sec:fingerprinting} for features other than
the hostname that a censor might try to use.)
The well-known drawback of domain fronting
is the high cost of CDN bandwidth.
Because we use it only for rendezvous,
the cost is much less than if we were to use it
for all data transfer.
% "AMP cache rendezvous" https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/50
\item[AMP cache]
AMP is a framework for web pages written in a restricted dialect of HTML.
Part of the framework is a free-to-use
cache server~\cite{amp-cache}.
The cache fetches AMP-conformant web pages on demand,
which means that it is, effectively, a~restricted sort of HTTP proxy.
We~have a module that encodes rendezvous messages to AMP specifications,
allowing them to be exchanged with the broker via the AMP cache.
Rendezvous through the AMP cache is not easily blocked
without blocking the cache server as a whole.
This rendezvous method still technically requires domain fronting,
because the AMP cache protocol would otherwise expose the
broker's hostname in the TLS SNI,
but it increases the number of usable intermediaries and front domains.
\end{description}
Anything that can be persuaded to convey a message
of about 1500 bytes indirectly to the broker,
and return a response of about the same size,
can work as a rendezvous module.
% "Broker: investigate non-domain-fronting secure client / proxy registrations" https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/25594
For example, encrypted DNS
% "DNS-based rendezvous for Snowflake" https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/25874
or a chat bot
% "Example: there is a chat bot (say, Telegram), that acts as a broker." https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/25594#note_2823395
would serve.
Though some systems (flash proxy was one)
may need only a single, outgoing rendezvous message,
Snowflake needs a two-way exchange,
to support the SDP offer and answer.
% Flash proxy email rendezvous would not work for Snowflake, because unidirectional.
% https://gitweb.torproject.org/flashproxy.git/tree/flashproxy-reg-email
% https://gitweb.torproject.org/flashproxy.git/tree/facilitator/fp-registrar-email
Rendezvous is not unique to Snowflake.
Other examples of rendezvous in circumvention include
the DEFIANCE Rendezvous Protocol~\cite[\S 3]{Lincoln2012a},
the facilitator interaction in flash proxy~\cite[\S 3]{Fifield2012a},
and the registration proxy in Conjure~\cite[\S 4.1]{Frolov2019b}.
A~key property of Snowflake and the mentioned systems
is that they do not rely on preshared secret information.
The client needs only to acquire the necessary software;
whatever additional information is required to establish a circumvention session
is exchanged dynamically, at runtime.
This stands in contrast to another class of systems in which,
prior to making a connection,
a~client must acquire some secret,
such as an IP address or password,
through an out-of-band channel
presumed to be unavailable to the censor---and
the system's blocking resistance depends on
keeping that information hidden from the censor.
A~corollary of the no-secret-information property
is that an adversary---the censor---is
at no special disadvantage in attacking the system.
The censor may download the client software,
run it, study its network connections---and
the system must maintain its blocking resistance despite this.
The disadvantage of a separate rendezvous step
is that it is one more thing to get right.
Not only the main circumvention channel
but also the rendezvous must resist blocking:
the system is only as strong as the weaker of the~two.
\subsection{Peer-to-peer connection establishment}
\label{sec:connection}
Now the client and the proxy connect to each other directly.
Even in the absence of censorship,
making a direct connection between two Internet peers is not always easy,
because of NAT (network address translation) and firewalls.
Snowflake clients and proxies alike run in diverse networks
with varying NATs and ingress policies.
Fortunately for us,
WebRTC is designed with this use case in mind,
and has built-in support for traversing NAT, in~the form of
ICE (Interactive Connectivity Establishment)~\cite{rfc8445},
a procedure for testing candidate pairs of peer network addresses
to find one that works.
ICE~makes use of third-party
STUN (Session Traversal Utilities for NAT)~\cite{rfc8489}
servers that, among other things,
enable a host to learn its external IP addresses.
The first part of ICE took place at the beginning of rendezvous,
when the client and proxy contacted STUN servers to gather
external address candidates and included them in their respective
SDP offer and answer.
There is no guarantee that two hosts will be able to make
a connection using the facilities of STUN alone.
Some address mapping and
filtering setups are simply incompatible.
In~such a case,
ICE would normally fall back to using
TURN (Traversal Using Relays around NAT)~\cite{rfc8656},
a~kind of UDP proxy.
Such a fallback would be problematic for Snowflake,
because the TURN relays themselves
would become a target of blocking by the censor.
% "Configure TURN servers for the proxy and/or client" https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/25596
But Snowflake has an advantage most WebRTC applications do not.
Most WebRTC applications want to connect \emph{a particular} pair of peers,
whereas we are satisfied when a client can connect to \emph{any} proxy.
Snowflake clients and proxies self-measure their NAT type
and report it to the broker,
which takes NAT compatibility into account
and avoids cases that would require a fallback to TURN.
% "Investigate Snowflake proxy failures" https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/33666#note_2595319
% "Okay here's a summary of what I've found: ..."
We condense the possible combinations of NAT and firewall features
that impact
a Snowflake client or proxy's
ability to make a peer-to-peer connection
into the following well-known variations:
% https://datatracker.ietf.org/doc/html/rfc3489#section-5
\begin{description}
\item[Full cone]
The same internal IP--port pair always maps to the same external port.
Any remote host may send a packet to an internal IP address and port by sending a packet to the
mapped external port.
\item[Restricted cone]
Like full cone,
but incoming packets
are allowed only if
there has recently been an outgoing packet
to the same remote IP address.
\item[Port-restricted cone]
Like restricted cone,
but incoming packets are allowed only if
there has recently been an outgoing packet
to the same remote IP--port pair.
\item[Symmetric]
The external port depends on both
the internal IP--port pair and the remote IP--port pair.
Incoming packets are allowed only if
there has recently been an outgoing
packet to the same remote address.
\end{description}
\begin{table}
\definecolor{Ycolor}{Gray}{14}
\definecolor{ncolor}{Gray}{13}
\newcommand{\Y}{\cellcolor{Ycolor}\ding{51}}
\newcommand{\n}{\cellcolor{ncolor}--}
\newcommand{\rotlabel}[1]{\rotatebox{30}{#1}}
% \vphantom is to make the labels take up vertical space;
% \rlap is so they don't expand the horizontal size of table columns.
\newcommand{\rot}[1]{\vphantom{\rotlabel{#1}}\rotlabel{\rlap{#1}}}
\centering
% Make normal cells taller by default.
\renewcommand{\arraystretch}{1.25}
% Tighten line spacing in \makecell cells.
\renewcommand\cellset{\renewcommand\arraystretch{0.8}\setlength\extrarowheight{0pt}}
\begin{tabular}{@{}rcccccl@{\hspace{0.5ex}}l@{}}
& % empty
\rot{No NAT} &
\rot{Full cone} &
\rot{Restricted cone} &
\rot{Port-restricted cone} &
\rot{Symmetric} &
&
\\
No NAT & \Y & \Y & \Y & \Y & \Y & \multirow{3}{*}{\kern-\tabcolsep\kern0.5ex\footnotesize\(\left.\kern-\nulldelimiterspace\rule[-19pt]{0pt}{38pt}\right\}\)} & \multirow{3}{*}{\footnotesize\makecell[l]{unrestricted\\proxy}} \\
Full cone & \Y & \Y & \Y & \Y & \Y & \\
Restricted cone & \Y & \Y & \Y & \Y & \Y & \\
Port-restricted cone & \Y & \Y & \Y & \Y & \n & \multirow{2}{*}{\kern-\tabcolsep\kern0.5ex\footnotesize\(\left.\kern-\nulldelimiterspace\rule[-12pt]{0pt}{24pt}\right\}\)} & \multirow{2}{*}{\footnotesize\makecell[l]{restricted\\proxy}} \\
Symmetric & \Y & \Y & \Y & \n & \n & \\[\dimexpr 0.5ex - \dimexpr 0.5\dimexpr\arraystretch\normalbaselineskip]
% \hfill hack to make \upbracefill work with colortbl:
% https://tex.stackexchange.com/questions/202138/upbracefill-filling-entire-tabular-cell-and-package-colortbl#comment474604_202143
& \multicolumn{4}{@{~}c@{~}}{\footnotesize\def\hfill{\hskip 0pt plus 1filll}\upbracefill} & \clap{\footnotesize\def\hfill{\hskip 0pt plus 1filll}\upbracefill} & \\
& \multicolumn{4}{c}{\footnotesize\makecell{unrestricted\\client}} & \clap{\footnotesize\makecell{restricted\\client}} \\
\end{tabular}
\caption{
Pairwise compatibility of NAT variants,
using the facilities of STUN alone
(no fallback to TURN).
The incompatible cases are when one peer's NAT is symmetric
and the other's is symmetric or port-restricted cone.
Note the asymmetry in what NAT variants we consider ``restricted''
in client and proxy.
}
\label{tab:nat-matching}
\todo[inline]{Check style guide to see if caption should be before or after a table.}
\end{table}
\autoref{tab:nat-matching}
shows the pairwise compatibility of NAT variations.
As~the incompatible cases always involve a symmetric NAT,
we further simplify matching by categorizing the variations into the two types
\firstterm{unrestricted} (works with most other NATs) and
\firstterm{restricted} (works only with more permissive NATs).
Unrestricted proxies may be matched with any client;
restricted proxies may be matched only with unrestricted clients.
The broker prefers to match unrestricted clients with restricted proxies,
% https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/9edaee65470a1483bbdbe984e5e15a885f1e95d2/broker/ipc.go#L236
in~order to conserve unrestricted proxies
for the clients that need them.
Symmetric NAT is always considered restricted,
but port-restricted cone NAT differs
depending on the peer:
for proxies it is restricted, but
for clients it is unrestricted.
The asymmetric categorization is an approximation
to help conserve unrestricted proxies
for clients with symmetric NATs.
Though it creates the potential for an incompatible match,
we believe this to be uncommon in practice.
In~case of a connection failure,
clients re-rendezvous and try again.
To self-assess their NAT type,
clients use the NAT behavior discovery feature of STUN~\cite{rfc5780}.
% "Use STUN to determine NAT behaviour of peers" https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/34129
% "Add utility to help user discover their NAT type" https://github.com/pion/stun/issues/8
Proxies cannot use the same technique,
because the necessary STUN features are not exposed
to JavaScript.
% "Have a remote probe service to test snowflake proxy NAT compatability" https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/40013
Instead,
we adapt a technique from MassBrowser~\cite[\S \mbox{V-A}]{Nasr2020a}:
we~run a centralized, always-on WebRTC testing peer
behind a simulated symmetric NAT.
Proxies try connecting to this peer:
if~the connection succeeds, the proxy's type is unrestricted;
otherwise it is restricted.
Clients and proxies retest their NAT type periodically,
to account for potential changes in their local networking environment.
If~a client or proxy is unable to determine its NAT type for some reason,
it reports the type ``unknown,''
which the broker conservatively treats as if it were restricted.
\begin{figure}
\includegraphics{figures/proxies/proxy-nat-type}
\caption{
Proxy NAT types, in unique IP addresses per day.
The places in 2021 and 2022
where there is an increase in the ``unknown'' NAT type
and a decrease in the other types
were the result of operational problems with
NAT type testing.
% "Increase of 'unknown' NAT assignments by probetest since 2021-10-25" https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/40071
% "Move snowflake-broker to a systemd based setup" https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/40147
}
\label{fig:proxy-nat-type}
\end{figure}
\autoref{fig:proxy-nat-type}
shows that unrestricted proxies form
a relatively small fraction of the proxy population.
In~absolute terms, there are enough,
thanks in large part to the volunteers who run
the command-line
version of the Snowflake proxy
on networks unencumbered by NAT.
Though stable, long-term proxies go
somewhat against the ethos of Snowflake,
it~has proved useful, as a matter of practicality,
to~sacrifice a measure of address diversity
for better NAT compatibility in a common case.
We~can estimate how many tries it takes a client
to be matched with a proxy, on average,
by counting failed and successful rendezvous attempts at the broker,
under the assumption that clients repeat rendezvous attempts
until getting a match.
In~July 2023,
unrestricted clients almost always got a match on the first attempt,
while restricted clients needed an average of
1.07 attempts (standard deviation 0.05).
% > library(tidyverse)
% > read_csv("figures/proxies/client-match.csv") %>%
% filter(date < "2024-01-01") %>%
% group_by(month = format(date, "%Y-%m")) %>%
% summarize(
% unrestricted_n = mean((matched_count + unrestricted_denied_count) / matched_count),
% unrestricted_sd = sd((matched_count + unrestricted_denied_count) / matched_count),
% restricted_n = mean((matched_count + restricted_denied_count) / matched_count),
% restricted_sd = sd((matched_count + restricted_denied_count) / matched_count),
% .groups = "drop"
% ) %>% tail(10)
% # A tibble: 10 × 5
% month unrestricted_n unrestricted_sd restricted_n restricted_sd
% <chr> <dbl> <dbl> <dbl> <dbl>
% 1 2023-03 1 0 1.42 0.280
% 2 2023-04 1 0 1.15 0.152
% 3 2023-05 1.00 0.00204 1.23 0.113
% 4 2023-06 1.00 0.0000834 1.23 0.0734
% 5 2023-07 1 0 1.07 0.0498
% 6 2023-08 1 0 1.02 0.0568
% 7 2023-09 1 0 1.00 0.0148
% 8 2023-10 1.00 0.000000552 1.01 0.0137
% 9 2023-11 1 0 1.00 0.00499
% 10 2023-12 1 0 1.00 0.0121
While the proxy is connecting to its client,
% "While": https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/40228
it also connects to the bridge.
This connection
uses WebSocket~\cite{rfc6455}, which
offers a TCP-like, client--server connection
layered on HTTPS.
The choice of protocol for the proxy--bridge link is arbitrary,
and could be changed
without affecting the rest of the system.
It does not need to be resist blocking,
it~just needs to be available to JavaScript code in web browsers.
WebRTC, for example, would work for this link too.
\subsection{Data transfer}
\label{sec:data-transfer}
No~complicated processing takes place at the proxy.
The main value of a Snowflake proxy is its IP address:
it~gives the client a peer to connect to that is not on the censor's address blocklist.
Having provided that,
the proxy assumes a role of pure data transfer.
Snowflake uses a stack of nested protocol layers.
We~will walk though the layers and describe the purpose of each.
\bigskip
\noindent
\begin{tabular}{@{}l@{\,}l@{}l@{\,}l}
UDP & \rdelim\}{3}{*} & \multirow{3}{5em}{WebRTC data channel} & \rdelim\}{3}{*}[\,ephemeral, per proxy]\\
DTLS \\
SCTP \\
KCP & \rdelim\}{2}{*} & \multirow{2}{*}{Turbo Tunnel} & \rdelim\}{3}{*}[\,persistent, per session] \\
smux \\
\multicolumn{3}{@{}l}{Tor protocol} \\
\multicolumn{4}{@{}l}{application streams} \\
\end{tabular}
\bigskip
\noindent
This is the stack for the client--proxy link,
which is the place where WebRTC is used, and which is
exposed to observation by the censor (\autoref{fig:architecture}).
The stack for the proxy--bridge link is the same,
but with WebSocket in place of the
WebRTC data channel at the top.
The layers marked ``ephemeral'' are skimmed off
and replaced as proxies come and~go.
The layers marked ``persistent'' are instantiated once
in each circumvention session,
hold long-term state,
and are end-to-end between client and bridge.
The connection between a client and its proxy is
a WebRTC data channel~\cite{rfc8831},
which provides a way to send arbitrary binary messages between peers.
A~data channel is its own stack of three protocols:
UDP for network transport,
DTLS (Datagram TLS)
for confidentiality and integrity, and
SCTP (Stream Control Transmission Protocol)
for delimiting message boundaries
and other features like congestion control.
Working UDP port numbers will have been discovered
using ICE in the previous phase.
The peers authenticate one another
at the DTLS layer using certificate fingerprints
that were exchanged during rendezvous~\cite[\S 5.1]{rfc8842}.
Data channels are well-suited to Snowflake's needs.
(The specification even lists circumvention as a use case~\cite[\S 3.2]{rfc8831}.)
But data channels are not the only option:
WebRTC also offers \firstterm{media streams}
for unreliable transport of real-time audio and video.
Which of these is used may be a fingerprinting vector.
We~will take up this topic in \autoref{sec:fingerprinting}.
If~clients only ever used one proxy,
a~WebRTC data channel alone would be sufficient.
But a Snowflake proxy might
disappear at any moment,
and when that happens, its data channel goes with~it.
If~the client was in the middle of a long download,
for example, it should be possible to resume the download
without interruption after rendezvousing with a new proxy.
For this we need a shared notion of session state that exists
at the client and the bridge, not tied to any temporary proxy.
A~lack of session continuity across proxy failures
had been an unsolved problem in flash proxy~\cite[\S 5.2]{Fifield2012a}.
We~adopt the
Turbo Tunnel design pattern~\cite{Fifield2020a}
and insert a userspace
session and reliability protocol
between the ephemeral proxy data channels
and the client's own application streams.
% "[anti-censorship-team] Turbo Tunnel in Snowflake" https://lists.torproject.org/pipermail/anti-censorship-team/2020-February/000059.html
This part of the protocol stack
outlives any single proxy; it~belongs to
the client and the bridge.
Its primary function is to attach sequence numbers and acknowledgements
to packets of data,
so~that both ends know what parts of the data stream
need to be retransmitted after a temporary loss of proxy connectivity.
The client tags its traffic
with a random session identifier string that remains
consistent throughout a session,
which the bridge uses to index a map of session variables.
For the inner session layer we use a combination of
KCP~\cite{kcp} and
smux~\cite{smux}.
KCP provides reliability,
and smux detects the end of idle sessions and terminates them.
KCP and smux have shown their worth in other deployments,
and are easy to program,
but there is nothing about them on which we depend essentially.
Any other transport protocol that provides the necessary features
and can be implemented in userspace would~do,
such as QUIC, TCP, or (another layer~of) SCTP.
We~prototyped successfully with QUIC before deciding on KCP/\allowbreak smux.
% Not going to get into how we currently use reliable, ordered (TCP-like) data
% channels, but plan to switch to unreliable, unordered (UDP-like) data
% channels, a better fit for the underlying datagram-oriented Turbo Tunnel
% layer.
%
% "turn off reliable mode for WebRTC DataChannel" https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/109
% "Snowflake is currently using network resource in a so suboptimal way..." https://bugs.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/40251#note_2891751
Snowflake can be seen as an instance of the
``untrusted messengers'' model of Feamster et~al.~\cite[\S 3]{Feamster2003a}:
our \firstterm{proxies} and \firstterm{bridge} correspond to their