forked from mikeckennedy/talk-python-transcripts
-
Notifications
You must be signed in to change notification settings - Fork 5
/
Copy path032_PyPy_js_and_Python_in_the_browser.txt
1538 lines (769 loc) · 62.3 KB
/
032_PyPy_js_and_Python_in_the_browser.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
00:00:00 Imagine a future where you're building a rich client-side web app.
00:00:03 You start by creating some backend services, maybe in Flask or Node, create an HTML page, you throw in a few divs and ULs,
00:00:10 and then you type script source equals main.py language equals Python.
00:00:15 That future might just be possible for the right type of applications with Ryan Kelly's PyPy.js project.
00:00:22 This is Talk Python to Me with guest Ryan Kelly.
00:00:27 Show number 32 recorded Wednesday, September 30th, 2015.
00:00:54 I have many interests, sometimes conflates my creativity.
00:00:57 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities.
00:01:04 This is your host, Michael Kennedy.
00:01:06 Follow me on Twitter where I'm @mkennedy.
00:01:08 Keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via at Talk Python.
00:01:16 This episode is brought to you by Hired and Codeship.
00:01:19 Thank them for supporting the show on Twitter via at Hired underscore HQ
00:01:23 and at Codeship.
00:01:24 Hey, everyone.
00:01:26 Thanks so much for listening in.
00:01:28 No news today, so let me introduce Ryan and we'll get right to the interview.
00:01:31 Ryan is a software developer based in Melbourne, Australia.
00:01:36 Most of his days are spent coding in Python and JavaScript, commercially as an engineer at Mozilla,
00:01:41 as well as for a variety of open source projects.
00:01:44 He also maintains a strong interest in logic programming, mainly as a result of his doctoral thesis.
00:01:49 Ryan, welcome to the show.
00:01:52 Thanks for having me.
00:01:53 I'm super excited that you're here today.
00:01:55 We're going to talk about Python, but in a place that you typically don't find it,
00:01:59 in the browser, right?
00:02:00 Yep.
00:02:02 Yeah, there's a lot of cool projects out there, and I'm a huge fan of the one that you've been working on.
00:02:07 Before we get to that, though, let's talk about how you got into programming.
00:02:11 What's your story?
00:02:11 Sure.
00:02:12 So I think it was quite a while ago now, and I actually started programming in JavaScript in high school way back in the day
00:02:19 for a class, right?
00:02:21 I was never the kid that sort of always grew up with computers.
00:02:24 I very much just got into it for a school assignment and pretty much fell in love straight away.
00:02:30 You know, I just fell in love with the ability to have a thought and kind of capture that in some concrete form
00:02:37 and have it executed and have effects on the world.
00:02:39 It's pretty amazing.
00:02:40 And actually, that kind of subject changed my entire career path.
00:02:44 I'd always planned to go into the sciences, and I had sort of a half a semester of computer programming in high school
00:02:51 and changed my whole career trajectory into engineering and computer science.
00:02:54 I think a lot of us who are in programming had that experience where maybe we didn't think that we were programmers per se, right?
00:03:02 But then you get some experience where then you're like, wow, this is really amazing.
00:03:06 This is really way more rewarding than a lot of other stuff.
00:03:08 I had that experience in math.
00:03:10 I studied lots of math, and I'm like, but we don't build anything here.
00:03:13 Programming is so much more concrete but creative.
00:03:17 I love it.
00:03:18 Yeah, it's that beautiful combination of kind of creativity in the mix, right?
00:03:24 The discovering and creating is pretty cool.
00:03:27 Yeah, sure.
00:03:28 So how did you get to Python?
00:03:29 I was handed Python as a requirement for an internship I was doing in university.
00:03:37 I went to work for a company that was doing engineering simulations.
00:03:40 So I think like a lot of people, I sort of come to Python because it has really strong tools for doing data analysis and visualizations.
00:03:46 So that was in a kind of typical intern project kind of way.
00:03:52 It's kind of like, here is a relatively well-scoped project, and you're using this language, and off you go.
00:03:57 But I just found that it really fit my head in a way that a lot of other languages that I've been dabbling around with didn't.
00:04:04 It was just kind of super easy to get up and running and dive in, and I found that the more time I spent with it,
00:04:11 and the more I came to understand the language and find all of these more and more interesting corners of it,
00:04:16 it just kind of grew from there.
00:04:17 And sort of since from that point, it's always been my kind of go-to language for side projects or projects where I'm not kind of locked into a language for other reasons.
00:04:29 Yeah, I think Python has this rare combination of being very capable and yet very simple.
00:04:36 There's a lot of capable languages, and there's a lot of simple languages, but they don't often intersect.
00:04:41 So that's great.
00:04:43 I've worked in some interesting languages like Prologue, and there's a language called Mozart, which is like a logic programming language.
00:04:49 And they're incredibly powerful, but yeah, they don't have that just clarity that Python has.
00:04:55 So before we get to talk about your project, you work at Mozilla, right?
00:05:00 That's right.
00:05:01 I'm a huge fan of Mozilla.
00:05:03 Firefox is definitely my browser.
00:05:06 Cool. Good to hear.
00:05:07 Yeah, I try to spread the word.
00:05:09 What do you do there?
00:05:09 I used to work with the cloud services team.
00:05:12 We're not an independent team anymore.
00:05:13 So when I first started out, I was working on the web servers behind Firefox Sync.
00:05:18 So doing sort of data storage and RESTful JSON APIs.
00:05:23 They're all Python-based servers.
00:05:27 So it's pretty standard kind of Python backend web gig.
00:05:32 Through a series of kind of mergers and reorgs and so forth, I actually wound up on a project that's operating in Node.js.
00:05:40 So my time these days is spent on Firefox accounts, which is the login system for Sync and for all of the new kind of cloud-based services that Mozilla is doing.
00:05:49 And it's a 100% JavaScript shop, that team.
00:05:53 So it's an interesting change of pace.
00:05:56 But it does leave me kind of itching to get more involved in Python stuff on the side.
00:06:00 Yeah, that's really interesting.
00:06:02 I think that's a good lead into your topic because it's like Python and JavaScript, those two worlds colliding, right?
00:06:10 Right.
00:06:12 So maybe at a really high level, we could just talk about your project, PyPyJS, and then we could talk a little more specifically about it.
00:06:20 Sure.
00:06:22 So at a super high level, I like to describe it as an experiment in building a fast and compliant Python environment for the web.
00:06:31 So concretely, it means like I want a Python interpreter that will run on top of JavaScript.
00:06:37 So anywhere you would go that you would want to do something in JavaScript, you should be able to take Python to that platform as a first-class citizen.
00:06:45 And obviously, a big place where JavaScript shows up is in the browser.
00:06:49 And the browser is just dominated by JavaScript.
00:06:52 There's really not a lot of other games in town if you want to do scripting in the browser.
00:06:56 But I'm also kind of interested in how it might fit on the server side in Node.js or other JavaScript frameworks as well.
00:07:02 But basically, being able to take other languages such as Python and put them anywhere where JavaScript is currently claiming as its home turf.
00:07:10 Yeah, that's great.
00:07:11 I mean, there's definitely been people talking how JavaScript is becoming the new assembly language of the web.
00:07:16 And we'll get into that.
00:07:17 So the way that I heard about your project was you gave a really excellent presentation at PyCon 2015.
00:07:24 Oh, thank you.
00:07:26 Yeah, yeah, definitely.
00:07:27 What was the title of it?
00:07:28 I forgot.
00:07:29 PyPy.js.
00:07:31 What, how, why?
00:07:32 I think from memory.
00:07:33 Yeah, that's perfect.
00:07:35 Yeah, I think you're right.
00:07:35 And it's on the YouTube channel for PyCon US 2015.
00:07:39 So I'll be sure to link to that in the show notes.
00:07:42 One of the ways you opened your talk was you talked about how Mozilla loves the web.
00:07:47 And not just the technology of it, but sort of the whole idea and the culture of it.
00:07:51 Can you maybe talk about that a bit?
00:07:53 Sure.
00:07:54 Sure.
00:07:54 So we have this saying at Mozilla.
00:07:57 And this was sort of one of the things that really inspired me to get started on this project.
00:08:00 We like to say like the web is the platform.
00:08:03 Well, this is kind of the world we want to live in is where the web is the platform.
00:08:08 And so the web, not necessarily from a technology standpoint, but from a kind of computing platform standpoint, is pretty unique and pretty amazing.
00:08:17 It's a platform that doesn't have any kind of all-powerful gatekeepers.
00:08:23 It's a platform where anyone can kind of show up and start participating.
00:08:28 You can publish stuff on the web without having to be locked into a particular hardware platform or a particular set of tools.
00:08:34 You can show up and start consuming stuff from the web without any of the similar sets of restrictions or requirements.
00:08:43 And so at Mozilla, we kind of see the web as a really powerful tool for empowerment and participation in the global kind of information economy.
00:08:55 Of course, the web, you know, from a social standpoint, that's all very well.
00:09:01 But when you get right down to the technology of the web, it doesn't necessarily tick all of those boxes, right?
00:09:07 In a sense, the technology stack of the web is a little bit kind of restrictive.
00:09:11 It's all kind of JavaScript and HTML in the browser.
00:09:13 There's kind of a lot of nice sociological aspects of the web that are really unique and really valuable.
00:09:21 And there are some technical aspects of the web that are not always to everyone's liking.
00:09:25 So that's an interesting platform in that respect.
00:09:27 It really is.
00:09:29 You know, you could have sold me on this idea with that statement in 2005, no problem.
00:09:37 But I think it's even more relevant now because we all have these app platforms and these proprietary APIs, you know, iOS, Android, OS X.
00:09:48 There's just more and more walled gardens and proprietary stuff.
00:09:53 Walled gardens and app stores.
00:09:55 Yes, absolutely.
00:09:56 And there's, you know, good financial reasons for that, right?
00:09:58 Absolutely.
00:09:59 And not just financial reasons.
00:10:01 Like curated content has a lot of user value and security value and stuff like that.
00:10:06 So the web has not completely undeserved reputation as a little bit of a security badlands because you don't know whose code you're running half the time.
00:10:14 You just could have got to trust that the browser is going to do the right thing for you.
00:10:17 That's right.
00:10:18 And that's, I think, partly why the JavaScript component of the web is so crucial because if it involves Flash or ActiveX or some other, you know, high privileged C-based plugin, it's not the web.
00:10:33 Not really.
00:10:34 Absolutely.
00:10:34 And you can't really trust it, right?
00:10:35 Absolutely.
00:10:35 And so your mission to bring things like Python to the web have to involve JavaScript somehow, right?
00:10:41 Right.
00:10:43 And I think there's been a long history and pretty much since the web has been a thing of trying to bring more kind of native platform functionality to the web.
00:10:53 You know, you mentioned Flash.
00:10:54 They've had, you know, ActiveX or Java applets, you know, way back in the day.
00:10:59 And the web has a really long history of trying to compete with native platforms by bringing native platform technologies kind of into the web by expanding the scope of the technology stack of the web.
00:11:14 As, you know, one after another have been kind of failures.
00:11:18 They fail to stick for any significant length of time.
00:11:23 And a lot of the time it's because of aspects like that security, right?
00:11:25 You're just kind of opening up too many holes, too many opportunities for things to go wrong.
00:11:29 Whereas if you take kind of JavaScript as your base level, then you can build on this sandbox that's had 20-odd years of, you know, figuring out how to get that right and build that model correctly.
00:11:42 So if you take that as your foundation, you kind of get a lot of the good security stuff and the lessons learned from the web for free.
00:11:48 Yeah, I totally agree.
00:11:50 And it's also worth noting the birthplace of JavaScript is Mozilla, right?
00:11:54 Well, it wasn't Mozilla back then.
00:11:56 But yeah, sort of the heritage of JavaScript sort of comes from the same place as Mozilla.
00:12:02 Yeah, absolutely.
00:12:03 Yeah, absolutely.
00:12:03 So when I think about JavaScript frameworks, there's the kind of more lightweight ones where it's, I'm writing, I'm taking some JavaScript API and I'm writing some more JavaScript code to write maybe a single page application.
00:12:20 So I'm thinking of like Angular, you know, those types of things, jQuery, so on.
00:12:26 And then at the other end, there's an idea of bringing more stuff into the JavaScript world using the concept of something called asm.js.
00:12:38 Can you speak to that a little bit?
00:12:40 So asm.js is a funny beast.
00:12:45 It's sort of this tiny little subset of JavaScript, which if you look at it in the right light and you kind of suspend disbelief for long enough,
00:13:00 inside of JavaScript are the semantics of a 32-bit CPU, essentially.
00:13:09 So JavaScript has, for what is a high-level language, JavaScript has a surprising number of kind of really low-level features.
00:13:17 Like it's got 32-bit bitwise integer operators and things like this.
00:13:24 Asm.js basically came out of an attempt by a very smart fellow named Alonza Kai to compile C code to the web.
00:13:33 And through a series of kind of experimenting with different ways of doing that,
00:13:38 came to realize that there was this subset of JavaScript that you could target that implemented the semantics of the machine more or less directly,
00:13:46 that you could then take C code and transliterate into this subset called asm.js.
00:13:52 And because of the kind of browser wars and the JavaScript performance wars that had been going on up to that time,
00:13:59 it also happened that that stuff would run pretty fast.
00:14:02 So he likes to describe it as discovering a kind of low-level 32-bit assembly language hidden inside JavaScript,
00:14:10 and I think that's pretty apt.
00:14:11 Very cool.
00:14:12 Have you seen the video or the PyCon presentation, Birth and Death of JavaScript by Gary Bernhardt?
00:14:19 I was present in the audience for that talk, and I loved it.
00:14:22 That was an amazing talk.
00:14:24 Those of you guys out there listening who haven't seen this, I'll put it in the links.
00:14:28 And I've referenced it before on the show.
00:14:30 But he did a really good job of showing off Asm.js and basically bringing anything that can be written in C to the web.
00:14:39 And one of those was like a pretty high-end video game like Quake or something like this, right?
00:14:46 They're doing sort of more – within half the speed of Native, I think they usually get these things running.
00:14:53 It's pretty amazing.
00:14:54 Yeah, very, very amazing.
00:14:56 And if you say high-end game like Quake, but I sort of sense that presentation has gone past,
00:15:01 I think they actually have been launching with even more kind of – the very latest and greatest gaming frameworks are starting to ship support for generating Asm.js JavaScript natively.
00:15:12 That's really amazing.
00:15:14 Okay, so there's this idea of we can take C code and we can compile it not to machine instructions,
00:15:19 but we can compile it to a special subset of JavaScript that then can be translated by the browsers to execute very fast.
00:15:28 Correct.
00:15:28 So how does this relate to your project, PyPy.js?
00:15:31 The PyPy.js is sort of a coming together of what to me are two amazing feats of engineering, right?
00:15:38 And one of them was Asm.js and Emscripten and this whole idea that you can treat the web more or less like a compile target,
00:15:46 as if it were a native CPU.
00:15:48 The other half of that, of course, is the PyPy Python interpreter, which I think you had an episode on not too long ago.
00:15:55 That's right.
00:15:56 That was show 21.
00:15:58 And PyPy, the interpreter, does some really amazing stuff.
00:16:01 It's incredible.
00:16:02 So one of the things that I think actually was talked about a little bit in that episode,
00:16:08 but one of the things I really like about PyPy is they weren't just focused on writing a really fast Python interpreter.
00:16:15 They were focused on creating a platform for experimenting with interpreters.
00:16:21 So it seemed to me the perfect kind of opportunity to learn more about both of these projects.
00:16:29 I'll learn more about PyPy.
00:16:30 I'll learn more about Emscripten and Asm.js.
00:16:33 Because PyPy is structured in such a way to make it really easy to dive in and kind of change bits in and out and play around with it,
00:16:40 I had this idea that I could get in there and like, well, I'll just find the just-in-time compiler backend inside PyPy,
00:16:48 and we'll just change it to emit Asm.js instead of, you know, x86 or ARM assembly,
00:16:54 which in a lot of projects would be kind of a terrifying proposition.
00:16:58 But because of the amount of kind of just good abstractions and so forth that are inside PyPy,
00:17:04 it actually was quite approachable as a problem.
00:17:08 So the coming together basically looks like the PyPy interpreter toolchain run through the Emscripten compiler toolchain
00:17:18 so that you get a version of it that is compiled to JavaScript.
00:17:22 And then at runtime, it uses the PyPy just-in-time compiler infrastructure
00:17:28 to take your Python code that you're running and spit out Asm.js at runtime,
00:17:34 in the same way that a native PyPy interpreter would spit out kind of specialized native code for your application.
00:17:39 So very much treating Asm.js as if it were just another dialect of some assembly language for a machine.
00:17:46 That's so amazing.
00:17:46 How long did it take you to get a proof of concept that was actually doing something?
00:17:51 I think my very first proof of concept without the just-in-time compiler part came together pretty quickly.
00:17:58 I lost maybe two weeks trying to figure out that Emscripten was actually assigning file descriptors
00:18:07 at different numbers than they usually are in Unix.
00:18:09 So like standard in is usually file descriptor zero, standard out is usually file descriptor one.
00:18:16 And it was giving them different numbers because it had never mattered before.
00:18:18 But PyPy really didn't like it.
00:18:20 So there are a few kind of just like toolchain compatibility problems like that.
00:18:24 But sort of once I got that figured out, right, is sort of just compile it as if you were running it through a cross compiler
00:18:32 and you get it up and running pretty quickly.
00:18:34 The JIT backend, I probably worked on, on and off for the better part of a year.
00:18:39 I conceived this scheme at the end of PyCon Australia probably three years ago.
00:18:45 And then I presented the kind of final first version of it at the following PyCon Australia the next year.
00:18:52 In your presentation, you said your goal was really to bring support to the web for Python.
00:18:58 And in your mind, that had to sort of meet three criteria, compatibility, performance, and webbishness.
00:19:04 Can you talk about that?
00:19:06 I wish I'd found a better word for that.
00:19:07 I think that's the official word, yeah.
00:19:10 Yeah.
00:19:12 So this is not the first time people have tried to do Python-y stuff on the web.
00:19:17 I think I even said in the talk it probably won't be the last.
00:19:21 But it's one of those classic cases that you get in software a lot of a triangle where the three goals are kind of more or less opposed to each other.
00:19:31 So you've had languages that compile to JavaScript.
00:19:35 That's a thing that we know how to do.
00:19:36 And we know how to do that relatively fast.
00:19:40 As long as your language semantically maps to JavaScript pretty well.
00:19:46 Python doesn't.
00:19:47 So in Python, you have big ints by default.
00:19:51 You have a variety of checks built into your arithmetic operators and a few things like that.
00:19:58 So the semantics of Python are actually quite different to the semantics of JavaScript, even though they look very similar.
00:20:05 On the one hand, it's easy to make something go fast in JavaScript.
00:20:08 If you're not too far from the semantics of JavaScript.
00:20:11 On the other hand, we know how to compile languages from one to the other, right?
00:20:16 It's sort of a Turing completeness argument.
00:20:18 So if we wanted to get a really compliant Python interpreter, you know, one that implements all of the edge cases of CPython up and running on a different platform, you know, it's not implausible that we can get that up and running.
00:20:34 So it's interesting to see how fast we can do it.
00:20:36 So there's kind of two relatively opposed goals there, right?
00:20:41 Being fast and being – I like to say being Python, right?
00:20:46 But doing all of the things that Python does, not kind of looking a bit like Python, but having JavaScript's number model and things like that.
00:20:53 Right.
00:20:53 Exactly.
00:20:53 There's all these – you know, when you talk to people about JavaScript, they're like, oh, there's all these JavaScript gotchas you have to be on the lookout for.
00:21:01 I don't typically hear people go, look out for all the Python gotchas because they're lurking everywhere, right?
00:21:06 Right.
00:21:06 Python's usually got your back.
00:21:19 This episode is brought to you by Hired.
00:21:22 Hired is a two-sided, curated marketplace that connects the world's knowledge workers to the best opportunities.
00:21:28 Each offer you receive has salary and equity presented right up front, and you can view the offers to accept or reject them before you even talk to the company.
00:21:37 Typically, candidates receive five or more offers in just the first week, and there are no obligations ever.
00:21:44 Sounds pretty awesome, doesn't it?
00:21:46 Well, did I mention there's a signing bonus?
00:21:48 Everyone who accepts a job from Hired gets a $2,000 signing bonus, and as Talk Python listeners, it gets way sweeter.
00:21:56 Use the link Hired.com slash Talk Python To Me, and Hired will double the signing bonus to $4,000.
00:22:04 Opportunity's knocking.
00:22:06 Visit Hired.com slash Talk Python To Me and answer the call.
00:22:09 It's usually got your back.
00:22:20 That's right.
00:22:20 And so you can't just, for example, translate floats in Python directly over necessarily to just run as JavaScript or numbers, especially integers, like you say, over to JavaScript numbers.
00:22:33 So there's all that.
00:22:34 That's the compatibility bit, right?
00:22:36 Right.
00:22:37 So, yeah, that's the compatibility aspect.
00:22:39 And things like parts of the standard library as well.
00:22:42 You know, do you have all of a full implementation of the sys module, including get frame and kind of the debugger and all of these things?
00:22:52 And it's interesting.
00:22:54 It's sort of like, well, I don't really use those things that often.
00:22:57 You know, I don't really depend on integer overflows escalating to big ints by default that often.
00:23:05 Or, you know, I don't use sys get frame that often.
00:23:07 But it's interesting.
00:23:08 But it's interesting.
00:23:09 As soon as you start trying to run, like, whole programs on Python, like, you'll find that, oh, you use some library somewhere that uses some library that happens to use one of those things, you know, in some corner of its API.
00:23:23 So even though they seem like edge cases, they're often not at a whole program level.
00:23:28 Right.
00:23:29 They're not necessarily edge cases for small, simple things.
00:23:32 But if you're going to really write major apps there, then all of a sudden those become things you have to worry about, right?
00:23:37 Right.
00:23:38 And something I'm very interested in is less about kind of writing Greenfield's Python code for the web.
00:23:43 Because, you know, personally, I am not that unhappy writing JavaScript.
00:23:47 I'm interested in taking Python code that you've written and putting it on the web without having to do a whole lot of work.
00:23:54 And the compatibility aspect really is really important for things like that.
00:23:57 Right.
00:23:59 Like script source equals some file dot py.
00:24:02 Something like this, right?
00:24:03 Right.
00:24:04 Well, so, something like that.
00:24:08 Yeah.
00:24:09 But it would be, you know, taking something that you've already written and being able to put it on the web and make use of all of the fun distribution aspects of the web without having to do a whole bunch of busy work to make it run in JavaScript.
00:24:20 Right.
00:24:21 Absolutely.
00:24:21 So it sounds to me like pypy.js completely nails the compatibility part because it's effectively PyPy.
00:24:29 Is that right?
00:24:30 Correct.
00:24:31 So the fun thing about basing this on PyPy is we essentially get the compatibility part for free because they put an amazing amount of work into being a really, really compatible with the Python language reference, cPython.
00:24:43 And then you get to the performance.
00:24:44 And performance has multiple aspects.
00:24:47 All right.
00:24:48 One is I've got my code loaded up.
00:24:50 And now I can run this and it's doing XYZ iterations per second with such and such amount of latency.
00:24:57 The other part of it is I visit the web page and then it responds.
00:25:04 Right.
00:25:05 Yes.
00:25:05 Maybe we could talk about those a bit.
00:25:08 Absolutely.
00:25:09 PyPy being interpreted with a just-in-time compiler generally targets kind of steady state performance of your application.
00:25:19 So, you know, you load some code and you run it for a while and the JIT has a chance to kick in and generate a specialized code and you get very nice kind of steady state performance for the code that you're running.
00:25:29 And I think, again, the combination of PyPy and Asm.js means we can kind of get a lot of that more or less, I don't want to say for free, but, you know, easily in a JavaScript environment.
00:25:43 What we really don't get and what is actually really important on the web, as you say, is that kind of initial startup experience.
00:25:51 Like the time between hitting a web page and being able to interact with it is even more critical on the web than it is in sort of places where you'd normally find Python.
00:26:05 And PyPy.js, realistically, right now does a pretty terrible job of it.
00:26:10 It's only so fast you can start interacting with the page and you've got to load sort of megabytes worth of JavaScript to even start running your code.
00:26:17 Right.
00:26:18 And so right now, if you look, I think, you know, you have a really great page, arewepythonyet.com, that talks about a lot of cool stuff.
00:26:25 We can get to that.
00:26:26 But on there you have the download size, both, you know, off of disk and sort of maybe on the wire, the GZIP representation on the wire.
00:26:35 And on disk, it's like it was a little higher, but you guys have gotten down to 15 megabytes, right?
00:26:41 Yep.
00:26:41 And what is that, like five on the wire?
00:26:44 Yeah, I guess down to comfortably on a single digits on the wire.
00:26:47 I think five is probably about right.
00:26:50 Which for a native application is not terrible.
00:26:55 Like that's kind of in the ballpark.
00:26:57 But for stuff that you have to load up in order to interact with a web page, it's not great.
00:27:01 Yeah.
00:27:02 Typically, we talk about minification of our JavaScript and it's smaller, usually smaller than five megs.
00:27:08 Yes.
00:27:09 Usually not measured in megabytes, let's be quite frank about it.
00:27:13 That's right.
00:27:14 That's huge.
00:27:16 What is that, 100K?
00:27:17 No, no, that's 20 megs.
00:27:18 But, you know, there are potential solutions, right?
00:27:24 I mean, there are CDNs, extreme caching.
00:27:28 Right.
00:27:29 Could alleviate this somewhat, right?
00:27:32 The core runtime parts could somehow be sort of shared across sites.
00:27:37 Indeed.
00:27:38 I think there's actually still a lot of low-hanging fruit in that size, right?
00:27:43 So that size includes, for example, an entire Unicode character database, which is built into the interpreter in PyPy.
00:27:52 It includes all of the standard library modules that are built into the interpreter, whether you're going to use them or not, and things like that.
00:27:59 So there is definitely some scope for making that smaller.
00:28:02 And I've made more minimal builds of it that are several megabytes smaller.
00:28:09 But I don't think it's ever going to get down to, you know, tens of kilobytes in size.
00:28:14 This is pretty unrealistic.
00:28:15 If you go and download CPython from python.org, compressed, that's like 22, 23 megabytes, right?
00:28:25 And so, you know, if you're going to ship the whole runtime, the implementation, the PyPy implementation, that's kind of a lower bound in some way.
00:28:32 Exactly.
00:28:34 But that's not always a problem.
00:28:38 You know, if you look at this as a way to write all of your kind of interactive website scripting in Python instead of JavaScript,
00:28:46 like you were saying, kind of script lang equals text Python, that's a tremendous amount of overhead to pay before you can do any of the interactivity on your site.
00:28:56 But if you're doing something like shipping a kind of more appy experience that you've built in Python,
00:29:03 kind of bootstrapping into, in the same way that you might if you're loading up a game on the web
00:29:08 or like some sort of more heavyweight application, it's less of a big deal.
00:29:12 So, and, you know, the games that are compiled for the web within script and have this problem as well, right?
00:29:18 You want to get into the game and start playing it.
00:29:20 You've got to download many more megabytes of like image assets and compiled code and all this sort of stuff.
00:29:26 But it's okay for that use case.
00:29:29 And I think that's sort of the point where we're at with the project now where the people that are sort of really showing an interest in starting to use it
00:29:36 are cases where that's going to be less and less of a concern, where you're okay to kind of wait for a few seconds while you're bootstrapping into a fairly elaborate experience.
00:29:47 Right.
00:29:47 And like I said, once you get that loaded up the first time, you can cache it.
00:29:51 So if you're able to wait five seconds or whatever it takes to download that, then, you know, maybe it's possible.
00:29:58 So games are obviously one concept here.
00:30:02 Maybe if you're doing some, like I said, single page app thing where you kind of load it up and then you just stay there for a long time.
00:30:09 Right.
00:30:11 So I had some preliminary kind of interactions with the Jupyter and IPython people trying to get this plugged in.
00:30:27 And in a sense, it actually went really well.
00:30:30 They've got a really excellent infrastructure for plugging in different language backends and different, you know,
00:30:35 they can do Ruby or whatever plugged into their infrastructure.
00:30:40 So that, in a sense, kind of works.
00:30:42 Like, hey, you can plug in, plug Pypy.js and run this completely client side.
00:30:46 Of course, then, what that's missing is all of the fun libraries that you might want to use with that.
00:30:51 So if you want to do scientific Python on the web, scientific kind of programming on the web in Python,
00:30:56 you're going to want things like NumPy and sort of all of the analysis libraries that typically go along with that.
00:31:03 So I'm still interested in kind of pushing ahead and seeing where we can go with that.
00:31:09 But I think it becomes more of a library problem at that point, making sure you can actually ship that,
00:31:15 all of the details of that experience that people would expect and not just the Python interpreter itself.
00:31:20 The Python interpreter itself is actually probably the easiest of the bits, right?
00:31:25 Because then you've got to get NumPy and SciPy and scikit-learn, and a lot of those have C extensions,
00:31:31 which then you've got to get into Asm.js again.
00:31:34 Right.
00:31:34 There's a lot of nuances that people building those libraries have never thought of, right?
00:31:39 Exactly.
00:31:40 And so in theory, that's all possible, right?
00:31:44 It's all the same principle of we've got code for a native platform where we can retarget it for the web.
00:31:49 But it's never quite that straightforward in practice.
00:31:52 Do you want to talk about arewepythonyet.com a bit?
00:31:57 Sure.
00:31:57 Yeah.
00:31:59 So what was the idea?
00:32:01 You said that was inspired by some sort of performance metrics at Mozilla historically.
00:32:07 The domain name in particular is a little kind of inside joke at Mozilla, right?
00:32:13 When the browser performance wars were really heating up, folks that were working on JavaScript performance at Mozilla needed a place to put their public JavaScript benchmarks.
00:32:24 And so they created this site called arewepastyet.com, which you could go to and just kind of look at all of the different JavaScript benchmarks that were running in the different engines.
00:32:33 And it kind of became a little bit of a thing at Mozilla.
00:32:35 So when people were talking about, oh, Firefox is a real memory hog and we need to do something about the memory usage of Firefox, you know, they started up a metrics tracking site called areweslimyet.com.
00:32:46 We were starting to really measure our community contributions and figure out, you know, how many contributors we're getting from different parts of the world and how big the community is growing.
00:32:55 Like they set up a metrics tracking website called arewemillionyet.com.
00:33:00 And sort of this ongoing habit of just trying to picking things that we really want to make a big difference to and measuring them and putting it up somewhere for everyone to see.
00:33:09 The other folks that are really keen on, you know, doing public performance measurements are the PyPy team.
00:33:15 So the benchmarks that are on arewepythonyet.com are all taken from the PyPy Speed Center, which I haven't managed to get all of their benchmarks running, but I've got a good subset of them.
00:33:26 But yeah, sort of both places, I guess, have a really strong history of being really upfront and public about all of their performance metrics tracking.
00:33:37 I think it's a wonderful philosophy to try and follow in the footsteps of.
00:33:40 Yeah, that's great.
00:33:42 And you go there and you can really quickly see where the performance is really good and where it might lag behind.
00:33:48 And one of the things I thought was really interesting is there are plenty of places where PyPy.js is literally faster than CPython.
00:33:58 Yep.
00:33:59 All of that magic, of course, is due to the PyPy JIT.
00:34:04 And if you change the comparison to compare it to a native PyPy, of course, it's going to be uniformly slower.
00:34:09 But yeah, I was pretty amazed.
00:34:12 That's sort of one of the challenges I set myself when I was starting out on this little experiment was to say, look, you know, is the combination of Asm.js, you know, being almost as fast as a native platform and PyPy being kind of consistently faster than CPython.
00:34:29 Can that in combination mean we can actually run Python benchmarks in the web faster than your standard Python interpreter?
00:34:36 And it's pretty astonishing to find out that that is, in fact, the case.
00:34:44 This episode is brought to you by CodeShip.
00:34:57 CodeShip has launched organizations, create teams, set permissions for specific team members, and improve collaboration in your continuous delivery workflow.
00:35:07 Maintain centralized control over your organization's projects and teams with CodeShip's new organizations plan.
00:35:12 And as Talk Python listeners, you can save 20% off any premium plan for the next three months.
00:35:18 Just use the code TALKPYTHON, all caps, no spaces.
00:35:21 Check them out at CodeShip.com and tell them thanks for supporting the show on Twitter where they're at CodeShip.
00:35:33 Yeah, I was, when I pulled this up, I was like, all right, let's see how many times slower it is.
00:35:39 Is it like something we could live with because it's going to run in the browser?
00:35:42 And I was like, wow, okay, that's really cool.
00:35:44 It's also fun to click off the D8 interpreter in those comparisons and get just the SpiderMonkey engine.
00:35:53 It'd get a little bit more times faster than CPython as well.
00:35:56 That's awesome.
00:35:57 And of course, D8 is Chrome and SpiderMonkey is Firefox, right?
00:36:02 V8 JavaScript engine, yes.
00:36:03 Same one that runs Node.js, right?
00:36:06 So very interesting.
00:36:07 So the performance story is really, really good on the execution side.
00:36:13 But then you also have the stuff about sort of the startup and the download size and so on.
00:36:18 Right.
00:36:19 So there's no point, you know, trying to hide any of that stuff.
00:36:22 There's a whole tab there about startup overhead.
00:36:24 And it does indeed measure the download size in megabytes.
00:36:27 And you can, you know, it's even got a little feature there.
00:36:30 You can actually click to just test how long it'd take to load up on your machine.
00:36:34 What's interesting here actually is the download size is one thing.
00:36:40 Like that's kind of understood and easy to grasp.
00:36:43 But there's also a graph of the time it takes.
00:36:45 Like once you've downloaded that, how long does it take for the JavaScript engine to actually kind of initialize this interpreter?
00:36:52 And that actually is on the order of a second or two just by itself, right?
00:36:56 Kind of taking all of that JavaScript code and parsing it and loading it up into the browser is actually, you know, makes the browser work really hard.
00:37:04 Yeah, it's a large quantity of JavaScript that you can understand, right?
00:37:09 So, you know, you work at Mozilla.
00:37:12 Maybe you can pull some strings.
00:37:13 Is it, would it be possible to take projects like this, like Asm.js and PyPy.js and have them sort of certain versions shipped with browsers?
00:37:26 So like when I load up Firefox and I hit a page and it says, give me PyPy.js, it goes, we already have that downloaded.
00:37:34 Let's go.
00:37:34 I'd love to think I could pull those strings.
00:37:37 I don't know if I really have that power.
00:37:39 But I mean, the thing about the web, and this has been a general principle of the web for a long time, right, is no one browser vendor really has the power to do that by themselves.
00:37:57 So, you know, we could kind of put this in the browser and say, yeah, like, you know, we want to treat Python as a first class language and we're going to special case it in the browser and get a really good experience for Firefox users.
00:38:10 But the individual market share of us or any other browser who tried to do that is probably not sufficient to, for people writing websites to be able to depend on it, right?
00:38:21 Yeah, absolutely.
00:38:22 And the most important use case of that would be on mobile.
00:38:26 And then that's an even harder sell.
00:38:28 Absolutely.
00:38:28 Absolutely.
00:38:30 What's really interesting and sort of segueing out of that a little bit, what you really need in order for something like this to kind of take the next step and to get over some of these kind of fundamental problems of code size or load time or whatever is, you know, if you can't find a way to hack around that in JavaScript, which for some of these things we really can't.
00:38:51 You kind of need all of the browser vendors to agree on a way forward and sort of everyone's got to ship something better together.
00:39:00 And one of the amazing things that's happened coming out of Asm.js and experiments like the Games initiative that have kind of proven that that's a really powerful thing is you're actually seeing all of the browser vendors sort of getting together and talking about, well, like, what's next after Asm.js?
00:39:19 Like, how can we make this better?
00:39:21 How can we really commit to this as the web rather than any particular browser platform?
00:39:27 That's really, I mean, that's the way the web works, right?
00:39:29 That's the way the web is successful anyway.
00:39:31 Like you said, people have tried to jam their way through, you know, like, IE6 type stuff that happened back in those days, but that's not really what sticks.
00:39:41 You've got to have a good story for getting from where the web is to where you want the web to be.