-
Notifications
You must be signed in to change notification settings - Fork 54
/
Copy pathGenerators_and_Comprehensions.html
794 lines (654 loc) · 71.7 KB
/
Generators_and_Comprehensions.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" />
<meta content="Topic: generators comprehensions and efficiency, Difficulty: Easy, Category: Section" name="description" />
<meta content="generator, range, list comprehension, generator comprehension, nested comprehensions, inline for-loop, filtered, iterator" name="keywords" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Generators & Comprehension Expressions — Python Like You Mean It</title>
<link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../_static/my_theme.css" type="text/css" />
<!--[if lt IE 9]>
<script src="../_static/js/html5shiv.min.js"></script>
<![endif]-->
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/doctools.js"></script>
<script src="https://www.googletagmanager.com/gtag/js?id=UA-115029372-1"></script>
<script src="../_static/gtag.js"></script>
<script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
<script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/x-mathjax-config">MathJax.Hub.Config({"tex2jax": {"inlineMath": [["$", "$"], ["\\(", "\\)"]], "processEscapes": true, "ignoreClass": "document", "processClass": "math|output_area"}})</script>
<script type="text/javascript" src="../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Python’s “Itertools”" href="Itertools.html" />
<link rel="prev" title="Iterables" href="Iterables.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../index.html" class="icon icon-home"> Python Like You Mean It
</a>
<div class="version">
1.4
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<p class="caption"><span class="caption-text">Table of Contents:</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../intro.html">Python Like You Mean It</a></li>
<li class="toctree-l1"><a class="reference internal" href="../module_1.html">Module 1: Getting Started with Python</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../module_2.html">Module 2: The Essentials of Python</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="Basic_Objects.html">Basic Object Types</a></li>
<li class="toctree-l2"><a class="reference internal" href="SequenceTypes.html">Sequence Types</a></li>
<li class="toctree-l2"><a class="reference internal" href="Variables_and_Assignment.html">Variables & Assignment</a></li>
<li class="toctree-l2"><a class="reference internal" href="Introduction.html">Introducing Control Flow</a></li>
<li class="toctree-l2"><a class="reference internal" href="ConditionalStatements.html">Conditional Statements</a></li>
<li class="toctree-l2"><a class="reference internal" href="ForLoops.html">For-Loops and While-Loops</a></li>
<li class="toctree-l2"><a class="reference internal" href="Iterables.html">Iterables</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">Generators & Comprehension Expressions</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#Introducing-Generators">Introducing Generators</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#The-range-generator">The <code class="docutils literal notranslate"><span class="pre">range</span></code> generator</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#Creating-your-own-generator:-generator-comprehensions">Creating your own generator: generator comprehensions</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#Storing-generators">Storing generators</a></li>
<li class="toctree-l4"><a class="reference internal" href="#Consuming-generators">Consuming generators</a></li>
<li class="toctree-l4"><a class="reference internal" href="#Chaining-comprehensions">Chaining comprehensions</a></li>
<li class="toctree-l4"><a class="reference internal" href="#Using-generator-comprehensions-on-the-fly">Using generator comprehensions on the fly</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#Iterating-over-generators-using-next">Iterating over generators using <code class="docutils literal notranslate"><span class="pre">next</span></code></a><ul>
<li class="toctree-l4"><a class="reference internal" href="#Iterables-vs. Iterators">Iterables vs. Iterators</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#List-&-Tuple-Comprehensions">List & Tuple Comprehensions</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#Nesting-comprehensions">Nesting comprehensions</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#Links-to-Official-Documentation">Links to Official Documentation</a></li>
<li class="toctree-l3"><a class="reference internal" href="#Reading-Comprehension-Exercise-Solutions:">Reading Comprehension Exercise Solutions:</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="Itertools.html">Python’s “Itertools”</a></li>
<li class="toctree-l2"><a class="reference internal" href="Functions.html">Basics of Functions</a></li>
<li class="toctree-l2"><a class="reference internal" href="Scope.html">Scope</a></li>
<li class="toctree-l2"><a class="reference internal" href="DataStructures.html">Data Structures (Part I): Introduction</a></li>
<li class="toctree-l2"><a class="reference internal" href="DataStructures_II_Dictionaries.html">Data Structures (Part II): Dictionaries</a></li>
<li class="toctree-l2"><a class="reference internal" href="DataStructures_III_Sets_and_More.html">Data Structures (Part III): Sets & the Collections Module</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../module_2_problems.html">Module 2: Problems</a></li>
<li class="toctree-l1"><a class="reference internal" href="../module_3.html">Module 3: The Essentials of NumPy</a></li>
<li class="toctree-l1"><a class="reference internal" href="../module_3_problems.html">Module 3: Problems</a></li>
<li class="toctree-l1"><a class="reference internal" href="../module_4.html">Module 4: Object Oriented Programming</a></li>
<li class="toctree-l1"><a class="reference internal" href="../module_5.html">Module 5: Odds and Ends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../changes.html">Changelog</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../index.html">Python Like You Mean It</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="../index.html" class="icon icon-home"></a> »</li>
<li><a href="../module_2.html">Module 2: The Essentials of Python</a> »</li>
<li>Generators & Comprehension Expressions</li>
<li class="wy-breadcrumbs-aside">
<a href="../_sources/Module2_EssentialsOfPython/Generators_and_Comprehensions.md.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<style>
/* CSS overrides for sphinx_rtd_theme */
/* 24px margin */
.nbinput.nblast.container,
.nboutput.nblast.container {
margin-bottom: 19px; /* padding has already 5px */
}
/* ... except between code cells! */
.nblast.container + .nbinput.container {
margin-top: -19px;
}
.admonition > p:before {
margin-right: 4px; /* make room for the exclamation icon */
}
/* Fix math alignment, see https://github.com/rtfd/sphinx_rtd_theme/pull/686 */
.math {
text-align: unset;
}
</style>
<div class="section" id="Generators-&-Comprehension-Expressions">
<h1>Generators & Comprehension Expressions<a class="headerlink" href="#Generators-&-Comprehension-Expressions" title="Permalink to this headline">¶</a></h1>
<div class="admonition warning">
<p class="admonition-title fa fa-exclamation-circle"><strong>Note</strong>:</p>
<p>There are reading-comprehension exercises included throughout the text. These are meant to help you put your reading to practice. Solutions for the exercises are included at the bottom of this page.</p>
</div>
<div class="section" id="Introducing-Generators">
<h2>Introducing Generators<a class="headerlink" href="#Introducing-Generators" title="Permalink to this headline">¶</a></h2>
<p>Now we introduce an important type of object called a <strong>generator</strong>, which allows us to generate arbitrarily-many items in a series, without having to store them all in memory at once.</p>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Definition</strong>:</p>
<p>A <strong>generator</strong> is a special kind of iterator, which stores the instructions for how to <em>generate</em> each of its members, in order, along with its current state of iterations. It generates each member, one at a time, only as it is requested via iteration.</p>
</div>
<p>Recall that a list readily stores all of its members; you can access any of its contents via indexing. A generator, on the other hand, <em>does not store any items</em>. Instead, it stores the instructions for generating each of its members, and stores its iteration state; this means that the generator will know if it has generated its second member, and will thus generate its third member the next time it is iterated on.</p>
<p>The whole point of this is that you can use a generator to produce a long sequence of items, without having to store them all in memory.</p>
<div class="section" id="The-range-generator">
<h3>The <code class="docutils literal notranslate"><span class="pre">range</span></code> generator<a class="headerlink" href="#The-range-generator" title="Permalink to this headline">¶</a></h3>
<p>An extremely popular built-in generator is <code class="docutils literal notranslate"><span class="pre">range</span></code>, which, given the values:</p>
<ul class="simple">
<li><p>‘start’ (inclusive, default=0)</p></li>
<li><p>‘stop’ (exclusive)</p></li>
<li><p>‘step’ (default=1)</p></li>
</ul>
<p>will generate the corresponding sequence of integers (from start to stop, using the step size) upon iteration. Consider the following example usages of <code class="docutils literal notranslate"><span class="pre">range</span></code>:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># start: 2 (included)</span>
<span class="c1"># stop: 7 (excluded)</span>
<span class="c1"># step: 1 (default)</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">7</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="c1"># prints: 2.. 3.. 4.. 5.. 6</span>
</pre></div>
</div>
<hr class="docutils" />
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># start: 1 (included)</span>
<span class="c1"># stop: 10 (excluded)</span>
<span class="c1"># step: 2</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">2</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="c1"># prints: 1.. 3.. 5.. 7.. 9</span>
</pre></div>
</div>
<hr class="docutils" />
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># A very common use case!</span>
<span class="c1"># start: 0 (default, included)</span>
<span class="c1"># stop: 5 (excluded)</span>
<span class="c1"># step: 1 (default)</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="c1"># prints: 0.. 1.. 2.. 3.. 4</span>
</pre></div>
</div>
<p>Because <code class="docutils literal notranslate"><span class="pre">range</span></code> is a generator, the command <code class="docutils literal notranslate"><span class="pre">range(5)</span></code> will simply store the instructions needed to produce the sequence of numbers 0-4, whereas the list <code class="docutils literal notranslate"><span class="pre">[0,</span> <span class="pre">1,</span> <span class="pre">2,</span> <span class="pre">3,</span> <span class="pre">4]</span></code> stores all of these items in memory at once. For short sequences, this seems to be a rather paltry savings; this is not the case for long sequences. The following graph compares the memory consumption used when defining a generator for the sequence of numbers <span class="math notranslate nohighlight">\(0-N\)</span> using <code class="docutils literal notranslate"><span class="pre">range</span></code>, compared to storing the sequence
in a list:</p>
<p><img alt="Memory consumption figure" src="../_images/Mem_Consumption_Generator.png" /></p>
<p>Given our discussion of generators, it should make sense that the memory consumed simply by defining <code class="docutils literal notranslate"><span class="pre">range(N)</span></code> is independent of <span class="math notranslate nohighlight">\(N\)</span>, whereas the memory consumed by the list grows linearly with <span class="math notranslate nohighlight">\(N\)</span> (for large <span class="math notranslate nohighlight">\(N\)</span>).</p>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Takeaway</strong>:</p>
<p><code class="docutils literal notranslate"><span class="pre">range</span></code> is a built-in generator, which generates sequences of integers.</p>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: Using ``range``</strong>:</p>
<p>Using <code class="docutils literal notranslate"><span class="pre">range</span></code> in a for-loop, print the numbers 10-1, in sequence.</p>
</div>
</div>
</div>
<div class="section" id="Creating-your-own-generator:-generator-comprehensions">
<h2>Creating your own generator: generator comprehensions<a class="headerlink" href="#Creating-your-own-generator:-generator-comprehensions" title="Permalink to this headline">¶</a></h2>
<p>Python provides a sleek syntax for defining a simple generator in a single line of code; this expression is known as a <strong>generator comprehension</strong>. The following syntax is extremely useful and will appear very frequently in Python code:</p>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Definition</strong>:</p>
<p>The syntax <code class="docutils literal notranslate"><span class="pre">(<expression></span> <span class="pre">for</span> <span class="pre"><var></span> <span class="pre">in</span> <span class="pre"><iterable></span> <span class="pre">[if</span> <span class="pre"><condition>])</span></code> specifies the general form for a <strong>generator comprehension</strong>. This produces a generator, whose instructions for generating its members are provided within the parenthetical statement.</p>
</div>
<p>Written in a long form, the pseudo-code for</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="o"><</span><span class="n">expression</span><span class="o">></span> <span class="k">for</span> <span class="o"><</span><span class="n">var</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">iterable</span><span class="o">></span> <span class="k">if</span> <span class="o"><</span><span class="n">condition</span><span class="o">></span><span class="p">)</span>
</pre></div>
</div>
<p>is:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">for</span> <span class="o"><</span><span class="n">var</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">iterable</span><span class="o">></span><span class="p">:</span>
<span class="k">if</span> <span class="nb">bool</span><span class="p">(</span><span class="o"><</span><span class="n">condition</span><span class="o">></span><span class="p">):</span>
<span class="k">yield</span> <span class="o"><</span><span class="n">expression</span><span class="o">></span>
</pre></div>
</div>
<p>The following expression defines a generator for all the even numbers in 0-99:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># when iterated over, `even_gen` will generate 0.. 2.. 4.. ... 98</span>
<span class="n">even_gen</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span> <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="mi">2</span> <span class="o">==</span> <span class="mi">0</span><span class="p">)</span>
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">if</span> <span class="pre"><condition></span></code> clause in the generator expression is optional. The generator comprehension</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="o"><</span><span class="n">expression</span><span class="o">></span> <span class="k">for</span> <span class="o"><</span><span class="n">var</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">iterable</span><span class="o">></span><span class="p">)</span>
</pre></div>
</div>
<p>corresponds to:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">for</span> <span class="o"><</span><span class="n">var</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">iterable</span><span class="o">></span><span class="p">:</span>
<span class="k">yield</span> <span class="o"><</span><span class="n">expression</span><span class="o">></span>
</pre></div>
</div>
<p>For example:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># when iterated over, `example_gen` will generate 0/2.. 9/2.. 21/2.. 32/2</span>
<span class="n">example_gen</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span><span class="o">/</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">21</span><span class="p">,</span> <span class="mi">32</span><span class="p">])</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">example_gen</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="c1"># prints: 0.0.. 4.5.. 10.5.. 16.0</span>
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre"><expression></span></code> can be any valid single-line of Python code that returns an object:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="p">((</span><span class="n">i</span><span class="p">,</span> <span class="n">i</span><span class="o">**</span><span class="mi">2</span><span class="p">,</span> <span class="n">i</span><span class="o">**</span><span class="mi">3</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">))</span>
<span class="c1"># will generate:</span>
<span class="c1"># (0, 0, 0)</span>
<span class="c1"># (1, 1, 1)</span>
<span class="c1"># (2, 4, 8)</span>
<span class="c1"># (3, 9, 27)</span>
<span class="c1"># (4, 16, 64)</span>
<span class="c1"># (5, 25, 125)</span>
<span class="c1"># (6, 36, 216)</span>
<span class="c1"># (7, 49, 343)</span>
<span class="c1"># (8, 64, 512)</span>
<span class="c1"># (9, 81, 729)</span>
</pre></div>
</div>
<p>This means that <code class="docutils literal notranslate"><span class="pre"><expression></span></code> can even involve inline if-else statements!</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="p">((</span><span class="s2">"apple"</span> <span class="k">if</span> <span class="n">i</span> <span class="o"><</span> <span class="mi">3</span> <span class="k">else</span> <span class="s2">"pie"</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">6</span><span class="p">))</span>
<span class="c1"># will generate:</span>
<span class="c1"># 'apple'..</span>
<span class="c1"># 'apple'..</span>
<span class="c1"># 'apple'..</span>
<span class="c1"># 'pie'..</span>
<span class="c1"># 'pie'..</span>
<span class="c1"># 'pie'</span>
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Takeaway</strong>:</p>
<p>A generator comprehension is a single-line specification for defining a generator in Python. It is absolutely essential to learn this syntax in order to write simple and readable code.</p>
</div>
<div class="admonition warning">
<p class="admonition-title fa fa-exclamation-circle"><strong>Note</strong>:</p>
<p>Generator comprehensions are <strong>not</strong> the only method for defining generators in Python. One can define a generator similar to the way one can define a function (which we will encounter soon). <a class="reference external" href="https://docs.python.org/3/tutorial/classes.html#generators">See this section of the official Python tutorial</a> if you are interested in diving deeper into generators.</p>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: Writing a Generator Comprehension</strong>:</p>
<p>Using a generator comprehension, define a generator for the series:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span><span class="o">..</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span><span class="o">..</span> <span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">)</span><span class="o">..</span> <span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span><span class="o">..</span> <span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">7</span><span class="p">)</span>
</pre></div>
</div>
<p>Note that (3, 5) is <em>not</em> in the series.</p>
<p>Iterate over the generator and print its contents to verify your solution.</p>
</div>
<div class="section" id="Storing-generators">
<h3>Storing generators<a class="headerlink" href="#Storing-generators" title="Permalink to this headline">¶</a></h3>
<p>Just like we saw with the <code class="docutils literal notranslate"><span class="pre">range</span></code> generator, defining a generator using a comprehension does <em>not</em> perform any computations or consume any memory beyond defining the rules for producing the sequence of data. See what happens when we try to print this generator:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># will generate 0, 1, 4, 9, 25, ..., 9801</span>
<span class="o">>>></span> <span class="n">gen</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100</span><span class="p">))</span>
<span class="o">>>></span> <span class="nb">print</span><span class="p">(</span><span class="n">gen</span><span class="p">)</span>
<span class="o"><</span><span class="n">generator</span> <span class="nb">object</span> <span class="o"><</span><span class="n">genexpr</span><span class="o">></span> <span class="n">at</span> <span class="mh">0x000001E768FE8A40</span><span class="o">></span>
</pre></div>
</div>
<p>This output simply indicates that <code class="docutils literal notranslate"><span class="pre">gen</span></code> stores a generator-expression at the memory address <code class="docutils literal notranslate"><span class="pre">0x000001E768FE8A40</span></code>; this is simply where the instructions for generating our sequence of squared numbers is stored. <code class="docutils literal notranslate"><span class="pre">gen</span></code> will not produce any results until we iterate over it. For this reason, generators cannot be inspected in the same way that lists and other sequences can be. You <strong>cannot</strong> do the following:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># you **cannot** do the following...</span>
<span class="o">>>></span> <span class="n">gen</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100</span><span class="p">))</span>
<span class="c1"># query the length of a generator</span>
<span class="o">>>></span> <span class="nb">len</span><span class="p">(</span><span class="n">gen</span><span class="p">)</span>
<span class="ne">TypeError</span><span class="p">:</span> <span class="nb">object</span> <span class="n">of</span> <span class="nb">type</span> <span class="s1">'generator'</span> <span class="n">has</span> <span class="n">no</span> <span class="nb">len</span><span class="p">()</span>
<span class="c1"># index into a generator</span>
<span class="o">>>></span> <span class="n">gen</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
<span class="ne">TypeError</span><span class="p">:</span> <span class="s1">'generator'</span> <span class="nb">object</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">subscriptable</span>
</pre></div>
</div>
<p>The sole exception to this is the <code class="docutils literal notranslate"><span class="pre">range</span></code> generator, for which all of these inspections are valid.</p>
</div>
<div class="section" id="Consuming-generators">
<h3>Consuming generators<a class="headerlink" href="#Consuming-generators" title="Permalink to this headline">¶</a></h3>
<p>We can feed this to any function that accepts iterables. For instance, we can feed <code class="docutils literal notranslate"><span class="pre">gen</span></code> to the built-in <code class="docutils literal notranslate"><span class="pre">sum</span></code> function, which sums the contents of an iterable:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">gen</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100</span><span class="p">))</span>
<span class="gp">>>> </span><span class="nb">sum</span><span class="p">(</span><span class="n">gen</span><span class="p">)</span> <span class="c1"># computes the sum 0 + 1 + 4 + 9 + 25 + ... + 9801</span>
<span class="go">328350</span>
</pre></div>
</div>
<p>This computes the sum of the sequence of numbers <em>without ever storing the full sequence of numbers in memory</em>. In fact, only two numbers need be stored during any given iteration of the sum: the current value of the sum, and the number being added to it.</p>
<p>What happens if we run this command a second time:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># computes the sum of ... nothing!</span>
<span class="c1"># `gen` has already been consumed!</span>
<span class="o">>>></span> <span class="nb">sum</span><span class="p">(</span><span class="n">gen</span><span class="p">)</span>
<span class="mi">0</span>
</pre></div>
</div>
<p>It may be surprising to see that the sum now returns 0. This is because <strong>a generator is exhausted after it is iterated over in full</strong>. You must redefine the generator if you want to iterate over it again; fortunately, defining a generator requires very few resources, so this is not a point of concern.</p>
<p>You can also check for membership in a generator, but this also consumes the generator:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># checking for membership consumes a generator until</span>
<span class="c1"># it finds that item (consuming the entire generator</span>
<span class="c1"># if the item is not contained within it)</span>
<span class="o">>>></span> <span class="n">gen</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">11</span><span class="p">))</span>
<span class="o">>>></span> <span class="mi">5</span> <span class="ow">in</span> <span class="n">gen</span> <span class="c1"># first 5 elements are consumed</span>
<span class="kc">True</span>
<span class="c1"># 1-5 are no longer contained in gen</span>
<span class="c1"># this check consumes the entire generator!</span>
<span class="o">>>></span> <span class="mi">5</span> <span class="ow">in</span> <span class="n">gen</span>
<span class="kc">False</span>
<span class="o">>>></span> <span class="nb">sum</span><span class="p">(</span><span class="n">gen</span><span class="p">)</span>
<span class="mi">0</span>
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Takeaway</strong>:</p>
<p>A generator can only be iterated over once, after which it is exhausted and must be re-defined in order to be iterated over again.</p>
</div>
</div>
<div class="section" id="Chaining-comprehensions">
<h3>Chaining comprehensions<a class="headerlink" href="#Chaining-comprehensions" title="Permalink to this headline">¶</a></h3>
<p>Because generators are iterables, they can be fed into subsequent generator comprehensions. That is, they can be “chained” together.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># chaining two generator comprehensions</span>
<span class="c1"># generates 400.. 100.. 0.. 100.. 400</span>
<span class="o">>>></span> <span class="n">gen_1</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="p">[</span><span class="o">-</span><span class="mi">20</span><span class="p">,</span> <span class="o">-</span><span class="mi">10</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">20</span><span class="p">])</span>
<span class="c1"># iterates through gen_1, excluding any numbers whose absolute value is greater than 150</span>
<span class="o">>>></span> <span class="n">gen_2</span> <span class="o">=</span> <span class="p">(</span><span class="n">j</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">gen_1</span> <span class="k">if</span> <span class="nb">abs</span><span class="p">(</span><span class="n">j</span><span class="p">)</span> <span class="o"><=</span> <span class="mi">150</span><span class="p">)</span>
<span class="c1"># computing 100 + 0 + 100</span>
<span class="o">>>></span> <span class="nb">sum</span><span class="p">(</span><span class="n">gen_2</span><span class="p">)</span>
<span class="mi">200</span>
</pre></div>
</div>
<p>This is equivalent to:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">total</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="p">[</span><span class="o">-</span><span class="mi">20</span><span class="p">,</span> <span class="o">-</span><span class="mi">10</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">20</span><span class="p">]:</span>
<span class="n">j</span> <span class="o">=</span> <span class="n">i</span> <span class="o">**</span> <span class="mi">2</span>
<span class="k">if</span> <span class="n">j</span> <span class="o"><=</span> <span class="mi">150</span><span class="p">:</span>
<span class="n">total</span> <span class="o">+=</span> <span class="n">j</span>
<span class="c1"># total is now 200</span>
</pre></div>
</div>
</div>
<div class="section" id="Using-generator-comprehensions-on-the-fly">
<h3>Using generator comprehensions on the fly<a class="headerlink" href="#Using-generator-comprehensions-on-the-fly" title="Permalink to this headline">¶</a></h3>
<p>A feature of Python, that can make your code supremely readable and intuitive, is that generator comprehensions can be fed <em>directly</em> into functions that operate on iterables. That is,</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">gen</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100</span><span class="p">))</span>
<span class="gp">>>> </span><span class="nb">sum</span><span class="p">(</span><span class="n">gen</span><span class="p">)</span>
<span class="go">328350</span>
</pre></div>
</div>
<p>can be simplified as:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nb">sum</span><span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100</span><span class="p">))</span>
<span class="go">328350</span>
</pre></div>
</div>
<p>If you want your code to compute the finite harmonic series: <span class="math notranslate nohighlight">\(\sum_{k=1}^{100} \frac{1}{n} = 1 + \frac{1}{2} + ... + \frac{1}{100}\)</span>, you can simply write:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nb">sum</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="n">n</span> <span class="k">for</span> <span class="n">n</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">101</span><span class="p">))</span>
<span class="go">5.187377517639621</span>
</pre></div>
</div>
<p>This convenient syntax works for any function that expects an iterable as an argument, such as the <code class="docutils literal notranslate"><span class="pre">list</span></code> function and <code class="docutils literal notranslate"><span class="pre">all</span></code> function:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># providing generator expressions as arguments to functions</span>
<span class="c1"># that operate on iterables</span>
<span class="o">>>></span> <span class="nb">list</span><span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">))</span>
<span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">25</span><span class="p">,</span> <span class="mi">36</span><span class="p">,</span> <span class="mi">49</span><span class="p">,</span> <span class="mi">64</span><span class="p">,</span> <span class="mi">81</span><span class="p">]</span>
<span class="o">>>></span> <span class="nb">all</span><span class="p">(</span><span class="n">i</span> <span class="o"><</span> <span class="mi">10</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">7</span><span class="p">])</span>
<span class="kc">True</span>
<span class="o">>>></span> <span class="s2">", "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="p">[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">200</span><span class="p">,</span> <span class="mi">4000</span><span class="p">,</span> <span class="mi">80000</span><span class="p">])</span>
<span class="s1">'10, 200, 4000, 80000'</span>
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Takeaway</strong>:</p>
<p>A generator comprehension can be specified directly as an argument to a function, wherever a single iterable is expected as an input to that function.</p>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: Using Generator Comprehensions on the Fly</strong>:</p>
<p>In a single line, compute the sum of all of the odd-numbers in 0-100.</p>
</div>
</div>
</div>
<div class="section" id="Iterating-over-generators-using-next">
<h2>Iterating over generators using <code class="docutils literal notranslate"><span class="pre">next</span></code><a class="headerlink" href="#Iterating-over-generators-using-next" title="Permalink to this headline">¶</a></h2>
<p>The built-in function <code class="docutils literal notranslate"><span class="pre">next</span></code> allows you manually “request” the next member of a generator, or more generally, any kind of <em>iterator</em>. Calling <code class="docutils literal notranslate"><span class="pre">next</span></code> on an exhausted iterator will raise a <code class="docutils literal notranslate"><span class="pre">StopIteration</span></code> signal.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># consuming an iterator using `next`</span>
<span class="o">>>></span> <span class="n">short_gen</span> <span class="o">=</span> <span class="p">(</span><span class="n">i</span><span class="o">/</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">])</span>
<span class="o">>>></span> <span class="nb">next</span><span class="p">(</span><span class="n">short_gen</span><span class="p">)</span>
<span class="mf">0.5</span>
<span class="o">>>></span> <span class="nb">next</span><span class="p">(</span><span class="n">short_gen</span><span class="p">)</span>
<span class="mf">1.0</span>
<span class="o">>>></span> <span class="nb">next</span><span class="p">(</span><span class="n">short_gen</span><span class="p">)</span>
<span class="mf">1.5</span>
<span class="o">>>></span> <span class="nb">next</span><span class="p">(</span><span class="n">short_gen</span><span class="p">)</span>
<span class="ne">StopIteration</span>
<span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">)</span>
<span class="o"><</span><span class="n">ipython</span><span class="o">-</span><span class="nb">input</span><span class="o">-</span><span class="mi">5</span><span class="o">-</span><span class="n">ed60a54ccf0b</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">module</span><span class="o">></span><span class="p">()</span>
<span class="o">----></span> <span class="mi">1</span> <span class="nb">next</span><span class="p">(</span><span class="n">short_gen</span><span class="p">)</span>
<span class="ne">StopIteration</span><span class="p">:</span>
</pre></div>
</div>
<p>This is a great tool for retrieving content from a generator, or any iterator, without having to perform a for-loop over it.</p>
<div class="section" id="Iterables-vs. Iterators">
<h3>Iterables vs. Iterators<a class="headerlink" href="#Iterables-vs. Iterators" title="Permalink to this headline">¶</a></h3>
<p>This subsection is <em>not</em> essential to your basic understanding of the material. I am including it to prevent this text from being misleading to those who already know quite a bit about Python. <strong>This is a bit advanced, feel free to skip it…</strong></p>
<p>There is a bit of confusing terminology to be cleared up: an iterable is not the same thing as an iterator.</p>
<p>An <em>iterator</em> object stores its current state of iteration and “yields” each of its members in order, on demand via <code class="docutils literal notranslate"><span class="pre">next</span></code>, until it is exhausted. As we’ve seen, a generator is an example of an iterator. We now must understand that every iterator is an iterable, but not every iterable is an iterator.</p>
<p>An <em>iterable</em> is an object that <em>can</em> be iterated over but does not necessarily have all the machinery of an iterator. For example, sequences (e.g lists, tuples, and strings) and other containers (e.g. dictionaries and sets) do not keep track of their own state of iteration. Thus you cannot call <code class="docutils literal notranslate"><span class="pre">next</span></code> on one of these outright:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># a list is an example of an iterable that is *not*</span>
<span class="c1"># an iterator - you cannot call `next` on it.</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="o">>>></span> <span class="nb">next</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="o">---------------------------------------------------------------------------</span>
<span class="ne">TypeError</span> <span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">)</span>
<span class="o"><</span><span class="n">ipython</span><span class="o">-</span><span class="nb">input</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="n">b9d20096048c</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">module</span><span class="o">></span><span class="p">()</span>
<span class="o">----></span> <span class="mi">1</span> <span class="nb">next</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">])</span>
<span class="ne">TypeError</span><span class="p">:</span> <span class="s1">'list'</span> <span class="nb">object</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">an</span> <span class="n">iterator</span>
</pre></div>
</div>
<p>In order to iterate over, say, a list you must first pass it to the built-in <code class="docutils literal notranslate"><span class="pre">iter</span></code> function. This function will return an <em>iterator</em> for that list, which stores its state of iteration and the instructions to yield each one of the list’s members:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># any iterable can be fed to `iter`</span>
<span class="c1"># to produce an iterator for that object</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="o">>>></span> <span class="n">x_it</span> <span class="o">=</span> <span class="nb">iter</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># `x_it` is an iterator</span>
<span class="o">>>></span> <span class="nb">next</span><span class="p">(</span><span class="n">x_it</span><span class="p">)</span>
<span class="mi">1</span>
<span class="o">>>></span> <span class="nb">next</span><span class="p">(</span><span class="n">x_it</span><span class="p">)</span>
<span class="mi">2</span>
<span class="o">>>></span> <span class="nb">next</span><span class="p">(</span><span class="n">x_it</span><span class="p">)</span>
<span class="mi">3</span>
</pre></div>
</div>
<p>In this way, a list is an <em>iterable</em> but not an <em>iterator</em>, which is also the case for tuples, strings, sets, and dictionaries.</p>
<p>Python actually creates an iterator “behind the scenes”, whenever you perform a for-loop over an iterable like a list. It feeds that iterable to <code class="docutils literal notranslate"><span class="pre">iter</span></code>, and then proceeds to call <code class="docutils literal notranslate"><span class="pre">next</span></code> on the resulting iterator for each of the for-loop’s iterations.</p>
</div>
</div>
<div class="section" id="List-&-Tuple-Comprehensions">
<h2>List & Tuple Comprehensions<a class="headerlink" href="#List-&-Tuple-Comprehensions" title="Permalink to this headline">¶</a></h2>
<p>Using generator comprehensions to initialize lists is so useful that Python actually reserves a specialized syntax for it, known as the list comprehension. A <strong>list comprehension</strong> is a syntax for constructing a list, which exactly mirrors the generator comprehension syntax:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="o"><</span><span class="n">expression</span><span class="o">></span> <span class="k">for</span> <span class="o"><</span><span class="n">var</span><span class="o">></span> <span class="ow">in</span> <span class="o"><</span><span class="n">iterable</span><span class="o">></span> <span class="p">{</span><span class="k">if</span> <span class="o"><</span><span class="n">condition</span><span class="p">}]</span>
</pre></div>
</div>
<p>For example, if we want to create a list of square-numbers, we can simply write:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># a simple list comprehension</span>
<span class="o">>>></span> <span class="p">[</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span>
<span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">25</span><span class="p">,</span> <span class="mi">36</span><span class="p">,</span> <span class="mi">49</span><span class="p">,</span> <span class="mi">64</span><span class="p">,</span> <span class="mi">81</span><span class="p">]</span>
</pre></div>
</div>
<p>This produces the exact same result as feeding the <code class="docutils literal notranslate"><span class="pre">list</span></code> function a generator comprehension. However, using a list comprehension is slightly more efficient than is feeding the <code class="docutils literal notranslate"><span class="pre">list</span></code> function a generator comprehension.</p>
<p>Let’s appreciate how economical list comprehensions are. The following code stores words that contain the letter “o”, in a list:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">words_with_o</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">word_collection</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'Python'</span><span class="p">,</span> <span class="s1">'Like'</span><span class="p">,</span> <span class="s1">'You'</span><span class="p">,</span> <span class="s1">'Mean'</span><span class="p">,</span> <span class="s1">'It'</span><span class="p">]</span>
<span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="n">word_collection</span><span class="p">:</span>
<span class="k">if</span> <span class="s2">"o"</span> <span class="ow">in</span> <span class="n">word</span><span class="o">.</span><span class="n">lower</span><span class="p">():</span>
<span class="n">words_with_o</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">word</span><span class="p">)</span>
</pre></div>
</div>
<p>This can be written in a single line, using a list comprehension:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">word_collection</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'Python'</span><span class="p">,</span> <span class="s1">'Like'</span><span class="p">,</span> <span class="s1">'You'</span><span class="p">,</span> <span class="s1">'Mean'</span><span class="p">,</span> <span class="s1">'It'</span><span class="p">]</span>
<span class="gp">>>> </span><span class="n">words_with_o</span> <span class="o">=</span> <span class="p">[</span><span class="n">word</span> <span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="n">word_collection</span> <span class="k">if</span> <span class="s2">"o"</span> <span class="ow">in</span> <span class="n">word</span><span class="o">.</span><span class="n">lower</span><span class="p">()]</span>
<span class="gp">>>> </span><span class="n">words_with_o</span>
<span class="go">['Python', 'You']</span>
</pre></div>
</div>
<p>Tuples can be created using comprehension expressions too, but we must explicitly invoke the <code class="docutils literal notranslate"><span class="pre">tuple</span></code> constructor since parentheses are already reserved for defining a generator-comprehension.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># creating a tuple using a comprehension expression</span>
<span class="o">>>></span> <span class="nb">tuple</span><span class="p">(</span><span class="n">i</span><span class="o">**</span><span class="mi">2</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">))</span>
<span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">16</span><span class="p">)</span>
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Takeaway</strong>:</p>
<p>The comprehensions-statement is an extremely useful syntax for creating simple and complicated lists and tuples alike.</p>
</div>
<div class="section" id="Nesting-comprehensions">
<h3>Nesting comprehensions<a class="headerlink" href="#Nesting-comprehensions" title="Permalink to this headline">¶</a></h3>
<p>It can be useful to nest comprehension expressions within one another, although this should be used sparingly.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Nested list comprehensions.</span>
<span class="c1"># This creates a 3x4 "matrix" (list of lists) of zeros.</span>
<span class="o">>>></span> <span class="p">[[</span><span class="mi">0</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">4</span><span class="p">)]</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">3</span><span class="p">)]</span>
<span class="p">[[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">]]</span>
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: List Comprehensions</strong>:</p>
<p>Use a list comprehension to create a list that contains the string “hello” 100 times.</p>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: Fancier List Comprehensions</strong>:</p>
<p>Use the inline <code class="docutils literal notranslate"><span class="pre">if-else</span></code> statement (discussed earlier in this module), along with a list comprehension, to create the list:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="s1">'hello'</span><span class="p">,</span>
<span class="s1">'goodbye'</span><span class="p">,</span>
<span class="s1">'hello'</span><span class="p">,</span>
<span class="s1">'goodbye'</span><span class="p">,</span>
<span class="s1">'hello'</span><span class="p">,</span>
<span class="s1">'goodbye'</span><span class="p">,</span>
<span class="s1">'hello'</span><span class="p">,</span>
<span class="s1">'goodbye'</span><span class="p">,</span>
<span class="s1">'hello'</span><span class="p">,</span>
<span class="s1">'goodbye'</span><span class="p">]</span>
</pre></div>
</div>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: Tuple Comprehensions</strong>:</p>
<p>Use a tuple-comprehension to extract comma-separated numbers from a string, converting them into a tuple of floats. I.e. <code class="docutils literal notranslate"><span class="pre">"3.2,2.4,99.8"</span></code> should become <code class="docutils literal notranslate"><span class="pre">(3.2,</span> <span class="pre">2.4,</span> <span class="pre">99.8)</span></code>. You will want to use the built-in string function <a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str.split">str.split</a>.</p>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: Translating a For-Loop</strong>:</p>
<p>Replicate the functionality of the the following code by writing a list comprehension.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># skip all non-lowercased letters (including punctuation)</span>
<span class="c1"># append 1 if lowercase letter is equal to "o"</span>
<span class="c1"># append 0 if lowercase letter is not "o"</span>
<span class="n">out</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="s2">"Hello. How Are You?"</span><span class="p">:</span>
<span class="k">if</span> <span class="n">i</span><span class="o">.</span><span class="n">islower</span><span class="p">():</span>
<span class="n">out</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="mi">1</span> <span class="k">if</span> <span class="n">i</span> <span class="o">==</span> <span class="s2">"o"</span> <span class="k">else</span> <span class="mi">0</span><span class="p">)</span>
</pre></div>
</div>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: Memory Efficiency</strong>:</p>
<p>Is there any difference in performance between the following expressions?</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># feeding `sum` a generator comprehension</span>
<span class="nb">sum</span><span class="p">(</span><span class="mi">1</span><span class="o">/</span><span class="n">n</span> <span class="k">for</span> <span class="n">n</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">101</span><span class="p">))</span>
</pre></div>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># feeding `sum` a list comprehension</span>
<span class="nb">sum</span><span class="p">([</span><span class="mi">1</span><span class="o">/</span><span class="n">n</span> <span class="k">for</span> <span class="n">n</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">101</span><span class="p">)])</span>
</pre></div>
</div>
<p>Is one expression preferable over the other? Why?</p>
</div>
</div>
</div>
<div class="section" id="Links-to-Official-Documentation">
<h2>Links to Official Documentation<a class="headerlink" href="#Links-to-Official-Documentation" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li><p><a class="reference external" href="https://docs.python.org/3/glossary.html#term-generator">Generator definition</a></p></li>
<li><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#typesseq-range">range</a></p></li>
<li><p><a class="reference external" href="https://docs.python.org/3/tutorial/classes.html#generator-expressions">Generator comprehension expressions</a></p></li>
<li><p><a class="reference external" href="https://docs.python.org/3/glossary.html#term-iterator">Iterator definition</a></p></li>
<li><p><a class="reference external" href="https://docs.python.org/3/library/functions.html#next">next</a></p></li>
<li><p><a class="reference external" href="https://docs.python.org/3/library/functions.html#iter">iter</a></p></li>
<li><p><a class="reference external" href="https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions">List comprehensions</a></p></li>
<li><p><a class="reference external" href="https://docs.python.org/3/tutorial/datastructures.html#nested-list-comprehensions">Nested list comprehensions</a></p></li>
</ul>
</div>
<div class="section" id="Reading-Comprehension-Exercise-Solutions:">
<h2>Reading Comprehension Exercise Solutions:<a class="headerlink" href="#Reading-Comprehension-Exercise-Solutions:" title="Permalink to this headline">¶</a></h2>
<p><strong>Using range: Solution</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># start=10, stop=0 (excluded), step-size=-1</span>
<span class="o">>>></span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">):</span>
<span class="o">>>></span> <span class="nb">print</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">end</span><span class="o">=</span><span class="s2">" "</span><span class="p">)</span> <span class="c1"># the "end" parameter is to avoid each value taking up a new line</span>
<span class="mi">10</span> <span class="mi">9</span> <span class="mi">8</span> <span class="mi">7</span> <span class="mi">6</span> <span class="mi">5</span> <span class="mi">4</span> <span class="mi">3</span> <span class="mi">2</span> <span class="mi">1</span>
</pre></div>
</div>
<p><strong>Writing a Generator Comprehension: Solution</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">g</span> <span class="o">=</span> <span class="p">((</span><span class="n">n</span><span class="p">,</span> <span class="n">n</span><span class="o">+</span><span class="mi">2</span><span class="p">)</span> <span class="k">for</span> <span class="n">n</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">6</span><span class="p">)</span> <span class="k">if</span> <span class="n">n</span> <span class="o">!=</span> <span class="mi">3</span><span class="p">)</span>
<span class="gp">>>> </span><span class="nb">list</span><span class="p">(</span><span class="n">g</span><span class="p">)</span> <span class="c1"># convert into a list to print values</span>
<span class="go">[(0, 2), (1, 3), (2, 4), (4, 6), (5, 7)]</span>
</pre></div>
</div>
<p><strong>Using Generator Comprehensions on the Fly: Solution</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nb">sum</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">101</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
<span class="go">2500</span>
</pre></div>
</div>
<p>or</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nb">sum</span><span class="p">(</span><span class="n">i</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">101</span><span class="p">)</span> <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="mi">2</span> <span class="o">!=</span> <span class="mi">0</span><span class="p">)</span>
<span class="go">2500</span>
</pre></div>
</div>
<p><strong>List Comprehensions: Solution</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="p">[</span><span class="s2">"hello"</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">100</span><span class="p">)]</span>
<span class="go">['hello', 'hello', ..., 'hello', 'hello'] # 100 hello's</span>
</pre></div>
</div>
<p><strong>Fancier List Comprehensions: Solution</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="p">[(</span><span class="s2">"hello"</span> <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="mi">2</span> <span class="o">==</span> <span class="mi">0</span> <span class="k">else</span> <span class="s2">"goodbye"</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span>
<span class="go">['hello', 'goodbye', 'hello', 'goodbye', 'hello', 'goodbye', 'hello', 'goodbye', 'hello', 'goodbye']</span>
</pre></div>
</div>
<p><strong>Tuple Comprehension: Solution</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">string_of_nums</span> <span class="o">=</span> <span class="s2">"3.2, 2.4, 99.8"</span>
<span class="gp">>>> </span><span class="nb">tuple</span><span class="p">(</span><span class="nb">float</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">string_of_nums</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">","</span><span class="p">))</span>
<span class="go">(3.2, 2.4, 99.8)</span>
</pre></div>
</div>
<p><strong>Translating a For-Loop: Solution</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">out</span> <span class="o">=</span> <span class="p">[(</span><span class="mi">1</span> <span class="k">if</span> <span class="n">i</span> <span class="o">==</span> <span class="s2">"o"</span> <span class="k">else</span> <span class="mi">0</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="s2">"Hello. How Are You?"</span> <span class="k">if</span> <span class="n">i</span><span class="o">.</span><span class="n">islower</span><span class="p">()]</span>
<span class="gp">>>> </span><span class="n">out</span>
<span class="go">[0, 0, 0, 1, 1, 0, 0, 0, 1, 0]</span>
</pre></div>
</div>
<p><strong>Memory Efficiency: Solution</strong></p>
<p>It is preferable to use the generator expression <code class="docutils literal notranslate"><span class="pre">sum(1/n</span> <span class="pre">for</span> <span class="pre">n</span> <span class="pre">in</span> <span class="pre">range(1,</span> <span class="pre">101))</span></code>, rather than the list comprehension <code class="docutils literal notranslate"><span class="pre">sum([1/n</span> <span class="pre">for</span> <span class="pre">n</span> <span class="pre">in</span> <span class="pre">range(1,</span> <span class="pre">101)])</span></code>. Using a list comprehension unnecessarily creates a list of the one hundred numbers, in memory, before feeding the list to <code class="docutils literal notranslate"><span class="pre">sum</span></code>. The generator expression need only produce a single value at a time, as <code class="docutils literal notranslate"><span class="pre">sum</span></code> iterates over it.</p>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="Itertools.html" class="btn btn-neutral float-right" title="Python’s “Itertools”" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
<a href="Iterables.html" class="btn btn-neutral float-left" title="Iterables" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
© Copyright 2021, Ryan Soklaski.
</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>