-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.xml
905 lines (846 loc) · 150 KB
/
index.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>James Sevedge</title>
<link>https://jamessevedge.com/</link>
<description>Recent content in Home on James Sevedge</description>
<generator>Hugo -- gohugo.io</generator>
<language>en-us</language>
<managingEditor>jjesc1031@gmail.com (James Sevedge)</managingEditor>
<webMaster>jjesc1031@gmail.com (James Sevedge)</webMaster>
<lastBuildDate>Wed, 30 Mar 2022 00:00:00 +0000</lastBuildDate><atom:link href="https://jamessevedge.com/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>Introduction to Jupyter Notebooks</title>
<link>https://jamessevedge.com/articles/introduction-to-jupyter-notebooks/</link>
<pubDate>Wed, 30 Mar 2022 00:00:00 +0000</pubDate>
<author>jjesc1031@gmail.com (James Sevedge)</author>
<guid>https://jamessevedge.com/articles/introduction-to-jupyter-notebooks/</guid>
<description><p>It has been on my bucket list for awhile now to expore <a href="https://jupyter.org">Jupyter Notebooks</a> as a solution for data analysis and computational story telling needs. I finally took the time to sit down and understand the tooling and authoring experience and figured I would document the process and share it.</p>
<h3 id="installation-process">Installation process</h3>
<p>My local development environment is a MacBook Pro running Catalina. I started out by installing and launching the Jupyter Notebook server using <a href="https://www.anaconda.com/products/distribution">Anaconda</a> which was uber simple. I did want to be able to reproduce the installation and execution steps in a containerized pipeline so I ended up just doing a python package install after a bit of research on core python packages necessary.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ pip install notebook pandas matplotlib
</code></pre></div><p><strong>Note:</strong> For a pipeline of course I would pin the package versions in a <code>requirements.txt</code> file to ensure I had reproducible builds.</p>
<h3 id="trying-it-out">Trying it out</h3>
<p>Now that I have the Jupyter CLI installed I can start the Jupyter notebook server which opens a browser to localhost:8088 with a simple file explorer and a few options including creating a new notebook.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ jupyter notebook
<span style="color:#f92672">[</span>I NotebookApp<span style="color:#f92672">]</span> Serving notebooks from local directory: /Users/a_user/Documents
<span style="color:#f92672">[</span>I NotebookApp<span style="color:#f92672">]</span> Jupyter Notebook 6.4.10 is running at:
<span style="color:#f92672">[</span>I NotebookApp<span style="color:#f92672">]</span> http://localhost:8888/?token<span style="color:#f92672">=</span>a_token
<span style="color:#f92672">[</span>I NotebookApp<span style="color:#f92672">]</span> or http://127.0.0.1:8888/?token<span style="color:#f92672">=</span>a_token
<span style="color:#f92672">[</span>I NotebookApp<span style="color:#f92672">]</span> Use Control-C to stop this server and shut down all kernels <span style="color:#f92672">(</span>twice to skip confirmation<span style="color:#f92672">)</span>.
</code></pre></div><p>I selected the option to create a notebook and found myself staring at an empty Jupyter Notebook.</p>
<p><img src="https://jamessevedge.com/jupyter-notebook-new.png" alt="Jupyter Notebook New"></p>
<h3 id="finding-some-data">Finding some data</h3>
<p>To create a useful Jupyter notebook I needed a simple data set so I downloaded a CSV file containing daily trading data on the S&amp;P 500 for the 20 years starting from 2000. Here is what a sample of that data looks like.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ wc -l data.csv
<span style="color:#ae81ff">5000</span> data.csv
$ tail data.csv
<span style="color:#e6db74">&#34;Jan 14, 2000&#34;</span>,<span style="color:#e6db74">&#34;1,465.20&#34;</span>,<span style="color:#e6db74">&#34;1,449.70&#34;</span>,<span style="color:#e6db74">&#34;1,473.00&#34;</span>,<span style="color:#e6db74">&#34;1,449.70&#34;</span>,<span style="color:#e6db74">&#34;-&#34;</span>,<span style="color:#e6db74">&#34;1.07%&#34;</span>
<span style="color:#e6db74">&#34;Jan 13, 2000&#34;</span>,<span style="color:#e6db74">&#34;1,449.70&#34;</span>,<span style="color:#e6db74">&#34;1,432.20&#34;</span>,<span style="color:#e6db74">&#34;1,454.60&#34;</span>,<span style="color:#e6db74">&#34;1,432.20&#34;</span>,<span style="color:#e6db74">&#34;-&#34;</span>,<span style="color:#e6db74">&#34;1.22%&#34;</span>
<span style="color:#e6db74">&#34;Jan 12, 2000&#34;</span>,<span style="color:#e6db74">&#34;1,432.20&#34;</span>,<span style="color:#e6db74">&#34;1,439.10&#34;</span>,<span style="color:#e6db74">&#34;1,445.30&#34;</span>,<span style="color:#e6db74">&#34;1,427.30&#34;</span>,<span style="color:#e6db74">&#34;-&#34;</span>,<span style="color:#e6db74">&#34;-0.44%&#34;</span>
<span style="color:#e6db74">&#34;Jan 11, 2000&#34;</span>,<span style="color:#e6db74">&#34;1,438.60&#34;</span>,<span style="color:#e6db74">&#34;1,457.60&#34;</span>,<span style="color:#e6db74">&#34;1,458.80&#34;</span>,<span style="color:#e6db74">&#34;1,434.40&#34;</span>,<span style="color:#e6db74">&#34;-&#34;</span>,<span style="color:#e6db74">&#34;-1.30%&#34;</span>
<span style="color:#e6db74">&#34;Jan 10, 2000&#34;</span>,<span style="color:#e6db74">&#34;1,457.60&#34;</span>,<span style="color:#e6db74">&#34;1,441.50&#34;</span>,<span style="color:#e6db74">&#34;1,464.40&#34;</span>,<span style="color:#e6db74">&#34;1,441.50&#34;</span>,<span style="color:#e6db74">&#34;-&#34;</span>,<span style="color:#e6db74">&#34;1.12%&#34;</span>
<span style="color:#e6db74">&#34;Jan 07, 2000&#34;</span>,<span style="color:#e6db74">&#34;1,441.50&#34;</span>,<span style="color:#e6db74">&#34;1,403.50&#34;</span>,<span style="color:#e6db74">&#34;1,441.50&#34;</span>,<span style="color:#e6db74">&#34;1,400.50&#34;</span>,<span style="color:#e6db74">&#34;-&#34;</span>,<span style="color:#e6db74">&#34;2.71%&#34;</span>
<span style="color:#e6db74">&#34;Jan 06, 2000&#34;</span>,<span style="color:#e6db74">&#34;1,403.50&#34;</span>,<span style="color:#e6db74">&#34;1,402.10&#34;</span>,<span style="color:#e6db74">&#34;1,411.90&#34;</span>,<span style="color:#e6db74">&#34;1,392.00&#34;</span>,<span style="color:#e6db74">&#34;-&#34;</span>,<span style="color:#e6db74">&#34;0.10%&#34;</span>
<span style="color:#e6db74">&#34;Jan 05, 2000&#34;</span>,<span style="color:#e6db74">&#34;1,402.10&#34;</span>,<span style="color:#e6db74">&#34;1,399.40&#34;</span>,<span style="color:#e6db74">&#34;1,413.30&#34;</span>,<span style="color:#e6db74">&#34;1,377.70&#34;</span>,<span style="color:#e6db74">&#34;-&#34;</span>,<span style="color:#e6db74">&#34;0.19%&#34;</span>
<span style="color:#e6db74">&#34;Jan 04, 2000&#34;</span>,<span style="color:#e6db74">&#34;1,399.40&#34;</span>,<span style="color:#e6db74">&#34;1,455.20&#34;</span>,<span style="color:#e6db74">&#34;1,455.20&#34;</span>,<span style="color:#e6db74">&#34;1,397.40&#34;</span>,<span style="color:#e6db74">&#34;-&#34;</span>,<span style="color:#e6db74">&#34;-3.83%&#34;</span>
<span style="color:#e6db74">&#34;Jan 03, 2000&#34;</span>,<span style="color:#e6db74">&#34;1,455.20&#34;</span>,<span style="color:#e6db74">&#34;1,469.20&#34;</span>,<span style="color:#e6db74">&#34;1,478.00&#34;</span>,<span style="color:#e6db74">&#34;1,438.40&#34;</span>,<span style="color:#e6db74">&#34;-&#34;</span>,<span style="color:#e6db74">&#34;-0.95%&#34;</span>
</code></pre></div><p>Now of course I could import this data set into Excel, Tableau, etc. to perform data visualization but the point of this exercise was to evaluate Jupyter notebooks which allow for not only data visualization but also data preparation (<a href="https://pandas.pydata.org">pandas</a>), machine learning (<a href="https://scikit-learn.org/stable/">scikit-learn</a>), etc. In general it has greater flexibility in functionality as the design allows for computational storytelling given the free form ability to add rows of markdown mixed in with the computational rows.</p>
<h3 id="creating-the-notebook">Creating the notebook</h3>
<p>Most of the learning curve came while trying to undertand how to do data preparation and visualization with <a href="https://pandas.pydata.org">pandas</a>. To do that I looked at the documentation on their <a href="https://pandas.pydata.org/docs/">site</a> until I got a basic understanding of the steps necessary to view, clean and visualize a data frame. A neat thing about Jupyter notebooks is you can run each step independently of the rest so if some steps take a long time to complete you can run those once and iterate on subsequent steps.</p>
<h3 id="exporting-the-notebook">Exporting the notebook</h3>
<p>Once I had a working notebook I wanted to export it so I could share it with other people easily. The Jupyter CLI has a command called <code>jupyter nbconvert</code> with support for a variety of output formats including popular options such as RST, HTML and PDF. Getting an HTML file is perfect as I can embed that as an iframe and share it right here in this article. I wrote a script to automate generating HTML from a folder of 1+ notebooks, if you take out the for loops and boilerplate this is the command I ended up running to turn the <code>.ipynb</code> notebook file into <code>.html</code>.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ jupyter nbconvert --execute --to html notebooks/stock-analysis/notebook.ipynb --stdout &gt; static/notebook-stock-analysis.html
</code></pre></div><h3 id="viewing-the-notebook">Viewing the notebook</h3>
<iframe src="https://jamessevedge.com/notebook-stock-analysis.html" style="height:100vh;width:100%;border:none;overflow:auto"></iframe>
<p>Looks like stocks have been doing OK. Definitely no need for any sort of disclaimer that <em>past performance does not predict future results</em> right?</p>
<h3 id="recap">Recap</h3>
<p>Authoring a Jupyter notebook from scratch and understanding the core data analysis libraries did not end up being a massive learning curve so the experience was great overall. I may use it more in the future for scenarios where Tableau is too heavy handed a solution or perhaps to glean insights using unsupervised machine learning given the right data set. I would like to explore the JupyterLab, JupyterHub and other projects that enable authoring and publishing like Voila.</p>
</description>
</item>
<item>
<title>Creating a Custom Tesla Light Show</title>
<link>https://jamessevedge.com/articles/custom-tesla-light-show/</link>
<pubDate>Tue, 22 Feb 2022 00:00:00 +0000</pubDate>
<author>jjesc1031@gmail.com (James Sevedge)</author>
<guid>https://jamessevedge.com/articles/custom-tesla-light-show/</guid>
<description><p>In late December Tesla pushed out a new major update to my Model Y dubbed <a href="https://www.tesla.com/blog/introducing-software-v11-0">&ldquo;version 11.0&rdquo;</a> which included a revamped UI and a bunch of new features. Probably the coolest feature released was the Light Show which allows creators to use a sequence show tool called xLights to turn on/off or ramp various lights, open/close the windows, open/close the trunk, etc. The Tesla also included a default light show which at the time was holiday themed and got a good reaction from friends/family, pretty sweet to get OTA updates on a car right? Anyways I finally got around to building a custom light show and figured I would take notes as I went and write about it in case others find it interesting and would like to do the same.</p>
<h3 id="documentation">Documentation</h3>
<p>The Tesla engineering team put together a great readme and linting tool which they published under the <code>teslamotors</code> Github organization <a href="https://github.com/teslamotors/light-show">here</a>. It covered most of the details I needed as long as I was willing to jump around a bit as I progressed.</p>
<h3 id="installed-software">Installed software</h3>
<p>Once I read through the documentation I went ahead and installed the xLights application using the Apple app store on my Mac. <strong>Note</strong> that xLights supports all the major OS platforms, see the <a href="https://xlights.org/releases/">download page</a>.</p>
<p>I then downloaded the sequenece model the Tesla engineering team created and provided a link to in this <a href="https://github.com/teslamotors/light-show#getting-started-with-the-tesla-xlights-project-directory">section</a> of the readme and imported it into xLights. This pulls in the 3D car model with the 46 channels (lights, windows, doors, etc) that are configurable and provides a nice visual layout when testing out the light show in xLights.</p>
<p><img src="https://jamessevedge.com/x-lights-overview.png" alt="xLights Overview"></p>
<h3 id="test-out-the-example-sequence">Test out the example sequence</h3>
<p>The Tesla engineering team provided an example <a href="https://github.com/teslamotors/light-show#opening-the-example-sequence">light show</a> (not the same one loaded in the car) which you can use to get a feel for how the sequence effects are laid out and validate your USB can be plugged into the car and correctly play a custom light show. I found this step useful to ensure I didn&rsquo;t take the time to create a light show from scratch and end up running into an issue with USB disk format, etc.</p>
<p>I downloaded the example, put it into <code>Downloads-&gt;Tesla_Light_Show-&gt;Smooth_Jazz-&gt;LightShow</code>, copied that folder over to the USB, plugged it into the car and tested it out. Thankfully it worked the first time! <strong>Note</strong> that I had to wait for a progress bar at the bottom of the modal to complete before pressing play or it would play the last loaded light show.</p>
<h3 id="creating-my-own-light-show">Creating my own light show</h3>
<p>First I needed the audio file for the song I wanted to play. It is surprisingly hard nowadays with streaming services like Spotify, Apple Music, etc. to just get a plain old mp3 anymore&hellip; but Amazon Music to the rescue! In a couple clicks I found the song, purchased the single and downloaded the mp3 file into a new folder under <code>Downloads-&gt;Tesla_Light_Show-&gt;Bang-&gt;LightShow</code>.</p>
<p>Now this is where the fun part actually begins, I needed to create the .fseq file using xLights which contains all the effects matched up to the audio. To do this I opened xLights, created a new sequence and imported the mp3 file. I put on some headphones and proceeded to try and grok the xLights interface and add effects 10-15 seconds at a time. Once I had a good chunk added I would play it back and modify the effects until they lined up with the audio (or close enough!). Once I got through the first chorus, verse and bridge I was able to start chunking the sequences with copy/paste. I would then overlay some additional effects to build the light show as it progressed through the song.</p>
<p>According to the Tesla light show readme there is a max of 651 effects allowed and so they wrote a python validator script to help sequence authors figure out if they were within the parameters. Once I completed the light show and saved the .fseq file to the LightShow folder I ran that tool. Apparently I had a lot more effects I could still add, maybe I will revist this sequence again another time.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash"> $ python validator.py ~/Downloads/Tesla_Shows/Bang/LightShow/lightshow.fseq
Found <span style="color:#ae81ff">3417</span> frames, step time of <span style="color:#ae81ff">50</span> ms <span style="color:#66d9ef">for</span> a total duration of 0:02:50.850000.
Used 45.23% of the available memory
</code></pre></div><p>Here is what the folder structure that was copied over to the USB drive looked like.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">├── LightShow
│ ├── lightshow.fseq
│ ├── lightshow.mp3
│ ├── lightshow.xsq
</code></pre></div><p>And this is a preview of how the light show turned out.</p>
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;">
<iframe src="https://www.youtube.com/embed/M-SK5_alK1A" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;" allowfullscreen title="YouTube Video"></iframe>
</div>
<p>If you have a Tesla and want to try this out you can download it from the very cool <a href="https://teslalightshare.io/light-show/291">teslalightshare.io</a> site.</p>
<h3 id="recap">Recap</h3>
<p>I spent roughly 4 hours getting from no idea how to create a light show for my car to understanding and creating one from scratch so kudos to the Tesla engineering team for good documentation and picking a tool (xLights) that has a decent interface for creating a sequence. I had a blast creating this light show and intend to create more in the future that makes full use of the advanced effects including tailgate open/close and light ramping.</p>
<p>One final note is it did occur to me that it would be nice to have a single USB with multiple custom light shows and the display would provide a picker with the list of shows available. I filed a Github RFE which was promptly closed as a dupe of <a href="https://github.com/teslamotors/light-show/issues/8">#8</a>. When I took a look at that issue it appears the team appreciated the suggestions but did not commit to implementing it. So a typical Tesla response.</p>
</description>
</item>
<item>
<title>Red Light Green Light</title>
<link>https://jamessevedge.com/articles/red-light-green-light/</link>
<pubDate>Mon, 31 Jan 2022 00:00:00 +0000</pubDate>
<author>jjesc1031@gmail.com (James Sevedge)</author>
<guid>https://jamessevedge.com/articles/red-light-green-light/</guid>
<description><p>I was listening to the Tim Ferriss Show <a href="https://tim.blog/2021/09/09/sheila-heen">podcast episode #532</a> with Sheila Heen (Harvard Negotiation Project, co-author of Difficult Conversations: How To Discuss What Matters Most) and a specific story she told during the conversation really resonated with me and I have been thinking alot about it since I heard it. Below is the excerpt of that story in her own words.</p>
<hr>
<p><em>It reminds me, I sometimes tell this story about my eldest son. His name is Ben. He’s 22 now, but when he was about three, we were driving down the street. We stopped at a traffic light, and we were working on both colors and also traffic rules, because at the time we lived on kind of a busy street in Cambridge. So we’re stopped at the light. And I say, “Hey, Ben. What color is the light?” And he says, “It’s green.” I said, “Ben, we’re stopped at the light. What color is the light? Take a good look.” And he goes, “It’s green.” And when it turns, he says, “It’s red. Let’s go.”</em></p>
<p><em>Now, the kid seemed bright in most other ways. So I just thought like, what is going on with him? My first hypothesis is maybe he’s color blind, which then that would be my husband’s fault. At least I thought at the time, it’s my husband’s fault. I’ve since been informed it would have been my fault.</em></p>
<p><em>So I started collecting data. I’m running a little scientific experiment of my own. So I start asking him to identify red and green in other contexts, and he gets it right every time. And yet every time we come to a traffic light, he’s still giving me opposite answers, because I get a little obsessed with this.</em></p>
<p><em>My second hypothesis, by the way, is that he is screwing with me, which I certainly had some data to support. This went on for about three weeks. It wasn’t until maybe three weeks later, and I think my mother-in-law was in town. So I was in the back seat sitting next to Ben, and we stopped at a traffic light. And I suddenly realized that from where he sits in his car seat, he usually can’t see the light in front of us, because the headrest is in the way or it’s above the level of the windshield, windscreen as they say in Europe. So he’s looking out the side window at the cross traffic light.</em></p>
<p><em>Now just think about the conversation from his point of view. He’s looking at the light, it’s green; I’m insisting that it’s red, and he’s like, you know, my mother seems right in most other ways, but she’s just wrong about this. The reason that that experience has stuck with me all these years is that it’s such a great illustration of the fact that where you sit determines what you see.</em></p>
<hr>
<p>Great story right? I considered titling this <strong>where you sit determines what you see</strong> since that seemed to be an appropriate TLDR but I opted for the pithier title, sorry couldn&rsquo;t help it. I am not going to pile on to the story with my own personal anecdotes since Sheila is a great story teller and I prefer to keep this short and sweet, but for anyone who may be frusturated because they are having an issue communicating with a colleague, spouse or friend about something… perhaps the problem has nothing to do with communication. Check your perspective!</p>
</description>
</item>
<item>
<title>SDK Generation from an OpenAPI Specification File</title>
<link>https://jamessevedge.com/articles/sdk-generation-from-an-openapi-specification-file/</link>
<pubDate>Wed, 01 Dec 2021 00:00:00 +0000</pubDate>
<author>jjesc1031@gmail.com (James Sevedge)</author>
<guid>https://jamessevedge.com/articles/sdk-generation-from-an-openapi-specification-file/</guid>
<description><p>Recently I spent a fair amount of time exploring the SDK generation space for a prototype our team was building at my current employer. Along the way I captured a few thoughts which I will break down here about what SDK generation is and some ways to go about it. Let&rsquo;s get some context first by starting with a problem statement.</p>
<p><strong>Problem Statement</strong></p>
<p>Let&rsquo;s say we are responsible for the usability of a SaaS service at XYZ company which has a RESTful API, and since this service is taking off we have begun to get requests for a better developer user experience (UX). Looking to peers in the industry with similar services a common composability pattern emerges, specifically something such as the following.</p>
<script src="https://unpkg.com/mermaid@8.5.1/dist/mermaid.min.js"></script>
<script>
mermaid.initialize({
theme: 'dark',
maxWidth: true
});
</script>
<pre class="mermaid" align="center">
graph TD;
api_service["Native API Service (Imperative)"];
api_service_declarative["Native API Service (Declarative)"];
sdk_python["SDK Binding (Python)"];
sdk_go["SDK Binding (Go)"];
sdk_js["SDK Binding (Javascript)"];
cli["CLI (written in Go)"];
terraform["Terraform (written in Go)"];
ansible["Ansible (written in Python)"];
api_service_declarative --> api_service;
sdk_python --> api_service;
sdk_python --> api_service_declarative;
sdk_go--> api_service;
sdk_go --> api_service_declarative;
sdk_js--> api_service;
sdk_js --> api_service_declarative;
cli --> sdk_go;
terraform --> sdk_go;
ansible --> sdk_python;
</pre>
<p>This pattern is used by most of the major SaaS providers, to varying degrees of maturity. It allows the developer UX to naturally evolve out and away from the core competencies (native API service + a declarative API service for companies that understand the customer value proposition of a declarative model and choose to provide that experience natively) to whatever integration point makes the most sense for the consumer of the service.</p>
<p>The technical issue with this pattern of course is how to propagate out changes that occur in the core competencies to all the upstream bindings and external integration points automagically. The assumption is in most cases it would not be feasible both from a manpower and delivery management perspective to try and do this without tooling and automation. So let&rsquo;s take some time to think about SDK generation since that is the set of bindings immediately upstream of the core APIs.</p>
<p><em>Future Post</em>: Once you have the SDK bindings problem solved you can use those in Terraform/Ansible and tackle the problem of if/how to automate updates to those providers/modules. One step at a time though!</p>
<p><strong>Existing SDK Generation Tooling</strong></p>
<p>Most SDK generation tooling research starts by googling &ldquo;SDK Generation Tool&rdquo; and ends with finding <a href="https://github.com/OpenAPITools/openapi-generator">OpenAPI Generator</a>, which is listed on <a href="https://openapi.tools/#sdk">openapi.tools</a>. OpenAPI Generator is probably the most popular open-source OpenAPI based SDK generator available, it was a fork of the original <a href="https://swagger.io/tools/swagger-codegen/">Swagger CodeGen</a>.</p>
<p>If you use one of these common generators they take an <a href="https://www.openapis.org">OpenAPI</a> specification file which describes the service API endpoints in as much detail as you like and uses common properties such as HTTP methods, HTTP uri and so on to generate bindings for 1+ programming languages. If the default output works for you without additional customization for company branding, etc. you can stop here. If you either don&rsquo;t want to learn how to customize those bindings (see <a href="https://openapi-generator.tech/docs/templating/">OpenAPI Generator templating</a>) or you want to understand how SDK generation works in more detail let&rsquo;s dive into the details.</p>
<p><strong>Building an SDK Generator</strong></p>
<p>In this section let&rsquo;s walk through the steps I took to go from an OpenAPI specification document to a working set of python client bindings.</p>
<ol>
<li>Given a simple OpenAPI specification document</li>
</ol>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#f92672">openapi</span>: <span style="color:#ae81ff">3.0.0</span>
<span style="color:#f92672">info</span>:
<span style="color:#f92672">title</span>: <span style="color:#ae81ff">Example API Specification</span>
<span style="color:#f92672">description</span>: <span style="color:#ae81ff">Example Description</span>
<span style="color:#f92672">version</span>: <span style="color:#ae81ff">1.0.0</span>
<span style="color:#f92672">servers</span>:
- <span style="color:#f92672">url</span>: <span style="color:#ae81ff">https://192.0.2.10</span>
<span style="color:#f92672">description</span>: <span style="color:#ae81ff">Mock Server</span>
<span style="color:#f92672">paths</span>:
<span style="color:#f92672">/applications</span>:
<span style="color:#f92672">get</span>:
<span style="color:#f92672">summary</span>: <span style="color:#ae81ff">List all applications</span>
<span style="color:#f92672">description</span>: <span style="color:#ae81ff">List all applications</span>
<span style="color:#f92672">operationId</span>: <span style="color:#ae81ff">getApplications</span>
<span style="color:#f92672">tags</span>:
- <span style="color:#ae81ff">Application</span>
<span style="color:#f92672">responses</span>:
<span style="color:#f92672">&#39;200&#39;</span>:
<span style="color:#f92672">description</span>: <span style="color:#ae81ff">OK</span>
<span style="color:#f92672">content</span>:
<span style="color:#f92672">application/json</span>:
<span style="color:#f92672">schema</span>:
<span style="color:#f92672">type</span>: <span style="color:#ae81ff">array</span>
<span style="color:#f92672">items</span>:
<span style="color:#f92672">type</span>: <span style="color:#ae81ff">object</span>
<span style="color:#f92672">properties</span>:
<span style="color:#f92672">name</span>:
<span style="color:#f92672">type</span>: <span style="color:#ae81ff">string</span>
<span style="color:#f92672">put</span>:
<span style="color:#f92672">summary</span>: <span style="color:#ae81ff">Add a new application</span>
<span style="color:#f92672">description</span>: <span style="color:#ae81ff">Add a new application</span>
<span style="color:#f92672">operationId</span>: <span style="color:#ae81ff">createApplication</span>
<span style="color:#f92672">tags</span>:
- <span style="color:#ae81ff">Application</span>
<span style="color:#f92672">requestBody</span>:
<span style="color:#f92672">description</span>: <span style="color:#ae81ff">Example Description</span>
<span style="color:#f92672">required</span>: <span style="color:#66d9ef">true</span>
<span style="color:#f92672">content</span>:
<span style="color:#f92672">application/json</span>:
<span style="color:#f92672">schema</span>:
<span style="color:#f92672">type</span>: <span style="color:#ae81ff">object</span>
<span style="color:#f92672">properties</span>:
<span style="color:#f92672">name</span>:
<span style="color:#f92672">type</span>: <span style="color:#ae81ff">string</span>
<span style="color:#f92672">responses</span>:
<span style="color:#f92672">&#39;200&#39;</span>:
<span style="color:#f92672">description</span>: <span style="color:#ae81ff">OK</span>
</code></pre></div><ol start="2">
<li>Parse and normalize into a model an SDK generator (and cooresponding templates) can understand</li>
</ol>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#f92672">packageMetadata</span>:
<span style="color:#f92672">name</span>: <span style="color:#ae81ff">Example API Specification</span>
<span style="color:#f92672">version</span>: <span style="color:#ae81ff">1.0.0</span>
<span style="color:#f92672">description</span>: <span style="color:#ae81ff">Example Description</span>
<span style="color:#f92672">globalConfiguration</span>:
<span style="color:#f92672">host</span>: <span style="color:#ae81ff">https://192.0.2.10</span>
<span style="color:#f92672">namespaces</span>:
- <span style="color:#f92672">namespace</span>: <span style="color:#ae81ff">Application</span>
<span style="color:#f92672">operations</span>:
- <span style="color:#f92672">name</span>: <span style="color:#ae81ff">getApplications</span>
<span style="color:#f92672">description</span>: <span style="color:#ae81ff">List all applications</span>
<span style="color:#f92672">transportSettings</span>:
<span style="color:#f92672">method</span>: <span style="color:#e6db74">&#39;GET&#39;</span>
<span style="color:#f92672">url</span>: <span style="color:#ae81ff">/applications</span>
- <span style="color:#f92672">name</span>: <span style="color:#ae81ff">createApplication</span>
<span style="color:#f92672">description</span>: <span style="color:#ae81ff">Add a new application</span>
<span style="color:#f92672">transportSettings</span>:
<span style="color:#f92672">method</span>: <span style="color:#e6db74">&#39;PUT&#39;</span>
<span style="color:#f92672">url</span>: <span style="color:#ae81ff">/applications</span>
<span style="color:#f92672">requestBodyRequired</span>: <span style="color:#66d9ef">true</span>
</code></pre></div><ol start="3">
<li>Render that generator model against a language-specific template set (Mustache, etc.), in this case Python</li>
</ol>
<pre><code class="language-mustache" data-lang="mustache">&quot;&quot;&quot;{{namespace}} Client&quot;&quot;&quot;
from mysdk.base_clients import BaseFeatureClient
class {{namespace}}Client(BaseFeatureClient):
&quot;&quot;&quot;{{namespace}} Client &quot;&quot;&quot;
def __init__(self, client, **kwargs):
&quot;&quot;&quot;Initialization &quot;&quot;&quot;
super({{namespace}}Client, self).__init__(
client,
logger_name=__name__,
uri='/'
)
{{#operations}}
def {{name}}(self, **kwargs):
&quot;&quot;&quot;{{description}}&quot;&quot;&quot;
{{#transportSettings.requestBodyRequired}}
return self._make_request(
method='{{transportSettings.method}}',
uri='{{transportSettings.url}}',
body=kwargs.pop('config', None)
)
{{/transportSettings.requestBodyRequired}}
{{^transportSettings.requestBodyRequired}}
return self._make_request(
method='{{transportSettings.method}}',
uri='{{transportSettings.url}}'
)
{{/transportSettings.requestBodyRequired}}
{{/operations}}
</code></pre><ol start="4">
<li>The render operation outputs the following functional SDK with a service client per namespace (group/tag/etc)</li>
</ol>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-python" data-lang="python"><span style="color:#75715e"># application.py</span>
<span style="color:#e6db74">&#34;&#34;&#34;Application Client&#34;&#34;&#34;</span>
<span style="color:#f92672">from</span> mysdk.base_clients <span style="color:#f92672">import</span> BaseFeatureClient
<span style="color:#66d9ef">class</span> <span style="color:#a6e22e">ApplicationClient</span>(BaseFeatureClient):
<span style="color:#e6db74">&#34;&#34;&#34;Application Client &#34;&#34;&#34;</span>
<span style="color:#66d9ef">def</span> __init__(self, client, <span style="color:#f92672">**</span>kwargs):
<span style="color:#e6db74">&#34;&#34;&#34;Initialization &#34;&#34;&#34;</span>
super(ApplicationClient, self)<span style="color:#f92672">.</span>__init__(
client,
logger_name<span style="color:#f92672">=</span>__name__,
uri<span style="color:#f92672">=</span><span style="color:#e6db74">&#39;/&#39;</span>
)
<span style="color:#66d9ef">def</span> <span style="color:#a6e22e">get_applications</span>(self, <span style="color:#f92672">**</span>kwargs):
<span style="color:#e6db74">&#34;&#34;&#34;List all applications&#34;&#34;&#34;</span>
<span style="color:#66d9ef">return</span> self<span style="color:#f92672">.</span>_make_request(
method<span style="color:#f92672">=</span><span style="color:#e6db74">&#39;GET&#39;</span>,
uri<span style="color:#f92672">=</span><span style="color:#e6db74">&#39;/applications&#39;</span>
)
<span style="color:#66d9ef">def</span> <span style="color:#a6e22e">create_application</span>(self, <span style="color:#f92672">**</span>kwargs):
<span style="color:#e6db74">&#34;&#34;&#34;Add a new application&#34;&#34;&#34;</span>
<span style="color:#66d9ef">return</span> self<span style="color:#f92672">.</span>_make_request(
method<span style="color:#f92672">=</span><span style="color:#e6db74">&#39;PUT&#39;</span>,
uri<span style="color:#f92672">=</span><span style="color:#e6db74">&#39;/applications&#39;</span>,
body<span style="color:#f92672">=</span>kwargs<span style="color:#f92672">.</span>pop(<span style="color:#e6db74">&#39;config&#39;</span>, <span style="color:#66d9ef">None</span>)
)
</code></pre></div><ol start="5">
<li>Ship it!</li>
</ol>
<p>Seems pretty easy right? Let&rsquo;s look at some of the mappings:</p>
<ul>
<li><code>path.method.tags[0]</code> -&gt; Class name (operation/method grouping)</li>
<li><code>path</code> -&gt; Method transport URI</li>
<li><code>path.method</code> -&gt; Method transport verb</li>
<li><code>path.method.operationId</code> -&gt; Method signature (name)</li>
<li><code>path.method.description</code> -&gt; Method description</li>
</ul>
<p>Of course this is the bare minimum necessary to generate a binding that could be useful, some other considerations that immediately present themselves are:</p>
<ul>
<li>What is necessary to define request/response interfaces for strongly typed languages?</li>
<li>Do management clients (auth, low-level transport) need to be auto-generated? If so what additional properties in the authentication endpoints (Basic, OAuth) OpenAPI specification files will need to be specified?</li>
<li>How should APIs with support for async operations be handled?</li>
<li>How would additional information such as namespace level usage examples be codified?</li>
<li>How much logic should be considered MVP in the core package, such as authentication token refresh, transport level retries and so on?</li>
</ul>
<p>Most of this initial set of problems are solvable simply by determining the minimal set of properties necessary for each endpoint to generate the appropriate generation model, enforcing that set of properties with a linter and then creating the appropriate templates/tooling to make use of that model.</p>
<p><strong>Industry SDK Generation Examples</strong></p>
<p>Here are some examples of popular industry API providers and the approach they took to SDK generation.</p>
<ul>
<li>Microsoft Azure:
<ul>
<li>Specification Location: <a href="https://github.com/Azure/azure-rest-api-specs">https://github.com/Azure/azure-rest-api-specs</a></li>
<li>SDK Generator: <a href="https://github.com/Azure/autorest">https://github.com/Azure/autorest</a>, <a href="https://github.com/Azure/autorest/blob/master/docs/trampoline.md">https://github.com/Azure/autorest/blob/master/docs/trampoline.md</a>, <a href="https://github.com/Azure/azure-rest-api-specs/tree/master/specification/compute/resource-manager">https://github.com/Azure/azure-rest-api-specs/tree/master/specification/compute/resource-manager</a></li>
</ul>
</li>
<li>AWS:
<ul>
<li>Specification Location: <a href="https://github.com/boto/boto3/tree/master/boto3/data">https://github.com/boto/boto3/tree/master/boto3/data</a>, <a href="https://github.com/aws/aws-sdk-js/blob/master/apis">https://github.com/aws/aws-sdk-js/blob/master/apis</a>,</li>
<li>SDK Generator: It appears they manually define the &ldquo;apis&rdquo; and SDK methods using JSON and use that during generation</li>
</ul>
</li>
<li>GCP:
<ul>
<li>Specification Location: <a href="https://github.com/googleapis/googleapis">https://github.com/googleapis/googleapis</a></li>
<li>SDK Generator: <a href="https://google.aip.dev/client-libraries/4210">https://google.aip.dev/client-libraries/4210</a>, <a href="https://github.com/googleapis/gapic-generator">https://github.com/googleapis/gapic-generator</a>, <a href="https://github.com/googleapis/gapic-generator-python">https://github.com/googleapis/gapic-generator-python</a></li>
</ul>
</li>
<li>Kubernetes
<ul>
<li>Specification Location: <a href="https://github.com/kubernetes/kubernetes/tree/master/api/openapi-spec">https://github.com/kubernetes/kubernetes/tree/master/api/openapi-spec</a>, <a href="https://github.com/kubernetes-client">https://github.com/kubernetes-client</a></li>
<li>SDK Generator: It looks like they might be using openapi-generator with their own customization. Here are the scripts they are using to invoke the generator <a href="https://github.com/kubernetes-client/gen">https://github.com/kubernetes-client/gen</a></li>
</ul>
</li>
</ul>
<p><strong>Final Thoughts</strong></p>
<p>As previously stated the deeper you go down the rabbit hole the more you have to consider, but getting a working prototype of a custom SDK generator is not that difficult. Most of the industry examples provided above ended up going down that route to provide the best developer UX possible for their customers since they have the scale and resourcing necessary.</p>
</description>
</item>
<item>
<title>An Excercise In Refactoring</title>
<link>https://jamessevedge.com/articles/an-excercise-in-refactoring/</link>
<pubDate>Tue, 09 Nov 2021 00:00:00 +0000</pubDate>
<author>jjesc1031@gmail.com (James Sevedge)</author>
<guid>https://jamessevedge.com/articles/an-excercise-in-refactoring/</guid>
<description><p>In my humble experience any discussion about refactoring should start with <a href="https://martinfowler.com/books/refactoring.html">Refactoring</a> (Martin Fowler/Kent Beck) and <a href="https://www.oreilly.com/library/view/clean-code-a/9780136083238/">Clean Code</a> (Robert Martin). Having at least a rudimentary understanding of the design patterns presented in <a href="https://www.amazon.com/gp/product/0201633612/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=0201633612&amp;linkCode=as2&amp;tag=martinfowlerc-20">Gang of Four</a> is helpful too.</p>
<p>It can be daunting to know where exactly to begin with those resources or understand why it matters so let&rsquo;s dive into the following concrete example to provide some clarity on what refactoring a module <em>may</em> look like.</p>
<h2 id="refactoring-excercise">Refactoring Excercise</h2>
<p>Let&rsquo;s start with some code that could use a little (or a lot of!) refactoring…</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#66d9ef">const</span> <span style="color:#a6e22e">axios</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">require</span>(<span style="color:#e6db74">&#39;axios&#39;</span>);
<span style="color:#66d9ef">function</span> <span style="color:#a6e22e">uploadFiles</span>() {
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">loginResponse</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1/login&#39;</span>,
<span style="color:#e6db74">&#39;GET&#39;</span>,
<span style="color:#a6e22e">auth</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">username</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;a_username&#39;</span>,
<span style="color:#a6e22e">password</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;a_password&#39;</span>
},
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#ae81ff">200</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">authToken</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">data</span>.<span style="color:#a6e22e">token</span>;
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">encodedCertificateProduction</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#e6db74">&#39;certificate for production environment&#39;</span>);
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">encodedKeyProduction</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#e6db74">&#39;key for production environment&#39;</span>);
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">productionFile</span> <span style="color:#f92672">=</span> {
<span style="color:#a6e22e">description</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;Production certificate/key&#39;</span>,
<span style="color:#a6e22e">name</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;production_file&#39;</span>,
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">encodedCertificateProduction</span>,
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">encodedKeyProduction</span>,
<span style="color:#a6e22e">metadata</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">env</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;production&#39;</span>
}
};
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">productionFileResponse</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1/storage/my-bucket&#39;</span>,
<span style="color:#e6db74">&#39;POST&#39;</span>,
<span style="color:#a6e22e">headers</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">Authorization</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`Bearer </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">authToken</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>
},
<span style="color:#a6e22e">data</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">productionFile</span>,
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">productionFileResponse</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#ae81ff">200</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">productionFileResponse</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">productionFileResponse</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">encodedCertificateStaging</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#e6db74">&#39;certificate for staging environment&#39;</span>);
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">encodedKeyStaging</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#e6db74">&#39;key for staging environment&#39;</span>);
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">stagingFile</span> <span style="color:#f92672">=</span> {
<span style="color:#a6e22e">description</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;Staging certificate/key&#39;</span>,
<span style="color:#a6e22e">name</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;staging_file&#39;</span>,
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">encodedCertificateStaging</span>,
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">encodedKeyStaging</span>,
<span style="color:#a6e22e">metadata</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">env</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;staging&#39;</span>
}
};
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">stagingFileResponse</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1/storage/my-bucket&#39;</span>,
<span style="color:#e6db74">&#39;POST&#39;</span>,
<span style="color:#a6e22e">headers</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">Authorization</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`Bearer </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">authToken</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>
},
<span style="color:#a6e22e">data</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">stagingFile</span>,
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">stagingFileResponse</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#ae81ff">200</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">stagingFileResponse</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">stagingFileResponse</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
}
<span style="color:#66d9ef">await</span> <span style="color:#a6e22e">uploadFiles</span>();
</code></pre></div><p>How hard was it tell the purpose of the code and did the code &ldquo;smell&rdquo; at all to you? If so, how?</p>
<h3 id="readability">Readability</h3>
<p>The first thing you notice may be how hard it is to understand what the code is doing, so let&rsquo;s add some comments (for now, more on that later) to help the developers reading this code.</p>
<p>Actions to take: 1) Add some comments to <code>uploadFiles</code></p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#66d9ef">const</span> <span style="color:#a6e22e">axios</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">require</span>(<span style="color:#e6db74">&#39;axios&#39;</span>);
<span style="color:#66d9ef">function</span> <span style="color:#a6e22e">uploadFiles</span>() {
<span style="color:#75715e">// login to &#34;the cloud&#34; and get a token
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">loginResponse</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1/login&#39;</span>,
<span style="color:#e6db74">&#39;GET&#39;</span>,
<span style="color:#a6e22e">auth</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">username</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;a_username&#39;</span>,
<span style="color:#a6e22e">password</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;a_password&#39;</span>
},
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#ae81ff">200</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">authToken</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">data</span>.<span style="color:#a6e22e">token</span>;
<span style="color:#75715e">// encode production certificate/key
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">encodedCertificateProduction</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#e6db74">&#39;certificate for production environment&#39;</span>);
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">encodedKeyProduction</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#e6db74">&#39;key for production environment&#39;</span>);
<span style="color:#75715e">// create production certificate/key file contents (axios will JSON stringify the JavaScript object for us)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">productionFile</span> <span style="color:#f92672">=</span> {
<span style="color:#a6e22e">description</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;Production certificate/key&#39;</span>,
<span style="color:#a6e22e">name</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;production_file&#39;</span>,
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">encodedCertificateProduction</span>,
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">encodedKeyProduction</span>,
<span style="color:#a6e22e">metadata</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">env</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;production&#39;</span>
}
};
<span style="color:#75715e">// upload production certificate/key file contents (uses name in HTTP body to create a file under &#34;my-bucket&#34;)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">productionFileResponse</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1/storage/my-bucket&#39;</span>,
<span style="color:#e6db74">&#39;POST&#39;</span>,
<span style="color:#a6e22e">headers</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">Authorization</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`Bearer </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">authToken</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>
},
<span style="color:#a6e22e">data</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">productionFile</span>,
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">productionFileResponse</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#ae81ff">200</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">productionFileResponse</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">productionFileResponse</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#75715e">// create staging certificate/key file contents (axios will JSON stringify the JavaScript object for us)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">encodedCertificateStaging</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#e6db74">&#39;certificate for staging environment&#39;</span>);
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">encodedKeyStaging</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#e6db74">&#39;key for staging environment&#39;</span>);
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">stagingFile</span> <span style="color:#f92672">=</span> {
<span style="color:#a6e22e">description</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;Staging certificate/key&#39;</span>,
<span style="color:#a6e22e">name</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;staging_file&#39;</span>,
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">encodedCertificateStaging</span>,
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">encodedKeyStaging</span>,
<span style="color:#a6e22e">metadata</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">env</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;staging&#39;</span>
}
};
<span style="color:#75715e">// upload staging certificate/key file contents (uses name in HTTP body to create a file under &#34;my-bucket&#34;)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">stagingFileResponse</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1/storage/my-bucket&#39;</span>,
<span style="color:#e6db74">&#39;POST&#39;</span>,
<span style="color:#a6e22e">headers</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">Authorization</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`Bearer </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">authToken</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>
},
<span style="color:#a6e22e">data</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">stagingFile</span>,
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">stagingFileResponse</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#ae81ff">200</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">stagingFileResponse</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">stagingFileResponse</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
}
<span style="color:#66d9ef">await</span> <span style="color:#a6e22e">uploadFiles</span>();
</code></pre></div><h3 id="duplication">Duplication</h3>
<p>Ok, that is a little better… at least it is easier to understand what is happening. But the function is still large and there are blocks of code that are almost exactly the same so let&rsquo;s extract those into some helper methods.</p>
<p>Actions to take: 1) Create <code>_generateFile</code> function and use it in <code>uploadFiles</code> for both environments 2) Create <code>_uploadFile</code> function and use it in <code>uploadFiles</code> for both environments</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#66d9ef">const</span> <span style="color:#a6e22e">axios</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">require</span>(<span style="color:#e6db74">&#39;axios&#39;</span>);
<span style="color:#66d9ef">function</span> <span style="color:#a6e22e">uploadFiles</span>() {
<span style="color:#75715e">// login to &#34;the cloud&#34; and get a token
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">loginResponse</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1/login&#39;</span>,
<span style="color:#e6db74">&#39;GET&#39;</span>,
<span style="color:#a6e22e">auth</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">username</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;a_username&#39;</span>,
<span style="color:#a6e22e">password</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;a_password&#39;</span>
},
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#ae81ff">200</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">authToken</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">data</span>.<span style="color:#a6e22e">token</span>;
<span style="color:#75715e">// create production certificate/key file contents (axios will JSON stringify the JavaScript object for us)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">productionFile</span> <span style="color:#f92672">=</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateFile</span>(<span style="color:#e6db74">&#39;production&#39;</span>, <span style="color:#e6db74">&#39;certificate for production environment&#39;</span>, <span style="color:#e6db74">&#39;key for production environment&#39;</span>);
<span style="color:#75715e">// upload production certificate/key file contents (uses name in HTTP body to create a file under &#34;my-bucket&#34;)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">uploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#a6e22e">productionFile</span>);
<span style="color:#75715e">// create staging certificate/key file contents (axios will JSON stringify the JavaScript object for us)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">stagingFile</span> <span style="color:#f92672">=</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateFile</span>(<span style="color:#e6db74">&#39;staging&#39;</span>, <span style="color:#e6db74">&#39;certificate for staging environment&#39;</span>, <span style="color:#e6db74">&#39;key for staging environment&#39;</span>);
<span style="color:#75715e">// upload staging certificate/key file contents (uses name in HTTP body to create a file under &#34;my-bucket&#34;)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">uploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#a6e22e">stagingFile</span>);
}
<span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateFile</span>(<span style="color:#a6e22e">environment</span>, <span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">key</span>) {
<span style="color:#66d9ef">return</span> <span style="color:#f92672">=</span> {
<span style="color:#a6e22e">description</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">environment</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> certificate/key`</span>,
<span style="color:#a6e22e">name</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">environment</span><span style="color:#e6db74">}</span><span style="color:#e6db74">_file`</span>,
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#a6e22e">certificate</span>),
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#a6e22e">key</span>),
<span style="color:#a6e22e">metadata</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">env</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">environment</span>
}
};
}
<span style="color:#66d9ef">async</span> <span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">uploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#a6e22e">file</span>) {
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">response</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1/storage/my-bucket&#39;</span>,
<span style="color:#e6db74">&#39;POST&#39;</span>,
<span style="color:#a6e22e">headers</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">Authorization</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`Bearer </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">authToken</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>
},
<span style="color:#a6e22e">data</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">file</span>,
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#ae81ff">200</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span>;
}
<span style="color:#66d9ef">await</span> <span style="color:#a6e22e">uploadFiles</span>();
</code></pre></div><h3 id="extract-functionality">Extract functionality</h3>
<p>Now <code>uploadFiles</code> is starting to look pretty good, but just because code is not duplicated does not mean it shouldn&rsquo;t be extracted. If there is a block of code that makes sense to extract out, do it.</p>
<p>Actions To Take: 1) Extract out <code>login</code> functionality and use it in <code>uploadFiles</code> 2) Extract out <code>_generateAndUploadFile</code> functionality and use it in <code>uploadFiles</code></p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#66d9ef">const</span> <span style="color:#a6e22e">axios</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">require</span>(<span style="color:#e6db74">&#39;axios&#39;</span>);
<span style="color:#66d9ef">function</span> <span style="color:#a6e22e">uploadFiles</span>() {
<span style="color:#75715e">// login to &#34;the cloud&#34; and get a token
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">authToken</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">login</span>(<span style="color:#e6db74">&#39;a_username&#39;</span>, <span style="color:#e6db74">&#39;a_password&#39;</span>);
<span style="color:#75715e">// create production certificate/key file contents (axios will JSON stringify the JavaScript object for us)
</span><span style="color:#75715e"></span> <span style="color:#75715e">// upload production certificate/key file contents (uses name in HTTP body to create a file under &#34;my-bucket&#34;)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateAndUploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#e6db74">&#39;production&#39;</span>, <span style="color:#e6db74">&#39;certificate for production environment&#39;</span>, <span style="color:#e6db74">&#39;key for production environment&#39;</span>)
<span style="color:#75715e">// create staging certificate/key file contents (axios will JSON stringify the JavaScript object for us)
</span><span style="color:#75715e"></span> <span style="color:#75715e">// upload staging certificate/key file contents (uses name in HTTP body to create a file under &#34;my-bucket&#34;)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateAndUploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#e6db74">&#39;staging&#39;</span>, <span style="color:#e6db74">&#39;certificate for staging environment&#39;</span>, <span style="color:#e6db74">&#39;key for staging environment&#39;</span>)
}
<span style="color:#66d9ef">async</span> <span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateAndUploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#a6e22e">environment</span>, <span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">key</span>) {
<span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">uploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateFile</span>(<span style="color:#a6e22e">environment</span>, <span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">key</span>));
}
<span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateFile</span>(<span style="color:#a6e22e">environment</span>, <span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">key</span>) {
<span style="color:#66d9ef">return</span> <span style="color:#f92672">=</span> {
<span style="color:#a6e22e">description</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">environment</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> certificate/key`</span>,
<span style="color:#a6e22e">name</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">environment</span><span style="color:#e6db74">}</span><span style="color:#e6db74">_file`</span>,
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#a6e22e">certificate</span>),
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#a6e22e">key</span>),
<span style="color:#a6e22e">metadata</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">env</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">environment</span>
}
};
}
<span style="color:#66d9ef">async</span> <span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">login</span>(<span style="color:#a6e22e">username</span>, <span style="color:#a6e22e">password</span>) {
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">loginResponse</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1/login&#39;</span>,
<span style="color:#e6db74">&#39;GET&#39;</span>,
<span style="color:#a6e22e">auth</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">username</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">username</span>,
<span style="color:#a6e22e">password</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">password</span>
},
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#ae81ff">200</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span>.<span style="color:#a6e22e">token</span>;
}
<span style="color:#66d9ef">async</span> <span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">uploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#a6e22e">file</span>) {
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">response</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1/storage/my-bucket&#39;</span>,
<span style="color:#e6db74">&#39;POST&#39;</span>,
<span style="color:#a6e22e">headers</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">Authorization</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`Bearer </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">authToken</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>
},
<span style="color:#a6e22e">data</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">file</span>,
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#ae81ff">200</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span>;
}
<span style="color:#66d9ef">await</span> <span style="color:#a6e22e">uploadFiles</span>();
</code></pre></div><h3 id="magic-strings">Magic strings</h3>
<p>Avoid <a href="https://en.wikipedia.org/wiki/Magic_string">magic strings</a>, there are a number of them still. In this refactoring example we will not delve into how some of these items (username, password, certificate/key content, etc.) should probably be provided as input or loaded from the environment, but let&rsquo;s at least not sprinkle them throughout the code.</p>
<p>Actions to take: 1) Move magic strings to the top of the module and update references</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#66d9ef">const</span> <span style="color:#a6e22e">axios</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">require</span>(<span style="color:#e6db74">&#39;axios&#39;</span>);
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">CLOUD_SERVICE_API</span> <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1&#39;</span>;
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">USERNAME</span> <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;a_username&#39;</span>;
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">PASSWORD</span> <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;a_password&#39;</span>;
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">TLS_CONTENT</span> <span style="color:#f92672">=</span> {
<span style="color:#e6db74">&#39;production&#39;</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;certificate for production environment&#39;</span>,
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;key for production environment&#39;</span>
},
<span style="color:#e6db74">&#39;staging&#39;</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;certificate for staging environment&#39;</span>,
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;key for staging environment&#39;</span>
}
}
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">HTTP_RESPONSE_CODES</span> <span style="color:#f92672">=</span> {
<span style="color:#a6e22e">SUCCESS</span><span style="color:#f92672">:</span> <span style="color:#ae81ff">200</span>
}
<span style="color:#66d9ef">function</span> <span style="color:#a6e22e">uploadFiles</span>() {
<span style="color:#75715e">// login to &#34;the cloud&#34; and get a token
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">const</span> <span style="color:#a6e22e">authToken</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">login</span>(<span style="color:#a6e22e">USERNAME</span>, <span style="color:#a6e22e">PASSWORD</span>);
<span style="color:#75715e">// create production certificate/key file contents (axios will JSON stringify the JavaScript object for us)
</span><span style="color:#75715e"></span> <span style="color:#75715e">// upload production certificate/key file contents (uses name in HTTP body to create a file under &#34;my-bucket&#34;)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateAndUploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#e6db74">&#39;production&#39;</span>, <span style="color:#a6e22e">TLS_CONTENT</span>[<span style="color:#e6db74">&#39;production&#39;</span>].<span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">TLS_CONTENT</span>[<span style="color:#e6db74">&#39;production&#39;</span>].<span style="color:#a6e22e">key</span>)
<span style="color:#75715e">// create staging certificate/key file contents (axios will JSON stringify the JavaScript object for us)
</span><span style="color:#75715e"></span> <span style="color:#75715e">// upload staging certificate/key file contents (uses name in HTTP body to create a file under &#34;my-bucket&#34;)
</span><span style="color:#75715e"></span> <span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateAndUploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#e6db74">&#39;staging&#39;</span>, <span style="color:#a6e22e">TLS_CONTENT</span>[<span style="color:#e6db74">&#39;staging&#39;</span>].<span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">TLS_CONTENT</span>[<span style="color:#e6db74">&#39;staging&#39;</span>].<span style="color:#a6e22e">key</span>)
}
<span style="color:#66d9ef">async</span> <span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateAndUploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#a6e22e">environment</span>, <span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">key</span>) {
<span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">uploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateFile</span>(<span style="color:#a6e22e">environment</span>, <span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">key</span>));
}
<span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateFile</span>(<span style="color:#a6e22e">environment</span>, <span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">key</span>) {
<span style="color:#66d9ef">return</span> <span style="color:#f92672">=</span> {
<span style="color:#a6e22e">description</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">environment</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> certificate/key`</span>,
<span style="color:#a6e22e">name</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">environment</span><span style="color:#e6db74">}</span><span style="color:#e6db74">_file`</span>,
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#a6e22e">certificate</span>),
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#a6e22e">key</span>),
<span style="color:#a6e22e">metadata</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">env</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">environment</span>
}
};
}
<span style="color:#66d9ef">async</span> <span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">login</span>(<span style="color:#a6e22e">username</span>, <span style="color:#a6e22e">password</span>) {
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">loginResponse</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">CLOUD_SERVICE_API</span><span style="color:#e6db74">}</span><span style="color:#e6db74">/login`</span>,
<span style="color:#e6db74">&#39;GET&#39;</span>,
<span style="color:#a6e22e">auth</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">username</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">username</span>,
<span style="color:#a6e22e">password</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">password</span>
},
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#a6e22e">HTTP_RESPONSE_CODES</span>.<span style="color:#a6e22e">SUCCESS</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span>.<span style="color:#a6e22e">token</span>;
}
<span style="color:#66d9ef">async</span> <span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">uploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#a6e22e">file</span>) {
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">response</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">CLOUD_SERVICE_API</span><span style="color:#e6db74">}</span><span style="color:#e6db74">/storage/my-bucket`</span>,
<span style="color:#e6db74">&#39;POST&#39;</span>,
<span style="color:#a6e22e">headers</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">Authorization</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`Bearer </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">authToken</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>
},
<span style="color:#a6e22e">data</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">file</span>,
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#a6e22e">HTTP_RESPONSE_CODES</span>.<span style="color:#a6e22e">SUCCESS</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span>;
}
<span style="color:#66d9ef">await</span> <span style="color:#a6e22e">uploadFiles</span>();
</code></pre></div><h3 id="remove-unnecessary-comments">Remove unnecessary comments</h3>
<p>Well now those comments are starting to look a little silly, the methods are named well enough to allow readers to understand what is going on without them. Let&rsquo;s just remove the comments.</p>
<p>Actions to take: 1) Remove unnecessary comments</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#66d9ef">const</span> <span style="color:#a6e22e">axios</span> <span style="color:#f92672">=</span> <span style="color:#a6e22e">require</span>(<span style="color:#e6db74">&#39;axios&#39;</span>);
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">CLOUD_SERVICE_API</span> <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;https://the-cloud/api/v1&#39;</span>;
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">USERNAME</span> <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;a_username&#39;</span>;
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">PASSWORD</span> <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;a_password&#39;</span>;
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">TLS_CONTENT</span> <span style="color:#f92672">=</span> {
<span style="color:#e6db74">&#39;production&#39;</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;certificate for production environment&#39;</span>,
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;key for production environment&#39;</span>
},
<span style="color:#e6db74">&#39;staging&#39;</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;certificate for staging environment&#39;</span>,
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#e6db74">&#39;key for staging environment&#39;</span>
}
}
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">HTTP_RESPONSE_CODES</span> <span style="color:#f92672">=</span> {
<span style="color:#a6e22e">SUCCESS</span><span style="color:#f92672">:</span> <span style="color:#ae81ff">200</span>
}
<span style="color:#66d9ef">function</span> <span style="color:#a6e22e">uploadFiles</span>() {
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">authToken</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">login</span>(<span style="color:#a6e22e">USERNAME</span>, <span style="color:#a6e22e">PASSWORD</span>);
<span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateAndUploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#e6db74">&#39;production&#39;</span>, <span style="color:#a6e22e">TLS_CONTENT</span>[<span style="color:#e6db74">&#39;production&#39;</span>].<span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">TLS_CONTENT</span>[<span style="color:#e6db74">&#39;production&#39;</span>].<span style="color:#a6e22e">key</span>)
<span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateAndUploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#e6db74">&#39;staging&#39;</span>, <span style="color:#a6e22e">TLS_CONTENT</span>[<span style="color:#e6db74">&#39;staging&#39;</span>].<span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">TLS_CONTENT</span>[<span style="color:#e6db74">&#39;staging&#39;</span>].<span style="color:#a6e22e">key</span>)
}
<span style="color:#66d9ef">async</span> <span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateAndUploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#a6e22e">environment</span>, <span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">key</span>) {
<span style="color:#66d9ef">await</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">uploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateFile</span>(<span style="color:#a6e22e">environment</span>, <span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">key</span>));
}
<span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">generateFile</span>(<span style="color:#a6e22e">environment</span>, <span style="color:#a6e22e">certificate</span>, <span style="color:#a6e22e">key</span>) {
<span style="color:#66d9ef">return</span> <span style="color:#f92672">=</span> {
<span style="color:#a6e22e">description</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">environment</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> certificate/key`</span>,
<span style="color:#a6e22e">name</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">environment</span><span style="color:#e6db74">}</span><span style="color:#e6db74">_file`</span>,
<span style="color:#a6e22e">certificate</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#a6e22e">certificate</span>),
<span style="color:#a6e22e">key</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">Buffer</span>.<span style="color:#a6e22e">from</span>(<span style="color:#a6e22e">data</span>).<span style="color:#a6e22e">toString</span>(<span style="color:#a6e22e">key</span>),
<span style="color:#a6e22e">metadata</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">env</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">environment</span>
}
};
}
<span style="color:#66d9ef">async</span> <span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">login</span>(<span style="color:#a6e22e">username</span>, <span style="color:#a6e22e">password</span>) {
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">loginResponse</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">CLOUD_SERVICE_API</span><span style="color:#e6db74">}</span><span style="color:#e6db74">/login`</span>,
<span style="color:#e6db74">&#39;GET&#39;</span>,
<span style="color:#a6e22e">auth</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">username</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">username</span>,
<span style="color:#a6e22e">password</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">password</span>
},
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#a6e22e">HTTP_RESPONSE_CODES</span>.<span style="color:#a6e22e">SUCCESS</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">loginResponse</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span>.<span style="color:#a6e22e">token</span>;
}
<span style="color:#66d9ef">async</span> <span style="color:#66d9ef">function</span> <span style="color:#ae81ff">_</span><span style="color:#a6e22e">uploadFile</span>(<span style="color:#a6e22e">authToken</span>, <span style="color:#a6e22e">file</span>) {
<span style="color:#66d9ef">const</span> <span style="color:#a6e22e">response</span> <span style="color:#f92672">=</span> <span style="color:#66d9ef">await</span> <span style="color:#a6e22e">axios</span>({
<span style="color:#a6e22e">url</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`</span><span style="color:#e6db74">${</span><span style="color:#a6e22e">CLOUD_SERVICE_API</span><span style="color:#e6db74">}</span><span style="color:#e6db74">/storage/my-bucket`</span>,
<span style="color:#e6db74">&#39;POST&#39;</span>,
<span style="color:#a6e22e">headers</span><span style="color:#f92672">:</span> {
<span style="color:#a6e22e">Authorization</span><span style="color:#f92672">:</span> <span style="color:#e6db74">`Bearer </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">authToken</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>
},
<span style="color:#a6e22e">data</span><span style="color:#f92672">:</span> <span style="color:#a6e22e">file</span>,
<span style="color:#a6e22e">validateStatus</span><span style="color:#f92672">:</span> <span style="color:#66d9ef">false</span>
});
<span style="color:#66d9ef">if</span> (<span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">status</span> <span style="color:#f92672">!==</span> <span style="color:#a6e22e">HTTP_RESPONSE_CODES</span>.<span style="color:#a6e22e">SUCCESS</span>) {
<span style="color:#66d9ef">throw</span> <span style="color:#66d9ef">new</span> Error(<span style="color:#e6db74">`HTTP request failed: </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">status</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span><span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span><span style="color:#e6db74">}</span><span style="color:#e6db74">`</span>)
}
<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">response</span>.<span style="color:#a6e22e">data</span>;
}
<span style="color:#66d9ef">await</span> <span style="color:#a6e22e">uploadFiles</span>();
</code></pre></div><h2 id="futher-optimizations">Futher optimizations</h2>
<p>There is more that can be done to this code of course, such as 1) moving the low level cloud service API interaction to a <code>CloudApiClient</code> class to encapsulate that functionality (and allow support via polymorphism for another cloud) 2) creating a <code>FilesClient</code> class to encapsulate cloud service file operation behavior 3) wrapping the top level certificate and key upload business logic into a class which handles loading the files from the environment and calling the other classes 4) and so on…</p>
<script src="https://unpkg.com/mermaid@8.5.1/dist/mermaid.min.js"></script>
<script>
mermaid.initialize({
theme: 'dark',
maxWidth: true
});
</script>
<pre class="mermaid" align="center">
classDiagram
class CloudApiClient{
+login()
+makeRequest()
}
class FilesClient{
+init()
+uploadFile()
}
class CertificateAndKeyUploader{
+uploadFiles()
}
CertificateAndKeyUploader ..> FilesClient
FilesClient ..> CloudApiClient
</pre>
<p>but this at least covers the broad strokes of what refactoring a module might look like.</p>
<h2 id="additional-resources">Additional Resources</h2>
<p>I find that once you have a foundational understanding of code refactoring, you simply need to excercise that knowledge over time to get better… but to get a refresher on specific principles you can go to the wonderful site <a href="https://refactoring.com">refactoring.com</a> and check out the <a href="https://refactoring.com/catalog/">catalog</a>. I also highly recommend, when the time is right, to find and spend the time to watch the &ldquo;Uncle Bob&rdquo; Clean Code <a href="https://www.youtube.com/watch?v=7EmboKQH8lM">series</a> where he goes into detail about the topics covered in his book.</p>
</description>
</item>
<item>
<title>Treat your Career Like a Startup</title>
<link>https://jamessevedge.com/articles/career-as-a-startup/</link>
<pubDate>Tue, 19 Oct 2021 00:00:00 +0000</pubDate>
<author>jjesc1031@gmail.com (James Sevedge)</author>
<guid>https://jamessevedge.com/articles/career-as-a-startup/</guid>
<description><p>Lately I have been having discussions with both close friends and distant acquaintances on &ldquo;leveling up&rdquo; in your career, along the way I have formulated a few thoughts on this topic.</p>
<p><strong>Do the job you want, not the job you have</strong></p>
<p>It is a well understood concept that you should start doing elements of the job you would like to be promoted into before you get the promotion, so figure out what those elements are and do them when opportunity allows.</p>
<p><strong>Raise your hand</strong></p>
<p>Is there a project that your management team has talked about but doesn&rsquo;t know who to assign it to? Raise your hand. Is there a learning opportunity relevant to your organization&rsquo;s needs that you can attend and bring back useful information? Raise your hand. Is there a project that requires a little travel to a different site? Raise your hand.</p>
<p><strong>Level up your organizations knowledge</strong></p>
<p>While doing the work expected of your current role, figure out what knowledge you could learn and bring back into the team to help the organization level up. It could be a new tool/library, testing methodology, way of working or anything else as long as it has some direct relevance.</p>
<p><strong>Level up your knowledge</strong></p>
<p>This is the most important one, do whatever it takes to continuously learn and surround yourself with peers who have the knowledge you would like to have. If you feel it has been awhile since you have learned something new, find something and spend a little (or a lot!) of time getting proficient at it. Always learning, always growing should be the mantra.</p>
<p><em>Qualifying statement</em></p>
<p>Let me qualify these statements by making clear that deriving satisfaction from work output and career trajectory is great, but there is more to life than a career so put this advice into context. Aspire to have balance in all things.</p>
</description>
</item>
<item>
<title>API Development (Beyond the Specification Document)</title>
<link>https://jamessevedge.com/articles/api-development-to-customer-ux/</link>
<pubDate>Wed, 20 May 2020 00:00:00 +0000</pubDate>
<author>jjesc1031@gmail.com (James Sevedge)</author>
<guid>https://jamessevedge.com/articles/api-development-to-customer-ux/</guid>
<description><p>Creating an API that is well thought out, scales and adopts industry standards can be quite difficult. Creating the API while keeping the end consumer User Experience (UX) at the forefront adds even more cognitive load, however the payoff is worth it to stay disciplined and consider it from the start. Let&rsquo;s talk about what that flow might look like at a high level including some example tooling and processes.</p>
<p>First let&rsquo;s assume you have a good API design in place which has been peer-reviewed and articulated in a human and machine readable format, for example if creating a REST API you may articulate this using <a href="https://openapis.org/">OpenAPI</a>, <a href="https://apiblueprint.org/">API Blueprint</a> or some other similar mechanism. In this example let&rsquo;s assume you have been using OpenAPI.</p>
<p>If the OpenAPI specification document describes your API, let&rsquo;s strive to keep it that way. That is the <em>source of truth</em>. Given that assumption all consumer UX should be derived from that document.</p>
<p>Ok great, but my API is already implemented so what more is needed here? And who is this mysterious &ldquo;consumer&rdquo; that demands more than simply a running instance of the API?</p>
<p>The most common consumer of API&rsquo;s is other developers trying to make use of your technology platform to accomplish a specific set of tasks which fit into a larger goal. To enable adoption of the API you must consider a variety of artifacts, including API documentation, hand-crafted examples to describe certain 80/20 usage patterns, multi-language binding SDK bindings and more.</p>
<p>Before picking any specific tool you need to do research to ensure any tooling selected makes good design sense (more on that later). One such resource is <a href="https://openapi.tools">https://openapi.tools</a>. That being said let&rsquo;s take a look at the below UML flow diagram and then map that to a specific tool.</p>
<script src="https://unpkg.com/mermaid@8.5.1/dist/mermaid.min.js"></script>
<script>
mermaid.initialize({
theme: 'dark',
maxWidth: true
});
</script>
<pre class="mermaid" align="center">
graph TD;
api_spec_a["API Specification (Offering A: /service-1)"] --> sdk_tooling;
api_spec_b["API Specification (Offering B: /service-1)"] --> sdk_tooling;
sdk_tooling["SDK Generation Tooling (Auth, low-level comms, Generator Templates)"] --> sdk_api_docs["SDK API Docs"];
sdk_tooling --> sdk_package["SDK Package"];
sdk_api_docs --> customer_docs["Consumer Docs"];
sdk_package --> package_index["Package Repository"];
</pre>
<p>As you can see everything in this diagram flows from the API specification document which should be considered the <em>source of truth</em>, that being said there is still plenty of work and tooling required to get to a polished consumer UX beyond just writing an API specification document.</p>
<ul>
<li>API Documentation Generator: <a href="https://github.com/Redocly/redoc">ReDoc</a></li>
<li>Curated Consumer Documentation: <a href="https://github.com/gohugoio/hugo">Hugo</a></li>
<li>SDK Generator: <a href="https://github.com/OpenAPITools/openapi-generator">OpenAPI Generator</a></li>
</ul>
<p>Thankfully there is open-source tooling which is available for use, you just have to pick a tool for the problem and hope that when you hit a blocker you will have used it and understand it enough to be able to contribute changes back.</p>
</description>
</item>
<item>
<title>Hello World</title>
<link>https://jamessevedge.com/articles/hello-world/</link>
<pubDate>Wed, 20 May 2020 00:00:00 +0000</pubDate>
<author>jjesc1031@gmail.com (James Sevedge)</author>
<guid>https://jamessevedge.com/articles/hello-world/</guid>
<description><p>Congratulations for making it to an obscure post on an obscure site, check out some of the other posts for better content!</p>
</description>
</item>
</channel>
</rss>