Parse_Blog.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xml.dom.minidom import parse\n",
    "import xml.dom.minidom\n",
    "from textwrap import wrap\n",
    "import string\n",
    "\n",
    "DOMTree = xml.dom.minidom.parse(\"../JonoBot/.venv/BlogExport.xml\")\n",
    "collection = DOMTree.documentElement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The Projects That Have Brought Me HereI have known I wanted to work in technology since I started writing software on my calculator that solved my math homework without me having to Calculus.\\xa0 This has led to a wide variety of projects and jobs throughout a 20 year career.\\xa0 Many of the old projects aren’t interesting at all anymore (the website for my Dad’s car dealership, the social network planning site I built with a friend, my specialty in rolling out good ITSM practices when switching to VMWare ESX), but I thought a few of them might provide some context to my current role as an advisor to companies looking to transform their technology.<img width=\"1884\" height=\"290\" src=\"http://jonathancavell.com/wp-content/uploads/2023/08/delta_c_r.png\" alt=\"\" loading=\"lazy\" srcset=\"https://jonathancavell.com/wp-content/uploads/2023/08/delta_c_r.png 1884w, https://jonathancavell.com/wp-content/uploads/2023/08/delta_c_r-300x46.png 300w, https://jonathancavell.com/wp-content/uploads/2023/08/delta_c_r-1024x158.png 1024w, https://jonathancavell.com/wp-content/uploads/2023/08/delta_c_r-768x118.png 768w, https://jonathancavell.com/wp-content/uploads/2023/08/delta_c_r-1536x236.png 1536w\" sizes=\"(max-width: 1884px) 100vw, 1884px\" /><strong>Delta Airlines - Cloud Transformation</strong>This was a project I started as an IBM employee and continued as a Kyndryl.\\xa0 I was one of the leaders who helped sell the <a href=\"https://www.wsj.com/articles/delta-taps-ibm-to-move-more-applications-to-the-cloud-11613653206\">massive transformation program</a> and then I stayed on to lead the first year plus of implementation of Delta’s Secure Cloud Foundations on AWS (one of 5 pillars of the whole program).\\xa0 I’m particularly proud of several of the outcomes of this project:<ol><li>My team and Delta recognized early on the need to provide different, fit-for-purpose environments for different types of applications depending on whether they were being rewritten to take advantage of cloud native services or simply migrating on to EC2s.</li><li>Delta recognized the need to build in security to the very foundations of the cloud. This was accomplished by incorporating team members and executive sponsors from both security and the infrastructure team.</li><li>The project was completed using a modified SAFe Agile approach that included team members from both Delta and IBM. This “co-creation” model allowed us to enable the Delta workforce while also helping to transform their IT.</li></ol>We measured our outcomes of the whole program based on business results for Delta.<img width=\"1024\" height=\"768\" src=\"http://jonathancavell.com/wp-content/uploads/2023/08/Celebrating-my-first-job-on-Wall-Street-back-in-the-day-for-Morgan-Stanley-1-1024x768.jpg\" alt=\"\" loading=\"lazy\" srcset=\"https://jonathancavell.com/wp-content/uploads/2023/08/Celebrating-my-first-job-on-Wall-Street-back-in-the-day-for-Morgan-Stanley-1-1024x768.jpg 1024w, https://jonathancavell.com/wp-content/uploads/2023/08/Celebrating-my-first-job-on-Wall-Street-back-in-the-day-for-Morgan-Stanley-1-300x225.jpg 300w, https://jonathancavell.com/wp-content/uploads/2023/08/Celebrating-my-first-job-on-Wall-Street-back-in-the-day-for-Morgan-Stanley-1-768x576.jpg 768w, https://jonathancavell.com/wp-content/uploads/2023/08/Celebrating-my-first-job-on-Wall-Street-back-in-the-day-for-Morgan-Stanley-1.jpg 1216w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" /><img width=\"1558\" height=\"618\" src=\"http://jonathancavell.com/wp-content/uploads/2023/08/IBM-57502b16.png\" alt=\"\" loading=\"lazy\" srcset=\"https://jonathancavell.com/wp-content/uploads/2023/08/IBM-57502b16.png 1558w, https://jonathancavell.com/wp-content/uploads/2023/08/IBM-57502b16-300x119.png 300w, https://jonathancavell.com/wp-content/uploads/2023/08/IBM-57502b16-1024x406.png 1024w, https://jonathancavell.com/wp-content/uploads/2023/08/IBM-57502b16-768x305.png 768w, https://jonathancavell.com/wp-content/uploads/2023/08/IBM-57502b16-1536x609.png 1536w\" sizes=\"(max-width: 1558px)'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'100vw, 1558px\" /><strong>IBM - Associate Partner, Cloud Advisory Services</strong>Obviously, this role eventually led to me being part of the <a href=\"https://www.cio.com/article/189224/what-is-kyndryl-ibms-managed-infrastructure-services-spin-off-explained.html\">Kyndryl spinoff</a>, but my role at IBM pre-Spin was different to my current role at Kyndryl.\\xa0 In the nearly two years I spent (back) at IBM, I was instrumental in building and expanding the Cloud Advisory Services team. My team was the Cloud Engineering team and I was responsible for creating service offerings, hiring/developing teams of engineers that could work with IBM sellers to sell and deliver projects in infrastructure automation, containerization, and DevOps..<ul><li>My most rewarding project at IBM was both selling and delivering a project at Delta Airlines. I was one of the top team members that sold <a href=\"https://www.wsj.com/articles/delta-taps-ibm-to-move-more-applications-to-the-cloud-11613653206\">the 9 figure deal</a> and I stayed on it for a year to lead the implementation of their Secure Cloud Foundations on AWS.</li><li>In my two years my team grew from just a handful of people to 23 engineers, architects, and managers.</li><li>We delivered over two dozen engagements around Kubernetes, Cloud Foundations, and Automation.</li><li>We accounted directly for ~$15M in revenue and helped sell tens of millions more in related engagements delivered by other parts of IBM.</li></ul><img width=\"1024\" height=\"768\" src=\"http://jonathancavell.com/wp-content/uploads/2023/08/Getting-the-team-together-at-IBMs-real-office-for-the-first-time-post-pandemic-1-1024x768.jpg\" alt=\"\" loading=\"lazy\" srcset=\"https://jonathancavell.com/wp-content/uploads/2023/08/Getting-the-team-together-at-IBMs-real-office-for-the-first-time-post-pandemic-1-1024x768.jpg 1024w, https://jonathancavell.com/wp-content/uploads/2023/08/Getting-the-team-together-at-IBMs-real-office-for-the-first-time-post-pandemic-1-300x225.jpg 300w, https://jonathancavell.com/wp-content/uploads/2023/08/Getting-the-team-together-at-IBMs-real-office-for-the-first-time-post-pandemic-1-768x576.jpg 768w, https://jonathancavell.com/wp-content/uploads/2023/08/Getting-the-team-together-at-IBMs-real-office-for-the-first-time-post-pandemic-1.jpg 1216w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" /><img width=\"3840\" height=\"2160\" src=\"http://jonathancavell.com/wp-content/uploads/2023/08/RBC-Emblem.png\" alt=\"\" loading=\"lazy\" srcset=\"https://jonathancavell.com/wp-content/uploads/2023/08/RBC-Emblem.png 3840w, https://jonathancavell.com/wp-content/uploads/2023/08/RBC-Emblem-300x169.png 300w, https://jonathancavell.com/wp-content/uploads/2023/08/RBC-Emblem-1024x576.png 1024w, https://jonathancavell.com/wp-content/uploads/2023/08/RBC-Emblem-768x432.png 768w, https://jonathancavell.com/wp-content/uploads/2023/08/RBC-Emblem-1536x864.png 1536w, https://jonathancavell.com/wp-content/uploads/2023/08/RBC-Emblem-2048x1152.png 2048w\" sizes=\"(max-width: 3840px) 100vw, 3840px\" /><strong>Royal Bank of Canada (RBC) - Director, Technology</strong>In my time at RBC, I oversaw three primary projects/teams:<ul><li>The first one was the Infrastructure Organizations DevOps Initiative. As part of this initiative we worked to evolve simple infrastructure automation into powerful DevOps tooling. We do this by exposing the infrastructure to developers and application teams as APIs so that they can use it in their CI/CD pipelines or to scale their application at runtime.</li><li>In mid-2018 I began overseeing RBC\\'s rollout of a Kubernetes cluster designed to be used by Application Development teams. My team has gone through the MANY ways that Kubernetes can be used and attempted to prioritize the most valuable ones, while building CI/CD pipelines, security scanning, and other tools to ease adoption.</li><li>From early 2017 to mid 2018 I oversaw the development, maintenance, and testing of RBC\\'s internal service catalog. With the service catalog we provide'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'innovative ways to connect the business to IT by applying similar UX principles to e-commerce. We also implemented agile development methodologies in an infrastructure ops team. Finally, we focused on providing advanced metrics to our service providers so they can get valuable feedback on when/where/how customers find their services valuable.</li></ul><img width=\"1216\" height=\"912\" src=\"http://jonathancavell.com/wp-content/uploads/2023/08/Showing-off-some-of-the-software-we-built-at-RBC.-1.jpg\" alt=\"\" loading=\"lazy\" srcset=\"https://jonathancavell.com/wp-content/uploads/2023/08/Showing-off-some-of-the-software-we-built-at-RBC.-1.jpg 1216w, https://jonathancavell.com/wp-content/uploads/2023/08/Showing-off-some-of-the-software-we-built-at-RBC.-1-300x225.jpg 300w, https://jonathancavell.com/wp-content/uploads/2023/08/Showing-off-some-of-the-software-we-built-at-RBC.-1-1024x768.jpg 1024w, https://jonathancavell.com/wp-content/uploads/2023/08/Showing-off-some-of-the-software-we-built-at-RBC.-1-768x576.jpg 768w\" sizes=\"(max-width: 1216px) 100vw, 1216px\" /><img width=\"2000\" height=\"1348\" src=\"http://jonathancavell.com/wp-content/uploads/2023/08/Fannie_Mae.png\" alt=\"\" loading=\"lazy\" srcset=\"https://jonathancavell.com/wp-content/uploads/2023/08/Fannie_Mae.png 2000w, https://jonathancavell.com/wp-content/uploads/2023/08/Fannie_Mae-300x202.png 300w, https://jonathancavell.com/wp-content/uploads/2023/08/Fannie_Mae-1024x690.png 1024w, https://jonathancavell.com/wp-content/uploads/2023/08/Fannie_Mae-768x518.png 768w, https://jonathancavell.com/wp-content/uploads/2023/08/Fannie_Mae-1536x1035.png 1536w\" sizes=\"(max-width: 2000px) 100vw, 2000px\" /><strong>Fannie Mae - Software Development Manager</strong>I held a number of roles at Fannie Mae, but the most rewarding was creating a piece of software that helped Fannie transform after the financial crisis by moving their actual securitization from being an internal function to Common Securitization Solutions.\\xa0 Obviously, this required a major transformation of Fannie’s internal Securitization applications.\\xa0 My team built a new piece of software that became the “Gateway” between Fannie’s existing securitization processes and Common Securitization Solutions.<br /><br />I built the team from scratch at Fannie Mae and incorporated the latest DevOps, Agile, and Testing techniques.\\xa0 The program ran across three agile teams and delivered 5 components in Java, Javascript, Axway Decision Insight, and Tibco Businessworks. We successfully implemented agile and CI/CD practices to get these teams delivering integrated, tested, working software every 2 weeks. From the time I began managing the project to when I left it was gained more than 30% in efficiency (from a productivity per dollar perspective) and became far more reliable about meeting our release commitments.\\xa0\\xa0<img width=\"1024\" height=\"411\" src=\"http://jonathancavell.com/wp-content/uploads/2023/08/The-Team-I-led-at-Fannie-Mae-1-1024x411.jpg\" alt=\"\" loading=\"lazy\" srcset=\"https://jonathancavell.com/wp-content/uploads/2023/08/The-Team-I-led-at-Fannie-Mae-1-1024x411.jpg 1024w, https://jonathancavell.com/wp-content/uploads/2023/08/The-Team-I-led-at-Fannie-Mae-1-300x120.jpg 300w, https://jonathancavell.com/wp-content/uploads/2023/08/The-Team-I-led-at-Fannie-Mae-1-768x308.jpg 768w, https://jonathancavell.com/wp-content/uploads/2023/08/The-Team-I-led-at-Fannie-Mae-1-1536x616.jpg 1536w, https://jonathancavell.com/wp-content/uploads/2023/08/The-Team-I-led-at-Fannie-Mae-1.jpg 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" /><img width=\"250\" height=\"67\" src=\"http://jonathancavell.com/wp-content/uploads/2023/08/adaptivity_logo_s_5B15_5D.png\" alt=\"\" loading=\"lazy\" /><strong>Adaptivity - Produce Manager</strong>It was at Adaptivity that I first made my transition from working on pure infrastructure to software development.\\xa0 We had started as a niche infrastructure consultancy designing platforms for banks.\\xa0 We eventually realized that many of the patterns we saw'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "\"could be coded into software.\\xa0 Beginning in July 2011 I stopped consulting on infrastructure and took on the role of Product Manager for two of Adaptivity's most critical products.<br /><br />In this capacity I was responsible for understanding and sizing the market for new features and additions to these products, recommending which features should make up a new version, working with the design and development teams to implement the new versions effectively and promoting the new features to both our internal sales and services teams and our customers. In the time I managed Visualization Studio it transformed from an underused feature of Adaptivity's Design Studio Product in to our top-selling tool.Adaptivity went on to sell to EMC shortly after I left the company.\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'2019 is starting off in a great place; I’m very happy with the friendships/relationships that I\\'ve formed in New York over the last couple years (and the ones I\\'ve maintained via distance), I have a chance to work on a really interesting project at work, and I\\'m in reasonably good physical shape. \\xa0The past few years  there have often been \"low hanging fruits\" in my life that desperately needed attention and where I could quickly get very rewarding progress (e.g. having let my weight go).  This year I feel the need to carefully select what I want to improve because it will take a lot of work to get significantly better than where I am.I ended up with three distinct goals for 2019 that can help take me to an even better place:<!-- wp:list {\"ordered\":true} --><ol><li>Enhance My Just For Fun Project - For the last 4 years I have run a website where my friends can pick NFL Football games (sort of like Fantasy Football).  I usually use it as a way to learn the technologies that my team at work has been using at a hands-on level even though I don\\'t have time to keep up with all of their progress on a line-by-line basis.  This year my team is working with such cool technologies that I\\'ve really learned a lot and built a product that\\'s pretty good.  I\\'d like to spend the off-season making it robust enough that I can open it up to users who aren\\'t just my friends.  This will require some functional enhancements; a new user interface, the ability to create and administer leagues without messing with the database, a few other odds and ends.  Most of the changes though will be creating better DevOps/Testing/Monitoring/Logging so that there are less disruptions.  My friends are pretty forgiving with outages and dumb mistakes... but I wouldn\\'t expect everyone to be.</li><li>Get a Sampling of More Cloud Technologies - My current project has me very focused on Kubernetes on premise.  I think I would benefit from a more well-rounded technical background so I can help influence other decisions.  With that in mind, I\\'m going to be working on getting a few AWS certifications and probably a GCP one.</li><li>Get Back in to Non-Technical Reading - My career has gotten a lot more technical in the last 2 years than it was before.  My focus on DevOps has only required some of the highest level knowledge of what\\'s going on in banking, best practices in development/agile, and management techniques.  I\\'ll be looking to modify my regular reading, get through a couple of books that talk about the industry at a much higher level, maybe even get certified in SAFe so that I still feel as at home in the developer\\'s scrum as I do in the devops scrum.  </li></ol><!-- /wp:list -->In addition to those 3, I want to make sure I don\\'t regress on a few areas of my life that are going well.  Most notably continuing to nurture my relationships with people and stay in reasonably good shape.  I have a few ways to measure those, but the goals are not remarkable.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'I was inspired to read this book by an interview with Botsman on a podcast I listen to.  It\\'s based on a compelling overall narrative... how have we reached a point where no one trusts the President of the United States, but people agree to stay in spare rooms of complete strangers on AirBnB and will literally put their life in the hands of a self-driving car?  Botsman came to fame writing about the sharing economy, so it makes sense to hear her perspective on the topic.The book isn\\'t as bold as I would have liked in predicting the future.  In books like these, I prefer to hear an experts opinion on what is likely to happen, whether it is good or not, and what that means we could/should be aware of.   Botsman prefers to focus only on the third of these topics; the dangerous trends we are working towards.  That said, she does offer a couple compelling cliffs that we appear to be confidently striding towards:<!-- wp:list {\"ordered\":true} --><ol><li>Not Evaluating Bot\\'s Motives - Botsman weaves together stories of her daughter interacting with Alexa and scientific studies to show that as people become more comfortable with technology, we are too trustworthy of its \"answers\".  Her concern is that these bots that we \"trust\" are not there as a public service, but to influence us in to certain behaviors.</li><li>Trusting Technology Before It\\'s Ready - Botsman also points out that there are a LOT of areas that artificial intelligence and algorithms have not figured out how to do well, but we seem to be willing to trust them without testing them.  There are several good academic studies where this behavior is exhibited.</li><li>Algorithmic Trust (Blockchain) Can Be Manipulated - Botsman covers the Ethereum DAO hack very well.  If you don\\'t know much about Block Chain you\\'ll be able to follow it.  She points out two concerns with the hack.  First, that the hacker didn\\'t really \"break\" the system, just found a way to use it that was unintended... imagine if Ethereum had become an international currency before that happened.  Second, that a relatively small group of people decided to effectively \"undo\" the change.  In this case they were doing \"good\", but it still served to demonstrate that Blockchains can be manipulated.</li><li>Reputation Based Trust Getting Out of Hand - The topic of the Social Credit Score in China is covered in more detail than I\\'ve ever read before and it is terrifying!  It\\'s essentially a credit score system that doesn\\'t just evaluate your wealth and default history, but you friends, your political leanings, the places you\\'ve gone, your grades in school, your hobbies, and more.  Essentially this will give the government access to incentivize all manner of behavior.  The consequences will be real too, \"people with low ratings will have slower internet connectivity; restricted access to more desirable night clubs, restaurants, and golf courses.  To quote a government document, \"allow the trustworthy to roam every where under heaven while making it hard for the discredited to take a single step.\"  While it is easy to say, that\\'s just China... we already evaluate people based on the number of followers they have, little blue check marks, star ratings on AirBnB or Uber, etc...</li></ol><!-- /wp:list -->Overall, I recommend the book, but only if you are interested in these topics and would like a little hand book of the current state of them.  I also don\\'t think this book will age well as these topics are evolving quickly.  If you happen to come across this blog post after 2019... I doubt I\\'d still recommend it.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'I had extremely high hopes for this book after listening to an interview with the author.  Unfortunately, most of what I liked about the book I heard in that 20 minute interview.  I believe that the core theme of this book is a topic that will shape the next 30 years of American life.  Unfortunately the vast majority of the book is spent detailing (and at times exaggerating) how evil the current system is instead of charting a course for correction without simply stopping progress.  With that in mind, I\\'m going to offer in this review a few things I like and then discuss a few places where she\\'s gone too far or failed to cover a topic.I love three aspects of the way the problem is framed:<!-- wp:list {\"ordered\":true} --><ol><li>I think it is brilliant that Zuboff connected Google\\'s (and then the rest of the Internet\\'s) switch from a \"pay per impression\" to \"pay per click\" model.  This means that Google is taking responsibility not only for displaying someone\\'s ad... but getting the user to click on it.  This incentivizes them to be actively matching customers and products.  This was the first step on to our present slippery slope.</li><li>I love the comparison of being able to commoditize and sell the ability to predict what we will click on to the ability to commoditize and sell labor which began in the industrial revolution.  Now, instead of needing 25 cobblers to make enough shoes for my city, I needed 25 laborers to operate the shoe factory.  This gave too much power to the capitalists who owned the factory and ended up resulting in unions (and eventually safety and wage related regulations).</li><li>Zuboff points out on page 192 that \"Demanding Privacy from surveillance capitalists or lobbying for an end to commercial surveillance on the internet is like asking Henry Ford to make each Model T by hand or asking a giraffe to shorten its neck.  Such demands are existential threats.\"  I agree, progress/evolution cannot be simply reversed.</li></ol><!-- /wp:list -->My problem with the book is how evil she claims the surveillance capitalists already are.  Though I credit Zuboff with the comparison to the labor market (#2 above) she spends far more time comparing it to the way totalitarian leaders take advantage of their people or the way Europeans took advantage of Native Americans.  Zuboff uses hundreds of pages trying to analogize surveillance capitalism to various evil periods in history, leaving very little of the book for the laws, regulations, social protests, etc... that might be necessary to cause a course correction.  Perhaps she\\'s already planning a sequel?Overall, I think the book is a worthwhile read because of how it brilliantly identifies one of the biggest problems of our time.  Even if you will have to glaze over the sections that over-elaborate problem and deal with the disappointment of a lack of resolutions.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'I\\'ve decided to start a bit of a series of blog posts called \"Clever Idea\".  I\\'ll use the space to talk about something I\\'m working on either in my hobby project (an app for picking Football games with your friends) or at work that I think is clever.  The intent will partly be to share a technique or technology that I think some people reading might be interested in.  I\\'ll also be hoping that occasionally someone points out to me that I\\'m not as clever as I thought and there\\'s actually a better way to accomplish what I\\'m trying to accomplish.</em><!-- wp:heading {\"align\":\"center\"} --><h2 style=\"text-align:center\">The Problem</h2><!-- /wp:heading -->As some of you know (and I\\'ll be posting about more), I have run a little app for picking football games the last 4 or 5 years.  It allows a \"league\" of people to keep score on who does the best picking games against the spread.  This year I am making an improvement to allow users who are already in the league to form new leagues.  A user is asked to specify the email addresses of everyone they\\'d like to participate in the league.  If you specify someone who is already in the league, then they are added to your list of players.  The first problem comes in when you want to invite someone who isn\\'t currently enrolled in the system.I use Google OAuth for authentication.  So I will need to \"pair\" the identity that an existing user has specified as wanting to be in their league with a logged in Google ID.  Note that I feel this is necessary because not everyone uses the email address associated with their Google ID as their primary email address, so I would not want to make it impossible to join unless the person inviting you happened to guess the right email.  One of my friends gave me the great suggestion of providing users a link.  If I send an email to the new player and say they\\'ve been invited to play in a league, I can include a link with a query string parameter that includes their \"user id\" (a random GUID created by my system).  This is where things get more difficult.  Unfortunately (in this case), I\\'m using Angular for my frontend.  Angular creates a downloadable browser app that players in the league use to interact with the system and that keeps itself refreshed through backend API calls.  I\\'m a huge fan of Angular, BUT it makes my \"pairing link\" impossible.Angular is a fully downloaded app that doesn\\'t actually go fetch other pages when the URL changes (these are handled client-side through the Angular \"router\").  Consequently, I can\\'t send someone directly to the \"pairing\" section of my app because the URL doesn\\'t \"exist\" on the web.  In <a href=\"https://angular.io/guide/deployment\">their deployment instructions</a>, Angular themselves recommend forcing all incoming traffic to a specific URL via your app server.  Seemingly this leaves me with no choice but to use an unfriendly system where I force users to land at my strange website for the first time, trust it enough to \"login\" even though they aren\\'t yet users, and paste their \"pairing ID\" (the GUID that represents them in my system) in to the system so it recognizes who they are.<!-- wp:heading {\"align\":\"center\"} --><h2 style=\"text-align:center\">The Clever Idea</h2><!-- /wp:heading -->Once you hear the idea, it seems obvious.  However, I had fully coded the crappy user experience mentioned above before I came up with it... Just build a simple webpage outside of your Angular deployment artifacts, but located alongside your app.  This means that Google will recognize the host as white listed when someone tries to login, your users will trust it as your site, AND you can send people to a specific page with query string parameters.  Below is the simple HTML/javascript page that I setup for the purpose (you can find it <a href=\"https://github.com/BurgherJon/SharingSamples/blob/master/pair.html\">here on git</a>):<!-- wp:code --><pre class=\"wp-block-code\"><code>&lt;html lang=\"en\">&lt;head>  &lt;meta name=\"google-signin-scope\"'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'content=\"profile email\">  &lt;meta name=\"google-signin-client_id\" content=\"&lt;id>.apps.googleusercontent.com\">  &lt;script src=\"https://apis.google.com/js/platform.js\" async defer>&lt;/script>&lt;/head>&lt;body>&lt;h1>Welcome to the New Player Page for LTHOI&lt;/h1>To pair, please follow these steps:&lt;ol>  &lt;li>If you don\\'t want to play Leave the House Out of It, do NOT fill in this form.  Your invitation will expire and the person who invited you will be alerted.  You will be removed from our database entirely.&lt;/li>  &lt;li>Signin to your Google Account.    &lt;div class=\"g-signin2\" data-onsuccess=\"onSignIn\" data-theme=\"dark\">&lt;/div>&lt;/li>  &lt;li>Fill in your first name and last initial.&lt;br>    &lt;input type=\"text\" id=\"fn\" value=\"Your First Name\">&lt;br>    &lt;input type=\"text\" id=\"li\" value=\"Your last initial\">&lt;br>    &lt;button align=\"center\" onclick=\"onSubmit()\">Accept Invitation to Play Leave The House Out of It&lt;/button>&lt;/li>&lt;/ol>&lt;p id=\"error\">&lt;/p>&lt;script>  var access_token=\"Unregistered\";  function onSignIn(googleUser)  {    // The ID token you need to pass to your backend:    access_token = googleUser.getAuthResponse().access_token;  }  function onSubmit()  {    // Get the user_id from the QSP to pass    var urlParams = new URLSearchParams(window.location.search);    var uid = urlParams.get(\\'uid\\');    var request = new XMLHttpRequest();    request.open(\\'POST\\', (\\'&lt;BackEnd>/bff/pair?uid=\\' + uid), true);    request.setRequestHeader(\\'googleToken\\', access_token);    try {      request.send(\\'{ \"first_name\" : \"\\' + document.getElementById(\"fn\").value + \\'\", \"last_initial\" : \"\\' + document.getElementById(\"li\").value + \\'\" }\\');      if (request.status != 200)      {        document.getElementById(\"error\").innerText = \\'That did not work.  Are you sure you were invited?  logged in?\\';      }      else      {        window.location.replace(\"&lt;dev location of LTHOI so they can login>\");      }    }    catch (err)    {      document.getElementById(\"error\").innerText = \\'That did not work.  Are you sure you were invited?  logged in?\\';    }  }&lt;/script>&lt;/body>&lt;/html></code></pre><!-- /wp:code -->'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'This book has a clever punch line... \"What can we as humans learn from computers about how to approach problems that face us in every day life?  Get the answer from a psychologist and a computer scientist.\"  A clever side effect is explaining some of the more interesting concepts in computer science (search, caching, communicating over an imperfect network, etc...) in the lens of every day life.  The result is a book that not only offers clever antictdotes on every day life, it also educates us on concepts that effect computers, networks, and computer science.Some of the interesting points that we can learn from computer science and apply to every day life:<!-- wp:list {\"ordered\":true} --><ol><li>I think their overall point is that there are a great deal of problems that, even with a super computer, cannot be solved unequivocally.  The key is to pick an algorithm that, within an appropriate amount of time, has the best chance of giving you the best answer.  Once you have confidence you\\'ve done that, live without regret.  The authors quote Bertrand Russell on this, \"it would seem we must take account of probability in judging of objective rightness... The objectively right act is the one which will probably be most fortunate.  I shall define this as the wisest act.\"  We can hope to be fortunate- but we should strive to be wise.  There are a bunch of examples that fall in to this, but my favorite is:<ul><li>If you are approaching a problem where you have 2 months to find an apartment, you don\\'t feel you can really compare two apartments without seeing both of them, and you run a very high risk of losing an apartment if you don\\'t make an offer the minute you see it.  Then it is optimal to spend 37% of your time looking but unwilling to commit, and 63% of your time ready to pounce if you find the right place.  There\\'s still only a 37% chance you actually end up in the best place you could have with that... but you will have done the \"wise\" thing.</li></ul></li><li> There\\'s a great little lesson on exponential backoff that I loved.  Essentially, exponential backoff is a practice of making repeated failures drastically reduce the frequency with which you make an attempt.  It\\'s common in computer science because computers can be so persistent if told to just retry infinitely.  My desktop application, for example, may try to communicate with the server and fail.  It then retries immediately (with a computer that a millisecond later).   If both fail it will wait a whole second (perhaps even doing some other task in the mean time) before trying again.  Then wait longer and longer between attempts because each time it fails the next time is less likely.  I had never really thought about it, but while exponential backoff is common in computer science, people much more commonly give something a certain number of tries in rapid succession and then give up forever.  The authors give this example:<ul><li>\"A friend of ours recently mused about a childhood companion who had a disconcerting habit of flaking on social plans.  What to do?  Deciding once and for all that she\\'d finally had enough and giving up entirely on the relationship seemed arbitrary and severe, but continuing to persist in perpetual rescheduling seemed naive, liable to leed to an endless amount of disappointment and wasted time.  Solution:  Exponential Backoff on the invitation rate.   Try to reschedule in a week, then two, then four, then eight.  The rate of retry goes toward zero-yet you never have to give up entirely\" [maybe they were just going through something personal]. </li></ul></li><li>They talk about how computationally intensive it is to do a full sort.  If you\\'re a computer nerd... you\\'ll love that they actually use big O notation to do so!!!  While sorting makes searching easier, you have to do a LOT of searching to justify a full sort of a large set.  Consequently they recommend a life filled with bucket sorts, where you just put things that are categorically'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'alike together and then search through the categorical subset when you need a specific item.  As a slob that treats his inbox as a steady stream... this seems to justify my actions!</li><li>There\\'s discussion of caching and different algorithms that computers use to decide what information to keep \"close at hand\" and what to file away for slow retrieval if/when needed; then brilliantly analogizes that to organizing your closet, filing cabinet, and house.  That also justifies a certain amount of apparent disorganization!</li><li>They end up basically justifying agile product/project management by pointing out the level of complexity in predicting things that are years away and have increasingly high numbers of factors influencing them.</li></ol><!-- /wp:list -->As I said though, it\\'s not all about learning life lessons.  If you don\\'t know much about computers you can expect to learn how the internet communicates messages over unreliable wires (or even thin air), how computers calculate the route between two locations, and the magic of Big O notation.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'I’ve decided to start a bit of a series of blog posts called “Clever Idea”. I’ll use the space to talk about something I’m working on either in my hobby project (an app for picking Football games with your friends) or at work that I think is clever. The intent will partly be to share a technique or technology that I think some people reading might be interested in. I’ll also be hoping that occasionally someone points out to me that I’m not as clever as I thought and there’s actually a better way to accomplish what I’m trying to accomplish.</em>This season will be the first season that I\\'m planning to open up my little app to more than just my immediate friends.  I\\'m planning to let my friends invite their friends to form leagues and hopefully end up with a couple dozen leagues.  Being somebody who has floated from infrastructure to software development and back again throughout my career, I know that this doesn\\'t just mean developing new screens for adding leagues and registering new players... This means, planning out my infrastructure so that the application can scale appropriately.Like any good architect, I started with the usual questions... how many concurrent users do I think I will have?  How many rows in the databases?  etc...  The problem here is, like most software projects, I don\\'t really know.  It could be that no one will be interested in the app and I\\'ll only have a couple users.  It could also be that sometime during the season it spreads like wildfire and I have a hundred leagues all of a sudden.I have a micro-service based architecture.  Last season it consisted primarily of containerized Spring Boot apps running on a EKS (Kubernetes) cluster and communicating with a relational database deployed on AWS RDS.  This architecture is certainly scalable relative to the vast majority of monolithic applications that exist in enterprise IT today.  I had an auto-scaling group setup to support the EKS cluster and it would scale down to two nodes and up as far as I needed.  Without re-architecting the database, it probably could have scaled to several hundred leagues.  It\\'s pretty flexible, allowing my AWS bill to run from ~$200/mo (a small RDS server, a couple small K8s application nodes, and the EKS control plane) to a cluster/db large enough to support a few hundred leagues with the only down-time being when I switched from smaller DB instances to larger ones. It\\'s not nearly as flexible as Lambda / DynamoDB though.  When I rebuilt the application this year it was with that flexibility specifically in mind.  The app now runs entirely on Lambda services and stores data (including cached calculations) in DynamoDB.  Both of these are considered serverless which means AWS ensures that my Lambda services always have a place to run and my DynamoDB data is always available, actually providing more reliability/availability than the K8s/Cluster architecture I had built.  More importantly for this post, Lambda and DynamoDB are both \"pay by the drink\".  With Amazon, those are very small drinks:<!-- wp:list --><ul><li>The base unit for Lambda is 50ms.  A typical call from the front-end of my app to the backend will result in a log line that reads: \"REPORT RequestId: 6419f5ca-f747-4b77-a311-09392fc6bcc3 Duration: 148.03 ms Billed Duration: 150 ms Memory Size: 256 MB Max Memory Used: 151 MB\".  AWS charges (past the free tier... which is a nice feature, but not the focus here) $0.0000002 per request and $0.0000166667  per GB/s.  For 10,000 calls similar to the one above, I\\'d be charged $0.002 for the number of calls $.0064 for the memory consumed.  We do need to remember that in a micro-service architecture, there are a lot of calls; some actions will result in 5 or 6 Lambda services to run.  However, based on the numbers above... if I end up with only a handful of users, my Lambda charges will be negligible.</li><li>For DynamoDB, the lower end of the scale is similarly impressive.  Charging $1.25 for a million writes, $0.25 for a'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'million reads, and $0.02 per 100,000 DynamoDB Stream Reads (more on these in another post).  I know from last seasons that if I only have a couple of leagues then, after refactoring for a big data architecture, I will end up with ~5,000 bets to keep track of that are rarely ever read (there are cached totals) but often get written and rewritten (let\\'s say 5 times per bet), ~300 games that are read every time a user loads their bet page (lets say 12,500 times players read all 300 games), and ~25 player/league records that are used on nearly every call from the UI (let\\'s say users are queried 50,000 times).  If I use those conservative guesses for usage, a small season would cost me $0.036 for the writes and the resulting DynamoDB Stream Reads and $0.95 to satisfy all the reads of the games and leagues.  That means my relatively small league is costing me less than $1.00 for the whole season.</li></ul><!-- /wp:list -->The reason I titled this post \"Serverless is Commitmentless\" is that I can build an app and host it on redundant compute and storage without really paying anything at all.  If I get lucky though, and the application is a huge success, this architecture would scale to thousands of leagues before I need to rethink my DynamoDB indexes.  As long as my revenue goes up faster than the AWS hosting fees when that happens, I have zero upside or downside risk from infrastructure cost when starting a new project.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'The book bills itself as being for the CEO that\\'s looking to make sense of how the changes in IT should change the way they view the CIO and IT.  It mostly lives up to that, though you\\'ll need to have some knowledge of agile/cloud.  As a technologist, I also found it valuable to hear the language and examples the author uses because I think they\\'ll be valuable as I talk to development managers and infrastructure executives who are trying to figure out how to sell the agile/devops/product/team approach to a business that is used to thinking about \"keep the lights on\"/multi-year initiatives/chargeback.  Overall, I\\'d recommend the book as a quick read.  Mostly to remind you of things you already know/feel and to give you words that\\'ll help you say them.A few things I liked in particular:<!-- wp:list --><ul><li>One of the primary points that he makes is that the world is changing so quickly right now that the biggest risk facing most companies is that they won\\'t be able to change fast enough.  With that in mind he argues that DevOps and Cloud are actually investments in Risk Management.  I find this powerful for two reasons:<ol><li>In many companies it\\'s the people carrying the title \"Risk Manager\" who want to stop things like deployments during business hours and hosting data outside the data center.  This points out that (often, not always) those risk managers are protecting the company from the wrong/lesser risk of small outages at the expense of the big risk (not being able to innovate as fast as your competitors).</li><li>It helps justify the cost of some of the DevOps Transformations and Cloud Transformations that need to happen.  Often these are hard to justify when they\\'re compared with the opportunity to deliver on business requirements or the opportunity to use automation to save money in infrastructure.  Framing DevOps as a risk management play, helps explain why there\\'s a need to invest in making the company better able to change.</li></ol></li><li>He actively fights the urge to make \"IT as a Business\".  Arguing that it needs to be part of the company like finance or marketing.  He, rightly, points out that in most companies IT is expected to operate like a sort of sub-contractor but not given the ability to hire freely and set salaries optimally, IT can\\'t choose to avoid business customers that become untenable, it can\\'t charge more for higher demand items instead of passing on costs (at least in most chargeback models I\\'ve seen).  Additionally, making IT a sub-contractor means adding red tape to the change process.  IT is inscentivized to offer standard products, to require the business \"pay\" for changing requirements, etc...</li><li>He uses one of my favorite business people analogies about agile software development vs project based development.  That it is like buying options.  Business people understand the risk of buying assets vs buying options... most even remember that moment of surprise in business school when you realized just how different the price is between an option and an asset.  Agile software development is like that.  You can build an MVP for the smallest slice of the market you can think of and then wait and see if they like it before agreeing to pay for the rest of the project.</li></ul><!-- /wp:list -->'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'One of my favorite experiences has always been going to Las Vegas on an NFL Sunday.&nbsp; Looking at all of the potential wagers, reading up on all of the games, talking with my friends about what they see, and placing a few bets while drinking that stale coffee at an abandoned Blackjack table is a great way to start a day.&nbsp; The best part of any weekend in Vegas though, is sitting at the little bar outside the sportsbook at the Venetian (the one where you can get free drinks by occasionally playing $1 Video Poker) and watching 14 games over 12 hours on a dozen big screen TVs.&nbsp; I try to make it out a time or two per season to risk a couple hundred dollars.Unfortunately, as a software developer and lover of discrete math… I know just how stupid this is mathematically.&nbsp; The longer you gamble on anything against the house (the gambler’s term for the casino), the less likely you are to be ahead.&nbsp; Casinos use a “spread” to make sure they’re getting more than their fair share of every bet.  The easiest way to illustrate this is to look at a bet that is perfectly even, like the flip of a coin.&nbsp; The way the House prices bets, if you wanted to win $100 on “Heads” you would have to bet $110; same with tails, risk $110 to win $100.  While you might win this once or twice, the House will eventually catch up with you.&nbsp; That’s why I only make Football bets in Vegas, I only do it a couple times a year, and I never bet much.Over the last few years though, I’ve found better and better ways to enjoy many Sundays the same way I enjoy them in Vegas (although I trade the Venetian for my local sports bar with Sunday Ticket) and without paying a tax to the House.&nbsp; It started 6 years ago when my brother and I hit upon the idea that each of us could pick a few NFL games for a friendly wager and the other person had to take the “House” side, but couldn’t charge the tax.  This way as long as I picked better than he did, I would end up winning money.&nbsp; This allowed us to watch the games together and enjoy them like we were at the casino, but without the need to fly to Vegas and pay the casinos (worst case scenario, I lost money and was paying my brother).Four years ago the idea grew again, we wanted to find a way to reduce the record keeping burden and include more of our friends.\\xa0 That year, the first version of <a href=\"http://www.lthoi.com\">Leave The Houst Out of It (LTHOI.com)</a> was born.  The idea behind it is a simple extension of what we’d been doing for a couple of years.\\xa0 We would build a “League” of friends and when any one of them wanted to pick a particular game, we would all take a tiny position against them.\\xa0 Having a lot of people playing with us made the system work even better; now when someone takes a bet that I am forced to take the other side of, I am sharing the responsibility with all the other people in the league.\\xa0 Over time, these “house positions” tend to even out such that we are truly “Leaving the House Out of It”.We soon discovered that this is an idea that doesn’t even require that money be involved.&nbsp; With the website version, instead of actual gambling… my friends and I discovered that we can just play for pride.&nbsp; Our Win-Loss record and earning against each other are tracked alongside the rest of the members of the “league” and we can battle for who has the best record.&nbsp; Now I get all of the thrill of the Venetian with none of the risk!<br>'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'There are shops that can setup a platform so all the developers need to do is code.  This is important when all the apps are similar or one platform.  Also important where regulatory concerns can be built in to the platform.Then there are shops where the developers and the teams own their infrastructure via IaC.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'What do you thin<!-- wp:list --><ul><li>Mainframes</li><li>Legacy Data</li><li>SAP</li><li>Containers</li></ul><!-- /wp:list -->'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'Blog<ul><li><a href=\"https://jonathancavell.com/2019/11/28/hundreds-of-thousands-a-still-more-glorious-nights-around-art-table/\" title=\"Hundreds of thousands a still more glorious nights around art table\"><img width=\"768\" height=\"356\" src=\"https://jonathancavell.com/wp-content/uploads/2019/11/blog_01-768x356.jpg\" alt=\"\" decoding=\"async\" loading=\"lazy\" srcset=\"https://jonathancavell.com/wp-content/uploads/2019/11/blog_01-768x356.jpg 768w, https://jonathancavell.com/wp-content/uploads/2019/11/blog_01-300x139.jpg 300w, https://jonathancavell.com/wp-content/uploads/2019/11/blog_01.jpg 970w\" sizes=\"(max-width: 768px) 100vw, 768px\" /></a><h4><a href=\"https://jonathancavell.com/2019/11/28/hundreds-of-thousands-a-still-more-glorious-nights-around-art-table/\">Hundreds of thousands a still more glorious nights around art table</a></h4>Bearable only through love hydrogen atoms bits of moving fluff culture shores of the cosmic ocean paroxysm of global death rich in heavy atoms ...<a href=\"https://jonathancavell.com/2019/11/28/hundreds-of-thousands-a-still-more-glorious-nights-around-art-table/\">READ MORE</a></li><li><a href=\"https://jonathancavell.com/2019/11/27/preserve-and-cherish-that-pale-blue-dot-are-creatures-of-the-cosmos-light-years/\" title=\"Preserve and cherish that pale blue dot are creatures of the cosmos light years\"><img width=\"768\" height=\"356\" src=\"https://jonathancavell.com/wp-content/uploads/2019/11/blog_02-768x356.jpg\" alt=\"\" decoding=\"async\" loading=\"lazy\" srcset=\"https://jonathancavell.com/wp-content/uploads/2019/11/blog_02-768x356.jpg 768w, https://jonathancavell.com/wp-content/uploads/2019/11/blog_02-300x139.jpg 300w, https://jonathancavell.com/wp-content/uploads/2019/11/blog_02.jpg 970w\" sizes=\"(max-width: 768px) 100vw, 768px\" /></a><h4><a href=\"https://jonathancavell.com/2019/11/27/preserve-and-cherish-that-pale-blue-dot-are-creatures-of-the-cosmos-light-years/\">Preserve and cherish that pale blue dot are creatures of the cosmos light years</a></h4>Bearable only through love hydrogen atoms bits of moving fluff culture shores of the cosmic ocean paroxysm of global death rich in heavy atoms ...<a href=\"https://jonathancavell.com/2019/11/27/preserve-and-cherish-that-pale-blue-dot-are-creatures-of-the-cosmos-light-years/\">READ MORE</a></li></ul>'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'How to Reach MeI’m always open to meeting interesting people in tech. Let’s have a discussion about what I can learn from you or how I can help you and your enterprise!Contacts<ul><li><a href=\"https://twitter.com/BurgherJon\" target=\"_blank\" rel=\"noopener\"></a></li><li><a href=\"https://www.linkedin.com/in/jonathancavell/\" target=\"_blank\" rel=\"noopener\"></a></li><li><a href=\"mailto:Jonathan@CavellandCavell.com\" target=\"_blank\" rel=\"noopener\"></a></li></ul><p role=\"status\" aria-live=\"polite\" aria-atomic=\"true\"> <ul></ul><form action=\"/jonathan/wp-admin/admin-ajax.php#wpcf7-f5-o3\" method=\"post\" aria-label=\"Contact form\" novalidate=\"novalidate\" data-status=\"init\"><input type=\"hidden\" name=\"_wpcf7\" value=\"5\" /><input type=\"hidden\" name=\"_wpcf7_version\" value=\"5.8\" /><input type=\"hidden\" name=\"_wpcf7_locale\" value=\"en_US\" /><input type=\"hidden\" name=\"_wpcf7_unit_tag\" value=\"wpcf7-f5-o3\" /><input type=\"hidden\" name=\"_wpcf7_container_post\" value=\"0\" /><input type=\"hidden\" name=\"_wpcf7_posted_data_hash\" value=\"\" /><br /><input size=\"40\" aria-required=\"true\" aria-invalid=\"false\" value=\"\" type=\"text\" name=\"your-name\" />Name<br /><input size=\"40\" aria-required=\"true\" aria-invalid=\"false\" value=\"\" type=\"email\" name=\"your-email\" />Email<br /><input size=\"40\" aria-invalid=\"false\" value=\"\" type=\"text\" name=\"your-subject\" />Subject<br /><textarea cols=\"40\" rows=\"10\" aria-invalid=\"false\" name=\"your-message\"></textarea>Message<input type=\"submit\" value=\"SEND MESSAGE\" /></form>Publications and Speaking Engagements:<a href=\"https://www.kyndryl.com/au/en/perspectives/articles/2023/05/modern-core-systems-for-banks\" target=\"_blank\" rel=\"noopener\">Kyndryl.com</a><a href=\"https://d1.awsstatic.com/events/Summits/reinvent2022/PRT310_From-strategy-to-delivery-Modernize-your-applications-and-cloud-sponsored-by-Kyndryl-.pdf\" target=\"_blank\" rel=\"noopener\">Presentation from AWS Re:Invent 2022\\xa0</a><a href=\"https://www.amazon.com/Bpel4ws-Business-Processes-Websphere-Integration/dp/0738491640\" target=\"_blank\" rel=\"noopener\">A book I helped write for IBM in 2004 that you can, for reasons I do not know, still buy on Amazon</a>\\xa0<a href=\"https://app.qwoted.com/opportunities/event-chicago-cio-executive-summit-2022-b\" target=\"_blank\" rel=\"noopener\">Chicago CIO Summit</a>'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "\"I'm sure the nearly no one who almost never read it will rejoice. &nbsp;It’s not a cgrand re-entry; I’m as ambivelant as ever about how many people read or enjoy it. &nbsp;It’s back online because I have missed having a place for long form expression.  I enjoy thinking through something long enough to have a coherent thought on it and I have found the best way to be sure those thoughts are coherent is to put them somewhere where someone might read them.<br>If you're curious... the old blog posts, all 1050 or so of them, are gone forever. &nbsp;Around this time last year I was cleaning up my AWS account and inadvertently deleted the instance that had my blog on it. &nbsp;Any of you that have worked with me professionally will appreciate the humor in the fact that I, for a brief moment, was forced to recognize the importance of segregation of duties!<br>\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'I\\'m really excited to announce that I\\'ve taken a job with IBM, in their advisory cloud consulting service.  The team I\\'ll be leading will be looking at how organizations can architect internal and external cloud solutions on both IBM/RedHat products/clouds as well as on other external providers like AWS/Azure/GCP.  I will be based out of New York City, but have responsibility across all of North America.  I\\'m excited about this opportunity both because I think IBM is well positioned to be successful and because I think this job is a great fit for me personally.  I\\'ll use the balance of this post to give you two reasons for each.I am excited that IBM has a chance to be the preferred provider for people who care deeply about their internal/hybrid cloud.  With the acquisition of RedHat and its suite of tools for containers/kubernetes and devops, IBM has more capability than the leading external cloud providers to help you build your internal cloud and then reach out.  I believe this will be important to companies that have legacy applications that are not cloud native (and therefore run inefficiently in a cloud pricing model), have large investments in their datacenters and enough demand to cost-effectively use them, and/or have applications that for compliance/security reasons they\\'re not comfortable running from the public cloud.  I am also excited that IBM can be a trusted advisor to the \"second wave\" of companies moving to the cloud.  The first wave of cloud computing was dominated by the Netflix, Facebook/Instagram, Uber, Twitter, etc... unicorns.  They still represent a huge portion of the current public cloud resource consumption.  These first-wave companies are, primarily, software engineering companies with no legacy software.  They have been able to capitalize on cloud models because they could invest in the engineering to do so with only a few applications that serve billions of people.  For companies in this second wave (I\\'m mostly familiar with Financial Services... but much of the Fortune 500 is with them), the cloud is more challenging.  It requires deciding when to re-platform an application that only has 2 developers supporting it.  It requires retraining/retooling a development staff (or hiring one and teaching it the intricacies of a business that\\'s more sophisticated than a tweet).  It requires replacing legacy infrastructure services/products with what the new cloud-based applications require.  These second-wave companies have relied on IBM for decades for their infrastructure, middleware, etc... and I am happy to be part of the group that will let them rely on us for cloud too.Personally, this gives me a chance to stay in New York City!  I have started to fall in love with this city and the friends I have met here.  If you\\'ve known me for a while you know that I\\'ve moved a lot for jobs in my career; in fact, this will be the first time that I have taken a job in the same city as the previous one.  I\\'m looking forward to putting down some roots and really enjoying all that the Big Apple has to offer (all while getting to travel a lot!).Finally, I\\'m also excited to be back in consulting.  It gives me a chance to do what I do best, identifying opportunities.  While I think my last two jobs (standing up Kubernetes at RBC and completing a large scale software development project at Fannie Mae) have shown I can excel at implementing/operationalizing; I am excited to get back to what I think I do best, helping people identify opportunities and proving out their value.  I\\'m sure in a few months when I hear that a client has dropped the ball on a project I proposed, I will remember the brown grass on the other side of this hill.  For now though, I am really excited to get to know lots of clients and see what they\\'re doing with the cloud!If you\\'ve made it this far, I thank you for taking an interest in what\\'s going on with me.  Hopefully it\\'s because you think we can collaborate in the new position?  My group will be'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'working with all of the major external cloud providers and we\\'re going to be agressively hiring engineers, developers, architects, and consultants who can help IBM\\'s clients with their \"Journey to the Cloud\".  If you\\'re interested in seeing how we can partner, reach out to me on LinkedIn.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'This \"book\" is actually a set of conversations recorded by John Willis and Gene Kim.  The main thing I can offer by way of advice is; absolutely do NOT buy a printed version, this isn\\'t really a book.  The book is apparently a inspired by a similar \"Beyond the Goal\" lecture done by Eliyahu Goldratt who authored \"The Goal\" the novel about Lean that inspired Kim to write a novel to explain DevOps.  I highly recommend getting the book, even if you only listen to the modules on Lean and Safety Culture.The first two chapters are not the most interesting.  They cover Goldratt and Deming.  They aren\\'t uninteresting, but it was more detail than I needed on the individuals.The next two chapters are fantastic and go through a lot of the concepts that they borrowed from when creating DevOps.  Goldratt and Deming as the people who they borrowed from most thoroughly when coming up with some of the DevOps concepts.  They then turn to Lean and Safety Culture, covering the history of each discipline and the most important parts that were borrowed.  I learned a TON from these sections.  I particularly enjoyed the section on the Andon Chord (which is intended to be pulled on a production line if anything goes wrong).  I had of course glanced at the concept before, but had always dismissed it as something that was only really useful for the most mature organizations.  Kim and Willis exposed me to how Toyota actually deals with it.  Learning that they actually panic when it\\'s not pulled enough and that not every pull stops everything, it\\'s more of an escalation.  The comparisons to a pipeline are obvious.  The next chapter is actually a recording of a conversation between leaders in Lean, Safety Culture, and DevOps there were a few good nuggets, but I generally would say you can skip it. The only remaining module that I found interesting was the case studies.  Particularly the stories about Target and Nordstrom.  It also closes with an interesting observation that Willis had with the CFO of Goldman Sachs.  Apparently he is tracking roll outs of DevOps technologies specifically by name (Kafka is mentioned specifically).  It says something about critical agility is becoming, independent of the business functionality that\\'s being built on top of it.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'If you\\'re going to be working in technology for more than 10 years, you should read this book.  It\\'s written by a set of economists associated with the Creative Destruction Lab at the University of Toronto.  As you might imagine from an incubator with that name, they have made many investments in Machine Learning companies.  The observations in the book are less about how machine learning works (though there is obviously some introduction required) and more about the implications for the wider economy. As the title suggests, they believe that the current advancements in artificial \"intelligence\" are not related to general intelligence, but are making advancements in computers\\' ability to distill lots of data about what humans have done to make a prediction.  For example, Google\\'s image recognition doesn\\'t actually \"recognize\" images, it is only able to say, \"based on other sets of pixels that people have identified as tables, I think that there is a 98% chance that is a photo of a table.\"  This key assumption then allows them to predict how further advancements in these prediction machines will impact the kinds of companies, jobs, social programs, etc... that will go up (or down) in economic value.As they put it, \"Economics offers clear insights regarding the business implications of cheaper prediction.  Prediction machines will be used for traditional prediction tasks (inventory and demand forecasting) and new problems (like navigation and translation).  The drop in the cost of prediction will impact the value of other things, increasing the value of complements (data, judgement, and action) and diminishing the value of substitutes (human prediction).\"The remainder of the book explores logical deductions from that fact.  Some of the more interesting of which are:<!-- wp:list --><ul><li> Some areas of prediction only become real game changers when the machines reach near 100% certainty.  For example, imagine if when Amazon made a recommendation they could be 98% sure that you\\'d want to buy the product... would they send automatically send the product to a distribution center near your house?  Maybe just send it straight to you and let you return one product in fifty that they\\'re actually wrong about.</li><li>The value of data is going up quickly.</li><li>Machines and humans have distinct strengths... humans tend not to be thorough while machines can\\'t discover independently a new type of data that should be incorporated.  The two will be paired with machines doing the prediction and humans doing the judgement.</li><li>They identify 6 types of risks associated with Prediction Machines.  The most interesting of which is that on things like driving an airplane, humans are getting less and less skilled at it while auto-pilot does more.</li></ul><!-- /wp:list -->'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'I confess, Value Stream Mapping as a technique is something I already do with clients and I was reading this book mostly to make sure that I could say that I did (and now it\\'s on the Internet!).  I will say that after being taught value stream mapping primarily as it relates to the DevOps movement and CI/CD pipelines, I was surprised to see how general the approach is.  It seems like it would be really valuable for enterprises to do in areas outside of manufacturing (it\\'s based on Lean and started as one of the Toyota techniques) and IT.  Maybe someday I\\'ll focus more on my MBA and less on my Comp Sci background and actually use it that way.In this book \"review\", I thought I\\'d spend some time in this blog post talking about the nuggets throughout the book that are a bit different when our team looks at creating a value stream to find opportunities for IT Automation.  Feel free to leverage this in your business, or give us a call and we can help you get started!First of all, we are usually doing a couple Value Stream Maps in our spare time in the first week of an Ansible Tower Foundation engagement.  While also installing Ansible, connecting it to our client\\'s Git, Active Directory, Credential Management, etc...  Consequently they are usually a lot less involved.  This different results in a few big changes:<!-- wp:list {\"ordered\":true} --><ol><li>The scope has to be different.  The book talks about focusing on customer value streams being from \"ring to ring\" (from the phone call to order to the ring of the cash register).  We still like to focus on this, but we often take IT Operations customers ring (a ServiceNow request to get a new VM) to service fulfillment (a notification to the user that they can log in).</li><li>The session itself is shorter.  I like to try to keep it to either one hour or two depending on the complexity.  This allows for nice crisp time keeping.  10 minutes to intro the concept, 20 minutes for current state, 10 minutes to brainstorm opportunities for improvement, 20 minutes for future state.</li><li>I think the charter becomes MORE important when you want to do a short session.  It is important to have your sponsor identify the limitations of the exercise and the precise scope.  This will limit wasteful conversation during the workshop.</li></ol><!-- /wp:list -->Since our focus is typically on finding out how best to improve an IT Operations value stream via automation, we are looking for slightly different things in the value stream map:<!-- wp:list {\"ordered\":true} --><ol><li>We do not limit ourselves in looking only for automation opportunities.  We are looking for all manner of improvements... avoiding batching, identifying opportunities to run in parallel, etc...  It\\'s not our goal to fix those as part of the project (though IBM has some excellent process consultants).  It\\'s merely our goal to avoid the investment of automating a process that shouldn\\'t exist at all or is not a bottleneck.</li><li>The Gemba walk is usually not a physical one.  I like to have actual operators walk an actual ticket on screen where possible.  In longer running processes it may be constructive to view several tickets in different phases.</li><li>We use all of the resulting automation \"kaizens\" to start our clients\\' automation backlog.  Because of the value stream mapping we can be clear about just how much value can be expected out of each automation.</li></ol><!-- /wp:list -->Overall, I definitely recommend reading the book.  Even if you\\'re only planning to use the technique for CI/CD or IT Operations value streams.  It will give you a lot of perspective on the history of the technique and its uses in the wider enterprise, in addition to arming you with the vocabulary you need to leverage it for your use case.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'If you\\'ve read much of Gene Kim\\'s work you\\'ve heard of the transformation that HP Printers made to their firmware development process.  Kim cites it so frequently because it is an example of one of the oldest software development groups around making software that\\'s some of the hardest software to test (printer emulators anyone?) transforming from a waterfall development process to an agile one.  Gruver and Mouser are the executives behind that transformation and this book is full of great lessons that everyone who\\'s had an IT department for longer than Amazon\\'s been a bookstore can learn from.Let me give you three of my favorite things about this book, and I encourage you to read it and come up with your own:<!-- wp:list {\"ordered\":true} --><ol><li>In the very first chapter they talk about how many companies make the mistake of trying to start an enterprise agile transformation by just having one team transform.  It doesn\\'t work because, at least in most companies, a meaningful development team can\\'t go to production on its own.  You have that one transformed team running scrums and using whiteboards full of sticky notes to deliver code to an integration test environment where it will sit for 6 months.  The key is to find ways to iteratively improve the whole organization (or at least ALL the parts that have to develop together) towards agile.</li><li>Having worked in technology all my life, I had never really considered why software development should be managed so much differently then your other cost centers.  I really liked their point that what makes software development so hard to plan for is that it\\'s hard to see and often changes from looking 80% done to looking destined for the trash heap overnight.  That ever-changing aspect is also what\\'s great about software.  It\\'s hard to predict where you\\'ll be in 6 months in a software dev project, but it\\'s much much easier to change your mind today about where you want to be tomorrow.  Doing agile software development makes so much sense because it\\'s just using the inherent strengths of software rather than trying to mold it in to a waterfall techniques.</li><li>Finally, this book isn\\'t afraid to make the tough point that executive teams must understand technical concepts like continuous delivery to be successful.  They can\\'t just be great people managers or inspirational leaders.  They have to know when to make exceptions to the \"no branches\" rule and when to require automated tests.  These are things that reasonable development managers can disagree on, but a knowledgeable executive must set a standard for.</li></ol><!-- /wp:list -->'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'2020 was a year where I expected to just entrench and work on establishing myself at IBM.  It was a really good year for that, and (thanks to a lack of travel and a bit of a COVID slow period) also allowed me to set some <a href=\"https://www.linkedin.com/posts/jonathancavell_covid-has-made-it-hard-to-do-a-lot-of-the-activity-6735798923555864576-Jz47\">pretty ambitious fitness goals</a>.  That leaves me feeling pretty good about what I was able to accomplish, making it possible for 2021 to be more about refining and doubling those successes.Specifically, I have set three goals:<!-- wp:list {\"ordered\":true} --><ol><li>I want to really focus on working with CIOs/CTOs to make their \"Infrastructure and Operations\" teams in to a value add part of IT.  As I spend more time with more companies I see them consistently able to articulate how a better customer experience or using AI can bring value to their bottom line, but then they are unable to understand why they can\\'t securely and efficiently create the platforms necessary for those innovations to occur on top of.  I am intending to spend 2021 moving beyond the containerization, cloud, and automation discussions I\\'m leading customers on now and pull that together in to a more strategic view of the organization.  You\\'ll see this in more frequent observations on Twitter, some blog posts, and (if you\\'re lucky enough to be a client) in a new model we\\'ve been working on for how to change your culture and tooling.</li><li>I am planning to be much better about keeping up with my professional contacts.  I realized yesterday that I haven\\'t done an inventory of my contacts since very early 2020.  My plan will be to do better at connecting with people I\\'ve worked with on LinkedIn and reaching out to folks I worked with a few years ago to see how they\\'re doing.</li><li>I managed to finish 2020 in reasonable shape.  I have not traditionally been good at staying in good shape once I reach it.  I tend to yo-yo back and forth between in shape and out of it.  My goal this year is to stay around my current level of fitness for the whole year.</li></ol><!-- /wp:list -->'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'I found a lot of great nuggets in this book, but I\\'m not sure I\\'d recommend it to everyone.  If you\\'re trying to sell your boss on a move to AWS, definitely give her this book.  If you\\'re trying to figure out what your IT priorities are, I\\'d recommend <a href=\"http://jonathancavell.com/2020/07/28/book-review-leading-the-transformation-by-gruver-mouser/\">Leading the Transformation</a> or The Pheonix Project.  If you insist on reading something that\\'s written by an AWS Employee, I\\'d even try <a href=\"http://jonathancavell.com/2019/08/01/book-review-war-peace-it-by-mark-schwartz/\">War, Peace, and IT by Mark Schwartz</a>.  While I found some nuggets that I enjoyed and found informative, I found most chapters just identifying a problem and then explaining how being in the cloud mitigates that problem.    It\\'s not really surprising considering it\\'s 53 chapters in only 298 pages (at least how Kindle counts it).  One big exception to this is Chapter 5 and Chapter 53 outline processes and are a little more detailed (but each of them could be an entire book).  The other little nuggets that I liked were:<!-- wp:list --><ul><li>Chapter 23\\'s discussion of the future of Managed Services in the cloud was interesting.  I believe the combining of migrating to cloud and managing applications (essentially turning them in to SaaS) was a great point.  Also, the highlighting of DevOps as an important feature of an MSP.</li><li>I liked Chapter 33\\'s discussion of Hybrid Cloud.  Too often AWS and their surrogates talk about Hybrid Cloud as a fantasy.  Unless you have the kind of capacity that it\\'s worth building an entire datacenter that runs just for you (like Netflix), you\\'re probably better off having everything in the cloud.  The problem is, you can\\'t just put it there by magic.  Orban does a nice job recognizing Hybrid Cloud as a stage, maybe a VERY VERY long one, but not a destination.</li><li>I liked the discussion in Chapter 46 about a company\\'s vision for moving from Centralized IT to Exponentially Growing IT.</li></ul><!-- /wp:list -->'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "\"I rarely bother to post books I read for pleasure on the blog, but I thought that a lot of the people that I know professionally may be interested in this one.  I started reading it in November when we were just starting to be able to think about what a post-Trump America might look like.  After four years of hating the news so much I rarely watched it, I was trying to remember what it felt like when I had agreed to let an Obama staffer live in my extra bedroom when the 2008 campaign was tight in Pennsylvania.  This book did not restore that optimism.The book is simultaneously interesting and boring because President Obama really dives in to telling about what his goals were, what challenges he met, and how he and the team dealt with them.  The book would be more inspirational, and more of a page turner, if he left out those details.  It would also feel like a commercial for his presidency instead of a memoir.  If you're in the mood to hear the details of the political process and how Obama and team tried to manipulate it, you'll enjoy this book.  If you're looking for a compelling narrative that's fun to read, there's an abridged version.One last recommendation.  Definitely get the audiobook, he reads it himself and I think being able to hear his emotions and emphasis while he reads is worth the slower pace.\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'Jeff Bezos built a company that\\'s worth about the same on the stock market ($1.67T) as the entire circulating supply of Pakistani Ruppees ($1.68T).  That means you could BARELY trade every Pakistani Rupee in existence for all the stock in Amazon.  I guess I\\'ll read some of his wisdom and see what he has to say.I found the book pretty interesting.  It\\'s not exactly a page turner and there are parts that are a little repetitive (the guy has clearly practiced telling his life story), but you can leaf through those parts quickly and be back in good material.It starts with a republishing of all of his shareholder letters as CEO of Amazon (the last one you\\'ll have to find on the internet until they publish a new edition).  They\\'re a fantastic read.  It\\'s almost a retelling of the whole internet.  From an optimistic Bezos reporting that Amazon had \"established long-term relationships with many strategic partners, including America Online, Yahoo!, Excite, Netscape, GeoCities, AltaVista, @Home, and Prodigy.\" in 1997.  To a one word title of \"Ouch\" in 2001 after he saw some 90% of the value of Amazon wiped out.  To explanations of how Amazon refuses to take a short-term perspective, even in a world where quarterly earnings are analyzed so heavily. To discussions of how, being a big company allows it to make big bets like the Fire Phone (dud) and AWS (clear winner) that it might not always win.  To the mentality of \"customer first\" that Bezos uses to push employees because customer loyalty only lasts until your competition shows your customer something they never knew they wanted.The remainder is a collection of Bezos\\' speeches.  This got repetitive with stories of how he admired his grandfather for being resourceful on a remote farm and sticking up for his mom when she was a pregnant high school. You should feel free to skip around, but don\\'t miss the speech on Blue Origin (his space company).  The company is endeavoring to build the \"infrastructure\" for future generations to reinvent industries in space.  He talks about the need to have big Lunar landers to send enough equipment to the moon so that the natural resources on the moon can start to be used for construction of more items in space (this makes sense since it\\'s much more efficient than trying to launch HEAVY things out of earth\\'s gravitational pull).  It\\'s a fascinating way to choose to spend a personal fortune that is roughly 2,000,000 times larger than the average American\\'s.Because of where I am professionally (with Kyndryl about to break off of IBM and provide me the opportunity to really grow the size, number of services, and the value we can provide the customers of my Cloud Advisory Service practice), I found the 2016 shareholder letter the most impactful.  He talks about techniques for always staying in Day 1 because \"Day 2 is stasis.  Followed by irrelevance.  Followed by excruciating, painful decline.  Followed by death.  And that</em> is why it\\'s always</em> day 1.\"  Resisting the urge to just get what you can from customers and instead keep innovating for customers, is the energy we\\'re going to need at Kyndryl.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'I can usually tell by 15 pages in to a self-help book whether it\\'s going to resonate with me; usually, it\\'s not going to.  I can\\'t stand being told about how much I can get done<a href=\"https://readingraphics.com/book-summary-the-miracle-morning/\"> in the morning</a> if I start at 3am or that if I work <a href=\"https://jamesclear.com/book-summaries/10x-rule#:~:text=The%2010X%20Rule%20says%20that,not%20setting%20goals%20high%20enough.\">10x harder</a> than everyone else it will pay off.  These are obvious, hard work usually pays off (although if you think it automatically does, you should read <a href=\"https://www.amazon.com/Peak-Secrets-New-Science-Expertise/dp/1531864880\" data-type=\"URL\" data-id=\"https://www.amazon.com/Peak-Secrets-New-Science-Expertise/dp/1531864880\">Peak by Anders Ericson</a>).Dweck\\'s book is entirely different.  It suggests that if you want to find fulfilling success, you should look at the world a little differently.  You should stop asking yourself if you are successful and start asking what you can do to grow.  I have to admit that 15 pages in to this book, I thought the idea was too simple to be useful.  The more I read though, the more I liked the way it challenged me to think about life.I\\'ve always been one to enjoy my successes with a little pat on the back (or more likely a celebration scotch).  What I considered a success though was often what the rest of the world would view that way.  For example, if my team at work won a new big project I would celebrate.  If we had a month where we didn\\'t get a new project, I\\'d feel bad.  It always felt a little cheap celebrating when the project win was just lucky and it was always tough to feel too bad when we made a great pitch to a customer that I knew would help them in the long-run, but that got scuttled when a key stakeholder retired.  This book gave me a better way to look at these projects than pure outcome based success or failure.Dweck would have us focus on \"growing\".  In the above example, the team clearly grew in our capability to identify and sell key projects by making that presentation (maybe we also learned how to identify clients we shouldn\\'t bother with).  Dweck points out that it\\'s not that luck doesn\\'t exist; it\\'s just that you don\\'t want to let let your luckiest moments be the way you define yourself.  This construct has given me a more consistent way to approach my days.  Even on a bad luck day I start thinking about what I can learn from the situation and the best possible way I can move from here.  If I\\'ve done my best and learned, I feel content and even more ready to take on the next day.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'If you were trying to get a gist of the overall maturity level of an IT Ops and Infrastructure organization by asking only one question, I think you’d be hard pressed to beat the question, “How mature is your DevOps?”\\xa0 I’d prefer that to “How do you use the cloud?” or “How fast can you provision a server?”.\\xa0 The reason is that DevOps is the only industry buzz word that has a built-in “why”.\\xa0 We get better at DevOps for a specific reason… to lower the friction for agile development teams.\\xa0 There’s lots of reasons to go to cloud and a lot of reasons to automate things… some of them are good and some of them are bad, but if you’re always seeking to make dev teams more agile, you’re likely on the right track.That’s why I’m always so enthusiastic to read the “State of DevOps” each year (a report commissioned by Puppet).\\xa0 It helps you see where high-performing DevOps organizations (ones that can deploy to production regularly) are differentiating themselves from lower performers.\\xa0 This can help prioritize your strategic goals and initiatives.This year I had three key takeaways from the report:<!-- wp:list {\"ordered\":true,\"type\":\"1\"} --><ol type=\"1\"><li><strong>Automation and cloud are key to DevOps, but definitely not the only keys</strong>.&nbsp; 62% of companies stuck in “mid-evolution” on their DevOps Journey say that they have high levels of automation.&nbsp; 65% of those “mid-evolution” companies are on public cloud, but only 20% using it effectively.</li><li><strong>The DevOps Journey is something we’ll be focusing on for a while.</strong>&nbsp; From 2018 to 2021 only 8% of companies graduated to high performing DevOps teams (going from 10% to 18% of all companies).</li><li><strong>Platform Teams are becoming key.</strong>&nbsp; This is something that I’ve been working with my customers at Kyndryl on for the last couple years.&nbsp; Platform teams are highly correlated not only with high performance on the devops scale, but also with employees feeling like they know their role.</li></ol><!-- /wp:list -->Check out the report <a href=\"https://puppet.com/resources/report/2021-state-of-devops-report/\" target=\"_blank\" rel=\"noreferrer noopener\">here</a> and let me know if you think I missed something.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'The word \"optimist\" in the subtitle is very well earned here.  Friedman\\'s book explains why/how someone can hope that the same technologies and macro-trends that are leading to hyper-nationalism, extreme divisiveness, and massive pollution might actually be harnessed for good.  He tells that story with his typical great story telling and insightful anecdotes.As usual I found a bunch of tidbits interesting and had a few realizations while reading the book that I\\'ll just list here:<!-- wp:list --><ul><li>One of the points that\\'s made over and over again that resonated with me is how fast technology is moving and how slow our institutions (particularly laws) are adapting to it.  Especially since those technologies are contributing to the gridlock that\\'s keeping us from effectively regulating them (let alone adopting them for public purposes).</li><li>I found interesting that Friedman suggests his hometown in Wisconsin as a place well suited to adapt to the new world, mostly because of the strong community and the small size enabling a single suburb to really make it theirs.  I understood the argument, but couldn\\'t help thinking that I\\'m glad I live in New York City... the combination of wealth and bright people I think will help push our leaders to adopt new advances more quickly in spite of our large size and less than community feel.</li><li>We\\'ve all heard Moore\\'s law, but I thought this explanation of just how fast computing power has grown to be compelling: \"If a 1971 Volkswagen Beetle improved at the same rate as microchips did... In 2015 that Beetle could go 300,000 mph, get 2,000,000 mpg, and would cost $0.04.\"</li><li>He makes a compelling argument for us entering the \"cognitive\" era of computing.  While I still think this will be slower to take hold than people expect, it is fascinating that we have enough compute power now to just throw a bunch of data at the cloud and let the computers sort out if it means anything and what it means (as opposed to old school computing where you gave it an algorithm to make sense of the data with).</li><li>Since the book is optimitistic it makes the point that while artificial intelligence will do a lot of what we do for jobs today, that is likely to actually lead to more jobs.  He brings up the example of automation in the textile industry.  It actually caused MORE people to be employed making clothes because the price dropped so far that individuals owned more (far more) than one set of clothes. </li></ul><!-- /wp:list -->'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'<!-- wp:heading --><h2>Why do a Big Data Project over the Holiday?</h2><!-- /wp:heading -->In 2021 the machine learning market was a little over $15B.  That is projected to increase 10x between now and 2028.  It\\'s the fastest growing area of technology (think mobile 10 years ago) and therefor it is top of mind for my clients.  In addition, the sophisticated (read as expensive) hardware, software, and staff required to do on-premise, original Machine Learning is cost prohibitive for many current companies.  I believe that, increasingly, \"access to the hardware and off-the-shelf software that are provided by the hyperscalers\" will become one of the primary reasons clients begin or accelerate their cloud journey.  Right alongside \"closing a datacenter\" or \"decreasing time-to-market\" or \"increasing availability.\"I\\'m certainly not new to creating cloud environments to support machine learning.  I have created several kubernetes clusters and cloud environments across multiple clients with the explicit goal of supporting their AI/ML or Big Data efforts.  In spite of that, I had little knowledge of what actually happened in those environments.  With that in mind, I decided to embark on building an AI based \"player\" for the fantasy/gambling app that I already use to keep my hands-on skills sharp.<!-- wp:heading --><h2>Introducing Book-E the robot gambler.</h2><!-- /wp:heading -->As many of you know, I currently run an \"app\" that lets my friends and I keep score on our football predictions. It\\'s described reasonably well on the homepage (<a rel=\"noreferrer noopener\" href=\"https://lthoi.com/\" target=\"_blank\">https://lthoi.com/</a>).  The TLDR version is that it allows players to chose wagers that should have even odds (they are coin flips) and then forces each of the other players in the game to take a portion of the other side of the wager.  So, our AI/ML \"player\" in the game will have to pick which over/under and spread position bets they want to make each week.  In order to have some fun with this, will call our AI/ML player \"Book-E\"Book-E (assuming I can finish the project) will do a few things:<!-- wp:list {\"ordered\":true} --><ol><li>Keep an up-to-date data set of all of the relevant football games and the data about them.</li><li>Use machine learning to create a \"model\" of what kinds of bets will win.</li><li>Evaluate each game just before betting closes (to have the best data) and pick which bets (if any) to make.</li></ol><!-- /wp:list --><!-- wp:heading --><h2>What tools/training am I going to use?</h2><!-- /wp:heading -->I\\'m going to have a lot to learn to complete this project!  I will need to gather the data, to process the data in to data set(s) that can be used for machine learning, to create and then serve a machine learning model, and (finally) to integrate that model with my current game so that we have a new \"player\".Given my focus in 2021/2022 on AWS, I\\'m planning to focus on AWS technologies.  I plan to leverage all of the AI technology in SageMaker for capturing the data and creating/serving the machine learning model.  Also, since my application is AWS based (<a href=\"http://jonathancavell.com/2021/11/05/clever-idea-serverless-cloud-native-ci-cd/\" data-type=\"URL\" data-id=\"http://jonathancavell.com/2021/11/05/clever-idea-serverless-cloud-native-ci-cd/\" target=\"_blank\" rel=\"noreferrer noopener\">a set of lambdas, dynamodb tables, SQS queues, and an API Gateway</a>), I will be adding a few lambdas and cloudwatch triggers to make the AI Player actually place \"bets\" and update models without the need for human intervention.For the aggregating of the data, I am going to be using Python and Jupyter Notebooks as my workspace.  Since I\\'m planning to be very AWS dependent I\\'m going to use the <a rel=\"noreferrer noopener\" href=\"https://aws.amazon.com/sagemaker/studio/\" target=\"_blank\">AWS Sagemaker Studio</a> as my IDE.  The data will come from existing tables in my application (which I will access using the AWS SDK known as'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'boto3) and from the company I use to provide my scores/data for the game (which I will access through the <a rel=\"noreferrer noopener\" href=\"https://github.com/MySportsFeeds/mysportsfeeds-python\" target=\"_blank\">Python wrapper they provide</a>).For creating and serving the actual machine learning model, I plan to use AWS SageMaker.  Specifically, I\\'m really excited about the <a rel=\"noreferrer noopener\" href=\"https://aws.amazon.com/sagemaker/autopilot/\" target=\"_blank\">AWS Autopilot</a> functionality which will select the best machine learning model for me without me having to be a data scientist.This is going to require some training!  At the onset of this project, I do not know much about AWS Sagemaker, AWS Sagemaker Studio, Python, the AWS SDK for Python, Jupyter notebooks, or machine learning!  I identified the following Udemy courses that I plan to go through:<!-- wp:list --><ul><li><strong><a rel=\"noreferrer noopener\" href=\"https://www.udemy.com/course/practical-aws-sagemaker-6-real-world-case-studies\" target=\"_blank\">AWS SageMaker Practical for Beginners | Build 6 Projects</a></strong> - This is my primary course.  It does a great job introducing the concepts of machine learning, the different types of models, and the ways to evaluate models.  Even better, it does this using AWS Sagemaker and Sagemaker Studio as the tools.</li><li><a href=\"https://www.udemy.com/course/mastering-boto3-with-aws-services/\"><strong>AWS - Mastering Boto3 &amp; Lambda Functions Using Python</strong></a> - This course was a great way to get started with both Python in general and with Boto3 (which is the AWS SDK for Python).  If you\\'re a bit of an idiot (like me) and jumping in to this project without background in Python, let me HIGHLY recommend chapter 5 which covers a lot of what you need to know about Python generally in 58m.  This would probably only be a sufficient overview if you have a decent amount of programming experience.</li><li><a href=\"https://www.udemy.com/course/data-manipulation-in-python\"><strong>Data Manipulation in Python: A Pandas Crash Course</strong></a> - This course was great for an introduction to Pandas (a library in Python that\\'s useful for data manipulation/review) and Jupyter notebooks.  While these are both touched on in the first course I mentioned above, if you\\'re going to actually do some of your own coding, you\\'ll need a more in-depth review.</li></ul><!-- /wp:list -->'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'If you\\'ve done any reading on AI/ML you\\'ve probably heard someone say that the real challenge is collecting and organizing the data.  That discussion is usually about finding good data, but I can tell you that it\\'s also a bit tricky to get data that you have access to organized enough for ML algorithms to run.  This is especially true when you\\'re learning Python and Panda for the first time.  Since this is just a learning experience for me, I cut myself off at about 10 hours of data gathering and sorting.The big decision I had to make before creating the data is what should I make the \"target\" value.  I could have either taken a direct path and asked the model to predict whether we should make a particular bet or I could take an indirect path and ask the model to predict what the score in the game would be and then derive whether the bet would be smart.  I chose to take a direct path.  I will explain this further below, but I have some data that relates to the actual bets and not the game.  For example, I have data on how many people from my league have made a particular bet.Another problem/issue with my data was that in order to create this \"MVP\" I used only 2021 data (data for previous seasons is harder for me to obtain since I delete most of the data out of LTHOI at the end of the season).  This means that through week 16 I only have 208 data points.  The SageMaker AutoPilot requires 500 data points.  In order to solve this, I logged each game that I had three times.  While this trick will let me process the data, it will make things like which two teams are playing a little bit too predictive.As I am writing this the AI models are currently running, so I have no idea whether any of these have proven useful.  Here are the data points I\\'ve given the model, how I gathered them, and what I\\'m hoping to get from them.  When the model is done running, I should be able to add information about how actually predictive it is.  I have also posted <a rel=\"noreferrer noopener\" href=\"https://github.com/BurgherJon/AI_LTHOI_Player_Notebooks/blob/master/Gather_Data.ipynb\" target=\"_blank\">the Jupyter notebook that I used to gather the data to my git repo</a>.  In the notes below I tell you where in the code I gathered the data.  At the bottom you can see the graph that AWS provides of how each field impacted the inferences both for over bets and bets on the home team.<!-- wp:list --><ul><li>The teams that are playing in the game came with my base data.  When you watch shows about gambling you\\'ll always here statistics like, \"The Steelers have never failed to cover when more than 5 point underdogs.\"  I am highly skeptical that individual teams help predict outcomes of games independent of their statistics.  However, with the duplicative data I expect this to end up being a key indicator.<ul><li>Source:  This comes with the base data about the games from mysportsfeeds.com</li><li>Section of Jupyter Playbook: 3, 4, 5</li><li>Actual Impact:  Because </li></ul></li><li>Which team is the home team.  Since I have the data in front of me, I can tell you that on average home teams win by 1.2 points this season.  I assume that will play in to the model in some way.  I could also see with the previous data point that certain home teams have a bigger advantage than others.<ul><li>Source: This comes with the base data about the games from mysportsfeeds.com.  There is also a field that contains information about whether the home team is actually playing at their home field.  For example when the NFL played a game in London.  Technically there was a \"home\" team, but the venue did not have allegiance to the home team.</li><li>Section of Jupyter Playbook: 3, 4, 5</li><li>Actual Impact:</li></ul></li><li>The line and over/under line that I used in LTHOI.com.  These are produced by the oddsmakers and are designed to make the game 50/50.  The line is in terms of the home team, for example if the home team is favored by 8 points I will have an 8, if they are 8'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'point underdogs I will have a negative 8.  The lines continue to shift over time, but in order to  make LTHOI.com less confusing, I freeze the lines at midnight the night before the game.  I doubt this will have much impact on the outcome, but I could imagine that sometimes bookmakers have tendencies that could be exploited.<ul><li>Source: This is retrieved from the database of my LTHOI game.  I used the boto3 SDK to access that database and pull the information.</li><li>Section of the Jupyter Playbook: 5</li><li>Actual Impact: </li></ul></li><li>The average points scored and points against for each team.  I calculate this by cycling through each team\\'s previous games and adding them up.  There might have been some fancy data science way to get these together by combining spreadsheets, but I\\'m still more of a developer than a data scientist!<ul><li>Source: This data was pulled from the mysportsfeeds.com statistics API.</li><li>Section of the Jupyter Playbook: 6</li><li>Actual Impact:</li></ul></li><li>The number of people in my league who made each type of bet (over, under, home team, away team).  I am thinking there may be something interesting here in the wisdom of crowds.  Also, if there is news or injuries that the model doesn\\'t capture this will capture part of it.<ul><li>Source: This data is available from the LTHOI table on bets.  Unfortunately, I use a dynamodb and a very flat database so there\\'s a lot of expensive querying in here.  If I keep using this AI model, I may have to add an index that will allow me to query this more cheaply.</li><li>Section of the Jupyter Playbook: 7</li><li>Actual Impact: </li></ul></li><li>The final line for the game at kickoff.  Since LTHOI.com freezes the line at midnight before the game starts, there are sometimes factors that cause the line to move significantly (a player is injured or sentiment shifts).  Some of the people in my league like to focus on this and others like to ignore it.  We\\'ll let the artificial intelligence decide whether it is important.<ul><li>Source: This data is available from the ODDS feed of mysportsfeeds.</li><li>Section of the Jupyter Playbook: 8</li></ul></li></ul><!-- /wp:list --><!-- wp:gallery {\"ids\":[144,145],\"linkTo\":\"none\"} --><figure class=\"wp-block-gallery columns-2 is-cropped\"><ul class=\"blocks-gallery-grid\"><li class=\"blocks-gallery-item\"><figure><img src=\"http://jonathancavell.com/wp-content/uploads/2021/12/over_bet_predictors.png\" alt=\"\" data-id=\"144\" data-full-url=\"http://jonathancavell.com/wp-content/uploads/2021/12/over_bet_predictors.png\" data-link=\"http://jonathancavell.com/?attachment_id=144\" class=\"wp-image-144\"/><figcaption class=\"blocks-gallery-item__caption\">How each feature influenced a bet on the Over.</figcaption></figure></li><li class=\"blocks-gallery-item\"><figure><img src=\"http://jonathancavell.com/wp-content/uploads/2021/12/home_bet_predictors.png\" alt=\"\" data-id=\"145\" data-full-url=\"http://jonathancavell.com/wp-content/uploads/2021/12/home_bet_predictors.png\" data-link=\"http://jonathancavell.com/?attachment_id=145\" class=\"wp-image-145\"/><figcaption class=\"blocks-gallery-item__caption\">How each feature influenced a bet on the Home Team against the spread.</figcaption></figure></li></ul></figure><!-- /wp:gallery -->After creating this data, I used a separate Jupyter notebook to create the actual training data.   It\\'s not as exciting as choosing which data to use, but you can find it on <a rel=\"noreferrer noopener\" href=\"https://github.com/BurgherJon/AI_LTHOI_Player_Notebooks/blob/master/Clean_Data.ipynb\" target=\"_blank\">my github here</a>.  I decided to make the AI have four separate models that will make a binary choice on each bet.  My intention is then to interpret the results and only place a bet if the models agree.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "\"I recently took the job within Kyndryl of helping to establish and expand our Application and Data Consulting Practice.  Kyndryl is known as an infrastructure company, there's no way around that and I got lots of questions about why anyone would want to run the software portion of a company so focused on infrastructure.  The truth is, for the first time in decades, the developers need the help!In considering whether to take the job, I recalled a moment a few years ago while I was still working for a big bank with thousands of developers.  I had made the switch from working on the software development side of the house to running the implementation of this bank's Kubernetes clusters.  I distinctly remember a morning when a young engineer and his manager came in to my office with a question about a support ticket they had received.  One of the application teams had entered a ticket for our monitoring team to install a performance monitoring agent on a particular container.  They gave the container's full name and included approval from their management team for us to log in to the container for the install.The application team had the ability to modify their own docker file to install the agent.  Further, the fact that the application team wanted to software installed meant they had missed that containers were immutable by design and the value of having their container image and dockerfile stored and versioned.  I realized then that the tables had turned.  For the first time since punch cards gave way to Cobol and Assembler, the infrastructure teams were not holding back the development teams.Of course it's not universally true.  The most advanced dev teams at most companies are still constantly challenging the infrastructure and security teams (even the cloud providers themselves) to provide more tools and technologies faster.  However, there are a lot of software development teams that are not ready to make use of the advanced infrastructure that's available to them.  There are several reasons for this:<!-- wp:list --><ul><li>The teams aren't trained on and don't understand how to use infrastructure as code, horizontal scaling, asynchronous communication, and so many other things that are required for them to unlock the power of the infrastructure they've been given.</li><li>They are working with workloads that are stuck on legacy infrastructure like the mainframe.</li><li>The code they are working with is too ingrained in legacy development models.</li><li>The data is too disorganized and not secure enough to move to the cloud.</li><li>ERP or COTS workloads won't allow them to leverage more advanced infrastructure.</li></ul><!-- /wp:list -->My new team in Kyndryl is focused on helping your development team overcome these challenges and unlock the value of the infrastructure now available to you.\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'For an organization with more than a handful of development teams, the hardest decision in technology right now is to decide where the cutoff between your platform teams and your software teams should be.  As with most things in the world today there are loud people making loud claims on both side of the debate but the real answer is somewhere in the middle.In one camp there are those that are screaming \"developers just want to code\".  They recognize that every member of the product team is expensive and they don\\'t want them spending hours selecting and troubleshooting infrastructure that they aren\\'t expert in.  They also recognize the efficiencies possible if the infrastructure team creates standards.In the other camp there is a really solid argument for self-sufficient product teams.  We have all seen software product teams that know more about what infrastructure they need than the infra team trying to work with them.  This is how shadow IT starts.  Creating true DevSecOps teams that are responsible for everything their app needs also allows the organization to more easily invest in (or divest of) individual product teams.  How much will I sound like a consultant trying to make a few bucks if I say, \"the answer is somewhere in the middle and really needs to be determined on a case by case basis\"?  Let me try to reward you for reading this far by breaking down a few of the things worth considering as you make this decision:<!-- wp:list --><ul><li>Are the products that your software teams are creating infrastructure dependent (e.g. low latency, require GPUs, edge, etc...).   In this case, lean toward creating product teams that build their own infrastructure.  Avoid the temptation to create single workload platforms.</li><li>Public cloud (and a willingness to commit to one cloud instead of attempting to maintain the ability to deploy any workload to any cloud) is a short cut in this debate.  They allow you to build platforms that product teams can leverage simply through IaC and GUIs that would be cost prohibitive to create on premise.</li><li>Consider the use of \"paved roads\".  Make it really easy for app teams to \"just code\" without taking away the ability to customize the infrastructure if required.</li><li>If you\\'re going to try to change your organization\\'s focus from one side of the continuum to the other, do NOT underestimate the cultural inertia.</li><li>The absolute worst thing you can do is to not make a decision on this.  You\\'ll end up with platforms that are over specialized and dev teams that don\\'t know how to use them.  You must pick where you want to land on this continuum and make sure both your dev and infra teams are funded and motivated accordingly.</li></ul><!-- /wp:list -->'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "\"I actually finished the Unicorn Project a couple weeks ago and didn't take great notes on it, so this won't be as extensive of a review as I often provide.  I did enjoy the book, and like it's predecessor (the Pheonix Project) it does a great job of showing how some of the best processes in the industry can come to life in a specific company.  It does a beautiful job covering both the DevOps revolution at the team level with containerized dev environments, versioned APIs, and everything else required to decouple deploys, automate test, and implement CI/CD.  It also touches on the transformations necessary to abstract your mainframe/legacy environment and the evolution of management thinking around experimenting with parts of your org while gaining efficiencies in others.My only real critique is that it's not very realistic.  It demonstrates how powerful these concepts CAN be by taking an ideal company.  The ficticious company in the book has a CIO who understands the need to empower engineers and keep the vision high-level (rather than command and control), a great agile coach, an operations team that wants to put in the extra work required to give control to the developers, and a development team that is excited about experimentation... not something I often see in clients.  It also glosses over things that I've seen very difficult.  At one point it takes a development team over three weeks of hard work to put together an automated test suite for their application that includes (presumably) either mocks or test instances of upstream and downstream systems.  They're right to call this hard work, but I've rarely seen developers tackle it in a few weeks, even on a backend system without a UI.The best use of this book is to give it to a CIO or CTO who struggles with what success looks like in their organization.  If you still have someone who's trying to count server consolidation ratios or incident tickets, give them this book.\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'This book didn’t change my life like how discovering an agile methodology for software development or reading <a href=\"https://jonathancavell.com/book-review-mindset-by-carol-dweck/\">Mindset</a> did, but it is a great book if you need a subtle reminder on how to build a good company or a good team. Essentially, Sinek applies the concept of a “growth mindset” to companies in a way that’s easy to understand and very motivating.Sinek asks readers to recognize that business is not a “finite game” like a game of football where there is a prescribed end and a declared winner. It is an “infinite game” where you can’t “win”, you can just hope to leave the company you worked for better than it was when you started.Sinek creates five steps for building a company or team ready for the infinite game. I’m going to list them all here, but I really think that 1 and 4 are the core. Of course, Sinek had to write a whole book, and I only have to write this blog post:<!-- wp:list {\"ordered\":true,\"type\":\"1\"} --><ol type=\"1\"><!-- wp:list-item --><li>Advance a Just Cause – For you and your company to function in an infinite game, you must have a just cause</em> worthy of an infinite pursuit, otherwise, you will revert to measuring yourself against quarterly outputs. Sinek makes a good observation that America succeeds because the founding fathers sought to establish a country that would seek to protect “inalienable rights” rather than one that sought to lower the taxes that Britain had levied on the colonies, and that is why we are still in the infinite game.</li><!-- /wp:list-item --><!-- wp:list-item --><li>Build Trusting Teams – Sinek argues that you need to build trust amongst your teams. Individuals need to believe that they are being looked after beyond finite results.</li><!-- /wp:list-item --><!-- wp:list-item --><li>Worthy Rival – The argument is that we need to understand what our competition is doing. Not to identify weaknesses and beat them in finite ways, but to understand our strengths and weaknesses and refine our just cause</em>.</li><!-- /wp:list-item --><!-- wp:list-item --><li>Existential Flexibility – We need to understand that the way that we apply our just cause</em> today may not be the best or only way to apply it tomorrow. When we see opportunities to improve on how we perform our just cause</em>, we must be ready to move to it. Sinek provides the example of Walt Disney realizing that he could also use Theme Parks in addition to movies to create a joyous departure from reality.</li><!-- /wp:list-item --><!-- wp:list-item --><li>Courage to Lead – The argument here is that you &nbsp;must have the courage to make tough decisions consistent with your just cause</em> even if those trying to apply finite game measurements are resistant.</li><!-- /wp:list-item --></ol><!-- /wp:list -->Overall, I think the book is worth a quick read especially if you have recently taken over a new team or are considering a pivot.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'Ten years ago, everyone was talking about industries getting “Ubered.” The expectation was that every company would be a software company in a very short order. The question at the time was, “Could you become a software company faster than a software company becomes you?”\\xa0 Ten years later, we’re not all software companies; the big banks are still the big banks (despite a couple of little digital banks), Ford still makes cars, and Tesla still looks like a car manufacturer more than a software company. Do you run or work in IT for a company that never became a software company? The good news is you’re still a company, and I think the risk of getting “Ubered” is low. The bad news is that you still need a Digital Transformation.Over the last five years, I’ve seen that successful Digital Transformations vary across industries and companies. The key to determining what will make you successful is knowing what differentiates you from your competitors and then figuring out how technology can enhance your USP (unique selling proposition). My observation is that when companies do this introspection and plan a Digital Transformation based on what works for them, they fit into three patterns (see the infographic above):<!-- wp:list {\"ordered\":true,\"type\":\"1\"} --><ol type=\"1\"><!-- wp:list-item --><li>They convert to a software company.</li><!-- /wp:list-item --><!-- wp:list-item --><li>They Infuse Innovation</em> across the company.</li><!-- /wp:list-item --><!-- wp:list-item --><li>They optimize the cost of technology by managing it as a cost center.</li><!-- /wp:list-item --></ol><!-- /wp:list -->As a technology consultant, I want to believe that everyone’s Digital Transformation will be a game changer, but as an automobile enthusiast, I know better. A great example of a company whose digital transformation would likely only provide incremental value is Ferrari. They famously make their cars by hand, so improvements to the manufacturing process are likely to be small. Ferrari owners don’t expect to interact with their car or the dealership over an app. They will benefit from a digital transformation, but it will be more cost-saving and efficient than game-changing tech. If they don’t already, they’ll want to implement an off-the-shelf ERP system to help them efficiently manage their supply chain, inventory, and distribution. I’m sure there are some -intensive parts of the car and the race teams, but those will likely be point solutions. Overall, for their core business, IT will feel more like a support function (HR, Finance, etc…) that needs to be optimized than a differentiator, even after their digital transformation.&nbsp; &nbsp;It is the companies in the middle that are most interesting. These companies do not become software companies, but they infuse technology into every portion of their business to create a more differentiated product. Take, for example, a bank like Capital One. As a bank with branches, tellers, and loan officers, they cannot simply “become a software company.”\\xa0 While I have not worked closely with Capital One, a quick perusal of their website shows that they have talented software development teams working on the <a href=\"https://www.capitalone.com/tech/cloud/capital-one-mobile-auto-failover-machine-learning-model-for-resiliency/\">mobile experience</a>, <a href=\"https://www.capitalone.com/tech/cloud/using-aws-to-enable-phygital-at-scale/\">software for the branches</a>, and <a href=\"https://www.capitalone.com/tech/cloud/transforming-credit-card-security-with-google/\">credit cards</a>. My guess is that talented software teams can be found in every corner of Capital One’s business, but unlike AirBnB or Uber, Capital One (and almost any bank) can’t just be software. A large part of their differentiation will be in how friendly their tellers are, what types of financial products and companies they invest in, and so on. A company like Capital One should make the goal of their digital transformation to become'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'“Innovation Infused.” Companies like this should use their Digital Transformation to accomplish three key tasks:<!-- wp:list {\"ordered\":true,\"type\":\"1\"} --><ol type=\"1\"><!-- wp:list-item --><li>Invest in infrastructure and development platforms that the different technology teams throughout the company can leverage to create this innovation efficiently. (I posted an article about <a href=\"https://jonathancavell.com/summit-talk-part-iii-fit-for-purpose-platforms/\">infrastructure platforms here</a> based on a talk I did at the AWS Summit, and there are a lot of great articles on creating Internal Developer Platforms, like <a href=\"https://internaldeveloperplatform.org/\">this one</a>).<!-- wp:list {\"ordered\":true} --><ol><!-- wp:list-item --><li>Note that this should include rolling out low-code and no-code platforms as well as best-of-breed SaaS solutions that can be configured for differentiation. Companies are increasingly finding ways to stand out with software, even if they don’t code it all.</li><!-- /wp:list-item --></ol><!-- /wp:list --></li><!-- /wp:list-item --><!-- wp:list-item --><li>Build data platforms and AI tools that allow Data and AI to be effectively integrated into many of these separate development teams. Too many companies have all of their data engineers and data scientists in one team that is separated from the real business problems. Similar to the infrastructure and development platforms I spoke about above, your data needs to exist in a platform where APIs, queries, and standard models for ingestion/governance are available to engineers throughout the organization.</li><!-- /wp:list-item --><!-- wp:list-item --><li>Change the way you think about projects! Build out teams that know their customers and focus on products. These teams can be spread across the company, but to maintain a strategic vision at the company level, it is important that there is some consistency (Usually formed from an Agile CoE or Agile PMO). It is also critical that companies change their budgeting and planning mechanisms to support this product transformation. For advice on how to be successful on this item, I highly recommend the book <a href=\"https://www.amazon.com/gp/product/1449368425/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&amp;psc=1\">Lean Enterprise</a>.</li><!-- /wp:list-item --></ol><!-- /wp:list -->Over the last couple of years, my team and I have carved out significant expertise in this middle category of companies. While the threat of getting “Ubered” never materialized in most industries, the need to “Infuse Innovation” via software development will become paramount over the next decade. I’ve worked personally with a leading airline, a leading electricity company, and several financial companies that understand this and are making tremendous progress. Helping people and companies infuse innovation</em> into what makes them great has become my overall goal.'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get everything I can out of the posts and pages on JonathanCavell.com.\n",
    "items = collection.getElementsByTagName(\"item\")\n",
    "posts = []\n",
    "\n",
    "for item in items:\n",
    "    if (item.getElementsByTagName('wp:post_type')[0].childNodes[0].data == \"page\"):\n",
    "        if (len(item.getElementsByTagName('content:encoded')[0].childNodes) > 0):\n",
    "            text = ''\n",
    "            text = text + item.getElementsByTagName('content:encoded')[0].childNodes[0].data\n",
    "            text = text.replace('<!-- wp:paragraph -->', '')\n",
    "            text = text.replace('\\n', '')\n",
    "            text = text.replace('<p>', '')\n",
    "            text = text.replace('<em>', '')\n",
    "            text = text.replace('</p>', '')\n",
    "            text = text.replace('<!-- /wp:paragraph -->', '')\n",
    "            text = text.replace('<h1>', '')\n",
    "            text = text.replace('</h1>', '')\n",
    "            text = text.replace('\\t', '')\n",
    "            if text.find(\"wp:group\") == -1 and text.find(\"wp:image\") == -1 and text.find(\"<style>\") == -1 and text.find(\"hello-world\") == -1 and text.find(\"<h1>Blog</h1>\") == -1:\n",
    "                texts = wrap(text, width=4000)\n",
    "                for text in texts:\n",
    "                    posts.append(text)\n",
    "        \n",
    "    if (item.getElementsByTagName('wp:post_type')[0].childNodes[0].data == \"post\"):\n",
    "        if (len(item.getElementsByTagName('content:encoded')[0].childNodes) > 0):\n",
    "            text = ''\n",
    "            text = text + item.getElementsByTagName('content:encoded')[0].childNodes[0].data\n",
    "            text = text.replace('<!-- wp:paragraph -->', '')\n",
    "            text = text.replace('\\n', '')\n",
    "            text = text.replace('<p>', '')\n",
    "            text = text.replace('<em>', '')\n",
    "            text = text.replace('</p>', '')\n",
    "            text = text.replace('<!-- /wp:paragraph -->', '')\n",
    "            text = text.replace('<h1>', '')\n",
    "            text = text.replace('</h1>', '')\n",
    "            text = text.replace('\\t', '')\n",
    "            if text.find(\"wp:group\") == -1 and text.find(\"wp:image\") == -1:\n",
    "                texts = wrap(text, width=4000)\n",
    "                for text in texts:\n",
    "                    posts.append(text)\n",
    "\n",
    "for post in posts:\n",
    "  display(post)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "training_data = []\n",
    "\n",
    "for post in posts:\n",
    "  line = {\"input\": post}\n",
    "  training_data.append(json.dumps(line))\n",
    "  training_data.append(\"\\n\")\n",
    "\n",
    "with open(\"training_data_from_blog.jsonl\", \"w\") as f:\n",
    "  f.writelines(training_data)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "\n",
    "s3 = boto3.client('s3')\n",
    "\n",
    "s3.upload_file(\n",
    "  'training_data_from_blog.jsonl',\n",
    "  'jonobot', \n",
    "  'Training_Data/training_data_from_blog.jsonl'\n",
    ")\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}