From dd7b7cbc20e9e58b0c5a94e718e4ef369454d0ee Mon Sep 17 00:00:00 2001 From: Manikantagit Date: Thu, 27 May 2021 22:19:03 +0530 Subject: [PATCH] Updated 316 317 --- transcripts/316.txt | 63 +++++++++++++++++++++++++++++++++++++++++++++ transcripts/317.txt | 62 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 125 insertions(+) create mode 100644 transcripts/316.txt create mode 100644 transcripts/317.txt diff --git a/transcripts/316.txt b/transcripts/316.txt new file mode 100644 index 00000000..d634bfa8 --- /dev/null +++ b/transcripts/316.txt @@ -0,0 +1,63 @@ + + +00:00 Flask is one of the most popular Python web frameworks and they have a huge news to share with us, Flask 2.0 just released after a ton of work. And it's as big a deal as the version number suggests. Async changes are coming Python3.5 and below including Python 2 support has been dropped and much more. Join me as I discussed Flask 2.0 with David Lord and Philip Jones. This is 'Talk Python to Me' Episode 316 recorded may 10 2021. + +00:41 Welcome to talk Python to me, a weekly podcast on Python, the language, the libraries, the ecosystem and the personalities. This is your host Michael Kennedy. Follow me on Twitter where I'm @mkennedy and keep up with the show and listen to past episodes at talk python.fm and follow the show on Twitter via @talkpython. This episode is brought to you by us over at talk Python training. If you want to learn Flask, we built a fantastic course called 'Building data driven web apps with flask and SQL Alchemy'. In this course, we build a 'PyPI.org' clone from scratch using Flask and SQL alchemy, you'll learn many of the major ingredients needed to build most web apps. If this sounds amazing. Just visit 'talkpython.fm/flask' or email us at 'sales@talk python.fm'. David Phil welcome back both of you to talk with me. Hello. Hello. Thank you. Yeah, it's great to have you back. David, I've had you on to talk about flask. And Phil, I've had you on to talk about derivatives of flask, I suppose about core your project. And they're both kind of approaching the same type of thing and that sort of zeroing in so now we have you here together, I suppose working on flask 2.0. Right. It's all merging into one not really just our efforts. + +01:55 So several libraries. Yeah. But it's really cool to see you working closely together, rather than just two disjoint things right. I think that gives a lot of credibility, especially the Quart , right? Because it was kind of this experimental thing. And now it's if not exactly is it a Pallets project officially or is it just working more closely? It's working more closely. In addition, it uses 'Werkzeug' now, which never used before. So it's been quite a lot of work in in that project to make that possible. Yeah, that's really cool. And 'Werkzeug' is like one of these foundational bits of flask. We'll get into I'm sure. Yeah. So the the idea is, because Quart is supposed to be API compatible with Flask, we want to get it using as much of what flask is using as possible. Our long term plan is to get it using the more Quarts and fixed. Yeah, yeah, super cool. Maybe some it's dangerous as well could get in there Who knows? + +02:43 It's risky, but you guys can make it happen. All right. Now before we jump into the main topic, flask 2.0. You both have been on the show before, I often ask people how they got in programming in Python or whatnot. You've you both answered that question. Previously, people can go back and hear it if they want. So how much to catch up? What do you been up to? Since you've been on the show? Last, Bill, You wanna go first? Sure. I think you're an open source sensor. It's, it's been Yeah, working a lot on Quart and working more. At the time, I didn't think I was really helping with the 'Pallets'project. So I've been helping with them since. And, yeah, just kind of developing those two, and then a lot of work trying to get well, the Async support that's now coming to flask is quite exciting. I kind of personal note. Yeah. Very exciting. Yeah. On a personal note, I work in London. So I've just been, I've actually worked for a few companies tried my own startup it didn't work, sadly. But working in London now. So it's nice. I love London. It's a fantastic town. I haven't been there in a year and a half. Oddly, I don't know. + +03:42 David, How about yourself? Where do you been up to? So I'm trying to remember when we did our podcast last, but it must have been like three years ago. It's been a while. So yeah, after that. I've just been working down the backlog of all the issues on flask, and pallets and all that. And now we're finally at the point where I'm confident about releasing all this stuff. So yeah, it's just been more and more open source work, and then still working for the same company I've been working for for 10 years now. So that hasn't changed at all. Yeah, that's cool. And I suspect they're pretty supportive of flask. Oh, yeah, we get to use it internally. I actually introduced Python and Flask to the whole company. We don't know. Yeah, I mean, how much of an advantage is that to have somebody like you? Yeah, so they definitely go with it. Everything right? Yeah. When we're bidding for contracts and stuff. They definitely Yeah, like use that as like, hey, we've got one of the maintainers of the libraries we want to use. What do you bring? Exactly what + +04:38 what's the advantage you have? Well, have you met David? Now? That's fantastic. That's fantastic. And check this out. You were back on episode 177. And the title was flask goes 1.00 my gosh. Yeah, this is September 15 2018. So it's been a while, but also, it's time to have you back because now you guys have incremented the big number in the version + +05:00 Well, not quite almost one more day. + +05:03 One more day. Yeah, smart. Okay, so for people listening on your podcast player, that's like, a week ago. Oh, + +05:11 One more day. Fantastic. Super cool. So I see some really interesting questions from people who are watching the live stream. But let's dive into a little bit of background stuff. And then I think we'll go from there. So I kind of wanted to start with the larger landscape of the web. And maybe also you could tell us about, I just threw that out there as if people would know, the pallets project. We talked about it a little bit with both of you, David, give us an overview of what the pallets project is because we have flask, and we have Quart . And then we have these other libraries as well. What's the relationship of all these things? I know Quart is its own thing. But it uses like 'Werkzeug'. So like, for example? Yeah. Well, pallets is the organization like just an open source organization, not an actual company, or anything that maintains flask and the libraries that flask uses. And so that's flask Werkzeug, Jinja, Click, 'ItsDangerous' 'MarkupSafe'. And it was started, all these libraries used to be maintained by the original author Arman and his other group 'Poku'. And eventually, they kind of grew larger than that. And the original team kind of grew to do other things. And so he formed pallets. And that's about when I joined and started maintaining everything. And although recently, I've come to the understanding that as much as I push the name pallets, I will never have as much name recognition as flask. And so you can also refer to things as flask. And I will know what you're talking about. Yeah, no doubt, like the general flask, obviously, is made up of the foundational bits as well. Yeah, yeah. And so palates, for a very long time was just the GitHub organization that these projects, repos were under. And I've been working every time to expand that into an actual open source organization, kind of, not nowhere near this yet, but inspired by like how Django Software Foundation works, and those sorts of things where they do things besides just develop. So we've been working on you know, growing the community getting more maintainers involved. Running 'Flaskcon' last July, I think, and we're Another one is coming up. I think November is this tentative schedule. And this is obviously I will guess, online at this point. Yeah, it'll be online. So it was online last time also, and it worked out pretty well. You can go find all the videos on 'Py video'. They're up there. Yeah. Fantastic. Do you have Sprint's or anything like that around it? Or is it just presentations? Oh, we didn't do Sprint's last time, because it was just a first time running a conference. We'll see what happens this year. Yeah. No concrete plans yet. Yeah. Fantastic. All right, I kind of wanted to start off this conversation with a wider view, looking at the Python web space. So I put up here for us to look at the Python developer survey from the PSF and JetBrains. For 2020. Obviously, it's 2021. But the results for that was the last survey that they've come out. And over there, we've got the web frameworks being used. And you know, these things are never comprehensive. Like there's 6% of other whatever that means. But we have flask, now clearly being the most popular one. So first of all, congratulations on that. That's awesome. I wonder if 'Quart' was folded into flask, or if it went into the other, you know, innovations mind when they clicked the button, which they click, you know, I imagine is probably in the other hand, a small percentage. Very small. Yeah. Nice. Okay, well, so this is really neat. I guess the big interesting thing here as well, the beyond, I think it was you in Django, at least in this previous survey, 2019. We're just tied, basically. So now it's you talked about how there's a lot of growth and a lot of momentum. And I think that's even more than what you see here on the screen. Like I think a lot of the API's that Flask embraces in the way of doing web programming has been adopted by many of the other new frameworks as well. So for example, we have Fast API making a jump up to 12%. Here, which is probably the biggest difference from the year before, as well. What do you think about this picture here? And then like your What do you think about just the Python web space in 2021 . Fast API is definitely gaining ground. Flask I'm continually surprised that so many people use flask never, it's hard to wrap your head around like that many people, I was just looking at the downloads this morning, and it was 14 million in the last 30 days in the last 30 days. That's so insane. That's just flask like if you look at like ginger, because ginger is used by things besides flask like Ansible, right? We're like the 20. Somethings very cool. And you can use Ginger just on its own for super interesting things as well. Like, I recently did a project where I needed to generate a PDF that was like out of you know, here's data in a dictionary, and I need to make a PDF that looks formatted. And I just use Ginger and a Ginger template to generate the HTML, then feed it off to a PDF library that made it into a PDF, right. There's all sorts of little edge cases like that. And same with click I mean, people use click to make actual command line applications not just to provide a little CLI for flask so yeah, when you look at this, like you like you realize that like oh, they're also using like all these other libraries, but + +10:00 I never really like think about the numbers, I guess every now and then I remember flask having more stars than Django on GitHub. But that's because Django joined GitHub late. And they surpassed us now. But that's about the only joke metric I follow. Really? + +10:13 Yeah. But yeah, I've seen a lot of interest in fast API. And I think it's a, it's nice to just see that, like, none of nothing stagnating, you know, like everything. Like there's, there's new ideas out there, people are developed, like, they're stable libraries that continue to develop like us. And we were still adding new features. And there's like completely new ideas, or new formulations of libraries. Yeah, I feel like we're in a bit of a Cambrian sort of 1000 flowers blooming type of time, and many of them won't survive that. But there's a lot of fresh ideas out there. And it's pretty exciting. Yeah, it's like having a healthy ecosystem having choice that you can choose whether a certain framework is more appropriate for, you know, the project you're starting or working on, I think is really powerful for like users and for maintainers. Yeah. And you can borrow ideas to like, Oh, I see the way fast API is doing types, like mean this, maybe that makes sense to like, allow that as an option. somewhere else. Yeah, who knows? Right. But one thing that I quite like to take from this list is Falcon has a really quite exciting router. And I think it might be a little too, too much for Werkzeug, perhaps, but I've been playing around with taking the ideas for Falcon and bringing it to 'Werkzeug'. So what makes it exciting is it's very fast matching system. And yeah, should outperform the effects of 'Werkzeug' get it in? By Okay, like, a good fraction. So that'd be exciting. Yeah, there's a lot of interplay here that I think is pretty neat. And why we're on this sort of broad topic. Rohit asks, in the live stream, you know, what are the advantages of using flask over Django? Maybe that pairs them a little bit too tight. But you know, looking at this list, these are the ones that probably the trade offs people are thinking about. Yeah, this gets harder for me to answer over time, because I haven't used Django in a long time. But it's hard to compare one to one. It's like apples and oranges, right? Like I said, a couple minutes ago, like you're choosing the project that's most appropriate, or the the framework that's more appropriate for your project and the way you want to work. So it's not necessarily that you're going to be losing out or getting any given feature by choosing one or the other. It's about how they feel when you're working on them. You know, like, sometimes Django makes more sense to people or fast API makes more sense. Or Yeah. And then like, they also have different goals, right? Like flask is deliberately just the web framework and wrapping some other libraries around it. But it's cool is that if you want other features, somebody develops an extension that is specific to that. And you add that on. And I mean, Django and I don't know about fast API, but Django has the same idea. It's just how many batteries they decide to include is different. Yeah. Phil, you want to take a shot of this before I give my thoughts as well, maybe? Yeah, I think specifically flask, and Django is full, it came down to a kind of a choice about how to do things like flask doesn't really give you an answer. And you can go and choose from the extensions, whereas Django does. And yeah, I think it's, like David said, a bit of what your preference is really what you prefer. Yeah. I mean, Django is making huge strides. And like async, also, and we're adapting to that, too. But they all have different solutions to the same thing. approaching it from different directions. Yeah, definitely. There's a lot of interesting stuff happening in Django around this as well. The way I see it is, you know, flask, it's like an empty canvas. And it's got just a few really nice little building blocks. And then you can build what you want, right? You wanna use Postgres and SQL alchemy. Great. That's totally easy. You wanna use MongoDB and Mongo engine, equally easy, go have fun, right? Like you, you can pick just the little bits that you really like, you don't have any other bits to worry about. And then you're good to go, right? Django, like, well, if you want to break from the Django ORM, probably you can do it. But it's, there's a little bit of a mismatch there, you kind of got to work around it, its advantages kind of become less good, right? Like, it's Admin Tools might not work as well, if you try to say switch to Mongo, or something along those lines. And if what you want is like, I'm not really sure what I'm doing to build this website, and I just want some guidance, like, Just show me the main way, and I'll just do that I don't, I just would like to go along for the flow. I think Django is fantastic for that it gives you like, really good pre built tools, like the admin stuff, and whatnot. But if you really care very much about which little piece you pick, and you want to put them together, just so I feel like you're better off with a microframework or some sort of like flask. That's my, my view. And I know some people love one side and hate the other and vice versa. So it's, you know, it's also a personal challenge or a personal thing you got to work out for where do you fit on that spectrum? Yeah, I mean, I personally started with Django flask was just being started when I was starting to learn Python and web frameworks. And then I eventually ran into something where I wanted to do things that Django wasn't designed to do, obviously, yeah. And I started running into a bunch of stuff. And so I switched to flask. Yeah, that's cool. I know if you come from an ecosystem, where there's like, clearly one choice for a lot of things. You know, if you come from Ruby, you got your Rails. + +15:00 And the Ruby ORM, from Microsoft, you've got ASP. NET Entity Framework, like you don't have decisions, you just do the thing. And then eventually you build your app, right? You come to Python, there's like, you know, 20 different choices at each level. And you're like, this is like the Paradox of Choice this, even though it's in some ways, it's great in other ways, like, I have no idea what to do. And so I think that can be a challenge for people as well. Yeah, but also an advantage. Once you get into it, I think your best but if you're asking the question, should I use Django or flask is to just pick one and learn enough? You know about that framework? And I think you will be better equipped to answer that question for your project. Next time you have to ask. Yeah, it doesn't really matter what you start with. Alright, before we get into more specific, so sort of a general flask question as well. Project Yvonne says since I started April Fool's joke does the core team. David and crew have any traditions around April Fool's. I thought about that. Because you know, every time somebody brings up that it was made April Fool's. I remember that it was made I overhauls. I always forget that. So I'm guessing not not to do any. Do anyone's okay. Don't have like a birthday party or anything or celebration. Maybe we should bring it up to the community team. There you go. There you go. I think you put out a blog for last year 10 years, didn't you? That was quite a good celebration. But I think that was about it. Oh, maybe it? Yeah, it's been 10 years for a lot of libraries at this point. Yeah. Very cool. So let's talk flask 2.0, you know, it's been a couple years it went to one Oh, it was on the zero over train, like so many projects were before then. Right? That doesn't really mean that it was such a huge transition. But there's a lot of stuff changing. Not necessarily in a breaking way. But a lot of new things. And flask to give us the exciting news. Well, the biggest one is probably Async. I love Phil talks about that. + +16:47 Yeah, there's like a async for flask directly. And then like maybe tighter integration with Quart type of next level async, right. Yeah, exactly. So with flask 2, you'll be able to write async route handlers, depending on what you like to call them before and after request handlers and teardown function. So it won't require any special effort other than installing flask with the extra support, but then you can just write async and await and carry on with your current setup. You don't have to change anything else. Yeah, that's super neat. And you actually gave a talk may 1 or something 10 days ago, what was it called? It was why isn't flask async? And yes, obviously, that's like leading into where things are going. You want to tell people a little bit about some of the ideas there. You talked about the history, obviously, flask being a whiskey or wsgi application. Those are all naturally having a hard time moving to async. Because, right, the server itself needs to know. But then you also talked about some of the things that are happening to allow some async stuff to run directly without any server changes. And then also moving to an ASGI server. Yeah. So what's the story there? So yeah, so obviously, flask started as a WSGI framework. I think it was probably WSGI from the start. Right. So yeah, I know. Yeah. I can't quite remember what whiskey is introduced. So yeah. And obviously, before async await is introduced, we have all these different servers. And I don't want to have just this framework run on that server. So WSGI born, right. But born in a world where async was less common. Yes, but what it has supported the whole time. And I think what's good to point out is flask has been async, the entire time just using green nets. And a lot of people use that very effectively, I think it's absolutely fine. And that, for us, I think was one of the key things to bear in mind that plus works Async in , if we introduce async await, we can't be breaking that at the same time. So that was one of the big constraints really. And then. So for the one side is keeping support with these existing async solutions. And the other is introducing the async await keywords which I think everyone has heard of the color problem and the virality associated with that, which makes it actually really hard to get a little bit deep into the call stack, if you will async without really having to change the whole call stack to async in one swoop. The other challenge, maybe I'll try to give a little bit of background just for people listening is if I've got some low level data access library or service access library, like h using HTTP x or something, in order to await the thing at the bottom, that has to be an async method, the next level up in order to call that method, it's got to be async. Then you got to await it to its methods got to be async. And it just it goes up the entire call stack in a lot of cases, right? Yeah. So what it came down to and what we what we did was follow Django what Django have already done really. And what they they've figured out is that you can run the async event loop in one thread than the greenlit based one or whatever you're doing previously in the main thread, and then everything coexists nicely you just need these wrappers to to move things from one thread to another. And that library if you're interested is called ASCII Ref, but using that allows us to have this spacing + +20:00 code in the View handler, keep flask synchronous in the rest and keep support with green, which is really nice. Yeah, that's really nice. And in production and a lot of the servers, it has threaded modes already that if like, you're awaiting something, and then that something talks to a network, like a database call, or a web service call or something, it's going to release the GIL anyway, and the other threads can run. So that can probably push out pretty far. Just on its own. Right. Yeah, it's I think, has been very effective to a lot of companies. I know companies that use it, I think it'd be really nice. So yeah, that's what we've we've got to and I think you were talking about, where can we go? Can we go fully async and ASCII with flask? And this is possibly more of my personal opinion, I'm not sure it is possible to go there with flask. Because I think if you do, say your end up breaking extension support, because it's quite common for extensions to extend the flask class can change the methods, but of course, they won't be async. So how are they going to deal with that? Yeah, there's the compatibility problem. And then there's also just the problem of having mark, everything is async. And await if you mark one thing, and we might discover I'm a little more optimistic than Phil, but I'm also not as experienced with it. But I would like it to be possible, like at some point for us, you know, we're focusing on like making a 'Werkzeug' stands IO. So it doesn't require blocking IO operations, when you make requests and stuff so that you can use it in a ASGI situations. And that's gonna be a slow process. We're not like 100% done with it or a thing like that. But my hope is that as we make 'Werkzeug' suite completely, like IO, agnostic, we might discover some ways to adopt flask. Yeah, but it's also not 100% necessary, because Quart exists also. + +21:46 That's an interesting point, maybe there's a way to implement some of those extension, the SU The problem is the extensions override certain functions that are part of the lifecycle of requests. And those are expected to be synchronous functions, because they had been for all of time, you can't just make those async and have those extensions still work, you know, at best are going to be kicking out unstarted co routines. You know, maybe it's probably worse than that. But maybe, maybe that's what they do, I don't know something is not going to go super well. But maybe you can create synchronous wrappers around the async stuff that can grabber an event loop and run it or some I don't know, like some there could be some sort of an adaptation thing where just the sync version, maybe it's not quite as fast or something I'm not sure. As much experience you guys have, I have way less than that. Just as a caveat, I was gonna say, I think we fast too. I think this is, in my opinion, where you want to be because the motivation in my mind is you want to use some async based library. And there's there's some exciting ones out there. So I can fully understand that. And you can now in flask and it will work will work with your existing setups. I think the only reason you could want to go fully async is because you only use an async libraries perhaps or perhaps you might see a bit of performance improvement there, which isn't necessarily guaranteed, by the way. For that, though, I think this Quart, so I kind of think all the users motivations might be covered now, which I think is really exciting. That is very exciting. Yeah. And our goal is just to get Quart and flask as API compatible as possible. And they're already very, very close. But you know, moving 'Werkzeug' that and so like, ideally, it should just be a drop in like name replace, for everything to keep. Yeah, and you could probably pull it off with just the right import statement, import court as flask or something potentially almost impossibly? + +23:38 No, this isn't some monkey patching in court, where you can tell everything else that caught his flask. So you can use some flask extensions we've caught by allowing Quart to visit this flask is is kind of kind of fun. Yeah, that is quite quite wild. I did see a question. Will flask break current extensions? No, our goal was still compatibility and stability. We are making major releases, and we are adding new features, but it shouldn't break shouldn't permanently break anything. You know, there might be some unforeseen in compatibility that a library needs to update for, but it's not like they will suddenly have to rewrite their whole thing. async or something like there's not if it breaks it, it's not on purpose. + +24:18 To sound bad, too, but no, no, I mean, like your intention is to have it be compatible, right? Yeah, yeah. That's always the intention of these libraries. They're meant to be very stable. And sectionally, one of the things I've been working on as a maintainer, as I've been maintaining these over the years, is when I do make releases, or when I make changes, I try to make everything show deprecation warning first, and then only remove it in the version after that. So I do want to evolve things and refactor things or, you know, identify patterns that more great or that don't, aren't needed anymore. Since we're dropping Python 2 do you want to evolve the libraries, but I don't want to break anybody without warning them first. And so + +25:00 There's a ton of deprecation warnings in all the libraries this time that you'll see if you're running tests. It's cool. So speaking that Yeah, I saw this really nice flask 2.0 is coming. Please help us test by statio. I don't know. If you know, maybe give him a shout out. Oh, it's me. Yeah. Is that your name? Perfect. All right. Yeah. So from you, Phil. And he said, from the change log, one of the things coming? Well, first of all, one of the things coming is Python 3+, sorry, 3.6+ only dropping two for sure. And even 3.5. That's pretty interesting. Well, 3.5 is end of life now. Yeah, it seems so new in my mind. But it's clearly clearly not that new anymore. When I first made the announcement of dropping Python 2 . 3.5 I wasn't going to go end of life until like nine months later, but I decided to I was going to drop 3.5 anyway, we won't usually, usually we're going to our plan is to try to wait for the actual official end of life's for pythons for no one. But 3.5 just, it was so much more convenient to drop it at the same time. And 3.6 is still supported. But the next versions after these will be 3.7 only because 3.6 will have gone end of life, probably by the time making your releases. Right, right. Okay, very cool. And how much the 3.5 vs 3.6? Was it just that 3.5 is going out or versus f strings? We do use f strings all over the place now. But that wasn't the reason. I mean, since we were doing 3.6+, I decided to like use f strings and all the places I don't remember. What was inconvenient about 3.5, but there I mean, I do remember there was something this was a year and a half ago. Now when I made this decision, I think async was slightly different. I mean, look, if you're switching to two, it's a chance for a nice break from some of these things either. Exactly left, yeah, I was originally like, originally 2.0 was just going to be like dropping Python 2, and making maybe a few minor changes. And then I got anxiety about making releases and breaking stuff. And so I just kept working on more and more stuff into 2.0 instead of releasing it. But because it's 2.0 that also kind of gave me the opportunity to like, identify more stuff. So I wanted to deprecate the normal, maybe or more bigger features. Yeah, but it's a perfect time to do it with such a big version number change. I want to pull out a few more things from now that I know it's Phil, Bill's Reddit post here, which got 1.3 1300 uploads. That's pretty awesome. Short format decorators. So instead of for routes or routes for you in London, instead of 'app.route', you know, specifying GET or POST, you can just do 'app.get' , 'app.post'. That's easy, but nice. Yeah, it's just a bit of syntactic sugar. Really, it's, yeah, should make it look nicer. It's going back to the ecosystem. It's something the other frameworks are seem to adopt. And it's nice to see Yeah, yeah, that's one of the big differences I've been seeing from other frameworks that they're very much like flask, they kind of put the verbs on the decorators a little bit more, so why not take it back, if you inspired them in the first place. Also, you know, blueprints, and blueprints are something I think, do not get enough attention, and love because they are so nice for organizing your code into little bits and little categories of functionality. Instead of, you know, trying to associate them all with the main app, the app. So there's some news around that for updates as well. Right? Yeah, one of the oldest standing open issues, Phil decided to just like, Oh, I can solve this. And he's like, wrote something over a weekend. I think that way to go, Phil, now it's blueprints all the way down. I'll let Phil describe it. + +28:36 Yeah, it's exactly that you can nest blueprints in blueprints. So I think they kind of co-structure is really helpful, because now it doesn't have to be all in that one group plan, right? You can split over a few files and bring them in, which helps a lot. And there's other structures where maybe you want to reuse some part of something in another blueprint. So you just bring in parts. But yeah, you can just use that. And you could use the command register blueprint on a blueprint to register the blueprint. And yeah, used to have to say 'app.registerblueprint'. And they'll give you the blueprint, and that would help you build up the various pieces. But now you can. You can nest them. And they can be more hierarchical, which is great. Right? Before it was just one layer, you have the flask app, and then the blueprints. And so like each blueprint can have a prefix, but people haven't actually asked for it for a while. But it's still been an open issue. It doesn't seem unreasonable. People have wanted to, like have prefixes for prefixes. And so then you get nested blueprints, and now it's possible. Yeah. Awesome. Let me take a couple of the questions out of the first Doug Pharaoh out there. Hey, Doug, says, Hey, I'm working on a book that uses Python flask as a primary example application. When might flask 2.0 be released? And should I plan on updating the book to use it? I'm guessing Doug didn't catch the very first opening bit because you said tomorrow Tomorrow, if you're interested in following pallets news in general, there's '@palletsteam' on Twitter, and there's also 'palletsprojects.com/blog', which has an RSS feed and so + +30:00 They actually have announced this already. But yeah, it's may 11. Tomorrow. Fantastic. And then john Shanahan says you right, we don't hear enough about blueprints, love blueprints? Yes, absolutely. right oh one says, in our talk Python training web training course, which is a flask covers, it's as fast basically rebuild PyPI org with flask is used as safe to use flask 2.0. I think so like David said, There shouldn't be any breaking changes, maybe deprecations. If you see them, let me know, I can update that. But I think it should be fine, right? Same reason that production apps work for, like example option where this is the first time we've tried to do release candidates before we've made releases. And so we actually have had release candidates up for about a month now for all the libraries where you could test your existing code with the new versions. But I think in the future, when we do this, we need to do a better job at advertising that so something will get better over time. But yeah, cool. Well, people listening now now, that also, I think, interestingly, related to both the microframework side of the story, but also the new features, like especially the async stuff, Rohit asks, which will be the most compatible databases supporting framework for flask, on one hand, like, just do whatever you want, right? But on the other, you've got flask SQL alchemy, as an extension, you've got the ability to use like Tortoise ORM. Because the async support, like there are new options, right? What do you think? Yeah, I mean, flask makes no requirement, it doesn't affect what database you use 'Flask SQL alchemy', that will have a 3.0 release similar to these soonish. I think I'm going to add a little break after this release first, but it should, it should continue to work. What's the as you know, SQL alchemy, 1.4, just released with the first async API, and they're moving towards 2.0 as well with a new API. What's the story around their their new API and the SQL Alchemy Async. Support? Yeah, Flask SQL Alchemy? Is there anything overlap in there? So SQL , Flask is compatible with 1.4? Right now, I did make a little bug fix release recently for that 2.0 is just deprecating things, so and nothing that flask SQL alchemy uses is going away. So it'll work with SQL alchemy. 2.0, when that happens, in terms of the async stuff, we might flask SQL alchemy, like the extension might have to do some thing if we want to enable using that pattern. But we might also not because it's using pretty clever use of 'Greenlet' to enable compatibility between sync and async code and flasks. async support should support that out of the box out of the box. But it's possible to extend it to do that. Supercool. Another thing I saw in the release notes was that around 'Werkzeug', there's performance improvements coming in Flask and 'Werkzeug', so we wanted to give people a little insight into what's coming there. Yeah, one of the things I think the one that got the most attention, got some interest on the Python subreddit was an improvement we did to the forum passing. So I think there's a this issue that's about five years old, saying it was a bit slow, especially for large files. And as David mentioned earlier, we were looking at moving that to be Sands IO. And in the work to do that, the original author of that issue actually came back and said, Oh, if you do this, you'll make it a lot quicker as well. And when we did that, he pointed out, we've made it about 10 to 15 times quicker than the previous form parser. So yeah, that's been sent files to flask server, it should be a lot quicker. Yeah, form data in general is faster. And then if you're uploading large files that's way, way faster than what it used to be. Oh, wow. Like multi part form upload type of files. Yeah. So if you're uploading even like hundreds of megabytes, it could be really slow in some cases. And so now you can just upload our big gigabytes ISOs if you want. should work fine. Just like streams and or something else will take as long as it takes to upload. Yeah, shut up onto the internet. Yeah, it doesn't make it worse. I see. Cool, but I don't know. I think I'm trying to think there were any other performance improvements in 'Werkzeug' I know a couple releases ago, we made a pretty big one were to our URL building, which made it like 10 times faster or something that's already released. So but it has been getting faster. Yeah, okay. We haven't measured it. I think the the work taking out all the compatibility shims. And just getting pure PY Free should should be noticeable. Yeah, sounds really interesting. All the compatibility code for Python two stuff, like literally the.com , 'compat.py' files are gone. And all the code that was calling that there's still some more work to be done on that in terms of like accepting both bytes and strings and a lot of places where it doesn't really make sense anymore. So there's still some overhead but there's a lot less compatibility overhead than there was before. But yeah, it hasn't been measured. But I expect that to be noticeable because when 'Werkzeug', and flask first added support for Python3, going the other direction, so it was Python2 only and then it became Python3, some people stuck with like really old version. + +35:00 of 'Werkzeug' in class, because there was a noticeable performance degradation by adding all that compatibility code. So now that should be better. Yeah, fantastic. I know the NumPy people, they actually got a lot better. They shrunk their code a bunch by dropping 2.0 or python 2.7. And similar for you. Yeah, it's just, I mean, besides the performance improvements, it's just more maintainable. Now, I pushed really hard to drop Python 2, even before it went end of life, I started thinking about how to do it, because I never used Python2 back then. And I still don't know. So every time I had to fix something, I had to like, oh, gosh, now I gotta run the Python 2 version of the tests, and figure out how to, like make some weird compatibility Shim, you know, between the two. So it's a nightmare maintenance for like maintainers. And contributors? Yeah, I was gonna say it's probably easier for contributors as well now to come in, because they don't have to know a lot of people these days, they don't know Python to write great, exactly. They don't want to come in and like learn, you know, something from 10 years ago, just so they can keep it compatible. They just want to add the new feature. Yeah, if you somehow have like a Python 3.5 application, so you can stick with flask 1.1, you, you'll still get your application working just fine. I mean, we're not going to be supporting it, because we just don't have that capacity. But the libraries are fairly stable, you know, like, we almost never get bug reports asking us to backport something, for example, you know, I use the panda bot, like pretty much anybody who uses GitHub does, at least for security, right? I think people have become a lot more aware overall of that they should be pinning dependencies and how to do that, and how to upgrade them intentionally. And so hopefully, over time that you know, less people will be surprised by updates and their applications will continue working with the versions they work with. Right, exactly. And if you're worried about these changes, just pin it to 1.11, or whatever it is, and just let it be until you're really ready. But also, you'll get notified of any security issues that you might need to jump in and fix right. So I was gonna say is I don't remember seeing any around plascon werkzeug so I almost ever so they're pretty stable. It seems it's pretty rare for there to be like critical bugs or security related bugs reported. We get stuff every now and then. But we just make a Py release. Yeah. And Rohit asked previously about like, why would I use something like flask over say something like Django? I noticed security releases for Django more often. And I think it's because they just have so much more going on. Like they have a whole admin section with forums and input. And you guys just, if it's not there, it can't have a vulnerability. And there's a much larger attack surface. Yeah, yes, exactly. But it's not that Django is less secure overall, though. I mean, first of all, they're releasing all these security updates. So they're staying secure, but I think they have a much larger team. And so they have the capacity to do these sorts of analysis that maybe we don't, I'm not saying like, I think flask is insecure. But it could be that, you know, they're just, they've just got more eyeballs on these things. Fuck, like noticing like, Oh, yeah, we shouldn't do this. We can make it safer. Or something like Yeah, yeah. And I just think the surface area is so much larger. Yeah, it's bound to happen. Maybe someday we'll be in that position for flask to + +38:14 Doug says, oh, man tried to get into Django, my new job, so much magic, too much magic happening there for me. But he also has a question that I think maybe you know, Phil, you spoke about in your presentation, like, does this release change how you run in production, like Unicorn versus micro whiskey, versus who knows what hyper corn, the good thing to say is that, no, it doesn't change at all, it still works the same. And you can still use the unicorn and use the async capabilities. Now in flask. If you wish to use an ASCII server, you can as well, you need some middleware and adapter to do it. And there's some examples in the pipe in their flask docs, if you want to see which one we recommend. So you can choose any server. But yeah, very important for me to point out that you don't have to change anything, you can just carry on as before. Yeah. So if you want, how would you categorize you, you actually, in your talk spoke about a spectrum of async capabilities. On one hand, you have flask with its decent async support, but not full, AGSI server capabilities. And then you've got core, which can be 100% ASGI. And so you kind of thought of it as a spectrum, right? Yeah. So my idea is that if your codebase is mostly sync, as you may have started, and flask is a great choice, if you want to add a bit of async here, and that's still a great choice. It's only when perhaps you want to go fully async or you've mostly async, maybe a few Sync, pieces of code that you actually would choose caught. I think they're really nice thing for the users, you can hopefully just go across the spectrum, eventually change the name, but then just continue and it is a pleasant experiences, my hope, Yeah, awesome, you should still be able to use the same WSGI servers, basically, like we've said a couple times, like, we're not breaking anything, we're not changing how flask works or anything like that. So you + +40:00 still deploy it the same way. And if you do want to use an ASGI framework for summary or server for some reason, you can always use an adapter to serve a WSGI app over ASGI to you just don't get any of the benefits Really? Yeah. Hey, Doug makes a good point out there, that flask for the wind, still making g unicorn a good solution. With g unicorn, you can change the worker type, right, you can let it be the regular one. Or you can say it's dash K, for the opposite, you said change the worker type to UV corner workers, which would switch it to be an ASGI version. So you don't even necessarily have to change anything but like party, your command line for your startup of your G unicorn. And that could be the total change you have to do. And going back to a point that Phil made earlier. In the same way you could change the worker to use like Java or something. And already have an async flask, just you know, with an older version of async code, or an older implementation. Yeah. And Joe out there says I'd look at the new async IO features, but it seems from the docs that it's early days, and maybe later versions will get more async. io adoptions. I guess that's a good direction to have a quick chat about is a what's the two? Oh, is this where you see it? landing? Or do you see this as a progression? I think it's fair to say it's a it's a progression, definitely. I mean, the ideal outcome in the end would be to somehow merge flask in Quart uses be to use any amount of async and or be ASCII or whiskey and everything just just works nicely for them. Really. Again, as I said earlier, I'm a bit skeptical about whether that could be the one namespace I just call flask, whatever you need to split it up as it is now. But yeah, I think it's the start. Hopefully, we can we can keep going. Yeah, like the async stuff we've gotten there so far is for supporting the most important thing that we thought people were needing async for, which is I have some async code, where like, I'm using discord.py, for example, to write a discord bot or something. That's all async. Great. So if you want to write that code, you need to type async def, and put your code in there. Right? So we're enabling that we've got extension points to change, you know, what async IO loop you're using, like async. io versus trio versus greenback, or UV loop or something maybe or you I'm not sure if that would tie in, I feel might be able to answer that. And then like, you know, addressing things like being able to do WebSockets. And that sort of thing. We might be able to do that in flask eventually. Or that might be the place where you switch over to the court namespace. And yeah, but you're using the same API. It's just a different name. field. That's something I wanted to ask you about it. Just let you get a give a quick shout out about is the WebSocket support that you all have in Quart? Oh, yeah, as I said, I just want to say level that it is definitely support it and it just go ahead and use it. It was supports WebSockets. You did bring some of that stuff back to fix like though, because you're using Werkzeug in Quart, you needed to be able to route to WebSocket URLs, as opposed to http URLs. And so we now have a way to identify in our very sweet router, whether something is intended to be a WebSocket are not sort of changed based on the the scheme you're using. I think it's good to point out that Miguel has a kind of experimental support, I think in flask as well, for WebSockets, which is good to keep an eye on have to see how that goes. Oh, yeah, that was really cool. I don't have the I don't remember what it was called or anything. But there's like a way that g unicorn will like let you take control of the HTTP socket from an endpoint. And so within your flask view, you could like take over controlling the socket and start treating it like a WebSocket. Instead, like an upgrade type thing. Yeah. So you still are in flask and WSGI mode. So you're still like sync in terms of one worker is blocked handling that one thing, but it's pretty clever. Yeah. And Joe out there throws out that it's Miguel Greenberg, Miguel, you're talking about? Yeah, there's also flask socket. io. But this is like a more native thing where you can just do this directly in flask and certainly in the extension. So okay, cool. So yeah, super cool. All right. We're getting sort of short on time. So I want to ask a couple of questions, hit on some high notes as well here. The one of the things that I think is pretty exciting that GitHub has done, you know, get up I think continues to be pretty neat. And one of the things they've done is they've added sponsor support for projects. Has that been helpful to you all? Yeah, we've gotten quite a lot of support, or part of the PSF. And so we take donations through them. If you go to "palletsproject.com/donate", you get the PSF donation page. We've got lots of people, you know, making one time donations or scheduled, you know, monthly donations. And we're also part of tide lift, which is a commercial sort of open source support service that we're getting funding through. And we were also I mean, this isn't really a sponsor link. But since we host all our docs on read the docs, we're partnered with ethical ads, which is their advertising platform to share the advertising revenue on docs, and so we're getting quite a bit of funding. Put it towards Flask conferences. Yeah, that's fantastic. That's good to hear. I do feel like there's probably a + +45:00 handful of billion dollar companies that are using flask at the core of that, maybe click that link and do a little more. But yes, no. But if you're using flask in your company, take a look at the sponsor links. Yeah, we have lots of playlists for what to do with it. Yeah, I was really happy to see get up with this in here, because like, just having like a little, you know, buy me a coffee link on the side of a page that didn't seem like it worked. I think this sort of Central way. And what's interesting is, I didn't realize that I that you could link to external sponsors, like my thought was that that sponsor button really was going through the GitHub sponsor, channel, or framework or whatever. people recommend that we use GitHub sponsors as well, because a lot of companies who are already in GitHub ecosystem have an easier time clicking that button than something external. So that's something I will be looking at, like App Store versus not apps, like vs go to our site to buy a type of integration, right? I feel like it's almost like that. I would definitely recommend it. I've sponsored some things through the sponsor button there. And it's super easy. Okay, bars. How about that? Yeah, flask is on Mars are held to the lander get to Mars. And I bet if I pull this up, let me see if I can find your user profile here. I just have to hack the URL, it's gonna have to be that way. And I go down here. Here we go. You have the official achievement Mars 2020. Helicopter contributor. How awesome is that? Phil, do you have one of these as well? Yeah, I think so. Yeah, I'm so jealous. I want one of these. Everybody who's gotten a code contribution to the repositories should have that badge. And there's been so many more people who have contributed in other ways that should have that badge. But don't maybe you should also have the Artic vault contributor, I have the article in but I turned it off a while ago, because I thought it was kind of silly. Yeah, that's like, way more. I don't know how that whole system works. But if they had archived flask, 2.0, after all, my work was like done, I'd be much happier. + +47:02 They did like two years ago now. Yeah. So there's a lot of ways in which Python is being used on Mars, when the lander first landed there. And, you know, people said, Oh, look, there's the f prime library that was used to power the the ingenuity, helicopter. That means pythons there like that looks like maybe Python just trained the model and c++ was there. But it turned out the lander that went down actually, the cameras, all the different angles were controlled by Python, and then getting those images back to jpo was part of Python. And yeah, what's the story of the flask? and Mars? This is so cool. Yeah. So flask is actually part of f prime. So f prime is like their, I'm not an expert on this, but it's like their flight control software. So it's like a monitoring dashboard for everything. So they're using flask to provide like all this information about the launches and flights and landings, and get updates in real time. So I think that's where our stuff came in. But I'm sure other parts of their code are using Ginger. Or click in other ways, not necessarily through flask. Yeah. That's so cool. What did you think when you heard this every now and then I'll see like a, you know, some big thing like this f prime, and I'll, they won't have mentioned flask or anything, I'll have to like somehow have like this intuition that it was used. And so like I, I went and just looked at f prime, and I like, I'm like, Oh, it's Python requirements.txt. flask + +48:30 it on GitHub. I'm like, wow, check this out. And that's super cool. Go to the dependency graph. I'm sure we'll see. Yeah, look, what number one dependency right there. There we are in flask. restful. Yeah. Something to do with API is here, right? Yeah. Cool. All right. Let's, I guess, round this out with two things. Actually, real quick, I want to take a question, I think or follow up comment from Keanu might be off topic. They say they would love to know more about how to convince a large employer to spend money sponsoring an open source project. I wish I had a good answer for this. I know, there has been thought on this, but I'm just not the best person to talk about it. Next, I would say look at tide lift their company. And their whole like job is to be that in between, between open source who doesn't know how to convince companies to do this, and companies who are using a lot of open source, but don't know how to analyze their dependencies and divide up their support. And so if you go look at tide lifts, you know, their their whole goal is to enable that sort of thing and discuss it with manage management, all that if not guys lift it, maybe tide lifts. That's not good. If they're getting me down temporarily. They're seeing that before maybe HTTP versus HTTPS connection. Oh, yeah. So that place, I do think this is a good option as well. You know, they sponsored talk python to me for a little while, and I think they're definitely trying to be creative. And that's right. They're trying to offer something to employers. That's not just put it on your charity column in the balance sheet. They're like, we don't have a charity column in the balance yet. What are you talking about? Right? So + +50:00 Doing something a little bit different there, you can look at it. I think as a company as part of your recruiting spend. I think every company knows how competitive the market is. And I do think engineers do like it, what interest in what you do in the open source community and look at it in a positive light. So I think that might help. I totally agree. A couple other quick follow up questions. Olga asks, Can f prime connect to API's on earth like Wi Fi? Is there anything about this? I think it is used on Earth as well. I mean, like I said, it's like a monitoring dashboard. So it's not something that's actively running on the rover, I think, from what I understand, I think it is accepting data from sources like the rover and other systems and then displaying that and coordinating it. Very cool. And then Joe s if Armen runacres still involved with the project, any Armen is not, he's still part of the GitHub organization. And he's like in our maintainer channel on discord but he's moved on to other things, you know, he's he's working in RUST a lot and others. Yeah, my impression that he's kind of moved on to rust and it's just enjoying his time there, which, yeah, totally fine. Like people don't have to stick with one technology, that whole life, right? contribution. So that's one of the things I've been trying to do to since I became a maintainer was get more maintainers Great. So it's not just Armin and it's and then it after Armin, it's not just me. You know, I'm trying to expand it. So it doesn't matter if Arman or I are still involved. Yeah. And box of ninjas out there. It says, I need the Mars endpoint. + +51:31 Yeah, I don't think they just anybody. Yeah, there's probably they don't, but often there's hard to find the endpoint. No, it just is a satellite the right time. Exactly. You can just turn over there. Alright, so I think that's probably all the stuff we got time for. But you know, thank you guys, for covering this. Let me ask you these. Last two questions we got here. You're gonna work on flask or Quart. What editor Do you use these days? I've been using PyCharm for almost a decade now. I feel like pretty much and so I started using Python. So PyCharm all the way right on. I use Emacs. Although I've used VS code a few times. If I ever record something, I use VS code. But yeah, Emacs is why I do the way I do write down the bare metal. + +52:15 Cool, and then recommend some library package you all come across recently that you're like, Oh, this is super cool. People should know about x. Any ideas? I'm really enjoying using Pydantic to validate the incoming requests, like just as a data class setup. I know you can do it with different ones as well. But the syntax seems so clean just to say this is looks exactly like a data class. And I just use it to validate my incoming it's great. That's fantastic. Yeah, one of the things I've started to do is like the first line in my flask API endpoints, is to say, pydantic model = you know, class **. What does that request '.JSON', something like that. Like just immediately try to convert it to invalidate it with pydantic something I learned recently I had Jamie on the show who creates and maintains Pydantic is I didn't learn about how to show someone else pointed out to me, there's a like a secrets capability built right into pedantic that like works with like, '.CSV files', and all sorts of stuff. It's got this whole mechanism. So yeah, that the library does a bunch of cool stuff. I don't have a good answer for this, because I've been just dealing with flask itself for so long, and getting those ready. But I haven't really been exploring new stuff for myself, this just may not get enough, you can have another. Yeah. So hopefully, I get to learn about some new libraries. As soon after, you know, flask 2.0 stable, I will say at work. This isn't official yet, because we're going to, we need to write documentation. It's totally unusable right now, because it's not documented. But we've been working on a open source library called auto invent, which takes a data model like such as your SQL Alchemy models, and produces an entire GraphQL API and React front end from just the data model plus like customizations to submitted data. Now, that's cool. It's super handy. It's really cool. It's completely undocumented. So I will, at some point, maybe half a year from now announce the actual project. But if you go to an 'autoinvent.dev', you can see all our messy code right now. Nice. You got one more Go for it. Oh, yeah. So Trio, if you've not heard of it, it's a different async await based event loop implementation. And I think I really like it because the structure concurrency it introduces is the way I think the most pythonic way to do async and await Yeah, true is neat. I had to remember that maintainers name but had him on the show and Nathan Nathaniel. That's right. If they add Nathaniel on the show, and yeah, it's super cool. Like how you have dependencies between you know, this task created those tasks and you cancel one, you cancel them all. There's a lot of interesting patterns that come out of there during each. Alright, I think that's it. David, Phil, thank you. quick shout out or a quick final call to action. People want to get into flask 2.0, they want to upgrade their stuff. Maybe they want to take advantage of the async stuff. + +55:00 For whatever, what do you tell them? Where do they go from here? Well look for that look for that new version releasing tomorrow or a week ago, depending on where you listen to this. Follow pallets team on Twitter, if you want to see announcements for new things. And also, if you're interested in like contributing to these projects, I would encourage more people to go on GitHub, you know, at least start the projects you're using, but also watch them, you know, watch for new issues coming in. Anybody can pick up one of these things and help out. Yeah, you know, I think star and fork are the ones that get all the attention. You know, one thing that GitHub just recently had, I know, I said that, but I just want to give a little quick awareness to people I think is super cool. If you forked a repo, it used to be you'd have to go to a line, you know, add an upstream origin. And then you have to kind of keep that in sync. Now they've added a button that if you go to a fork, and as your as unions, click a button, go sync my fork with the original now, and I think that, you know, encourage people to forget and to stay up to date a lot a little bit easier. So that's pretty cool as well. Bill, final call to action. What do you say to folks? Maybe some core focus, I think it would be go and use it and let us know if it solves your needs, if it does what you want. Yeah. Awesome. All right. Thank you both for being here. Thanks. Yep. Bye. Bye. + +56:18 This has been another episode of talk Python to me. Our guests on this episode were David Lord and Philip Jones, and it's been brought to you by us over at talk Python training. Want to level up your Python. We have one of the largest catalogs of Python video courses over at talk Python. Our content ranges from true beginners to deeply advanced topics like memory and async. And best of all, there's not a subscription insight. Check it out for yourself at "training.talkpython.fm". Be sure to subscribe to the show, open your favorite podcast app and search for Python. We should be right at the top. You can also find the iTunes feed at /iTunes, the Google Play feed at /play and the direct RSS feed at /RSS on talkpython.fm. We're live streaming most of our recordings these days. If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at 'talkpython.fm/youTube'. This is your host Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code diff --git a/transcripts/317.txt b/transcripts/317.txt new file mode 100644 index 00000000..26b15f05 --- /dev/null +++ b/transcripts/317.txt @@ -0,0 +1,62 @@ + + +00:00 When you think of government software development and projects, do fast apps and modern tech stacks jumped to mind? Probably not. So you'll be delighted to hear from our guest, Laura Buford. She's the tech lead at the US Federal Election Commission . She and her team have built a very modern tech stack running modern flask web apps with API's powered by SQL Alchemy and Flask Restful. This app and its API are available as open source on GitHub, and they deploy it with continuous delivery, right out to cloud.gov. There are lots of lessons to learn for governmental agencies around the world as well as private organizations, small and large. This is talk Python to me, Episode 317, recorded may 19 2021. + +00:52 Welcome to talk Python to me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy, follow me on Twitter, where I'm @mkennedy, and keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via @talkpython. This episode is brought to you by Square and US over at talk Python training. Please check out what we're offering during our segments. It really helps support the show. Laura, welcome to talk Python to me. Thank you very much, Michael. Happy to be here. Yeah, I'm so happy to have you here as well. You're doing a whole bunch of interesting things around open source and the US government. And I find government to be so interesting. Because there's a lot of data. There's a lot of technology. There's many users. All right, we got it. 300 million users in the US are so in some sense. And yet a lot of times the technology there feel so old and ancient. And yet what you're building and some of the change that you're leading is, is very modern, right? You could put this up beside a lot of the startups in Silicon Valley, and it would look amazing still, so well done. Well, thank you so much for your kind words. Yeah, absolutely. Now before we dive into all the cool Python things, let's just start with your story. How'd you get into programming and Python? I have always been interested in computers. I love to tinker with my computers at home, like couldn't say that I had a natural aptitude for software development. I took classes in high school and college, I thought I wanted to be a computer science major and really struggled with writing software. It was a very exciting time politically at the time. So I decided want to be a political science and history major. And my mom really encouraged me to finish my minor which was great advice, Mom, thank you, I finished my computer science minor. When when I first graduated, I really wanted to get involved in campaigns. It was 2007 2008. And I found a I'm from DC, I came back to DC, I found a local political software company that was hiring. And I had originally interviewed for administrative assistant position and the founder recommended I work in tech support. I was really surprised because I felt like I didn't know what I was doing. But it was a really great organization. It taught me everything I needed to know and really loved. That's cool. Did you have one of those experiences where you're like, oh, there's no way I can help out with this tech support stuff. And then you get there, you're like, I'm actually able to help quite a bit. Yeah, you learn some skills. So we had one major system that we supported, and then a couple other smaller software. And I never really learned how to use myself. So you learned some tricks like, so walk me through the exact steps you've taken so far, you know, and then you sort of follow along with them. And you're like, have you tried clicking this, this looks promising. Let's try clicking on this and see what happens. So how funny. Yeah, tech support was a great first job out of college, I learned a lot helped me with my creative problem solving skills. And I really learned the software well to the point where, after a few years of tech support, I would send the developers the line of code where the bug occurred and say, you know, here's the bug that the user reported. So they asked me to come work on the development side. And I did that for a little while. And then I was still struggling with my confidence. It wasn't really a great fit. And I couldn't quite figure out why. I didn't have a lot of support. This is before a lot of the meetup culture on this was 2012. There are a lot of now I found there a lot of communities for helping people get into tech, but DC Yeah, the communities have really grown a lot last five years or so it's a lot a lot nicer in that absolutely . Yeah, and DC wasn't really a tech hub. It wasn't really known for being much of a tech hub. So I had a lot of self doubt. And I wasn't sure that being a developer was right for me. So but I while I was working at this company, it was a company that helped campaigns file their campaign finance disclosures, the FEC. So political campaigns in the in the US need to disclose money that they've raised and spent towards their election. This is if they've privately done it or through they get government support or something like that, right? You have options FEC provides free filing software and it's everything you might imagine. + +05:00 Free government software might look like and we're working on that. But they're also third party companies like, like equivalent of like a Turbo Tax for filing campaign finance disclosures. And so that's what that company did. And I just fell in love with all the rules in the campaign finance, it's sort of like super nerdy, like anyone who gets excited about filing their taxes and all the different scenarios, it's very similar amount of rules for who can give how much under what circumstances there are situations where individuals can gift goods and services to campaigns, but there are restrictions on those types of gifts. So I really just enjoyed that. World campaign finance. Yeah, even though I had been a political science major in college actually hadn't learned that much about the FEC, and campaign finance laws and regulations. And it was kind of before campaign finance was a hotbed issue, might can sometimes be a hotbed issue now. But this was not something we covered a lot in class. It's not something that was getting covered a lot in the media. So those three work that I learned about FEC disclosure rules and campaign fundraising, whereas before had really learned more about voting. So fundraising is what allows the running of the campaign to take take place. Yeah, so much, at least in the US is about running ads, and hiring people to go out and knock on doors and all sorts of stuff like that, right? So even though really, you'd hope it's kind of some sort of meritocracy and you debate and then you choose the idea there's money's involved. Yes, it has become a hot, hot topic and a lot of ways, but I think at the time I was learning about it was the money was often thought to be an indicator of support. So if you could raise a lot of money from a lot of different people, it was an indicator that you had widespread support, and that those folks were even if they're just giving you $20, towards the campaign, they would help you support, like have office space, and like you said, print ads and write some sort of if you're going to give money, you're likely you would vote for them, right, giving money as a bigger signal than actually voting. So it's an early signal, I guess. Absolutely. So I wasn't sure being a software developer was right for me. So I went to go work for the FEC in a less technical role, where I worked to help the FEC with more of the enforcement side of things, making sure that the campaign finance reports were filled out properly, if people have questions about how to comply with the law, if there's any disparities or anything that was unclear on the reports, I would help clear, like, send correspondence on the public record, asking the regulated community that, you know, the campaigns and the packs and parties to further clarify their reports. And there was an element of tech support in that role. I also did tech support for the free filing software. And anyone who's done tech support, sometimes it can get pretty, like answering the same question over and over again, no matter how much you care it can can sort of wear on you. So I was looking for a challenge. I'm so happy to have your question. I'm really here to support you for the 100th time. Yes. And this is the first time they're asking, you know, so you have to? Of course, they it's not there, they don't know. Right? And there are a lot of very confusing rules. And yeah, some of the best advice I got for my supervisor was you don't have to defend the agency, you know, so if someone says, This is really confusing, you can say it is confusing. Let me see if I can help you work through it instead of, you know, trying to defend the regulations for. And so after about five years of doing that job, I wanted to challenge I wanted to change. And there was an opening on the website, Project team@ftc.gov. And my mentor at the time had recommended that I come work on this project was really cool project. And I can talk about why it's cool. And I said, I haven't written a line of code in five years, you know, there's no way I can work there. And I met with the team. And they were willing to if I was willing to work hard to get up to speed, they were willing to work with me since I had a lot of knowledge about the area and past experience. So basically did a crash course in Python, with a lot more support this time around, there are all these organizations in DC for under represented genders and tech and just new developers. And so really did a deep dive and kind of was able to talk myself onto the team. And it was drinking from the firehose, and really just, yeah, really had fun, though, it was a super fun. That's cool. You know, one of the things that I hear in your story here, that sounds a little bit different, between the two experiences, you're going to college and getting your minor and CS versus, you know, being really passionate about the politics in elections, and then learning code to sort of advance that mission. And those are really different in that, you know, so often in computer science, it's like, well, here's some random algorithm, we're going to study the algorithm, then you can reproduce the algorithm and how boring and dry and separate from like what you're interested in, is that right? Whereas this is like, well, I've got to go through this technology to use it as like a way to really achieve the goals I'm after, and I think those are just such different experiences. Would you agree? Absolutely. I distinctly remember one of my projects in college was to build a calendar. + +10:00 And I was like, I have a calendar. + +10:04 You have to remember, like, the paperwork and I got a digital and + +10:07 yeah. And so it just wasn't very passionate about some of the projects. And they're good projects for teaching the concepts, but not necessarily the can be difficult to find. And I think there's been a re-evaluation of how certain ways of writing software is taught a little more beginner friendly. If you have a goal, you have something you want to build, and you know why you want to work on it? Well, then it's super exciting. But if it's just dry, I think we lose a lot of people who don't have some sort of personal drive that's going to get them through that is not necessarily part of the lessons are part of the official experience, you know, absolutely. And the thing I wanted to add to that is that I didn't learn at the time I was in school, I studied computer science, and I've sort of figured out that software development and computer science are can be somewhat different disciplines. And yes, I never learned how to really write software in college. And what I've learned the hard way is to really start small, get the smallest piece working, change the smallest part of it. And can you make that even smaller, maybe just hard code something for now. Whereas before I would write in college, I would write top to bottom, write the whole thing out and then just explode about a semicolon buried on line 463. Exactly. So those learning those skills has helped me to a lot more fun. A lot less miserable. Yeah. Fantastic. So I think you've given us quite a bit of background already about the FEC, Federal Election Commission. But I think two things are interesting to talk about real quickly here. One is this whole agency was born, I guess, a tiny bit before I was actually but it was born out of a financial scandal in 1972. Right, yes. After the Watergate scandal. Yeah, I bet people have heard of that one. I hope so. + +11:50 I would hope so do. The other thing is, you know, you look at the FEC. And he said, it's not a huge organization, the government, but how many people work there, we have about 300 full time employees. That's federal employees. So they work for the US government. And for on fec.gov. We have a team of about, you know, we have a cross functional team. But we have about nine developers and the user experience designer who work on the technical side of the site. Yeah, cool. It's pretty small group of developers probably all work pretty closely. I'm sure it's a fun place to be. It really is. Yeah. And before we move on too far from the opening, Kim van wyck, who was one of the Ask me anything hosts for me, Hey, Kim says I strongly suspect that also helps you remember a software topic six months later, if you did something you found interesting. while learning? Absolutely, I would definitely say so. Alright, so as we get into the code side of things, maybe we could talk a little bit about the type of data to give people a sense of what we're working with here. So one of the things you can do is you can go to fec.gov. And you go to all data, and there's all kinds of stuff, you can learn about raising money spending money, candidates, I could go here, I could select my state, Oregon, I could go pick my location around Portland, and pull that up. And it'll do things like I could go up and say learn about the house, I guess it will show this person and this party raised $64,000 and had $110,000 just disbursements, how much cash they have on hand right now. So apparently $711,000, which is kind of insane. And then you can go see the sources for this data. Tell us about the kind of data and how people use it there. Yeah, that's a great our electoral profile pages, I'm really proud of those, we can see who's running in which in each election, yeah, it's beautiful. And you get like a colorful map of the areas, you just click on it, it pulls it right up. It's really neat. Thank you so much. And as you were demoing it live here, it was looking snappy, which just really warmed my heart. + +13:44 So when we think about filing our taxes with the IRS, in the US, at least we have to disclose you know how much money we made and various other basically the money we made in our taxes. It's all summary level data. campaigns need to file the not only the summary level data, but also the transaction level data. So from every contributor who's given aggregate of $200, or more in a given election, they have to disclose their name and address employer occupation and the details of that transaction. And so it's a large volume of data, especially at the presidential level, or, or national level. Yes, yes, presidential level and like hundreds of millions of records, I think we're on track to have a total of 1 billion records by the end of the 2022 elections. It's just a lot of data. So it's not just how much they've raised. It's the individual contributions that go into that. So from here, if you wanted to click on the name of the candidate you're looking at for your district, it'll take you to the candidates profile page. It looks like this is a house candidate, and they run every two years. So there's a browse receipts button so you can actually see the individual contributions that made up and it's 2022. There hasn't been a lot of data yet for 2022 because it's early in the elections. + +15:00 cycle Oh my gosh. So I'm in here looking at this and it says, Here's Diane snow, who had donated how much you know, $50. Like these are individual contributions and all sorts of stuff, right? This is crazy, the crazy detail. Yes. And we've had detailed we've we it's part of our mission to make detailed contributor data and not just contributor but expenditure data, how that the committee's are spending their money, since back to the late 70s. So we do have all the historical data here to explore. You could export this to excel spreadsheet, or techniques, a CSV, but at the top of the page, you could filter for, like all the money that's been received for 2022. So you could eliminate your there, the breadcrumbs there with the little X's on them, you could, you know, just maybe that one candidate across all time or all the money for 2022. And so it is for all the data is is again, like about coming up on, we had about 500 million transactional transactions last in the 2020 election, that equals about four terabytes of data. And the expectation is it is filterable. It's searchable, it's exportable, and it user expectations that you'll be able to get the information quickly. So we spend a lot of time focusing on performance improvement for the site, which is powered by our campaign finance Data API under the hood. + +16:18 This portion of talk python to me is brought to you by Square. Payment acceptance can be one of the most painful parts of building a web app for a business. When implementing Checkout, you want it to be simple to build secure and slick to use. "Squares new web payment SDK" raises the bar in the payment acceptance developer experience and provides a best in class interface for merchants and buyers. With it, you can build a customized branded payment experience and never miss a sale. deliver a highly responsive payments flow across web and mobile that integrates with credit cards and debit cards, digital wallets like Apple Pay and Google ACH bank payments and even gift cards. For more complex transactions. Follow up actions by the customer can include completing a payment authentication step, filling in a credit line application form or doing background risk checks on the buyers device. And developers don't even need to know if the payment method requires validation. Square hides the complexity from the seller and guides the buyer through the necessary steps. Getting started with a new web payment SDK is easy. Simply include the web payment SDK, JavaScript blog and element on the page where you want the payment form to appear. And then attach hooks for your custom behavior. Learn more about integrating with squares web payments SDK at talkpython.fm/square, or just click the link in your podcast player show notes. That's talk python.fm/square. + +17:42 There's a whole bunch of API's. And you can even check out the code on GitHub, which is super interesting. This is the kind of data that people get. And I think it's it's pretty good. So one of the thing that's interesting to me is if I'm over here on 'FEC.GOV', this looks like a pretty well designed website. It looks nice, you can jump around. But then as you were showing me those different locations, it looks pretty data driven. Like there's a decent amount of logic behind it. But you said that this is built on wagtail. Is that right? So there are when you go to ftc.gov. It's mainly to applications, okay, the content management system we use is built with wagtail. So they're kind of three different buckets on the site. There's the campaign finance data, which we were just discussing, right, which is mostly powered by the API, we use some Ajax calls just basically JavaScript and data tables to display the data that the API provides. And this application that we're looking at is the Django based CMS application, right? And so for people who don't know, maybe just tell them really, what's the elevator pitch on wagtail? What is this thing? Oh, wagtail is a really great admin for the Django framework. So Django has a built in admin framework that so just to kind of give some context, the disclosure, the help for candidates and committees section of our site has information for a regulated community. So they want to find out, can I take this money from the source? Or what are the limits on contributions, they can go to help for candidates and committees and find the answers to their questions. And so that's stored in a in a in the Django database? Well, it's stored in a Postgres database that our Django application connects to. And Django as a web application framework has a built in admin. So you know, if you're working on this, if you've worked on like the Django tutorial or anything like that, you can go slash admin and use the built in Django admin to manage your content. So if you wanted to modify an article or you know you're working on a blog, or whatever, that's how you do it. wagtail is an open source tool that is that has more user friendly functionality on top of the existing Django admin little closer to like WordPress ish, kind. Yeah, it's got some great features. You can do custom templates. I don't work as I work more on the data sides, but we have a content team that uses wagtail to manage the content on the site. It seems like it's very customizable and just has more features than the + +20:00 Django admin that I'm sure you can build yourself. But this is a free open source project that allows you to have a more robust admin interface. Yeah, a lot of times, that's a big platform decision, people have to make like, Oh, well, we want to have this admin thing that people let people go in. And just developers don't have to add the informational pages and just maintain the basic data of the site. So I can either build something in Django, and then it's kind of on me, or we got to go through the admin backend in a clunky way. Or I've got to build something or we go WordPress, and then but now you're out of Python, and you're in this other Joomla, or WordPress world that you're not really able to extend and it just becomes almost a static site. And so things like wagtail are cool, because they let you stay in your world if you're a Python developer, but then also have a nicer experience on top of it for the users. Yeah, wagtail it's been seems like it's been great for us. Yeah. Very cool. Very cool. Okay, so we've got that. And then we've also got the open FEC API is what I'm trying to say here, which is RESTful API around all this data that we've been talking about, right? Tell us about this. Sure. So if anyone hasn't worked with API's before, the API is just the source for all the campaign finance data. So one of my friends put it in a cute way we drink our own champagne. So any of the data you see on fec.gov is coming from the API. So you don't need to know how to use an API, you never need to touch the API. If you don't want to you can you go to fec.gov and get all of the data that you that you want out of this. That's really cool. I've heard the term dog feeding, eat, you're + +21:33 drinking your own champagne sounds way better? Same thing, right? Way more unpleasant? I imagine. Yeah, it's absolutely. When we were interacting with the homepage, the old data section, that was JavaScript talking to this as well, just so you're basically the first client of this API. Correct? exactly correct. Okay. However, if the FEC went fec.gov doesn't have the information you need, or you want to build your own tools on top of the API, we do make that available to you directly. So you can bypass fec.gov and go directly to the source to get the data. How old is this API? How long has it been around, we realized around 2014, that the legacy systems we were running, were going to have a really big problem with our exponentially growing data. So in 2015, we partnered with 18 F, which is a government organization within consulting for government, by government. And we wanted to build an open source cloud based application that had an API. So we actually built the we launched the API before we launched fec.gov. So it's it's been, we've been drinking our own champagne since around 2015. It's been a party for 16 years. + +22:41 So I've heard of 18 F . It sounds like these are some really tech savvy folks that can drop in and help the other developers on other organizations or divisions and the government say, look, you want to, you want to modernize what you're doing. Let us come spend six months or a year and we'll help you get going and then hand it off to you. I was hugely fortunate to have overlapped with 18F when I joined the project, because I again, joined the project hadn't done a crash course in learning Python, but not having had a lot of other skills that were needed to be successful in the project. So I was able to have really, really excellent mentors that helped me learn in a lot of government agencies. This was our like our first cloud based application, I think, hopefully, I'll get a chance to talk about 'cloud.gov', which is the platform that we run. Yeah, absolutely. But this is all new to us. And so 18F was really wonderful. They taught us a lot. They taught us how to build software, an agile way how to practice open source, and not just right, not just share your code, but work in the open. So if you go to our code repositories, you can see what I'm currently working on all of our issues, all of our tickets are public, and you can see that the priorities and how things are going. And now there's my change, or I was tweaking our workers for an event. And actually, I'm thinking of moving to a different mechanism for our asynchronous requests. And you can follow along with my research and if you have opinions about workers versus threads for G unicorn, I think, I don't know. I call it g unicorn. I say G unicorn. Some people say gunicorn to me, I don't know it's like a unicorn. But + +24:15 if you have opinions on this, like please chime in all those issues are public. I've done some research on pros and cons of the different configuration. But yeah, 18F led the way on how to work in agile way and open way. And we were super lucky to be able to work with Yeah, this is really neat. So I'm sitting here looking@github.com/fec.gov/openfec and this is the GitHub repository for the API. Right? Correct. And start some forked off like well, we'll dump to it every now and then, like you said, this is where you all work. Yes, this is where we deploy from, we use circle ci, they have a really great open source tear. So when we merged the develop branch, it deploys to conduct the develop space. So this is our source of truth for the code nice and looks like you're using sort of feature branch style + +25:00 PR development which is feels modern and fresh to me. Another thing that's interesting is if I come down here we've got our, my requirements for my JSON here and we've got flask, we've got flask restful and flask SQL alchemy. This looks like a fun tech stack. I think you are. Yeah, I've enjoyed it. I gave a talk at flask con, which I think was the first time they ran it. I guess about a year ago, that goes into a little bit more technical detail about how we use flask for the API. Yeah, super interesting. One thing I'm noticing here, there's a flask. + +25:30 1.1.1 , which is great. Last week, just last week, maybe five days ago, 2came out. They also have new async plans going on? Are you all thinking about that already? Or are you just still putting out features for now with our data doubling every two years, and we are down a couple staff members? We're sort of in the hopefully hang on until next year and more money plays if it's not broken, but we do have a regular dependency checking for security issues, or do you kind of if it's not, if there's an issue, we do update it? Basically, we're not in the place to take that on quite yet. But I did what I did hear the news, and I'm excited to see what that might that might help us accomplish. Yeah, absolutely. You've mentioned code.gov. I think that's a pretty interesting place. Let's talk about a couple of other things. Let's talk about this communities of practice @digital.gov. So he talked about working on the open. So let's talk about some of the things going on there that I think are pretty interesting. The story of these community of practices. So I'm here I can see that there's like an AI section. And there's a blockchain section and their cloud and infrastructure, which is getting close to cloud. gov stories and DevOps. What is this place? Yeah, so one of the things that surprised me about coming to government is how so FEC is an independent regulatory agency, which means it's not part of the one of the big cabinet agencies. And so kind of surprised me, we're all sort of doing similar things. And we sort of struggle to talk to each other about, you know, what's working, what's not working, it's not always clear who you might talk to just at other agencies about ideas and suggestions they have were pretty small potatoes compared to a lot of the bigger agencies, and we have a lot to learn from them. But I don't know how I'd feel about just sort of cold emailing someone I stopped on LinkedIn or something like that, it can be hard to connect to other people working on very similar problems, because you could hear that maybe the people in I don't know, IRS or something, they're doing something cool. And you because it's kind of related to what you're doing. And in some sense, it'd be neat to see how that might work for you all, but even don't know those people, right? Like the government is huge. And it's like 100,000, or more that are huge, huge company that many people I've never met, they're not close to each other. Right. So this is a way to set up some of those communities around ideas. Is that right? Absolutely. Yeah, it's really wonderful. And we've talked to people at the CFPB. They use I know, they use wagtail. And we've sort of traded ideas about what's working well, and what's some of the challenges we're facing with the communities of practice, or a kind of a formal way of bringing people together. And I've really enjoyed the DevOps community of practice. And they have a monthly short half hour Tech Talk, where they invite people to talk about the types of things they're working on, and successes and challenges. Yeah, super neat. Then another one was this digital services playbook. So I think the digital services is something I just learned about a little while ago, three years ago, I think I had, oh, gosh, I'm sorry, I don't remember the guests name. But I had one of the developers on from the US digital service, which I thought was a really interesting, just a really interesting organization that I hadn't heard about, tell people what that is real quick. And then what is this digital services playbook? I think this seems pretty neat. Sure. So I'm not affiliated with the US digital service in any way. However, I have met people who work there. And I've always been super impressed by them. And they'd have a similar mission to at Neff, there are people out there who've written blog posts about the difference, but they're through the Executive Office of the President. And they do have a similar mission of helping modernize government, government technologies. I think one of the biggest challenges and everyone I've ever spoken to who works in government is contracting. So if you were lucky enough to have staff, a federal employees that you can have some more flexibility around how you do the work. But if you are there are some advantages and disadvantages to contracting. But it's it can be very difficult to get right. And so one of the resources I often point my acquaintances towards that are trying to modernize their technology is the digital US digital services playbook. And it goes through how to approach your modernizing your technology, it talks about best practices for contracting and kind of gives you a roadmap. And another resource I'd like to touch on in a second. There's a statistic that I think like 80% of all government, custom build software projects in government fail. And I think that contracting is a really hard thing to get right. It's sort of like imagine outsourcing for the private sector. It has risks to outsource because it just has trade offs and the the amount of management and oversight you might need to have for a contractor. + +30:00 Or outsourced project might differ from if it's in house. So this is a really great the digital services playbook is a really great resource for other federal employees or federal people working on federal projects. And then also just anyone looking to modernize their, their systems might find this helpful as well. Yeah, they've got 13 apparently, they're not superstitious, which is cool. + +30:25 Got a 13 Digital Service plays. So things like that are pretty, I guess, pretty standard, but also good, you know, I understand what people need, make it simple and intuitive. But then stuff that maybe people within government don't necessarily, they wouldn't necessarily think about, like default to open or choose a modern technology stack, and things like that, you know, default to open for example, right? You might think, well, this is data about our people. And it's super sensitive. That's one reason to keep it hidden. I mean, obviously, what you're given away supposed to be public in FEC. And then another is, you know, for example, like, if you share your source code, maybe people who want to do bad things with it will come in, you know, use that to find some way to do something that they shouldn't, right. We've seen a lot of security in open source, as well, right? By having having multiple eyes on it. Absolutely. And I think us building in the open from the beginning is the best way to successfully have an open source project, because retro actively open sourcing that has been built using the assumption that you'll have protection throughout the station, can be really risky. And unfortunately, if you're building a closed source system, it can make it a little easier to have some bad practices like hard coding credentials, I think there can be some misunderstandings about how open source works, like no one can come in and just change the code for fec.gov. They need to submit a request which is reviewed by one of our team members and gone through a security review as part of that process. So it kind of open source sort of encourages best practices as you might think that you're you've the strongest network security that ever exists. If something phishing is, I think one of the biggest weaknesses that organizations face. And so I think that's sometimes called zero trust. + +32:10 assume they're gonna get to the code either way, so you might as well, we use best practices for securing your code. And the best way to do that is from the very beginning, assume that that you're going to have eyes on it. + +32:22 Talk Python to me partially supported by our training courses. Flask is one of if not the most popular Python web frameworks and developers are adopting it for the smallest in many of the largest Python based websites and API's. If you want to learn flask, we built a fantastic course called "Building data driven web apps with flask and SQL Alchemy". In this course, we build a "PyPI.org clone" from scratch using flask and SQL Alchemy, you'll learn many of the major ingredients needed to build most web apps. If this sounds amazing, just visit 'talkpython.fm/flask' or email us at sales@talk python.fm. + +33:00 Before we move off this digital services Kim on live stream says David Holmes was your guest. Thank you, Kim. for that. Oh, yes, I remember that now. And says, as a non US citizen, he's from South Africa. Do organizations like the FEC use the US digital services? Or is it all in house? It sounds like it's you kind of separate from them. We partnered with 18F back in 2015. Again, they're sort of like consulting for government by government. They rolled off the 18F rolled off our project in 2018 ish. And we haven't partnered with us Digital Service, I think they choose their projects that can be influenced by the Executive Office of the President. So we haven't worked with them. But I've met, we do have an exchange of ideas for we're lucky to go to PyCon , for example, and meet other people who work there. And we can talk about what's working well, and so that is super valuable to be able to have that sort of relationship. But yeah, fantastic. One other resource that I think you were hinting at is this de risking guide from 18F, is that right? Yes, I'm a huge fan of this. They have a federal and non federal version. So if your state, that's a whole nother thing in the US that we have to deal with, where we have beings, the federal government and the state government, and just like all the federal agencies are out there replicating similar efforts, all the states are replicating similar efforts with their own budgets and their own challenges. And in some ways, if we're looking at things like security, we have a disadvantage, because we're splitting up our resources in many into many slices. So I love that 18F also built a non federal guide to help with the state and local governments as well. Yeah, there's a I think this also applies to large organizations that have maybe been around a while but you know, here's a stat from government says only 13% of large government IT projects succeed, and implementing custom software projects can be extraordinarily costly and risky in a government setting. And here's the big thing that waterfall software development remains the standard at all levels. + +35:00 So in budgeting and management and rollout, all those things, which is very different from here's my PR, let's push that with CI to cloud.gov. Right? I absolutely feel like someone said recently, government contracting is really designed to buy tanks. And building software isn't really very, doesn't have a lot in common buying a tank. And so yes, it is difficult to get right. And this guide has some really practical recommendations for how to manage the process. And one of my biggest things I'd like to highlight is the quality assurance surveillance plan or cost that's mentioned in this guide. You can build it into your contract as a way to measure whether the quality of the software is acceptable. It can help you as the person who's managing the contract and help your contractors understand what acceptable levels of quality are. So it looks like the cost is highlighted there at the top and one of the menus. Oh, yeah, there you go. And they have a sample class, which is really cool. And so if you scroll down a little bit there in this page, you can see some example quality measurement ideas. So using an automated tool to check for accessibility and security vulnerabilities, you should be able to deploy with a single command following a wasp best practices, there are tools that can be used to check for vulnerabilities doing user research and every sprint. These are just really great recommendations. Oh, yeah. Honey, there's little. How do you do this measurement of a set of like, what is the acceptable quality level? What How do you measure it? So on? Yeah, this is really neat. I like it huge. I'm a huge fan. Yeah, cool. Now let me pull up one thing that I thought was pretty interesting here, I'm seeing this. I'm seeing this all over the different sites that where I'm seeing on your fec.gov, and so on, if I click here, it says, right at the top in official website of the US government, how do you know? And it says, well, it's .gov. And then it talks about this little lock. And if I look at the lock, it says this is all done through Let's Encrypt, which Let's Encrypt is awesome. You know, it's a way to basically automate getting SSL certificates. But it feels like very modern and, and cool and hip. And to see the government rolling that out, as that was kind of a surprise. And I feel like this ties a little bit back to our next topic, which is cloud.gov. Absolutely, yes. The first thing I want to point out is the banner you mentioned at the top of the page is part of the US web development standards webt development design system. It's an initiative to bring the look and feel of government websites in line with each other. So it can help build trust, and users can be assured that they're they are in the right place. If you're if this is the first time you've heard of it, they have some really great resources, USWDS and I might be getting an acronym like the what it stands for slightly wrong, but they have components you can use on your site. It's all open source, web design system. They go, thank you. And so you can see on their website, they have a official website of the US government, here's how you know, at the top of their page, just like we do not even have versioning, like version 2.11.2. Yeah, this is a really great resource as well. It's all open source. So even if you aren't in government, if you wanted to borrow any design is a design system. And people who know more about design systems would know more about what that means. But we do use parts of that in our site. But you mentioned, Let's Encrypt as part of how we manage our certificates. fec.gov is run on a cloud.gov. And cloud. gov is a platform as a service that's available to government agencies. And so when you build a cloud based system, you have the option of building your own cloud infrastructure like AWS, or Azure, Google Cloud, from scratch, or you could use a platform as a service. And there are pros and cons trade offs of each approach. But we find as a small agency, cloud misconfiguration is one of the leading causes of security vulnerable, like security breaches. Yeah, how often do we hear about here is a open source server running in the cloud without without authentication, or here is 10 million records of sensitive data in a non secured public s3 bucket because it was a pain to deal with security went to integrate that with the app or whatever, right? These are real problems. Absolutely. And the cloud itself is very secure. But you need to make sure you're configuring it properly, take advantage of those security features. And so that's not easy to do, or I haven't found it easy to do there. I would say like they give you enough. Well, I won't use that phrase phrase ology, but it's very powerful and difficult to get right. Yeah, I feel like AWS and Azure, those types of places. They're very powerful but you go there and it's just you know, deer in the headlights so many choices, just even at the what area Do you want to go into and then you'd factor well what are the security implications of changing this switch down in some level? It's crazy. Yeah, yeah, I won't even sign up for a free tier account cuz I'm convinced knowing me someone's going to get access in mind Bitcoin. I'm gonna have a $10,000 + +40:00 Like they they make you put your credit card in there? And I'm not Yeah. + +40:04 So it takes a lot of expertise to build your own infrastructure in the cloud in a secure way, our agency being smaller, we could, we could go ahead and do that. But cloud.gov has a lot of advantages. And so what it does is it sits on top of AWS as a management layer. And so it manages cloud security. It manages authentication, you know, who has access to what it makes it super easy to deploy, really easy to keep our infrastructure as code, you can use infrastructure as code in AWS. But I still haven't really figured out how to do that. It seems wolf again, so many Yeah, haven't, I find it no matter how many trainings I take, I still find it difficult to work with so cloud.gov has a lot of advantages over building your own infrastructure has fixed pricing again, like if you mess something up and running a machine for too long, and you get a surprise, $10,000 bill, that doesn't happen in cloud.gov, because they have cost engineers and they have fixed pricing. in government, if you want to build a technical system, you have to have an it's called an ATO an authority to operate. It's all this paperwork that you need to do in order to prove that your system is secure enough, by using cloud.gov. You can inherit from their ATO and kind of save yourself a lot of paperwork, just put the checkbox I'm using cloud. gov, essentially. I mean, you mentioned we use, Let's Encrypt, and my first reaction was cool. + +41:28 They managed a lot for us and make it super easy to deploy. We have scalability through cloud. gov. And we can focus more of our time and energy on the data and application side of development, which in and of itself is pretty time consuming for us. Yeah, maybe you could talk about two things. Maybe you could talk about what is your deployment look like? I think using AWS at the moment, is that right? And then you're using Postgres, maybe just talk about how you're running all that the API's and stuff out there. Sure. So application architecture, we run our application tier on Cloud. gov, and our database tier is actually we have our own managed AWS, Aurora, Postgres that we run. Because while cloud.gov does offer database services, our database is so massive that we wanted and our we have really experienced DBAs, who wanted more granular control over the settings on the database. Yeah, you talked about how you were in a place where if you want to scale horizontally by adding more read servers, read replicas, that was a redeploy for app and a big challenge and moving to Aurora made that automatic behind the scenes, right? Yes, I did. And I can probably put this in the show notes. I did a talk on how we use Aurora for scale database scalability. So AWS, RDS, Postgres just kind of plain, I think it's like relational database service management, basically managed database service, when you have read replicas. So at some point, we need to run like three to four read replicas equivalent. So you wouldn't like the difference there. But we need to run on numerous machines at the same time on the database side, in order to meet our demand, I suspected this data is very read heavy, the way it's been used. Well, it is very read heavy, but we also have the data is always changing. So we have some really big filers. That's true if if you're doing like a big campaign people are every little contribution is streaming in, right, yeah. And there's some big packs that will have like 30 million records, 40 million records need to insert all in one night. So So we do have a lot of it's pretty insert heavy as well. But in the past, when you use already, AWS RDS, each read replica had its own connection string, we are using the 12 factor framework for application development, which is a few is that that's published online that basically considered best practices for cloud based application development. And it recommends using your attached services connecting to them through connection strings. So basically, you should be able to swap out your database for one database to be able to swap out to another. So this is backing services. Number four there treat backing services attached resources. So our application connect to the database through environment variable connection string, and we're using plain RDS before each replica had its own connection string. So in order to scale, we could have set up auto scaling on that database side. But we would have had to come up with a fairly fancy way of consuming that new connection string, adding it to the environment variable, the application and then redeploying the application to reflect that that new connection string, so basically wasn't functionally, it wasn't very feasible to us, for us to have database auto scaling. And after doing some research, we decided to move to AWS, Aurora, Postgres, which has one connection string that you connect to, and it handles auto scaling on its end. So it doesn't call them replicas, it calls them instances but they're functionally replicas. So we connect to one connection string and the could have two instances running and then if it + +45:00 has high CPU, it'll spin up some more machines to handle the database side. Cool. So you can set up like auto scaling rules and things along those lines. Yes. Okay, so that's the machine under the hood for the API. Yeah. And during one of your talks, was that presentation that I linked to? Is that one given to the community of practice group? Is that where that was, is? Yeah, that's all coming together. I see. So I'll link to that as well. But during that talk, you showed some need so see allies for deployment and configuring scale and things like that it was really cool. You over there, say, Hey, I just want five more web Brennan's boom, and you have them right. That's, that was quite neat how that works. Yeah. cloud.gov is based off of Cloud Foundry, which is an open source platform management tool. And so it can be a little bit difficult to run this on your own. But there are managed services that run Cloud Foundry. And one of the big ideas, as I understand it, behind Cloud Foundry is making the developer experience really easy. So you can deploy with a single command CF push, so we use on cloud.gov, all the Cloud Foundry commands. And there you just with one command, you can scale up your the number of instances you're running your application. And it's practically immediate. I mean, I would say like under under three seconds to quadruple your mission number of machines. Yeah, you made an interesting comparison to that about your previous model with physical servers. Yeah, so our legacy system was run on physical servers. And it was really difficult to predict how much we were going to need we'd sometimes we'd overbuy, sometimes we would under buy, it could take three months to get a new server setup. And now it takes three seconds to get a new instance running. In addition, when we moved to Cloud.gov , we saved about $1.2 million a year in infrastructure costs, which was really big for us as a small agency. That's a huge thing. And I believe one of the numbers you threw out there previously is 1.4 million was around the infrastructure cost before so to go from 1.4 to 200,000. That's a huge difference. It's not like, well, it's 100 million, and it went to 99 million, it was much, much better, right? percentage wise, that's a great point. Yeah. And we were able to again, like minded, the thing I love about cloud.gov is not taking on that risk ourselves, both the security risk, and also just the the risk of miss configuring something and something going down it for a small agency like ours, it's really huge to be able to depend on them for that. You don't have to have a team of network and DevOps engineers just to keep it going. I suspect, like that's the cloud. gov site, and they just deal with it. Yeah, I think when I talked to them, if we had taken all of this in house, the amount of services that they offer, we would have needed to hire like six or seven more people, and we have a team of nine. So it was just not cost effective for us to bring that the infrastructure and house at this time. Alright, so let me ask you a question. I've been thinking about this, as we've been talking about all these things, seeing all the cool stuff that you're doing, seeing how 18F came in and helped and all of those. Now, I don't necessarily expect you to have an answer to this. But I just want to hear your thoughts. So the rollout of the Affordable Care Act in all of the drama around the websites and the challenge of scaling that and stuff? Do you think it would be different? I think it would be smoother, if like this world existed in a more polished way. And we tried to roll it out again, in the same way. I think that there are a lot of challenges in developing and delivering digital services. I think a lot of the expectations for digital service delivery are high, you know, it's like, well, Google can find this for me, and and if you compare the operating budget between my agency and Google, it's like, yeah, Google just talked about rolling out like their own supercomputer for this other thing, right? That's like, a different set of capabilities or whatever. Totally, totally, I would say it. To me, it all boils down to as boring as it sounds, it boils down to contracting. It really depends on how the contract is written and how its managed. And I don't know very much about it. But I do know that if you don't get it, right, it's very difficult to course correct. So absolutely, we can do better. But it's there are still a lot of challenges to preventing those sorts of issues. Yeah, of course. All right. Well, I think that's probably all the time, we have to dive deep in this. But this has been a super interesting look at what's happening over there. And, you know, honestly, it's heartening to see how much the government is advancing, uh, using modern tech and really putting these systems in place. So well done. Thank you so much. It was a pleasure to speak with you today. Yeah, you bet. Now, before you get out of here, I got the two final questions, though. If you're going to work on the API, for example, write some flask and Python code. What editor do you use? I use sublime. And VS code does package management better, I think but I just can't think myself move and notable PyPI package. You know, maybe something cool that used during the project that you're like, oh, people should really know about this. It doesn't get enough press. Sure. I'm a maintainer for flask, SQL alchemy, so it's a really great package community has been super welcoming, and it's the only one I've been able to really get my hands on. So if you're looking for a project to work on all the flask projects are great to contribute to + +50:00 Oh, fantastic. Tell people real quick. Why would you choose Flask SQL alchemy over just SQL Alchemy? I think it depends on like getting started. It's pretty friendly to get started. And you can maybe start with that and then re evaluate what sort of custom settings you might need. Yeah. Cool. All right. couple comments from the live stream. Antonio says no, don't go. I really enjoyed the chat. And then Brandon says, super nursing subject. Thank you. Yeah, thanks. Thanks both for being here. final call to action, you know, people broadly, or maybe even people with in governments around the world who want to do better work with modern tech stack know, what do you tell them? Check out fec.gov the link to our GitHub is in the footer. We look at all the feedback as public anything you can get out of our open source project, take a look at who's running for office and would just love it if you got involved in our project. Very cool. Well, like I said, well done. Very Nice work. It seems like when I was working with the site, it was super snappy and beautiful Impaler so Excellent. Thanks for being here, Laura. I'll pass it along to my team. Thanks so much. Yeah, you bet. Thanks. Okay, bye. + +51:03 This has been another episode of talk Python to me. Our guests on this episode was Laura Beauford has been brought to you by Square and US over at talk Python training. With Square, your web app can easily take payments seamlessly accept debit and credit cards as well as digital wallet payments. Get started building your own online payment form in three steps with "Squares Python SDK" at 'talkpython.fm/square'. + +51:29 Want to level up your Python we have one of the largest catalogs of Python video courses over at talk Python. Our content ranges from true beginners to deeply advanced topics like memory and async. And best of all, there's not a subscription insight. Check it out for yourself at training.talk python.fm Be sure to subscribe to the show, open your favorite podcast app and search for Python. We should be right at the top. You can also find the iTunes feed at /iTunes, the Google Play feed at /play and the direct RSS feed at /RSS on talkpython.fm. We're live streaming most of our recordings these days. If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at +talkpython.fm/youtube. This is your host Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code