025_effective_python.txt

00:00:00 What if you could bottle up all the wisdom and hard-fought experience of many expert Python developers and power up your skills?

00:00:05 That's what Brett Slatkin did, and he put it in his new book, Effective Python.

00:00:10 This is episode number 25 of Talk Python to Me, recorded Wednesday, August 12, 2015.

00:00:16 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the library's,

00:00:46 the ecosystem, and the personalities.

00:00:48 This is your host, Michael Kennedy.

00:00:50 Follow me on Twitter, where I'm @mkennedy.

00:00:52 Keep up with the show and listen to past episodes at talkpython.fm.

00:00:56 And follow the show on Twitter via at Talk Python.

00:00:59 This episode is brought to you by Hired and CodeChip.

00:01:02 Thank them for supporting the show on Twitter via at Hired underscore HQ and at CodeChip.

00:01:07 It's another week with a book giveaway.

00:01:10 And just like last week's, this one is a really excellent one.

00:01:14 Make sure you're a friend of the show and you'll be in the running to win the free e-book version of Effective Python.

00:01:20 Just visit talkpython.fm, click on Friends of the Show, and boom, winning happens.

00:01:25 Well, sometimes, if you're lucky.

00:01:27 Now let me introduce Brett so we can get right to the show.

00:01:30 Brett Slatkin is the author of Effective Python.

00:01:33 He's the engineering lead and co-founder of Google Consumer Surveys.

00:01:37 He formerly worked on Google App Engine, the PubSub Hubbub protocol, and managing Google's fleet of servers.

00:01:44 Outside his day job, he works on open source tools and writes about software, bicycles, and other topics on his personal website.

00:01:50 And today, he lives in San Francisco.

00:01:53 Brett, welcome to the show.

00:01:55 Thanks for having me, Michael.

00:01:57 Yeah, I'm really excited to talk to you about your book and the other stuff you have going on with Python.

00:02:01 We're going to talk about this book that you recently released called Effective Python.

00:02:06 And I was a big fan of the Effective series back in the C++ days.

00:02:11 And so when I saw your book title, I was like, oh, this is going to be awesome.

00:02:14 So it's going to be a fun conversation.

00:02:17 Yeah, definitely.

00:02:17 And I similarly, Effective C++ was the best programming book that I ever had.

00:02:23 And so I totally agree with that.

00:02:27 So that was 1997 or 1998 when that came out when I first read it.

00:02:31 And being able to make the Python version of that was really a great honor for me to be able to write.

00:02:38 Yeah, I bet it was.

00:02:39 That's great.

00:02:40 Before we get into the details of your book, though, maybe we could just talk a bit about how you got to where you are.

00:02:47 How did you get into programming in Python?

00:02:49 Yeah, so I won't bore you with the super long details of my history of writing code.

00:02:55 But I've been writing code since I was about 10 or 11 years old.

00:02:58 My first job out of college was at Google.

00:03:02 And I showed up for my first day of work.

00:03:06 And they said, OK, here's this Python code base.

00:03:10 You need to make it better.

00:03:11 Here's a book.

00:03:12 And Alex Martelli sits right there.

00:03:15 Figure it out.

00:03:16 So I had seen Python.

00:03:18 I hadn't written any Python at all.

00:03:20 And I went from basically knowing nothing about Python and then actually writing a book about it in about 10 years, which is pretty funny.

00:03:28 Yeah, that's great.

00:03:29 The student becomes the master sort of thing, right?

00:03:32 Yeah.

00:03:32 I hope so.

00:03:33 I hope it's worthwhile.

00:03:34 Well, I think what was funny is that my first impression of Python was from the BitTorrent code base.

00:03:40 Because Bram Cohen had had this post all about Python and how it makes him more efficient.

00:03:44 And that's why BitTorrent's so awesome.

00:03:47 I don't think people realize that BitTorrent was written by Python.

00:03:50 But then like Zope and Plone, I thought were just really, they were code bases that I was very unhappy with using when I tried Python for the first time.

00:03:58 And that kind of turned me off.

00:03:59 So then when I showed up at Google and they said, OK, you're using Python, I was cautiously optimistic.

00:04:06 But it ended up turning out real well.

00:04:09 And so I can just go real quick, just the kind of projects that I used Python for at Google.

00:04:15 Yeah, people would love to hear that.

00:04:16 My first job there was kind of doing janitorial services for data centers.

00:04:23 Google has a bunch of machines in their data centers.

00:04:26 Those machines have a life cycle.

00:04:28 They're built.

00:04:28 They're repaired.

00:04:29 They break.

00:04:31 And I worked on a bunch of the tools to help that life cycle, which are primarily written in Python.

00:04:37 And I got to scale that out into Google's kind of worldwide network of data centers, a lot of system level code, a lot of networking, kind of database accessing, all kinds of stuff like that, workflows.

00:04:50 So that was my first kind of big role.

00:04:53 I did some other things similar to that around security and securing the machines in a data center.

00:05:01 And then shortly after that, that was a couple of years.

00:05:04 And then shortly after that, I heard of this really cool project that was being created called App Engine that had just started.

00:05:09 And I thought it really had a lot of potential.

00:05:14 And so I went and talked to that team.

00:05:16 I think it was four people at that point, the original founding team.

00:05:19 And I was like, what can I do to get on this team?

00:05:22 How can I work on this?

00:05:23 And in my 20% time, I built the App Engine Dev App Server, which is, I don't know if you've ever used App Engine, but you spend a lot of your time using this development tool to actually write your apps.

00:05:37 And so, yes, I built that.

00:05:38 And they're like, oh, this is pretty good.

00:05:39 And then they invited me to join the team.

00:05:41 And then I helped build App Engine out and launch it, which was, so it's, you know, platform as a service, Google's cloud kind of system.

00:05:49 Yeah, that's, it's really cool.

00:05:51 And was Python the very first language or was it Python and Go?

00:05:55 What was it?

00:05:56 It launched with just a restricted set, right?

00:05:58 Yeah.

00:05:58 So Python was the first language.

00:06:00 Java came like a year and a half or two years later.

00:06:02 And then, and then Go came a little bit after, like maybe two, two or three years after that.

00:06:09 I think the first language is actually supposed to be JavaScript, server-side JavaScript in the old Netscape, LiveScript scheme of things.

00:06:17 But in 2008, that seemed like a completely ridiculous proposition, which is pretty funny looking at all the Node.js stuff that's happened in the last few years with V8.

00:06:25 This is way before V8.

00:06:27 JavaScript is still really slow.

00:06:29 And so anyway, so they went with Python because it was one of the main languages that Google had used.

00:06:35 And Google had a lot of expertise in Python in general.

00:06:37 So on the App Engine team, I spent a lot of time working on Python infrastructure, Python APIs,

00:06:43 just kind of the feel of what it was like to write apps on top of App Engine using Python and fix a lot of bugs,

00:06:50 vetted new APIs to go out to make sure they felt good, built infrastructure around task queues and MapReduce and offline processing and all kinds of different things.

00:07:01 Did that for a few years.

00:07:04 And then also started this project called PubSubHubbub in there, which was a real-time RSS project for making RSS feeds real-time,

00:07:14 which is kind of hilarious also in hindsight now since that's pretty – it's kind of like saying I made something to design pogs or something like that.

00:07:22 It's kind of old.

00:07:23 Yeah, that's pretty funny.

00:07:25 But that was the Google Reader days and all that kind of stuff, right?

00:07:27 Yes, the good old days.

00:07:29 Yeah.

00:07:29 I kind of miss that thing still.

00:07:30 I do too, all the time.

00:07:33 I've been using News Blur recently, which I believe is also a Python app written in Django.

00:07:38 But, you know, anyway, I miss Reader.

00:07:40 Yeah, definitely.

00:07:42 So that is a bunch of really cool stuff.

00:07:44 And we might have to come back and talk about that at some later date in more detail.

00:07:49 That'd be really fun.

00:07:49 But let's talk about your book today.

00:07:51 It's called Effective Python.

00:07:53 What's the subtitle?

00:07:55 It's 59 Specific Ways to Write Better Python.

00:07:58 It's kind of based in this – the heritage of the Effective series, which, as we kind of hinted at the beginning,

00:08:04 I think that started out with Scott Myers and Effective C++, right?

00:08:08 Yeah, he invented the format.

00:08:10 And, you know, he's really the guy who came up with this way of educating people.

00:08:17 And all the other books are in his style.

00:08:20 Yeah, that's cool.

00:08:20 And so I remember when I was learning C++, I could sort of do stuff with it.

00:08:25 You know, I could write code and so on.

00:08:27 But after I read his book, I felt like it really took my understanding and effectiveness to a new level.

00:08:34 And so you're kind of trying to bring this to the Python developers, right?

00:08:37 Definitely.

00:08:39 Yeah, that's the goal of the book.

00:08:41 It's a bit of an audacious goal, but yeah.

00:08:44 Yeah, I mean, I think it's really simple is that, like, I read an introductory Python book.

00:08:49 This Effective Python is the book I would have given to myself as a second book if I had had the chance.

00:08:54 So there was a gap in my knowledge that took me years and years and years to fill.

00:08:59 And this book is kind of a way to shortcut and get all that practical knowledge without having to actually pay the dues.

00:09:05 So I wish I had had this book when I was starting out.

00:09:09 You know, we talk a lot about the concept of Pythonic code and idiomatic Python and stuff.

00:09:14 And there's a lot of that concept in there.

00:09:16 You know, like, this is the way you should do things in Python, even if you already know if you could accomplish it using your old idioms from, say, Java or C# or something.

00:09:26 But then there's another part, another angle to it that's just sort of broadening your horizon, I think.

00:09:35 So it's not just about writing Pythonic code.

00:09:37 And I'm thinking of things like how you would use, say, subprocess to manage child processes and parallelism from your Python app, right?

00:09:47 That's not technically a Pythonic thing, but it definitely ups your game and what you can do.

00:09:51 Yeah, I think, you know, a lot of the advice is pure language construct kind of stuff.

00:09:58 And then other things are like, yeah, the subprocess is item.

00:10:01 But, you know, it's important to remember that one thing that they say about Python is, you know, batteries are included, right?

00:10:06 And so, you know, the difference between Python as a language and Python as an ecosystem or Python as a set of libraries that you can use.

00:10:15 When people think of Python, they think of all of it together, both the syntax, the language features that it has, and then the libraries that you know you can rely on to always be there.

00:10:23 So those are tools you should have in your toolbox that are part of being a Python programmer, just like, you know, the collections API if you're a Java programmer, because it's basically all you write all the time.

00:10:34 Yeah, absolutely.

00:10:35 I have many friends that work in other languages, and they often want to compare their language to Python.

00:10:43 And like you said, I think that's not even a conversation you can begin to have if you don't think about the entire standard library and the 60,000 PyPI packages.

00:10:54 And, you know, you've got to take it as a whole, right?

00:10:56 It's not just, here's how you do properties here and properties there.

00:10:58 So this is either better or less good, right?

00:11:02 Yeah, totally.

00:11:02 I mean, a lot of these things, you have so much, I like to use the word leverage, you know, to describe, you get to stand on the shoulders of giants.

00:11:10 There's a lot of different ways to say it.

00:11:11 But yeah, things like NumPy, as an example of a library, it's a whole ecosystem unto itself.

00:11:17 And trying to do the equivalent in Java or C++ is extremely difficult.

00:11:22 Yeah, that's for sure.

00:11:24 So I thought one way that would be fun to have a conversation about your book is kind of go from section to section or chapter to chapter and pick out a few interesting pieces of guidance from each one of them.

00:11:36 Yeah, definitely.

00:11:37 Yeah, cool.

00:11:38 So you broke your book into six parts, and we'll just take them one at a time.

00:11:42 So the first one is this concept of sort of the libraries and Pythonic thinking, right?

00:11:48 Yeah, that's the first part.

00:11:49 It's actually eight separate chapters is how many there are, because it keeps going.

00:11:53 But yeah, the first one is Pythonic thinking.

00:11:58 Excellent.

00:11:59 So why'd you start with that?

00:12:01 Yeah, I think that it's kind of like the strange universe of Python syntax.

00:12:07 You have to get used to doing things.

00:12:09 Even simple expressions like, I remember when I first started writing Python, and the not operator, as opposed to using the bang or exclamation point, getting used to where that not goes.

00:12:19 And things like is not, as opposed to, you could do A is not B, or you could do not A is B.

00:12:26 And, you know, understanding that the Pythonic way is A is not B. That's how you'd say it.

00:12:30 So you really need this base to start with, so that all the other things make sense or are in context.

00:12:37 Right. I also think it's probably a good way to kind of shock the system a little bit to prepare them for, like, this is a different world, probably, than you're coming from.

00:12:45 Yeah.

00:12:46 And so get prepared to think differently.

00:12:48 Yeah, it's to open, I think it's a good way to say it, because I think it's, if you try to fit it into the way you're writing other languages, you're going to have a bad time.

00:12:56 And you really need to embrace the Pythonic way of doing it.

00:13:00 I had one person told me once that my C++ code looks like Python.

00:13:04 I thought that was a really great compliment, I guess.

00:13:08 But you don't really want your Python to look like C++, right?

00:13:10 Yeah.

00:13:11 Absolutely.

00:13:12 Cool.

00:13:13 So the very first thing that you talk about, literally the first piece of guidance, is to know what version of Python you're using.

00:13:21 And I do a lot of training in Python, and it's always a problem.

00:13:25 Someone's always running the wrong version of Python, or even worse, they're trying to use a package and they install it into the wrong version of Python,

00:13:33 and then they're running the right one or something like that, right?

00:13:35 Yeah, that's a big problem, especially with the Python 3 move that's going on.

00:13:39 You know, you can get confused really easily.

00:13:41 And then a lot of the features that people like to use, like even with statements, are actually more recent developments.

00:13:46 So if you're on some old version, some versions of macOS are like running Python 2.5, and you don't know it,

00:13:52 and you try to do simple things like a with statement, it doesn't work.

00:13:54 And so that can be very surprising.

00:13:57 So it's always good to check your assumptions.

00:13:58 Yeah, absolutely.

00:13:59 So sort of level set, all right, we all know what we're talking about.

00:14:02 Now we can move forward, right?

00:14:03 Yeah, exactly.

00:14:05 So what I think probably a lot of people are familiar with, but some are not, is sort of working with sequences.

00:14:12 And you say you should prefer using list comprehensions over like chaining a map and a filter call together.

00:14:18 Yeah.

00:14:21 Yeah, so I think if you, yeah, I think the story, so it's interesting, because if you look at the history of Python, people will talk about how, oh, I like Python, because it's kind of functional, but it's kind of object oriented, and it's kind of scripty.

00:14:33 And so you'll still see examples or guides today.

00:14:36 You'll be searching around the internet, and you'll find some old guide on active state from like 2006.

00:14:40 And it's like, here's how you should use map and filter.

00:14:42 And, or even 2003, like, you know, a long time ago.

00:14:47 And those are useful tools, sometimes they're useful, but the list comprehension syntax is just so much more pithy.

00:14:54 I think that Python over the years has added more and more tools to maximize the readability of code, and to minimize the visual noise, the extra parentheses and brackets and various symbols that you need to express something.

00:15:07 And so I think that map and filter are kind of just antiquated tools that are in the language, because they're hard to take out, because they're, you know, kind of keywords, built in functions.

00:15:18 If I had my way, I would just take them out, because I think list comprehensions are better.

00:15:22 This episode is brought to you by Hired.

00:15:36 Hired is a two-sided, curated marketplace that connects the world's knowledge workers to the best opportunities.

00:15:42 Each offer you receive has salary and equity presented right up front, and you can view the offers to accept or reject them before you even talk to the company.

00:15:52 Typically, candidates receive five or more offers in just the first week, and there are no obligations, ever.

00:15:58 Sounds pretty awesome, doesn't it?

00:16:00 Well, did I mention there's a signing bonus?

00:16:02 Everyone who accepts a job from Hired gets a $2,000 signing bonus.

00:16:06 And as Talk Python listeners, it gets way sweeter.

00:16:10 Use the link Hired.com slash Talk Python To Me, and Hired will double the signing bonus to $4,000.

00:16:19 Opportunity's knocking.

00:16:20 Visit Hired.com slash Talk Python To Me and answer the call.

00:16:34 But yeah, the main thing is that, you know, if you have two ways of doing something, I think the Pythonic way of doing it is always the more clear and more explicit and obvious way of doing it.

00:16:45 And so yeah, so that's why I think list comprehensions are definitely the way to go.

00:16:48 Yeah, that's cool.

00:16:50 I totally agree.

00:16:51 It's not exactly a more declarative way of programming, but it's closer, right?

00:16:56 That's nice.

00:16:57 Yeah, it's more obvious.

00:16:58 Sort of related to that, one of your other pieces of guidance was that you should use generators for large list comprehensions.

00:17:05 Yeah.

00:17:06 It's so easy to switch from one to the other, but when should I use one?

00:17:09 When shouldn't I?

00:17:10 What's the story of that?

00:17:11 Yeah, so I think that, you know, Python can use a lot of memory is one big thing.

00:17:16 And depending on what you're trying to do, if you're dealing with a small data set that entirely fits within memory, then you don't have to really think about this.

00:17:24 You can just read everything as a list and just work on lists.

00:17:27 So if you're reading a file, like a CSV file, and you want to deal with it line by line or something like that, then reading the whole thing into memory is fine.

00:17:35 But what always happens is you end up wanting to do something a little bit bigger than you expected, especially with a lot of the data processing stuff that people are doing in Python these days.

00:17:44 And so I'm trying to change the culture a little bit to say, hey, why not use a generator if you can?

00:17:52 Because if you use a generator, then all of these memory problems go away.

00:17:57 A generator returns one item at a time.

00:17:59 It doesn't fully materialize the list.

00:18:01 So the total memory space that's occupied is just the last thing you return from the generator.

00:18:07 The rest of it gets cleaned up, garbage collected.

00:18:10 Yeah, that's fantastic.

00:18:11 Especially if you're like chaining one to another to another, and they're kind of building up.

00:18:17 It's really efficient.

00:18:19 Yeah, and to that point, I mean, I think that if you start by at the base level, like if you want to switch to generator code, it's difficult because you have to, you have to,

00:18:27 put generators all the way down to the leaves of your call stacks.

00:18:30 But if you start with generators at the leaves, then it's very easy to start saying, hey, you know, I'm going to turn this into a streaming generator.

00:18:35 And so creating those cascades of generators becomes very straightforward.

00:18:39 And so you can get quick, you know, very fast executing code that's easy to follow and uses a low amount of memory very, you know, very cheaply.

00:18:48 And it's very readable.

00:18:50 So that's why I suggest that.

00:18:53 So the final language one that I wanted to sort of point out is something that I always thought was weird is the whole concept of the else statement on a loop.

00:19:04 Yeah.

00:19:05 And something I recently learned is that try except finally also supports else.

00:19:12 It does.

00:19:13 Yeah.

00:19:13 Yeah.

00:19:14 And you have really interesting guidance.

00:19:15 You say, you know what, these, this else statement on loops, maybe not so much, but on exception handling, it's very cool.

00:19:20 It's like an alternative to the catch.

00:19:22 Yeah, definitely.

00:19:23 Yeah.

00:19:24 So I think else, else makes a lot of sense for try except finally, because you're basically saying, you know, try to do this.

00:19:31 If an exception happens, do this.

00:19:33 And if no exception happens, then do this other thing.

00:19:36 And that's what the else block is on a, on an exception, kind of try except.

00:19:40 and so that makes things really clear, clearly delineated what will and will not happen inside your exception handling.

00:19:45 and when you're doing exception handling, you want the, the try block to be as small as possible to narrow what you're catching.

00:19:52 So that, that exceptions you didn't plan to catch are raised back up.

00:19:56 That's, that's like a really important thing.

00:19:57 Yeah, that makes sense.

00:19:58 And the other thing is, you know, you might think, well, just the last bit of code in your try bit, your try try block could be what would go in the else statement, but that doesn't account for like early returns.

00:20:11 And so, you know, if you were to do, try to do some stuff, if this, oh, just return, it would still run that else, but then you'd have to be really careful, you know, trying to put that cleanup all over the try.

00:20:22 So yeah, it's really nice.

00:20:23 Yeah, it's nice.

00:20:24 And yeah, you can put like final cleanup stuff in the finally, and that'll always run if you have an early return.

00:20:29 And if you have an else with an early return, that'll also still run the finally.

00:20:32 So it's, it's super nice.

00:20:34 Going to that back to that for loop part of it.

00:20:36 Yeah, I think that's one of the big things.

00:20:37 Like when people first learn about that in Python, they're like, oh, this is so great.

00:20:40 I can, I can use it for doing this and that.

00:20:43 And it's, there's really, yeah, I think it's something to avoid.

00:20:47 It's kind of like, it's a shiny new toy and you want to use all the different parts of it for you, you know, and you have to like stab yourself in the eye before you realize that you're getting, that wasn't a good idea.

00:20:57 That thing hurts.

00:20:58 Yeah, it hurts.

00:20:58 Like, oh, it's short.

00:20:59 It's pointy.

00:21:00 I shouldn't touch that.

00:21:00 And I was talking to, I was talking to Guido about this one in particular because he said, you know, cause I was asking him what he thinks about this one.

00:21:10 Cause it's kind of, it's kind of tough.

00:21:12 I had worked with Guido on App Engine for a few years and, so, and I see him at PyCon from time to time.

00:21:17 And so, you know, I felt bad by saying like, hey, you shouldn't use this part of the language.

00:21:20 I just, you know, wanted to make sure it wasn't ridiculous to him.

00:21:23 And, you know, he said that it's actually an implementation detail of how the Python, the CPython runtime is implemented.

00:21:29 the way that the for else block, the else with loops works has to do with the way these go to statements actually work in the Python interpreter.

00:21:37 and so if you, if you, if you wrote that code, then it makes perfect sense because the else has to do with the way the certain switch or go to statement works.

00:21:47 it's switch statements.

00:21:48 Sorry.

00:21:48 It's not a go to thing.

00:21:49 It's a switch.

00:21:49 And, but if you don't have that mental model, then it makes no sense.

00:21:53 And, you know, I think that, so for, for Guido and maybe other people who've hacked on the core, they just get it and it makes sense to them, but everyone else they have, it does the opposite of what you would expect.

00:22:04 And so just because of that, I think it's too sharp and I think it's something you should avoid.

00:22:08 Talk Python To Me is partially supported by our training courses.

00:22:13 Do you want to learn Python, but you can't bear to subscribe to yet another service at Talk Python Training?

00:22:19 We hate subscriptions too.

00:22:20 We hate subscriptions too.

00:22:21 That's why our course bundle gives you full access to the entire library of courses for one fair price.

00:22:26 That's right.

00:22:27 With the course bundle, you save 70% off the full price of our courses and you own them all forever.

00:22:33 That includes courses published at the time of the purchase, as well as courses released within about a year of the bundle.

00:22:40 So stop subscribing and start learning at talkpython.fm/everything.

00:22:46 All right.

00:22:47 So the next chapter was functions and there's a lot of good stuff in there.

00:22:50 One of them that I'm a fan of is the whole sort of concept of it's easier to ask for forgiveness than permission.

00:22:57 Okay.

00:22:59 And, you know, that sort of manifests in your recommendation of saying prefer exceptions rather than returning none from functions.

00:23:05 Yeah.

00:23:06 Yeah, definitely.

00:23:06 Yeah, this one I think that people, they're always looking, I don't know if you've ever heard of like a tribool.

00:23:13 There's always this joke about a tribool.

00:23:15 It's a Boolean value of three, you know, values, tribool.

00:23:18 People are always dying for a tribool because it's, you know, it's true or it's false or it's some exceptional case that's neither true or false.

00:23:24 And none in Python ends up playing that role a lot of the time.

00:23:28 And that's a problem because in Python, both false and none evaluate false.

00:23:37 They're falsy values.

00:23:38 And so what ends up happening is it introduces a lot of bugs into your code because you say, hey, you know, x equals this function call.

00:23:45 And then if x, then do this other thing.

00:23:48 Or if not x, do this, then there's an error.

00:23:51 But if you're returning none, then that if not x kind of statement is going to run no matter what.

00:23:59 And so my whole point, I've been hit by this bug in production a whole bunch of times.

00:24:03 I had to learn this one the hard way over the course of many years.

00:24:07 And I'm still managing a code base that has a bunch of this in it and trying to take it out.

00:24:13 And so my advice here is if you have anything exceptional, just always raise an exception.

00:24:16 That's what exceptions are for.

00:24:18 I think people coming from other languages are used to exceptions costing a lot of CPU time or somehow, you know, cause a lot of cache problems.

00:24:28 And that's true for C++ and other languages.

00:24:31 But in Python, raising exception is a little bit of cost to it, but it's not enough to matter.

00:24:36 And the clarity is worth it.

00:24:38 Not enough to introduce bugs into your code, right?

00:24:41 Yeah, exactly.

00:24:42 It's like, okay, well, it's fast, but it doesn't work correctly.

00:24:44 It's like, all right, well, if that's what you want, then that's fine.

00:24:46 Yeah, that's not a trade-off you want to make, right?

00:24:49 Yeah, not the right trade-off.

00:24:50 Sure.

00:24:51 So another one you had around functions that I thought was good advice was if you're going to return a lot of data, just make that function a generator rather than returning a list.

00:24:59 Yeah, and this kind of goes hand in hand with, you know, generator statements that I was talking about before.

00:25:08 I think I kind of covered both of those when I was talking before.

00:25:10 So, you know, generator expressions, there's list comprehensions, which, you know, are a fastest way of doing map and filter on lists.

00:25:17 Then there's generator expressions, which is a way of doing exactly the same thing, but the data is generated one at a time.

00:25:22 And then a generator function is just a, it's not just a one-liner, it's like a full function that does this.

00:25:28 And that's, so what I was talking about before, I think I kind of conflated both of those together.

00:25:32 Yeah, I think just having functions that return, that are generators, like I was saying, lets you kind of start with the leaves and move up to make things streaming.

00:25:39 Yeah, and it's so easy to just throw in a yield, you know, instead of list.append or whatever, right?

00:25:47 Yeah, exactly.

00:25:47 It's very clear.

00:25:48 And what's interesting is that, you know, JavaScript and other languages are adopting the yield semantics because it's such a powerful tool that people love to use, which is really great.

00:25:58 And I'm sure Python stole it from something else.

00:25:59 I actually don't know exactly where it yields from originally, but it's a really great tool.

00:26:03 Yeah, it's really great.

00:26:05 And it's one of those that I find that a lot of people, they haven't taken the time to really learn it and appreciate it.

00:26:12 They know it's there and then they just kind of ignore it, right?

00:26:15 But knowing, sort of raising the awareness is great.

00:26:18 Yeah, definitely.

00:26:19 And it should be the default.

00:26:21 I think you should just always, you should like plan to yield.

00:26:24 Plan, like if you're returning and it's not, if you're returning a sequence, like question what you're doing.

00:26:29 Yeah, exactly.

00:26:31 The thing better be pretty short.

00:26:33 So the last one I want to talk about in functions, which I haven't necessarily decided my feelings on it yet, but is if you use keyword arguments, that can help in the expressivity and clarity of functions.

00:26:47 Yes.

00:26:48 So I think the actual recommendation was to use just keyword arguments if you get a chance sometimes, right?

00:26:55 Go ahead.

00:26:56 Yeah.

00:26:56 Maybe speak to that a bit.

00:26:57 Sure.

00:26:57 So, well, I was curious what part you don't necessarily agree with because I'm curious.

00:27:02 So, yeah, I mean, because I have a few different parts to my advice, but yeah, keyword arguments are extremely powerful.

00:27:08 They let you be explicit about your intention.

00:27:13 And yeah, I think that using them in general for optional parameters is really important.

00:27:21 So if you have a function that takes an optional flag, then you should always use, you know, keyword argument is a great way to do that.

00:27:26 No, I was going to say that I totally agree with.

00:27:28 I think where there's an optional thing, it definitely helps because it's not part of what you expect to pass all the time.

00:27:36 Right.

00:27:36 Yeah, because Python doesn't have polymorphism like other languages.

00:27:39 So in Java, you can define the same function name that takes three different sets of parameters.

00:27:43 You know, C++ can do that too.

00:27:44 And in Python, you can't do that.

00:27:46 So the only way to deal with this is to basically add additional optional parameters to one function and have it kind of deal with the various types of input or define a totally different function of the different name, which can get verbose.

00:27:57 Right.

00:27:59 Yeah, I'm not sure I disagree with you.

00:28:01 I just haven't fully internalized that one yet.

00:28:03 Yeah.

00:28:03 So the use case that I was thinking of is if you have some kind of rich IDE, something like PyCharm or Python tools for Visual Studio or something, and you hit open parenthesis and there's like a nice listing of all the parameters without even looking at the docs, which for optionals, I think that still does show up.

00:28:20 But if you just do like the star star KWRs.

00:28:23 Oh, yeah.

00:28:23 Then you got a problem.

00:28:24 Then it's really like, okay, what the heck can I pass?

00:28:28 Yeah, no, that's totally true.

00:28:29 So star star KWRs, which basically is a catch all.

00:28:33 It's you catch any keyword arguments no matter what they are, including garbage.

00:28:36 It can be a problem.

00:28:37 And this is one of the things that I've tried really hard in the book to do, which is I provide advice that applies both to Python 2 and 3.

00:28:46 So it's not a Python 2 book.

00:28:48 It's not a Python 3 book.

00:28:49 It's like the most overlapping subset that I could find of both.

00:28:52 So it should be relevant to everyone.

00:28:55 And this is a great example.

00:28:56 Python 3 has specific language features that would make it so your IDE will continue to work properly, whereas Python 2 does not.

00:29:04 So in Python 2, you can't say these arguments can only be passed by keyword.

00:29:10 And you want to do that in situations where you're passing two integer values and you don't know which is which.

00:29:16 And so you want to use the keyword part in the function call to make it really clear that, like, hey, this is the numerator and this is the denominator.

00:29:23 And it's not the other way around.

00:29:24 So here's a label in the function call.

00:29:28 Numerator equals 5 and denominator equals 10 or something like that.

00:29:32 And you want to enforce that behavior so that people can't call it by just passing 5 and 10 or 10 and 5 on accident.

00:29:37 And so in Python 3, you can just add this star into the arguments list and then that's enforced by the compiler and everything's great.

00:29:46 And in Python 2, you can't do that.

00:29:47 So that star star kwrks is a way to deal with that.

00:29:50 And you're right.

00:29:51 It breaks your IDE.

00:29:52 It makes it harder to understand the documentation.

00:29:54 You have to make that tradeoff for you and say, hey, how error-prone is this function?

00:29:58 Is it worth doing this, you know, having these optional parameters that are in this way?

00:30:04 And, yeah, maybe there's a better design that you could have, maybe a helper class that would be a better way to approach it than having kwrks, star star kwrks, sitting there.

00:30:11 Right.

00:30:12 It's definitely a cool language feature.

00:30:13 It's just I think sometimes it gets overused and you're like, okay, I've got to basically keep the documentation up for a while so I know what my possibilities are, you know?

00:30:21 You're totally right.

00:30:22 I would say don't use star star kwrks unless you're doing something like this.

00:30:27 I think it's one of the few times you should use it.

00:30:30 There's some other glue kind of infrastructure times that are good to use it if you're doing, like, generic wrappers of functions and stuff like that.

00:30:36 But in general, I try to stay away from it.

00:30:38 And I'm really happy that Python 3 has this extra forcing keyword arcs feature.

00:30:43 Yeah, that's cool.

00:30:44 Okay, then I definitely agree with you.

00:30:46 Okay.

00:30:47 Thanks.

00:30:47 This episode is brought to you by Codeship.

00:31:05 Codeship has launched organizations, create teams, set permissions for specific team members, and improve collaboration in your continuous delivery workflow.

00:31:14 Maintain centralized control over your organization's projects and teams with Codeship's new organizations plan.

00:31:19 And as Talk Python listeners, you can save 20% off any premium plan for the next three months.

00:31:25 Just use the code TALKPYTHON, all caps, no spaces.

00:31:28 Check them out at Codeship.com and tell them thanks for supporting the show on Twitter where they're at Codeship.

00:31:40 So let's move on to classes.

00:31:41 One thing that Python supports is multiple inheritance.

00:31:46 And speaking about, oh, that's sharp, that hurts, is I did a lot of C++ and even COM with insane templating, multiple inheritance.

00:31:56 And I just, I kind of prefer single inheritance and just keep it really simple.

00:32:03 And so you were, you're saying that to some degree and you're saying, look, this multiple inheritance stuff should be used for mix-ins.

00:32:09 Yeah, definitely.

00:32:10 Yeah.

00:32:11 So I.

00:32:11 Can you tell people what you mean by mix-ins in this case and like how that would go?

00:32:15 Yeah.

00:32:16 So I think that a mix-in to me is, is a set of functionality that you can add to a class.

00:32:22 You can add to any class.

00:32:23 So helpers for serialization, helpers for logging, helpers for, you know, doing some kind of introspection on functions and stuff like that.

00:32:34 Those are the mix-ins where you're like, hey, I have this class.

00:32:37 Wouldn't it be nice if it automatically logged every function call or something like that?

00:32:40 Or every attribute access.

00:32:41 Let me, let me, let me inherit from this mix-in utility to do that for me.

00:32:45 So I think that those are the times where you want to use multiple inheritance, where the class structure doesn't actually matter.

00:32:52 It's more like aspect-oriented programming or something like that, where, which I'm not necessarily a huge fan of, but I'm just trying to say that it's nice to be able to kind of compose functionality.

00:32:59 And so mix-ins are a way of doing composition with class inheritance.

00:33:05 If there were another way to express that composition in Python, that'd be great.

00:33:09 But the tool that we have is, is multiple inheritance.

00:33:12 Yeah.

00:33:13 And I have seen some really beautiful mixing code that, that when done right, right, is nice.

00:33:18 Yeah.

00:33:19 When it's done correctly, it's, it's really, really nice.

00:33:21 And, and it doesn't, it's not brittle.

00:33:24 I think a lot of the time, multiple inheritance is, is really brittle and it breaks in weird ways when you refactor it.

00:33:29 And mix-ins are built not to be brittle.

00:33:32 And so that's why it works out.

00:33:33 So it's about composition, primarily.

00:33:36 Nice.

00:33:37 So let's move on to the next section, meta classes.

00:33:40 Yeah.

00:33:41 And in there, one thing that was sort of stood out to me was you're talking about that you can register class existence with the meta class.

00:33:49 What's the story of that?

00:33:51 Yeah.

00:33:52 So, you know, meta classes are like the sharpest tool you can poke yourself with in Python.

00:33:57 And it's kind of like, once you use it enough, Python enough, you're like, okay, I'm going to try out these meta classes thing.

00:34:02 And maybe you read Eric Raymond's thing about meta classes and some email filtering thing he wrote with them and how awesome that was.

00:34:10 And you realize that essay is from 12 years ago and it's worth to think about that again.

00:34:14 But it's a really powerful tool.

00:34:18 I have three different kind of main ways I think you should use it in Python.

00:34:22 I don't think you should use it in other ways unless you really know what you're doing.

00:34:25 And the registration is one of the big ones.

00:34:28 So it's really nice to know, you know, if you're creating infrastructure and you're creating, you know, classes to represent database tables or sensors that self-register,

00:34:38 a lot of that kind of like, you want the program to kind of sign itself up and initialize itself up so you don't have to write a lot of boilerplate to say,

00:34:45 oh yeah, hey, remember, configure this sensor, configure that sensor, configure this database row.

00:34:50 You know, you have a lot of programs end up having these main functions that are just like lists of registration calls for 100 lines of registration.

00:34:57 And then it says go after that.

00:34:58 And so meta classes give you a really nice way of saying, hey, anytime anyone creates a class,

00:35:07 it's part of this hierarchy, register it in a central database of classes or central, you know, dictionary of classes so that when I start my program,

00:35:15 I can go through and do all the housekeeping I need to do for initialization.

00:35:20 So that's one thing, lets you do initialization really easily.

00:35:23 And it also lets you do lookups really easily.

00:35:25 So you say, hey, you know, is there a class that does this in the system?

00:35:29 Yes, there is.

00:35:29 I know exactly where it is.

00:35:30 It's right here.

00:35:31 I've already imported it.

00:35:31 I've already looked at it.

00:35:32 I know what it's capable of.

00:35:34 So anytime you see yourself doing that pattern where you have one of something now and you're going to have 50 of them later and you need a registry of those things,

00:35:41 there's no reason to explicitly register things.

00:35:45 You should always just have them automatically register themselves so that it's less error prone.

00:35:49 And that's the big thing that I'm trying to kind of advise you to do here.

00:35:54 It's, you know, we're humans.

00:35:55 We leave things out all the time.

00:35:58 And so helping people make fewer mistakes is really important.

00:36:02 And so metaclasses are a great way of helping people make fewer mistakes.

00:36:05 And that's what that item is specifically about.

00:36:09 Nice.

00:36:10 The other sort of topic you talk a little bit about there is attributes.

00:36:15 I think you talk about like private versus somewhat private, you know, the sort of underscore versus double underscore attributes in there.

00:36:23 Is that right?

00:36:23 Yeah, I think it's a common thing also that people have when they come from other languages that have like strict definitions of public and private.

00:36:29 And, you know, Python doesn't have those things.

00:36:31 It has kind of best equivalents of that.

00:36:35 And the gist of it is, you know, you should just make everything public by default in almost all cases.

00:36:42 Because people are going to go in there and reach in and make things that are private, public anyway.

00:36:46 There's some nuance to it, but that's the gist of it.

00:36:49 And Guido refers to this as, you know, we're all consenting adults.

00:36:53 You know, you should let people reach into the class and use it however they want if they want to.

00:36:59 As long as you know you're getting yourself into trouble, then, you know, if you're willing to take on that risk, then it's up to you.

00:37:04 So, yeah, I think the whole idea here is that Python is a dynamic language.

00:37:08 So things are mutable and more fluid.

00:37:09 So you don't have to have this kind of draconian public-private enforcement that maybe Java programmers have.

00:37:16 It's interesting that other languages are so focused on not letting people do things by default,

00:37:22 where Python is focused on really letting anyone do anything with a class by default.

00:37:26 Yeah, that's an interesting just language philosophy statement altogether, isn't it?

00:37:31 Yeah, it is.

00:37:32 And some people say that's why Python gets difficult for really large projects,

00:37:35 because that's why defaulting to closed is good.

00:37:39 It's because once the project's a million lines of code, then, you know, it's too late.

00:37:42 I understand that.

00:37:44 But I think, you know, a million line Python project is a very large program.

00:37:48 A lot of people have never worked on something that size and probably never will.

00:37:50 So it's kind of like one size does not fit all.

00:37:53 Yeah, that's a good point.

00:37:55 I mean, I see a lot of people that learn design patterns and other things, and they just want to apply them.

00:37:59 Exactly.

00:38:00 To whatever, right?

00:38:01 But these design patterns maybe make sense on very large scales or certain circumstances, right?

00:38:06 Absolutely.

00:38:06 And that's why I think all the advice needs to be in context.

00:38:09 You know, okay, you're giving me this advice.

00:38:10 Who are you?

00:38:12 What are you doing?

00:38:12 How big is the code base?

00:38:13 You know, like, how often do you have to do it?

00:38:15 Like, I need to know those things because your design constraints are different than mine.

00:38:18 So.

00:38:19 Right.

00:38:20 How similar are we?

00:38:21 Yep.

00:38:21 So one of the sections I really liked, and I'm looking forward to digging into it more,

00:38:27 is the concurrency and parallelism section.

00:38:30 Yeah.

00:38:31 And you draw an interesting distinction between what you're calling concurrency and parallelism.

00:38:36 Can you maybe speak to that first?

00:38:38 And I think this goes back to, there's this great talk that Rob Pike, who's one of the

00:38:42 creators of the Go programming language, a language who also works at Google, he did called concurrency

00:38:47 is not parallelism.

00:38:48 It's a great, great talk to check out.

00:38:51 But yeah, it's basically, you know, concurrency is a programming pattern.

00:38:54 It's a way of doing multiple things at the same time and having tools to let you do that at the programming level.

00:38:59 So threads are one of those tools.

00:39:00 Coroutines are one of those tools.

00:39:02 And then parallelism is actually running two lines of execution on a processor at the same time.

00:39:07 And that's not necessarily a programming tool.

00:39:11 It's just the way that the program runs.

00:39:13 And Python is great at doing concurrency style stuff.

00:39:18 But Python can actually do, cannot do parallelism because of the global interpreter lock, which I'm sure people listening to this have probably heard of.

00:39:26 But that's one of its big shortcomings as a language.

00:39:29 Concurrently, you're thinking of maybe I'm going to make two web service calls at the same time and I can wait for one to come back.

00:39:37 But parallelism is I'm trying to compute this financial algorithm and it's just a computational thing.

00:39:45 And breaking that up into steps and trying to run that with threads is not really going to do much for you, right?

00:39:50 Yeah.

00:39:51 Or if you're trying to decode two frames from a video stream simultaneously on two different CPU cores, you can't do that in Python without actually going down to the C API.

00:40:00 Whereas asynchronous things, you can do it in parallel.

00:40:04 You can even run threads where that computation looks like it's happening in parallel, but it's actually not.

00:40:09 It's happening concurrently.

00:40:11 So you can't increase your throughput of computation with threads or any kind of concurrent infrastructure in Python.

00:40:19 You can't get parallelism, cannot get parallelism that way.

00:40:23 Right.

00:40:23 So your first piece of guidance was to use the subprocess module to manage child processes.

00:40:29 And those might be Python subprocesses or they might just be other executables, right?

00:40:34 Yeah.

00:40:35 Yeah.

00:40:36 And that's a cheap way to get some parallelism because, you know, at the system level, multiple processes can be doing multiple things on multiple cores.

00:40:43 So you can really take advantage of your computer that way.

00:40:45 Yeah.

00:40:46 Nice.

00:40:46 Another one that you had was to use threads for blocking IO, but to avoid threads for CPU parallelism.

00:40:52 And that kind of is a little bit what we were talking about, right?

00:40:55 Yeah.

00:40:56 Your web service calls exactly that.

00:40:57 So you can do a web service call in one thread, another web service call in another thread, and they will happen concurrently.

00:41:04 But they actually won't process in parallel.

00:41:07 So it's great for kind of blocking tasks or IO bound tasks, threads are, but they don't actually help you go faster.

00:41:15 Right.

00:41:15 So verify my mental model here.

00:41:17 I think I recall that if I have a thread in Python and it does some blocking IO, that thread will release the global lock, right?

00:41:28 Exactly.

00:41:29 Yeah.

00:41:29 So you can use threads to avoid waiting.

00:41:32 It's basically all you get.

00:41:33 But you can't use them to.

00:41:35 And so the gill, yeah, will unlock and then another thread will start running.

00:41:39 And that's nice.

00:41:41 And that's what threads are good for.

00:41:42 But that's as far as it'll go.

00:41:45 Right.

00:41:45 Sort of the Node.js style of parallelism, maybe.

00:41:47 Yeah.

00:41:48 It's very similar, actually.

00:41:49 Node has a lot of the same constraints and problems.

00:41:51 It only has one executing thread at any time.

00:41:54 Okay.

00:41:55 So another one was to use or consider coroutines for running many functions concurrently.

00:42:00 How's that look?

00:42:02 Yeah.

00:42:02 Yeah.

00:42:03 This is my favorite item in the book, item 40.

00:42:05 Yeah.

00:42:07 I implement Conway's Game of Life as a set of coroutines.

00:42:09 I would say if you understand this one, then you've understood the Xenopython pretty

00:42:14 well.

00:42:15 And it's basically just trying to say that you can express very complicated, high-level

00:42:21 ideas of the way that things should interact.

00:42:23 Whole system flows should work, workflows, using coroutines in this really abstract way that's

00:42:29 extremely powerful, easy to test, easy to expand.

00:42:33 It's really hard to go into detail without explaining the whole thing from the beginning.

00:42:36 Sure.

00:42:37 It's definitely worth checking out.

00:42:38 It's actually online, fully published.

00:42:39 If you search for it, I think you can actually find it on effectivepython.com because I'm so

00:42:44 proud of that one.

00:42:44 Yeah, nice.

00:42:44 That's awesome.

00:42:45 So you kind of pulled it out so everyone can get to it?

00:42:47 Yeah, definitely.

00:42:47 Cool.

00:42:48 And then the final concurrency one that I thought was interesting was to consider concurrent

00:42:52 dot futures for true CPU parallelism.

00:42:56 Yeah, and this is the kind of hack that someone came up with.

00:42:59 So there's this multiprocessing module that's built into Python now that will actually farm

00:43:04 out work to sub-processes that are also Python.

00:43:08 It's really ridiculous how it works, but for certain cases, it's extremely powerful, and

00:43:14 it will actually speed up your program by the number of cores on your machine.

00:43:17 It doesn't always work.

00:43:19 It also breaks in a lot of weird ways, but when it does work, it's magical because you can go

00:43:25 from something that was slow to something that's 10 times faster.

00:43:28 So it's worth checking out, at least trying.

00:43:30 I just wouldn't go too far.

00:43:31 I think you just summed up multi-threaded parallel programming right there.

00:43:36 It sometimes works, it sometimes breaks in weird ways, and when it works really well,

00:43:41 it's magical, right?

00:43:42 Yeah, it's true.

00:43:43 Yeah, that's very true.

00:43:44 I guess that's no exception.

00:43:45 Yeah, but Python has its own peculiarities about how it breaks and how it works and whatever,

00:43:51 right?

00:43:51 That's a really good point.

00:43:52 People forget to mention that, oh, Python only has one thread.

00:43:55 It's like, that is a problem.

00:43:56 It's a problem.

00:43:57 But multiprocessing, multiprocess programming, multithreading in any other language was also

00:44:01 really hard and error-prone, so let's not forget that.

00:44:04 That's a great point.

00:44:05 There was a group that was referring to the bugs that you get from multithreading as

00:44:11 Heisen bugs, and I really like that way of thinking about it.

00:44:14 Yeah, definitely.

00:44:15 They're very hard to track down.

00:44:16 So let's talk about just one more section in the book, and that was collaboration, like

00:44:22 how you work well with other people and sort of like on teams and so on, right?

00:44:27 Yeah, yeah.

00:44:28 And I think Python is a huge community, like we were talking about before, and so working

00:44:31 with others in Python is really important.

00:44:33 Yeah, that's cool.

00:44:34 And one of your pieces of guidance was to use packages rather than just patch around a bunch

00:44:39 of files, right?

00:44:40 Yeah.

00:44:41 And this all depends on the size of your code base.

00:44:43 You know, when you're starting out, it doesn't matter.

00:44:44 Like until you have 10 files or something like that, it really doesn't matter.

00:44:48 But you know, you should be willing to refactor your code into packages once you realize you

00:44:52 have enough different working pieces, especially if you're working with different people.

00:44:57 It's nice to split things a little earlier than you might have expected.

00:45:01 So you give them room to grow.

00:45:02 It's kind of like you'd buy, you know, if you have a kid, you buy them shoes a little, like

00:45:06 one size a little bit bigger so that they can grow into them.

00:45:08 Yeah.

00:45:09 So packages are kind of like that.

00:45:10 I think it's a good way to modularize your code base before it becomes too late.

00:45:15 And I've been in code bases where it's been way too late.

00:45:17 And we have files, six lines of Python code in them.

00:45:20 And you're just like, why?

00:45:20 How did this even happen?

00:45:22 It's terrible.

00:45:24 Yeah, I can imagine.

00:45:26 Do you recommend having like an internal private package server?

00:45:29 You know, it really depends on your company.

00:45:33 I think if you're comfortable with PIP, then yeah, it's pretty good, especially if you're

00:45:37 doing deployments off of it.

00:45:38 It can be really useful.

00:45:39 If you have a security team that needs to vet things, it can be really valuable.

00:45:42 Right.

00:45:44 Maybe one team that builds the API and another team that builds the web part that consumes

00:45:48 the API or something like that.

00:45:50 Yeah, you can do stuff like that.

00:45:51 I mean, it also depends on your source code repo.

00:45:53 Google, we have just one giant repo for the entire company.

00:45:56 So for us, I don't need a package server because I have the code.

00:45:58 But yeah, it's good hygiene to have people publishing APIs that are stable and to think

00:46:04 about things in that way.

00:46:05 It's kind of the Netflix or Amazon approach to software engineering.

00:46:08 And it works really well, more like a service-oriented architecture, but at the API level.

00:46:12 Yeah, that's cool.

00:46:13 So another piece of advice you had for collaboration was to consider the Reaper method for debugging

00:46:22 output and helping people understand the state of your program.

00:46:26 Yeah.

00:46:26 Yeah.

00:46:27 So yeah, Reaper Reaper print representation.

00:46:32 I don't know why they call it that.

00:46:33 I think it's pretty silly.

00:46:34 But yeah, this is the most valuable tool for me in Python.

00:46:37 Like, print is the first most valuable tool for me in Python.

00:46:40 But the second most valuable tool besides print is Reaper.

00:46:43 Because you can get detailed information about an object.

00:46:48 And you can basically make an object that'll self-describe itself so that when you're debugging, you can

00:46:53 print it out and say, hey, what's in this thing?

00:46:55 What were the parameters that created this object?

00:46:56 How did I even get here?

00:46:57 Reaper and under-under-repper is a really great way to do that.

00:47:01 And Python has built-in support for it all over the place.

00:47:03 So it's definitely worth checking out if you don't use it.

00:47:05 Right.

00:47:05 Absolutely.

00:47:06 If you just take a list and print it, you'll get that representation for all the items it

00:47:11 contains.

00:47:11 That's right.

00:47:12 And it basically evaluates back to the Python statement that it would have taken to create the

00:47:16 same list.

00:47:17 And so you usually can copy and paste reppers back into an interpreter to test them out.

00:47:22 So it's a really nice way to do kind of an interactive debugging session with your own code.

00:47:26 That's cool.

00:47:27 And the final collaboration thing that I actually hadn't heard of was you can understand memory

00:47:32 allocations and leaks with something called trace malloc.

00:47:35 Yeah.

00:47:37 And this is another Python 3 only thing.

00:47:39 And it's wonderful.

00:47:44 Because if you've ever had memory leaks in Python, for especially long-running processes,

00:47:47 it's really hard to track down where your memory usage is going.

00:47:51 Any project of size that I've been on, we run a lot of servers at Google, so we have this

00:47:56 a lot.

00:47:56 You end up having this problem where you look at your garbage collection or your heap, and

00:48:00 it just keeps growing.

00:48:03 You're like, where's the actual space going?

00:48:05 And the tools to do that in Python are horrible.

00:48:07 Python 2.7 and before, they're all bad.

00:48:10 And trace malloc, they finally fixed this in Python 3.

00:48:13 And yeah, it basically will tell you exactly like, this is the object you have too many of.

00:48:18 This is exactly where it was allocated.

00:48:19 Here are the parameters that were used at the time.

00:48:22 And so now it's like, orders of magnitude easier to figure it out, which is really great.

00:48:26 There is a port of that code to Python 2.7, I believe.

00:48:31 But I don't know how well-maintained it is.

00:48:33 And I don't think it's actually, it has to actually modify the CPython run, like binary

00:48:38 itself.

00:48:38 It can't be done as a C extension module.

00:48:40 So I think it'll never be added to Python 2, which is unfortunate.

00:48:44 Right.

00:48:45 So you might have to like build your own version of the source code with that baked in and then

00:48:49 run your code on that.

00:48:50 Which is pretty scary, but it might be worth it.

00:48:53 Yeah, maybe not for use in production too much.

00:48:55 Yeah, I wouldn't use in production.

00:48:56 For sure.

00:48:58 Okay.

00:48:59 So those are a bunch of really awesome things that I think will help, you know, pretty

00:49:04 much people of all level.

00:49:05 Thanks, man.

00:49:06 Yeah.

00:49:07 How long has the book been out?

00:49:09 It's been out since April, basically.

00:49:12 And it's doing pretty well.

00:49:15 And this week we also have a video that's coming out, which is a lot of the same content, but

00:49:20 in me just typing out the actual code format.

00:49:23 So some people who learn better from seeing demonstrations, it might be better for them.

00:49:29 But yeah, it's had a lot of good feedback, a lot of good reviews on Amazon and other places.

00:49:34 A lot of bugs reported of things I did wrong, but not too many bugs.

00:49:38 There's only one bug so far that I'm really embarrassed about where I did a multiply instead

00:49:43 of divide, but the rest of them are all pretty good.

00:49:46 Well, the multiply, divide, what's the difference?

00:49:50 Yeah.

00:49:50 You know, I just, I got confused.

00:49:52 Math is hard.

00:49:57 So people can go get it on Amazon and I'll definitely link to the book, but you have

00:50:03 a book code that we can give to listeners so they can get a bit of a discount, right?

00:50:07 Yeah.

00:50:07 And it's, it's the, the, the code is, FPI, E F F P Y. And the simplest way to use

00:50:12 it is you go to inform it.com like I N F O R M I T.com slash F PI, all caps E F F P Y.

00:50:21 And, that, that lets you buy the book directly from the publisher.

00:50:23 And for some places, that are not in the United States also, it can be cheaper

00:50:28 this way than going through Amazon.

00:50:29 and it's a, it gives a discount off the ebook version, the physical book, and then also

00:50:35 this new video that just came out this week.

00:50:36 you can get them all there.

00:50:38 Yeah.

00:50:38 You just released that.

00:50:39 You, released this, this nice new video training version of the book.

00:50:43 So, I'll be sure to link to that as well.

00:50:45 That'd be cool.

00:50:45 Yeah.

00:50:46 Thank you.

00:50:46 Yeah.

00:50:47 You bet.

00:50:48 So Brett, anything else you want to give a shout out to or let people know about that

00:50:52 we somehow didn't touch on?

00:50:53 the only other thing is, yeah, like the, the book websites, effective python.com and,

00:50:58 everything you want to know about me, you can find there, and, please report

00:51:03 any bugs you have.

00:51:04 Those are on, on GitHub with all the example code.

00:51:06 And I'm trying to make sure that, anything that's wrong, I, I, I address in there.

00:51:10 but yeah, if you have questions, there's also a link to email me directly.

00:51:13 So if you, if you hit an issue, I'd love to hear from you.

00:51:16 I'm happy to answer your question.

00:51:18 Excellent.

00:51:19 So two final questions before I let you go.

00:51:21 What is your current favorite code editor?

00:51:24 If you're going to sit down and write some Python, what do you, what do you open up?

00:51:26 I, I am a sublime fan.

00:51:28 I use text mate for a long time.

00:51:29 I still use Emacs, when I'm on a, on a terminal.

00:51:32 So I use Emacs every day also.

00:51:34 I don't use VI, but I can use less so I can, I can do the read part of it, I guess.

00:51:39 and, but yeah, sublime is my favorite, editor.

00:51:43 Nice.

00:51:44 And then of all the 60,000 plus, whatever it is, PyPI packages, what, what ones do you

00:51:52 think people should know about that?

00:51:53 Maybe they don't know.

00:51:53 NumPy, obviously, I guess that's, everyone knows about that one.

00:51:58 SQLAlchemy is an amazing tool, especially the relational package, for doing, database

00:52:04 lookups.

00:52:04 flask is a great web framework that I like a lot.

00:52:07 and then there's a bunch of stuff around, speeding up programs that I'm, I'm trying

00:52:13 to get more into.

00:52:13 Theano is one and Numba is another.

00:52:16 These are things that let you use your GPU through Python, which is super cool.

00:52:20 And you can use that for things like neural network training and machine learning and stuff

00:52:23 I haven't really spent enough time doing.

00:52:24 So I think I'm really excited to try out some more of these computational tools and Python

00:52:28 packages.

00:52:29 Yeah, that's really awesome.

00:52:30 If you, you know, people who've looked at it, they know, but if you've not looked at

00:52:35 the parallel capabilities of GPUs, it's like, it's astounding, right?

00:52:39 It's crazy.

00:52:40 And the fact that you can like use that in three lines of Python is crazy.

00:52:43 So I think it's just speaks to the community that Python has of scientists and, and, you

00:52:48 know, it's just such a, such a diverse community.

00:52:49 So it's really great to see these packages, blossoming in Python before many other languages.

00:52:54 Yeah, absolutely.

00:52:55 Brett, it's been a very interesting conversation.

00:52:58 I appreciate it.

00:52:59 Yeah.

00:53:00 Thanks a lot for having me on and asking all those great questions.

00:53:02 Yeah.

00:53:02 And I love the book and I encourage the readers to go check it out and they'll enjoy it.

00:53:07 Thanks very much.

00:53:08 Yeah.

00:53:08 Talk to you later.

00:53:09 All right.

00:53:10 See ya.

00:53:10 This has been another episode of talk Python to me.

00:53:14 Today's guest was Brett Slacken.

00:53:16 And this episode has been sponsored by hired and code chip.

00:53:19 Thank you guys for supporting the show.

00:53:21 Hired wants to help you find your next big thing.

00:53:23 Visit hired.com slash talk Python to me to get five or more offers with salary.

00:53:28 And equity presented right up front and a special listener signing bonus of $4,000.

00:53:32 Code chip wants you to always keep shipping.

00:53:36 Check them out at code chip.com and thank them on Twitter via at code chip.

00:53:40 Don't forget about the discount code for listeners.

00:53:42 It's easy.

00:53:43 Talk Python, all caps, no spaces.

00:53:46 You can find the links from the show on talkpython.fm/episodes slash show slash 25.

00:53:52 While you're there, be sure to subscribe to the show.

00:53:55 Open your favorite podcatcher and search for Python.

00:53:57 We should be right at the top.

00:53:58 You can also find the iTunes and direct RSS feeds in the footer of the website.

00:54:03 Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.

00:54:08 You can hear the entire song at talkpython.fm/home slash music.

00:54:12 This is your host, Michael Kennedy.

00:54:15 Thanks for listening.

00:54:16 Smix, take us out of here.

00:54:21 Chilled in, haven't been sleeping.

00:54:23 I've been using lots of rest.

00:54:25 I'll pass the mic back to who rocked it best.

00:54:28 Developers, developers, developers.

00:54:31 Developers, developers, developers.

00:54:38 developers.

00:54:38 you you