024_fluent_python.txt

00:00:00 Are you fluent in Python, or do you speak the language with an accent?

00:00:03 Python's ease of learning can also lead to non-Pythonic patterns for even experienced developers.

00:00:08 Luciano Romalo is here to give us a deeper understanding of this language we love.

00:00:12 It's episode number 24, recorded Thursday, August 6, 2015.

00:00:17 I'm a developer in many senses of the word, because I make these applications, but I also use these verbs to make this music.

00:00:28 I construct it line by line, just like when I'm coding another software design.

00:00:33 In both cases, it's about design patterns. Anyone can get the job done, it's the execution that matters.

00:00:39 I have many interests, sometimes conflict.

00:00:42 Welcome to Talk Python to Me, a weekly podcast on Python.

00:00:46 The language, the libraries, the ecosystem, and the personalities.

00:00:49 This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy.

00:00:53 Keep up with the show and listen to past episodes at talkpython.fm.

00:00:57 And follow the show on Twitter via at Talk Python.

00:01:00 This episode is brought to you by Hired and CodeChip.

00:01:03 Thank them for supporting the show on Twitter via at Hired underscore HQ and at CodeChip.

00:01:09 Before we get to the conversation with Luciano, he generously agreed to give away an electronic copy of his book, Fluent Python.

00:01:15 All you have to do to be eligible to win is to be a friend of the show.

00:01:19 Visit talkpython.fm, choose friend of the show in the nav bar, and sign up.

00:01:24 Now, let me introduce Luciano.

00:01:25 Luciano Romalo is the author of Fluent Python.

00:01:28 Romalo was a web developer before the Netscape IPO in 1995 and switched from Perl to Java and finally to Python in 1998.

00:01:36 Since then, he's worked on some of the largest news portals in Brazil using Python.

00:01:40 He's taught Python web development in the Brazilian media, banking, and government sectors.

00:01:45 He's spoken multiple times at OSCON, PyCon, Python Brazil, FISL, and Rupy.

00:01:50 Romalo is a fellow at the Python Software Foundation and a co-founder at the Garoa Hacker Club, the first hackerspace in Brazil.

00:01:58 Finally, he's the managing partner at python.pro.br at Python Training Company.

00:02:03 Luciano, welcome to the show.

00:02:06 Thanks, Mike.

00:02:06 Thanks for having me.

00:02:07 Yeah, I'm really excited to talk to you about your new book, Fluent Python.

00:02:11 Yeah, I'm excited about it too.

00:02:13 I bet.

00:02:14 You've been working on it for a while.

00:02:16 Before we get into talking about your book, which is really an amazing book, I'll have more to say on that in a minute.

00:02:21 Thanks.

00:02:22 You're welcome.

00:02:23 Let's talk about how you got started in the programming and Python at all.

00:02:27 You know so much.

00:02:29 I can tell from your book.

00:02:30 But at some point, you had to start at the beginning, like we all do, right?

00:02:34 Where did you get started?

00:02:36 Okay.

00:02:36 Now, I'm 51 years old.

00:02:39 So when I was a teenager in Brazil, there were no microcomputers.

00:02:44 But I got a hand on a programmable calculator, a TA-58.

00:02:51 And I learned to program by programming it.

00:02:54 And actually, my first interesting program was a port of a game from the HP-25 calculator to the TI-59.

00:03:02 It was the lunar landing game that was famous.

00:03:06 Oh, that was a great game.

00:03:07 Yeah.

00:03:08 So that was my first significant program.

00:03:12 And then I was an exchange student in 81 in the US.

00:03:16 I lived in Little Harrisburg, Illinois, population 10,000.

00:03:21 And there I got – there were – in the library, in the school, there were a few Apple II computers that had just arrived when I arrived.

00:03:31 And nobody knew how to use them.

00:03:33 The teachers who were supposed to teach about them hadn't yet taken the course to do it.

00:03:40 So they were basically free for anyone to do whatever they wanted.

00:03:43 And I learned – I taught myself basic programming in them.

00:03:48 Anyway, so that's a long time.

00:03:50 And then I went through lots of languages after basic, Pascal.

00:03:54 And Pascal was one of the most important ones for me.

00:03:58 And then Delphi.

00:04:00 And then Visual Basic.

00:04:02 And I studied a little bit of small talk.

00:04:07 And then when the web started here in Brazil, again, we had a kind of a different situation because there was – all telecom companies in Brazil were state-owned at that time.

00:04:19 Later they were privatized, but they were state-owned in 94.

00:04:24 And so I was hired by a large publishing company here to start developing their strategy for online.

00:04:30 And I learned – I taught myself pro programming because everybody did everything on the web that was automated using Perl at the time.

00:04:37 Right?

00:04:38 Right.

00:04:39 And I loved Perl.

00:04:41 I thought it was awesome.

00:04:42 But I also found it – well, I'm saying I just said I love it.

00:04:48 But I loved it, but I also hated it at the same time because it was very powerful and quick.

00:04:53 But it was also full of traps.

00:04:56 And it was difficult to read afterwards and so on.

00:05:01 Then I tried Java and I probably was one of the first people in Brazil to do server-side Java because when Java was first released, Sun was trying to market it as a tool for client-side programming, right, with the applets and so on.

00:05:18 It took a while for them to realize that server-side was where Java was going to thrive.

00:05:24 But I didn't enjoy the verbosity of Java.

00:05:27 I liked the fact that it was really object-oriented, which Perl was not at the time.

00:05:32 But it was too verbose.

00:05:34 And then after a while, I went back to Perl again.

00:05:37 And Perl had acquired object-oriented features in around Perl 5.

00:05:43 And then something happened.

00:05:45 As the Perl community was discussing how to do things in a dynamic language, but also in an object-oriented way,

00:05:54 they kept repeating the sentence.

00:05:58 Here's how they do it in Python.

00:06:00 In Python, this is like that.

00:06:03 And that's how I first heard of Python.

00:06:05 It was discussions on the Perl mailing list about how to do things in an object-oriented way,

00:06:13 but in a language that was a scripting language, in a dynamic language.

00:06:16 So after reading mentions of Python a few times, I decided to go study it.

00:06:20 And then I read the tutorial, which was shorter at the time than it is now.

00:06:25 And I just fell in love immediately because for me, it had the best features of Python, of Java and Perl.

00:06:35 It was really object-oriented with an object-oriented standard library like Java and had exceptions and other things that I liked about Java,

00:06:47 but was not verbose.

00:06:49 So it was, you know, very concise and to the point and agile like Perl.

00:06:57 And so it was for me the combination of the best features of both languages.

00:07:02 And then I basically never looked back.

00:07:04 That's a great story.

00:07:06 I think we've all had those sorts of feelings about some of our first programming languages.

00:07:12 We love them because we really got into programming, but, you know, they're not necessarily the best.

00:07:17 So you have those words as well, right?

00:07:19 Exactly.

00:07:20 But Python's where you've sort of settled into.

00:07:24 I can tell you've been doing it for a long time from your book.

00:07:26 Yeah.

00:07:28 So I started doing it in 98.

00:07:30 That was when I found the tutorial that I mentioned.

00:07:33 And then I immediately pitched a Python solution to a client at the time for an online news site.

00:07:43 And then I discovered Zope, which was just then released as open source.

00:07:51 And I started using it like a couple weeks after it was released as open source.

00:07:58 And then a couple months later, we actually launched in Brazil a new site that was based on Zope.

00:08:05 And it was one of the first big Zope cases around the world.

00:08:09 Oh, that's excellent.

00:08:11 Yeah.

00:08:11 And so I owe a lot to Zope.

00:08:13 I don't use Zope anymore these days, but I owe a lot to it because it was because of Zope that I was able to deliver the kinds of products that the clients were looking for me to deliver, like content management systems.

00:08:29 And so I was able to get paid for writing Python pretty early, at least in the context of Brazil, since 98.

00:08:39 That's a great feeling, right?

00:08:40 Yeah.

00:08:41 You find this thing you love and, wow, people pay me to do it.

00:08:43 Exactly.

00:08:44 Exactly.

00:08:45 It's a good time to be a programmer indeed.

00:08:47 So let's talk a little bit about your book.

00:08:49 The title is Fluent Python.

00:08:51 And it's coming out under, being published by O'Reilly.

00:08:54 And it's coming out, I think, September 4th is what Amazon.com tells me anyway.

00:08:58 Is that right?

00:09:00 Yeah.

00:09:00 To be honest, I don't know exactly when the print book is going to come out because different pages that I look at give different dates.

00:09:06 I've seen August.

00:09:07 I've seen September.

00:09:08 And I've seen October all in the last two weeks.

00:09:11 So, yeah.

00:09:14 But soon, right?

00:09:15 Pretty soon.

00:09:16 Just about the same time that this podcast comes out, I think, should be the time that the book is released.

00:09:20 Wow.

00:09:21 That's excellent.

00:09:22 But let me say, the e-book is now complete in the final form.

00:09:29 So people who buy the e-book or who have bought it previously can now download the final version of, you know, first edition, first revision.

00:09:40 And it's also available from Amazon.com for the Kindle in the final form.

00:09:47 So we are only waiting for the print book at this time.

00:09:50 Right.

00:09:51 Okay.

00:09:51 Excellent.

00:09:52 Yeah.

00:09:52 I think having the electronic book works really well.

00:09:55 I was reading it on my Kindle and it, on my Kindle, paper white, and it was really nice.

00:09:59 You know, the code samples came through really well.

00:10:02 Like sometimes that doesn't work so well.

00:10:04 Wow.

00:10:04 That's cool.

00:10:05 Nice to know.

00:10:05 Yeah.

00:10:06 Yeah.

00:10:06 It came out really, really well.

00:10:07 And, you know, so I've only had the book for a few days.

00:10:11 So I've only been able to read maybe the first five chapters.

00:10:14 But my impression of the first five chapters, if it keeps going like this, I'm sure it does, is this book is a masterpiece.

00:10:22 It is really, really good.

00:10:25 Well, thank you very much.

00:10:26 I think it's going to go down as a classic book in the Python community.

00:10:29 Really.

00:10:30 Thank you very much.

00:10:31 Yeah.

00:10:31 I'm not just saying that because you're on the show.

00:10:33 But I was reading it and it's a little hard for me to put in words, I think, for people to really understand.

00:10:41 But it seemed like everything that you covered, and we'll talk a bit more about that.

00:10:46 But it seemed like everything you covered, I'm kind of like, I mostly know this.

00:10:49 Oh, wait.

00:10:50 There's this really cool piece of information or motivation that I was never really aware of.

00:10:57 And just underneath the surface of the knowledge that I do have.

00:11:02 Right.

00:11:02 And that was like all the time happening to me while I was reading your book.

00:11:06 And so I think it's a great contribution.

00:11:08 Thank you.

00:11:09 Yeah.

00:11:10 So let me read a quick quote right from the beginning to give people a sense of, I think, what you're going for with this book.

00:11:17 So in the introduction or one of the first chapters, you say, Python is an easy-to-learn, powerful programming language.

00:11:24 Those are the first words of the official Python tutorial.

00:11:26 That's true, but there's a catch.

00:11:28 Because the language is so easy to learn and put to use, many practicing Python programmers leverage only a fraction of its powerful features.

00:11:35 Does that kind of sum up the problem you're trying to solve with this book?

00:11:40 Exactly.

00:11:41 And that's why I called it Fluent Python.

00:11:42 You know, as somebody who is fluent in two languages, in Portuguese and English, and that knows how difficult it is to become fluent in a language,

00:11:52 because it's easy to become fluent in the language in your native language, right?

00:11:56 You grow with it, and that's basically people don't even notice.

00:12:01 But getting fluent in another language is hard.

00:12:05 And so I've been teaching Python to lots of people.

00:12:11 When I had a company, whenever we hired people, we never could hire people that knew Python because nobody knew Python at the time.

00:12:18 So we hired people who knew Java and then taught them Python.

00:12:22 But I also worked as a Python instructor, and that's what I do now these days.

00:12:28 So I realized that this thing that I say over there, that the language is easy to use, it's practical.

00:12:36 So people start solving real problems real fast.

00:12:40 But that means that sometimes they are already using it for several years and don't know why some things are the way they are.

00:12:47 And another thing that I say right after that in the second paragraph is that when you're learning a new language,

00:12:57 you look for things that are familiar to you.

00:13:01 Like, for instance, maybe this language, probably this language has a way to use a regular expression.

00:13:08 Let's see how that works.

00:13:09 But if there is something in the language that is not matched by something that you know from another language,

00:13:17 like, for instance, the concept of generators that many languages don't have,

00:13:22 then you're unlikely to go look for it, right?

00:13:25 Because you don't even know the thing exists.

00:13:27 Another simpler example than generators is like tuple unpacking.

00:13:32 It's just a nice synthetic sugar.

00:13:35 But if you've never seen that in a language, you won't look for it.

00:13:39 And maybe you're going to spend years working with Python without ever using it.

00:13:43 But it's a really nice thing to use because it makes code more readable, even safer in some ways.

00:13:49 Python tuple unpacking has this characteristic because it actually forces the interpreter to count the number of items.

00:13:59 So if there's one more or one less, an error will be raised.

00:14:03 So it's a very nice feature, but it's something that people who come from other languages may never use because they won't think about it.

00:14:10 Right.

00:14:10 If you came from something like C#, for example, that doesn't have this concept of tuple unpacking,

00:14:15 and so you just wouldn't even, you might not ever find it.

00:14:19 And that kind of gets at the broader issue or concept of Pythonic versus non-Pythonic code, right?

00:14:29 Yes.

00:14:29 Yes.

00:14:30 It's pretty easy to come over to Python and learn it really quickly so that you can do the same thing that you do.

00:14:37 But it's not always the best, is it?

00:14:39 Some of your mental models don't really carry over so well.

00:14:42 Exactly.

00:14:43 Yeah.

00:14:43 Yeah.

00:14:44 It's very, it's, of course, it's a very elusive thing to define what is Pythonic.

00:14:51 I actually mentioned it several times in the book and tried to address it afterwards.

00:14:58 But basically, I just send people links to other people that I respect a lot in the Python community who have attempted to define it directly.

00:15:11 Right.

00:15:11 But it's like, it's about a good definition which doesn't really explain what is Pythonic is to say it's idiomatic Python, right?

00:15:20 Right.

00:15:21 But okay, so what is idiomatic Python?

00:15:23 Right?

00:15:24 It's hard to define.

00:15:27 Yeah.

00:15:28 I think your best bet is to show a bunch of people a bunch of examples, say, all of these things are Pythonic.

00:15:34 All of these things are not Pythonic because they carry over concepts that work well in other languages but are not the proper way to do them here.

00:15:44 And once you get enough experience of going, bad example, good example, then you kind of get this sense, like almost a smell for like, yeah, it seems like this is right.

00:15:54 It seems like that's not right.

00:15:55 But it's very hard to be very concrete.

00:15:57 Exactly.

00:15:58 Yeah.

00:15:59 Yeah.

00:15:59 I tried to do that in the book.

00:16:01 And actually, a decision that I made early on was to be as Pythonic as I could in the program examples, even when being Pythonic might not have provided the clearest source code from something that's not familiar with the language.

00:16:20 You know?

00:16:21 Like, for instance, right in the first example in the book, I use named tuple, right?

00:16:29 Yep.

00:16:29 And then I give it just a couple sentences to explain.

00:16:34 It's probably not enough if the person has never seen it, but it's actually something useful that I start using all over the book.

00:16:41 And if that picks up the person's curiosity, then the person can go and look for it.

00:16:48 So this is what I decided.

00:16:50 Like, for instance, before I actually have a chapter that formally discusses generated expressions, I use them in the book whenever it was natural to do so.

00:16:59 Even before I actually cover it.

00:17:02 Because of what I just said.

00:17:03 You know, I wanted to write idiomatic Python whenever I could, even if that made the...

00:17:09 Because the thing is, when you're speaking to a native speaker of English, for example, the person won't dumb down his or her speech for you, right?

00:17:21 That's right.

00:17:22 And that's cool.

00:17:23 Sometimes maybe you don't understand everything, but you're catching, you know, there's something new about the way this person is expressing herself.

00:17:33 And then you're going to try and find out what that means.

00:17:37 That's, you know, if you always talk to people that spoke English, easy English, you would only learn easy English and not real English, fluent English.

00:17:49 You wouldn't get that chance to grow into a fluent speaker, kind of like you said.

00:17:53 So your example with the name tuple kind of hits on the point that I was talking about.

00:17:58 Like, I've used name tuples.

00:18:00 I'm familiar with them.

00:18:01 I think they're great.

00:18:02 But, you know, you go a little bit deeper and you're like, hey, it's really cool to use a name tuple because the actual dictionary that stores the attributes is stored in the class object, which is a singleton in the runtime.

00:18:16 Not in the individual instances.

00:18:17 So storage is actually cheaper than regular custom classes.

00:18:21 Yes.

00:18:21 So if all you want to have is just a bunch of pieces of data that have names in an object, but not functionality, maybe you should use these name tuples.

00:18:31 This episode is brought to you by Hired.

00:18:44 Hired is a two-sided, curated marketplace that connects the world's knowledge workers to the best opportunities.

00:18:50 Each offer you receive has salary and equity presented right up front, and you can view the offers to accept or reject them before you even talk to the company.

00:18:59 Typically, candidates receive five or more offers in just the first week, and there are no obligations.

00:19:05 Ever.

00:19:05 Sounds pretty awesome, doesn't it?

00:19:07 Well, did I mention there's a signing bonus?

00:19:10 Everyone who accepts a job from Hired gets a $2,000 signing bonus, and as Talk Python listeners, it gets way sweeter.

00:19:18 Use the link Hired.com slash Talk Python To Me, and Hired will double the signing bonus to $4,000.

00:19:25 Opportunities knocking.

00:19:27 Visit Hired.com slash Talk Python To Me and answer the call.

00:19:31 Exactly.

00:19:41 Yeah.

00:19:42 And another thing.

00:19:43 They are, because they are inexpensive, you can use it.

00:19:47 And they are inexpensive and they are completely compatible with tuples, right?

00:19:52 You can unpack them, you can iterate over the items and so on.

00:19:55 So, for instance, whenever you need to return more than one value from a function, I recommend you return a name tuple.

00:20:03 Because it will make, for instance, understanding what the function does easier, because there's going to be the definition of this name tuple.

00:20:11 If the person is looking at the result of the function in the console or in the debugger, the person is going to actually see the names of the parts of the tuple that the function is returning.

00:20:24 So, this is a recommendation.

00:20:26 Like, for instance, if you're going to return more than one value from a function, usually people do it with a tuple, and that's fine.

00:20:34 But with the name tuple is even better.

00:20:36 And it's cheap, like you said, because of the way it's implemented.

00:20:40 It doesn't require more memory.

00:20:44 Yeah.

00:20:45 You don't have to store all the keys, copies of the keys for every instance, right?

00:20:49 Like you would with regular classes or objects.

00:20:52 Yeah.

00:20:53 And, of course, when those come back, if you're unsure what's coming back from that method, you could print it out and you get a nice listing of actually not just what's in there, but the meaning of them by the name.

00:21:02 Exactly.

00:21:03 Exactly.

00:21:03 Or if you're in a debugger like PyCharm, you could probably hover over it and actually see it pop up in the watch window.

00:21:10 Yeah, it's excellent.

00:21:11 Yeah.

00:21:11 Yeah.

00:21:12 So, one thing that you said you used in your book is something called Doctest to check the code in the console listings.

00:21:18 Can you talk about that a little bit?

00:21:20 Yeah.

00:21:20 So, that's something that I think was created by the people in the ZOOP community.

00:21:28 And it's not...

00:21:32 So, in the ZOOP community, this was always very big.

00:21:35 So, the main idea of Doctest is to make sure that whatever code snippets appear in documentation, particularly listings of console sessions demonstrating stuff, that those actually match the behavior of the program at that time.

00:21:56 So, this was the starting point for Doctest.

00:22:01 In the ZOOP community, which is still active, there's a lot of development going on still in ZOOP.

00:22:07 There's a lot of also files that are testing files that are not strictly documentation that are written in the Doctest format.

00:22:18 Because it's really easy to...

00:22:19 Because it's really easy to...

00:22:20 I find it excellent to...

00:22:22 When I have to design a new API.

00:22:24 Because I...

00:22:27 It's sort of a BDD thing, you know?

00:22:30 You start typing those fake console sessions where you're thinking how it would feel to interact with this API.

00:22:41 And then the result would look like that and so on.

00:22:44 And then you...

00:22:46 It's really good to do TDD.

00:22:49 Because you write a couple of stanzas of Doctest and then you implement the code that makes it pass and so on and on and on.

00:22:59 You know, the whole idea of trying to envision what your API looks like from the outside and then implement it generates much cleaner APIs in the end.

00:23:10 And this is, you know, one flavor of that, right?

00:23:12 That's cool.

00:23:12 Yeah.

00:23:14 One thing, though, I have to say about Doctest is that after a while you'll realize there are some limitations to it.

00:23:20 And even recently, like last week, I was...

00:23:24 I had to come up with a difficult solution to an algorithmic problem.

00:23:29 And I started implementing some tests using Doctests.

00:23:34 But then, for instance, Doctest doesn't let you easily run just one part of the test.

00:23:40 It runs everything from top to down.

00:23:43 And sometimes when you're debugging, it's useful to be able to run just a specific part.

00:23:48 Anyway, the unit testing frameworks, not only unit tests, but py tests, which is my favorite, are much more powerful than Doctests.

00:23:59 And they can actually also run Doctests.

00:24:02 So there is a way to integrate them.

00:24:04 So you can have...

00:24:05 I think the best idea these days is to really use Doctests just for the documentation.

00:24:13 And just the straightforward cases, not corner cases, not weird outcomes, not testing whether an exception is raised or not.

00:24:23 Because although all of that can be done in Doctests, in the end, it becomes difficult to maintain the tests.

00:24:30 Right.

00:24:30 Maybe it's better done in pytest, something like that.

00:24:32 Exactly.

00:24:33 Yeah.

00:24:33 I use Py.Test.

00:24:35 I prefer it much better than unit tests because I think unit tests has a very strong Java flavor of forcing you to do everything inside classes.

00:24:44 Mm-hmm.

00:24:44 And I much prefer the syntax that pytest introduced and the nodes also supports.

00:24:50 But I prefer Py.Test of having tests as functions.

00:24:54 And you can create classes if you want, but you can also have module level things.

00:24:58 I prefer the syntax.

00:25:01 Yeah, cool.

00:25:03 So one of the very first things that you talk about in your book is the Python data model.

00:25:08 And you talk about it's sort of the foundation for making your own classes fit this Pythonic programming model.

00:25:17 And you have a quote in there.

00:25:18 It's something like the – you know, talking about this data model or just learning about it is the tip of the iceberg.

00:25:25 So the iceberg is called the Python data model, and it describes the API that you can use to make your own objects play well with the most idiomatic language features.

00:25:33 Yes.

00:25:34 Mm-hmm.

00:25:35 Do you think people are – people who are sort of come to Python from other languages, you know, ones that add functionality to their classes very much by overriding virtual methods and those kinds of things sort of miss out on some of the Python data model, like under-under init or, you know, the iter or str methods, things like that?

00:25:57 Yeah, for instance, this is something that happened to me many years ago when I developed the French deck, the card deck example that I use in that chapter.

00:26:08 So many years ago when I had the idea of implementing that, so I was – the idea was to show how easy it was to create a new class that behaved like a sequence, right?

00:26:25 So you implement dunder lan and you implement dunder get item and basically you are done.

00:26:31 But then something really interesting happened because I was implementing – the idea was to implement a card deck, right?

00:26:37 So what is an obvious thing you want to do with a card deck is to shuffle it, right?

00:26:41 Right.

00:26:42 So the first time that I created this example, I actually had a method called shuffle, which was not a special method.

00:26:50 It was not dunder shuffle.

00:26:51 It was just shuffle, right?

00:26:53 So the idea was – so you have this card deck and you can look at the cards and you can call the shuffle method and see the cards shuffle.

00:27:00 Okay.

00:27:01 But then a few weeks after that, I realized that when I looked back at the example given a class, I said, wait, wait a minute.

00:27:14 There is already a shuffle function in the random module that shuffles any mutable sequence that you give it.

00:27:27 So it actually doesn't make sense to implement the shuffle.

00:27:31 If the whole point of my exercise was to show how easy it was to create a sequence, even if the sequence represented a card deck, a very specific domain, specific thing, it didn't make sense for me to implement my own shuffle method.

00:27:47 Because there was already a shuffle method and it is designed to work with mutable sequences.

00:27:54 Right.

00:27:55 Right.

00:27:55 And it's already tested, probably optimized, all that kind of stuff, right?

00:27:58 Exactly.

00:27:59 So then that was an insight for me of an advantage of being Pythonic.

00:28:05 It's the fact that when you are Pythonic, your objects suddenly become more compatible with things that are already out there in the standard library and in other packages that you can download from PyPI.

00:28:19 Yeah, I think this is a lesson that people need to learn.

00:28:24 Because, for instance, I don't know about C#.

00:28:29 I studied Java a lot for a while.

00:28:31 And even today, sometimes I spend some time every year I try to sit down and read something about Java.

00:28:39 Because I get a lot of clients in my course that studied Java before and it's useful to make analogies.

00:28:47 But, for instance, a big advantage of Python over Java is the fact that we have these operators that are more generic.

00:28:55 Like, for instance, the square bracket operator, which is the operator to get an item or to set an item, right?

00:29:03 And they don't have that in Java.

00:29:07 That operator in Java is a magical thing that only works with the primitive arrays.

00:29:14 Anything else, you have to use some other, you know, you have to use method syntax.

00:29:20 Also, these days I'm studying Go, the Go language, and I really like it very much.

00:29:27 But one of the things that I most dislike about it is the fact that it follows this Java philosophy of having some built-in data structures that are magical.

00:29:38 And you cannot create your own data structures that emulate them precisely.

00:29:43 Like, for instance, in Go, there's a way to...

00:29:48 You can iterate over some primitive types, like maps and arrays and slices.

00:29:55 But you cannot create your own type that is iterable using the for-range construct of the language.

00:30:02 Right.

00:30:03 That's too bad.

00:30:04 Unlike the for-in-loop in Python, which is totally accessible.

00:30:08 Yeah, so I think Java, I think Python is actually better than most languages in this regard, of enabling you to create structures that behave exactly like the most fundamental data structures in the language.

00:30:22 And the way to accomplish that is through the data model.

00:30:25 So, I think it's very nice to have that documented and exposed, and I wanted to give it more visibility.

00:30:33 So, that's why I started the first chapter with that.

00:30:35 Even if in the following chapters, my strategy then is to say, okay, but let's look at what is already implemented in Python.

00:30:44 Because I didn't want people to go out and reimplementing new sequence types when maybe they don't even leverage everything that the built-in sequence types can offer.

00:30:55 Right?

00:30:56 Right.

00:30:56 I think that's a good point.

00:30:57 People coming from other languages, they don't necessarily know what's available.

00:31:01 So, they'll find themselves reimplementing stuff.

00:31:03 A really simple example would be I would like to randomly get an item out of a list.

00:31:09 So, I might import the random module, get a random index based on the length, and then go index it out.

00:31:14 Or you could just do random.choice.

00:31:16 Oh, I didn't know random.choice exists.

00:31:18 Exactly.

00:31:19 Yeah.

00:31:19 Yeah.

00:31:20 So, one thing that you talk about around the Python data model is the len method.

00:31:25 Mm-hmm.

00:31:26 Yeah.

00:31:27 And, you know, being such an object-oriented language, I always felt that it should be, you know, collection.len or length or something like that, maybe even as a property.

00:31:38 And this sort of special function on the outside of the classes seemed a little bit odd.

00:31:43 But you had a good conversation around that in there.

00:31:46 You want to elaborate on that?

00:31:47 Yeah.

00:31:48 Sure.

00:31:48 So, that is also something that bothered me for a while.

00:31:52 Of course, you get used to it because it's really easy to get used to it and it's really easy to type.

00:31:58 It's actually one of the arguments that Guido uses to defend that decision is that it's easier to type than having a dot.

00:32:05 You know, if it was a method call, you would have to write, like, you know, x.len open close paren.

00:32:14 Right?

00:32:15 So, there's one dot extra there.

00:32:19 And it's also something.

00:32:21 And here's another thing that's funny.

00:32:23 Because I think it's of the human nature.

00:32:28 When people come to learn a new language, sometimes they are not pleased.

00:32:33 Sometimes their boss told them to learn it.

00:32:35 Right?

00:32:37 And then, some people try to resist by pointing out, you know, things that they don't like or things that they think are inconsistent.

00:32:46 And bugs, although bugs are extremely difficult for a beginner to find in Python.

00:32:52 I've never seen a beginner find a bug in Python.

00:32:55 Every time a beginner thinks… I've seen them write bugs.

00:32:57 Yeah, yeah.

00:32:58 Sure.

00:32:58 Yeah.

00:33:00 But most of the time, in my life, every time I've seen a beginner complain about a Python bug in a mailing list, it was invariably because the person didn't understand what exactly was going on.

00:33:11 That's right.

00:33:11 It was not a real bug.

00:33:12 But anyway, here's about inconsistencies.

00:33:15 So, they say, ah, this is inconsistent because it should be spelled as a method.

00:33:21 Okay.

00:33:22 Now, let's think about how this problem is solved in Java.

00:33:26 In Java, there's actually an inconsistency because there is the… in the array type, there is like a… a property, as it were.

00:33:37 It's actually a field, like a public field that you read.

00:33:41 So, you don't… you just write myarray.length without parentheses, and that's how you get the length of the array.

00:33:50 Because Java is just basically giving you the value that's… it's internally stored in a structure that represents the array.

00:33:58 So, that's very cheap, and that's how they do it.

00:34:01 And it's important to be cheap because it's a very common operation.

00:34:05 Then, for other types that don't have this structure, there's a method called length.

00:34:12 But also, in the Java API, there are other classes that could perfectly have methods named length, but have methods named size.

00:34:21 Size or count or something, yeah.

00:34:23 Yeah.

00:34:24 So, there's at least three different ways that are common in Java code to do the same thing.

00:34:32 And in Python, we have only one way.

00:34:34 It's the length of the thing, called as a function.

00:34:37 And so, first of all, it's more consistent.

00:34:41 The second thing is, it's pragmatic.

00:34:44 Because the way it works is, if you say length of X, and X is a built-in type, this built-in type, if the built-in type has many elements,

00:34:56 there's actually a struct, I forgot the name, but it's spelled out in the book,

00:35:01 there's actually a C struct that represents the type, and it has a field that has the count of the numbers, of the number of items.

00:35:08 So, that's the same thing that Java does, except that it doesn't expose a special field, but uses the length function.

00:35:15 So, the length function, the implementation of the length function in the Python interpreter does that.

00:35:20 It just goes to look at, if there is a built-in type, and it's a multi-valued built-in type,

00:35:26 then return the value of this field in the struct.

00:35:29 So, that's very cheap, and it's actually faster than doing the resolution of an attribute, right?

00:35:36 So, because if you had to write X dot length, this dot requires some advanced machinery in Python,

00:35:43 because of the dynamic nature of Python.

00:35:45 The price we pay for all this dynamic behavior is that the dot operator is kind of expensive.

00:35:52 Right.

00:35:52 Basically, a dictionary lookup, and then the function calls themselves are also kind of expensive.

00:35:57 Exactly.

00:35:58 Yeah, and it's more expensive than a dictionary lookup, because there's all that machinery that I explain later in the book.

00:36:08 Basically, in the last part of the book, the metaprogramming part, is where I explain actually how descriptors work.

00:36:16 And descriptors, the infrastructure below properties, but also below another thing that's really fundamental that everybody uses in Python,

00:36:27 which are methods.

00:36:28 Because what makes a method behave as a method, as an instance method, as a bound method, or as an unbound method,

00:36:38 has to do with the way the descriptor mechanism resolves the dot operator.

00:36:45 So, it's an expensive thing.

00:36:47 So, by making it be written as len paren X, close paren, the interpreter can resolve this much faster.

00:36:58 And then, if the structure is not a built-in structure, then the interpreter falls back to looking up,

00:37:06 and then there's the cost of the attribute lookup, but it will look up the under len method.

00:37:15 So, I think this is actually very elegant, because in the end, the user interface is always the same.

00:37:21 It's always len of X.

00:37:22 But it allows, but it gives us, creators of APIs, a way to provide a consistent interface.

00:37:30 An interface that's consistent with the built-in method, with the built-in types,

00:37:35 while at the same time ensuring that the built-in types have the fastest possible performance for doing this operation,

00:37:42 which is crucial, right?

00:37:45 It's often something done in loops, right?

00:37:47 And you want to optimize everything that's done in a loop.

00:37:51 So, getting the lane of something is important to be fast.

00:37:54 This episode is brought to you by Codeship.

00:38:11 Codeship has launched organizations, create teams, set permissions for specific team members,

00:38:16 and improve collaboration in your continuous delivery workflow.

00:38:19 Maintain centralized control over your organization's projects and teams with Codeship's new organizations plan.

00:38:25 And, as Talk Python listeners, you can save 20% off any premium plan for the next three months.

00:38:31 Just use the code TALKPYTHON, all caps, no spaces.

00:38:35 Check them out at CodeChip.com and tell them thanks for supporting the show on Twitter where they're at, CodeChip.

00:38:45 Yeah, that's excellent.

00:38:46 And you talked a lot about the internals, which I think can be kind of tricky to discover in Python.

00:38:52 The previous show that I recorded but is not yet released is with Philip Guao, all about this university class he did and then put online on the internals of Cpython.

00:39:05 I think he did a 10-hour series on like walking through Cpython.

00:39:08 And I just – I want to watch that.

00:39:11 Yeah, I just want to recommend to listeners like if that kind of stuff that we were just talking about sounds a little like, I'm not really sure where this is going.

00:39:18 Like watch Philip's videos, right?

00:39:20 Maybe I'll put them in the links here because it'll all of a sudden click together and be like, oh, I see, exactly.

00:39:25 And you talk a little bit about the same kinds of things that he does by importing the disassembly module, the dis module, and disassembling functions and actually looking at what it is you're talking about to see what's really happening.

00:39:37 Yeah.

00:39:38 The thing is I'm not – to be honest, I'm not an authority on the internals of Python.

00:39:46 But sometimes I was curious about some things and I went to look for it.

00:39:50 But I also want to tell your listeners that the book is not about that kind of thing.

00:39:56 It has some moments that has these kinds of look into, you know, under the covers.

00:40:03 But most of the time it's not like that.

00:40:05 It's something that I enjoy, but I also found it challenging to read the source code for Cpython.

00:40:15 So I would love to see Python developing – here's a wish that I have.

00:40:21 I would love to see some time in the future, perhaps, a Python interpreter written in Go.

00:40:29 Because Go is much more readable than C.

00:40:32 It's simpler also.

00:40:34 The syntax is simpler.

00:40:35 But it has the problem of concurrency resolved in a very good way.

00:40:41 And it would be awesome to see a Python interpreter written in Go.

00:40:45 It would make it also much easier to understand what's going on.

00:40:48 Yeah.

00:40:49 How about something – a future Python that the runtime parts are easier to read because it's Go.

00:40:56 And it doesn't have the global interpreter lock.

00:40:58 Exactly.

00:40:59 Maybe because you can somehow resolve that with simpler threading models in Go.

00:41:03 Yeah.

00:41:04 It's important to – since you mentioned the global interpreter lock, you know, the JITON and the Iron Python implementations of Python don't have it.

00:41:14 Because they already can count on an underlying implementation of objects and threads that is thread safe.

00:41:21 So it's something that happens with CPython and also with PyPI.

00:41:28 But Iron Python and JITON don't have the global interpreter lock.

00:41:32 Oh, that's interesting.

00:41:33 Yeah.

00:41:34 So basically there's two cases of this is a possible thing.

00:41:38 It's just we're not there yet, right?

00:41:40 Yeah.

00:41:41 No.

00:41:41 Not at all.

00:41:42 Yeah.

00:41:45 Let's move on maybe to like the array section and the sequences.

00:41:48 You did some really interesting things there, a lot of which was kind of familiar to me like list comprehensions and generator expressions.

00:41:58 Those are – Yeah.

00:41:59 You know, if you've been doing Python for a while, it'll make a lot of sense.

00:42:02 But then you also have like little gems in there.

00:42:04 Like one of them that really stuck with me is said basically if you implement the iter method, you can automatically do iteration.

00:42:14 But if the collection you have doesn't have a dunder contains, then any sort of is check will actually be a sequential scan.

00:42:23 An in check, not is.

00:42:26 Yeah.

00:42:26 Oh, sorry.

00:42:27 Yes, of course.

00:42:27 Yes, an in check.

00:42:28 So if you say object in sequence and it does – it implements iteration but not the contains, all of a sudden – it can still answer the question, but it's a huge performance problem potentially.

00:42:39 Yeah.

00:42:40 Yeah.

00:42:41 And that's because if you look on – right at the second page of chapter two, there's a diagram of the – of the mutable sequence and sequence ABCs and then container iterable and sized.

00:42:56 And so those ABCs pretty much define the standard API for mutable sequences.

00:43:02 And in the sequence ABC, contains is a concrete method.

00:43:08 So even though that's an abstract class, the only abstract method in it is dunder get item.

00:43:14 But dunder contains is already implemented.

00:43:17 And the way it's implemented is by just using iteration.

00:43:21 Yeah.

00:43:22 This is one thing that I – that – one thing that makes me really like Python is the fact that as a teacher, I found it easier to explain over the years.

00:43:33 Although it has also grown more features that take a lot of time to explain.

00:43:38 But for a certain specific set of features, for instance, if you're talking about sequences, these days it's easier to talk about sequences, I think, than it was before.

00:43:51 Because although there's more content, you have to talk about comprehensions and so on.

00:43:55 The basic concept of a sequence was not very well defined earlier in the language because there were no ABCs.

00:44:02 In the documentation, you read about, oh, so this expects a sequence and that's – but there was no strict definition of what a sequence was.

00:44:11 After a while – It's like a vague concept of, well, a sequence is kind of these – yeah, what does that actually mean precisely?

00:44:17 Exactly.

00:44:17 Yeah, exactly.

00:44:19 So people who were comfortable with the language intuitively knew what a sequence was.

00:44:24 People who knew about the implementation of the language in C knew exactly what a sequence was, but in a way that was very complicated to convey.

00:44:33 And now with the introduction with the ABCs, it's easier because you can say, no, so here is what a sequence is.

00:44:40 A sequence has to have a way of – it's a container, an iterable-sized container.

00:44:46 So it has to be a way to – there's got to be a way to get the lane, to iterate over it and to determine whether something is or is not in it.

00:44:55 And then it has to have a – so there's a definition that people can look up in the documentation.

00:45:03 And in the book, I drew some UML diagrams, which I also think it's – they make it easier to see the relationship between those ABCs because in the documentation, there is no diagram.

00:45:15 So it's all – Yeah, those pictures are really nice.

00:45:18 I like them.

00:45:19 Yeah.

00:45:20 Yeah.

00:45:20 So one thing I thought was really helpful, and I think probably I learned a decent amount from it, was when you're talking about this whole thing on data structures, you have this part that's called – that you entitled, you know, when a list is not the answer.

00:45:37 Uh-huh.

00:45:37 Yeah.

00:45:37 So, like, you can almost always use a list for everything, and it does have a, you know, effectively a pop method and, you know, push is just append and things like that, right?

00:45:50 So – but it's maybe not the best choice for that.

00:45:54 So you talk about things like arrays.

00:45:57 Yeah, exactly.

00:45:58 And the DQ class or sequence for double-ended queues, and I thought that was really interesting.

00:46:05 So maybe – do you want to just talk quickly about your message there?

00:46:09 Yeah, so that was the idea, because it's something that I've always – not always, but I've already caught myself doing in the past, was to use a list when I should be using an array.

00:46:21 An array is easy – it was really easy to use, and it's much more efficient in terms of memory, and also in terms of execution, because it's a packed data structure where one number is right after the other.

00:46:38 Yeah, it might be worth just, like, expanding on that just a little bit, you know, because in normal Python, even a number, if I say x equals 7, that allocates under, you know, down in the runtime, like a full-on object, which has all sorts of additional information, and it's a pointer out to that thing.

00:46:58 Exactly.

00:46:59 Whereas array allocates, you know, literally four bytes for ints or whatever in a huge long array, right?

00:47:05 Exactly.

00:47:06 That's because – yeah, exactly.

00:47:07 That's why when you create a – I think the first stumbling block to create an array is that you have to look up in the documentation, because when you create an array, you have to write this type code that tells the array what are the size of the numbers that are going to – the size and the type of the numbers that are going to be there, because it can also have an array of floats, right?

00:47:31 So you have to look up in the documentation to remember what are the letters that you use to specify the numeric type that goes into the array.

00:47:40 But then, when you do that, you have to look up in the array.

00:47:40 But then, when you do that, you're all set.

00:47:42 One thing that I try to do, like, for instance, in the discussion about arrays, there is that, and there's other places, is to compare – I wrote those tables with, like, all the methods and operations of a list and array.

00:47:58 And so you can just briefly look and realize what the difference is.

00:48:03 Oh, here's something that I can do with an array that I cannot do with a list or vice versa, right?

00:48:08 So that's why I decided to add those tables, because there's a lot of functionality, and I wanted to pretty much graphically show people that they're almost the same.

00:48:20 But there are some differences, and when you look at the tables, it's easy to tell really quickly what the differences are.

00:48:28 Yeah, it was really helpful for me, so I appreciate that you added that.

00:48:32 That's cool.

00:48:32 Mm-hmm.

00:48:34 So we're kind of coming up towards the end of the show.

00:48:36 Maybe you want to touch on just a couple more points that you think are important parts of the book that maybe I haven't gotten to.

00:48:43 Yeah.

00:48:44 So, basically, the book is divided in six parts, right?

00:48:49 We talked about the part one, which is just one chapter where I present the data model in a very introductory way.

00:48:58 But the idea of the data model and demonstrate how useful it is with the card deck example, and then there's a vector class with operators.

00:49:07 Anyway, and then the second part is data structures, the part that we were talking about.

00:49:13 That part has a chapter.

00:49:16 I think the longest chapters in the book are there.

00:49:20 The one about all the sequence types, the one about dictionaries and sets, and the one about texts versus bytes.

00:49:27 About the text versus bytes chapter, I was very happy because one of my favorite programming book authors, Bruce Echel, who wrote the famous Thinking in Java book.

00:49:39 These days, he's doing some Scala writing, but also he loves Python, and he was reading my book, and he said that my chapter about text versus bytes, which covers Unicode, what Unicode means, and all the different ways of converting Unicode from and to bytes.

00:49:59 He thought they were very useful for him.

00:50:02 Anyway, so this is the data structures part of the book.

00:50:07 Then part three is about functions as objects.

00:50:10 I decided to introduce that before classes, also because I think it's a Pythonic way of thinking.

00:50:16 We start by coding stuff in functions, and we usually only create classes when realized that just plain functions won't do.

00:50:26 Right.

00:50:27 Maybe some data needs to come along with those functions or something, right?

00:50:30 Exactly, exactly.

00:50:32 If you are trained in Java, you are trained to do everything in classes because that's the only way to do it.

00:50:39 So that's why I decided in the book to give a total explanation of functions and design patterns using functions,

00:50:46 how you can simplify some design patterns by getting rid of classes and just using functions.

00:50:53 And then I wrap up with decorators and closures.

00:50:56 And then part four is about object-oriented programming.

00:51:02 It starts with an explanation of how references work and how the whole thing about mutability works.

00:51:10 And then I give some examples of implementing what I call PyTonic objects,

00:51:16 which are first a vector class, a 2D vector class, and then a two-dimensional vector class,

00:51:23 and then a multi-dimensional vector class.

00:51:26 And those examples are excuses to show the use of many special methods in practice.

00:51:33 I thought your comment on how you don't like to call the Dunder methods magic methods,

00:51:42 but special methods was interesting.

00:51:44 You're like, there's nothing magic about these.

00:51:46 This is how you do it.

00:51:47 Exactly, exactly, exactly.

00:51:50 And the thing is, when you call something magical, you're saying, okay, so this is not for mere mortals.

00:51:55 But that's exactly the opposite.

00:51:57 Like I was saying before about the criticism that I was criticizing Java and Go

00:52:06 because of the fact that it has some data structures that are magical in the sense that you cannot really emulate them in your own code.

00:52:14 And so, exactly.

00:52:18 Anyway, so this is the part of the book where I explain how to do that.

00:52:22 And also, there's some discussions about the use of interfaces.

00:52:28 And something that's a first, and I'm very proud that Alex Martelli, who was one of the reviewers of the book, one of the tech reviewers of the book,

00:52:36 wrote an essay introducing, after he introduced duck typing, at least the Wikipedia credits him with the creation of,

00:52:48 or at least with the first use of the expression duck typing in the context of discussing programming languages.

00:52:56 Alex Martelli now invented the concept of goose typing.

00:53:02 Goose typing.

00:53:03 Okay.

00:53:03 What is goose typing?

00:53:06 So, goose typing is a variation of duck typing where you have some type hints,

00:53:12 this new feature of Python 3.5.

00:53:15 But it doesn't require, actually, the new feature, the type hints.

00:53:19 The idea is to use ABCs as a way to specify types.

00:53:27 But here's the thing.

00:53:27 Python uses ABCs in a way that's very influenced by its origins as a dynamic-typed language with duck typing.

00:53:39 And the main reason for that is that it allows you to actually create a class

00:53:45 that does not inherit from an ABC, but then you register it as a virtual subclass of that ABC.

00:53:53 So, you're actually promising.

00:53:55 I do implement that thing.

00:53:57 Although I do not inherit from it, I do implement that.

00:54:00 And you can actually do that after the fact, for instance.

00:54:04 Maybe because it's a third-party library that implements something that you look at it

00:54:09 and say, hey, this actually looks like a sequence, and it would be useful in my code to be able to ask if that's a sequence

00:54:17 and to have the answer yes.

00:54:18 So, in your own code, you can affirm that this other data structure that somebody else did is, in fact, a virtual subclass of a sequence.

00:54:28 So, this is a facility that Python has with the way the ABCs were implemented.

00:54:34 And there's this register method of ABCs that allow you to register some other type as a virtual subtype of this type.

00:54:43 That's cool.

00:54:44 Yeah.

00:54:45 So, that's the idea of goose typing.

00:54:48 It's a different kind of waterfall, but it's related to ducks in some way.

00:54:54 Yes.

00:54:54 Yeah.

00:54:55 Nice.

00:54:57 And then there is a chapter about inheritance where I discuss the multiple inheritance,

00:55:04 which, again, is something that a lot of people don't have contact with because many OO languages don't implement that.

00:55:11 And then there's the chapter about control flow.

00:55:14 This was the hardest chapter for me to do.

00:55:18 And there was a lot because, basically, because the AsyncIO was very new,

00:55:24 still very new.

00:55:25 Yeah.

00:55:26 You have a lot of stuff on, basically, parallelism or async here.

00:55:30 You've got concurrency with AsyncIO, concurrency with futures.

00:55:33 What's the story with the futures part?

00:55:36 Yeah.

00:55:36 So, the thing is, there is a module called concurrent futures, right?

00:55:41 And this module actually works with threads and processes under the covers.

00:55:48 But it was the first module in the first package in the Python standard library

00:55:55 that implemented this idea of futures, which is something that is gaining a lot of currency in other languages as well.

00:56:03 So, futures are basically objects that represent a pending computation.

00:56:08 So, the idea is that when you submit to some concurrent API a task to be done,

00:56:17 usually you submit, please run this function, right?

00:56:21 So, the API immediately and in a non-blocking way returns to you a future

00:56:28 that represents that pending computation.

00:56:32 And then you can query this future to know whether the thing that you asked to be done

00:56:37 is actually done or not.

00:56:38 And when it's done, you can actually get the results from the future as well.

00:56:43 And another thing that the future does is, that is extremely useful whenever you are doing async work or threaded work,

00:56:53 that it encapsulates some exception that was raised when the system was running that function that you asked it to run.

00:57:01 And this is really important because what happens is, when you're running code asynchronously and there's an exception happens,

00:57:09 it's happening in another context, right?

00:57:12 So, how do you catch it?

00:57:14 And it's really interesting.

00:57:16 The way it works is, the future object catches it and stores it for you.

00:57:22 And then later, when you actually go look at the future object, and for instance, if you request the result from the future,

00:57:28 if instead of a result, an exception was raised, the exception is raised at this point in your program.

00:57:35 So, it teleports, in a way, the exception that happened in this other context that you have,

00:57:42 where you have no control, to a context where you actually either, you know,

00:57:46 you want to look at the result of the computation, but actually there was an exception,

00:57:50 and this is where the exception is raised, and you get to handle it.

00:57:54 Yeah, that's excellent.

00:57:55 Because normally you think of the exception as going up the call stack.

00:57:58 Yeah, exactly.

00:57:59 Until somebody catches it.

00:58:00 But on the other thread that you didn't start or manage, right, that's not what you need.

00:58:05 Exactly.

00:58:06 Cool.

00:58:06 Yeah.

00:58:07 Awesome.

00:58:07 And then the last section you have is on metaprogramming, right?

00:58:11 Exactly.

00:58:12 Yeah.

00:58:12 So, the metaprogramming part is about how attributes actually work in Python.

00:58:19 So, I start with some simple examples using the dundergetutter special method,

00:58:25 for example, that allows you to emulate like a virtual attribute.

00:58:29 And I use it, I have a real data stream from OSCON last year that had all the talks.

00:58:37 So, it had a few thousand records in JSON format.

00:58:41 And I use, I implement a few classes to make it easy to navigate through this OSCON data.

00:58:47 And then, what else?

00:58:50 Then I talk about properties.

00:58:53 And then I go to, the next chapter is about descriptors, which is the infrastructure for properties.

00:58:58 And then I developed an example that actually evolved from a talk that I gave at PyCon in 2013 in Santa Clara.

00:59:12 It was called encapsulation with descriptors, the talk.

00:59:15 But in the chapter, I can spend much more than 45 minutes with that.

00:59:19 Of course.

00:59:20 Or 30 minutes, actually, it was.

00:59:21 Anyway.

00:59:22 And the last chapter is about class metaprogramming.

00:59:25 And I didn't call it metaclasses, because there's an area.

00:59:30 Although metaclasses is the most famous way of doing metaprogramming with classes in Python and other languages that allow it.

00:59:40 We also, we now have an easier way of doing metaprogramming with classes, which is using class decorators.

00:59:46 So, then I show some examples with that and a real metaclass.

00:59:51 And that wraps up the book.

00:59:54 Yeah, that's really cool.

00:59:57 There's a bunch of stuff that I'm looking forward to digging into if I have a few more days.

01:00:03 You know, especially the concurrency looks really interesting.

01:00:05 So, yeah, congratulations on your book.

01:00:09 It's very well done, and people are going to get a lot out of it.

01:00:12 Thanks a lot.

01:00:13 Let me just mention one thing that I'm proud of in the concurrency part of the book, and actually the control flow part.

01:00:20 In the cool routines chapter, I wanted to show people cool routines used in an interesting way, which was not linked to asynchronous programming, which is the main use case for them these days.

01:00:33 And so, I decided to implement this discrete event simulation of taxi cabs.

01:00:39 And so, this is something that, an example that I'm proud of, that I came up with this example that shows and gives you, when you run the simulation, if you actually run the script, it gives you an intuition of how actually you can use cool routines to manage many things happening at the same time.

01:00:56 In the example, there are cab trips, different cabs that are on the street catching passengers and dropping off passengers.

01:01:05 And so, but it doesn't involve an event loop, but it's a simulation.

01:01:11 And so, there's a very simple scheduler implemented in about 10 lines of Python and makes all those cabs run.

01:01:19 And actually, the cabs are implemented as co-routines.

01:01:24 That sounds really cool.

01:01:25 Definitely want to check that out.

01:01:27 So, before I let you go, there's two questions I always ask my guests on the way out the door.

01:01:32 First one is, what's your favorite code editor?

01:01:34 I use Sublime Text.

01:01:37 Sublime.

01:01:38 That's a good one.

01:01:38 Yeah.

01:01:39 Although, I have a partner in my training company that uses PyCharm.

01:01:45 And sometimes I get, I think I'm moving to PyCharm.

01:01:49 You get sucked in there.

01:01:50 Yeah.

01:01:51 Yeah, exactly.

01:01:53 I'm not an IDE person.

01:01:56 Maybe someday I will be, but now I prefer simpler things and I'm very happy with Sublime Text.

01:02:02 Yeah, excellent.

01:02:02 And the other question is, what's your favorite PyPI package?

01:02:06 You know, there's all these different things out there, 60,000 plus, whatever.

01:02:10 Yeah.

01:02:11 There's just so much to raise people's awareness of.

01:02:15 So, what's your top list?

01:02:16 Well, requests is probably a common answer.

01:02:21 That's one of my favorites.

01:02:23 Other people have maybe given you this answer.

01:02:26 But I think it's really cool.

01:02:27 And I actually use it in the book.

01:02:29 It's one of the few external libraries that I use, even though there is a library that does pretty much the same in the standard library.

01:02:36 And I use it because it's much simpler, but also because it runs the same on Python 2 and Python 3.

01:02:45 But our HTTP client functionality in the standard library has always been not very good.

01:02:51 And because it's not been very good, it's been changed a lot in Python 3 without getting much better, but enough to become incompatible.

01:03:03 So, I think requests, if you want to write an HTTP client, is something that you really have to go for because then your code will be easier, nicer, and it will run in Python 2 and Python 3 at the same time.

01:03:22 Yeah.

01:03:23 The only thing that I wish requests had, and it doesn't, is support for asynchronous requests.

01:03:31 One of the things that you learn in my book is that you can actually do very effective network code in Python, despite the GIL, with threads.

01:03:42 Because of the fact that every single function in the standard library of Python that does blocking I.O. releases the GIL.

01:03:49 So, if you test it, if you run it, you will see it.

01:03:53 Actually, you know, except in very extreme cases, the performance of threaded applications and asynchronous applications in Python is pretty much the same.

01:04:04 So, you can use requests with threads for doing very fast HTTP clients.

01:04:11 Yeah, that's great.

01:04:12 Because most of you are blocked waiting on the network anyway, right?

01:04:14 Exactly.

01:04:15 Exactly.

01:04:16 Cool.

01:04:16 Yeah, we had Kenneth Wright on the show, the sixth episode, and we talked a lot about requests.

01:04:22 That was very cool.

01:04:23 Yeah.

01:04:24 I'm very, I was very glad.

01:04:26 I think Kenneth, I don't know, Kenneth gave some manifestation that was positive about my book on Twitter.

01:04:34 That maybe he favored somebody else's tweet that was positive about my book.

01:04:38 But I was very proud of that because he's really an example of a guy who can do things in a very Pythonic way.

01:04:47 If you want to learn what Pythonic is, study Kenneth's code.

01:04:51 Yeah, he's very good at creating APIs.

01:04:53 Yeah.

01:04:54 Mm-hmm.

01:04:54 All right.

01:04:56 Any final shout-outs or call to actions before we go?

01:05:01 People should go read your book.

01:05:02 They should go check out your book, right?

01:05:04 Yeah, sure.

01:05:05 Yeah, it was a huge investment for me.

01:05:09 I don't know if most of your readers know, but it's pretty impossible to make a living writing books, right?

01:05:15 It's sort of like how to make a living playing guitar.

01:05:22 Yes.

01:05:23 There's about very few people in the world that do it, that manage that.

01:05:28 Anyway, so I have to recover the investment in time that was huge.

01:05:33 I spent 18 months writing this book.

01:05:36 And the second half of that time was full-time doing it because my editor started complaining that it was, you know, I get, I was not feeling my...

01:05:48 You weren't on the deadline.

01:05:50 I was not in the schedule, yes.

01:05:52 So there was a couple of postponements, but then she noticed that I was really writing a lot.

01:05:58 The book grew.

01:05:59 It started as a project for a 300 to 400 pages book.

01:06:04 But I'm really happy with the result.

01:06:07 I'm really proud.

01:06:08 And I'm really happy with the chance that I had to work with Megan Blanchett, my editor, O'Reilly.

01:06:15 She was awesome.

01:06:17 And the support that I had from the technical editors, Alex Martelli, Anna Ravenscroft, Leonardo Rochael, and Leonard Regebru, and Victor Steiner, who was a special guest tech editor for the AsyncIO chapter.

01:06:32 They were awesome.

01:06:34 And it's really been a great experience.

01:06:36 And I'm really happy to see how people are enjoying the book.

01:06:41 Yeah.

01:06:42 It's quite an investment, but hopefully it pays off for you because it's definitely good work.

01:06:46 Yes.

01:06:47 Thanks a lot.

01:06:48 Yeah, you're welcome.

01:06:49 Luciano, thanks for being on the show.

01:06:51 Really appreciate it.

01:06:51 Yeah.

01:06:52 Thank you very much, Mike.

01:06:53 You bet.

01:06:53 Bye.

01:06:54 Bye.

01:06:55 This has been another episode of Talk Python to Me.

01:06:57 Today's guest was Luciano Romalo, and this episode has been sponsored by Hired and Cochip.

01:07:02 Thank you guys for supporting the show.

01:07:03 Hired wants to help you find your next big thing.

01:07:06 Visit Hired.com slash Talk Python to Me to get five or more offers with salary and equity presented right up front and a special listener signing bonus of $4,000.

01:07:14 Cochip wants you to always keep shipping.

01:07:18 Check them out at Cochip.com and thank them on Twitter via at Cochip.

01:07:21 Don't forget the discount code for listeners.

01:07:23 It's easy.

01:07:24 Talk Python.

01:07:25 All caps.

01:07:26 No spaces.

01:07:26 You can find the links from today's show at talkpython.fm/episodes slash show slash 24.

01:07:32 Be sure to subscribe to the show.

01:07:34 Open your favorite podcatcher and search for Python.

01:07:36 We should be right at the top.

01:07:38 You can also find the iTunes and direct RSS feeds in the footer of the website.

01:07:43 Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.

01:07:48 You can hear the entire song at talkpython.fm/home slash music.

01:07:52 This is your host, Michael Kennedy.

01:07:54 Thanks for listening.

01:07:55 It's Mix.

01:07:56 Take us out of here.

01:07:57 He's dating with my voice.

01:07:59 There's no norm that I can feel within.

01:08:01 Haven't been sleeping.

01:08:02 I've been using lots of rest.

01:08:03 I'll pass the mic back to who rocked it best.

01:08:06 I'll pass the mic back to who rocked it best.

01:08:19 Thank you.