Ranked Skills

August 10th, 2019

Technical resumes are challenging to produce, and challenging to accurately interpret. One of the biggest obvious challenges is that candidates are highly motivated to exaggerate, and interpreters of resumes are highly motivated to make a quick decision erring on the side of rejection so as to waste a minimal amount of their precious time. Another big challenge, however, one which I think might be solved, is the difficulty in commnicating exactly what the resume author means by how skilled they are in a particular technology or specialty. Even if one were to assume candidates are 100% truthful and resume readers give 100% effort to read and parse every word of every resume that comes across their desks, what does "Expert at Python" or "5 years experience C++" really mean? Worse yet, someone says (or writes on their resume) "I am 8 / 10 at Ruby" - what does that mean? Eight what? Years? Projects? This person thinks they know 80% of everything there is to know about Ruby? Or are in the 80th percentile for Ruby programmers?

In discussing this problem with others, I had a brilliant suggestion made to me, and I deeply regret not remembering who to credit for it. As an interviewer, they suggested, one could ask "on a zero to ten point scale, rate yourself on X". This immediately sounds like a complete bullshit softball question - but stay with me. The candidate will hopefully say something between six and nine. There is a strong urge among people to rate themselves at least a five on most things they are even passingly familiar with. Similarly, except for the most insanely confident candidate, pretty much nobody should claim to be a ten - If a candidate claims to be a ten in Python, they had better be Guido Van Rossum. Let's say a candidate claims to be a nine out of ten in Python. Great, now immediately follow up with "What would you teach someone who is an eight, in order to make them a nine?". This is not a trap per say, but it does force the candidate to say things which give you a very good idea what they meant by nine out of ten. If they answer "recursion", you know their nine means bupkis. If they answer "how function decorators work", you know that this candidate is familiar with function decorators and considers it a very advanced language feature. Since I would not currently rate myself a nine at Python, if they talked about things which I am not myself comfortable with, I could conclude that they really are probably more advanced at Python than I am myself.

Next, I would follow up with the inverse question: "What would you expect someone who is a ten to teach you, or what would you expect to learn in order to call yourself a ten?". Like before, if you get a disappointing answer about some standard language feature, you know this candidate is not very advanced, but if you get an answer like "implementation details, familiarity with the codebase and PEP process, experience with implementing new language features", that is an answer which shows that the candidate understands what it means to be at the very top of the Python game.

Using both answers, it becomes much easier to understand where a candidate is really at, rather than trying to perform mind-reading tricks to turn an arbitrary number into a complex vector in skill-space. That said, I also propose a standard ranking for what numbers mean which is moderately technology agnostic and can be used to communicate skills in a "best effort" fashion, knowing that not every interviewer can or will use the trick I've outlined above. Additionally, a standard metric like this yields benefits for non-interactive cases such as reviewing a resume.

When I rate myself on a ten point scale, I use this guide to do so:

10: I could have a protracted discussion with the inventors of this technology including deep discussions about implementation details without prior preparation or embarrassment. I honestly believe I have little or nothing more to learn about this technology. I, personally, am unlikely ever to claim this rating.
9: I could follow a deep technical discussion among core developers of this technology regarding most implementation details.
8: I am very familiar with all user-facing aspects of this technology, as well as some implementation details necessary for mastery.
7: I am able to use this technology as a competent power user with little need to seek references.
6: I am a competent power user of this technology, though I occasionally seek references or examples.
5: I have a firm understanding of this technology but often seek examples or references.
3: I am a user of this technology, but not a power user.
1: I have a passing familiarity with this technology, having used it a handful of times or interacted with it superficially. I am listing it primarily because I am interested in learning more.
0: I refuse to use this technology or acknowledge any ability in it whatsoever. Think PHP.

I personally think of my resume as a method for communicating my skills and fit as accurately as possible. I don't want to waste anyone's time, including my own, so accuracy is paramount.

====

Title: Ranked Skills Date: 2019-08-10 Tags: interviewing, programming