Merge pull request maxmind#1 from maxmind/dave/add-evaluation-doc

Evaluation doc

EVALUATION.md

@@ -0,0 +1,58 @@
+This is a guide for those involved with hiring at MaxMind to evaluate the
+homework (but if you're an applicant you're welcome to read this).
+
+## Questions to Ask When Reviewing Homework
+
+* Is the code in one of the specified languages, or did the applicant get a
+  special dispensation (papal or otherwise) to use another language?
+* Is the code runnable on an Ubuntu Trusty or Xenial system without too much
+  difficulty (installing 3rd party libraries is not too much difficulty as
+  long as clear instructions are provided. Installing ten PPAs from ten
+  different sources *is* too much difficulty)?
+  * If the code is not in Perl or Go, are the instructions written for people
+    without much experience in that language?
+* Does the homework include instructions in a `README` or as the output of
+  `command --help`?
+  * If the code isn't in Perl or Go, there really needs to be a `README` of
+    some sort!
+* Does the code do more or less what it should (run it and eyeball the output
+  for sanity)?
+  * We should expect some slight variance from the internal answer we have
+    because of small ambiguities in the instructions. For example seeing a
+    slightly different order of countries or states, or one different top URL,
+    is usually not an indication of any serious mistakes in the
+    implementation. If all of the countries or states are different, it's
+    worth investigating the applicant's implementation further to see if they
+    did something really wrong.
+* Does the output include the specified data?
+  * Top 10 countries by visitor count, with name of country in English and
+    visitor count for the country.
+    * Plus top page for that country that is not the root page (`/`).
+  * Same top 10 list for US states.
+* Does the code handle the following cases sanely ...
+  * IP does not exist in the GeoIP2 database?
+  * IP exists but does not have country info?
+  * IP exists and is in the US but does not have state info?
+* Is the code organized in a reasonable manner?
+  * Is the split between various packages (if any) done in a way that breaks
+    down well across responsibilities?
+  * Is the split between the invoking CLI program and the modules sensible?
+* Does the code use appropriate 3rd party libraries?
+* Does the code use a GeoIP2 client library?
+  * It's ok if they just use a low level MaxMind DB reader too, since the
+    distinction between the two types of libraries is not obvious if you're
+    new to our products.
+* Is the code written in a modern style for the language, avoiding known-bad
+  idioms or techniques?
+* Does the implementation avoid repeated code for easily abstracted things
+  (like counting and/or displaying top N things)?
+
+### Bonus points
+
+These things are nice to have, but depending on the applicant's level of
+experience, they may not be able to get to them in the time specified.
+
+* Is there any sort of API documentation for some or all of the code?
+  * If so, is it well organized and clearly written?
+* Are there any sort of unit and/of functional tests?
+  * If so, are they well organized and clearly written?

README.md

@@ -2,7 +2,7 @@ Please note that while this is a relatively simple matter of programming, we are
 
 The given log file is a standard log file in the [Apache combined log format](https://httpd.apache.org/docs/1.3/logs.html#combined) (with referrer information stripped out). You will write code to parse this file and generate a report which breaks down some of the information based on the country and US state where visitors came from. This code should be delivered in the form of one or more classes. Please include any tests you wrote along the way as well.
 
-**Please write your example in Perl, Python, Ruby, JavaScript, or PHP. If you want to use a different language please contact us first.** You are also encouraged to use any third party libraries you like. However, you must deliver the end result in a way that allows us to run the code on a standard Ubuntu Precise or Trusty system. That means either packaging up all of your dependencies or giving us very clear instructions on how to install them. Unless the language you choose is Perl, assume that we do not know anything about the package management tools for that language! We will run your code to look at the output, as well as looking at the code itself.
+**Please write your example in Perl, Python, Ruby, JavaScript, PHP, or Go. If you want to use a different language please contact us first.** You are also encouraged to use any third party libraries you like. However, you must deliver the end result in a way that allows us to run the code on a standard Ubuntu Trusty or Xenial system. That means either packaging up all of your dependencies or giving us very clear instructions on how to install them. Unless the language you choose is Perl or Go, assume that we do not know anything about the package management tools for that language! We will run your code to look at the output, as well as looking at the code itself.
 
 Include instructions on how to run the code, either as the output of `your-command --help` or as a separate README.md file. Your instructions should tell us how to specify the access log file to parse, as well as how to specify the MaxMind GeoIP2 City database to use. This can either be done via command line flags or by telling us where to put these files relative to your script.