Policy re: examples and tests

[jennybc]

API wrapping packages present headaches in terms of runnable examples and tests. They can be slow and there are myriad reasons why they might not work, that don't necessarily reveal a fundamental problem.

Do you have any Big Picture "policy thoughts" before I even contemplate adding examples to function docs or putting some real tests in?

[cscheid]

This is all really ugly, you're right.

What I really would like to get right is testing the paths that require authentication.

I have thought of using some of Hadley's infrastructure for testing using private tokens via encrypted key-value pairs on the travis configuration file, but I haven't tried that yet. RGithub's develop branch needs merging, but it has the travis configuration file. The idea would be to get an authentication token by decrypting the values in Travis, and then running real tests against GitHub's API (create.gist, etc.)

These would necessarily be slow, but the only alternative that I can think of is mocking the entire GitHub backend. (That's impractical, of course.)

I really don't know how people write test suites for API wrappers, to be honest.

What do you think?

[jennybc]

I am working on a package to wrap the Google spreadsheets API (or rather, someone working for me is doing the grunt work 😄 ) and we are confronting many of the same issues. This was just a probe to see if you had some pre-existing master plan 😄 I don't have any real answers either.

But I do propose not waiting for the glorious but complicated stuff you describe above to start adding tests. What if I just started to gradually put in tests where possible, i.e. where authentication is not needed?

I have done some informal testing of my own using a test Organization, because my current crisis is a need to create many Teams and repos and create associations between them. Seems like GitHub needs some dummy repos and Organizations and users where you can count on various properties and build tests around querying them! Won't help for creating repos, Teams, etc. though.

[cscheid]

What if I just started to gradually put in tests where possible, i.e. where authentication is not needed?

That would be enourmously helpful.

[cscheid]

I'm going to close this, since half of it is going to #48, and I just created #49 for the other half.