Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Tests? #47

Open
tmcw opened this Issue · 15 comments

5 participants

@tmcw
Owner

Since this is a minimal new start and we have some freedom, could we write tests? KIF, kiwi and gh-unit look like decent options for ObjC.

@wsnook

Testing sounds like a good idea to me. I haven't had the occasion yet to dig into Xcode 5's built in testing features, but I'd suggest those go on the list of options to consider:

https://developer.apple.com/technologies/tools/whats-new.html
https://developer.apple.com/library/mac/documentation/ToolsLanguages/Conceptual/Xcode_Overview/UnitTestYourApp/UnitTestYourApp.html

@incanus
Owner

HELL YES. I would love to get on this before the project keeps growing, though it slipped my mind. Testing culture in Objective-C is not good, but this is a light enough library that I think it makes sense to cut our teeth on it, find our way around, and then expand it into our other iOS work.

@mick
Owner

Cedar is a good option. You can see what it looks like in action on @boundsj's mapbox-ios-sdk test branch

@wsnook

@tmcw and @incanus, what sort of requirements do you see in terms of where, how, and when the tests get run, and how they integrate with other things? I'm thinking along the lines of the continuum from "make sure the tests pass in Xcode before you commit" all the way out to "you can't commit unless the tests pass because it's all integrated with GitHub hooks and such".

My suggestion would be to error on the side of using the thing which pulls in the smallest number of external dependencies to hit whatever level of integration is truly necessary. We already have kind of an issue with getting the installation dependencies figured out for MBXMapKit itself, so it seems like a good idea to be careful about pulling in more dependencies unless they provide something really beneficial.

On the other hand, maybe it's time to sit down and solve the installation problem, and pull in a good testing package as part of that.

@incanus
Owner
@wsnook

My first thoughts at approaching this would be:
1. Explore the possibility of using Xcode's unit testing features for testing the key internal functionality of library code, like what happens if you try to fetch a bad tile url that should return a 404?
2. Write a sample app which exercises the full API in such a way that it's easy to manually check that everything looks okay (perhaps a tableview where you can navigate to each of several test configurations).
3. See what things look like once we've gotten that far

@wsnook

One thing that would be cool is to have some automated process which periodically makes sure that the podspec works and that it isn't too far out of sync with mapbox/mbxmapkit/master (e.g. #34 ).

@tmcw
Owner

Let's not think about Travis / running this periodically / blocking git commits at this point: just being able to run test in the repository and see a ✔ would be great for contributors and for maintainers doing refactoring.

@incanus
Owner

Agree with @tmcw. I do like the CocoaPods automation idea @wsnook, but honestly that's more useful to me in http://github.com/mapbox/mapbox-ios-sdk than here, though we can reevaluate as the project grows.

@wsnook

@incanus sounds good. I'm definitely thinking about this in terms of stuff that might need to happen as part of managing future complexity. So far things are still pretty sane, so it's not a big deal yet.

@wsnook wsnook referenced this issue from a commit in boundsj/mbxmapkit
@boundsj boundsj Add tests
- Create a project (currently ios static lib)
- Add cedar
- Add cedar specs target
- Add rakefile
- Add travis config file
9df9f34
@boundsj

Hi @incanus @wsnook @tmcw. I set up cedar specs, a rake file for running the tests from the command line, and have automated builds and tests running in travis for my fork of mbxmapkit.

Where I work, all of our objc projects are test driven using cedar (a BDD framework that we built in house that eventually inspired others like kiwi) and with full CI - usually on travis or or Jenkins instance on a mac mini.

You can see the CI status here: https://travis-ci.org/boundsj/mbxmapkit
The (albeit massive) commit to make all of this work is here: boundsj@bbc2683
(note: my fork has the super awesome travis build status badge in the readme: https://github.com/boundsj/mbxmapkit :sunglasses: )

Most of the changes were administrative - I housed the files in a project as an ios static lib, (I know the MO is to keep things simple but it seems it might be time for this project to grow up just a little bit... it'll still be simple enough as a static lib for ios/osx), I added a rake file, I added cedar specs. You can read all about how cedar works on it's repo but basically, you install it system wide and it comes with xcode plugins so things "just work".

The most salient addition is https://github.com/boundsj/mbxmapkit/blob/master/mbxmapkit/Specs/MBXMapKitSpec.mm. In this file I demonstrate a few of the basics of BDD testing objc code. An MBXMapView is initialized and then we test that it's behavior is correct. For example, did it make a request for tiles? Spies and fakes are used to control flow under spec.

It's probably worth nothing that this test really ends up testing the "updateOverlay" method of MBXMapView. The current design is a bit odd and a little hard to test. For example, updateOverlay calls itself recursively, infinitely if there are network and/or non-200 response codes. But, I'm not sure I would've noticed this awkwardness if I were not thinking about how to write a behavioral test for it. Also, there are GCD calls made in "updateOverlay" that would be nice to pull out into separate objects so the asynchronous behavior can be forced to be synchronous under test.

In any case, I hope you consider doing this or something like this. I'd encourage you to use a BDD framework like cedar (it is truly awesome and well supported but I'm biased) or, if not, kiwi. BDD frameworks have several advantages over the more traditional unit testing frameworks (like Apple provides). I think an amazing open source project like this deserves a great testing framework for it's contributors and for the sake of long term sustainability.

If any of you are ever in the SF office and want to pair on writing tests let me know - I'd be happy to stop by and give a more detailed tour in person.

@wsnook

Hi @boundsj, thanks for all this. Over at #61 I've been working on a refactoring experiment for the 0.3.0 milestone, and I think it may address some of what you were talking about with the updateOverlay stuff.

@incanus
Owner

@boundsj, this is great stuff. I think I follow. I see how MBXMapKitSpec.mm is testing that the use of a mapID in effect adds an overlay and that overlay results in a TileJSON request by spying on the NSURLSession. I would be interested in learning conceptually how this could be taken further, especially with a lot of UI results being the end state.

If any of you are ever in the SF office and want to pair on writing tests let me know - I'd be happy to stop by and give a more detailed tour in person.

In fact, I will be there tomorrow through Friday, though I know this is late notice. Would love to sit down and discuss this a bit if you've got some time.

@boundsj

Thanks @incanus. I'll check with @mick to see when it might work out to meet up.

@incanus
Owner

We've got a good thing going with KIF & Travis over in GL Cocoa and should replicate that if we do another sprint on this project.

@incanus incanus added the refactor label
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.