Introducing Cukes #431

merged 156 commits into from Jan 24, 2014


None yet

Please note this is not merge-able yet, this is mostly for discussion purposes.

Hey all! One of the first things I've been tasked with is improving the API specification. Over the weekend I asked around my network, did some research, and spiked out an idea, and I wanted to run it all past you.


As always, the first step is to gather requirements. Here's what I gather we need for the API spec:

  1. A way to define what each aspect of the API does.
  2. An automated way to turn those definitions into executable tests.
  3. A way to run those tests regularly to ensure conformance.

All three of these are pretty straightforward, but might as well spell them out.

  1. These specifications must be easy to understand by a large variety of people.

This one is complicated. There are two kinds of concerns here: {in,out}side of Balanced, and across software language groups.

The first is the in/out part. Basically, while we have a very clear idea of what everything does, people who are new to Balanced won't. Just today I got an email from someone, asking if Balanced was appropriate for them, when their marketplace was basically identical to Rent My Bike. I'm happy to answer those kinds of questions, but we need something that will be easy for people who don't live and breathe payments to grok.

Secondly, software language groups. If we write our specs in Python, Pythonistas will be able to understand it, but Haskellers might not. Well, they probably will, but you know what I mean. Everyone wants their preferred language, but our product is an API, and it's language agnostic. Therefore, we need something that's not incredibly closely tied to a particular software language.

  1. The specification has to facilitate collaboration on what the specification actually does.

This is related to #4, but is a bit different. It's not enough that the spec is legible to everyone, but that it's easy to bikeshed. ;)

What we have

Right now, the API spec looks largely like this:

It's a bunch of YAML files, with a totally custom, bash/Python runner.

This situation is good in that it's language agnostic, but bad in that it's 100% custom, so it's not natively understood by anyone, and while it communicates to some degree of what the specs do, it's almost 100% code. It took me some time to be able to actually understand what's going on, which will always happen, of course, but it's still hard to turn into actual words, which impedes #4 and #5.


I've spiked out a temporary solution based on a tool called "Cucumber." Before I tell you more, let's compare:



The 'current' has two features, and the cucumber only has one, so ignore past line 18 on 'current.'

Cucumber, if you haven't heard of it, is a tool for collaboration between stakeholders. The idea is that you write high level descriptions of what you want your software to do in psudo-English ("Gherkin"), and then that gets transformed into code to actually run and test those descriptions.

While I've implemented the scenarios in Ruby, you can write the 'code' aspect of the tests in a variety of languages. You can also write the features in a variety of languages, too. For example:

As for how this gets turned into code, I have two versions:

JSON Schema:


Since we already assert against JSON Schema, I think sticking with that makes sense, as it'll be easier to port.

I actually think the "200 code" assertion may be too low level.


Basically, this is the exact use case for Cucumber. Because the specifications are written in English, we help make it easier for people to understand, as well as making it easier from people to collaborate across language boundaries. I have some issue with English being the de facto human language that technologists communicate in, but it is what it is, at least for now. This solves #4 and #5, our biggest problems, nicely.

Furthermore, collaboration can be done on the high-level language, without worrying about the details. For example, Matin had some concern over my wording of the last part of that scenario, because 'that's not exactly what that does.' Excellent, we have some sort of communication error here. Cucumber's high-level descriptions bring that kind of thing out, and people can submit pull requests with just the Gherkin, and we can add the implementation ourselves.

I wonder if "elevator pitches for your code" is a good way of describing Cucumber. Maybe I've been in San Francisco too long. ;)


Cucumber is... not a tool without some controversy. It was originally described as an 'acceptance testing' tool, and so people got into 'test mode' and started writing tests. Cucumber is very bad when used in this way. Eventually, even some helpers were removed [1] to discourage writing tests at a low level. Basically, if you're not writing very high-level descriptions, you are Cuking It Wrong. ;)

Cucumber can be slow. There's the whole Gherkin-code 'compilation' step, so it can take longer than other frameworks. This is mitigatable, but we're already planning on making HTTP requests against our API for our tests, so they'll be slow no matter what the framework is.

As I mentioned above, some people really, really dislike cukes. I think that's mostly due to having terrible Cucumber suites, people have developed some scars. This can be overcome, but there may be some initial 'eww.' There may be with our specs as of right now. ;)

Other options

Basically, if this is a bad idea, we should just move to some sort of very basic unit testing library in a not-to-unusual language. I'd basically use Python and Requests to do this, since it's what we're most familiar with overall. I think that the downsides of being language specific is pretty big, though.


I'm sure I've forgotten a few things, but I'd love to hear feedback, good and bad. I think this is the right move, but I don't always have all the right answers... I would especially love feedback from @mattwynne and @aslakhellesoy .



CRUD tests

Here's all the cases we need to cover to merge. From :


  • Create a Key
  • Get a Key
  • List API Keys
  • Delete API Key


  • Create a Bank Account Verification
  • Get a Bank Account Verification
  • Confirm a Bank Account Verification


  • Tokenize a Bank Account (Direct)
  • Get a Bank Account
  • List Bank Accounts
  • Delete a Bank Account
  • Credit a Bank Account
  • Debit a Bank Account


  • Create a Callback
  • Get a Callback
  • List all Callbacks
  • Delete a Callback


  • Tokenize a Card (Direct)
  • Get a Card
  • List All Cards
  • Update a Card
  • Deleting a Card
  • Debit a Card


  • Create a Credit
  • Get a Credit
  • List All Credits


  • Create a Customer
  • Get a Customer
  • List Customers
  • Delete a Customer
  • Associate a Card
  • Associate a Bank Account


  • Create a Debit
  • Get a Debit
  • List All Debits
  • Update a Debit
  • Refund a Debit


  • Get an Event
  • List all Events


  • Create a Refund
  • Get a Refund
  • List All Refunds
  • Update a Refund


  • Create a Reversal
  • Get a Reversal
  • List All Reversals
  • Update a Reversal

non-crud tests

That's exhaustive. Let's do this.

I like it!

You'll have to see how it plays out to stick to this kind of high-level language in all of your scenarios. Some people might want to see more detail, yet not want to read the Ruby code underneath in the step definition. You can either address this with hand-crafted user docs (which could go in the Gherkin), or you could consider having one scenario per feature that uses more detail and has concrete examples of the JSON request / responses.

I'm interested to see how this develops.

tef commented Dec 3, 2013

I think as it stands, your requirements are manufactured to want Cucumber, so let's look at them:

A way to define what each aspect of the API does.
An automated way to turn those definitions into executable tests.
A way to run those tests regularly to ensure conformance.

The second step is the one you'll have to justify, and I'm not convinced.

. Basically, while we have a very clear idea of what everything does, people who are new to Balanced won't. Just today I got an email from someone, asking if Balanced was appropriate for them, when their marketplace was basically identical to Rent My Bike

Writing these tests in pseudocode won't make people read them. (Unit) tests themselves do not give big picture views of your code, and should not serve as documentation, or example code.

If you have a comprehensive test suite, would you expect people to browse through them one-by-one to see if their use case works?

Secondly, software language groups.

There is an xkcd cartoon here, but instead of using a language some people know, you're going to use a language no-one knows. Although some tests become 'readable', the maintenance of tests becomes labourious. Frequently for each line of english you have to write a custom regular expression to parse it. Your test runner won't be language agnostic either.

Therefore, we need something that's not incredibly closely tied to a particular software language.

Except the implementation will be, and you will need to rewrite the ad-hoc regex rules in each language if you want to avoid this. This is a huge burden for tests and maintenance.

Basically, this is the exact use case for Cucumber

The use case of cucumber (afaik) was so that clients signing off on a project could agree to pages and pages of legalese that was executable, to enable the customer to both understand and specifiy what they want.

This is not your use case. Your use case as presented is "We need better documentation and examples to showcase features and make it easier to get started with the API". Cucumber solves this if you think reading tests is a good substitute for documentation and worked, comprehensive examples in other languages :-)

My experience with cucumber is that it made writing tests slow, and running tests slow, and debugging tests harder.

BRMatt commented Dec 3, 2013

Sorry if I've misunderstood the problem, but have you considered using a tool like api-blueprint to document the API, then use tools that test the API against the defined documentation?

It adds some semantics on top of Markdown for documenting requests/responses/schemas etc. and it's pretty readable, even when browsing via Github.

Admittedly it probably won't capture intricate scenarios as easily as a cuke scenario/bash script, but thought it might be worth bringing up.

tef commented Dec 3, 2013

(nb I'm not against having documentation and examples which can be tested to check they work, but I don't think that is the same as having tests that serve as documentation and examples. c.f python's doctests and the problems with using them)


@BRMatt , I'm not opposed, but one of the tricky bits here is that as Balanced moves even further towards hypermedia, we want to move away from documenting exact URLs and HTTP methods, and focus more on workflows. I don't have the most experience with api-blueprint, but generally, these kinds of tools tend to focus on 'RESTish' style documentation rather than this kind, so that's something that's in here too.


@tef, I think your 'use case for cukes' is the same thing, just more cynical. ;)

I actually didn't even come up with cukes until the last minute, so maybe they are described as a fit for cukes, but it happened in the other direction.

I don't think new people will go read a bunch of cukes, but it would be nice to show as a response to this kind of question.

@tef makes an important point that the demands of testing and documentation can come into conflict. The RSpec project have tried this, and found that the comprehensive test suite made for pretty boring user documentation. Users don't want to see every single edge case.

My advice in this situation is usually to push the edge cases down into unit tests. If you think there's still benefit in having that complete specification readable by everyone, consider writing two sets of cukes - one that provides a user manual, and another that provides your complete specification.

Kosmas commented Dec 3, 2013

@tef regarding the issue of speed with the tests, I think that is something that will change soon but @mattwynne will be able to confirm this.

I also believe that the cukes would help to have a quick understanding of how things are supposed to work, but in order to fully understand you would have to go further deep into the code.

MattyO commented Dec 3, 2013

The python equivalent to cucumber is called lettuce. It a little immature compared to cucumber. Its usefulness may be found in the use cases of interoperability with any current testing tools and libraries you might have or want to use in the future written in python.

Mainly wanted to let you know of its existence.

BRMatt commented Dec 3, 2013

@BRMatt , I'm not opposed, but one of the tricky bits here is that as Balanced moves even further towards hypermedia, we want to move away from documenting exact URLs and HTTP methods, and focus more on workflows. I don't have the most experience with api-blueprint, but generally, these kinds of tools tend to focus on 'RESTish' style documentation rather than this kind, so that's something that's in here too.

@steveklabnik Fair enough, those are reasonable points. I believe the maintainers of api-blueprint are trying to work in support for hypermedia relations, but I'm not sure how that fits in with your plans.

mjallday commented Dec 3, 2013

@steveklabnik do you see Balanced then implementing the underlying scenarios for the Gherkin in each client library to ensure they conform to a basic functionality (the spec)?

As I'm sitting here figuring out how we can implement the rev1.1 clients I really like the idea of a basic set of functional tests across all client libraries so we don't end up with the issues we had last round such as under-utliized client libraries having issues opened because they are missing esoteric functionality and we don't realise until someone actually tries to use it.


Here are my thoughts:

I think Gherkin is the way to go here, definitely +1 on it.

First and foremost, there are many Gherkin/Ragel parsers in many different languages. The official client libraries that are supported are: Ruby, Python, PHP, Java, C# and Node.js -- all of which have Gherkin/Ragel runners. So this is largely aiding #4 as well and helps with the training wheels.

We attempted to something very similar to this in our documentation: -- maybe instead of having scenarios like this, we organize tests implemented in different languages and use those to feed into our API? Ben might have more thoughts on this topic.

This is definitely a step in the right direction and however, I'd like to compare it to something like or RAML.


@mjallday maybe; I'm not sure how directly applicable they are, as the clients would need testing that they consume things correctly, and these are tests that stuff is produced correctly.

That said, we should integration test both together, at some point.


I just realized that I wrote a blog post a few years back that's directly relevant to this conversation

JonRowe commented Dec 5, 2013

What'd be great is a way to visualise the gherkin documentation online, with each step able to sho the raw response / differing language implementations...


@JonRowe , that's what is for :)

JonRowe commented Dec 5, 2013

It allows you to visualise the documentation, but it's not very non-developer friendly, I found @chrismdp's rack-usermanual idea very intriguing in this respect. It also doesn't currently offer a means to inspect what it actually does / provide sample responses.


Oh that is super cool!

@steveklabnik we'd be happy to set you up with a free account once we go live. Cucumber Pro will replace Relish.

Regarding the idea to document the API with Cucumber. If I was a developer using the API, I would like to see scenarios like this:

Feature: Payment

  Scenario: tokenize
    Given some precondition here
    When I http POST to /cards with JSON body:
        "number": "4111111111111111",
        "expiration_month": "12",
        "expiration_year": 2016
    Then the response code should be 201 with:
        ... whatever the expected reply is...

This is more verbose than what you suggested, but I think this level of detail is needed for the audience you're targeting (developers using the API)


@aslakhellesoy thanks! Yeah, that makes sense; with something like this, I'm never 100% sure what the right level of abstraction is

To find the right level of abstraction, ask some stakeholders (in your case, users of the API):

  1. Does this make sense?
  2. Would you be able to do this yourself with this info?
  3. Do you need more info?
  4. Is anything puzzling you?

Looking at your feature I think most developers unfamiliar with your API would answer like this:

  1. Yes, I think so
  2. No. I have no idea what HTTP request to issue
  3. Yes, can you give me an example?
  4. I don't understand how the secret key is relevant. Can you explain? (that's what the scenario description is for)
Feature: feature name
  Scenario: scenario name
    Some descriptive text here is useful

    Given ...

@aslakhellesoy that looks really good! 👍

We've been using a similar setup at @nutshellcrm. I think it has been working pretty well, though I admit it's tough to find the right level of abstraction.

chrismdp commented Dec 6, 2013

@steveklabnik thanks. Just realised the Rack::Usermanual demo links were broken: should be fixed now.


@andrewsardone would you be interested in some help and / or collaboration to turn this into a gem, maybe?


@tef, does this style do anything to mitigate your concerns, or you still think it's a terrible idea?




@aslakhellesoy @mattwynne , one more question: We have scenarios that operate like this:

This (kinda terrible because of the runner) test basically says "run the 'tokenize' scenario, and then make a request to the URI that's located at cards.href in its response."

This kind of thing is going to come up a lot as we move more and more towards hypermedia; "execute this thing, if it passes, execute this other thing, based on what comes back." These feel like two different tests to me, but GivenScenario was depreciated, so I'm not sure how to accomplish this kind of task.


mamund commented Dec 9, 2013

I've thought about this, too. Another possible testing challenge is that you're building a client that could possibly travel several paths to a successful outcome. So it's not just if this then that but also something like start at A and IF you eventually get to Z that's a pass ELSE that's a fail

A likely situation for hypermedia, but not something I am aware that current testing tools can handle.

matin commented Dec 9, 2013

@steveklabnik Could you also add a scenario that debits/charges the card that you just tokenized?

One of the most painful parts of our current scenarios is referencing the result of a previous scenario. This becomes even worse if you want to use some sort of a fixture to avoid doing repetitive tasks like tokenizing a card or creating a customer for different scenarios.


Yup, that's the next one I'm working on. That's how I found out #431 (comment) , because they all chain off of each other.

@andrewsardone would you be interested in some help and / or collaboration to turn this into a gem, maybe?

Absolutely! It would be good to get a few minds together to see if there’s any place for a specific tool. Our current setup is very simple – it really just glues together lostisland/faraday & collectiveidea/json_spec within features/support/env.rb, but perhaps we should all focus on not reinventing the wheel whenever we try to spec out a new API. I think your and @balanced’s approach here is the right one, and hopefully more API builders will follow your lead.

I’ve created a blank slate umbrella project for the gem. If others think there’s value in trying to build a simple tool, open up some issues and let’s see what we need!


This looks good so far. I think that the main thing that I am waiting to see is how referencing the result from other scenarios will work, but there seems to already be conversation on this front.

I haven't run this locally yet, so I am wondering what the error reporting will look like for cukes. With the current yaml based scenarios, it first tests to make sure that the scenarios themselves are internally consistent, which basically ensure that you are not referencing non existent scenarios and files. Once it has finished the first step it then tests the api, and if there are any differences it basically ends up printing out all the json and reporting that there is some error. I am not sure if this is the right way to go about it, but how can you detect errors in the scenarios vs errors in the api.

I know that you are trying to go in the direction of including json in the scenarios (as that is what the target audience cares about), but I am questions what is the right level of detail in the scenarios. Currently you are listing that I will get back a cards with is an array of something. ( ) In the yaml based scenarios we had try to divide up what was content of the response by using specifically json schema and the the scenario just contained specific details for that instance. ( )


Here's one example if your cukes are off: It's good to draw the distinctions between the two, you're right, and cucumber absolutely does. Here's an assertion failure:

Then I should get a 200 OK status code                                     # features/step_definitions/card_steps.rb:17                                        
      <200> expected but was                                                                                                                                       
      <401>. (MiniTest::Assertion)                                                                                                                                 
      ./features/step_definitions/card_steps.rb:18:in `/^I should get a (.+) status code$/'                                                                        
      features/cards/tokenize.feature:38:in `Then I should get a 200 OK status code'                                            

And I used the schema here because that's what the YAML version had done. You're right that that test would end up having different ones.


@andrewsardone awesome, I will end up checking this out. 👍


I found , which might be a reasonable answer: I forgot that the actual Given/When/Then are for communication only. I like the idea of just adding a space and then describing the second bit.

This won't help reduce duplication, but it will make some things easier.


I used the SO technique to add the second follow-on scenario:

Two big problems:

  1. Our schemas claim to be version 4, but they're actually version 3.
  2. Our schemas aren't actually valid schemas. They use the required style of version 3, but the version 4 of everything else.

Using non-standard versions of these schemas make me :'(


I fixed the version 4 issue, still have to worry about relative URI resolution.

This test also now demonstrates two different ways of handling the schemas: inline and in a file. I think I like inline for shorter things, but file stuff for longer ones, rather than just choosing one or the other.


Okay! So now it's just failing because of a schema validation. For some reason, it properly de-references the relative schemas when I pass in a File object rather than just the parsed JSON: 0cf9ff5#diff-371d49fabfca7e84475425c67890d2f9R43

I pushed it up to Travis while it fails so you can see what a failure would look like:

Tomorrow morning I'm going to figure out why it's actually failing, and then we can start really cranking out the scenarios. 👍

@steveklabnik these are looking good. It's nice to have some real examples to help the discussion.

Are you still worried about chaining scenarios together, or is it working out for you just to have a longer scenario?

The danger with having fewer long scenarios is that the feedback you get when it fails can be crude (or even obscure other failures later in the scenario). Smaller scenarios generally give you better feedback when they fail, but you need to be pragmatic. It often makes sense to assert a few things in the same scenario as you're doing in that one.

When I want to have several scenarios that test different stages or paths through a workflow, I'll generally do it like this:

Scenario: Complete stage one
  Given I have asked for a loan
  When I fill out my name
  And I fill out my age
  And I submit the form
  Then I should be on stage 2

Scenario: Complete stage two
  Given I have completed stage 1
  When I fill out my address
  And I fill out my favourite colour
  And I submit the form
  Then I should be on stage 3

Scenario: Complete stage three
  Given I have completed stages 1 and 2
  When I fill out my weight

Point being, you roll up the details of the previous scenario into a single step. In the step def for that higher-level step, you can either use steps to literally delegate to Gherkin, or (preferably) just directly call the necessary Ruby to get the job done.


@mattwynne I'd still prefer ultimately to not have longer scenarios: for example, creating a card will be used by basically every other scenario, and I'd prefer to organize them by use case rather than have the same things over and over in the background. Not to mention the duplication, as well as what you just pointed out with regards to feedback.

I think that your example is basically perfect, it's just slightly annoying that when I call steps I have to enumerate all of the steps, when I know I always want to run every step in a given scenario. Does that make sense?

I really appreciate everyone on this thread. ❤️

Yeah, it's slightly annoying, but suck it up ;)

If it were me, I'd make sure I had a Ruby method called create_card and delegate to that in all those other scenarios. You only need one or two scenarios that explain how the card creation process works in detail. You might well find that those detailed scenarios actually do things you don't need to do in other scenarios that just need a card to exist.


Cool. Everyone, I added a rake task and formatter generously adapted from to handle CI, so I can work on these specs while we determine what exactly is up.

@steveklabnik steveklabnik Nuke everything 💣
Also, start over again with Cucumber, and import the fixture schemas
from before.

I am in a happy place with all of this now. @matthewfl, what do you think? Ready to give me a hand in porting other ones over?

matin commented Dec 13, 2013

@steveklabnik I can't open the first link in your last comment


@matin edited, that's what i get for copy/paste


I forgot to move the card tokenization file up to be like the API key feature, and to mention that I'd like to make one feature file per section in our docs. That's basically what I did with the API keys steps.

Sent from Mailbox for iPhone

On Thu, Dec 12, 2013 at 5:36 PM, Matin Tamizi

@steveklabnik I can't open the first link in your last comment

Reply to this email directly or view it on GitHub:
#431 (comment)

@steveklabnik steveklabnik "Tokenize a card without a secret" works.
There's a teeny bit of extra infrastructure to get a token. I built it by
accident, but we'll need it eventually, so I just left it in.

I added a checklist of scenarios that we have, so we can measure progress.


All of the CRUD scenarios for all the resources are complete. It does not make since to get a single event in most cases as everything provides a events index that can be used to list all events, eg: /debits/:debit_id/events

For referencing components of other scenarios, there is now hydrate that will substitute in :name_of_something in url and json body that are getting posted. We are still not really using the power of hypermedia as we are currently doing stuff like /cards/:card_id/debits whenever we need to reference another portion of a response.

steveklabnik and others added some commits Dec 10, 2013
@steveklabnik steveklabnik Set up rake tasks to make running features easier
Three things here:

1. I'm going to translate features over, and mark them as failing when
they are failing. I'll also tag them with gh-N to mark GitHub issue #N
that they go along with.
2. `bin/rake cruise` has been set up to run for CI. Basically, it runs all
the things that aren't failing, and makes sure they pass. Then, it runs
all thet hings that are supposed to be failing, and makes sure they
fail. If either of those are not true, it fails the build, and if
they both pass, they pass. This way we know when something starts
passing again, and to remove the failing tag.
3. `bin/rake focus` is added when developing new scenarios. You just tag
the scenario you're working on with `@focus`, and then `bin/rake focus`
will run just that one scenario.
@steveklabnik steveklabnik New feature: creating API keys.
There was no existing schemas for the API key response, so I made one.
@steveklabnik steveklabnik New feature: retrieve API keys e494c51
@steveklabnik steveklabnik New feature: list all API keys 82780ea
@steveklabnik steveklabnik New feature: remove API keys c3e3f5b
@steveklabnik steveklabnik Refactor/reorganize steps for clarity.
It's important not to refactor steps _too_ much, but these were certainly
http steps and not card steps.
@steveklabnik steveklabnik Update README. 65a81b8
@matthewfl @steveklabnik matthewfl moving root_url and accept_headers into global variables 8397390
@matthewfl @steveklabnik matthewfl adding card tokens schema 6f28333
@matthewfl @steveklabnik matthewfl Refactor schema error handling, reintroduce failure
We should handle these errors better, and now that we are,
let's reintroduce the failing second half of the scenario.

I'm actually not entirely sure that flunk-ing with the message
is any different than just bubbling up the error, but at least
it's a real assertion failure error, rather than a JSON API
parsing error.
@matthewfl @steveklabnik matthewfl basic create customer scenario 224b089
@matthewfl @steveklabnik matthewfl micro library to contain interaction with the api 43349be
@matthewfl @steveklabnik matthewfl New feature: Customer index c318fb3
@steveklabnik steveklabnik Adding technical documentation. 424c741
@steveklabnik steveklabnik clean up some whitespace 26bbee3
@steveklabnik steveklabnik New feature: get customer f385d4f
@steveklabnik steveklabnik 'fix' failing address spec.
This was failing, but for the wrong reasons. Let's fail for the right ones.
@steveklabnik steveklabnik Clean up some small formatting issues.
I missed these in my earlier cleanups.
@steveklabnik steveklabnik New feature: remove a customer 4d05364
@steveklabnik steveklabnik New feature: create a callback 146c870
@steveklabnik steveklabnik New feature: get callback info 6b9aae4
@steveklabnik steveklabnik refactor to use the client
Thanks @matthewfl
@matthewfl @steveklabnik matthewfl adding hydrater 1e61843
@matthewfl @steveklabnik matthewfl start changing over to hydrater bbb2eff
@matthewfl @steveklabnik matthewfl feature: debiting a card 7caf6fd
@matthewfl @steveklabnik matthewfl feature: Retrieving debits for a card e7eaf45
@steveklabnik steveklabnik New features: list/delete callbacks c88e3e7
@steveklabnik steveklabnik New feature: get a card 9a4cb93
@steveklabnik steveklabnik New features: list and remove cards 9a6603e
@steveklabnik steveklabnik client library refactor.
I took @matthewfl's work on the client library and fixed it up a bit.
There were some idiom issues, some conventions that we're followed,
and a couple of OOP things that _really_ bugged me.

I'm happy with this now, other than two things: we are doing OO terribly
with this whole response thing, and we should also implement DELETE.
@steveklabnik steveklabnik Implement PUT in the client. 26878a3
@steveklabnik steveklabnik New feature: update card 647f631
@matthewfl @steveklabnik matthewfl feature: debit a customer 7c2c8e6
@matthewfl @steveklabnik matthewfl spacing is hard 492b2f8
@matthewfl @steveklabnik matthewfl bank accounts feature failing as address is not coming back in response b6cfe2b
@matthewfl @steveklabnik matthewfl credit bank account feature ca0e231
@matthewfl @steveklabnik matthewfl basic crud for bank accounts 125f31b
@matthewfl @steveklabnik matthewfl bank account debit and credit scenarios 45654ef
@matthewfl @steveklabnik matthewfl bank account verification feature aaa5fc4
@matthewfl @steveklabnik matthewfl finishing debit scenarios and forcing json schema formatting fe968e5
@matthewfl @steveklabnik matthewfl refund features e52e3f9
@steveklabnik steveklabnik Fix broken callback scenario.
Thanks for setting me straight, @mjallday
@steveklabnik steveklabnik new feature: debit a card 72b5546
@steveklabnik steveklabnik Move features into rest
We'll keep these CRUD ones here, and do more intersting cases in the main folder.
@matthewfl @steveklabnik matthewfl customer add card/bank account 6ecfbee
@matthewfl @steveklabnik matthewfl credits features b7e3b68
@matthewfl @steveklabnik matthewfl reversal features 07fe36d
@matthewfl @steveklabnik matthewfl events schemas 8db133e

are you supposed to be checking the refund went to the original card (as opposed to the debit)


You fetch the debit, and then assert on the body of the debit, see 3 lines down where it checks for the original card id.


@steveklabnik @rserna2010

I am just quickly looking over the changes for the last few weeks, and I am noticing that there are a lot of scenarios that are marked @failing I am thinking that this is getting to the point of being a smell where we are allowing ourselves to write stuff that looks good, but doesn't actually function. I am thinking that we should go through and carefully review all the cases where we are making scenarios as failing and probably remove the failing tag all together and attempt to work out the minor issues in the api that originally provoked that tag.

In my quick browsing of the code here are two cases where @failing is being abused. (making a credit will not validate against the debit schema) (afaik we did not start supporting the ruby style #{})


@matthewfl yup, the idea was that @rserna2010 would be able to pitch in where he could to help keep moving this forward, since you were out. I told him that I'd handle making sure they actually worked, I just merged them in as @failing so it'd be simpler to knock them out.

When this gets merged there should be zero @failing.s

steveklabnik and others added some commits Jan 6, 2014
@steveklabnik steveklabnik Some movement toward #467 7736c9e
@steveklabnik steveklabnik Passing: reverse a credit on a bank account.
I had the wrong fixture data, so it was creating a failed credit, rather than a succeeded one.
@steveklabnik steveklabnik remove 'set the default funding source
In 1.1, only instruments can be credited, so a default funding source doesn't make sense.

See #460
@steveklabnik steveklabnik Some movement on underwriting a customer
See #468 for more.
@steveklabnik steveklabnik Moving forward on 'create an order'
There's still a failure, unsure which part is right or wrong.
@steveklabnik steveklabnik progress on checking escrow on orders.
Same delivery address issue
@steveklabnik steveklabnik Add in debugging steps.
Previously, @matthewfl had added some logging stuff, this is a bit
heavier, but often useful.
@steveklabnik steveklabnik Making some progress on 'checking escrow of order'
Erroring for the same reason the other Orders are.
@steveklabnik steveklabnik Making progress on 'orders can't be credited more...
... than escrow balance.
@steveklabnik steveklabnik Making progress on 'Create a refund' 5d33e55
@steveklabnik steveklabnik Update interpolation syntax
I missed this one because the scenario fails before it hit this part.
But it's pretty obvious it needs fixed.
@steveklabnik steveklabnik Typo fix: should be an order not an error
The scenario failed before this, but I want to catch it now. Bady
copy/paste on my part
@steveklabnik steveklabnik Making progress on create a reversal 51c6a76
@steveklabnik steveklabnik fix interpolation syntax
let's just do this all at once
@matthewfl matthewfl baby steps 4ce588a
@steveklabnik steveklabnik New feature: underwrite a customer
Closes #468
@matthewfl matthewfl Merge remote-tracking branch 'origin/cukes' into cukes
@steveklabnik steveklabnik making some movement on #471 585f67f
@steveklabnik steveklabnik Make progress on 'transactions inherit descriptions' 48ec874
@steveklabnik steveklabnik make some progress on crediting an unverified merchant.
#474 is tracking this particular failure.
@steveklabnik steveklabnik Making progress on 'cancelling an order' 413775d
@steveklabnik steveklabnik Making progress on 'existing buyer makes a purchase with a new card'
#476 has the report

And with this latest commit, we now have 100% of the scenarios ported over.

There are still failures due to backend bugs, but we're almost there!

matthewfl and others added some commits Jan 13, 2014
@matthewfl @steveklabnik matthewfl some cleaning 37eb86b
@steveklabnik steveklabnik making progress on 'existing buyer makes a purchase with an existing …

Yet another #469 victim.
@steveklabnik steveklabnik Making progress on 'new buyer makes a purchase'
Another #469.
@steveklabnik steveklabnik Progress on existing buyer makes a purchase with a new card.
Closes #476.
@rserna2010 @steveklabnik rserna2010 Add credit card scenarios. 3cb8958
@steveklabnik steveklabnik making progress on 'adding a card to a customer' 8a3ffbf
@steveklabnik steveklabnik New feature: AVS postal code matches 0fb808b
@steveklabnik steveklabnik New feature: AVS postal code does not match 0c1d810
@steveklabnik steveklabnik New scenario: AVS Postal card is unsupported 7de997d
@steveklabnik steveklabnik Making progress on 'AVS postal code is unused.
#438 will fix this.
@steveklabnik steveklabnik New feature: AVS street matches 5381186
@steveklabnik steveklabnik New feature: AVS street does not match 87aafe9
@steveklabnik steveklabnik Making progress on 'avs street match is null'
#438 will fix that.
@steveklabnik steveklabnik Progress on 'detect a visa card brand'
#438 fixes
@steveklabnik steveklabnik Make progress on 'detect a mastercard brand'
#438 fixes
@steveklabnik steveklabnik Make progress on 'detect an amex brand'
#438 fixes
@steveklabnik steveklabnik make progress on 'detect a discover card' ae18309
@steveklabnik steveklabnik Make some progress on retrieving card info
#480 is tracking this.
@steveklabnik steveklabnik Progress on 'tokenizing a card'
#438 is the problem.
@steveklabnik steveklabnik Making progress on luhn test failure e51d3a9
@steveklabnik steveklabnik new feature: unstore a card b47d8d0
@steveklabnik steveklabnik New feature: CVV matches 7924eb0
@steveklabnik steveklabnik progress on 'CVV doesn't match'
#438 fixes this.
@steveklabnik steveklabnik Progress on 'CVV is unsupported'
#438 strikes again
@steveklabnik steveklabnik Making progress on CVV is unused
#438 again
@steveklabnik steveklabnik 'Add a card to a customer' now works 2c2a4b0
@steveklabnik steveklabnik Feature fixed: Debit a card 9a9e032
@steveklabnik steveklabnik Making progress on 'basic order flow' 2e7a0e6
@matthewfl @steveklabnik matthewfl removing failing tag from card scenarios 9ba89de
@steveklabnik steveklabnik New feature: card metadata 84d3b4b
@steveklabnik steveklabnik New feature: fail to confirm a bank account verification e0ff12a
@steveklabnik steveklabnik progress on 'inferring bank names' 2044e73
@steveklabnik steveklabnik Make progress on 'add a bank account to a customer' 9513dd2
@matthewfl @steveklabnik matthewfl cleaning orders features 5f9f1c7
@matthewfl @steveklabnik matthewfl trying new link resolve system 1392564
@matthewfl @steveklabnik matthewfl fix e4bc413
@matthewfl @steveklabnik matthewfl another one working 2eec38b
@matthewfl @steveklabnik matthewfl fixing bank account rest scenarios 2770646
@matthewfl @steveklabnik matthewfl fixed 404 error 7d774b5
@matthewfl @steveklabnik matthewfl canceling an order f802d28
@matthewfl @steveklabnik matthewfl the end is nearing 4b13e43
@matthewfl @steveklabnik matthewfl one more to go 5a1a65f
@matthewfl @steveklabnik matthewfl I did it, its done, no more failing bf659a8
@matthewfl @steveklabnik matthewfl some issues with test credit card numbers df27867
@steveklabnik steveklabnik properly tag luhn test failure.
Should be linked to #481
@steveklabnik steveklabnik Tagging CVV is unsupported failure. 8027255
@matthewfl @steveklabnik matthewfl cvv card number scenarios working 79702b8
@steveklabnik steveklabnik Properly tagging cvv does not match a6fa221
@matthewfl @steveklabnik matthewfl it is all working now f08a98d

I am currently re-running the tests over every commit to make sure that bisect works. Assuming that's successful, it's time to finally merge this sucker.



@steveklabnik I just noticed that there are some scenarios that do not appear to be implemented, might we double check that we have everything


Okay! I have re-gone through and double checked. We were missing one or two. Let's get that CI building.


🎊 :shipit:

@steveklabnik steveklabnik merged commit 7c22c05 into revision1 Jan 24, 2014

1 check passed

default The Travis CI build passed

oh snap! 💃

@steveklabnik steveklabnik referenced this pull request in apiaryio/api-blueprint Jan 29, 2014

Hypermedia #13

@steveklabnik have you looked at 'behave' instead of Cucumber? It's Python and allegedly better than Lettuce.


@julianpistorius I have not, no.

@steveklabnik steveklabnik deleted the cukes branch Feb 5, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment