New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support response stubbing for all instances of a service client #187

Closed
kalpitad opened this Issue Jan 2, 2015 · 22 comments

Comments

Projects
None yet
6 participants
@kalpitad

kalpitad commented Jan 2, 2015

This is the follow up issue from our discussion in the comments of this blog post: http://ruby.awsblog.com/post/Tx15V81MLPR8D73/Client-Response-Stubs

My goal is to set stub_responses in my test code that will impact the DynamoDB calls in my app code, when the tests are run.

My Rspec test code sets Aws.config[:dynamodb] = {stub_responses: true} in a helper file. However, since my code calls Aws::DynamoDB::Client.new.query (i.e. I always create new clients), how can my test code call the stub_responses method on a client instance that it doesn't have access to? I would like a way to be able to stub responses for all instances of a service client (e.g. DynamoDB), so that I can affect the return values of the clients within my app code when the test is running.

Thanks!

@trevorrowe

This comment has been minimized.

Show comment
Hide comment
@trevorrowe

trevorrowe Jan 12, 2015

Member

To support this, the SDK would need to support statically providing response stubs for a service client. For example:

Aws.config[:dynamodb] = {
  stub_responses: true,
  stubbed_responses: {
    list_tables: ...,
    get_item: ...
  }
}

This would not be very difficult, but users relying on a globally configured list of stubbed responses would need to be certain to clear this shared state between tests:

# or replace the stubs
Aws.config[:dynamodb].delete(:stubbed_responses)

Another option would be to monkey patch Aws::DynamoDB::Client.new to provide the proper stubs:

class Aws::DynamoDB::Client
  def self.new(options = {})
     client = super(options.merge(stub_responses: true))
     # don't hard-code these, but instead load them from a fixtures file perhaps
     client.stub_responses(:list_tables, table_names:['aws-sdk'])
     client
  end
end

A more expanded example might look like:

Given fixtures/dynamodb_stubs.json as:

{
  "list_tables": { "table_names": ["aws-sdk"] }
}
class Aws::DynamoDB::Client
  def self.new(options = {})
     client = super(options.merge(stub_responses: true))
     # don't hard-code these, but instead load them from a fixtures file perhaps
     stubs = MultiJson.load(File.read('fixtures/dynamodb_subs.json'))
     stubs.each do |operation_name, responses|
       client.stub_resposnes(operation_name, responses)
     end
     client
  end
end

Thoughts?

Member

trevorrowe commented Jan 12, 2015

To support this, the SDK would need to support statically providing response stubs for a service client. For example:

Aws.config[:dynamodb] = {
  stub_responses: true,
  stubbed_responses: {
    list_tables: ...,
    get_item: ...
  }
}

This would not be very difficult, but users relying on a globally configured list of stubbed responses would need to be certain to clear this shared state between tests:

# or replace the stubs
Aws.config[:dynamodb].delete(:stubbed_responses)

Another option would be to monkey patch Aws::DynamoDB::Client.new to provide the proper stubs:

class Aws::DynamoDB::Client
  def self.new(options = {})
     client = super(options.merge(stub_responses: true))
     # don't hard-code these, but instead load them from a fixtures file perhaps
     client.stub_responses(:list_tables, table_names:['aws-sdk'])
     client
  end
end

A more expanded example might look like:

Given fixtures/dynamodb_stubs.json as:

{
  "list_tables": { "table_names": ["aws-sdk"] }
}
class Aws::DynamoDB::Client
  def self.new(options = {})
     client = super(options.merge(stub_responses: true))
     # don't hard-code these, but instead load them from a fixtures file perhaps
     stubs = MultiJson.load(File.read('fixtures/dynamodb_subs.json'))
     stubs.each do |operation_name, responses|
       client.stub_resposnes(operation_name, responses)
     end
     client
  end
end

Thoughts?

@jtrost

This comment has been minimized.

Show comment
Hide comment
@jtrost

jtrost Jan 23, 2015

I would like to see something similar to AWS.stub!. In my Rails application I create S3 buckets and Cloudfront distributions in a service object, which is called from an after_create method. Because of this setup, using the stub_responses: true option isn't practical.

Using the webmock gem, I am able to kind of accomplish this with:

before(:each) do
    stub_request(:any, /cloudfront.amazonaws.com/).to_return({distribution: {id: "ABCDEFGHIJ"}})
end

However, the returned object is coerced into an instance of Aws::PageableResponse, so I cannot access the stubbed distribution ID. I think there's a way for me to work around this, but it would be nice if stubbing requests was easier.

jtrost commented Jan 23, 2015

I would like to see something similar to AWS.stub!. In my Rails application I create S3 buckets and Cloudfront distributions in a service object, which is called from an after_create method. Because of this setup, using the stub_responses: true option isn't practical.

Using the webmock gem, I am able to kind of accomplish this with:

before(:each) do
    stub_request(:any, /cloudfront.amazonaws.com/).to_return({distribution: {id: "ABCDEFGHIJ"}})
end

However, the returned object is coerced into an instance of Aws::PageableResponse, so I cannot access the stubbed distribution ID. I think there's a way for me to work around this, but it would be nice if stubbing requests was easier.

@kalpitad

This comment has been minimized.

Show comment
Hide comment
@kalpitad

kalpitad Jan 26, 2015

@trevorrowe -- my apologies for the long delay in replying. I'm not exactly clear on how the monkey patch would work.

Is the following correct?

  1. Add the monkey patch to my test code
  2. My test runs the monkey patch on the Aws::DynamoDB::Client class first (e.g. in a before block) and then invokes a part of my app code.
  3. Using your example, if my app code then calls Aws::DynamoDB::Client.new.list_tables, the values from the stubbed responses in my monkey patch will be returned.

If this flow is correct, it would be probably work fine for me. Let me know and I'll put it on my todo list to try out. Thanks!

@trevorrowe -- my apologies for the long delay in replying. I'm not exactly clear on how the monkey patch would work.

Is the following correct?

  1. Add the monkey patch to my test code
  2. My test runs the monkey patch on the Aws::DynamoDB::Client class first (e.g. in a before block) and then invokes a part of my app code.
  3. Using your example, if my app code then calls Aws::DynamoDB::Client.new.list_tables, the values from the stubbed responses in my monkey patch will be returned.

If this flow is correct, it would be probably work fine for me. Let me know and I'll put it on my todo list to try out. Thanks!

@trevorrowe

This comment has been minimized.

Show comment
Hide comment
@trevorrowe

trevorrowe Jan 26, 2015

Member

@kalpitad Doing this in a before block would be a bit tricky, as the patch is persistent. This could allow for state leak between tests. The patch would need some modifications to do this one when setting up your tests (i.e. in a spec or test helper). Then it could use a shared / global list of stubs. Its not pretty, but a workable solution is possible.

@jtrost You can simulate AWS.stub! with the following:

Aws.config[:stub_responses] = true

To specify the stubs when using the resource interfaces, you have to access the client object:

# enable stubs globally
Aws.config[:stub_responses] = true

# stub responses for a single client
s3 = Aws::S3::Resource.new
s3.client.stub_responses(:list_buckets, { buckets: ['aws-sdk'] })

s3.buckets.map(&:name)
#=> ['aws-sdk']
Member

trevorrowe commented Jan 26, 2015

@kalpitad Doing this in a before block would be a bit tricky, as the patch is persistent. This could allow for state leak between tests. The patch would need some modifications to do this one when setting up your tests (i.e. in a spec or test helper). Then it could use a shared / global list of stubs. Its not pretty, but a workable solution is possible.

@jtrost You can simulate AWS.stub! with the following:

Aws.config[:stub_responses] = true

To specify the stubs when using the resource interfaces, you have to access the client object:

# enable stubs globally
Aws.config[:stub_responses] = true

# stub responses for a single client
s3 = Aws::S3::Resource.new
s3.client.stub_responses(:list_buckets, { buckets: ['aws-sdk'] })

s3.buckets.map(&:name)
#=> ['aws-sdk']
@trevorrowe

This comment has been minimized.

Show comment
Hide comment
@trevorrowe

trevorrowe Jan 26, 2015

Member

It sounds like it would be helpful if you could enable stubs and specify stubbed data globally as well as per instance. Would the following be helpful?

# global stubbing
Aws.config[:s3] = {
  stub_responses: true
  stubs: {
    list_buckets: { buckets: 'aws-sdk' }
  }
}

# per client or per service resource
s3 = Aws::S3::Resource.new(  
  stub_responses: true
  stubs: {
    list_buckets: { buckets: 'aws-sdk' }
  }
)

Thoughts?

Member

trevorrowe commented Jan 26, 2015

It sounds like it would be helpful if you could enable stubs and specify stubbed data globally as well as per instance. Would the following be helpful?

# global stubbing
Aws.config[:s3] = {
  stub_responses: true
  stubs: {
    list_buckets: { buckets: 'aws-sdk' }
  }
}

# per client or per service resource
s3 = Aws::S3::Resource.new(  
  stub_responses: true
  stubs: {
    list_buckets: { buckets: 'aws-sdk' }
  }
)

Thoughts?

@ktheory

This comment has been minimized.

Show comment
Hide comment
@ktheory

ktheory Jan 27, 2015

Contributor

TIL about Aws.config[:stub_responses] = true. Yay! 🎉 That's exactly what I want in my test helpers.

(@trevorrowe: would be great if you mentioned that on your blog post about stubbing.)

Contributor

ktheory commented Jan 27, 2015

TIL about Aws.config[:stub_responses] = true. Yay! 🎉 That's exactly what I want in my test helpers.

(@trevorrowe: would be great if you mentioned that on your blog post about stubbing.)

@jtrost

This comment has been minimized.

Show comment
Hide comment
@jtrost

jtrost Jan 28, 2015

Thanks @trevorrowe. Aws.config[:stub_responses] = true works perfectly! Is this documented anywhere? I was looking around for this solution for a while before asking here.

jtrost commented Jan 28, 2015

Thanks @trevorrowe. Aws.config[:stub_responses] = true works perfectly! Is this documented anywhere? I was looking around for this solution for a while before asking here.

@trevorrowe

This comment has been minimized.

Show comment
Hide comment
@trevorrowe

trevorrowe Jan 29, 2015

Member

@ktheory I've updated the blog post with mention of Aws.config[:stub_responses] = true. Thanks for the suggestion.

@jtrost Is is documented in each of the client constructors, and then Aws.config is briefly documented as a set of default params applied to each client. Its very non-obvious at this point. Currently I am working on putting together a developer guide that should cover these sort of features.

Member

trevorrowe commented Jan 29, 2015

@ktheory I've updated the blog post with mention of Aws.config[:stub_responses] = true. Thanks for the suggestion.

@jtrost Is is documented in each of the client constructors, and then Aws.config is briefly documented as a set of default params applied to each client. Its very non-obvious at this point. Currently I am working on putting together a developer guide that should cover these sort of features.

@trevorrowe

This comment has been minimized.

Show comment
Hide comment
@trevorrowe

trevorrowe Jun 12, 2015

Member

This has been resolved now as of v2.1.0. You can specify the stub responses to Aws.config.

Aws.config[:s3] = {
  stub_responses: {
    list_buckets: { buckets:[{name:'aws-sdk'}]}
  }
}

Aws::S3::Client.new.list_buckets.buckets.map(&:name)
#=> ["aws-sdk"]

I plan to blog about this and a few other of the new 2.1 features shortly.

Member

trevorrowe commented Jun 12, 2015

This has been resolved now as of v2.1.0. You can specify the stub responses to Aws.config.

Aws.config[:s3] = {
  stub_responses: {
    list_buckets: { buckets:[{name:'aws-sdk'}]}
  }
}

Aws::S3::Client.new.list_buckets.buckets.map(&:name)
#=> ["aws-sdk"]

I plan to blog about this and a few other of the new 2.1 features shortly.

@trevorrowe trevorrowe closed this Jun 12, 2015

@kalpitad

This comment has been minimized.

Show comment
Hide comment

Awesome!

@ktheory

This comment has been minimized.

Show comment
Hide comment
@ktheory

ktheory Jun 16, 2015

Contributor

Yay, thanks @trevorrowe!

Contributor

ktheory commented Jun 16, 2015

Yay, thanks @trevorrowe!

@kalpitad

This comment has been minimized.

Show comment
Hide comment
@kalpitad

kalpitad Jul 3, 2015

Hi @trevorrowe, I'm trying to stub a DynamoDB ProvisionedThroughputExceededException. After looking at the current docs and the old SDK blog, I still couldn't get this to work. What am I missing?

    Aws.config[:dynamodb] = {
      stub_responses: {
        query: { error: Aws::DynamoDB::Errors::ProvisionedThroughputExceededException}
      }
    }

I'm doing my stubbing in Rspec, so my other question is how to turn stubbing on (e.g. in a before block) and then off (e.g. in the after block), so that other tests aren't affected?

Thanks and Happy 4th!

kalpitad commented Jul 3, 2015

Hi @trevorrowe, I'm trying to stub a DynamoDB ProvisionedThroughputExceededException. After looking at the current docs and the old SDK blog, I still couldn't get this to work. What am I missing?

    Aws.config[:dynamodb] = {
      stub_responses: {
        query: { error: Aws::DynamoDB::Errors::ProvisionedThroughputExceededException}
      }
    }

I'm doing my stubbing in Rspec, so my other question is how to turn stubbing on (e.g. in a before block) and then off (e.g. in the after block), so that other tests aren't affected?

Thanks and Happy 4th!

@trevorrowe

This comment has been minimized.

Show comment
Hide comment
@trevorrowe

trevorrowe Jul 6, 2015

Member

@kalpitad If you are stubbing a response error, you can give just the error code:

Aws.config[:dynamodb] = {
  stub_responses: {
    query: 'ProvisionedThroughputExceededException'
  }
}

To disable stubbing, simply remove the relevant option from Aws.config:

Aws.config[:dynamodb].delete(:stub_responses)
Member

trevorrowe commented Jul 6, 2015

@kalpitad If you are stubbing a response error, you can give just the error code:

Aws.config[:dynamodb] = {
  stub_responses: {
    query: 'ProvisionedThroughputExceededException'
  }
}

To disable stubbing, simply remove the relevant option from Aws.config:

Aws.config[:dynamodb].delete(:stub_responses)
@kalpitad

This comment has been minimized.

Show comment
Hide comment
@kalpitad

kalpitad Jul 6, 2015

Thanks @trevorrowe!

Update: The stubbing is working. It turns out that I had a separate issue with my DynamoDB local, which was causing an error. I think I'm all good now! :)

kalpitad commented Jul 6, 2015

Thanks @trevorrowe!

Update: The stubbing is working. It turns out that I had a separate issue with my DynamoDB local, which was causing an error. I think I'm all good now! :)

@iDiogenes

This comment has been minimized.

Show comment
Hide comment
@iDiogenes

iDiogenes Sep 16, 2015

@trevorrowe the stub_responses works great for the client object, but I am having trouble resource objects when globally stubbing. I am trying to write a test that will stub out a S3 move_to request which is in S3::Object. Trying to add move_to into the global stub_responses throws an error during Aws::S3::Client.new, which is no surprise. Is what I am trying to do possible on a global level?

Aws.config[:s3] = {
            stub_responses: {
                list_objects: { contents: [{key: 'offers/1/img.jgp', storage_class: "STANDARD"}]},
                move_to: { delete_marker: true}
            }
        }

@trevorrowe the stub_responses works great for the client object, but I am having trouble resource objects when globally stubbing. I am trying to write a test that will stub out a S3 move_to request which is in S3::Object. Trying to add move_to into the global stub_responses throws an error during Aws::S3::Client.new, which is no surprise. Is what I am trying to do possible on a global level?

Aws.config[:s3] = {
            stub_responses: {
                list_objects: { contents: [{key: 'offers/1/img.jgp', storage_class: "STANDARD"}]},
                move_to: { delete_marker: true}
            }
        }
@trevorrowe

This comment has been minimized.

Show comment
Hide comment
@trevorrowe

trevorrowe Sep 16, 2015

Member

@iDiogenes It would be helpful to see more context, including a stack trace on the raised error. Also, would you consider posting this as an issue instead against aws/aws-sdk-ruby?

Member

trevorrowe commented Sep 16, 2015

@iDiogenes It would be helpful to see more context, including a stack trace on the raised error. Also, would you consider posting this as an issue instead against aws/aws-sdk-ruby?

@iDiogenes

This comment has been minimized.

Show comment
Hide comment
@iDiogenes

iDiogenes Sep 17, 2015

@trevorrowe Happy to post as in issue if you think it warrants it rather than just a mistake on my part. For context I have the following code in a ruby module

bucket = Aws::S3::Bucket.new(Rails.application.secrets.aws['bucket'])
object = bucket.object(s3_obj.key)
object.move_to(bucket.object(s3_obj.key.split('archive/').last), acl: 'public-read')

I would like to stub out the response of object.move_to for testing purposes. When I try and add it to Aws.config[:s3] as previously shown, I receive the following error when: Aws::S3::Bucket.new(Rails.application.secrets.aws['bucket']) is run.

ArgumentError: unknown operation :move_to

@trevorrowe Happy to post as in issue if you think it warrants it rather than just a mistake on my part. For context I have the following code in a ruby module

bucket = Aws::S3::Bucket.new(Rails.application.secrets.aws['bucket'])
object = bucket.object(s3_obj.key)
object.move_to(bucket.object(s3_obj.key.split('archive/').last), acl: 'public-read')

I would like to stub out the response of object.move_to for testing purposes. When I try and add it to Aws.config[:s3] as previously shown, I receive the following error when: Aws::S3::Bucket.new(Rails.application.secrets.aws['bucket']) is run.

ArgumentError: unknown operation :move_to
@trevorrowe

This comment has been minimized.

Show comment
Hide comment
@trevorrowe

trevorrowe Sep 17, 2015

Member

@iDiogenes Oh, I see the issue. There is no client operation call #move_to that you can stub. The Aws::S3::Object#move_to method actually calls multiple operations, #copy_object followed by a #delete_object.

Do you need a specific response from the object.move_to call? I suspect not, so you can probably get away with simply enabling response stubbing, without specific responses.

Aws.config[:s3] = { stub_responses: true }
Member

trevorrowe commented Sep 17, 2015

@iDiogenes Oh, I see the issue. There is no client operation call #move_to that you can stub. The Aws::S3::Object#move_to method actually calls multiple operations, #copy_object followed by a #delete_object.

Do you need a specific response from the object.move_to call? I suspect not, so you can probably get away with simply enabling response stubbing, without specific responses.

Aws.config[:s3] = { stub_responses: true }
@iDiogenes

This comment has been minimized.

Show comment
Hide comment
@iDiogenes

iDiogenes Sep 18, 2015

@trevorrowe A specific response would be helpful. Currently the stub shows a delete_marker of false, which is the response for when the delete part of the #move_to fails. Therefore, with the stub I can only test what happens during a failed attempt and not the successful ones. Another example would be testing the response of

Aws::S3::Object.new(bucket_name: Rails.application.secrets.aws['bucket'], key: s3_obj.key).restore

In practice it returns a string of: "ongoing-request="true"", "ongoing-request="false"" or nil. When using

Aws.config[:s3] = { stub_responses: true }

The response is "Restore", which is not useful for testing code logic that depends on the result of #restore

@trevorrowe A specific response would be helpful. Currently the stub shows a delete_marker of false, which is the response for when the delete part of the #move_to fails. Therefore, with the stub I can only test what happens during a failed attempt and not the successful ones. Another example would be testing the response of

Aws::S3::Object.new(bucket_name: Rails.application.secrets.aws['bucket'], key: s3_obj.key).restore

In practice it returns a string of: "ongoing-request="true"", "ongoing-request="false"" or nil. When using

Aws.config[:s3] = { stub_responses: true }

The response is "Restore", which is not useful for testing code logic that depends on the result of #restore

@trevorrowe

This comment has been minimized.

Show comment
Hide comment
@trevorrowe

trevorrowe Sep 18, 2015

Member

@iDiogenes The default stubbed responses are really just placeholders. If you prefer to specify the values for the response, you need to configure them via :stub_responses, or you can use your own testing framework to return stubbed responses. This can be useful if you only care the stub the final operation. For example:

# example using rspec
resp = obj.client.stub_data(:delete_object,  version_id:'id', delete_marker: true)
allow(obj).to recieve(:restore).and_return(resp)

You can use any instance of any service client and call #stub_data. This will return the structured data object for the named operation. If you pass a hash, it will merged with the placeholder values. It will also validate the structure to ensure you have no typos.

Member

trevorrowe commented Sep 18, 2015

@iDiogenes The default stubbed responses are really just placeholders. If you prefer to specify the values for the response, you need to configure them via :stub_responses, or you can use your own testing framework to return stubbed responses. This can be useful if you only care the stub the final operation. For example:

# example using rspec
resp = obj.client.stub_data(:delete_object,  version_id:'id', delete_marker: true)
allow(obj).to recieve(:restore).and_return(resp)

You can use any instance of any service client and call #stub_data. This will return the structured data object for the named operation. If you pass a hash, it will merged with the placeholder values. It will also validate the structure to ensure you have no typos.

@iDiogenes

This comment has been minimized.

Show comment
Hide comment
@iDiogenes

iDiogenes Sep 18, 2015

"If you prefer to specify the values for the response, you need to configure them via :stub_responses" Correct, that is exactly what I am trying to do and where my issue is happening. The #stub_data looks perfect, but it does not seem to work when setting in the Aws.config

Aws.config[:s3] = {
            stub_responses: {
                list_objects: { contents: [{key: 'archive/offers/1/img.jgp', storage_class: "GLACIER"}]},
            },
            stub_data: {
                delete_object: { delete_marker: true}
            }
        }

Throws ArgumentError: invalid configuration option:stub_data'`

The issue I am having is the same as the individual who created this thread. "how can my test code call the stub_responses method on a client instance that it doesn't have access to? I would like a way to be able to stub responses for all instances of a service client (e.g. DynamoDB), so that I can affect the return values of the clients within my app code when the test is running." My client is being initiated within the method that needs to be tested so I can't send in the object with the stub response set, it needs to be set globally.

"If you prefer to specify the values for the response, you need to configure them via :stub_responses" Correct, that is exactly what I am trying to do and where my issue is happening. The #stub_data looks perfect, but it does not seem to work when setting in the Aws.config

Aws.config[:s3] = {
            stub_responses: {
                list_objects: { contents: [{key: 'archive/offers/1/img.jgp', storage_class: "GLACIER"}]},
            },
            stub_data: {
                delete_object: { delete_marker: true}
            }
        }

Throws ArgumentError: invalid configuration option:stub_data'`

The issue I am having is the same as the individual who created this thread. "how can my test code call the stub_responses method on a client instance that it doesn't have access to? I would like a way to be able to stub responses for all instances of a service client (e.g. DynamoDB), so that I can affect the return values of the clients within my app code when the test is running." My client is being initiated within the method that needs to be tested so I can't send in the object with the stub response set, it needs to be set globally.

@rocifier

This comment has been minimized.

Show comment
Hide comment
@rocifier

rocifier Sep 6, 2017

Hi, how can I use the above technique to stub the responses of an S3::Bucket object in production code? Specifically I am trying to mock the response for #objects(prefix). Am I correct in seeing that you can only stub methods of S3::Client ??

rocifier commented Sep 6, 2017

Hi, how can I use the above technique to stub the responses of an S3::Bucket object in production code? Specifically I am trying to mock the response for #objects(prefix). Am I correct in seeing that you can only stub methods of S3::Client ??

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment