Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Waiting on #174] WIP - Add rake task to sync items with Amazon #175

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

[Waiting on #174] WIP - Add rake task to sync items with Amazon #175

wants to merge 8 commits into from

Commits on Nov 29, 2017

  1. Extract endpoint superclass 

    I'm pretty torn on this commit-an abstract superclass doesn't seem like
a very ruby thing to do. Maybe I've been doing too much Java?

There are two main goals of this commit:

  1. It should be trivial to add new endpoints. We're going to have to
     add a few more, and being able to customize only the specialized
     bits will save a ton of time and prevent bugs.

  2. Shared code should exist in one location. Already, we've got a bug
     in our signing process, and keeping this all in a superclass means
     we only need to fix it in one place.

I don't like how I'm handling `@aws_credentials` at the moment
(requiring each subclass to initialize it); any ideas of a better way to
handle it?
    @leesharma
    leesharma committed Nov 29, 2017
    Configuration menu
    Copy the full SHA
    7450a60 View commit details
    Browse the repository at this point in the history

  2. Add Amazon Product API README 

    Since the instructions are getting more complicated, this commit adds a
README (with new endpoint instructions) to the Amazon Product API. This
has three benefits:

  * It keeps the root README clean and relevant to most people.
  * It allows us to go into more documentation detail.
  * It facilitates an eventual extraction of the Amazon Product API lib.

It might be worth extracting more from the "Getting Amazon Product
Advertising API Working Locally" section into the lib README, but for
now there's just endpoint information.
    @leesharma
    leesharma committed Nov 29, 2017
    Configuration menu
    Copy the full SHA
    4cd254a View commit details
    Browse the repository at this point in the history

  3. Only load AWS test credentials once 

    Whenever the tests run, the following error appeared:

    warning: already initialized constant AWSTestCredentials

This moves the AWSTestCredentials init from the individual specs to a
spec helper class: `spec/support/helpers/amazon_helpers.rb`.
    @leesharma
    leesharma committed Nov 29, 2017
    Configuration menu
    Copy the full SHA
    3b0bb6a View commit details
    Browse the repository at this point in the history

  4. Add items:sync rake task 

    `rake items:sync` will sync all items in the database. Some chunking was
done in `ItemSyncJob` in order to get around Amazon's rate limitations.

TODO:
  * Batch the Amazon query (ItemLookup can take up to 10 ASINs)
  * Add a delayed job queue (new issue?)
  * Add specs
    @leesharma
    leesharma committed Nov 29, 2017
    Configuration menu
    Copy the full SHA
    fd43cfc View commit details
    Browse the repository at this point in the history

  5. Rubocop fixes

    @leesharma
    leesharma committed Nov 29, 2017
    Configuration menu
    Copy the full SHA
    07a6d62 View commit details
    Browse the repository at this point in the history

  6. Add item sync specs 

    A couple notes:

  1. I added a new exception to rubocop to allow for the
     `expect { action_a }.to change { dynamic_attr }` form.
  2. The 2 sec Amazon rate limit delay doesn't apply to tests.

All tests are green and no new rubocop violations are added.
    @leesharma
    leesharma committed Nov 29, 2017
    Configuration menu
    Copy the full SHA
    85757bd View commit details
    Browse the repository at this point in the history

  7. Item lookup endpoint supports up to 10 ASINs 

    As a first step for batching the requests, this commit makes the client
endpoint take a list of asins instead of just an individual one. Other
files are changed in order to work with the new interface (asins ->
items)

Next up: bulk-searching with the items sync job!
    @leesharma
    leesharma committed Nov 29, 2017
    Configuration menu
    Copy the full SHA
    1229548 View commit details
    Browse the repository at this point in the history

  8. Batch the item sync job 

    This commit takes advantage of the multi-asin item lookup requests and
performs the item sync job in batches of 10.
    @leesharma
    leesharma committed Nov 29, 2017
    Configuration menu
    Copy the full SHA
    3da732f View commit details
    Browse the repository at this point in the history