Skip to content

bmwracer0/data-engineering

 
 

Repository files navigation

Welcome to The Importer

Running the app:

After cloning the code, cd into the directory, and run bundle install to pull down the required dependencies. Following the dependency resoultion, run rake db:migrate in order to create the required tables. Finally, start the server with rails server. You can now open your favorite browser, and navigate to localhost:3000. You'll be able to login with your Google account, which is the OpenID provider I chose to implement, since it's one of the most popular account types on the web. It would be trivial to add additional providers.

Testing:

You can run the Rspec tests for this application by first running rake db:test:prepare to prepare the database, and then running rspec from the root of the project directory.

Future:

If this app was to be used in a production environment, it might be recommended to do the processing asyncronously by using a queue or something similar. Parsing the CSV during the request is not optimal from a user's point of view, because large files may appear to make the app unresponsive. In addition to the asyncronous processing, but it might be useful to save the files on disk somewhere for archiving purposes, rather than store the contents of the file in the database.

Sorry this was lumped in all as one commit, I started working on it with a different Github account (the one I use for work stuff) and wanted to move it over to a different account.

About

Challenge for those applying to the Software Engineer, Big Data position

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Ruby 98.1%
  • JavaScript 1.4%
  • CoffeeScript 0.5%