An app to search startup jobs scraped from websites written in Elixir, Phoenix, React and styled-components.
Clone or download
Latest commit 5fe1afc May 22, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.deliver Bump up to 0.1.1 May 20, 2017
apps Remove an unused file May 22, 2017
config Create Umbrella application Dec 30, 2016
rel Fix search bug May 21, 2017
.gitignore Fix config settings Apr 2, 2017
LICENSE Add more guideline May 21, 2017
README.md Fix a typo May 21, 2017
mix.exs Make deploy setup Mar 30, 2017
mix.lock Use cron for Customer jobs May 18, 2017

README.md

license Deps Status

Startup Job

Startup Job is a sample project to search startup jobs scraped from various websites written in Elixir/Phoenix(Backend) and React/Redux(Frontend).

This project is an umbrella project.

  • Customer app is for processing data and rendering contents.
  • Scraper app is for scraping data from websites.

Demo

http://demo.job-search.tsurupin.com/

Motivation

I created this app to understand Elixir/OTP and get more familliar with React.

Main Technology Stack

Requirements

  • Elixir 1.4+
  • Phoenix 1.3+
  • Node 7.0+
  • PostgreSQL 9.4+
  • Elasticsearch

Development

Setup

  1. Get the repo.

     % git clone git@github.com:tsurupin/job_search.git
    
  2. Install Elasticsearch in local environment.

    Mac OS

  3. Change username and password of PostgreSQL

    % vi apps/customer/config/dev.exs
    
  4. Setup your environment.

     % cd apps/customer
     % mix deps.get
     % mix ecto.setup
     % cd assets
     % npm install 
    
  5. Scrape data

     % cd ../../scraper         
     % mix deps.get 
     % iex -S mix
     % Scraper.Site.Accel.Show.perform("http://google/com", "Test", "Software engineer", "San Francisco, CA, US", :test)
     % Scraper.Site.A16z.Show.perform("http://google/com", "Sample", "Senior software engineer", "Seattle, WA, US", :test)
     % Scraper.Site.Sequoia.Show.perform("http://google/com", :test)
     % Customer.Builder.EsReindex.perform  
    
  6. Create a new OAuth account(URL) (Optional. Google OAuth account is needed to login and logout)

     1. Click `Create credentials` and Choose OAuth client ID
     2. Select Web Application and Set Authorizedredirect URIs as `http://localhost:4000/auth/google/callback`
     3. Set Client ID, Client secret and Authorized redirect URI of your OAuth account in apps/customer/config/dev.exs
    
  7. Run customer application.

     % cd ../customer
     % mix phx.server
    
  8. Verify that the app is up and running.

     % open http://localhost:4000
    

Todo

  1. Make Selenium work in background in server.
  2. Enable users to download their favorite jobs in csv.
  3. Synchronize favorite jobs with Google Sheets.

License

The project is available as open source under the terms of the MIT License.

Troubleshooting

Please create an issue.