Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
app
 
 
bin
 
 
 
 
db
 
 
 
 
lib
 
 
log
 
 
 
 
 
 
tmp
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Table of Contents

  1. Introduction
  2. Installation
  3. Backend Features
  4. Surf Database
  5. JSON Web Api
  6. React Native App
  7. Forecast Engine
    a. Retrieving Weather Forecast via Api
    b. Creating Cron Jobs
    c. Calculating Surf Forecast
  8. Things To Improve

Introduction

Get it on Google Play

Installation Instructions

ruby version ruby 2.5.0

rails 5.1.3

Installation

git clone git@github.com:fabriziobertoglio1987/surfbackend.git
bundle install
yarn install

run rspec for running test suite. Specs are currently not completed and some will fail. They will be fixed in the future.

Backend Features

  • API/Web Authentication was built with Devise as explained in this stackoverflow answer, simple token authentication. I enhanced the devise registration and sessions controllers to handle API-Authentication
  • Web/API pictures upload built with carrierwave and the following solution
  • Responsive WebPage built with Twitter-Bootstrap 4
  • Geocoder to reverse geocode database entries based on the GPS latitude and longitude coordinates
  • Geospatial Queries by user coordinates or bounding box
  • Offering over 10.000 surfspots information in the world with gps coordinates, pictures, wave forecast and surfspot information available in the webapplication, Iphone and Android native apps and json-api.
  • Sidekiq and Cron-Sidekiq jobs to retrieve and calculate forecasts information
  • Hosting on Digital Ocean
  • Native Mobile Iphone and Android application

The OpenSource project is a mirror of the backend and includes 99% of the functionalities. Some features are kept private and they are not disclosed to the public.

Surf Database

The surf-rails backend includes around 10.000 surfspots from Europe, Oceania, Africa, America and Asia, it's the first public api endpoint, currently the magicseaweed api does not provide this information.

Json Web Api

Each SurfSpot is represented as a location model in Ruby on Rails and it is reverse geocoded with the longitude and latitude. Each location includes information relative to the surf spot which are essential to calculate the surf forecast, for example best_wind_direction and best_swell_direction to display the green and red indicators in the mobile app. The green/red indicator represent wind or waves coming from optimal/non-optimal directions (offshore or onshore).

The Mobile App retrieves the information through the /locations endpoint. The /locations endpoint accepts three type of queries:

  1. Locations filtered by distance from specific coordinates (#set_nearby_locations)
  2. Locations filtered within a bounding box (#set_locations_with_box)
  3. Locations that have videos of the surf conditions (#set_locations_with_cameras)

An example curl request:

curl --location --request GET 'https://surfcheck.xyz/api/v1/posts.json?longitude=115.165524&latitude=-8.730649&page=1&per_page=4' \
--header 'X-User-Email: testing@torino1' \
--header 'X-User-Token: bGEfYLE72KtkGyQ21bcP' \
--header 'Accept: {{Accept}}' \
--header 'Content-Type: {{Content}}' \
--data-raw ''

More updated information and test cases are available in the /locations api endpoint documentation.

React Native App

The react native app retrieves from the /locations api endpoint the locations, forecast, videos information in the Locations component using the Api class. The information are retrieved at the application startup and cached. The same endpoint is used to display the different surfspots during map navigation with the Map component and the Map class.

The MapScreen renders the MapView Component and uses the Map class below to determine if the locations #shouldUpdate().
The map instance variable is used inside the react MapScreen component #handleRegionChange callback to re-render the page relevant surfspots. The method shouldUpdate() will detect if the user scrolled to a new region of the map.

class Map {
  constructor (coords) {
    this._current = coords
    this._previous = coords
  }

  get zoomOut() { return this.delta > 5; }
  get zoomIn() { return 0.5 > this.delta > 0.1 }
  get noZoom() { return 0.5 < this.delta < 1.5; }

  get shift() {
    return this.noZoom && this.scroll
  }

  get scroll() {
    return this.scroll_horizontal || this.scroll_vertical
  }

  get shouldUpdate() { 
    return this.zoomIn || this.zoomOut || this.shift 
  }
}

Additionally the ReactNative app allows users to record and upload videos. The functionality is included in the Camera Component which uses the Recorder component to record the video and the Player component to upload it.

Forecast Engine

The forecast information are retrieved from the stormglass api endpoint /weather/point. The application targets a specific country (Indonesia, Bali) with the plan and potential to expand to other areas in the world. Forecast are retrieved only for locations featured in the landing page with at least one uploaded surfing video, limiting background jobs memory usage and video playback server expenses.

The forecast are calculated with Sidekiq cron jobs. The #set_job method inside Location model starts the process which consist of the following steps:

STEP 1 Retrieving Information via Api

Runs the WeeklyForecastWorker to retrieve the forecast from the api and persist them in the Forecast model json weather column.

The #set_job method is called from the Location class.

class Post < ApplicationRecord
  before_save :set_forecast
  def set_forecast
    location = self.camera.location
    location.set_job unless location.with_forecast
  end
end

class Location < ApplicationRecord
  def set_job
    # seeds for the first time the weather data
    WeeklyForecastWorker.perform_async({ id: self.id })
  end
end

Each Location has_one :forecast, the result of the api call is cached in the weather jsonb column for later postprocessing.

class WeeklyForecastWorker
  include Sidekiq::Worker

  def perform(args)
    # calls execute_job
  end

  def execute_job
    return unless @location.storm.success?
    # saves the stormglass weather information in the database
    @location.forecast.update({ 
      weather: @location.storm.getWaves,
      tides: @location.storm.getTides,
    })
  end
end

STEP 2 Creating Cron Jobs

Creates a cron job to repeat the WeeklyForecastWorker 3 times a week.

class Location
  def weekly_cron_tab
    first_day = @now.wday
    second_day = first_day.next_day
    # returns a string for ex. 56 8 * * 4,6,2
    # to repeat the cron job 3 times a week
    # https://crontab.guru
    "#{@now.minute} #{@now.hour - 1} * * #{first_day},#{second_day},#{second_day.next_day}"
  end

  def set_job
    @now = DateTime.now
    location_text = "Location name: #{self.name}, id: #{self.id}"
    # Loads in Sidekiq the cron job
    Sidekiq::Cron::Job.load_from_array(
      [
        {
          name: "#{location_text} update forecast data - every 3 days",
          id: "#{location_text} update forecast data - every 3 days",
          cron: weekly_cron_tab,
          class: 'WeeklyForecastWorker',
          args: { id: id }
        },
      ]
    )
  end
end

STEP 3 Calculating Surf Forecast

Surf Forecast calculations are handled by the Weather plain ruby class. The Weather class inherits from Array and it is used to parse the forecasts.weather jsonb column, calculates the hourly and daily average wave height, wave period, wind speed, wind direction...

class Forecast < ApplicationRecord
  belongs_to :location

  # overwrites the default wether column attribute reader
  def weather
    # returns an instance of Weather class
    Weather.new(read_attribute(:weather) || [])
  end
end

Forecast.last.weather
=> [
      {
        "time"=>"2019-11-09T21:00:00+00:00",
        "seaLevel"=>[{"value"=>1.14, "source"=>"sg"}],
        "windSpeed"=>[{"value"=>10.14, "source"=>"sg"}, {"value"=>12.44, "source"=>"icon"}],
        "..." => "...",
      }, 
      {
        "time"=>"2019-11-09T22:00:00+00:00" 
      }
   ]
Forecast.last.weather.class
=> Weather

The Weather class creates accessors to retrieve informations from the json structure

class Weather < Array
  # a symbol for each property to retrieve from the json response
  KEYS = %w(time swellHeight waveHeight windSpeed windDirection waveDirection 
  swellDirection swellPeriod)

  # generates def swellHeightRange get method
  # returns the current hour swellHeight as a range 
  # for example 1-5 (meters) of swell
  # 1 is the minimum for that hour and 5 is the maximum
  %w(swellHeight waveHeight windSpeed swellPeriod).each do |method|
    define_method("#{method}Range") { current[method].minMaxString }
  end

  # generates def waveHeights
  # returns an array of the current hour
  # wave heights based on different sources
  # for ex. [1,1.5,1,3]
  %w(waveHeight swellPeriod).each do |method|
    define_method(method.pluralize) do 
      current[method].collect { |x| x["value"] } 
    end
  end
 
 # generates def time, def swellHeight 
  KEYS.each do |method|
    define_method(method) { current.value(method) }
  end

  # generates def swellHeight_at(time), def windSpeed_at(time)
  KEYS.each do |method|
    define_method("#{method}_at(time)".to_sym) do
      select { |row| row["time"] == time }.first.value(method)
    end
  end
end

The Forecast Model uses the Weather class #daily method to calculate the daily average forecast.

class Forecast < ApplicationRecord
  belongs_to :location
  KEYS = %w(swellHeight waveHeight windSpeed windDirection 
  waveDirection swellDirection swellPeriod)

  def weather
    Weather.new(read_attribute(:weather) || [])
  end
  
  # generates the following methods
  # def get_swell_height(days) def get_wave_height(days) 
  # def get_wind_speed(days) def get_wind_direction(days) 
  # def get_wave_direction(days) def get_swell_direction(days) 
  # def get_swell_period(days)
  KEYS.each do |field|
    define_method("get_#{field.duckTyped}(days)") do |days|
      weather.daily(field, days)
    end
  end

  # uses above get methods to calculate daily forecast
  def get_daily(days)
    result ||= KEYS.map do |field|
      # send(:get_wind_speed(days), days)
      daily_forecast = send("get_#{field.duckTyped}(days)".to_sym, days)
      [field, daily_forecast]
    end
    size = result.first.second.size - 1
    result += [["days", days.in_words[0..size]]]
    result.to_h
  end
end

The DailyForecastWorkers schedules the calculation of daily forecast.
Forecast are calculated based on the Location timezone, as surfing is possible only at daytime.

class Location < ApplicationRecord
  # returns a list of local dates based on the location Timezome
  #  => [Sun, 26 Jan 2020 19:47:22 WITA +08:00, ...]
  def week_days
    @week_days ||= (DateTime.now..DateTime.now + 6).map do |day|
      day.in_time_zone(timezone["timeZoneId"])
    end
  end
  
  def get_daily
    # uses the local timezones to calculate the forecast
    daily = forecast.get_daily(week_days)
    # checks if wind/swell directions are optimal
    %w(wind swell).each do |attr|
      daily["optimal_#{attr}"] = daily["#{attr}Direction"].map do |value|
        send("optimal_#{attr}?(#{attr})", value.in_word)
      end
    end
    daily
  end
end

The Weather #daily method is responsible for calculating the dailyAverage swell and wind based of the location timezone.

class Weather < Array
  def daily(key, week_days)
    return nil unless available?
    week_days.map { |day| dailyAverage(key, day) }.delete_if { |x| x.nil? }
  end
end

Things to Improve

  1. Full Test Coverage (jest unit test and detox e2e tests) of the react native mobile app
  2. Refactor the React application
  3. Fixing rspec tests
  4. Speeding up rspec tests with vcr
  5. Optimizing weather engine
    a. retrieve weather forecast based on closest buoy and not gps
    location belongs_to :weather_station
    b. refactor the weather class and avoid code repetition between location and forecast model
    c. Weather class does not have a constructor
    d. Refactor json response structure
  6. Continuos Integration
  7. Fastlane

About

monitor surf conditions at different time of the day

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published