Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

third party api integration service example

branch: testtextile

This branch is 0 commits ahead and 25 commits behind master

Fetching latest commit…

Cannot retrieve the latest commit at this time

README.textile

Disclaimer!

The following document describes Chronatog as a web cron service. The descriptions presented are accurate with one important exception: ACTUAL CRON IS NOT YET IMPLEMENTED. This means you can register as many callbacks as you want, see them, delete them etc… but they will never actually run. Chronatog is currently entirely missing it’s underlying actual cron component.

Chronatog

Chronatog is:

  1. A simple service providing basic web cron. (lib/server)
  2. A ruby client implementation for the chronatog.engineyard.com web service. (lib/client, chronatog-client.gemspec)
  3. An example service demonstrating how to integrate with the EngineYard services API. (lib/ey_integration)
  4. A gem for use by the internal EngineYard services API implementation, for testing. (chronatog.gemspec)
  5. This document designed for helping partners get started with the EY services API. (README.textile)

Getting Started: Deploying your own Chronatog

Become a Partner

First you need a partner account with EngineYard. Once you have one, you should be able to login at https://services.engineyard.com.

Save your credentials

In Chronatog, credentials are stored in config/ey_partner_credentials.yml.

Example to generate this file in script/console:

$ script/console
> Chronatog::EyIntegration.save_creds('ff4d04dbea52c605', 'e301bcb647fc4e9def6dfb416722c583cf3058bc1b516ebb2ac99bccf7ff5c5ea22c112cd75afd28')
=> #<struct Chronatog::EyIntegration::Credentials auth_id="ff4d04dbea52c605", auth_key="e301bcb647fc4e9def6dfb416722c583cf3058bc1b516ebb2ac99bccf7ff5c5ea22c112cd75afd28">

Test your connection

To test your connection to services.engineyard.com, you can make a GET request to the registration url. This returns a listing of currently registered services.

Example:

$ script/console
> Chronatog::EyIntegration.connection.list_services(registration_url)
=> []

Behind the scenes, Chronatog is calling out to EY::ServicesAPI.

list_services is a method on EY::ServicesAPI::Connection.

Register your services

For the remainder of the setup steps, you will need to have Chronatog running somewhere with a publicly accessible url.

To register your service, you make a POST request to the registration url, passing a JSON body describing your service. Included in that description are callback URLS, so in order to generate them Chronatog needs to know it’s public-facing url.

Example:

$ script/console
> registration_url = http://services.engineyard.com/api/1/partners/11/services
> chronatog_url = "https://chronatog.engineyard.com"
> registered_service = Chronatog::EyIntegration.register_service(registration_url, chronatog_url)
=> #<struct Chronatog::EyIntegration::Service url="http://services.engineyard.com/api/1/partners/11/services/10">

Behind the scene, Chronatog is calling register_service on a EY::ServicesAPI::Connection. The first parameter is the registration_url. The second parameter is a hash describing the service being registered.

In the case of this example it looks something like:

{
  :name                     => "Chronatog", 
  :label                    => "chronatog", 
  :description              => "Web cron as a service.", 
  :service_accounts_url     => "https://chronatog.engineyard.com/eyintegration/api/1/customers", 
  :home_url                 => "https://chronatog.engineyard.com/", 
  :terms_and_conditions_url => "https://chronatog.engineyard.com/terms", 
  :vars                     => ["service_url", "auth_username", "auth_password"]
} 

Viewing your service on cloud.engineyard.com

If your service registration succeeded, you should see it’s information displayed when you visit https://services.engineyard.com. From there you can enable testing of your service with any cloud account you have access to.

If you don’t have any cloud accounts, you can create a free trial account at: https://cloud.engineyard.com/.

Once enabled for testing, you should see your service available if you navigate to “Services” in the menu bar from https://cloud.engineyard.com.

Verifying requests from Engine Yard

By using the EY::ApiHMAC::ApiAuth::LookupServer middleware in the API controller, Chronatog verifies that each request to it’s API is correctly signed by the requester. The block passed to the middleware is expected to return the auth_key correspondent to the auth_id given. It is then up to EY::ApiHMAC to calculate a signature and verify that it matches the one in the request (env).

use EY::ApiHMAC::ApiAuth::LookupServer do |env, auth_id|
  EyIntegration.api_creds && (EyIntegration.api_creds.auth_id == auth_id) && EyIntegration.api_creds.auth_key
end

Enabling your service

When you click ‘enable’, EngineYard will make a call to your service_accounts_url to create a service account. In the case of Chronatog, this callback is handled by creating a customer record.

The request will look something like this:

POST https://chronatog.engineyard.com/eyintegration/api/1/customers
{
  "name": "some-account", 
  "url": "http://services.engineyard.com/api/1/partners/9/services/9/service_accounts/9", 
  "messages_url": "http://services.engineyard.com/api/1/partners/9/services/9/service_accounts/9/messages", 
  "invoices_url": "http://services.engineyard.com/api/1/partners/9/services/9/service_accounts/9/invoices"
}

Chronatog will handle the callback with the implementation defined in the API controller:

request_body = request.body.read
service_account = EY::ServicesAPI::ServiceAccountCreation.from_request(request_body)
create_params = {
  :name         => service_account.name,
  :api_url      => service_account.url,
  :messages_url => service_account.messages_url,
  :invoices_url => service_account.invoices_url
}
customer = Chronatog::Server::Customer.create!(create_params)

As part of handling the callback, a Customer will be created:

#<Chronatog::Server::Customer id: 1, service_id: nil, name: "some-account", created_at: "2011-11-09 11:12:54", updated_at: "2011-11-09 11:12:54", api_url: "http://services.engineyard.com/api/1/partners/0/ser...", messages_url: "http://services.engineyard.com/api/1/partners/0/ser...", invoices_url: "http://services.engineyard.com/api/1/partners/0/ser...", plan_type: "freemium", last_billed_at: nil>

Chronatog returns a JSON response that tells EngineYard some information about the customer.

The code for generating that response:

response_params = {
  :configuration_required   => false,
  :configuration_url        => "#{sso_base_url}/customers/#{customer.id}",
  :provisioned_services_url => "#{api_base_url}/customers/#{customer.id}/schedulers",
  :url                      => "#{api_base_url}/customers/#{customer.id}",
  :message                  => EY::ServicesAPI::Message.new(:message_type => "status", 
                                                            :subject      => "Thanks for signing up for Chronatog!")
}
response = EY::ServicesAPI::ServiceAccountResponse.new(response_params)
content_type :json
headers 'Location' => response.url
response.to_hash.to_json

Notice EY::ServicesAPI::Message in the code above. The subject text should now appear in the context of the Chronatog service on https://cloud.engineyard.com.

What the generated response looks like:

{
  "service_account": {
    "url": "https://chronatog.engineyard.com/eyintegration/api/1/customers/1", 
    "configuration_required": false, 
    "configuration_url": "https://chronatog.engineyard.com/eyintegration/sso/customers/1", 
    "provisioned_services_url": "https://chronatog.engineyard.com/eyintegration/api/1/customers/1/schedulers"
  }, 
  "message": {
    "message_type": "status", 
    "subject": "Thanks for signing up for Chronatog!", 
    "body": null
  }
}

Visiting Chronatog over SSO

With the service enabled, a “Visit” link should appear. Following this link will redirect to the configuration_url provided in the response to service enablement.

The configuration url provided by Chronatog in this example was:

https://chronatog.engineyard.com/eyintegration/sso/customers/1

When EY signs the url it provides additional parameters, such that it looks like this:

https://chronatog.engineyard.com/eyintegration/sso/customers/1?access_level=owner&ey_return_to_url=https%3A%2F%2Fcloud.engineyard.com%2Fdashboard&ey_user_id=123&ey_user_name=Person+Name&timestamp=2011-11-09T11%3A12%3A56-08%3A00&signature=AuthHMAC+123edf%3A0%2F9spogTgwHiWvgawtLN92O7BnE%3D

Chronatog will verify the SSO request with a before filter that looks like this:

before do
  if session["ey_user_name"]
    #already logged in
  elsif EY::ApiHMAC::SSO.authenticated?(request.url,
                                        Chronatog::EyIntegration.api_creds.auth_id,
                                        Chronatog::EyIntegration.api_creds.auth_key)
  then
    session["ey_return_to_url"] = params[:ey_return_to_url]
    session["ey_user_name"] = params[:ey_user_name]
  else
    halt 401, "SSO authentication failed. <a href='#{params[:ey_return_to_url]}'>Go back</a>."
  end
end

Provisioning a Chronatog Scheduler

With the service enabled in EngineYard cloud, it should appear available for provisioning when you drill-down in the UI to viewing a single environment within the context of a single application.

Clicking “Provision” will cause EngineYard to call to your provisioned_services_url to create a provisioned service. In the case of Chronatog, this callback is handled by creating a scheduler.

The request will look something like this:

POST https://chronatog.engineyard.com/eyintegration/api/1/customers/1/schedulers
{
  "url": "http://services.engineyard.com/api/1/service_accounts/11/provisioned_service/3", 
  "messages_url": "http://services.engineyard.com/api/1/partners/13/services/12/service_accounts/11/provisioned_service/3/messages", 
  "app": {
    "id": 3, 
    "name": "myapp"
  }, 
  "environment": {
    "id": 3, 
    "name": "myenv", 
    "framework_env": "production"
  }
}

Chronatog will handle the callback with the implementation defined in the API controller:

request_body = request.body.read
provisioned_service = EY::ServicesAPI::ProvisionedServiceCreation.from_request(request_body)

customer = Chronatog::Server::Customer.find(customer_id)
create_params = {
  :environment_name => provisioned_service.environment.name,
  :app_name => provisioned_service.app.name,
  :messages_url => provisioned_service.messages_url,
  :usage_calls => 0
}
scheduler = customer.schedulers.create!(create_params)

As part of handling the callback, a Customer will be created:

#<Chronatog::Server::Scheduler id: 1, customer_id: 1, auth_username: "Ue166c7a5c160f9", auth_password: "Pa948f36fd816461ef9a7800235", created_at: "2011-11-09 11:12:56", updated_at: "2011-11-09 11:12:56", environment_name: "myenv", app_name: "myapp", messages_url: "http://services.engineyard.com/api/1/partners/12/se...", decomissioned_at: nil, usage_calls: 0>

Chronatog returns a JSON response that tells EngineYard some information about the created scheduler.

The code for generating the response:

response_params = {
  :configuration_required => false,
  :vars     => {
    "service_url"   => "#{true_base_url}/chronatogapi/1/jobs",
    "auth_username" => scheduler.auth_username,
    "auth_password" => scheduler.auth_password,
  },
  :url      => "#{api_base_url}/customers/#{customer.id}/schedulers/#{scheduler.id}",
  :message  => EY::ServicesAPI::Message.new(:message_type => "status", 
                                            :subject      => "Your scheduler has been created and is ready for use!")
}
response = EY::ServicesAPI::ProvisionedServiceResponse.new(response_params)
content_type :json
headers 'Location' => response.url
response.to_hash.to_json

Notice EY::ServicesAPI::Message in the code above. The subject text should now appear in the context of the relevant application and environment on https://cloud.engineyard.com.

What the response JSON looks like:

{
  "provisioned_service": {
    "url": "https://chronatog.engineyard.com/eyintegration/api/1/customers/1/schedulers/1", 
    "configuration_required": false, 
    "configuration_url": null, 
    "vars": {
      "service_url": "https://chronatog.engineyard.com/chronatogapi/1/jobs", 
      "auth_username": "U8e2343e7d4cffb",
      "auth_password": "P4d16fb58ca892bed8de5ea41b8"
    }
  }, 
  "message": {
    "message_type": "status", 
    "subject": "Your scheduler has been created and is ready for use!", 
    "body": null
  }
}

Using the provisioned Chronatog service in your app

The Chronatog service has been enabled and provisioned. Values for service_url, auth_username, and auth_password have been generated and sent back to EngineYard. Now it’s time to make use of the service in your application. Just check-in few changes to your app and deploy!

The public client gem for the Chronatog API is called chronatog-client. Add it to your Gemfile like this:

gem 'chronatog-client', :require => 'chronatog/client'

To initialize the Chronatog client with the provisioned configs, you’ll also need another gem, ey_config:

gem 'ey_config'

With these 2 gems installed, setting up Chronatog can be done like so:

@client = Chronatog::Client.setup!(EY::Config.get(:chronatog, 'service_url'), 
                                   EY::Config.get(:chronatog, 'auth_username'), 
                                   EY::Config.get(:chronatog, 'auth_password'))

EY::Config works by reading the contents of config/ey_services_config_local.yml or config/ey_services_config_deploy.yml. In order to develop with a fake version of the Chronatog API locally, you can create config/ey_services_config_local.yml with the contents:

---
chronatog:
  service_url: in-memory
  auth_username: 123-ignored
  auth_password: 456-also-ignored

When you deploy, EngineYard will create config/ey_services_config_deploy.yml. It might look something like this:

---
chronatog:
  service_url: https://chronatog.engineyard.com/chronatogapi/1/jobs
  auth_username: U3c292206caf4e5
  auth_password: P8481b7f41fa672965d22eeb9be

The existence of config/ey_services_config_deploy.yml will override all settings in config/ey_services_config_local.yml.

Disabling the Chronatog Service

As amazing as our Chronatog web service is, we still need to support user’s deciding they don’t need it anymore and disabling it.

When the service is disabled, EngineYard will make a DELETE call to the url provided when the service account was first created.

The request will look something like this:

DELETE https://chronatog.engineyard.com/eyintegration/api/1/customers/1

Chronatog will handle that request with the implementation defined in the API controller:

customer = Chronatog::Server::Customer.find(customer_id)
customer.bill!
customer.destroy
content_type :json
{}.to_json

Notice the call to customer.bill!. This causes Chronatog to send a final bill for services before destroying the customer. The request sent looks something like this:

POST http://services.engineyard.com/api/1/partners/9/services/9/service_accounts/9/invoices
{
  "invoice": {
    "total_amount_cents": 27, 
    "line_item_description": "For service from 2011/11/08 to 2011/11/09 includes 1 schedulers and 5 jobs run."
  }
}

Billing

Hopefully, normally, customers will use the Chronatog service for a long period of time before canceling. So we need a way to charge them periodically as well.

The mechanism for this is currently a manual process run once a month via script/console.

Simply run:

$ script/console
> Chronatog::Server::Customer.all.each(&:bill!)

This, of course, will call bill! on all customers, which calculates charges and sends an invoice to the invoices_url for each customer.

The implementation as defined in Chronatog::EyIntegration::CustomerExtensions calculates the total amount owed based on the last time billing was run for each customer (last_billed_at or created_at). It then sends an invoice to the invoices_url and sets the last_billed_at to now.

def bill!
  #don't bill free customers
  return if plan_type == "freemium"

  self.last_billed_at ||= created_at
  billing_at = Time.now
  #having the awesome service active costs $0.02 per day
  total_price = 2 * (billing_at.to_i - last_billed_at.to_i) / 60 / 60 / 24

  total_jobs_ran = 0
  schedulers.each do |schedule|
    #add $0.05 for every time we called a job
    usage_price = 5 * schedule.usage_calls
    total_jobs_ran += schedule.usage_calls
    schedule.usage_calls = 0
    schedule.save
    total_price += usage_price
  end
  if total_price > 0
    line_item_description = [
      "For service from #{last_billed_at.strftime('%Y/%m/%d')}",
      "to #{billing_at.strftime('%Y/%m/%d')}",
      "includes #{schedulers.size} schedulers", 
      "and #{total_jobs_ran} jobs run.",
    ].join(" ")

    invoice = EY::ServicesAPI::Invoice.new(:total_amount_cents => total_price,
                                           :line_item_description => line_item_description)
    Chronatog::EyIntegration.connection.send_invoice(self.invoices_url, invoice)

    self.last_billed_at = billing_at
    save!
  end
end

More

TODO: provisioned service SSO.

TODO: using those API keys works. Chronatog automatically updates the status to tell the user they are now using the service. Tell them how many jobs are scheduled.

TODO: using the API to create more than 10 jobs on the free plan and Chronatog sends a notification prompting you to upgrade.

TODO: Examining the monthly billing job Chronatog created in itself and forcing it to run.

Something went wrong with that request. Please try again.