Permalink
Find file
Fetching contributors…
Cannot retrieve contributors at this time
524 lines (345 sloc) 23.2 KB

Engine Yard Add-ons API

This document describes how partners work with Engine Yard to provide Add-on services that integrate with Engine Yard Cloud and are made available to Engine Yard users. A Ruby implementation of this API is available via: gem install ey_services_api (Source code here: https://github.com/engineyard/ey_services_api).

Key Topics in this Document:

General API properties and specific APIs are discussed in this document. All the APIs are JSON-based and RESTful.

General API properties:

  • API Authentication
  • SSO / Collaboration
  • API Error Handling

Specific APIs are:

  • Partner Service Registration — The way in which partners create and expose Add-on services to Engine Yard users.
  • Service Enablement for User Account — The way in which Engine Yard users sign up for Add-on services.
  • Service Provisioning — The way in which Engine Yard users provision an Add-on service for a particular use.
  • User Feedback — The way in which partners can send messages to Engine Yard Cloud and users about their Add-on service.
  • Billing — The way in which partners submit Add-on related invoice line items to Engine Yard and customers.

Definitions

These definitions are helpful for understanding this document:

Partner A third party integrating one or more Add-on services with Engine Yard Cloud.
Service Functionality provided by a partner.
Application A web application and its deployment configuration on Engine Yard Cloud.
Environment A set of one or more running servers hosting one or more running Applications.
Provisioned Service A piece of functionality or capacity provisioned on-demand for a specific application and environment.
Account A single billed entity (Engine Yard customer), having one or more Applications running in one or more Environments.
Service Account A particular Add-on Service as activated for a specific Account. A parent entity to Provisioned Service.
User An individual user (with unique email and login credential) having access to one or more Accounts.

API Authentication

When a partner signs up, Engine Yard creates a unique authentication ID, auth_id, and a unique authentication key, auth_key. Those tokens are used to sign every API call in both directions, with HMAC.

The easiest way to sign requests is to use the ey_api_hmac gem and make your HTTP request with rack-client. EY::ApiHMAC::BaseConnection is a convenient wrapper for which automatically included the needed EY::ApiHMAC::ApiAuth::Client middleware. (both are part of ey_api_hmac)

If you’re using a different Ruby HTTP library, the auth-hmac gem should be able to help you sign requests.

Implementation

Each request includes an HTTP Authorization header containing its signature. This signature, encoded with HMAC, follows the format:

Authorization: AuthHMAC {auth_id}:{signed_string}

Where auth_id is the unique partner ID, and signed_string is an HMAC digest of the canonical_string for the current request.

The canonical_string can be calculated by joining the following strings with \n as the separator:

  1. The HTTP request method (e.g. GET or POST).
  2. The content type, as specified by the “Content-Type” HTTP header.
  3. The content MD5, as specified by the “Content-MD5” HTTP header. Or in its absence, the MD5 hash of the request body.
  4. The date, as specified by the “Date” HTTP header.
  5. The request path, (URL without schema, host, post, or query parameters).

Thus, to obtain the signed_string, partners use their secret auth_key to sign this cannonical_string and generate its base64 encoded representation using sha1-hmac digest. As an example, this Ruby code would generate our signed_string:

[OpenSSL::HMAC.digest(OpenSSL::Digest::Digest.new('sha1'), auth_key, canonical_string)].pack('m').strip

Example of a signed request

Given the following request:

GET /api/1/service_accounts/1324/messages HTTP/1.1
Content-Type : application/json
Accept : application/json
Date : Tue, 16 Aug 2011 20:55:55 GMT
User-agent : EY-ServicesAPI/0.0.1
Host : services.engineyard.com
Authorization : AuthHMAC ff4d04dbea52c605:o3wmVM41ihTXIHWDj6SkROBAg2g=

{"message":{"message_type":"status","subject":"Everything looks good.","body":null}}

The canonical_string (to be HMAC digested) would be:

GET\napplication/json\ne8fa80541e3726e2cf4c71d07a7bd9fd\n2011-08-16 13:55:55 -0700\n/api/1/service_accounts/1324/messages

Becomes the signed_string:

o3wmVM41ihTXIHWDj6SkROBAg2g=

Which can then be compared to the value provided in the Authorization header of the request.

The method used is compatible with the auth-hmac Ruby gem (https://rubygems.org/gems/auth-hmac). A simpler, rack-only implementation is provided as part of the ey_api_hmac gem (https://github.com/engineyard/ey_api_hmac/blob/master/lib/ey_api_hmac.rb).

SSO / Collaboration

Engine Yard users can be linked with multiple accounts. User memberships are distinguished as being either “collaborator” or “owner”. Services are enabled per-account. SSO access to services happens per-user-membership.

The SSO process signs a partner-provided URL to include identifying information about the user. The identifying parameters are:

  • ey_user_id – the unique identifier for the user.
  • ey_user_name – the full name of the user in plain text. Example: “John Doe”.
  • access_level – either “owner” or “collaborator”.

What’s NOT included is information about the account being accessed. It is suggested that partners include this information somehow in the configuration_url generated for each service account.

Having parsed who the user is and the account they are accessing, the partner should be able to display something appropriate. Some integrations might choose to ignore ID and name and treat all access to the account as a single user. Other deeper integrations may take advantage of this additional information for per-user customizations.

It is the responsibility of partners to determine what makes the most sense for their integration in terms of handling multiple users per account.

The SSO signing process also add the following other parameters:

  • ey_return_to_url – the url to be used when sending the user back to Engine Yard.
  • timestamp – time the signature was calculated, URL should be considered invalid if timestamp is more than 5 minutes off.
  • signature – HMAC digest of the full url, minus this parameter (using the API auth_id and auth_key)

SSO signing and verification are implemented as part of the ey_api_hmac gem. To verify a url, call:

EY::ApiHMAC::SSO.authenticated?(url, auth_id, auth_key)

SSO implementation

Users reach partner sites by being redirected to a configuration_url (at service_account or provisioned_service level) which is communicated to Engine Yard via API described below. The configuration_url is “signed” by Engine Yard to add additional parameters and make it possible to validate. If the signature validates, the partner should assume the user is allowed to access the account or resource exposed.

The process of signing adds some parameters to the URL query string. The value of the signature parameter is base64 encoded HMAC digest of the full url, minus the signature, using the partner’s API auth_key as the secret.

An example configuration_url might look like:

 http://partner/sso/customers/1/generators/1

Signed with these parameters:

{
 :timestamp=>"2011-08-16T11:48:39-07:00",
 :ey_user_id=>"1",
 :ey_user_name=>"Bob",
 :ey_return_to_url=>"http://awsm/deployments/1",
 :access_level=>"owner"
}

Using:

auth_id = "ff4d04dbea52c605"
auth_key = "e301bcb647fc4e9def6dfb416722c583cf3058bc1b516ebb2ac99bccf7ff5c5ea22c112cd75afd28"

Would become:

http://partner/sso/customers/1/generators/1?access_level=owner& \
ey_return_to_url=http%3A%2F%2Fawsm%2Fdeployments%2F1&ey_user_id=1&ey_user_name=Bob& \
timestamp=2011-08-16T11%3A48%3A39-07%3A00&signature=AuthHMAC+ff4d04dbea52c605%3A38HUpyqVWcPqeeoSAgYm4IH1cp4%3D

The process of validation

Using the URL without the signature param:

http://partner/sso/customers/1/generators/1?access_level=owner& \
ey_return_to_url=http%3A%2F%2Fawsm%2Fdeployments%2F1&ey_user_id=1&ey_user_name=Bob& \
timestamp=2011-08-16T11%3A48%3A39-07%3A00

Calculate the signature (including auth_id and auth_key) and compare it to the one provided:

AuthHMAC ff4d04dbea52c605:38HUpyqVWcPqeeoSAgYm4IH1cp4=

The HMAC digest method is the same as that used for API Authentication, except that we use the full URL instead of a canonical_string.

An example implementation of SSO is available here: https://github.com/engineyard/ey_api_hmac/blob/master/lib/ey_api_hmac/sso.rb

API Error Handling

The Engine Yard API returns RESTful HTTP status codes to indicate any failures processing requests. (http://en.wikipedia.org/wiki/List_of_HTTP_status_codes)

It is assumed that the partner API will do likewise.

Error HTTP responses should have a JSON body with an “error_message”.

If a 4xx series error code is returned, the request is considered to have failed and should be modified in some way before being retried.

If a 5xx series error code is returned, the caller is expected to either retry the request later or message the failure to the requesting user to try again manually.

Examples:

503 Service Unavailable
{
  "error_messages": ["We are doing maintenance, try again later"]
}
422 Unprocessable Entity
{
  "error_messages": ["description cannot be blank", "name is already taken"]
}

Partner Service Registration

To create a new service:

POST to the service_registration_url provided on partner sign-up

{
  "service": {
    "name":                      "Compliment service",
    "description":               "We post friendly messages to your dashboard daily.  Sign-up is free!"
    "vars":     [
      "api_key", 
      "daily_supplement_path"
    ],
    "home_url":                   "http://compliments-r-us.net",
    "terms_and_conditions_url":   "http://compliments-r-us.net/terms",
                                  // (optional: if provided, user will see a link)
    "service_accounts_url":       "http://api.compliments-r-us.net/engineyard-api/service_accounts",
                                  // RESTful URL for creating service accounts for this service in partner system
  }
}

Engine Yard responds with a service, with its url provided in the Location header:

201 Created
Location: http://addons.engineyard.com/api/1/partners/8/services/1232

To update your service configurations, you can:

PUT to the service’s url

{
  "service": {
    "description":       "We post friendly messages to your dashboard daily.  Only $1/month."
  }
}

You may not be allowed to update certain fields after you move out of Alpha.

To facilitate having a “test” or “staging” version of your service, register the service a second time with slightly different attributes. For example, include “test” in the name and description, and point to a “staging” service_accounts_url. It’s OK to register multiple services with the same vars, the gating happens at (public) Beta.

Service Enablement for User Account

When users go to the Add-ons page of Engine Yard Cloud, they see a variety of services to enable. Enabling a service triggers an API call to set up the user’s account with the default plan for that service. The user might either be forced to immediately SSO into the service to configure it or be presented with an optional link to configuration. Partners can also use this entry point to allow the user to upgrade/downgrade from the default plan provided.

Service account creation

When a user enables the service from the services listing page, Engine Yard will:

POST to the service’s service_accounts_url

{
    "url":           "http://services.engineyard.com/api/1/partners/8/services/1232/service_accounts/333"
                     // Identifier/URL for the service account in EY systems
    "name":          "foo-corp",
                     // Name of the account in EY systems
    "messages_url":  "http://services.engineyard.com/api/1/partners/8/services/1232/service_accounts/333/messages"
                     // RESTful URL for creating messages for this service_account in Engineyard system
    "invoices_url":  "http://services.engineyard.com/api/1/partners/8/services/1232/service_accounts/333/invoices"
                     // RESTful URL for creating invoices for this service_account in Engineyard system (aka getting paid)
}

The partner should return a service account:

{
    "service_account": {
      "url":                      "http://api.compliments-r-us.net/engineyard-api/customers/4331"
                                   // Identifier/URL for the service account in partner systems
      "configuration_required":    true
                                   // true or false
      "configuration_url":        "http://compliments-r-us.net/accounts/4331/configure",
                                   // if configuration_required is true, user will be redirected here after enabling the service
      "provisioned_services_url":  "http://api.compliments-r-us.net/engineyard-api/service_accounts/4331/provisioned_services",
                                   // RESTful URL for creating provisioned_services for this service_account in partner system
    },
    "message":                  {}
                                // optional, see "User Feedback" for formatting/contents
}

Partner account cancellation

When/If users cancel their accounts, Engine Yard will:

DELETE to the service_account’s url

The partner should return:

200 OK

Also upon cancellation, an invoice for usage to-date should be sent as soon as possible.

Account configuration

If configuration_required was true:

  • Service is not yet considered active (can’t be provisioned or billed)
  • User is redirected to configuration_url via SSO
  • When configuration is completed:
    • The partner should redirect the user (or provide a link) to ey_return_to_url
    • The partner should make an API call to Engine Yard to note that config is complete

Account updates API

The partner should:

PUT to Engine Yard’s service account url

{
  "service_account": {
    "configuration_required":    false
                                 // false to indicate that the service can now be used
  }
}

Engine Yard should return:

200 OK

Post account creation and configuration

After configuration_required is true, the services dashboard will continue to show a link to configure.

Extra steps in Alpha phase:

  • Login to the partner account on services.engineyard.com
  • Turn on the “test” of your service for an Engine Yard Cloud account
    • Choose from a list of all accounts that you as a user are collaborating with
  • Note: Certain Engine Yard internal users automatically see all “alpha” services (via a feature flag)
  • Note: “Alpha” charges filed with Engine Yard will never actually be billed or payed out

Service Provisioning

  • The user can choose to provision services per application per environment.
  • In the background, Engine Yard makes calls to all selected services.

AccountP

Provision API call

Engine Yard will:

POST to the service’s provisioned_services_url

{
    "url":           "http://services.engineyard.com/api/1/partners/8/services/1232/service_accounts/333/provisioned_services/32"
                     // Identifier/URL for the provisioned_service in EY systems
    "messages_url":  "http://services.engineyard.com/api/1/partners/8/services/1232/service_accounts/333/provisioned_services/32/messages"
                     // RESTful URL for creating provisioned_services for this service_account in Partner system
    "environment": {
      "name":          "foo_production",
      "framework_env": "production"
      "id":            "123"
    },
    "app": {
      "name": "foo",
      "id":   "456"
    }
}

The partner should return a provisioned service:

{
  "provisioned_service": {
    "url":                "http://api.compliments-r-us.net/engineyard-api/service_accounts/4331/provisioned_services/23"
                          // Identifier/URL for the provisioned_service in PARTNER systems
    "configuration_url":  "http://compliments-r-us.net/accounts/4331/compliment-generator/1234",
                          // Should be a URL specific to the provisioned service
    "vars": {
      "api_key": "987698AFB0987EFBB983", 
      "daily_supplement_path": "/etc/"
    }
  },
  "message":            {}
                        // optional, see "User Feedback" for formatting/contents
}

The service MAY not immediately be available after provisioning, but it SHOULD be possible to return all the relevant configs.

De-provision API call

Engine Yard will:

DELETE to the provisioned_service’s url

The partner should return:

200 OK

Configuration on user instances

The configuration_variables returned from the provision call are populated on the user’s instances.

These values are:

  • Available in the “DNA” provided to chef recipes that may need them for configuration.
  • Constant until the user de-provisions.
  • Available as configration variables for users’ scripts and applications

User Feedback

Messages

To post a message to an account dashboard, the partner should:

POST to the service_account’s messages_url

{
  "message": {
    "message_type": "notification",
    "subject":      "The Compliments Service is please to announce a new service: Insults!",
    "body":         "You mother was a...",
                    // Optional, will show as collapsed until user clicks 'read more'
  }
}

Engine Yard should return:

201 Created

To post a message to a specific provisioned_service, the partner should:

POST to the provisioned_service’s messages_url

{
  "message": {
    "message_type": "notification",
    "subject":      "That's a nice looking app deployment you've got there",
    "body":         "And a db_slave, spiffy!",
                    // Optional, will show as collapsed until user clicks 'read more'
  }
}

Engine Yard should return:

201 Created

Messages appear in the context of the sending service, with a link to the appropriate configuration_url to SSO into the service if action is required.

message_type may be either “status”, “notification” or “alert”.

Status

Status messages replace each other. This means that only one status message per provisioned service will be presented at a time. Or, one status message per account if sent at the account level.

Examples:

  • Messaging the current monthly cost of the service as configured (account level status message).
  • The service is provisioned and some configuration is recommended (via SSO).
  • The service is provisioned but hasn’t actively been used yet.
  • The service is now configured correctly and up and running.
  • Providing statistics from the service.

Notification

Notifications are persistent and dismissible. Many notifications can be presented to the user at any given time. If a service accumulates too many un-dismissed notifications, they will appear in a roll-up with a count. Notifications are displayed to Engine Yard Cloud users along side App and Environment management pages in the following manner:

Alerts

Examples:

  • Letting the user know they are approaching plan limits.
  • Normal service messages that could be classified as warnings.

Alert

Alerts are identical to Notifications, but are also emailed to each user associated with the account.

Examples:

  • Letting the user know they have exceeded plan limits.
  • Normal service messages that could be classified as errors.

Billing

Engine Yard bills its users monthly, for usage by calendar month. A few days after the beginning of each month, the previous month’s billing cycle is closed and an invoice is calculated for it.

In order to collect for services, partners are required to submit invoices to Engine Yard. Submitted invoice will be placed in the next available (open) billing cycle, and will be billed the user when that billing cycle closes. Generally, invoices should be submitted for usage for the previous month shortly after the close of the month. Submitting multiple invoices per billing cycle is allowed, but not expected to be the norm.

If a user cancels service, an invoice for usage should be submitted within 24 hours of cancellation (preferably immediately).

Billing

To send an invoice, the partner should:

POST to the service_account’s invoices_url

{
    "invoice":
    {
      "total_amount_cents":     "3050",
                                //USD amount in cents ($30.50)
      "line_item_description":  "Invoice ID: 122. For service from Jan 1 to Feb 1 of 2012, rendered in a complimentary fashion.",
    }
}

Engine Yard should return:

201 Created

The total_amount_cents is a numeric USD value in cents (but without the “$” sign), and must be an integer greater than zero. This is the only field actually used to invoice the user. This is the total amount charged to the user, irrespective of the division of royalties to Engine Yard.

The line_item_description is a breakdown of the calculation used to produce the total_amount, and will appear on the invoice provided to the user.

Beta process

The Beta pipeline is:

  1. Testing
  2. Alpha (sometimes called Private Beta)
  3. Beta (optional)
  4. General Availability (GA)

Services can be transitioned down the pipeline through the partner dashboard. Each transition requires manual approval by Engine Yard.

Testing

  • The service is not visible on the service listing for the general public.
  • The service is visible for any account owned by the user who created the service.
  • The service can optionally be made visible to any account collaborated on by the user who created the service.

Alpha (Private Beta)

  • Service is present on services listing page, with:
    • “Coming soon” tag.
    • “Request access” button.
  • Partners can see all users who have requested access and can grant access via the partner dashboard.

Beta (Public Beta)

Identical to GA (below), but the service is tagged as “Beta” on the services screen. This is intended to set expectations to users as to what level of support they can expect from partners.

GA

The service is visible on the service listing and can be enabled by any user.