Architecture Summary
To understand what is happening with the current Eucalyptus Console architecture the first thing you need to know is that it is in the middle of being refactored. The Eucalyptus technical management had determined that there was not sufficient time to "start from scratch" and that new features needed to be delivered to users in tandem with incremental architectural improvements. The system design becomes much clearer when you see it as a series of elements that are in transition rather than a complete and intentional design. We will explain that transition by describing where the 3.2 design originated, where it is headed and the progress of that conversion process.
The 3.2 version of the Eucalyptus console revolves around jQuery UI. Each page is modeled as a jQuery widget and data sources are displayed and primarily managed with jQuery Datatables "landing pages". The landing pages are implemented in a series of files in the "js" directory off of the web root (instance, volume, sgroup, etc.). Each of these files are implemented as a jQuery "widget" extending a base implementation called "eucawidget".
HTML in 3.2 is either drawn from a series of templates embedded in the monolithic index.html file at the root or constructed with a series of jQuery calls (html(), append(), etc.) on existing elements. It is very typical to see a sequence like:
var parent = thisObj.connectDialog.find('a#password-link').parent();
parent.append($('<span>').addClass('on-error').text(instance_dialog_password_error));
It is also typical for the DOM to be interrogated directly for values that are then passed to AJAX calls to do work with the back end. For instance:
var desc = $add_dialog.eucadialog("getValue", "#sgroup-description");
$.ajax({
type:"POST",
url:"/ec2?Action=CreateSecurityGroup",
data:"_xsrf="+$.cookie('_xsrf') + "&GroupDescription=" + toBase64(desc),
... etc ...
The result is that there is a fairly tight coupling between code that manipulates the state of the CLC via AJAX and the HTML/UI interfaces that activates it.
The activation of different landing pages from URL hash locations (#instance, #volume, etc.) is handled by the primary jQuery widget defined in the file main_container.js. This code examines the hash and, depending on its content, activates specific child jQuery widgets which are then instantiated on main_container's top level element (ie. this.element.sgroup(), this.element.keypair(), etc.).
For this to work, all of the landing pages must be loaded in advance in index.html. In essence each landing page is just a large version of the kind of jQuery widget that could be used for autocompletes, tabs or any other jQuery UI element.
Help file loading is managed by js/help.js. Each help file must have a corresponding data structure created here and be added to the top level load() function. All of the help files are loaded at startup.
The proxy would act as a pass-through for all API calls between the browser and the CLC with the exception of caches that were put in place on the "Describe" operations. That meant that frequent requests for resource lists (i.e. instances or volumes) would hit the cache unless the cache was deemed too old. By default most caches had a age limit of 30 seconds with images set to 60 seconds. The browser code (eucadata.js) ran timers to periodically fetch all resource lists on a regular basis. This led to very current data in the browser and a highly responsive UI when moving from screen-to-screen, but at the cost of high traffic between the browser and proxy, as well as constant polling of the CLC by the proxy.
In 3.3 a number of changes were made in an effort to decouple AJAX server communication from the UI and to modularize and incrementally load the system logic. The components selected were:
- RequireJS: To incrementally load logic as needed and to explicitly describe the dependencies of each module.
- Backbone.js To provide a "model driven" infrastructure that facilitates a clean separation between the business logic operating on models and other dependent functions such as rendering views of those models.
- Rivets To provide two-way data binding between the Backbone.js models and the DOM.
In contrast, the model driven template approach looks more like this:
<span data-text="volume.name"></span>
var volume = new Volume();
var scope = new Backbone.Model({
volume: volume
});
rivets.bind($el, scope);
volume.set('name', 'foobar');
// Inside Volume
$.ajax({
type:"POST",
url:"/ec2?Action=CreateVolume",
data:"_xsrf="+$.cookie('_xsrf') + "&size=" + this.get('size'),
... etc ...
Dialogs have made the most progress in the refactoring effort. Scaling Group, Launch Configuration, CloudWatch Metric and CloudWatch Alarm dialogs have been implemented using rivets templates. Volume and snapshot dialogs have successfully been removed from their landing page parent and moved into a separate .js and .html template in js/views/dialogs. The .html templates use rivets to bind the model data that is loaded into the scope by the js controller file. That controller also contains the implementation of event managers that the template binds for mouse clicks, keystrokes and other activities.
Dialogs for the following resources have not been fully converted to separate .js and .html templates taking advantage of rivets:
- Security Groups (partially converted)
- Instances (except those that relate to volume actions like attach and detach)
- Elastic IPs
- Key Pairs
- About Your Cloud
- Log In
- Change Password
- Dashboard
For this version the responsibility of polling the CLC was shifted to the proxy. The proxy runs timers to keep caches fresh. The merged eucadata/backbone code now polls the proxy for resources that are needed by the specific page the user is looking at. That results in less traffic between the browser and the proxy. The dashboard also uses a new summary method to get a list of values to be displayed. The summary information is calculated on the proxy from cache content, so users sitting on the dashboard won't have large amounts of data being passed to the browser (instead, just the totals are passed).
Landing pages are in the midst of being refactored for the 3.4 release. The refactoring is partial because while the landing page templates and controllers are loaded via RequireJS they are still wrapped in a jQuery widget that is used when main_container instantiates them. Rivets is now used to render row data rather than Datatable and most of the row HTML can be directly edited in the templates defined in js/views/landing_pages.
The big change for this version is the addition of push notifications. This significantly affects how the data is updated on the browser. The proxy still refreshes the caches, but now compares the new contents with the old contents and sends a short message to the browser if changes are detected. When a push notification is received (in eucadata.js), it causes a refresh to the backbone collection mentioned in the notification. This means that if no data is changing on the resources the user is looking at, there will be no traffic between the browser and the proxy.
Further, the information about which resources the current screen requires to function is used by the proxy to limit which caches are refreshed. That lets us reduce the load on the CLC for each user by only running periodic "Describe" operations for a subset of the resources at any given time.
From a high level, we have a proxy service we run that sits in between the browser(s) and the back-end. The back-end could be either Eucalyptus or AWS. The proxy serves static files required by the browser and provides a number of web service endpoints need for both authentication and interaction with the various back-end services. The APIs supported by the proxy are directly related to the EC2/CloudWatch/ELB and AutoScaling services on the back-ends.
Each web service endpoint has a stack behind it. The top level is an API handler that validates the authentication, parses parameters and creates any objects needed to be passed to the next layer. Below that is the caching layer which handles a lot of things. It either passes calls down to a mock layer (for isolation testing) or to a boto layer which communicates with the back-end. In the caching layer is thread handling so that if any call requires communication to the back-end, that operation is performed in a background thread. Because of this, Tornado handles the calls asynchronously. This allows Tornado to process many requests simultaneously.
The caching bit is involved on the "Describe" operations. All of those simply read from the caches. The caches are kept current by timers that use a configurable value to perform the describe operations in the background. Modification type operations cause the cache to be expired and causes the timer to be reset so that another fetch happens quickly.
Push notifications are connected to cache refreshes. If a describe operations results in new data in the cache (based on content hash comparison), a push notification is sent to the browser. This allows us to reduce the amount of requests from the browser to only those times when new data is available.
There are 3 cases we should cover to get a general understanding of data flow between the browser, proxy and CLC.
As the user navigates through the application, a SetDataInterest call is made which tells the proxy which resources that screen is interested in. This causes the proxy to adjust which caches get refreshed (with timer threads). When refreshing caches, we use an MD5 hash of the data to compare old vs. new and see if anything has changed. If the cache contents have changed, a push notification is sent to the browser which causes the backbone collection to fetch new data.
When a backbone collection's fetch method is called, the Describe call is satisfied by the proxy using the cache exclusively. Since the fetch is generally only called after a push notification is received, there is minimal network traffic to get updates.
Operations that modify the resources (Create, Delete, Attach, etc.) are sent to the proxy which invalidates the cache for that resource and makes the call on a background thread. Once the CLC responds, that response is passed all the way back to the browser and appropriate HTTP status codes are set.
In the original implemenation of the proxy, Describe operations were performed on demand when the browser required data. Caching was used, but the cache was only refreshed based on a browser-initiated fetch. One big problem with this approach was that as the CLC got slower under load, it would take longer to respond to requests. This response time sometimes exceeded the browser AJAX timeout of 30 seconds. When that happened, the user would get logged out of their session since the code assumed there was some problem with the proxy. Now, the fetch from the CLC is decoupled from the browser requests by use of the background threads and the fact the browser fetches are served exclusively from the cache.
The admin console uses account/username/password authentication for login and we wanted to leverage that for the user console so that user login can be administered in the same way. For security purposes, we also wanted to avoid having any permenent credentials (access key/secret key) on the proxy. By avoiding having permanent credentials on the proxy, we limit exposure of those credentials.
To accomplish this, we're using the GetSessionTokens call from the STS service. That call has been specially augmented to also support basic authentication with acct/user/password. The process the console follows is this;
- A login form captures the account/username/password
- Javascript invokes an ajax call to the proxy, passing this information
- The proxy performs the call to the STS service, using a Basic auth header.
- The proxy receives the credentials and stores them in the user session object Once the session expires, those credentials discarded.
To support AWS login, the console asks the user for access key/secret key. For security reasons, we don't want to send those to the proxy. We also want to leverage the GetSessionToken feature of AWS and then the proxy can authenticate API calls in the same way for both types of login. When the user logs in with their AWS credentials, these things happen;
- The access key and secret key are captured from the form
- A request is created and signed in the browser
- The signed request is encoded and sent to the proxy in an ajax call
- The proxy performs the STS call against AWS
- The proxy receives the credentials and stores them in the user session object