Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/master' into lua-router-integration
Browse files Browse the repository at this point in the history
  • Loading branch information
GUI committed Sep 27, 2015
2 parents 860a80a + 6403a23 commit 8ff556b
Show file tree
Hide file tree
Showing 83 changed files with 30,022 additions and 53 deletions.
27 changes: 26 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,33 @@
# See http://help.github.com/ignore-files/ for more about ignoring files.
#
# If you find yourself ignoring temporary files generated by your text editor
# or operating system, you probably want to add a global ignore instead:
# git config --global core.excludesfile ~/.gitignore_global

# Ignore Vagrant config
/.vagrant

# Ignore the development working code directories
/workspace

# Local chef/knife configuration.
/chef/cookbooks
/chef/tmp

# Ignore bundler config
/.bundle

# Ignore the build directory for the website
/build

# Ignore Circle CI items, so the GitHub pages publish task can work.
/.ruby-version
/vendor

/deps
/log/*
/node_modules
/test/config/.overrides.yml
/test/log/*
/test/tmp/*
/vendor
/ci_cache
120 changes: 120 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# API Umbrella Change Log

## 0.8.0 (2015-04-26)

This update fixes a couple of security issues and a few important bugs. It's highly recommended anyone running earlier versions upgrade to v0.8.0.

[Download 0.8.0 Packages](http://nrel.github.io/api-umbrella/download/)

### Upgrade Instructions

If you're upgrading a previous API Umbrella version, you must first stop API Umbrella manually (`sudo /etc/init.d/api-umbrella stop`) before installing the new package.

### Hightlights

* **Fix cross-site-scripting vulnerability:** In the admin, there was a possibility of a cross-site-scripting vulnerability. (See [api.data.gov#214](https://github.com/18F/api.data.gov/issues/214))
* **Make it easier to route to new website pages:** Any non-API request will be routed to the website backend, making it easier to manage your public website content. In addition, different website content can now be served up for different hostnames. (See [api.data.gov#146](https://github.com/18F/api.data.gov/issues/146) and [#69](https://github.com/NREL/api-umbrella/issues/69))
* **New analytics querying interface:** The new interface for querying the analytics allows you to filter your analytics using drop down menus and form fields. This should be much easier to use than the raw Lucene queries we previously relied on. (See [#15](https://github.com/NREL/api-umbrella/issues/15) and [api.data.gov#168](https://github.com/18F/api.data.gov/issues/168))
* **Add ability to set API response headers:** This feature can be used to set headers on the API responses, which can be used to force CORS headers with API Umbrella. (See [#81](https://github.com/NREL/api-umbrella/issues/81) and [api.data.gov#188](https://github.com/18F/api.data.gov/issues/188))
* **Add feature to specify HTTPS requirements:** This feature can be used force HTTPS usage to access your APIs and can also be used to help transition new users to HTTPS-only. (See [api.data.gov#34](https://github.com/18F/api.data.gov/issues/34))
* **Allow for better customization of the API key signup confirmation e-mail:** The contents for the API key signup e-mail can now be better tailored for different sites. (See [api.data.gov#133](https://github.com/18F/api.data.gov/issues/133))
* **Fix file descriptor leak:** This could lead to an outage by exhausting your systems maximum number of file descriptors for setups with lots of API backends using domains with short-lived TTLs. (See [api.data.gov#188](https://github.com/18F/api.data.gov/issues/188))

### Everything Else

* **Fix possibility of very brief 503 errors:** For setups with lots of API backends using domains with short-lived TTLs, there was a possibility of rare 503 errors when DNS changes were being reloaded. (See [api.data.gov#207](https://github.com/18F/api.data.gov/issues/207))
* **Fix server log rotation issues:** There were a few issues present with a default installation that prevented log files from rotating properly, and may have wiped previous log files each night. This should now be resolved. (See [api.data.gov#189](https://github.com/18F/api.data.gov/issues/189))
* **Fix couple of edge-cases where custom rate limits weren't applied:** There were a couple of edge-cases in how API backends and users were configured that could lead to rate limits being ignored. (See [#127](https://github.com/NREL/api-umbrella/issues/127), [api.data.gov#201](https://github.com/18F/api.data.gov/issues/201), [api.data.gov#202](https://github.com/18F/api.data.gov/issues/202))
* **Fix situations where analytics may have not been logged for specific queries:** If a URL contained UTF-8 character or if a query parameter contained a date or time, there were certain situations where that request would fail to be logged in the analytics database. (See [api.data.gov#198](https://github.com/18F/api.data.gov/issues/198) and [api.data.gov#213](https://github.com/18F/api.data.gov/issues/213))
* **Fix proxy transforming backslashes into forward slashes in the URL:** If a URL contained a backslash character, it may have been transformed into a forward slash when the API backend received the request. (See [api.data.gov#199](https://github.com/18F/api.data.gov/issues/199))
* **Gracefully handle MongoDB replicaset changes:** API Umbrella should continue to serve requests with no downtime if the MongoDB primary server changes. (See [api.data.gov#200](https://github.com/18F/api.data.gov/issues/200))
* **Add registration source information to admin user list:** The user registration source is now shown in the user listing and can also be searched by the free-from search field. (See [api.data.gov#190](https://github.com/18F/api.data.gov/issues/190))
* **Fix broken pagination on the admin list of API backends:** The list of API backends didn't properly handle pagination when more than 50 backends were present. (See [api.data.gov#209](https://github.com/18F/api.data.gov/issues/209))
* **Fixes to URL encoding for advanced request rewriting:** If you were doing complex URL rewriting with "Route Pattern" rewrites under the Advanced Request Rewriting section, this fixes a variety of URL encoding issues.
* **Reduce duplicative nginx reloads for DNS changes:** If your system has several API backends with domains that have short-lived TTLs, there were a couple race conditions that could lead to nginx reloading twice on DNS changes. This is now fixed so the unnecessary, duplicate reload commands are gone. (See [api.data.gov#191](https://github.com/18F/api.data.gov/issues/191))
* **Fix incorrectly logging HTTPS requests as HTTP:** API Umbrella v0.7 introduced a bug the led to HTTPS requests being logged as HTTP requests in the analytics database. (See [api.data.gov#208](https://github.com/18F/api.data.gov/issues/208))
* **Fix analytics charts during daylight saving time:** During daylight saving time, the daily analytics charts in the admin may have contained an extra duplicate day with 0 results. (See [api.data.gov#147](https://github.com/18F/api.data.gov/issues/147))
* **Prevent all URL prefixes from being removed from API backends:** In the admin, it was possible to remove all URL prefixes from an API backend's configuration, leaving it in an invalid state (See [api.data.gov#215](https://github.com/18F/api.data.gov/issues/215))
* **Improve compatibility of install on systems with other Rubies present:** If you're installing API Umbrella on a system that already had something like rbenv/rvm/chruby installed, this should should fix some compatibility issues.
* **Build process improvements:** Various improvements to our build process for packaging new binary releases.
* **Upgrade bundled dependencies:**
* Bundler 1.7.12 -> 1.7.14
* ElasticSearch 1.4.2 -> 1.5.1
* MongoDB 2.6.7 -> 2.6.9
* nginx 1.7.9 -> 1.7.10
* ngx_headers_more 0.25 -> 0.26
* ngx_txid a41a705 -> f1c197c
* Node.js 0.10.36 -> 0.10.38
* OpenSSL 1.0.1l -> 1.0.1m
* Ruby 2.1.5 -> 2.1.6
* RubyGems 2.4.5 -> 2.4.6
* Varnish 4.0.2 -> 4.0.3

## 0.7.1 / 2015-02-11

This update fixes a couple of important bugs that were discovered shortly after rolling out the v0.7.0 release. It's highly recommended anyone running v0.7.0 upgrade to v0.7.1.

[Download 0.7.1 Packages](http://nrel.github.io/api-umbrella/download/)

### Upgrade Instructions

If you're upgrading a previous API Umbrella version, you must first stop API Umbrella manually (`sudo /etc/init.d/api-umbrella stop`) before installing the new package.

### Changes

* Fix 502 Bad Gateway errors for newly published API backends. Due to the DNS changes introduced in v0.7.0, newly published API backends may have not have properly resolved and passed traffic to the backend servers. (See [#107](https://github.com/NREL/api-umbrella/issues/107))
* Fix broken admin for non-English web browsers. The translations we introduced in v0.7.0 should actually now work (whoops!). (See [#103](https://github.com/NREL/api-umbrella/issues/103))
* Cut down on unnecessary DNS changes triggering reloads.
* Adjust internal API Umbrella logging to reduce error and warning log messages for expected events.
* Disables Groovy scripting in default ElasticSearch setup due to [CVE-2015-1427](http://www.elasticsearch.org/blog/elasticsearch-1-4-3-and-1-3-8-released/).

## 0.7.0 / 2015-02-08

[Download 0.7.0 Packages](http://nrel.github.io/api-umbrella/download/)

### Upgrade Instructions

If you're upgrading from API Umbrella v0.6.0, you must first stop API Umbrella manually (`sudo /etc/init.d/api-umbrella stop`) before installing the new package.

### Highlights

* **Admin UI Improvements:** Lots of tweaks and fixes have been made to the various parts of the admin to make it easier to use. There are better defaults, better notifications, and a lot more error validations to make it easier to manage API backends and users. (Related: [api.data.gov#160](https://github.com/18F/api.data.gov/issues/160), [api.data.gov#158](https://github.com/18F/api.data.gov/issues/158), [#49](https://github.com/NREL/api-umbrella/issues/49))
* **Improved DNS handling for API backends:** Fixes edge-case scenarios where DNS lookups may have not refreshed too quickly for backend API domain names with short TTLs (typically affecting API backends hosted behind Heroku, Akamai, or an Amazon Elastic Load Balancer). In certain rare cases, this could have temporarily taken down an API. (Related: [api.data.gov#131](https://github.com/18F/api.data.gov/issues/131))
* **Improved analytics gathering:** Fixes edge-case scenarios where analytics logs may have not been gathered. Request logs should also now show up in the admin analytics more quickly (within a few seconds). (Related: [#37](https://github.com/NREL/api-umbrella/issues/37), [api.data.gov#138](https://github.com/18F/api.data.gov/issues/138), [api.data.gov#106](https://github.com/18F/api.data.gov/issues/106))
* **Improved server startup:** Lots of fixes for various startup issues that should make starting API Umbrella more reliable on all platforms. API Umbrella v0.6 was our first package release across multiple platforms, so thanks to everyone in the community for reporting issues, and apologies if things were a bit bumpy. Hopefully v0.7 should be a bit easier to get running for everyone, but please let us know if not. (Related: [#42](https://github.com/NREL/api-umbrella/issues/42), [#89](https://github.com/NREL/api-umbrella/issues/89), [#92](https://github.com/NREL/api-umbrella/issues/92), [#100](https://github.com/NREL/api-umbrella/issues/100)
* **Dyanmic HTTP header rewriting:** Thanks to [@darylrobbins](https://github.com/darylrobbins) for this new feature, you can now perform more complex header rewriting by referencing existing header values during the HTTP header rewriting phase. (Related: [#96](https://github.com/NREL/api-umbrella/issues/96), [api-umbrella-gatekeeper#7](https://github.com/NREL/api-umbrella-gatekeeper/pull/7))
* **Admin Internationalization:** We've begun work to allow the admin interface to be translated into other languages. This is still incomplete, but the main admin menus and a good portion of the API Backends screen should now be available in Finnish, French, Italian, and Russian (with some translations started in German and Spanish too). Many thanks to [@perfaram](https://github.com/perfaram), [@kyyberi](https://github.com/kyyberi), Vesa Härkönen, vpilo, and enizev! (Related: [#60](https://github.com/NREL/api-umbrella/issues/60))

### Everything Else

* Fix analytics CSV downloads. (Related: [api.data.gov#173](https://github.com/18F/api.data.gov/issues/173))
* Fix default API key signup form in IE8-9. (Related [api.data.gov#174](https://github.com/18F/api.data.gov/issues/174))
* Give a better error message to restricted admins when they try to create an API outside of their permission scope. (Related: [api.data.gov#152](https://github.com/18F/api.data.gov/issues/152))
* Improve the admin UI for publishing backend changes to provide more sane checkbox defaults. (Related: [api.data.gov#169](https://github.com/18F/api.data.gov/issues/169))
* Treat admin logins case insensitively. (Related [api.data.gov#170](https://github.com/18F/api.data.gov/issues/170))
* Fix bugs preventing the GitHub OAuth based logins for admins from working. (Related: [#46](https://github.com/NREL/api-umbrella/issues/46), [#88](https://github.com/NREL/api-umbrella/issues/88))
* Fix limited admin account not having privileges to assign the special "api-umbrella-key-creator" role. (Related: [api.data.gov#157](https://github.com/18F/api.data.gov/issues/157))
* Fix analytics permissions for restricted admins for API paths containing uppercase characters. (Related: [api.data.gov#154](https://github.com/18F/api.data.gov/issues/154))
* Fix admin permissions for API backends with multiple URL prefixes. (Related: [api.data.gov#156](https://github.com/18F/api.data.gov/issues/156))
* Increase the default number of concurrent HTTP connections the various processes can accept.
* Fix inability to unset referrer or IP restrictions on user accounts once set. (Related [#97](https://github.com/NREL/api-umbrella/issues/97), [api.data.gov#155](https://github.com/18F/api.data.gov/issues/155))
* Fix issues surrounding default log rotation setup
* Retry connections to MongoDB in the event of MongoDB disconnects.
* Add the ability to selectively reload API Umbrella components via the `api-umbrella reload` command.
* Add a [deployment process](http://nrel.github.io/api-umbrella/docs/deployment/) for deploying non-packaged updates for API Umbrella components directly from git. (Related: [api.data.gov#159](https://github.com/18F/api.data.gov/issues/159), [api.data.gov#161](https://github.com/18F/api.data.gov/issues/161), [#99](https://github.com/NREL/api-umbrella/issues/99))
* Upgrade bundled dependencies
* Bundler 1.7.4 -> 1.7.12
* ElasticSearch 1.3.4 -> 1.4.2
* MongoDB 2.6.5 -> 2.6.7
* nginx 1.7.6 -> 1.7.9
* Node.js 0.10.33 -> 0.10.36
* OpenSSL 1.0.1j -> 1.0.1l
* Redis 2.8.17 -> 2.8.19
* Ruby 2.1.3 -> 2.1.5
* RubyGems 2.4.2 -> 2.4.5
* Ruby on Rails 3.2.19 -> 3.2.21
* Supervisor 3.1.2 -> 3.1.3

## 0.6.0 / 2014-10-27

* Initial package releases for CentOS, Debian, and Ubuntu.
17 changes: 17 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,23 @@
source "https://rubygems.org"

group :development do
gem "middleman", "~> 3.3.7"

# Live-reloading plugin
gem "middleman-livereload", "~> 3.3.4"

# For faster file watcher updates on Windows:
gem "wdm", "~> 0.1.0", :platforms => [:mswin, :mingw]

# Syntax highlighting
gem "middleman-syntax", "~> 2.0.0"

# Deploy to GitHub Pages
gem "middleman-gh-pages", "~> 0.0.3"

# Markdown
gem "kramdown", "~> 1.5.0"

# Deployment
gem "capistrano", "~> 3.3.5"
gem "capistrano-npm", "~> 1.0.1"
Expand Down
133 changes: 133 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
GEM
remote: https://rubygems.org/
specs:
activesupport (4.1.13)
i18n (~> 0.6, >= 0.6.9)
json (~> 1.7, >= 1.7.7)
minitest (~> 5.1)
thread_safe (~> 0.1)
tzinfo (~> 1.1)
capistrano (3.3.5)
capistrano-stats (~> 1.1.0)
i18n
Expand All @@ -9,17 +15,138 @@ GEM
capistrano-npm (1.0.1)
capistrano (>= 3.0.0)
capistrano-stats (1.1.1)
celluloid (0.16.0)
timers (~> 4.0.0)
chunky_png (1.3.4)
coffee-script (2.4.1)
coffee-script-source
execjs
coffee-script-source (1.9.1.1)
colorize (0.7.5)
compass (1.0.3)
chunky_png (~> 1.2)
compass-core (~> 1.0.2)
compass-import-once (~> 1.0.5)
rb-fsevent (>= 0.9.3)
rb-inotify (>= 0.9)
sass (>= 3.3.13, < 3.5)
compass-core (1.0.3)
multi_json (~> 1.0)
sass (>= 3.3.0, < 3.5)
compass-import-once (1.0.5)
sass (>= 3.2, < 3.5)
dotenv (1.0.2)
em-websocket (0.5.1)
eventmachine (>= 0.12.9)
http_parser.rb (~> 0.6.0)
erubis (2.7.0)
eventmachine (1.0.8)
eventmachine (1.0.8-java)
execjs (2.6.0)
ffi (1.9.10)
ffi (1.9.10-java)
haml (4.0.7)
tilt
hike (1.2.3)
hitimes (1.2.3)
hitimes (1.2.3-java)
hooks (0.4.1)
uber (~> 0.0.14)
http_parser.rb (0.6.0)
http_parser.rb (0.6.0-java)
i18n (0.7.0)
json (1.8.3)
json (1.8.3-java)
kramdown (1.5.0)
listen (2.10.1)
celluloid (~> 0.16.0)
rb-fsevent (>= 0.9.3)
rb-inotify (>= 0.9)
middleman (3.3.12)
coffee-script (~> 2.2)
compass (>= 1.0.0, < 2.0.0)
compass-import-once (= 1.0.5)
execjs (~> 2.0)
haml (>= 4.0.5)
kramdown (~> 1.2)
middleman-core (= 3.3.12)
middleman-sprockets (>= 3.1.2)
sass (>= 3.4.0, < 4.0)
uglifier (~> 2.5)
middleman-core (3.3.12)
activesupport (~> 4.1.0)
bundler (~> 1.1)
erubis
hooks (~> 0.3)
i18n (~> 0.7.0)
listen (>= 2.7.9, < 3.0)
padrino-helpers (~> 0.12.3)
rack (>= 1.4.5, < 2.0)
rack-test (~> 0.6.2)
thor (>= 0.15.2, < 2.0)
tilt (~> 1.4.1, < 2.0)
middleman-gh-pages (0.0.3)
rake (> 0.9.3)
middleman-livereload (3.3.4)
em-websocket (~> 0.5.1)
middleman-core (~> 3.2)
rack-livereload (~> 0.3.15)
middleman-sprockets (3.4.2)
middleman-core (>= 3.3)
sprockets (~> 2.12.1)
sprockets-helpers (~> 1.1.0)
sprockets-sass (~> 1.3.0)
middleman-syntax (2.0.0)
middleman-core (~> 3.2)
rouge (~> 1.0)
minitest (5.8.1)
multi_json (1.11.2)
net-scp (1.2.1)
net-ssh (>= 2.6.5)
net-ssh (2.9.2)
padrino-helpers (0.12.5)
i18n (~> 0.6, >= 0.6.7)
padrino-support (= 0.12.5)
tilt (~> 1.4.1)
padrino-support (0.12.5)
activesupport (>= 3.1)
rack (1.6.4)
rack-livereload (0.3.16)
rack
rack-test (0.6.3)
rack (>= 1.0)
rake (10.4.2)
rb-fsevent (0.9.6)
rb-inotify (0.9.5)
ffi (>= 0.5.0)
rouge (1.10.1)
sass (3.4.18)
sprockets (2.12.4)
hike (~> 1.2)
multi_json (~> 1.0)
rack (~> 1.0)
tilt (~> 1.1, != 1.3.0)
sprockets-helpers (1.1.0)
sprockets (~> 2.0)
sprockets-sass (1.3.1)
sprockets (~> 2.0)
tilt (~> 1.1)
sshkit (1.6.1)
colorize (>= 0.7.0)
net-scp (>= 1.1.2)
net-ssh (>= 2.8.0)
thor (0.19.1)
thread_safe (0.3.5)
thread_safe (0.3.5-java)
tilt (1.4.1)
timers (4.0.4)
hitimes
tzinfo (1.2.2)
thread_safe (~> 0.1)
uber (0.0.15)
uglifier (2.7.2)
execjs (>= 0.3.0)
json (>= 1.8.0)

PLATFORMS
java
Expand All @@ -29,3 +156,9 @@ DEPENDENCIES
capistrano (~> 3.3.5)
capistrano-npm (~> 1.0.1)
dotenv (~> 1.0.2)
kramdown (~> 1.5.0)
middleman (~> 3.3.7)
middleman-gh-pages (~> 0.0.3)
middleman-livereload (~> 3.3.4)
middleman-syntax (~> 2.0.0)
wdm (~> 0.1.0)
Loading

0 comments on commit 8ff556b

Please sign in to comment.