Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Import database version 1.2.0

  • Loading branch information...
commit c84e534d99951c1486e11c9b3634ad1ac8c2bef8 1 parent 836c9eb
Steven Scott authored
Showing with 2,241 additions and 0 deletions.
  1. +29 −0 cookbooks/database/CHANGELOG.md
  2. +29 −0 cookbooks/database/CONTRIBUTING
  3. +201 −0 cookbooks/database/LICENSE
  4. +383 −0 cookbooks/database/README.md
  5. +97 −0 cookbooks/database/libraries/provider_database_mysql.rb
  6. +76 −0 cookbooks/database/libraries/provider_database_mysql_user.rb
  7. +130 −0 cookbooks/database/libraries/provider_database_postgresql.rb
  8. +83 −0 cookbooks/database/libraries/provider_database_postgresql_user.rb
  9. +109 −0 cookbooks/database/libraries/provider_database_sql_server.rb
  10. +106 −0 cookbooks/database/libraries/provider_database_sql_server_user.rb
  11. +111 −0 cookbooks/database/libraries/resource_database.rb
  12. +90 −0 cookbooks/database/libraries/resource_database_user.rb
  13. +34 −0 cookbooks/database/libraries/resource_mysql_database.rb
  14. +34 −0 cookbooks/database/libraries/resource_mysql_database_user.rb
  15. +35 −0 cookbooks/database/libraries/resource_postgresql_database.rb
  16. +35 −0 cookbooks/database/libraries/resource_postgresql_database_user.rb
  17. +34 −0 cookbooks/database/libraries/resource_sql_server_database.rb
  18. +34 −0 cookbooks/database/libraries/resource_sql_server_database_user.rb
  19. +45 −0 cookbooks/database/metadata.json
  20. +21 −0 cookbooks/database/metadata.rb
  21. +20 −0 cookbooks/database/recipes/default.rb
  22. +89 −0 cookbooks/database/recipes/ebs_backup.rb
  23. +201 −0 cookbooks/database/recipes/ebs_volume.rb
  24. +78 −0 cookbooks/database/recipes/master.rb
  25. +62 −0 cookbooks/database/recipes/snapshot.rb
  26. +8 −0 cookbooks/database/templates/default/app_grants.sql.erb
  27. +3 −0  cookbooks/database/templates/default/aws_config.erb
  28. +10 −0 cookbooks/database/templates/default/chef-solo-database-snapshot.cron.erb
  29. +1 −0  cookbooks/database/templates/default/chef-solo-database-snapshot.json.erb
  30. +6 −0 cookbooks/database/templates/default/chef-solo-database-snapshot.rb.erb
  31. +2 −0  cookbooks/database/templates/default/ebs-backup-cron.erb
  32. +8 −0 cookbooks/database/templates/default/ebs-db-backup.sh.erb
  33. +10 −0 cookbooks/database/templates/default/ebs-db-restore.sh.erb
  34. +27 −0 cookbooks/database/templates/default/s3cfg.erb
29 cookbooks/database/CHANGELOG.md
View
@@ -0,0 +1,29 @@
+## v1.2.0:
+
+* [COOK-970] - workaround for disk [re]naming on ubuntu 11.04+
+* [COOK-1085] - check RUBY_VERSION and act accordingly for role
+* [COOK-749] - localhost should be a string in snapshot recipe
+
+## v1.1.4:
+
+* [COOK-1062] - Databases: Postgres exists should close connection
+
+## v1.1.2:
+
+* [COOK-975] - Change arg='DEFAULT' to arg=nil, :default => 'DEFAULT'
+* [COOK-964] - Add parentheses around connection hash in example
+
+## v1.1.0
+
+* [COOK-716] - providers for PostgreSQL
+
+## v1.0.0
+
+* [COOK-683] - added `database` and `database_user` resources
+* [COOK-684] - MySQL providers
+* [COOK-685] - SQL Server providers
+* refactored - `database::master` and `database::snapshot` recipes to leverage new resources
+
+## v0.99.1
+
+* Use Chef 0.10's `node.chef_environment` instead of `node['app_environment']`.
29 cookbooks/database/CONTRIBUTING
View
@@ -0,0 +1,29 @@
+If you would like to contribute, please open a ticket in JIRA:
+
+* http://tickets.opscode.com
+
+Create the ticket in the COOK project and use the cookbook name as the
+component.
+
+For all code contributions, we ask that contributors sign a
+contributor license agreement (CLA). Instructions may be found here:
+
+* http://wiki.opscode.com/display/chef/How+to+Contribute
+
+When contributing changes to individual cookbooks, please do not
+modify the version number in the metadata.rb. Also please do not
+update the CHANGELOG.md for a new version. Not all changes to a
+cookbook may be merged and released in the same versions. Opscode will
+handle the version updates during the release process. You are welcome
+to correct typos or otherwise make updates to documentation in the
+README.
+
+If a contribution adds new platforms or platform versions, indicate
+such in the body of the commit message(s), and update the relevant
+COOK ticket. When writing commit messages, it is helpful for others if
+you indicate the COOK ticket. For example:
+
+ git commit -m '[COOK-1041] Updated pool resource to correctly delete.'
+
+In the ticket itself, it is also helpful if you include log output of
+a successful Chef run, but this is not absolutely required.
201 cookbooks/database/LICENSE
View
@@ -0,0 +1,201 @@
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS
+
+APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+Copyright [yyyy] [name of copyright owner]
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
383 cookbooks/database/README.md
View
@@ -0,0 +1,383 @@
+Database Cookbook
+=================
+
+The main highlight of this cookbook is the `database` and `database_user` resources for managing databases and database users in a RDBMS. Providers for MySQL, PostgreSQL and SQL Server are also provided, see usage documentation below.
+
+This cookbook also contains recipes to configure mysql database masters and slaves and uses EBS for storage, integrating together with the application cookbook utilizing data bags for application related information. These recipes are written primarily to use MySQL and the Opscode mysql cookbook. Other RDBMS may be supported at a later date. This cookbook does not automatically restore database dumps, but does install tools to help with that.
+
+Requirements
+============
+
+Chef 0.10.0 or higher required (for Chef environment use).
+
+Platform
+--------
+
+* Debian, Ubuntu
+* Red Hat, CentOS, Scientific, Fedora
+
+Cookbooks
+---------
+
+The following Opscode cookbooks are dependencies:
+
+* mysql
+* postgresql
+* xfs
+* aws
+
+Resources/Providers
+===================
+
+These resources aim to expose an abstraction layer for interacting with different RDBMS in a general way. Currently the cookbook ships with providers for MySQL, PostgreSQL and SQL Server. Please see specific usage in the __Example__ sections below. The providers use specific Ruby gems to execute commands and carry out actions. These gems will need to be installed before the providers can operate correctly. Specific notes for each RDBS flavor:
+
+- MySQL: leverages the `mysql` gem which is installed as part of the `mysql::client` recipe.
+- PostgreSQL: leverages the `pg` gem which is installed as part of the `postgresql::client` recipe.
+- SQL Server: leverages the `tiny_tds` gem which is installed as part of the `sql_server::client` recipe.
+
+`database`
+----------
+
+Manage databases in a RDBMS. Use the proper shortcut resource depending on your RDBMS: `mysql_database`, `postgresql_database` or `sql_server_database`.
+
+### Actions
+
+- :create: create a named database
+- :drop: drop a named database
+- :query: execute an arbitrary query against a named database
+
+### Attribute Parameters
+
+- database_name: name attribute. Name of the database to interact with
+- connection: hash of connection info. valid keys include :host, :port, :username, :password
+- sql: string of sql to execute against the database. used by :query action only
+
+### Providers
+
+- **Chef::Provider::Database::Mysql**: shortcut resource `mysql_database`
+- **Chef::Provider::Database::Postgresql**: shortcut resource `postgresql_database`
+- **Chef::Provider::Database::SqlServer**: shortcut resource `sql_server_database`
+
+### Examples
+
+ # create a mysql database
+ mysql_database 'oracle_rules' do
+ connection ({:host => "localhost", :username => 'root', :password => node['mysql']['server_root_password']})
+ action :create
+ end
+
+ # create a sql server database
+ sql_server_database 'mr_softie' do
+ connection ({:host => "127.0.0.1", :port => node['sql_server']['port'], :username => 'sa', :password => node['sql_server']['server_sa_password']})
+ action :create
+ end
+
+ # create a postgresql database
+ postgresql_database 'mr_softie' do
+ connection ({:host => "127.0.0.1", :port => 5432, :username => 'postgres', :password => node['postgresql']['password']['postgres']})
+ action :create
+ end
+
+ # create a postgresql database with additional parameters
+ postgresql_database 'mr_softie' do
+ connection ({:host => "127.0.0.1", :port => 5432, :username => 'postgres', :password => node['postgresql']['password']['postgres']})
+ template 'DEFAULT'
+ encoding 'DEFAULT'
+ tablespace 'DEFAULT'
+ connection_limit '-1'
+ owner 'postgres'
+ action :create
+ end
+
+ # externalize conection info in a ruby hash
+ mysql_connection_info = {:host => "localhost", :username => 'root', :password => node['mysql']['server_root_password']}
+ sql_server_connection_info = {:host => "localhost", :port => node['sql_server']['port'], :username => 'sa', :password => node['sql_server']['server_sa_password']}
+ postgresql_connection_info = {:host => "127.0.0.1", :port => 5432, :username => 'postgres', :password => node['postgresql']['password']['postgres']}
+
+ # same create commands, connection info as an external hash
+ mysql_database 'foo' do
+ connection mysql_connection_info
+ action :create
+ end
+ sql_server_database 'foo' do
+ connection sql_server_connection_info
+ action :create
+ end
+ postgresql_database 'foo' do
+ connection postgresql_connection_info
+ action :create
+ end
+
+ # create database, set provider in resource parameter
+ database 'bar' do
+ connection mysql_connection_info
+ provider Chef::Provider::Database::Mysql
+ action :create
+ end
+ database 'bar' do
+ connection sql_server_connection_info
+ provider Chef::Provider::Database::SqlServer
+ action :create
+ end
+ database 'bar' do
+ connection postgresql_connection_info
+ provider Chef::Provider::Database::Postgresql
+ action :create
+ end
+
+ # drop a database
+ mysql_database "baz" do
+ connection mysql_connection_info
+ action :drop
+ end
+
+ # query a database
+ mysql_database "flush the privileges" do
+ connection mysql_connection_info
+ sql "flush privileges"
+ action :query
+ end
+
+ # vacuum a postgres database
+ postgres_database "vacuum databases" do
+ connection postgresql_connection_info
+ database_table "template1"
+ sql "VACUUM FULL VERBOSE ANALYZE"
+ action :query
+ end
+
+`database_user`
+---------------
+
+Manage users and user privileges in a RDBMS. Use the proper shortcut resource depending on your RDBMS: `mysql_database_user` or `sql_server_database_user`.
+
+### Actions
+
+- :create: create a user
+- :drop: drop a user
+- :grant: manipulate user privileges on database objects
+
+### Attribute Parameters
+
+- username: name attribute. Name of the database user
+- password: password for the user account
+- database_name: Name of the database to interact with
+- connection: hash of connection info. valid keys include :host, :port, :username, :password
+- privileges: array of database privileges to grant user. used by the :grant action. default is :all
+- host: host where user connections are allowed from. used by MySQL provider only. default is 'localhost'
+- table: table to grant privileges on. used by :grant action and MySQL provider only. default is '*' (all tables)
+
+### Providers
+
+- **Chef::Provider::Database::MysqlUser**: shortcut resource `mysql_database_user`
+- **Chef::Provider::Database::SqlServerUser**: shortcut resource `sql_server_database_user`
+
+### Examples
+
+ # create connection info as an external ruby hash
+ mysql_connection_info = {:host => "localhost", :username => 'root', :password => node['mysql']['server_root_password']}
+ sql_server_connection_info = {:host => "localhost", :port => node['sql_server']['port'], :username => 'sa', :password => node['sql_server']['server_sa_password']}
+
+ # create a mysql user but grant no priveleges
+ mysql_database_user 'disenfranchised' do
+ connection mysql_connection_info
+ password 'super_secret'
+ action :create
+ end
+
+ # do the same but pass the provider to the database resource
+ database_user 'disenfranchised' do
+ connection mysql_connection_info
+ password 'super_secret'
+ provider Chef::Provider::Database::MysqlUser
+ action :create
+ end
+
+ # create a sql server user but grant no priveleges
+ sql_server_database_user 'disenfranchised' do
+ connection sql_server_connection_info
+ password 'super_secret'
+ action :create
+ end
+
+ # drop a mysql user
+ mysql_database_user "foo_user" do
+ connection mysql_connection_info
+ action :drop
+ ends
+
+ # bulk drop sql server users
+ %w{ disenfranchised foo_user }.each do |user|
+ sql_server_database_user user do
+ connection sql_server_connection_info
+ action :drop
+ end
+ end
+
+ # grant select,update,insert privileges to all tables in foo db from all hosts
+ mysql_database_user 'foo_user' do
+ connection mysql_connection_info
+ password 'super_secret'
+ database_name 'foo'
+ host '%'
+ privileges [:select,:update,:insert]
+ action :grant
+ end
+
+ # grant all privelages on all databases/tables from localhost
+ mysql_database_user 'super_user' do
+ connection mysql_connection_info
+ password 'super_secret'
+ action :grant
+ end
+
+ # grant select,update,insert privileges to all tables in foo db
+ sql_server_database_user 'foo_user' do
+ connection sql_server_connection_info
+ password 'super_secret'
+ database_name 'foo'
+ privileges [:select,:update,:insert]
+ action :grant
+ end
+
+Recipes
+=======
+
+ebs\_volume
+-----------
+
+*Note*: This recipe does not currently work on RHEL platforms due to the xfs cookbook not supporting RHEL yet.
+
+Loads the aws information from the data bag. Searches the applications data bag for the database master or slave role and checks that role is applied to the node. Loads the EBS information and the master information from data bags. Uses the aws cookbook LWRP, `aws_ebs_volume` to manage the volume.
+
+On a master node:
+* if we have an ebs volume already as stored in a data bag, attach it.
+* if we don't have the ebs information then create a new one and attach it.
+* store the volume information in a data bag via a ruby block.
+
+On a slave node:
+* use the master volume information to generate a snapshot.
+* create the new volume from the snapshot and attach it.
+
+Also on a master node, generate some configuration for running a snapshot via `chef-solo` from cron.
+
+On a new filesystem volume, create as XFS, then mount it in /mnt, and also bind-mount it to the mysql data directory (default /var/lib/mysql).
+
+master
+------
+
+This recipe no longer loads AWS specific information, and the database position for replication is no longer stored in a databag because the client might not have permission to write to the databag item. This may be handled in a different way at a future date.
+
+Searches the apps databag for applications, and for each one it will check that the specified database master role is set in both the databag and applied to the node's run list. Then, retrieves the passwords for `root`, `repl` and `debian` users and saves them to the node attributes. If the passwords are not found in the databag, it prints a message that they'll be generated by the mysql cookbook.
+
+Then it adds the application databag database settings to a hash, to use later.
+
+Then it will iterate over the databases and create them with the `mysql_database` resource while adding privileges for application specific database users using the `mysql_database_user` resource.
+
+slave
+-----
+
+_TODO_: Retrieve the master status from a data bag, then start replication using a ruby block. The replication status needs to be handled in some other way for now since the master recipe above doesn't actually set it in the databag anymore.
+
+snapshot
+--------
+
+Run via Chef Solo. Retrieves the db snapshot configuration from the specified JSON file. Uses the `mysql_database` resource to lock and unlock tables, and does a filesystem freeze and EBS snapshot.
+
+Deprecated Recipes
+==================
+
+The following recipe is considered deprecated. It is kept for reference purposes.
+
+ebs\_backup
+-----------
+
+Older style of doing mysql snapshot and replication using Adam Jacob's [ec2_mysql](http://github.com/adamhjk/ec2_mysql) script and library.
+
+Data Bags
+=========
+
+This cookbook uses the apps data bag item for the specified application; see the `application` cookbook's README.md. It also creates data bag items in a bag named 'aws' for storing volume information. In order to interact with EC2, it expects aws to have a main item:
+
+ {
+ "id": "main",
+ "ec2_private_key": "private key as a string",
+ "ec2_cert": "certificate as a string",
+ "aws_account_id": "",
+ "aws_secret_access_key": "",
+ "aws_access_key_id": ""
+ }
+
+Note: with the Open Source Chef Server, the server using the database recipes must be an admin client or it will not be able to create data bag items. You can modify whether the client is admin by editing it with knife.
+
+ knife client edit <client_name>
+ {
+ ...
+ "admin": true
+ ...
+ }
+
+This is not required if the Chef Server is the Opscode Platform, instead use the ACL feature to modify access for the node to be able to update the data bag.
+
+Usage
+=====
+
+Aside from the application data bag (see the README in the application cookbook), create a role for the database master. Use a role.rb in your chef-repo, or create the role directly with knife.
+
+ % knife role show my_app_database_master -Fj
+ {
+ "name": "my_app_database_master",
+ "chef_type": "role",
+ "json_class": "Chef::Role",
+ "default_attributes": {
+ },
+ "description": "",
+ "run_list": [
+ "recipe[mysql::server]",
+ "recipe[database::master]"
+ ],
+ "override_attributes": {
+ }
+ }
+
+Create a `production` environment. This is also used in the `application` cookbook.
+
+ % knife environment show production -Fj
+ {
+ "name": "production",
+ "description": "",
+ "cookbook_versions": {
+ },
+ "json_class": "Chef::Environment",
+ "chef_type": "environment",
+ "default_attributes": {
+ },
+ "override_attributes": {
+ }
+ }
+
+
+The cookbook `my_app_database` is recommended to set up any application specific database resources such as configuration templates, trending monitors, etc. It is not required, but you would need to create it separately in `site-cookbooks`. Add it to the `my_app_database_master` role.
+
+License and Author
+==================
+
+Author:: Adam Jacob (<adam@opscode.com>)
+Author:: Joshua Timberman (<joshua@opscode.com>)
+Author:: AJ Christensen (<aj@opscode.com>)
+Author:: Seth Chisamore (<schisamo@opscode.com>)
+Author:: Lamont Granquist (<lamont@opscode.com>)
+
+Copyright 2009-2011, Opscode, Inc.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
97 cookbooks/database/libraries/provider_database_mysql.rb
View
@@ -0,0 +1,97 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require 'chef/provider'
+
+class Chef
+ class Provider
+ class Database
+ class Mysql < Chef::Provider
+ include Chef::Mixin::ShellOut
+
+ def load_current_resource
+ Gem.clear_paths
+ require 'mysql'
+ @current_resource = Chef::Resource::Database.new(@new_resource.name)
+ @current_resource.database_name(@new_resource.database_name)
+ @current_resource
+ end
+
+ def action_create
+ unless exists?
+ begin
+ Chef::Log.debug("#{@new_resource}: Creating database #{new_resource.database_name}")
+ db.query("create database #{new_resource.database_name}")
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ def action_drop
+ if exists?
+ begin
+ Chef::Log.debug("#{@new_resource}: Dropping database #{new_resource.database_name}")
+ db.query("drop database #{new_resource.database_name}")
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ def action_query
+ if exists?
+ begin
+ db.select_db(@new_resource.database_name) if @new_resource.database_name
+ Chef::Log.debug("#{@new_resource}: Performing query [#{new_resource.sql}]")
+ db.query(@new_resource.sql)
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ private
+ def exists?
+ db.list_dbs.include?(@new_resource.database_name)
+ end
+
+ def db
+ @db ||= begin
+ ::Mysql.new(
+ @new_resource.connection[:host],
+ @new_resource.connection[:username],
+ @new_resource.connection[:password],
+ nil,
+ @new_resource.connection[:port] || 3306
+ )
+ end
+ end
+
+ def close
+ @db.close rescue nil
+ @db = nil
+ end
+
+ end
+ end
+ end
+end
76 cookbooks/database/libraries/provider_database_mysql_user.rb
View
@@ -0,0 +1,76 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require File.join(File.dirname(__FILE__), 'provider_database_mysql')
+
+class Chef
+ class Provider
+ class Database
+ class MysqlUser < Chef::Provider::Database::Mysql
+ include Chef::Mixin::ShellOut
+
+ def load_current_resource
+ Gem.clear_paths
+ require 'mysql'
+ @current_resource = Chef::Resource::DatabaseUser.new(@new_resource.name)
+ @current_resource.username(@new_resource.name)
+ @current_resource
+ end
+
+ def action_create
+ unless exists?
+ begin
+ db.query("CREATE USER '#{@new_resource.username}'@'#{@new_resource.host}' IDENTIFIED BY '#{@new_resource.password}'")
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ def action_drop
+ if exists?
+ begin
+ db.query("DROP USER '#{@new_resource.username}'@'#{@new_resource.host}'")
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ def action_grant
+ begin
+ grant_statement = "GRANT #{@new_resource.privileges.join(', ')} ON #{@new_resource.database_name || "*"}.#{@new_resource.table || "*"} TO '#{@new_resource.username}'@'#{@new_resource.host}' IDENTIFIED BY '#{@new_resource.password}'"
+ Chef::Log.info("#{@new_resource}: granting access with statement [#{grant_statement}]")
+ db.query(grant_statement)
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+
+ private
+ def exists?
+ db.query("select User,host from mysql.user where User='#{@new_resource.username}' AND host = '#{@new_resource.host}'").num_rows != 0
+ end
+
+ end
+ end
+ end
+end
130 cookbooks/database/libraries/provider_database_postgresql.rb
View
@@ -0,0 +1,130 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Author:: Lamont Granquist (<lamont@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require 'chef/provider'
+
+class Chef
+ class Provider
+ class Database
+ class Postgresql < Chef::Provider
+ include Chef::Mixin::ShellOut
+
+ def load_current_resource
+ Gem.clear_paths
+ require 'pg'
+ @current_resource = Chef::Resource::Database.new(@new_resource.name)
+ @current_resource.database_name(@new_resource.database_name)
+ @current_resource
+ end
+
+ def action_create
+ unless exists?
+ begin
+ encoding = @new_resource.encoding
+ if encoding != "DEFAULT"
+ encoding = "'#{@new_resource.encoding}'"
+ end
+ Chef::Log.debug("#{@new_resource}: Creating database #{new_resource.database_name}")
+ create_sql = "CREATE DATABASE #{new_resource.database_name}"
+ create_sql += " TEMPLATE = #{new_resource.template}" if new_resource.template
+ create_sql += " ENCODING = #{encoding}" if new_resource.encoding
+ create_sql += " TABLESPACE = #{new_resource.tablespace}" if new_resource.tablespace
+ create_sql += " CONNECTION LIMIT = #{new_resource.connection_limit}" if new_resource.connection_limit
+ create_sql += " OWNER = #{new_resource.owner}" if new_resource.owner
+ Chef::Log.debug("#{@new_resource}: Performing query [#{create_sql}]")
+ db("template1").query(create_sql)
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ def action_drop
+ if exists?
+ begin
+ Chef::Log.debug("#{@new_resource}: Dropping database #{new_resource.database_name}")
+ db("template1").query("drop database #{new_resource.database_name}")
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ def action_query
+ if exists?
+ begin
+ Chef::Log.debug("#{@new_resource}: Performing query [#{new_resource.sql}]")
+ db(@new_resource.database_name).query(@new_resource.sql)
+ Chef::Log.debug("#{@new_resource}: query [#{new_resource.sql}] succeeded")
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ private
+
+ def exists?
+ begin
+ Chef::Log.debug("#{@new_resource}: checking if database #{@new_resource.database_name} exists")
+ ret = db("template1").query("select * from pg_database where datname = '#{@new_resource.database_name}'").num_tuples != 0
+ ret ? Chef::Log.debug("#{@new_resource}: database #{@new_resource.database_name} exists") :
+ Chef::Log.debug("#{@new_resource}: database #{@new_resource.database_name} does not exist")
+ ensure
+ close
+ end
+ ret
+ end
+
+ #
+ # Specifying the database in the connection parameter for the postgres resource is not recommended.
+ #
+ # - action_create/drop/exists will use the "template1" database to do work by default.
+ # - action_query will use the resource database_name.
+ # - specifying a database in the connection will override this behavior
+ #
+ def db(dbname = nil)
+ close if @db
+ dbname = @new_resource.connection[:database] if @new_resource.connection[:database]
+ host = @new_resource.connection[:host]
+ port = @new_resource.connection[:port] || 5432
+ user = @new_resource.connection[:username] || "postgres"
+ Chef::Log.debug("#{@new_resource}: connecting to database #{dbname} on #{host}:#{port} as #{user}")
+ password = @new_resource.connection[:password] || node[:postgresql][:password][:postgres]
+ @db = ::PGconn.new(
+ :host => host,
+ :port => port,
+ :dbname => dbname,
+ :user => user,
+ :password => password
+ )
+ end
+
+ def close
+ @db.close rescue nil
+ @db = nil
+ end
+
+ end
+ end
+ end
+end
83 cookbooks/database/libraries/provider_database_postgresql_user.rb
View
@@ -0,0 +1,83 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Author:: Lamont Granquist (<lamont@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require File.join(File.dirname(__FILE__), 'provider_database_postgresql')
+
+class Chef
+ class Provider
+ class Database
+ class PostgresqlUser < Chef::Provider::Database::Postgresql
+ include Chef::Mixin::ShellOut
+
+ def load_current_resource
+ Gem.clear_paths
+ require 'pg'
+ @current_resource = Chef::Resource::DatabaseUser.new(@new_resource.name)
+ @current_resource.username(@new_resource.name)
+ @current_resource
+ end
+
+ def action_create
+ unless exists?
+ begin
+ db("template1").query("CREATE USER #{@new_resource.username} WITH PASSWORD '#{@new_resource.password}'")
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ def action_drop
+ if exists?
+ begin
+ db("template1").query("DROP USER #{@new_resource.username}")
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ def action_grant
+ begin
+ # FIXME: grants on individual tables
+ grant_statement = "GRANT #{@new_resource.privileges.join(', ')} ON DATABASE #{@new_resource.database_name} TO #{@new_resource.username}"
+ Chef::Log.info("#{@new_resource}: granting access with statement [#{grant_statement}]")
+ db(@new_resource.database_name).query(grant_statement)
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+
+ private
+ def exists?
+ begin
+ exists = db("template1").query("select * from pg_user where usename='#{@new_resource.username}'").num_tuples != 0
+ ensure
+ close
+ end
+ exists
+ end
+
+ end
+ end
+ end
+end
109 cookbooks/database/libraries/provider_database_sql_server.rb
View
@@ -0,0 +1,109 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require 'chef/provider'
+
+class Chef
+ class Provider
+ class Database
+ class SqlServer < Chef::Provider
+ include Chef::Mixin::ShellOut
+
+ def load_current_resource
+ Gem.clear_paths
+ require 'tiny_tds'
+ @current_resource = Chef::Resource::Database.new(@new_resource.name)
+ @current_resource.database_name(@new_resource.database_name)
+ @current_resource
+ end
+
+ def action_create
+ unless exists?
+ begin
+ Chef::Log.debug("#{@new_resource}: Creating database #{new_resource.database_name}")
+ db.execute("CREATE DATABASE [#{new_resource.database_name}]").do
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ def action_drop
+ if exists?
+ begin
+ Chef::Log.debug("#{@new_resource}: Dropping database #{new_resource.database_name}")
+ db.execute("DROP DATABASE #{new_resource.database_name}").do
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ def action_query
+ if exists?
+ begin
+ #db.select_db(@new_resource.database_name) if @new_resource.database_name
+ Chef::Log.debug("#{@new_resource}: Performing query [#{new_resource.sql}]")
+ db.execute(@new_resource.sql).do
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+ end
+
+ private
+ def exists?
+ exists = false
+ begin
+ result = db.execute("SELECT name FROM sys.databases")
+ result.each do |row|
+ if row['name'] =~ /#{@new_resource.database_name}/i
+ exists = true
+ break
+ end
+ end
+ result.cancel
+ ensure
+ close
+ end
+ exists
+ end
+
+ def db
+ @db ||= begin
+ ::TinyTds::Client.new(
+ :host => @new_resource.connection[:host],
+ :username => @new_resource.connection[:username],
+ :password => @new_resource.connection[:password],
+ :port => @new_resource.connection[:port] || 1433
+ )
+ end
+ end
+
+ def close
+ @db.close rescue nil
+ @db = nil
+ end
+
+ end
+ end
+ end
+end
106 cookbooks/database/libraries/provider_database_sql_server_user.rb
View
@@ -0,0 +1,106 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require File.join(File.dirname(__FILE__), 'provider_database_sql_server')
+
+class Chef
+ class Provider
+ class Database
+ class SqlServerUser < Chef::Provider::Database::SqlServer
+ include Chef::Mixin::ShellOut
+
+ def load_current_resource
+ Gem.clear_paths
+ require 'tiny_tds'
+ @current_resource = Chef::Resource::DatabaseUser.new(@new_resource.name)
+ @current_resource.username(@new_resource.name)
+ @current_resource
+ end
+
+ def action_create
+ begin
+ unless exists?(:logins)
+ db.execute("CREATE LOGIN #{@new_resource.username} WITH PASSWORD = '#{@new_resource.password}', CHECK_POLICY = OFF").do
+ @new_resource.updated_by_last_action(true)
+ end
+ unless exists?(:users)
+ if @new_resource.database_name
+ Chef::Log.info("#{@new_resource} creating user in '#{@new_resource.database_name}' database context.")
+ db.execute("USE #{@new_resource.database_name}").do
+ else
+ Chef::Log.info("#{@new_resource} database_name not provided, creating user in global context.")
+ end
+ db.execute("CREATE USER #{@new_resource.username} FOR LOGIN #{@new_resource.username}").do
+ @new_resource.updated_by_last_action(true)
+ end
+ ensure
+ close
+ end
+ end
+
+ def action_drop
+ begin
+ if exists?(:users)
+ db.execute("DROP USER #{@new_resource.username}").do
+ @new_resource.updated_by_last_action(true)
+ end
+ if exists?(:logins)
+ db.execute("DROP LOGIN #{@new_resource.username}").do
+ @new_resource.updated_by_last_action(true)
+ end
+ ensure
+ close
+ end
+ end
+
+ def action_grant
+ begin
+ if @new_resource.password
+ action_create
+ end
+ Chef::Application.fatal!('Please provide a database_name, SQL Server does not support global GRANT statements.') unless @new_resource.database_name
+ grant_statement = "GRANT #{@new_resource.privileges.join(', ')} ON DATABASE::#{@new_resource.database_name} TO #{@new_resource.username}"
+ Chef::Log.info("#{@new_resource} granting access with statement [#{grant_statement}]")
+ db.execute("USE #{@new_resource.database_name}").do
+ db.execute(grant_statement).do
+ @new_resource.updated_by_last_action(true)
+ ensure
+ close
+ end
+ end
+
+ private
+ def exists?(type=:users)
+ case type
+ when :users
+ table = "database_principals"
+ if @new_resource.database_name
+ Chef::Log.debug("#{@new_resource} searching for existing user in '#{@new_resource.database_name}' database context.")
+ db.execute("USE #{@new_resource.database_name}").do
+ end
+ when :logins
+ table = "server_principals"
+ end
+
+ result = db.execute("SELECT name FROM sys.#{table} WHERE name='#{@new_resource.username}'")
+ result.each.any?
+ end
+ end
+ end
+ end
+end
111 cookbooks/database/libraries/resource_database.rb
View
@@ -0,0 +1,111 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require 'chef/resource'
+
+class Chef
+ class Resource
+ class Database < Chef::Resource
+
+ def initialize(name, run_context=nil)
+ super
+ @resource_name = :database
+ @database_name = name
+ @allowed_actions.push(:create, :drop, :query)
+ @action = :create
+ end
+
+ def database_name(arg=nil)
+ set_or_return(
+ :database_name,
+ arg,
+ :kind_of => String
+ )
+ end
+
+ def connection(arg=nil)
+ set_or_return(
+ :connection,
+ arg,
+ :required => true
+ )
+ end
+
+ def sql(arg=nil)
+ set_or_return(
+ :sql,
+ arg,
+ :kind_of => String
+ )
+ end
+
+ def sql(arg=nil)
+ set_or_return(
+ :sql,
+ arg,
+ :kind_of => String
+ )
+ end
+
+ #
+ def template(arg=nil)
+ set_or_return(
+ :template,
+ arg,
+ :kind_of => String,
+ :default => 'DEFAULT'
+ )
+ end
+
+ def encoding(arg=nil)
+ set_or_return(
+ :encoding,
+ arg,
+ :kind_of => String,
+ :default => 'DEFAULT'
+ )
+ end
+
+ def tablespace(arg=nil)
+ set_or_return(
+ :tablespace,
+ arg,
+ :kind_of => String,
+ :default => 'DEFAULT'
+ )
+ end
+
+ def connection_limit(arg=nil)
+ set_or_return(
+ :connection_limit,
+ arg,
+ :kind_of => String,
+ :default => '-1'
+ )
+ end
+
+ def owner(arg=nil)
+ set_or_return(
+ :owner,
+ arg,
+ :kind_of => String
+ )
+ end
+ end
+ end
+end
90 cookbooks/database/libraries/resource_database_user.rb
View
@@ -0,0 +1,90 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require File.join(File.dirname(__FILE__), 'resource_database')
+
+class Chef
+ class Resource
+ class DatabaseUser < Chef::Resource::Database
+
+ def initialize(name, run_context=nil)
+ super
+ @resource_name = :database_user
+ @username = name
+
+ @database_name = nil
+ @table = nil
+ @host = 'localhost'
+ @privileges = [:all]
+
+ @allowed_actions.push(:create, :drop, :grant)
+ @action = :create
+ end
+
+ def database_name(arg=nil)
+ set_or_return(
+ :database_name,
+ arg,
+ :kind_of => String
+ )
+ end
+
+ def username(arg=nil)
+ set_or_return(
+ :username,
+ arg,
+ :kind_of => String
+ )
+ end
+
+ def password(arg=nil)
+ set_or_return(
+ :password,
+ arg,
+ :kind_of => String,
+ :required => true
+ )
+ end
+
+ def table(arg=nil)
+ set_or_return(
+ :table,
+ arg,
+ :kind_of => String
+ )
+ end
+
+ def host(arg=nil)
+ set_or_return(
+ :host,
+ arg,
+ :kind_of => String
+ )
+ end
+
+ def privileges(arg=nil)
+ set_or_return(
+ :privileges,
+ arg,
+ :kind_of => Array
+ )
+ end
+
+ end
+ end
+end
34 cookbooks/database/libraries/resource_mysql_database.rb
View
@@ -0,0 +1,34 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require File.join(File.dirname(__FILE__), 'resource_database')
+require File.join(File.dirname(__FILE__), 'provider_database_mysql')
+
+class Chef
+ class Resource
+ class MysqlDatabase < Chef::Resource::Database
+
+ def initialize(name, run_context=nil)
+ super
+ @resource_name = :mysql_database
+ @provider = Chef::Provider::Database::Mysql
+ end
+
+ end
+ end
+end
34 cookbooks/database/libraries/resource_mysql_database_user.rb
View
@@ -0,0 +1,34 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require File.join(File.dirname(__FILE__), 'resource_database_user')
+require File.join(File.dirname(__FILE__), 'provider_database_mysql_user')
+
+class Chef
+ class Resource
+ class MysqlDatabaseUser < Chef::Resource::DatabaseUser
+
+ def initialize(name, run_context=nil)
+ super
+ @resource_name = :mysql_database_user
+ @provider = Chef::Provider::Database::MysqlUser
+ end
+
+ end
+ end
+end
35 cookbooks/database/libraries/resource_postgresql_database.rb
View
@@ -0,0 +1,35 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Author:: Lamont Granquist (<lamont@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require File.join(File.dirname(__FILE__), 'resource_database')
+require File.join(File.dirname(__FILE__), 'provider_database_postgresql')
+
+class Chef
+ class Resource
+ class PostgresqlDatabase < Chef::Resource::Database
+
+ def initialize(name, run_context=nil)
+ super
+ @resource_name = :postgresql_database
+ @provider = Chef::Provider::Database::Postgresql
+ end
+
+ end
+ end
+end
35 cookbooks/database/libraries/resource_postgresql_database_user.rb
View
@@ -0,0 +1,35 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Author:: Lamont Granquist (<lamont@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require File.join(File.dirname(__FILE__), 'resource_database_user')
+require File.join(File.dirname(__FILE__), 'provider_database_postgresql_user')
+
+class Chef
+ class Resource
+ class PostgresqlDatabaseUser < Chef::Resource::DatabaseUser
+
+ def initialize(name, run_context=nil)
+ super
+ @resource_name = :postgresql_database_user
+ @provider = Chef::Provider::Database::PostgresqlUser
+ end
+
+ end
+ end
+end
34 cookbooks/database/libraries/resource_sql_server_database.rb
View
@@ -0,0 +1,34 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require File.join(File.dirname(__FILE__), 'resource_database')
+require File.join(File.dirname(__FILE__), 'provider_database_sql_server')
+
+class Chef
+ class Resource
+ class SqlServerDatabase < Chef::Resource::Database
+
+ def initialize(name, run_context=nil)
+ super
+ @resource_name = :sql_server_database
+ @provider = Chef::Provider::Database::SqlServer
+ end
+
+ end
+ end
+end
34 cookbooks/database/libraries/resource_sql_server_database_user.rb
View
@@ -0,0 +1,34 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require File.join(File.dirname(__FILE__), 'resource_database_user')
+require File.join(File.dirname(__FILE__), 'provider_database_sql_server_user')
+
+class Chef
+ class Resource
+ class SqlServerDatabaseUser < Chef::Resource::DatabaseUser
+
+ def initialize(name, run_context=nil)
+ super
+ @resource_name = :sql_server_database_user
+ @provider = Chef::Provider::Database::SqlServerUser
+ end
+
+ end
+ end
+end
45 cookbooks/database/metadata.json
View
@@ -0,0 +1,45 @@
+{
+ "name": "database",
+ "description": "Sets up the database master or slave",
+ "long_description": "Database Cookbook\n=================\n\nThe main highlight of this cookbook is the `database` and `database_user` resources for managing databases and database users in a RDBMS. Providers for MySQL, PostgreSQL and SQL Server are also provided, see usage documentation below.\n\nThis cookbook also contains recipes to configure mysql database masters and slaves and uses EBS for storage, integrating together with the application cookbook utilizing data bags for application related information. These recipes are written primarily to use MySQL and the Opscode mysql cookbook. Other RDBMS may be supported at a later date. This cookbook does not automatically restore database dumps, but does install tools to help with that.\n\nRequirements\n============\n\nChef 0.10.0 or higher required (for Chef environment use).\n\nPlatform\n--------\n\n* Debian, Ubuntu\n* Red Hat, CentOS, Scientific, Fedora\n\nCookbooks\n---------\n\nThe following Opscode cookbooks are dependencies:\n\n* mysql\n* postgresql\n* xfs\n* aws\n\nResources/Providers\n===================\n\nThese resources aim to expose an abstraction layer for interacting with different RDBMS in a general way. Currently the cookbook ships with providers for MySQL, PostgreSQL and SQL Server. Please see specific usage in the __Example__ sections below. The providers use specific Ruby gems to execute commands and carry out actions. These gems will need to be installed before the providers can operate correctly. Specific notes for each RDBS flavor:\n\n- MySQL: leverages the `mysql` gem which is installed as part of the `mysql::client` recipe.\n- PostgreSQL: leverages the `pg` gem which is installed as part of the `postgresql::client` recipe.\n- SQL Server: leverages the `tiny_tds` gem which is installed as part of the `sql_server::client` recipe.\n\n`database`\n----------\n\nManage databases in a RDBMS. Use the proper shortcut resource depending on your RDBMS: `mysql_database`, `postgresql_database` or `sql_server_database`.\n\n### Actions\n\n- :create: create a named database\n- :drop: drop a named database\n- :query: execute an arbitrary query against a named database\n\n### Attribute Parameters\n\n- database_name: name attribute. Name of the database to interact with\n- connection: hash of connection info. valid keys include :host, :port, :username, :password\n- sql: string of sql to execute against the database. used by :query action only\n\n### Providers\n\n- **Chef::Provider::Database::Mysql**: shortcut resource `mysql_database`\n- **Chef::Provider::Database::Postgresql**: shortcut resource `postgresql_database`\n- **Chef::Provider::Database::SqlServer**: shortcut resource `sql_server_database`\n\n### Examples\n\n # create a mysql database\n mysql_database 'oracle_rules' do\n connection ({:host => \"localhost\", :username => 'root', :password => node['mysql']['server_root_password']})\n action :create\n end\n\n # create a sql server database\n sql_server_database 'mr_softie' do\n connection ({:host => \"127.0.0.1\", :port => node['sql_server']['port'], :username => 'sa', :password => node['sql_server']['server_sa_password']})\n action :create\n end\n\n # create a postgresql database\n postgresql_database 'mr_softie' do\n connection ({:host => \"127.0.0.1\", :port => 5432, :username => 'postgres', :password => node['postgresql']['password']['postgres']})\n action :create\n end\n\n # create a postgresql database with additional parameters\n postgresql_database 'mr_softie' do\n connection ({:host => \"127.0.0.1\", :port => 5432, :username => 'postgres', :password => node['postgresql']['password']['postgres']})\n template 'DEFAULT'\n encoding 'DEFAULT'\n tablespace 'DEFAULT'\n connection_limit '-1'\n owner 'postgres'\n action :create\n end\n\n # externalize conection info in a ruby hash\n mysql_connection_info = {:host => \"localhost\", :username => 'root', :password => node['mysql']['server_root_password']}\n sql_server_connection_info = {:host => \"localhost\", :port => node['sql_server']['port'], :username => 'sa', :password => node['sql_server']['server_sa_password']}\n postgresql_connection_info = {:host => \"127.0.0.1\", :port => 5432, :username => 'postgres', :password => node['postgresql']['password']['postgres']}\n\n # same create commands, connection info as an external hash\n mysql_database 'foo' do\n connection mysql_connection_info\n action :create\n end\n sql_server_database 'foo' do\n connection sql_server_connection_info\n action :create\n end\n postgresql_database 'foo' do\n connection postgresql_connection_info\n action :create\n end\n\n # create database, set provider in resource parameter\n database 'bar' do\n connection mysql_connection_info\n provider Chef::Provider::Database::Mysql\n action :create\n end\n database 'bar' do\n connection sql_server_connection_info\n provider Chef::Provider::Database::SqlServer\n action :create\n end\n database 'bar' do\n connection postgresql_connection_info\n provider Chef::Provider::Database::Postgresql\n action :create\n end\n\n # drop a database\n mysql_database \"baz\" do\n connection mysql_connection_info\n action :drop\n end\n\n # query a database\n mysql_database \"flush the privileges\" do\n connection mysql_connection_info\n sql \"flush privileges\"\n action :query\n end\n\n # vacuum a postgres database\n postgres_database \"vacuum databases\" do\n connection postgresql_connection_info\n database_table \"template1\"\n sql \"VACUUM FULL VERBOSE ANALYZE\"\n action :query\n end\n\n`database_user`\n---------------\n\nManage users and user privileges in a RDBMS. Use the proper shortcut resource depending on your RDBMS: `mysql_database_user` or `sql_server_database_user`.\n\n### Actions\n\n- :create: create a user\n- :drop: drop a user\n- :grant: manipulate user privileges on database objects\n\n### Attribute Parameters\n\n- username: name attribute. Name of the database user\n- password: password for the user account\n- database_name: Name of the database to interact with\n- connection: hash of connection info. valid keys include :host, :port, :username, :password\n- privileges: array of database privileges to grant user. used by the :grant action. default is :all\n- host: host where user connections are allowed from. used by MySQL provider only. default is 'localhost'\n- table: table to grant privileges on. used by :grant action and MySQL provider only. default is '*' (all tables)\n\n### Providers\n\n- **Chef::Provider::Database::MysqlUser**: shortcut resource `mysql_database_user`\n- **Chef::Provider::Database::SqlServerUser**: shortcut resource `sql_server_database_user`\n\n### Examples\n\n # create connection info as an external ruby hash\n mysql_connection_info = {:host => \"localhost\", :username => 'root', :password => node['mysql']['server_root_password']}\n sql_server_connection_info = {:host => \"localhost\", :port => node['sql_server']['port'], :username => 'sa', :password => node['sql_server']['server_sa_password']}\n\n # create a mysql user but grant no priveleges\n mysql_database_user 'disenfranchised' do\n connection mysql_connection_info\n password 'super_secret'\n action :create\n end\n\n # do the same but pass the provider to the database resource\n database_user 'disenfranchised' do\n connection mysql_connection_info\n password 'super_secret'\n provider Chef::Provider::Database::MysqlUser\n action :create\n end\n\n # create a sql server user but grant no priveleges\n sql_server_database_user 'disenfranchised' do\n connection sql_server_connection_info\n password 'super_secret'\n action :create\n end\n\n # drop a mysql user\n mysql_database_user \"foo_user\" do\n connection mysql_connection_info\n action :drop\n ends\n\n # bulk drop sql server users\n %w{ disenfranchised foo_user }.each do |user|\n sql_server_database_user user do\n connection sql_server_connection_info\n action :drop\n end\n end\n\n # grant select,update,insert privileges to all tables in foo db from all hosts\n mysql_database_user 'foo_user' do\n connection mysql_connection_info\n password 'super_secret'\n database_name 'foo'\n host '%'\n privileges [:select,:update,:insert]\n action :grant\n end\n\n # grant all privelages on all databases/tables from localhost\n mysql_database_user 'super_user' do\n connection mysql_connection_info\n password 'super_secret'\n action :grant\n end\n\n # grant select,update,insert privileges to all tables in foo db\n sql_server_database_user 'foo_user' do\n connection sql_server_connection_info\n password 'super_secret'\n database_name 'foo'\n privileges [:select,:update,:insert]\n action :grant\n end\n\nRecipes\n=======\n\nebs\\_volume\n-----------\n\n*Note*: This recipe does not currently work on RHEL platforms due to the xfs cookbook not supporting RHEL yet.\n\nLoads the aws information from the data bag. Searches the applications data bag for the database master or slave role and checks that role is applied to the node. Loads the EBS information and the master information from data bags. Uses the aws cookbook LWRP, `aws_ebs_volume` to manage the volume.\n\nOn a master node:\n* if we have an ebs volume already as stored in a data bag, attach it.\n* if we don't have the ebs information then create a new one and attach it.\n* store the volume information in a data bag via a ruby block.\n\nOn a slave node:\n* use the master volume information to generate a snapshot.\n* create the new volume from the snapshot and attach it.\n\nAlso on a master node, generate some configuration for running a snapshot via `chef-solo` from cron.\n\nOn a new filesystem volume, create as XFS, then mount it in /mnt, and also bind-mount it to the mysql data directory (default /var/lib/mysql).\n\nmaster\n------\n\nThis recipe no longer loads AWS specific information, and the database position for replication is no longer stored in a databag because the client might not have permission to write to the databag item. This may be handled in a different way at a future date.\n\nSearches the apps databag for applications, and for each one it will check that the specified database master role is set in both the databag and applied to the node's run list. Then, retrieves the passwords for `root`, `repl` and `debian` users and saves them to the node attributes. If the passwords are not found in the databag, it prints a message that they'll be generated by the mysql cookbook.\n\nThen it adds the application databag database settings to a hash, to use later.\n\nThen it will iterate over the databases and create them with the `mysql_database` resource while adding privileges for application specific database users using the `mysql_database_user` resource.\n\nslave\n-----\n\n_TODO_: Retrieve the master status from a data bag, then start replication using a ruby block. The replication status needs to be handled in some other way for now since the master recipe above doesn't actually set it in the databag anymore.\n\nsnapshot\n--------\n\nRun via Chef Solo. Retrieves the db snapshot configuration from the specified JSON file. Uses the `mysql_database` resource to lock and unlock tables, and does a filesystem freeze and EBS snapshot.\n\nDeprecated Recipes\n==================\n\nThe following recipe is considered deprecated. It is kept for reference purposes.\n\nebs\\_backup\n-----------\n\nOlder style of doing mysql snapshot and replication using Adam Jacob's [ec2_mysql](http://github.com/adamhjk/ec2_mysql) script and library.\n\nData Bags\n=========\n\nThis cookbook uses the apps data bag item for the specified application; see the `application` cookbook's README.md. It also creates data bag items in a bag named 'aws' for storing volume information. In order to interact with EC2, it expects aws to have a main item:\n\n {\n \"id\": \"main\",\n \"ec2_private_key\": \"private key as a string\",\n \"ec2_cert\": \"certificate as a string\",\n \"aws_account_id\": \"\",\n \"aws_secret_access_key\": \"\",\n \"aws_access_key_id\": \"\"\n }\n\nNote: with the Open Source Chef Server, the server using the database recipes must be an admin client or it will not be able to create data bag items. You can modify whether the client is admin by editing it with knife.\n\n knife client edit <client_name>\n {\n ...\n \"admin\": true\n ...\n }\n\nThis is not required if the Chef Server is the Opscode Platform, instead use the ACL feature to modify access for the node to be able to update the data bag.\n\nUsage\n=====\n\nAside from the application data bag (see the README in the application cookbook), create a role for the database master. Use a role.rb in your chef-repo, or create the role directly with knife.\n\n % knife role show my_app_database_master -Fj\n {\n \"name\": \"my_app_database_master\",\n \"chef_type\": \"role\",\n \"json_class\": \"Chef::Role\",\n \"default_attributes\": {\n },\n \"description\": \"\",\n \"run_list\": [\n \"recipe[mysql::server]\",\n \"recipe[database::master]\"\n ],\n \"override_attributes\": {\n }\n }\n\nCreate a `production` environment. This is also used in the `application` cookbook.\n\n % knife environment show production -Fj\n {\n \"name\": \"production\",\n \"description\": \"\",\n \"cookbook_versions\": {\n },\n \"json_class\": \"Chef::Environment\",\n \"chef_type\": \"environment\",\n \"default_attributes\": {\n },\n \"override_attributes\": {\n }\n }\n\n\nThe cookbook `my_app_database` is recommended to set up any application specific database resources such as configuration templates, trending monitors, etc. It is not required, but you would need to create it separately in `site-cookbooks`. Add it to the `my_app_database_master` role.\n\nLicense and Author\n==================\n\nAuthor:: Adam Jacob (<adam@opscode.com>)\nAuthor:: Joshua Timberman (<joshua@opscode.com>)\nAuthor:: AJ Christensen (<aj@opscode.com>)\nAuthor:: Seth Chisamore (<schisamo@opscode.com>)\nAuthor:: Lamont Granquist (<lamont@opscode.com>)\n\nCopyright 2009-2011, Opscode, Inc.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n",
+ "maintainer": "Opscode, Inc.",
+ "maintainer_email": "cookbooks@opscode.com",
+ "license": "Apache 2.0",
+ "platforms": {
+ "debian": ">= 0.0.0",
+ "ubuntu": ">= 0.0.0",
+ "centos": ">= 0.0.0",
+ "suse": ">= 0.0.0",
+ "fedora": ">= 0.0.0",
+ "redhat": ">= 0.0.0",
+ "scientific": ">= 0.0.0"
+ },
+ "dependencies": {
+ "mysql": ">= 1.2.0",
+ "postgresql": ">= 0.0.0",
+ "aws": ">= 0.0.0",
+ "xfs": ">= 0.0.0"
+ },
+ "recommendations": {
+ },
+ "suggestions": {
+ },
+ "conflicting": {
+ },
+ "providing": {
+ },
+ "replacing": {
+ },
+ "attributes": {
+ },
+ "groupings": {
+ },
+ "recipes": {
+ "database": "Empty placeholder",
+ "database::ebs_backup": "Considered deprecated, older way of backing up EBS volumes",
+ "database::ebs_volume": "Sets up an EBS volume in EC2 for the database",
+ "database::master": "Creates application specific user and database",
+ "database::snapshot": "Locks tables and freezes XFS filesystem for replication, assumes EC2 + EBS"
+ },
+ "version": "1.2.0"
+}
21 cookbooks/database/metadata.rb
View
@@ -0,0 +1,21 @@
+maintainer "Opscode, Inc."
+maintainer_email "cookbooks@opscode.com"
+license "Apache 2.0"
+description "Sets up the database master or slave"
+long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
+version "1.2.0"
+
+recipe "database", "Empty placeholder"
+recipe "database::ebs_backup", "Considered deprecated, older way of backing up EBS volumes"
+recipe "database::ebs_volume", "Sets up an EBS volume in EC2 for the database"
+recipe "database::master", "Creates application specific user and database"
+recipe "database::snapshot", "Locks tables and freezes XFS filesystem for replication, assumes EC2 + EBS"
+
+depends "mysql", ">= 1.2.0"
+depends "postgresql"
+depends "aws"
+depends "xfs"
+
+%w{ debian ubuntu centos suse fedora redhat scientific }.each do |os|
+ supports os
+end
20 cookbooks/database/recipes/default.rb
View
@@ -0,0 +1,20 @@
+#
+# Author:: Joshua Timberman (<joshua@opscode.com>)
+# Cookbook Name:: database
+# Recipe:: default
+#
+# Copyright 2009, Opscode, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
89 cookbooks/database/recipes/ebs_backup.rb
View
@@ -0,0 +1,89 @@
+#
+# Author:: Joshua Timberman (<joshua@opscode.com>)
+# Cookbook Name:: database
+# Recipe:: ebs_backup
+#
+# Copyright 2009, Opscode, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+begin
+ aws = Chef::DataBagItem.load('aws', 'main')
+ Chef::Log.info("Loaded AWS information from DataBagItem aws[#{aws['id']}]")
+rescue
+ Chef::Log.fatal("Could not find the 'main' item in the 'aws' data bag")
+ raise
+end
+
+db_role = String.new
+db_master_role = String.new
+db_type = node[:database][:type]
+
+search(:apps) do |app|
+ db_role = app["database_#{db_type}_role"] & node.run_list.roles
+ db_master_role = app["database_master_role"]
+end
+
+ebs_info = Chef::DataBagItem.load(:aws, "ebs_#{db_master_role}_#{node.chef_environment}")
+
+gem_package "dbi"
+gem_package "dbd-mysql"
+
+directory "/mnt/aws-config" do
+ mode 0700
+ owner "root"
+ group "root"
+end
+
+template "/mnt/aws-config/config" do
+ source "aws_config.erb"
+ variables(
+ :access_key => aws['aws_access_key_id'],
+ :secret_key => aws['aws_secret_access_key']
+ )
+ owner "root"
+ group "root"
+ mode 0600
+end
+
+git "/opt/ec2_mysql" do
+ repository "git://github.com/jtimberman/ec2_mysql.git"
+ reference "HEAD"
+ action :sync
+ not_if { ::FileTest.directory?("/opt/ec2_mysql/.git") }
+end
+
+%w{backup restore}.each do |file|
+ template "/usr/local/bin/db-#{file}.sh" do
+ source "ebs-db-#{file}.sh.erb"
+ owner "root"
+ group "root"
+ mode 0700
+ variables(
+ :mysql_root_passwd => node['mysql']['server_root_password'],
+ :mysql_device => node['mysql']['ebs_vol_dev'],
+ :ebs_vol_id => ebs_info['volume_id']
+ )
+ end
+end
+
+if db_type == "master" && node.chef_environment == "production"
+ template "/etc/cron.d/db-backup" do
+ source "ebs-backup-cron.erb"
+ owner "root"
+ group "root"
+ mode 0644
+ backup false
+ end
+end
201 cookbooks/database/recipes/ebs_volume.rb
View
@@ -0,0 +1,201 @@
+#
+# Author:: Joshua Timberman (<joshua@opscode.com>)
+# Author:: AJ Christensen (<aj@opscode.com>)
+# Cookbook Name:: database
+# Recipe:: ebs_volume
+#
+# Copyright 2009-2010, Opscode, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+if node[:ec2]
+ include_recipe "aws"
+ include_recipe "xfs"
+
+ begin
+ aws = Chef::DataBagItem.load(:aws, :main)
+ Chef::Log.info("Loaded AWS information from DataBagItem aws[#{aws['id']}]")
+ rescue
+ Chef::Log.fatal("Could not find the 'main' item in the 'aws' data bag")
+ raise
+ end
+
+ ebs_vol_dev = node['mysql']['ebs_vol_dev']
+ if (platform?("ubuntu") && node['platform_version'].to_f >= 11.04)
+ ebs_vol_dev_mount = ebs_vol_dev.sub(/^\/dev\/sd/, "/dev/xvd")
+ else
+ ebs_vol_dev_mount = ebs_vol_dev
+ end
+ ebs_vol_id = String.new
+ db_type = String.new
+ db_role = String.new
+ master_role = String.new
+ slave_role = String.new
+ root_pw = String.new
+ snapshots_to_keep = String.new
+
+ search(:apps) do |app|
+ if (app["database_master_role"] & node.run_list.roles).length == 1 || (app["database_slave_role"] & node.run_list.roles).length == 1
+ master_role = app["database_master_role"]
+ slave_role = app["database_slave_role"]
+ root_pw = app["mysql_root_password"][node.chef_environment]
+ snapshots_to_keep = app["snapshots_to_keep"][node.chef_environment]
+
+ if (master_role & node.run_list.roles).length == 1
+ db_type = "master"
+ db_role = RUBY_VERSION.to_f <= 1.8 ? master_role : master_role.join
+ elsif (slave_role & node.run_list.roles).length == 1
+ db_type = "slave"
+ db_role = RUBY_VERSION.to_f <= 1.8 ? slave_role : slave_role.join
+ end
+
+ Chef::Log.info "database::ebs_volume - db_role: #{db_role} db_type: #{db_type}"
+ end
+ end
+
+ begin
+ ebs_info = Chef::DataBagItem.load(:aws, "ebs_#{db_role}_#{node.chef_environment}")
+ Chef::Log.info("Loaded #{ebs_info['volume_id']} from DataBagItem aws[#{ebs_info['id']}]")
+ rescue
+ Chef::Log.warn("Could not find the 'ebs_#{db_role}_#{node.chef_environment}' item in the 'aws' data bag")
+ ebs_info = Hash.new
+ end
+
+ begin
+ master_info = Chef::DataBagItem.load(:aws, "ebs_#{master_role}_#{node.chef_environment}")
+ Chef::Log.info "Loaded #{master_info['volume_id']} from DataBagItem aws[#{master_info['id']}]"
+ rescue
+ Chef::Application.fatal! "Could not load replication masters snapshot details", -41 if db_type == "slave"
+ end
+
+ ruby_block "store_#{db_role}_#{node.chef_environment}_volid" do
+ block do
+ ebs_vol_id = node[:aws][:ebs_volume]["#{db_role}_#{node.chef_environment}"][:volume_id]
+
+ unless ebs_info['volume_id']
+ item = {
+ "id" => "ebs_#{db_role}_#{node.chef_environment}",
+ "volume_id" => ebs_vol_id
+ }
+ Chef::Log.info "Storing volume_id #{item.inspect}"
+ databag_item = Chef::DataBagItem.new
+ databag_item.data_bag("aws")
+ databag_item.raw_data = item
+ databag_item.save
+ Chef::Log.info("Created #{item['id']} in #{databag_item.data_bag}")
+ end
+ end
+ action :nothing
+ end
+
+ aws_ebs_volume "#{db_role}_#{node.chef_environment}" do
+ aws_access_key aws['aws_access_key_id']
+ aws_secret_access_key aws['aws_secret_access_key']
+ size 50
+ device ebs_vol_dev
+ snapshots_to_keep snapshots_to_keep
+ case db_type
+ when "master"
+ if ebs_info['volume_id'] && ebs_info['volume_id'] =~ /vol/
+ volume_id ebs_info['volume_id']
+ action :attach
+ elsif ebs_info['volume_id'] && ebs_info['volume_id'] =~ /snap/
+ snapshot_id ebs_info['volume_id']
+ action [ :create, :attach ]
+ else
+ action [ :create, :attach ]
+ end
+ notifies :create, resources(:ruby_block => "store_#{db_role}_#{node.chef_environment}_volid")
+ when "slave"
+ if master_info['volume_id']
+ snapshot_id master_info['volume_id']
+ action [:create, :attach]
+ else
+ Chef::Log.warn("Couldn't detect snapshot ID.")
+ action :nothing
+ end
+ end
+ provider "aws_ebs_volume"
+ end
+
+ if db_type == "master"
+ Chef::Log.info "Setting up templates for chef-solo snapshots"
+ template "/etc/chef/chef-solo-database-snapshot.rb" do
+ source "chef-solo-database-snapshot.rb.erb"
+ variables :cookbook_path => Chef::Config[:cookbook_path]
+ owner "root"
+ group "root"
+ mode 0600
+ end
+
+ template "/etc/chef/chef-solo-database-snapshot.json" do
+ source "chef-solo-database-snapshot.json.erb"
+ variables(
+ :output => {
+ 'db_snapshot' => {
+ 'ebs_vol_dev' => node.mysql.ec2_path,
+ 'db_role' => db_role,
+ 'app_environment' => node.chef_environment,
+ 'username' => 'root',
+ 'password' => root_pw,
+ 'aws_access_key_id' => aws['aws_access_key_id'],
+ 'aws_secret_access_key' => aws['aws_secret_access_key'],
+ 'snapshots_to_keep' => snapshots_to_keep,
+ 'volume_id' => ebs_info['volume_id']
+ },
+ 'run_list' => [
+ "recipe[database::snapshot]"
+ ]
+ }
+ )
+ owner "root"
+ group "root"
+ mode 0600
+ end
+
+ template "/etc/cron.d/chef-solo-database-snapshot" do
+ source "chef-solo-database-snapshot.cron.erb"
+ variables(
+ :json_attribs => "/etc/chef/chef-solo-database-snapshot.json",
+ :config_file => "/etc/chef/chef-solo-database-snapshot.rb"
+ )
+ owner "root"
+ group "root"
+ mode 0600
+ end
+ end
+
+ execute "mkfs.xfs #{ebs_vol_dev_mount}" do
+ only_if "xfs_admin -l #{ebs_vol_dev_mount} 2>&1 | grep -qx 'xfs_admin: #{ebs_vol_dev_mount} is not a valid XFS filesystem (unexpected SB magic number 0x00000000)'"
+ end
+
+ %w{ec2_path data_dir}.each do |dir|
+ directory node['mysql'][dir] do
+ mode 0755
+ end
+ end
+
+ mount node['mysql']['ec2_path'] do
+ device ebs_vol_dev_mount
+ fstype "xfs"
+ action :mount
+ end
+
+ mount node['mysql']['data_dir'] do
+ device node['mysql']['ec2_path']
+ fstype "none"
+ options "bind,rw"
+ action :mount
+ end
+end
78 cookbooks/database/recipes/master.rb
View
@@ -0,0 +1,78 @@
+#
+# Author:: Joshua Timberman (<joshua@opscode.com>)
+# Cookbook Name:: database
+# Recipe:: master
+#
+# Copyright 2009-2010, Opscode, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# This is potentially destructive to the nodes mysql password attributes, since
+# we iterate over all the app databags. If this database server provides
+# databases for multiple applications, the last app found in the databags
+# will win out, so make sure the databags have the same passwords set for
+# the root, repl, and debian-sys-maint users.
+#
+
+db_info = Hash.new
+root_pw = String.new
+
+search(:apps) do |app|
+ (app['database_master_role'] & node.run_list.roles).each do |dbm_role|
+ %w{ root repl debian }.each do |user|
+ user_pw = app["mysql_#{user}_password"]
+ if !user_pw.nil? and user_pw[node.chef_environment]
+ Chef::Log.debug("Saving password for #{user} as node attribute node['mysql']['server_#{user}_password'")
+ node.set['mysql']["server_#{user}_password"] = user_pw[node.chef_environment]
+ node.save
+ else
+ log "A password for MySQL user #{user} was not found in DataBag 'apps' item '#{app["id"]}' for environment ' for #{node.chef_environment}'." do
+ level :warn
+ end
+ log "A random password will be generated by the mysql cookbook and added as 'node.mysql.server_#{user}_password'. Edit the DataBag item to ensure it is set correctly on new nodes" do
+ level :warn
+ end
+ end
+ end
+ app['databases'].each do |env,db|
+ db_info[env] = db
+ end
+ end
+end
+
+include_recipe "mysql::server"
+
+connection_info = {:host => "localhost", :username => 'root', :password => node['mysql']['server_root_password']}
+
+search(:apps) do |app|
+ (app['database_master_role'] & node.run_list.roles).each do |dbm_role|
+ app['databases'].each do |env,db|
+ if env =~ /#{node.chef_environment}/
+ mysql_database "create #{db['database']}" do
+ database_name db['database']
+ connection connection_info
+ action :create
+ end
+ %W{ % #{node['fqdn']} localhost }.each do |h|
+ mysql_database_user db['username'] do
+ connection connection_info
+ password db['password']
+ database_name db['database']
+ host h
+ action :grant
+ end
+ end
+ end
+ end
+ end
+end
62 cookbooks/database/recipes/snapshot.rb
View
@@ -0,0 +1,62 @@
+#
+# Author:: AJ Christensen (<aj@opscode.com>)
+# Cookbook Name:: database
+# Recipe:: snapshot
+#
+# Copyright 2009-2010, Opscode, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+include_recipe "aws"
+include_recipe "xfs"
+
+%w{ebs_vol_dev db_role app_environment username password aws_access_key_id aws_secret_access_key snapshots_to_keep volume_id}.collect do |key|
+ Chef::Application.fatal!("Required db_snapshot configuration #{key} not found.", -47) unless node.db_snapshot.has_key? key
+end
+
+connection_info = {:host => "localhost", :username => node.db_snapshot.username, :password => node.db_snapshot.password}
+
+mysql_database "locking tables for #{node.db_snapshot.app_environment}" do
+ connection connection_info
+ sql "flush tables with read lock"
+ action :query
+end
+
+execute "xfs freeze" do
+ command "xfs_freeze -f #{node.db_snapshot.ebs_vol_dev}"
+end
+
+aws_ebs_volume "#{node.db_snapshot.db_role.first}_#{node.db_snapshot.app_environment}" do
+ aws_access_key node.db_snapshot.aws_access_key_id
+ aws_secret_access_key node.db_snapshot.aws_secret_access_key
+ size 50
+ device node.db_snapshot.ebs_vol_dev
+ snapshots_to_keep node.db_snapshot.snapshots_to_keep
+ action :snapshot
+ volume_id node.db_snapshot.volume_id
+ ignore_failure true # if this fails, continue to unfreeze and unlock
+end
+
+execute "xfs unfreeze" do
+ command "xfs_freeze -u #{node.db_snapshot.ebs_vol_dev}"
+end
+
+mysql_database "unflushing tables for #{node.db_snapshot.app_environment}" do
+ connection connection_info
+ sql "unlock tables"
+ action :query
+end
+
+aws_ebs_volume "#{node.db_snapshot.db_role.first}_#{node.db_snapshot.app_environment}" do
+ action :prune
+end
8 cookbooks/database/templates/default/app_grants.sql.erb
View
@@ -0,0 +1,8 @@
+# Generated by Chef. Local modifications will be overwritten.
+<% @db_info.each do |env,db| -%>
+# Privileges for databases in <%= env %>
+GRANT ALL ON <%= db['database'] %>.* TO '<%= db['username'] %>'@'localhost' IDENTIFIED BY '<%= db['password'] %>';
+GRANT ALL ON <%= db['database'] %>.* TO '<%= db['username'] %>'@'<%= node['fqdn'] %>' IDENTIFIED BY '<%= db['password'] %>';
+GRANT ALL ON <%= db['database'] %>.* TO '<%= db['username'] %>'@'%' IDENTIFIED BY '<%= db['password'] %>';
+<% end -%>
+flush privileges;
3  cookbooks/database/templates/default/aws_config.erb
View
@@ -0,0 +1,3 @@
+AWS_ACCESS_KEY_ID=<%= @access_key %>
+AWS_SECRET_ACCESS_KEY=<%= @secret_key %>
+BUCKET_BASE_NAME=db-backups
10 cookbooks/database/templates/default/chef-solo-database-snapshot.cron.erb
View
@@ -0,0 +1,10 @@
+# Managed by Chef
+# m h dom mon dow command
+# Keep 1 day of hourly, 7 days of daily, 4 weeks of weekly and 1 year of monthly snapshots
+PATH=/usr/sbin:/usr/bin:/sbin:/bin
+<% cs = "chef-solo -j #{@json_attribs} -c #{@config_file}" %>
+00 * * * * root <%= cs %>
+15 0 * * * root <%= cs %>
+30 0 * * 0 root <%= cs %>
+45 0 1 * * root <%= cs %>
+
1  cookbooks/database/templates/default/chef-solo-database-snapshot.json.erb
View
@@ -0,0 +1 @@
+<%= require 'json'; JSON.pretty_generate(@output) %>
6 cookbooks/database/templates/default/chef-solo-database-snapshot.rb.erb
View
@@ -0,0 +1,6 @@
+executable_path ENV['PATH'] ? ENV['PATH'].split(File::PATH_SEPARATOR) : []
+<% if @cookbook_path.is_a? Array %>
+ cookbook_path [ <%= @cookbook_path.collect { |cb| "\"#{cb}\""}.join(", ") -%> ]
+<% else %>
+ cookbook_path "<%= @cookbook_path -%>"
+<% end %>
2  cookbooks/database/templates/default/ebs-backup-cron.erb
View
@@ -0,0 +1,2 @@
+# Chef Name: ebs_db_backup
+15 0 * * * root /usr/local/bin/db-backup.sh
8 cookbooks/database/templates/default/ebs-db-backup.sh.erb
View
@@ -0,0 +1,8 @@
+#!/bin/bash
+#
+# Back up a MySQL database via EBS snapshot
+
+. /mnt/aws-config/config
+
+/opt/ec2_mysql/bin/ec2_mysql -a $AWS_ACCESS_KEY_ID -s $AWS_SECRET_ACCESS_KEY -p '<%= @mysql_root_passwd %>' -k 5 master
+echo "done"
10 cookbooks/database/templates/default/ebs-db-restore.sh.erb
View
@@ -0,0 +1,10 @@
+#!/bin/bash
+#
+# Restore a MySQL database from EBS
+
+mkdir -p /mnt/restore
+
+. /mnt/aws-config/config
+
+/opt/ec2_mysql/bin/ec2_mysql -a $AWS_ACCESS_KEY_ID -s $AWS_SECRET_ACCESS_KEY -p '<%= @mysql_root_password %>' -v '<%= @ebs_vol_id %>' -m /mnt/restore -d <%= @mysql_device %> -r <%= @mysql_device %> -l debug -n slave
+echo "done"
27 cookbooks/database/templates/default/s3cfg.erb
View
@@ -0,0 +1,27 @@
+[default]
+access_key = <%= @aws['aws_access_key_id'] %>
+acl_public = False
+bucket_location = US
+debug_syncmatch = False
+default_mime_type = binary/octet-stream
+delete_removed = False
+dry_run = False
+encrypt = False
+force = False
+gpg_command = /usr/bin/gpg
+gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
+gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
+gpg_passphrase =
+guess_mime_type = False
+host_base = s3.amazonaws.com
+host_bucket = %(bucket)s.s3.amazonaws.com
+human_readable_sizes = False
+preserve_attrs = True
+proxy_host =
+proxy_port = 0
+recv_chunk = 4096
+secret_key = <%= @aws['aws_secret_access_key'] %>
+send_chunk = 4096
+simpledb_host = sdb.amazonaws.com
+use_https = True
+verbosity = WARNING
Please sign in to comment.
Something went wrong with that request. Please try again.