Goodwill, a Thrift types store
Goodwill is a store to share Thrift schemas.
It provides an HTML-based UI and a JSON API to create and document them. Goodwill is fully integrated with the action-core (github.com/pierre/action-core) to browse Thrift-based sequence files on HDFS. Another usecase is to export Thrift data from HDFS to a SQL-based data warehouse: Goodwill is the middle-man that defines field types and does the translation between Thrift types and SQL types.
To get started, one can use the CSV file store:
mvn -Dgoodwill.store.type=csv -Dgoodwill.store.csv.file.path=examples/file_store.csv jetty:run
This creates a store with a single type Awesomeness. You can browse it at:
To access the API, specify the Accept header:
curl -H'Accept: application/json' http://127.0.0.1:8080/registrar/Awesomeness
Note: the CSV file store is currently read-only. If you want to fix it, look at updateType in github.com/pierre/goodwill/blob/master/src/main/java/com/ning/metrics/goodwill/store/CSVFileStore.java.
A more common usecase is to use the MySQL backend. Simply create a goodwill database and create the following table:
CREATE TABLE `thrift_types` ( `event_type` varchar(255) DEFAULT NULL, `field_id` int(11) DEFAULT NULL, `field_type` varchar(255) DEFAULT NULL, `field_name` varchar(255) DEFAULT NULL, `sql_type` varchar(255) DEFAULT NULL, `sql_length` int(11) DEFAULT NULL, `sql_scale` int(11) DEFAULT NULL, `sql_precision` int(11) DEFAULT NULL, `description` varchar(255) DEFAULT NULL, `id` int(11) NOT NULL AUTO_INCREMENT, PRIMARY KEY (`id`), UNIQUE KEY `unique_fields` (`event_type`,`field_id`) ) ENGINE=InnoDB AUTO_INCREMENT=114 DEFAULT CHARSET=utf8
You can now run Goodwill via
You can configure your sink via Goodwill. A sink is where the data described in Goodwill is stored. For instance, it can JDBC to your data warehouse to create the tables that will hold the events. For now, only Netezza is supported:
mvn -Dgoodwill.sink.type=netezza -Dgoodwill.sink.db.host=netezza.company.com -Dgoodwill.sink.db.name=company \ -Dgoodwill.sink.db.user=pierre -Dgoodwill.sink.db.password=pierre -Dgoodwill.sink.db.port=5480 jetty:run
You may want to look at the configuration parameter goodwill.sink.db.extra_sql in GoodwillConfig.java to see how to customize it.
will create a war.
License (see LICENSE-2.0.txt file for full license)
Copyright 2010 Ning
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.