StreamSets Data Collector - Continuous big data and cloud platform ingest infrastructure
Clone or download
metadaddy SDC-10809. HTTP Client processor should accept request data for PATCH…
… request

Also fixed a minor bug I noticed in the destination where it wasn't treating
PATCH the same as POST etc.

Change-Id: Id68ede4581627031c709346777bcf3fdcb74adc3
Reviewed-on: https://review.streamsets.net/c/18990
Reviewed-by: Eddie Park <eddie@streamsets.com>
Latest commit eaa98e6 Jan 16, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
aerospike-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kafka_0_10-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kafka_0_11-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kafka_0_8_1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kafka_0_8_2-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kafka_0_9-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kafka_1_0-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kafka_1_1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kafka_2_0-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kudu_1_0-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kudu_1_1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kudu_1_2-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kudu_1_3-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kudu_1_4-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kudu_1_5-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kudu_1_6-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-kudu_1_7-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-pulsar_2-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
apache-solr_6_1_0-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
aws-lib SDC-10808. Add no more data event support Jan 17, 2019
aws-secrets-manager-credentialstore-lib SDC-10018. Add AWS Secrets Manager as an option for Credential Stores Jan 17, 2019
aws-support Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
azure-keyvault-credentialstore-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
azure-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
basic-lib SDC-10809. HTTP Client processor should accept request data for PATCH… Jan 19, 2019
bigtable-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
bootstrap Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cassandra-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cassandra_3-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cdh-spark_2_1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cdh-spark_2_2-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cdh-spark_2_3-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cdh_5_10-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_11-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_12-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_13-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_14-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_15-lib SDC-10678. Limit apache hadoop-common propagation Dec 17, 2018
cdh_5_2-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_3-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_4-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_5-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_7-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_8-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_5_9-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_6_0-lib SDC-10709. Fix classpath tests Dec 21, 2018
cdh_kafka_1_2-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cdh_kafka_1_3-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cdh_kafka_2_0-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cdh_kafka_2_1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cdh_kafka_3_0-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cdh_kafka_3_1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cdh_spark_2_1_r1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cli Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
client-api Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cloudera-integration Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cluster-bootstrap-api Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cluster-bootstrap Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cluster-common Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cluster-hdfs-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
cluster-kafka-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
common Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
commonlib SDC-10390. Netflow decoder need maxDataLen but the field is not avail… Jan 11, 2019
container-common SDC-10774: Make SDC Security Manager optional Jan 7, 2019
container SDC-10629. Refactor dynamic previewer code to integrate more with exi… Jan 7, 2019
couchbase-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
couchbase_5-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
crypto-lib SDC-10705: Encrypt/Decrypt stage error kills pipeline in preview mode Dec 31, 2018
cyberark-credentialstore-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
databricks-ml-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
databricks-ml_2-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
datacollector-ui SDC-10788. List of Stage Libraries in Package Manager shows undefined Jan 10, 2019
dataformats-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
dev-lib SDC-10777: Add event support to Raw Data Origin Jan 9, 2019
dev-support SDC-9524.Updating pom files to support global version update using ve… Jul 19, 2018
dir-spooler-protolib SDC-10704. HDFS Spool from last queued file instead of current file Jan 11, 2019
dist SDC-10018. Add AWS Secrets Manager as an option for Credential Stores Jan 17, 2019
docs SDC-10781 updated 3.7.0 help for the build Jan 8, 2019
e2e-tests Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
elasticsearch-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
elasticsearch_5-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
emr-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
emr_hadoop_2_8_3-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
flume-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
google-cloud-lib SDC-10701. Missing Upgrader for Google Pub-Sub destination Dec 19, 2018
google-common Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
groovy-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
groovy_2_4-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
guavasupport Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hadoop-common Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hbase-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdfs-protolib SDC-10762: Improve renaming error record reason in HDFS Executor Jan 4, 2019
hdp-stagelib-base Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_2-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_3-hive1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_3-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_4-hive1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_4-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_5-flume-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_5-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_6-flume-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_6-hive2-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_6-lib SDC-10587. Fix failing classpath tests Dec 13, 2018
hdp_2_6_1-hive1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hdp_2_6_2-hive1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
hive-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
httpcommonlib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
influxdb_0_9-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
integration-testing SDC-6435: SDC source code incorrectly says it's licensed to the ASF Jun 13, 2017
jdbc-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
jdbc-protolib SDC-10786. Handle big decimals scale jdbc Jan 16, 2019
jks-credentialstore-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
jms-lib SDC-10600: sdc_1.1.0_pipeline_JMS_Consumer_Trash.json Dec 18, 2018
json-dto SDC-10629. Refactor dynamic previewer code to integrate more with exi… Jan 7, 2019
jython-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
jython_2_7-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
kafka-common Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
kafka_multisource-0_10-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
kafka_multisource-0_9-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
kafka_multisource-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
kafka_source-protolib SDC-10601: sdc_1.1.0_pipeline_Kafka_consumer_trash.json Dec 18, 2018
kafka_target-protolib SDC-10659. Speed up Kafka Producer Jan 17, 2019
kinesis-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
kinetica-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
kinetica_6_0-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
kinetica_6_1-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
kinetica_6_2-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
kudu-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
lookup-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapr-cluster-bootstrap Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapr-cluster-bootstrap_2_2 Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapr-common Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapr_5_0-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapr_5_1-lib SDC-10730. Remove HBase dependencies from MapRDB Jan 4, 2019
mapr_5_2-lib SDC-10730. Remove HBase dependencies from MapRDB Jan 4, 2019
mapr_6_0-lib SDC-10730. Remove HBase dependencies from MapRDB Jan 4, 2019
mapr_6_0-mep4-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapr_6_0-mep5-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapr_json-5_2-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapr_json-6_0-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapr_json-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapr_spark_2_1_mep_3_0-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
maprdb-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mapreduce-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
maprfs-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
maprstreams-common Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
maprstreams-multisource-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
maprstreams-source-protolib SDC-10729. MapR Streams standalone and cluster tests failing Jan 11, 2019
maprstreams-target-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mesos-bootstrap Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
messaging-client SDC-10629. Refactor dynamic previewer code to integrate more with exi… Jan 7, 2019
miniSDC Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mleap-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mongodb-protolib SDC-10604: sdc_1.6.0.0_pipeline_Source_Destination.json Dec 18, 2018
mongodb_3-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
mysql-binlog-lib SDC-10519. Update Hikari Pool version Dec 13, 2018
net-commonlib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
omniture-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
pulsar-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
rabbitmq-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
rbgen-maven-plugin Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
redis-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
release Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
root-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
root-proto SDC-10519. Update Hikari Pool version Dec 13, 2018
root SDC-10218. Add dynamic preview support to existing preview resource Jan 7, 2019
rpm Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
salesforce-lib SDC-10694. Support aggregate functions in Salesforce Queries Jan 14, 2019
scripting-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-hbase-0_98 SDC-10691. Limit hdfs dependency propagation Dec 17, 2018
sdc-hbase-2_0 SDC-10691. Limit hdfs dependency propagation Dec 17, 2018
sdc-hbase-api SDC-10718. mapr libs should not include hadoop jars Dec 26, 2018
sdc-kafka-api Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-kafka_0_10 Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-kafka_0_11-common Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-kafka_0_11 Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-kafka_0_8 Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-kafka_0_9-common Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-kafka_0_9 Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-kafka_0_9_mapr_5_1 SDC-10729. MapR Streams standalone and cluster tests failing Jan 11, 2019
sdc-kafka_0_9_mapr_5_2 Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-kafka_1_0 Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-solr-api Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-solr_6 Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdc-solr_cdh_4 Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sdk Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
solr-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
spark-executor-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
spark-processor-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
sso SDC-10218. Add dynamic preview support to existing preview resource Jan 7, 2019
stage-lib-archetype Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
stage-lib-parent Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
stagesupport Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
stats-lib SDC-10218. Add dynamic preview support to existing preview resource Jan 7, 2019
tensorflow-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
testing Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
utils Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
vault-credentialstore-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
wholefile-converter-protolib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
wholefile-transformer-lib SDC-10732. WholeFileTransformer size zero Jan 11, 2019
windows-lib Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
.gitignore SDC-10092. - SDC SSO libraries should allow alternate REST endpoint f… Oct 5, 2018
BUILD.md Updated version to 3.8.0-SNAPSHOT Dec 13, 2018
CONTRIBUTING.md SDC-2077. Fix Contributor License Agreement link in datacollector/CON… Nov 25, 2015
LICENSE.txt SDC-6455: Remove license from license.md file Jun 14, 2017
NOTICE.txt SDC-3184: TCP/UDP/Syslog destination Aug 21, 2018
README.md SDC-10403. Link to build instructions on README Nov 17, 2018
datacollector_splash.png SDC-1765. Cleaning up markdown files and testing pull request workflow. Sep 14, 2015
dependency-check-suppression.xml SDC-6435: SDC source code incorrectly says it's licensed to the ASF Jun 13, 2017
pom.xml SDC-10018. Add AWS Secrets Manager as an option for Credential Stores Jan 17, 2019

README.md

What is StreamSets Data Collector?

StreamSets Data Collector is an enterprise grade, open source, continuous big data ingestion infrastructure. It has an advanced and easy to use User Interface that lets data scientists, developers and data infrastructure teams easily create data pipelines in a fraction of the time typically required to create complex ingest scenarios. Out of the box, StreamSets Data Collector reads from and writes to a large number of end-points, including S3, JDBC, Hadoop, Kafka, Cassandra and many others. You can use Python, Javascript and Java Expression Language in addition to a large number of pre-built stages to transform and process the data on the fly. For fault tolerance and scale out, you can setup data pipelines in cluster mode and perform fine grained monitoring at every stage of the pipeline.

To learn more, check out http://streamsets.com

Building StreamSets Data Collector

To build the StreamSets Data Collector from source code, click here for details.

License

StreamSets Data Collector is built on open source technologies, our code is licensed with the Apache License 2.0.

Getting Help

A good place to start is to check out http://streamsets.com/community. On that page you will find all the ways you can reach us and channels our team monitors. You can post questions on Google Groups sdc-user or on StackExchange using the tag #StreamSets. Post bugs at http://issues.streamsets.com or tweet at us with #StreamSets.

If you need help with production systems, you can check out the variety of support options offered on our support page.

Contributing Code

We welcome contributors, please check out our guidelines to get started.

Changelog

See the latest changelog