@awholegunch awholegunch released this Sep 26, 2014 · 28837 commits to develop since this release

Assets 2

Release Notes

New Features

Ad-hoc querying

  • Capability to write to Datasets using SQL
  • Added a CDAP JDBC driver allowing connections from Java applications and third-party business intelligence tools
  • Ability to perform ad-hoc queries from the CDAP Console:
    • Execute a SQL query from the Console
    • View list of active, completed queries
    • Download query results

Datasets

  • Datasets can be tested with TestBase outside of the context of an Application
  • CDAP now checks Datasets for compatibility in a verification stage
  • The Transaction engine uses server-side filtering for efficient transactional reads
  • Dataset specifications can now be dynamically reconfigured through the use of RESTful endpoints
  • The Bundle jar format is now used for Dataset libs
  • Increments on Datasets are now read-less

Services

  • Added simplified APIs for using Services from other programs such as MapReduce, Flows and Procedures
  • Added an API for creating Services and handlers that can use Datasets transactionally
  • Added a RESTful API to make requests to a Service via the Router

Security

  • Added authorization logging
  • Added Kerberos authentication to Zookeeper secret keys
  • Added support for SSL

Spark Integration

  • Supports running Spark programs as a part of CDAP applications in Standalone mode
  • Supports running Spark programs written with Spark versions 1.0.1 or 1.1.0
  • Supports Spark's MLib and GraphX modules
  • Includes three examples demonstrating CDAP Spark programs
  • Adds display of Spark program logs and history in the CDAP Console

Streams

  • Added a collection of applications, tools and APIs specifically for the ETL (Extract, Transform and Loading) of data
  • Added support for asynchronously writing to Streams

Clients

  • Added a Command-line Interface
  • Added a Java Client Interface

Major CDAP Bug Fixes

  • Fixed a problem with a HADOOP_HOME exception stacktrace when unit-testing an Application
  • Fixed an issue with Hive creating directories in /tmp in the Standalone and unit-test frameworks
  • Fixed a problem with type inconsistency of Service API calls, where numbers were showing up as strings
  • Fixed an issue with the premature expiration of long-term Authentication Tokens
  • Fixed an issue with the Dataset size metric showing data operations size instead of resource usage

Known Issues

  • Metrics for MapReduce jobs aren't populated on secure Hadoop clusters
  • The metric for the number of cores shown in the Resources view of the CDAP Console will be zero
    unless YARN has been configured to enable virtual cores