Permalink
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
127 lines (100 sloc) 5.62 KB

Introduction

Axibase Collector is a Java-based ETL application that queries external data sources on a defined schedule and uploads the data as series, properties, messages, and files into Axibase Time Series Database.

Collector supports data markers to minimize the load on the source systems as well as list-based automation to efficiently copy data from multiple sources with re-usable configurations.

Use Cases

  • Download a CSV/TSV file from a URL or an FTP server and upload the file into ATSD.
  • Offload data from a relational database for long-term storage in ATSD.
  • Historize metrics by querying current values and retaining their history in ATSD.
  • Query a web service, convert JSON document into CSV format, and upload the file into ATSD.
  • Collect performance metrics from Java applications (JMX), network devices (SNMP), etc.
  • Copy data from operational support systems and plant historians for advanced analytics and visualization.

Supported Databases

  • Oracle
  • IBM DB2
  • Microsoft SQL Server
  • MySQL
  • PostgreSQL
  • Sybase
  • Apache Derby
  • Vertica
  • ATSD
  • SAP HANA
  • OSIsoft PI Data Archive
  • OSIsoft PIOleDBEnt
  • Custom

Supported Data and Network Protocols

  • JDBC
  • SNMP
  • JMX
  • ICMP
  • TCP
  • HTTP

Supported File Formats

  • CSV
  • TSV
  • Text
  • JSON

Supported Schemas

Supported APIs

  • Kafka
  • Amazon Web Services CloudWatch
  • Docker Engine
  • HP OVPM (Performance Manager)

Installation

Jobs

Automation

Job Types

Type Description
AWS Collects AWS statistics using CloudWatch API.
Docker Collects container, image, and volume statistics using Docker Engine API.
File Downloads CSV/TSV files from remote servers or local file system and uploads them into ATSD for parsing.
Supports HTTP/s, FTP, SFTP, SCP, and FILE protocols.
HTTP Executes HTTP requests or Web Driver scripts and stores response status, response times, and SSL certificate status as metrics.
ICMP Pings hostnames/IP addresses and stores response status.
JDBC Executes SQL queries against relational databases.
Converts rows into series, property, or message commands.
JMX Collects MBean attribute values from Java applications.
JSON Downloads JSON files, parses the documents and converts JSON fields into series, property, and message commands using JSONPath.
Kafka Reads JSON messages from Kafka brokers, parses the messages and converts them into series, property, and message commands.
OVPM Offloads statistics from HP OpenView Performance Manager.
PI Extracts PI points archive data via JDBC driver.
SNMP Queries SNMP devices using built-in and custom MIB files.
Socrata Downloads JSON documents published in Socrata schema and converts them into series, property, and message commands.
TCP Connects to hostnames/IP addresses and stores connection status.

Administration

Examples

Name Job Type Description
ActiveMQ JMX Collect metrics about brokers, queues, pub/sub topics
Derby Database JMX Collect uptime metrics for the database
File File Download CSV files
HP OpenView OVPM Offload CODA metrics from OVPM
Jetty JMX Collect sessions, requests, status from the Jetty server
JSON JSON Download JSON file and convert the file to CSV format
JVM JMX Collect key JVM performance metrics
MySQL JDBC Collect database performance metrics
nginx File Collect key web server metrics for nginx
nginx-plus File Collect extended web server metrics for nginx-plus
Oracle EM JDBC Offload incremental database and application metrics collected by Oracle EM
PI PI Copy incremental tag values from PI Data Archive
PostgreSQL JDBC Collect database performance metrics
SCOM JDBC Offload incremental server metrics collected by SCOM
Socrata Socrata Download and parse datasets published in Socrata format
SolarWinds JDBC Offload incremental network and server metrics collected by SolarWinds
Tomcat JMX Collect key container metrics exposed by Tomcat
VMware JDBC Offload incremental cluster, host, VM metrics collected by VMware vCenter