OSDQ Desktop V6.3.3

Description

This project is dedicated to open source data quality and data preparation solutions. Data Quality includes profiling, filtering, governance, similarity check, data enrichment/alteration, real time alerting, basket analysis, bubble chart Warehouse validation, single customer view etc. This product is developing high performance integrated data management platform which will seamlessly do Data Integration, Data Profiling, Data Quality, Data Preparation, Dummy Data Creation, Meta Data Discovery, Anomaly Discovery, Reporting and Analytic.

It also had Hadoop ( Big data ) support to move files to/from Hadoop Grid, Create, Load and Profile Hive Tables. This project is also known as "Aggregate Profiler"

Changes

6.3.3 Changes:

a.) Create Rule based on Metric UDF b.) Schedule Metric UDF based Rules c.) Unspecified bug fixes

Features

• Mysql, Oracle,Postgres,Access,Db2,SQL Server certified Big data support - HIVE
• Create Hive table, Profile Hive table, Move file to/from Profiler System and Hadoop Grid
• Fuzzy Logic based similarity check, Cardinailty check between tables and files
• Export and import from XML, XLS or CSV format, PDF export
• File Analysis, Regex search, Standardization, DB search
• Complete DB Scan, SQL interface, Data Dictionary, Schema Comparison
• Statistical Analysis, Reporting ( dimension and measure based), Ad Hoc reports and Analytics
• Pattern Matching , DeDuplication, Case matching, Basket Analysis, Distribution Chart
• Data generation, Data Preparation and Data masking features
• Meta Data Information, Reverse engineering of Data Model
• Timeliness analysis , String length analysis, KMean, Prediction, Regression
• Address Correction, Single View of Customer, Product, Golden merge for records
• Record Match, Linkage and Merge added based on fuzzy logic
• Format Creation, Format Matching ( Phone, Date, String and Number), Format standardization

Getting Started

Using pre-compiled distribution - you can directly download the pre-compiled binaries from sourceforge

1. Unzip the download release file
2. Go to bin directory
3. run runprofiler.[sh][bat]

If you are a developer or you want to build the project yourself, you first need to build and install osdq-core into local maven repository of your machine.

1. git clone https://github.com/arrahtech/osdq-core
2. cd osdq-core
3. mvn install
4. cd ..
5. git clone https://github.com/arrahtech/osdq-desktop
6. cd osdq-desktop
7. mvn exec:java

Intellij Setup

1. Add a maven run configuration
2. Accept the defaults and set the "Command line" parameter as exec:java
3. Run and Debug should be available from Intellij

Disclaimer by one of author (arun-y): I am providing code in the repository to you under an open source license. Because this is my personal repository, the license you receive to my code is from me and not my employer (Facebook)