Skip to content

nh43de/DataToolChain

Repository files navigation

DataToolChain - Utilities for Data 🆕

For people that deal with data. A collection of utility applications that use DataPowerTools to do some cool stuff. (And more coming soon!!)

Installing Data Toolchain

DataToolChain is built with WPF on .NET 7. You can download and run the latest binaries from the Releases page. You may need to install the .NET 7 Runtimes available from Microsoft's website.

Tools Included

I. DataUploader

Upload data from multiple sources, including Excel, CSV, and more to SQL server. With data-fitting transformations and better error messages.

Home Page

✔️ Using DataUploader to Upload Excel & CSV Data

  1. Enter SQL Server credentials and database information
  2. Select files to upload by clicking "Browse..."
  3. Set options or leave them as default
  4. Click "GO"!

Data Uploader Options

Destination Server and database, username and password

Destination MS SQL Server server, database, and credentials (if using SQL server authentication). Cloud auth coming soon.

Configurable Destination Table

If entered in left-hand panel, all data will upload to this table. Alternatively can leave this blank and upload to specific tables per file.

If no table names are specified, data will upload to table of the same name as the file name minus extenson.

JSON Configuration

You can easily copy/paste JSON to allow for saving and storing uploader configuration.

🔢 Advanced Configuration
  • Apply Default Transform Group - gathers SQL schema information and attempts to transform any dates, numbers, bit values e.g. (Y/N), so that it will be able to upload to SQL server.
  • Truncate Tables First - clears data from all tables used by executing a truncate table command.
  • Use Ordinals - Normally the uploader will attempt to match by column name but use ordinals will override to upload based on column index instead.
  • Bulk Copy Rows per batch - Number of rows per batch to use for SQLBulkCopy protocol.

II. Excel Formula Extractor 🆕

Extracts formulas from given excel sheets.

✔️ Using Excel Formula Extractor

  1. Select files to parse
  2. Click "GO"!
  3. Output is tab-separated and can be copied directly into Excel

Excel Formula Extractor

III. Excel vLookup Mover

This tool moves all vLookups in an Excel workbook into a sheet called "__data". This sheet can then be pasted as values to avoid performance issues with vLookups.

IV. DbStringer

A collection of handy string operations. String together IN statements in SQL, or quickly sort or format items on the fly.

DbStringer

List of String Manipulation Features (DbStringer)
  • Comma-separated list to vertical list
  • Distinct
  • Escape Regex
  • Escape SQL String
  • Format JSON
  • Generalize Regex - use ^^^ to enclose groups, and ___ to indicate a wildcard
  • NULLIF
  • Params to Tabs
  • Smart NULLIF
  • Sort
  • Sort by Length
  • SQL Columns to DECLARE statement
  • Sql Input Params Into Declarations
  • Tabs to params
  • Tabs to rows
  • Tabs to SQL Columns
  • Trim
  • Unescape regex
  • Vertical list to comma-separated list
  • Vertical list to double quoted, comma-separated list
  • Vertical list to single quoted, comma-separated list
  • Vertical list to SQL literal string
  • Vertical list to SQL UNION ALL
  • Vertical list to SQL UNIONS
  • Parse hours - parses any hour ranges in the text i.e. 1:30am-3pm. Great for parsing time sheets.
  • NEW! UnPivot table CSV - unpivots 2-axis matrices into a 3-column table
  • NEW! Create Table from CSV - fits csv data into an appropriate CREATE TABLE sql statement
  • NEW! Generate SQL inserts from CSV - generates SQL insert statements from csv data
  • NEW! Generate JSON from CSV - fits csv data into a json object
  • NEW! Generate SQL inserts from JSON - generates SQL insert statements from an array of json objects
  • NEW! Generate CSV from JSON

V. Sql Query to .csv

A high performance streaming implementation of outputting a SQL query to a comma-separated values (.csv) file using DataPowerTools. Great for outputting millions or billions of rows to a csv file without using SSMS. Also faster than SSMS or any other tool we've tested.

Sql Query To Csv

VI. JUST Transformer Playground

Experiment with JUST transforms for querying and transforming JSON data (similar to XSLT).

More information here: https://github.com/WorkMaze/JUST.net

About

Using data power tools to do some cool stuff.

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages