Node.js package that manages CSV to JSON conversion supporting complex field mapping
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
modules
test
.gitignore
.travis.yml
LICENSE
README.md
index.js
package.json
yarn.lock

README.md

yap-csv

JavaScript Style Guide npm version Build Status codecov Known Vulnerabilities
The purpose of this module is to support converting a csv (with delimiter support) to an array of json objects.
Using a custom map that can handle both simple fields renames and more complicated fields that are a result of calculations

Usage

Add to project

yarn add @nielsen-oss/yap-csv

Structure

The module exposes two variables a Parser and ConverterConsts.
One is used for the actual parsing process and one provide consts helpers.

Simple Usage

const {Parser} = require('yap-csv')
// parserPromise is a Promise
const parserPromise = Parser.parseSingle(FILE_NAME, FILE_LOCATION, DELIMITER, FIELDS_MAP, EXTRA_FIELDS_MAP)
parserPromise
.then((formattedData) => {
  // Do what you want with formattedData
})
.catch((e) => {
  console.error(e.stack)
})

Stream Usage

const {Parser} = require('yap-csv')
const formattedData = []
// parserStream is a stream
const parserStream = Parser.parseSingleStream(FILE_NAME, FILE_LOCATION, DELIMITER, FIELDS_MAP, EXTRA_FIELDS_MAP)
parserStream.on('error', (e) => {
  console.error(e.stack)
})
parserStream.on('data', (chunk) => {
  formattedData.push(chunk)
})
parserStream.on('end', () => {
  // Do what you want with formattedData
})

Fields Specification

  • FILE_NAME - The name of the file being parsed
  • FILE_LOCATION - The location of the file being parsed
  • DELIMITER - The delimiter of the text
  • FIELDS_MAP - A conversion map of the fields
  • EXTRA_FIELDS_MAP - A conversion map for new fields that you would like to add

FIELDS_MAP

The following is an example of a map

const convertConsts = require('yap-csv').ConverterConsts

const FIELDS_MAP = {
  'FIELD1': 'simple_name_change',
  'FIELD2': {
    name: 'my_custom_number',
    type: convertConsts.NUMBER_TYPE
  },
  'FIELD3': {
    name: 'functional_change',
    type: (dataObj) => {
      return dataObj['FIELD3'].toLowerCase()
    }
  }
}

As you can see using the fields map you can:

  • Preform simple field name changes
  • Convert to the helper types that are provided in this module
  • Preform conversions using a specified function that receives the original data object (before conversion)

EXTRA_FIELDS_MAP

The following is an example of a map of extra fields

const EXTRA_FIELDS_MAP = {
  complicatedField: (convertedDataObj) => {
    return convertedDataObj['simple_name_change']
  }
}

As you can see using the fields map you can add fields that are not in the original data.
Each field should be a function that receives that data after it passed the original formatting using the FIELDS_MAP.

ConverterConsts

The exported consts are:

  • NUMBER_TYPE
  • NUMBER_DOUBLE_TYPE

According the the types we parse the results to a parseInt or parseFloat