Skip to content

Commit

Permalink
v1.2.0
Browse files Browse the repository at this point in the history
  • Loading branch information
roll committed Oct 4, 2017
1 parent 8f65286 commit ca2ce68
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 16 deletions.
15 changes: 13 additions & 2 deletions README.md
Expand Up @@ -175,10 +175,11 @@ It was onle basic introduction to the `Table` class. To learn more let's take a

Factory method to instantiate `Table` class. This method is async and it should be used with await keyword or as a `Promise`. If `references` argument is provided foreign keys will be checked on any reading operation.

- `source (String/Array[]/Function)` - data source (one of):
- `source (String/Array[]/Stream/Function)` - data source (one of):
- local CSV file (path)
- remote CSV file (url)
- array of arrays representing the rows
- readable stream with CSV file contents
- function returning readable stream with CSV file contents
- `schema (Object)` - data schema in all forms supported by `Schema` class
- `strict (Boolean)` - strictness option to pass to `Schema` constructor
Expand Down Expand Up @@ -597,7 +598,12 @@ The `descriptor` variable is now a JSON object:

This funcion is async so it has to be used with `await` keyword or as a `Promise`.

- `source (String/Array[]/Stream)` - data source
- `source (String/Array[]/Stream/Function)` - data source (one of):
- local CSV file (path)
- remote CSV file (url)
- array of arrays representing the rows
- readable stream with CSV file contents
- function returning readable stream with CSV file contents
- `headers (String[])` - array of headers
- `options (Object)` - any `Table.load` options
- `(errors.TableSchemaError)` - raises any error occured in the process
Expand Down Expand Up @@ -636,6 +642,11 @@ $ npm run build

Here described only breaking and the most important changes. The full changelog and documentation for all released versions could be found in nicely formatted [commit history](https://github.com/frictionlessdata/tableschema-js/commits/master).

### v1.2

New API added:
- `Table.load` and `infer` now accept Node Stream as a `source` argument

### v1.1

New API added:
Expand Down
2 changes: 1 addition & 1 deletion package.json
@@ -1,6 +1,6 @@
{
"name": "tableschema",
"version": "1.1.1",
"version": "1.2.0",
"description": "A library for working with Table Schema in Javascript.",
"license": "MIT",
"main": "lib/index.js",
Expand Down
28 changes: 15 additions & 13 deletions src/table.js
Expand Up @@ -53,15 +53,7 @@ class Table {
* https://github.com/frictionlessdata/tableschema-js#table
*/
async iter({keyed, extended, cast=true, relations=false, stream=false}={}) {

// Get row stream
let source = this._source
if (source.readable) {
const duplicateStream = this._source.pipe(new PassThrough())
this._source = duplicateStream.pipe(new PassThrough())
source = duplicateStream.pipe(new PassThrough())
}
const rowStream = await createRowStream(source, this._parserOptions)

// Prepare unique checks
let uniqueFieldsCache = {}
Expand All @@ -71,6 +63,16 @@ class Table {
}
}

// Multiplicate node stream
if (source.readable) {
const duplicateStream = this._source.pipe(new PassThrough())
this._source = duplicateStream.pipe(new PassThrough())
source = duplicateStream.pipe(new PassThrough())
}

// Get row stream
const rowStream = await createRowStream(source, this._parserOptions)

// Get table row stream
let rowNumber = 0
let tableRowStream = rowStream.pipe(csv.transform(row => {
Expand Down Expand Up @@ -228,17 +230,17 @@ async function createRowStream(source, parserOptions) {
stream = source()
stream = stream.pipe(parser)

// Node stream
} else if (source.readable) {
stream = source
stream = stream.pipe(parser)

// Inline source
} else if (isArray(source)) {
stream = new Readable({objectMode: true})
for (const row of source) stream.push(row)
stream.push(null)

// Row stream
} else if (source.readable) {
stream = source
stream = stream.pipe(parser)

// Remote source
} else if (helpers.isRemotePath(source)) {
if (config.IS_BROWSER) {
Expand Down

0 comments on commit ca2ce68

Please sign in to comment.