Skip to content
This repository was archived by the owner on May 18, 2026. It is now read-only.
This repository was archived by the owner on May 18, 2026. It is now read-only.

Port away from event based stream #24

Closed

Description

@thejmazz

No way to tell when an event based stream ends. So essentially need to port into a transform stream that takes obo and emits parsed [Term]s. Perhaps throws you header with .on('header') or something.

Best way to do this?

What happens now:

line events emitted, then lines are added into string buffers using some flags and buffers. Checking what the current line is, emit header once, and emit stanza which is piped through stanzaParser, filtered, ndjsonified, etc.

Events was a nice way to separate concerns. So it was good for prototyping. But the output stream not having a defined end is not good. So some way to either force in the end event, or move out of using events. A little bit stumped, so any suggestions would be awesome. Perhaps I should move back to "vanilla" streams instead of wrapping with highland, and build buffers based on current character and flags. If anyone knows any good examples of streaming parsers, that would be good to see too.

Activity

  1. self-assigned this
    on Mar 15, 2016
  2. added this to the 0.2.0 milestone on Mar 15, 2016
  3. bmpvieira commented on Mar 18, 2016

    @bmpvieira
    Member

    @thejmazz did you have a look at https://github.com/bionode/bionode-fasta?
    or https://github.com/mafintosh/csv-parser and https://github.com/maxogden/csv-write-stream @mafintosh and @maxogden are stream experts, maybe ask them on #dat IRC or http://gitter.im/datproject/discussions

    However, maybe the problem is the wrapping with highlandjs, I remember having some issues with it and ended up not using it. Instead, I made a quick and dirty module for some stream operations https://github.com/bmpvieira/tool-stream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions