Reading from a stream

High-level method parse

Here is an example to read a file or stream containing multiple protobuf messages using parse method. In this example, the input file contains multiple Alignment messages defined in vg.proto. But the input file can contain any other protobuf messages of the same type and. The file name or file object and the message type should be provided as an argument to parse method:

import stream
import vg_pb2  # or any other compiled protobuf module

# parse by file name
for message in stream.parse('test.gam', vg_pb2.Alignment):
   # work with `message`
   ...
# parse by file object
for message in stream.parse(f, vg_pb2.Alignment):
   # work with `message`
   ...

Low-level method open

open method opens a stream and returns an iterable Stream object. Iterating over Stream object yields the message raw data without parsing. It can be useful when different types of messages are present in the file/stream. Otherwise, parse is recommended.

import stream
import vg_pb2  # or any other compiled protobuf module

# open by file name
with stream.open('test.gam', 'rb') as istream:
    for data in istream:
        message = vg_pb2.Alignment()
        message.ParseFromString(data)
        # work with message

# open by file object
# NOTE that file-like object `f` is passed as a keyword argument `fileobj`
with stream.open(fileobj=f, 'rb') as istream:
    for data in istream:
        message = vg_pb2.Alignment()
        message.ParseFromString(data)
        # work with message

---

NOTE

The stream can be closed by calling close method explicitly, especially when Stream is opened without using context management (with statement).

---