Skip to content

Giraffe server protocol

Parsa Eskandar edited this page Nov 14, 2026 · 2 revisions

vg giraffe-server loads the Giraffe indexes once, then reads requests from standard input and writes answers to standard output until its input ends. This page describes what a client sends and what it gets back. It is meant to be enough to write a client without reading vg.

Everything is text. Fields are separated by one tab. Every line ends with one newline character (\n, not \r\n).

Overview

A client sends requests. There are two kinds:

  • a read, on one line, to be mapped to the graph; This is the main usage of vg giraffe-server.
  • a SURJECT_WITH_ANCHORS block, on several lines, to place a graph alignment the client already has on one path of the graph.

The server answers every request with exactly one frame: a result frame or an error frame. A frame carries the name of its request, so a client matches answers to requests by name.

client                                  server
------                                  ------
r1<TAB>ACGT...<TAB>IIII...
r2<TAB>ACGT...<TAB>IIII...
PROCESS_BATCH
                                        READ<TAB>r1<TAB>1
                                        r1<TAB>150<TAB>0<TAB>150<TAB>+<TAB>>12>13 ...
                                        READ<TAB>r2<TAB>1
                                        r2<TAB>150<TAB>0<TAB>150<TAB>+<TAB>>40>41 ...

Standard error carries log lines for people. They are not part of the protocol, and a client must not parse them.

Requests

A read

A read is one line with one to four fields:

SEQUENCE
NAME<TAB>SEQUENCE
NAME<TAB>SEQUENCE<TAB>QUALITY
NAME<TAB>SEQUENCE<TAB>QUALITY<TAB>TARGET
Field Rules
NAME Any text without a tab, and not empty. It comes back in the frame. Use a name that no other request waiting for an answer has, or the answers cannot be told apart. A read sent as a bare SEQUENCE is named read_0, read_1, and so on, in the order such reads arrive.
SEQUENCE One or more of A, C, G, T, N (can use lowercases).
QUALITY Base qualities as in FASTQ: one character per base, phred+33, from ! to ~. It is either empty or as long as SEQUENCE. Without qualities, the mapping quality is not capped by base quality, as for FASTA input to vg giraffe.
TARGET The name of a path to surject each alignment of this read onto, exactly as the graph names it. The server must have indexed the path's positions when it started. It indexes the paths given with --surject-target and, when at least one is given, also the reference paths and generic paths of the graph. An empty TARGET is the same as none. With a TARGET, QUALITY may be the empty string between two tabs.

When reads are mapped

The server collects reads into a batch and maps the batch at once. A read gets its answer when its batch is mapped, which happens when:

  • the batch reaches --batch-size reads (256 by default);
  • the client sends a line with only PROCESS_BATCH;
  • a SURJECT_WITH_ANCHORS block arrives;
  • the input ends (client closes the standard input).

A client that waits for answers must therefore send PROCESS_BATCH after its last read. PROCESS_BATCH itself gets no frame. Empty lines are skipped.

A read with a bad SEQUENCE or QUALITY is not put in the batch. The server writes its error frame as soon as it reads the line.

A SURJECT_WITH_ANCHORS block

This request places a graph alignment on a path whose positions the server has not indexed. The client supplies anchors: places on the target path whose offsets it already knows.

SURJECT_WITH_ANCHORS<TAB>NAME<TAB>TARGET<TAB>PATH_LENGTH<TAB>ANCHOR_COUNT
ANCHOR_COUNT anchor lines
one GAF line
Header field Meaning
NAME The name for the frame, as for a read.
TARGET The target path's name. If the graph has no path with this name, the server uses the path that the first anchor lies on. This lets a client name a haplotype loosely, as in HG002#2.
PATH_LENGTH The length of the target path in bases. With 0, the server walks the whole path to measure it, which is slow for a long path. Any other number must be right: the server does not check it.
ANCHOR_COUNT The number of anchor lines that follow. With 0 the server knows no position on the path, and the status is failed.

An anchor is a run of consecutive steps of the target path. An anchor line has six numbers:

Column Meaning
1 GBWT node of the run's first step. A GBWT node is 2 * node ID, plus 1 for the reverse orientation.
2 Offset of that step among the visits in the node's GBWT record. Columns 1 and 2 together are a step handle of a GBWTGraph.
3, 4 The same two numbers for the run's last step. The last step may be the first step.
5 Offset on the target path of the first base of the first step, counted from 0.
6 Offset on the target path of the first base of the last step, counted from 0.

The anchors must be right for the graph the server has loaded. The server checks that a walk along the path from each anchor's first step reaches its last step at the offset in column 6, and that the first anchor lies on TARGET when TARGET is a path name. It answers with an error frame if not. Together the anchors' runs must cover every step of the target path that the surjected alignment touches.

The GAF line is one alignment as this server writes it in a result frame. Its path must be a list of oriented node IDs (>12<13), and it needs the cs tag that the server writes. Tags that the server added may stay on the line.

The server reads all the lines of a block before it answers, also when the block has an error, so the next request starts on a line of its own. The one exception is a header whose ANCHOR_COUNT is not a number, or which does not have five fields: the server cannot tell how long the block is, so each of the block's other lines is then read as a request of its own, and gets an error frame.

Frames

Before the first frame, with --emit-header, the server writes the graph's GAF header lines, if the graph has any. Everything after that is frames.

Result frame

READ<TAB>NAME<TAB>N
N GAF lines

N is followed by a newline, and then come exactly N lines.

  • For a read, there is one line per reported alignment, best first. N is at least 1: a read that does not map gets one line with * for its path.
  • For a SURJECT_WITH_ANCHORS block, N is 1. The line is the GAF line that was sent, with the surjection tags added.

Error frame

ERROR<TAB>NAME<TAB>MESSAGE

An error frame is one line. MESSAGE is text for a person and has no tab in it. NAME is * for a line that could not be read as a request, and then MESSAGE starts with line K:, where K counts input lines from 1.

Order

  • The result frames of reads come out in the order the reads were sent.
  • The error frame of a read with a bad SEQUENCE or QUALITY, and of a line that is not a request, is written when the line is read. It can come before the frames of earlier reads that still wait in the batch.
  • The frame of a SURJECT_WITH_ANCHORS block can come at any time after the block was sent: before or after the frames of later requests, and also between two frames of one batch of reads.
  • A frame is never split. Its lines are not mixed with another frame's.

Errors

  1. A request is one read line or one SURJECT_WITH_ANCHORS block.
  2. Every request gets exactly one frame: a result frame or an error frame.
  3. An error in a request affects only that request. The server goes on.
  4. A line that is neither a request nor PROCESS_BATCH gets an error frame with the name *. Empty lines are skipped.
  5. A frame is written whole.
  6. The server stops when its input ends, after it has answered everything it read, and exits with status 0. Before the first request it also stops, with status 1 and no frame, if its options or indexes are bad. If the server dies, the client sees standard output close, and should treat every request without an answer as failed.

The server gives no sign that it has finished loading. A client that needs to know sends one read and PROCESS_BATCH, and waits for the frame.

Some of the messages:

Message Request
empty SEQUENCE read
SEQUENCE has characters other than A, C, G, T and N read
QUALITY has character code 32 at offset 4, outside phred+33 ('!' to '~') read
QUALITY length (4) does not match SEQUENCE length (10) read
line 32: expected 1-4 tab-separated fields or a known command, got 5 none (*)
line 33: not a known command or a nucleotide sequence none (*)
line 34: empty NAME none (*)
malformed SURJECT_WITH_ANCHORS header (need 5 tab fields, got 4) block
malformed anchor line 606 (need 6 tab-separated numbers) block
GAF path has node 999, which is not in the graph block
the anchors are not on the target path CHM13#0#chrM block

A read that cannot be mapped for another reason, and a surjection that fails inside vg, also get an error frame, with vg's own message.

The GAF lines

Each line is a GAF record as vg giraffe writes it: the twelve GAF columns, then vg's own tags (AS, bq, cs, dv and others). The server adds the tags below after those. Their names are local to this server.

A tag with type i holds an integer, and Z holds text.

Haplotype tags

These say which haplotypes of the graph follow the alignment's path. Every line of a result frame for a read has them.

The server cuts the path into segments. A segment is a run of the path's nodes that at least one haplotype follows without a break, and it is as long as it can be. A path that some haplotype follows from end to end is one segment. A node that no haplotype visits is in no segment. The tags hp to ht describe the longest segment, the one whose nodes have the most bases, and the first such segment if there are several.

A haplotype is named sample#phase#contig, as in HG002#1#chr1. This is shorter than a path name in the graph, which can also have a fragment number: all the fragments of a haplotype share one name here. A generic path, which has no sample, is named by its contig alone, as the graph names it. Where the GBWT has no name for a sample or a contig, its number stands in.

Tag Meaning
hp:Z The distinct names of the haplotypes that follow the longest segment, sorted, with commas between them. Empty if there is no segment.
hn:i The number of names in hp.
hb:Z One haplotype from hp that a client can use when it needs a single one, for example as a surjection target. It is a haplotype of a reference sample, or a generic path, if hp has one, and otherwise the one with the lowest sequence ID in the GBWT, so it is the same on every run. * if there is none.
hf:i 1 if hb is from a reference sample or is a generic path, else 0.
hc:i The number of times GBWT sequences follow the longest segment. A haplotype that follows it twice counts twice, so hc can be more than hn.
ht:i 1 if hp may be incomplete. The server looks up at most 1000 of the hc sequences.
hl:i The node ID of the boundary node before the path, in the path's direction, of the smallest snarl that contains the whole path.
hr:i The same for the boundary node after the path.
hq:i 1 if one segment is the whole path, else 0.
hv:i The share of the path that the longest segment covers, in percent, rounded down. Both lengths are sums of whole node lengths.
hs:i The number of segments.
hm:Z Only when hs is more than 1. One entry per segment, in path order, with ; between entries. An entry is LENGTH:COUNT:NAME: the bases in the segment's nodes, the number of haplotype names of the segment, and the segment's own hb.

hl and hr are each written only when the node was found. To find them the server takes the haplotypes that follow the whole path and walks out from each end while they all go to the same node, until the first node outside the snarl. A side is missing when no haplotype follows the whole path, when no snarl contains the whole path, when the haplotypes part before they leave the snarl, or when the walk takes more than 1000 steps.

An unmapped read has hp:Z: (empty), hb:Z:*, and 0 in the other tags.

Surjection tags

These say where the alignment lies on one path. A line has them when its read named a TARGET, and the line of a SURJECT_WITH_ANCHORS answer has them.

sj:Z is always there and holds the status. The other tags are there only when the status is ok.

sj:Z Meaning
ok The alignment was placed on the target path.
empty The alignment has no path in the graph: the read did not map.
unknown_path For a read: the graph has no path with the target's name. For a block: the name is not a path, and there is no anchor to take the path from.
not_indexed The graph has the path, but the server did not index its positions. A server started without --surject-target indexes no path, and gives this status to every line of a read with a TARGET, also when the read did not map or the name is not a path. Only for reads.
incompatible The target path does not visit, in order, the nodes of the alignment's longest segment, or the alignment has no segment. Only for reads.
failed vg found no alignment on the target path.
Tag Meaning
sn:Z The name of the path, as the graph has it.
sp:i Where the alignment starts on the path: the offset, counted from 0, of its leftmost base on the path's forward strand.
sr:i 1 if the read is on the path's reverse strand, else 0.
ss:i The alignment score on the path.
sm:i The mapping quality. It is the graph alignment's.
sc:Z The CIGAR string against the path's forward strand, as in SAM. It uses M, I, D and S. A match and a mismatch are both M. * if there is none.
sM:i The number of read bases that match the path.
sX:i The number of read bases that differ from the path.
sI:i The number of read bases that are not on the path.
sD:i The number of path bases that are not in the read.

The answer to a SURJECT_WITH_ANCHORS block has two more tags when the status is ok:

Tag Meaning
an:i The number of anchors that were sent.
ap:Z The name of the path the server used. It is the same as sn. It matters when TARGET was not a path name.

Why the surjection is on the GAF line

A surjection belongs to one graph alignment, and the tags keep the two in one record, so a frame still holds one line per alignment. The cost is that a line can report only one placement. When vg places an alignment on the path in several pieces, the server reports the piece with the best score.

An example

Two reads, the second one with a target, and a line that is not a request. The server was started with --surject-target CHM13#0#chrM.

r1<TAB>ACGTACGTAC<TAB>IIIIIIIIII
r2<TAB>ACGTACGTAC<TAB><TAB>CHM13#0#chrM
HELLO
PROCESS_BATCH
ERROR<TAB>*<TAB>line 3: not a known command or a nucleotide sequence
READ<TAB>r1<TAB>1
r1<TAB>10<TAB>0<TAB>10<TAB>+<TAB>>7>8<TAB>...<TAB>hp:Z:CHM13#0#chrM,GRCh38#0#chrM<TAB>hn:i:2<TAB>hb:Z:CHM13#0#chrM<TAB>hf:i:1<TAB>hc:i:2<TAB>ht:i:0<TAB>hq:i:1<TAB>hv:i:100<TAB>hs:i:1
READ<TAB>r2<TAB>1
r2<TAB>10<TAB>0<TAB>10<TAB>+<TAB>>7>8<TAB>...<TAB>hs:i:1<TAB>sj:Z:ok<TAB>sn:Z:CHM13#0#chrM<TAB>sp:i:3577<TAB>sr:i:0<TAB>ss:i:20<TAB>sm:i:60<TAB>sc:Z:10M<TAB>sM:i:10<TAB>sX:i:0<TAB>sI:i:0<TAB>sD:i:0

The error frame comes first here because the server answers a bad line as soon as it reads it, and the two reads only when PROCESS_BATCH arrives. The GAF lines are shortened: ... stands for the columns and tags that are left out.

Clone this wiki locally