/
doc.go
169 lines (131 loc) · 6.08 KB
/
doc.go
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
// Copyright 2017 The Upspin Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
/*
Package rpc provides a framework for implementing RPC servers and clients.
RPC wire protocol
The protocol for a particular method is to send an HTTP request with the
appropriate request message to an "api" URL with the server type and method
name; the response will then be returned. Both request and response are
sent as binary protocol buffers, as declared by upspin.io/upspin/proto. The
encoding procedure is described in more detail below.
As an example, to call the Put method of the Store server running at
store.example.com, send to the URL,
https://store.example.com/api/Store/Put
a POST request with, as payload, an encoded StorePutRequest. The response
will be a StorePutResponse. There is such a pair of protocol buffer
messages for each method.
For streaming RPC methods, the requests are the same, but the response is
sent as the bytes "OK" followed by a series of encoded protocol buffers.
Each encoded message is preceded by a four byte, big-endian-encoded int32
that describes the length of the following encoded protocol buffer. The
stream is considered closed when the HTTP response stream ends.
If an error occurs while processing a request, the server returns a 500
Internal Server Error status code and the response body contains the error
string.
Authentication
The client authenticates itself to the server using special HTTP headers.
In its first request to a given Upspin server, the client presents a signed
authentication request as a series of HTTP headers with key
'Upspin-Auth-Request'. The header values are, in order:
the user name,
the server host name,
the current time, and
the R part of an upspin.Signature,
the S part of an upspin.Signature.
The current time is formatted using Go's time.ANSIC presentation:
"Mon Jan _2 15:04:05 2006"
To generate the upspin.Signature, the client concatenates the user name, host
name, and formatted time, each string prefixed by its length as a big
endian-encoded uint32:
[len(username)][username][len(hostname)][hostname][len(time)][time]
The client then SHA256-hashes that string and signs it using its Factotum.
The server checks the signature and, if valid, returns an authentication token
that represents the current session in the 'Upspin-Auth-Token' response header.
In subsequent requests, the client presents that authentication token to the
server using the 'Upspin-Auth-Token' request header.
If there is an error validating an authentication request or token, the server
returns an error message in the 'Upspin-Auth-Error' response header.
TODO: document the 'Upspin-Proxy-Request' header.
Encoding
The arguments to each method, and the returned values, are encoded as
protocol buffers at the top level. These protocol buffers are declared in
the package upspin.io/upspin/proto and its .proto definition file,
upspin.io/upspin/proto/upspin.proto.
Unlike some other systems that use protocol buffers, Upspin does not use
the types defined in upspin.proto internally. The protocol buffer types are
used for transport only, and so the internal types must be transcoded for
use on the wire.
Some of the transcoding is straightforward. The scalar types Transport,
NetAddr, Reference, UserName, PathName, and so on are transmitted as their
underlying scalar types, int or string. (Strings are considered scalars
here as they are basic types in both Go and protocol buffers.)
The structured types Endpoint, Location, Refdata and User themselves
contain only scalar types and so are converted field by field. There are
helper functions in the upspin.io/upspin/proto package to automate the
conversion of these types. For instance, the function proto.UpspinLocations
converts from a slice of protocol buffer Location structs to a slice of the
internal type, upspin.Location.
The DirEntry type and its component DirBlock are treated a little
differently. In the upspin.io/upspin package, which defines these types,
there are Marshal and Unmarshal methods that encode and decode these types
to byte slices. (The details of these encodings are listed below.) Thus
within the protocol buffer types, these appear as proto type "bytes".
These types are handled differently so the DirServer can serialize them
easily for storage.
The errors.Error type is also handled specially so its full functionality
can be accessed across the wire. Methods defined in the errors package do
the marshaling and unmarshaling of Error values.
When these types appear in other protocol buffer types, such as
proto.Event, they are encoded as the byte slices generated by the process
described below.
How to Marshal
The fields of a structured type that marshals to a byte slice (one of
DirBlock, DirEntry, or Error) are encoded as follows using the methods of
the Go package encoding/binary to marshal their underlying type:
int:
varint
string:
varint count n followed by n bytes
byte slice:
varint count n followed by n bytes
Encoding of DirBlock
The fields of a DirBlock are encoded using the rules above in the following
order:
Location.Endpoint.Transport: 1 byte
Location.Endpoint.NetAddr: as string
Location.Reference: as string
Offset: as int
Size: as int
Packdata: as bytes
Encoding of DirEntry
The fields of a DirEntry are encoded using the rules above in the following
order:
SignedName: as string
Packing: 1 byte
Time: as int
Blocks:
len(Blocks): as int
Blocks: n DirBlocks as described above.
Packdata: as bytes
Link: as string
Writer: as string
Name: if different from SignedName, as string
Otherwise, 1 byte with value 0.
Attr: 1 byte
Sequence: as int
Encoding of Error
The fields of an Error are encoded using the rules above in the following
order.
Path: as string
User: as string
Op: as string
Kind: as int
Error:
If of type Error, as described here.
Otherwise the value of Error.Error() as string.
The Error value is always marshaled by address (*Error); if the pointer is
nil, nothing is marshaled. Similarly, for a nil Error.Err field, nothing is
marshaled.
*/
package rpc