forked from ipfs/go-pinning-service-http-client
/
client.go
498 lines (429 loc) · 18.9 KB
/
client.go
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
/*
* IPFS Pinning Service API
*
* ## About this spec The IPFS Pinning Service API is intended to be an implementation-agnostic API: - For use and implementation by pinning service providers - For use in client mode by IPFS nodes and GUI-based applications > **Note**: while ready for implementation, this spec is still a work in progress! 🏗️ **Your input and feedback are welcome and valuable as we develop this API spec. Please join the design discussion at [github.com/ipfs/pinning-services-api-spec](https://github.com/ipfs/pinning-services-api-spec).** # Schemas This section describes the most important object types and conventions. A full list of fields and schemas can be found in the `schemas` section of the [YAML file](https://github.com/ipfs/pinning-services-api-spec/blob/master/ipfs-pinning-service.yaml). ## Identifiers ### cid [Content Identifier (CID)](https://docs.ipfs.io/concepts/content-addressing/) points at the root of a DAG that is pinned recursively. ### requestid Unique identifier of a pin request. When a pin is created, the service responds with unique `requestid` that can be later used for pin removal. When the same `cid` is pinned again, a different `requestid` is returned to differentiate between those pin requests. Service implementation should use UUID, `hash(accessToken,Pin,PinStatus.created)`, or any other opaque identifier that provides equally strong protection against race conditions. ## Objects ### Pin object ![pin object](https://bafybeideck2fchyxna4wqwc2mo67yriokehw3yujboc5redjdaajrk2fjq.ipfs.dweb.link/pin.png) The `Pin` object is a representation of a pin request. It includes the `cid` of data to be pinned, as well as optional metadata in `name`, `origins`, and `meta`. ### Pin status response ![pin status response object](https://bafybeideck2fchyxna4wqwc2mo67yriokehw3yujboc5redjdaajrk2fjq.ipfs.dweb.link/pinstatus.png) The `PinStatus` object is a representation of the current state of a pinning operation. It includes the original `pin` object, along with the current `status` and globally unique `requestid` of the entire pinning request, which can be used for future status checks and management. Addresses in the `delegates` array are peers delegated by the pinning service for facilitating direct file transfers (more details in the provider hints section). Any additional vendor-specific information is returned in optional `info`. ## The pin lifecycle ![pinning service objects and lifecycle](https://bafybeideck2fchyxna4wqwc2mo67yriokehw3yujboc5redjdaajrk2fjq.ipfs.dweb.link/lifecycle.png) ### Creating a new pin object The user sends a `Pin` object to `POST /pins` and receives a `PinStatus` response: - `requestid` in `PinStatus` is the identifier of the pin operation, which can can be used for checking status, and removing the pin in the future - `status` in `PinStatus` indicates the current state of a pin ### Checking status of in-progress pinning `status` (in `PinStatus`) may indicate a pending state (`queued` or `pinning`). This means the data behind `Pin.cid` was not found on the pinning service and is being fetched from the IPFS network at large, which may take time. In this case, the user can periodically check pinning progress via `GET /pins/{requestid}` until pinning is successful, or the user decides to remove the pending pin. ### Replacing an existing pin object The user can replace an existing pin object via `POST /pins/{requestid}`. This is a shortcut for removing a pin object identified by `requestid` and creating a new one in a single API call that protects against undesired garbage collection of blocks common to both pins. Useful when updating a pin representing a huge dataset where most of blocks did not change. The new pin object `requestid` is returned in the `PinStatus` response. The old pin object is deleted automatically. ### Removing a pin object A pin object can be removed via `DELETE /pins/{requestid}`. ## Provider hints Pinning of new data can be accelerated by providing a list of known data sources in `Pin.origins`, and connecting at least one of them to pinning service nodes at `PinStatus.delegates`. The most common scenario is a client putting its own IPFS node's multiaddrs in `Pin.origins`, and then directly connecting to every multiaddr returned by a pinning service in `PinStatus.delegates` to initiate transfer. This ensures data transfer starts immediately (without waiting for provider discovery over DHT), and direct dial from a client works around peer routing issues in restrictive network topologies such as NATs. ## Custom metadata Pinning services are encouraged to add support for additional features by leveraging the optional `Pin.meta` and `PinStatus.info` fields. While these attributes can be application- or vendor-specific, we encourage the community at large to leverage these attributes as a sandbox to come up with conventions that could become part of future revisions of this API. ### Pin metadata String keys and values passed in `Pin.meta` are persisted with the pin object. Potential uses: - `Pin.meta[app_id]`: Attaching a unique identifier to pins created by an app enables filtering pins per app via `?meta={\"app_id\":<UUID>}` - `Pin.meta[vendor_policy]`: Vendor-specific policy (for example: which region to use, how many copies to keep) Note that it is OK for a client to omit or ignore these optional attributes; doing so should not impact the basic pinning functionality. ### Pin status info Additional `PinStatus.info` can be returned by pinning service. Potential uses: - `PinStatus.info[status_details]`: more info about the current status (queue position, percentage of transferred data, summary of where data is stored, etc); when `PinStatus.status=failed`, it could provide a reason why a pin operation failed (e.g. lack of funds, DAG too big, etc.) - `PinStatus.info[dag_size]`: the size of pinned data, along with DAG overhead - `PinStatus.info[raw_size]`: the size of data without DAG overhead (eg. unixfs) - `PinStatus.info[pinned_until]`: if vendor supports time-bound pins, this could indicate when the pin will expire # Pagination and filtering Pin objects can be listed by executing `GET /pins` with optional parameters: - When no filters are provided, the endpoint will return a small batch of the 10 most recently created items, from the latest to the oldest. - The number of returned items can be adjusted with the `limit` parameter (implicit default is 10). - If the value in `PinResults.count` is bigger than the length of `PinResults.results`, the client can infer there are more results that can be queried. - To read more items, pass the `before` filter with the timestamp from `PinStatus.created` found in the oldest item in the current batch of results. Repeat to read all results. - Returned results can be fine-tuned by applying optional `after`, `cid`, `name`, `status`, or `meta` filters. > **Note**: pagination by the `created` timestamp requires each value to be globally unique. Any future considerations to add support for bulk creation must account for this.
*
* API version: 0.1.1
* Generated by: OpenAPI Generator (https://openapi-generator.tech)
*/
package openapi
import (
"bytes"
"context"
"encoding/json"
"encoding/xml"
"errors"
"fmt"
"io"
"log"
"mime/multipart"
"net/http"
"net/http/httputil"
"net/url"
"os"
"path/filepath"
"reflect"
"regexp"
"strings"
"time"
"golang.org/x/oauth2"
)
var (
jsonCheck = regexp.MustCompile(`(?i:(?:application|text)/(?:vnd\.[^;]+\+)?json)`)
xmlCheck = regexp.MustCompile(`(?i:(?:application|text)/xml)`)
)
// APIClient manages communication with the IPFS Pinning Service API API v0.1.1
// In most cases there should be only one, shared, APIClient.
type APIClient struct {
cfg *Configuration
common service // Reuse a single struct instead of allocating one for each service on the heap.
// API Services
PinsApi *PinsApiService
}
type service struct {
client *APIClient
}
// NewAPIClient creates a new API client. Requires a userAgent string describing your application.
// optionally a custom http.Client to allow for advanced features such as caching.
func NewAPIClient(cfg *Configuration) *APIClient {
if cfg.HTTPClient == nil {
cfg.HTTPClient = http.DefaultClient
}
c := &APIClient{}
c.cfg = cfg
c.common.client = c
// API Services
c.PinsApi = (*PinsApiService)(&c.common)
return c
}
// selectHeaderContentType select a content type from the available list.
func selectHeaderContentType(contentTypes []string) string {
if len(contentTypes) == 0 {
return ""
}
if contains(contentTypes, "application/json") {
return "application/json"
}
return contentTypes[0] // use the first content type specified in 'consumes'
}
// selectHeaderAccept join all accept types and return
func selectHeaderAccept(accepts []string) string {
if len(accepts) == 0 {
return ""
}
if contains(accepts, "application/json") {
return "application/json"
}
return strings.Join(accepts, ",")
}
// contains is a case insenstive match, finding needle in a haystack
func contains(haystack []string, needle string) bool {
for _, a := range haystack {
if strings.EqualFold(a, needle) {
return true
}
}
return false
}
// parameterToString convert interface{} parameters to string, using a delimiter if format is provided.
func parameterToString(obj interface{}, collectionFormat string) string {
var delimiter string
switch collectionFormat {
case "pipes":
delimiter = "|"
case "ssv":
delimiter = " "
case "tsv":
delimiter = "\t"
case "csv":
delimiter = ","
}
if reflect.TypeOf(obj).Kind() == reflect.Slice {
return strings.Trim(strings.Replace(fmt.Sprint(obj), " ", delimiter, -1), "[]")
} else if t, ok := obj.(time.Time); ok {
return t.Format(time.RFC3339Nano)
}
return fmt.Sprintf("%v", obj)
}
// callAPI do the request.
func (c *APIClient) callAPI(request *http.Request) (*http.Response, error) {
if c.cfg.Debug {
dump, err := httputil.DumpRequestOut(request, true)
if err != nil {
return nil, err
}
log.Printf("\n%s\n", string(dump))
}
resp, err := c.cfg.HTTPClient.Do(request)
if err != nil {
return resp, err
}
if c.cfg.Debug {
dump, err := httputil.DumpResponse(resp, true)
if err != nil {
return resp, err
}
log.Printf("\n%s\n", string(dump))
}
return resp, err
}
// Allow modification of underlying config for alternate implementations and testing
// Caution: modifying the configuration while live can cause data races and potentially unwanted behavior
func (c *APIClient) GetConfig() *Configuration {
return c.cfg
}
// prepareRequest build the request
func (c *APIClient) prepareRequest(
ctx context.Context,
path string, method string,
postBody interface{},
headerParams map[string]string,
queryParams url.Values,
formParams url.Values,
formFileName string,
fileName string,
fileBytes []byte) (localVarRequest *http.Request, err error) {
var body *bytes.Buffer
// Detect postBody type and post.
if postBody != nil {
contentType := headerParams["Content-Type"]
if contentType == "" {
contentType = detectContentType(postBody)
headerParams["Content-Type"] = contentType
}
body, err = setBody(postBody, contentType)
if err != nil {
return nil, err
}
}
// add form parameters and file if available.
if strings.HasPrefix(headerParams["Content-Type"], "multipart/form-data") && len(formParams) > 0 || (len(fileBytes) > 0 && fileName != "") {
if body != nil {
return nil, errors.New("cannot specify postBody and multipart form at the same time")
}
body = &bytes.Buffer{}
w := multipart.NewWriter(body)
for k, v := range formParams {
for _, iv := range v {
if strings.HasPrefix(k, "@") { // file
err = addFile(w, k[1:], iv)
if err != nil {
return nil, err
}
} else { // form value
w.WriteField(k, iv)
}
}
}
if len(fileBytes) > 0 && fileName != "" {
w.Boundary()
//_, fileNm := filepath.Split(fileName)
part, err := w.CreateFormFile(formFileName, filepath.Base(fileName))
if err != nil {
return nil, err
}
_, err = part.Write(fileBytes)
if err != nil {
return nil, err
}
}
// Set the Boundary in the Content-Type
headerParams["Content-Type"] = w.FormDataContentType()
// Set Content-Length
headerParams["Content-Length"] = fmt.Sprintf("%d", body.Len())
w.Close()
}
if strings.HasPrefix(headerParams["Content-Type"], "application/x-www-form-urlencoded") && len(formParams) > 0 {
if body != nil {
return nil, errors.New("cannot specify postBody and x-www-form-urlencoded form at the same time")
}
body = &bytes.Buffer{}
body.WriteString(formParams.Encode())
// Set Content-Length
headerParams["Content-Length"] = fmt.Sprintf("%d", body.Len())
}
// Setup path and query parameters
url, err := url.Parse(path)
if err != nil {
return nil, err
}
// Override request host, if applicable
if c.cfg.Host != "" {
url.Host = c.cfg.Host
}
// Override request scheme, if applicable
if c.cfg.Scheme != "" {
url.Scheme = c.cfg.Scheme
}
// Adding Query Param
query := url.Query()
for k, v := range queryParams {
for _, iv := range v {
query.Add(k, iv)
}
}
// Encode the parameters.
url.RawQuery = query.Encode()
// Generate a new request
if body != nil {
localVarRequest, err = http.NewRequest(method, url.String(), body)
} else {
localVarRequest, err = http.NewRequest(method, url.String(), nil)
}
if err != nil {
return nil, err
}
// add header parameters, if any
if len(headerParams) > 0 {
headers := http.Header{}
for h, v := range headerParams {
headers.Set(h, v)
}
localVarRequest.Header = headers
}
// Add the user agent to the request.
localVarRequest.Header.Add("User-Agent", c.cfg.UserAgent)
if ctx != nil {
// add context to the request
localVarRequest = localVarRequest.WithContext(ctx)
// Walk through any authentication.
// OAuth2 authentication
if tok, ok := ctx.Value(ContextOAuth2).(oauth2.TokenSource); ok {
// We were able to grab an oauth2 token from the context
var latestToken *oauth2.Token
if latestToken, err = tok.Token(); err != nil {
return nil, err
}
latestToken.SetAuthHeader(localVarRequest)
}
// Basic HTTP Authentication
if auth, ok := ctx.Value(ContextBasicAuth).(BasicAuth); ok {
localVarRequest.SetBasicAuth(auth.UserName, auth.Password)
}
// AccessToken Authentication
if auth, ok := ctx.Value(ContextAccessToken).(string); ok {
localVarRequest.Header.Add("Authorization", "Bearer "+auth)
}
}
for header, value := range c.cfg.DefaultHeader {
localVarRequest.Header.Add(header, value)
}
return localVarRequest, nil
}
func (c *APIClient) decode(v interface{}, b []byte, contentType string) (err error) {
if len(b) == 0 {
return nil
}
if s, ok := v.(*string); ok {
*s = string(b)
return nil
}
if xmlCheck.MatchString(contentType) {
if err = xml.Unmarshal(b, v); err != nil {
return err
}
return nil
}
if jsonCheck.MatchString(contentType) {
if actualObj, ok := v.(interface{ GetActualInstance() interface{} }); ok { // oneOf, anyOf schemas
if unmarshalObj, ok := actualObj.(interface{ UnmarshalJSON([]byte) error }); ok { // make sure it has UnmarshalJSON defined
if err = unmarshalObj.UnmarshalJSON(b); err != nil {
return err
}
} else {
return errors.New("unknown type with GetActualInstance but no unmarshalObj.UnmarshalJSON defined")
}
} else if err = json.Unmarshal(b, v); err != nil { // simple model
return err
}
return nil
}
return errors.New("undefined response type")
}
// Add a file to the multipart request
func addFile(w *multipart.Writer, fieldName, path string) error {
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
part, err := w.CreateFormFile(fieldName, filepath.Base(path))
if err != nil {
return err
}
_, err = io.Copy(part, file)
return err
}
// Prevent trying to import "fmt"
func reportError(format string, a ...interface{}) error {
return fmt.Errorf(format, a...)
}
// Set request body from an interface{}
func setBody(body interface{}, contentType string) (bodyBuf *bytes.Buffer, err error) {
if bodyBuf == nil {
bodyBuf = &bytes.Buffer{}
}
if reader, ok := body.(io.Reader); ok {
_, err = bodyBuf.ReadFrom(reader)
} else if b, ok := body.([]byte); ok {
_, err = bodyBuf.Write(b)
} else if s, ok := body.(string); ok {
_, err = bodyBuf.WriteString(s)
} else if s, ok := body.(*string); ok {
_, err = bodyBuf.WriteString(*s)
} else if jsonCheck.MatchString(contentType) {
err = json.NewEncoder(bodyBuf).Encode(body)
} else if xmlCheck.MatchString(contentType) {
err = xml.NewEncoder(bodyBuf).Encode(body)
}
if err != nil {
return nil, err
}
if bodyBuf.Len() == 0 {
err = fmt.Errorf("invalid body type %s", contentType)
return nil, err
}
return bodyBuf, nil
}
// detectContentType method is used to figure out `Request.Body` content type for request header
func detectContentType(body interface{}) string {
contentType := "text/plain; charset=utf-8"
kind := reflect.TypeOf(body).Kind()
switch kind {
case reflect.Struct, reflect.Map, reflect.Ptr:
contentType = "application/json; charset=utf-8"
case reflect.String:
contentType = "text/plain; charset=utf-8"
default:
if b, ok := body.([]byte); ok {
contentType = http.DetectContentType(b)
} else if kind == reflect.Slice {
contentType = "application/json; charset=utf-8"
}
}
return contentType
}
// Ripped from https://github.com/gregjones/httpcache/blob/master/httpcache.go
type cacheControl map[string]string
func parseCacheControl(headers http.Header) cacheControl {
cc := cacheControl{}
ccHeader := headers.Get("Cache-Control")
for _, part := range strings.Split(ccHeader, ",") {
part = strings.Trim(part, " ")
if part == "" {
continue
}
if strings.ContainsRune(part, '=') {
keyval := strings.Split(part, "=")
cc[strings.Trim(keyval[0], " ")] = strings.Trim(keyval[1], ",")
} else {
cc[part] = ""
}
}
return cc
}
// CacheExpires helper function to determine remaining time before repeating a request.
func CacheExpires(r *http.Response) time.Time {
// Figure out when the cache expires.
var expires time.Time
now, err := time.Parse(time.RFC1123, r.Header.Get("date"))
if err != nil {
return time.Now()
}
respCacheControl := parseCacheControl(r.Header)
if maxAge, ok := respCacheControl["max-age"]; ok {
lifetime, err := time.ParseDuration(maxAge + "s")
if err != nil {
expires = now
} else {
expires = now.Add(lifetime)
}
} else {
expiresHeader := r.Header.Get("Expires")
if expiresHeader != "" {
expires, err = time.Parse(time.RFC1123, expiresHeader)
if err != nil {
expires = now
}
}
}
return expires
}
// GenericOpenAPIError Provides access to the body, error and model on returned errors.
type GenericOpenAPIError struct {
body []byte
error string
model interface{}
}
// Error returns non-empty string if there was an error.
func (e GenericOpenAPIError) Error() string {
return e.error
}
// Body returns the raw bytes of the response
func (e GenericOpenAPIError) Body() []byte {
return e.body
}
// Model returns the unpacked model of the error
func (e GenericOpenAPIError) Model() interface{} {
return e.model
}