Skip to content
This repository has been archived by the owner on Feb 27, 2023. It is now read-only.

Latest commit

 

History

History
1218 lines (800 loc) · 45.6 KB

apis.md

File metadata and controls

1218 lines (800 loc) · 45.6 KB

Dragonfly SuperNode API

Overview

API is an HTTP API served by Dragonfly's SuperNode. It is the API dfget or Harbor uses to communicate with the supernode.

Version information

Version : 0.1

URI scheme

Schemes : HTTP, HTTPS

Tags

  • Peer : Create and manage peer nodes in peer networks.
  • Piece : create and manage image/file pieces in supernode.
  • PreheatTask : Create and manage image or file preheat task in supernode.
  • Task : create and manage image/file distribution task in supernode.

Consumes

  • application/json
  • text/plain

Produces

  • application/json
  • text/plain

Paths

Ping

GET /_ping

Description

This is a dummy endpoint you can use to test if the server is accessible.

Responses

HTTP Code Description Schema
200 no error string
500 An unexpected server error occurred. Error

Example HTTP response

Response 200
json :
"OK"

register dfget in Supernode as a peer node

POST /api/v1/peers

Description

dfget sends request to register in Supernode as a peer node

Parameters

Type Name Description Schema
Body body
optional
request body which contains peer registrar information. PeerCreateRequest

Responses

HTTP Code Description Schema
201 no error PeerCreateResponse
400 bad parameter Error
500 An unexpected server error occurred. Error

get all peers

GET /api/v1/peers

Description

dfget sends request to register in Supernode as a peer node

Parameters

Type Name Description Schema Default
Query pageNum
optional
integer 0
Query pageSize
required
integer
Query sortDirect
optional
Determine the direction of sorting rules enum (ASC, DESC) "ASC"
Query sortKey
optional
"The keyword used to sort. You can provide multiple keys, if two peers have the same first key, sort by the second key, and so on" < string > array

Responses

HTTP Code Description Schema
200 no error < PeerInfo > array
400 bad parameter Error
500 An unexpected server error occurred. Error

get a peer in supernode

GET /api/v1/peers/{id}

Description

return low-level information of a peer in supernode.

Parameters

Type Name Description Schema
Path id
required
ID of peer string

Responses

HTTP Code Description Schema
200 no error PeerInfo
404 An unexpected 404 error occurred. Error
500 An unexpected server error occurred. Error

Produces

  • application/json

delete a peer in supernode

DELETE /api/v1/peers/{id}

Description

dfget stops playing a role as a peer in peer network constructed by supernode. When dfget lasts in five minutes without downloading or uploading task, the uploader of dfget automatically sends a DELETE /peers/{id} request to supernode.

Parameters

Type Name Description Schema
Path id
required
ID of peer string

Responses

HTTP Code Description Schema
204 no error No Content
404 no such peer 4ErrorResponse
500 An unexpected server error occurred. Error

Create a Preheat Task

POST /api/v1/preheats

Description

Create a preheat task in supernode to first download image/file which is ready. Preheat action will shorten the period for dfget to get what it wants. In details, after preheat action finishes downloading image/file to supernode, dfget can send request to setup a peer-to-peer network immediately.

Parameters

Type Name Description Schema
Body PreheatCreateRequest
optional
request body which contains preheat task creation information PreheatCreateRequest

Responses

HTTP Code Description Schema
200 no error PreheatCreateResponse
400 bad parameter Error
409 preheat task already exists Error
500 An unexpected server error occurred. Error

List Preheat Tasks

GET /api/v1/preheats

Description

List preheat tasks in supernode of Dragonfly. This API can list all the existing preheat tasks in supernode. Note, when a preheat is finished after PreheatGCThreshold, it will be GCed, then this preheat will not be gotten by preheat tasks list API.

Responses

HTTP Code Description Schema
200 no error < PreheatInfo > array
400 bad parameter Error
500 An unexpected server error occurred. Error

Get a preheat task

GET /api/v1/preheats/{id}

Description

get detailed information of a preheat task in supernode.

Parameters

Type Name Description Schema
Path id
required
ID of preheat task string

Responses

HTTP Code Description Schema
200 no error PreheatInfo
404 no such preheat task 4ErrorResponse
500 An unexpected server error occurred. Error

Produces

  • application/json

Delete a preheat task

DELETE /api/v1/preheats/{id}

Description

delete a preheat task

Parameters

Type Name Description Schema
Path id
required
ID of preheat task string

Responses

HTTP Code Description Schema
200 no error No Content
404 no such preheat task 4ErrorResponse
500 An unexpected server error occurred. Error

Produces

  • application/json

create a task

POST /api/v1/tasks

Description

Create a peer-to-peer downloading task in supernode.

Parameters

Type Name Description Schema
Body body
optional
request body which contains task creation information TaskCreateRequest

Responses

HTTP Code Description Schema
201 no error TaskCreateResponse
400 bad parameter Error
500 An unexpected server error occurred. Error

get a task

GET /api/v1/tasks/{id}

Description

return low-level information of a task in supernode.

Parameters

Type Name Description Schema
Path id
required
ID of task string

Responses

HTTP Code Description Schema
200 no error TaskInfo
404 An unexpected 404 error occurred. Error
500 An unexpected server error occurred. Error

Produces

  • application/json

update a task

PUT /api/v1/tasks/{id}

Description

Update information of a task. This endpoint is mainly for operation usage. When the peer network or peer meet some load issues, operation team can update a task directly, such as pause a downloading task to ease the situation.

Parameters

Type Name Description Schema
Path id
required
ID of task string
Body TaskUpdateRequest
optional
request body which contains task update information" TaskUpdateRequest

Responses

HTTP Code Description Schema
200 no error No Content
404 An unexpected 404 error occurred. Error
500 An unexpected server error occurred. Error

Consumes

  • application/json

Produces

  • application/json

delete a task

DELETE /api/v1/tasks/{id}

Description

delete a peer-to-peer task in supernode. This endpoint is mainly for operation usage. When the peer network or peer meet some load issues, operation team can delete a task directly to ease the situation.

Parameters

Type Name Description Schema Default
Path id
required
ID of task string
Query full
optional
supernode will also delete the cdn files when the value of full equals true. boolean "false"

Responses

HTTP Code Description Schema
204 no error No Content
404 no such task 4ErrorResponse
500 An unexpected server error occurred. Error

Get pieces in task

GET /api/v1/tasks/{id}/pieces

Description

When dfget starts to download pieces of a task, it should get fixed number of pieces in a task and then use pieces information to download the pieces. The request piece number is set in query.

Parameters

Type Name Description Schema
Path id
required
ID of task string
Query clientID
required
When dfget needs to get pieces of specific task, it must mark which peer it plays role of. string
Query num
optional
Request number of pieces of task. If request number is larger than the total pieces in supernode,
supernode returns the total pieces of task. If not set, supernode will set 4 by default.
integer (int64)
Body PiecePullRequest
required
request body which contains the information of pieces that have been downloaded or being downloaded. PiecePullRequest

Responses

HTTP Code Description Schema
200 no error < PieceInfo > array
404 no such task 4ErrorResponse
500 An unexpected server error occurred. Error

Produces

  • application/json

Update a piece

PUT /api/v1/tasks/{id}/pieces/{pieceRange}

Description

Update some information of piece. When peer A finishes to download piece B, A must send request to supernode to update piece B's info to mark that peer A has the complete piece B. Then when other peers request to download this piece B, supernode could schedule peer A to those peers.

Parameters

Type Name Description Schema
Path id
required
ID of task string
Path pieceRange
required
the range of specific piece in the task, example "0-45565". string
Body PieceUpdateRequest
optional
request body which contains task update information. PieceUpdateRequest

Responses

HTTP Code Description Schema
200 no error No Content
404 An unexpected 404 error occurred. Error
500 An unexpected server error occurred. Error

Consumes

  • application/json

Produces

  • application/json

report a piece error

POST /api/v1/tasks/{id}/pieces/{pieceRange}/error

Description

When a peer failed to download a piece from supernode or failed to validate the pieceMD5, and then dfget should report the error info to supernode.

Parameters

Type Name Description Schema
Path id
required
ID of task string
Path pieceRange
required
the range of specific piece in the task, example "0-45565". string
Body PieceErrorRequest
optional
request body which contains piece error information. PieceErrorRequest

Responses

HTTP Code Description Schema
200 no error No Content
404 An unexpected 404 error occurred. Error
500 An unexpected server error occurred. Error

Consumes

  • application/json

Produces

  • application/json

Get Prometheus metrics

GET /metrics

Description

Get Prometheus metrics

Responses

HTTP Code Description Schema
200 no error string

Example HTTP response

Response 200
json :
"go_goroutines 1"

report the heart beat to super node.

POST /peer/heartbeat

Description

This endpoint is mainly for reporting the heart beat to supernode. And supernode could know if peer is alive in strem mode.

Parameters

Type Name Description Schema
Body body
optional
request body which contains base info of peer HeartBeatRequest

Responses

HTTP Code Description Schema
200 no error HeartBeatResponse
500 An unexpected server error occurred. Error

peer request the p2p network info from supernode.

POST /peer/network

Description

In the new mode which dfdaemon will provide the seed file so that other peers could download. This endpoint is mainly for fetching the p2p network info.

Parameters

Type Name Description Schema
Body body
optional
request body which filter urls. NetworkInfoFetchRequest

Responses

HTTP Code Description Schema
200 no error NetworkInfoFetchResponse
500 An unexpected server error occurred. Error

report a piece has been success

GET /peer/piece/suc

Description

Update some information of piece. When peer A finishes to download piece B, A must send request to supernode to update piece B's info to mark that peer A has the complete piece B. Then when other peers request to download this piece B, supernode could schedule peer A to those peers.

Parameters

Type Name Description Schema
Query cid
required
the downloader clientID string
Query dstCid
optional
the uploader peerID string
Query pieceRange
required
the range of specific piece in the task, example "0-45565". string
Query taskId
required
ID of task string

Responses

HTTP Code Description Schema
200 no error ResultInfo
404 An unexpected 404 error occurred. Error
500 An unexpected server error occurred. Error

Produces

  • application/json

registry a task

POST /peer/registry

Description

Create a peer-to-peer downloading task in supernode.

Parameters

Type Name Description Schema
Body body
optional
request body which contains task creation information TaskRegisterRequest

Responses

HTTP Code Description Schema
200 no error ResultInfo
400 bad parameter Error
500 An unexpected server error occurred. Error

report a peer service will offline

GET /peer/service/down

Parameters

Type Name Description Schema
Query cid
required
the downloader clientID string
Query taskId
required
ID of task string

Responses

HTTP Code Description Schema
200 no error ResultInfo
404 An unexpected 404 error occurred. Error
500 An unexpected server error occurred. Error

Produces

  • application/json

Get pieces in task

GET /peer/task

Description

When dfget starts to download pieces of a task, it should get fixed number of pieces in a task and then use pieces information to download the pieces. The request piece number is set in query.

Parameters

Type Name Description Schema
Query dstCid
optional
the uploader cid string
Query range
optional
the range of specific piece in the task, example "0-45565". string
Query result
optional
pieceResult It indicates whether the dfgetTask successfully download the piece.
It's only useful when status is RUNNING.
enum (FAILED, SUCCESS, INVALID, SEMISUC)
Query srcCid
required
When dfget needs to get pieces of specific task, it must mark which peer it plays role of. string
Query status
optional
dfgetTaskStatus indicates whether the dfgetTask is running. enum (STARTED, RUNNING, FINISHED)
Query taskId
required
ID of task string

Responses

HTTP Code Description Schema
200 no error ResultInfo
404 no such task 4ErrorResponse
500 An unexpected server error occurred. Error

Produces

  • application/json

upload dfclient download metrics

POST /task/metrics

Description

This endpoint is mainly for observability. Dfget is a short-live job and we use this endpoint to upload dfget download related metrics.

Parameters

Type Name Description Schema
Body body
optional
request body which contains dfget download related information TaskMetricsRequest

Responses

HTTP Code Description Schema
200 no error ResultInfo
500 An unexpected server error occurred. Error

Get version and build information

GET /version

Description

Get version and build information, including GoVersion, OS, Arch, Version, BuildDate, and GitCommit.

Responses

HTTP Code Description Schema
200 no error DragonflyVersion
500 An unexpected server error occurred. Error

Definitions

CdnSource

Type : enum (supernode, source)

DfGetTask

A download process initiated by dfget or other clients.

Name Description Schema
cID
optional
CID means the client ID. It maps to the specific dfget process.
When user wishes to download an image/file, user would start a dfget process to do this.
This dfget is treated a client and carries a client ID.
Thus, multiple dfget processes on the same peer have different CIDs.
string
callSystem
optional
This attribute represents where the dfget requests come from. Dfget will pass
this field to supernode and supernode can do some checking and filtering via
black/white list mechanism to guarantee security, or some other purposes like debugging.
Minimum length : 1
string
dfdaemon
optional
tells whether it is a call from dfdaemon. dfdaemon is a long running
process which works for container engines. It translates the image
pulling request into raw requests into those dfget recognizes.
boolean
path
optional
path is used in one peer A for uploading functionality. When peer B hopes
to get piece C from peer A, B must provide a URL for piece C.
Then when creating a task in supernode, peer A must provide this URL in request.
string
peerID
optional
PeerID uniquely identifies a peer, and the cID uniquely identifies a
download task belonging to a peer. One peer can initiate multiple download tasks,
which means that one peer corresponds to multiple cIDs.
string
pieceSize
optional
The size of pieces which is calculated as per the following strategy
1. If file's total size is less than 200MB, then the piece size is 4MB by default.
2. Otherwise, it equals to the smaller value between totalSize/100MB + 2 MB and 15MB.
integer (int32)
status
optional
The status of Dfget download process. enum (WAITING, RUNNING, FAILED, SUCCESS)
supernodeIP
optional
IP address of supernode which the peer connects to string
taskId
optional
string

DragonflyVersion

Version and build information of Dragonfly components.

Name Description Schema
Arch
optional
Dragonfly components's architecture target string
BuildDate
optional
Build Date of Dragonfly components string
GoVersion
optional
Golang runtime version string
OS
optional
Dragonfly components's operating system string
Revision
optional
Git commit when building Dragonfly components string
Version
optional
Version of Dragonfly components string

Error

Name Schema
message
optional
string

ErrorResponse

It contains a code that identify which error occurred for client processing and a detailed error message to read.

Name Description Schema
code
optional
the code of this error, it's convenient for client to process with certain error. integer
message
optional
detailed error message string

HeartBeatRequest

The request is to report peer to supernode to keep alive.

Name Description Schema
IP
optional
IP address which peer client carries string (ipv4)
cID
optional
CID means the client ID. It maps to the specific dfget process.
When user wishes to download an image/file, user would start a dfget process to do this.
This dfget is treated a client and carries a client ID.
Thus, multiple dfget processes on the same peer have different CIDs.
string
port
optional
when registering, dfget will setup one uploader process.
This one acts as a server for peer pulling tasks.
This port is which this server listens on.
Minimum value : 15000
Maximum value : 65000
integer (int32)

HeartBeatResponse

Name Description Schema
needRegister
optional
If peer do not register in supernode, set needRegister to be true, else set to be false. boolean
seedTaskIDs
optional
The array of seed taskID which now are selected as seed for the peer. If peer have other seed file which
is not included in the array, these seed file should be weed out.
< string > array
version
optional
The version of supernode. If supernode restarts, version should be different, so dfdaemon could know
the restart of supernode.
string

NetworkInfoFetchRequest

The request is to fetch p2p network info from supernode.

Name Description Schema
urls
optional
The urls is to filter the peer node, the url should be match with taskURL in TaskInfo. < string > array

NetworkInfoFetchResponse

The response is from supernode to peer which is requested to fetch p2p network info.

Name Schema
nodes
optional
< Node > array

Node

The object shows the basic info of node and the task belongs to the node.

Name Description Schema
basic
optional
Basic node info PeerInfo
extra
optional
< string, string > map
load
optional
The load of node, which as the schedule weight in peer schedule. integer
tasks
optional
< TaskFetchInfo > array

PeerCreateRequest

PeerCreateRequest is used to create a peer instance in supernode. Usually, when dfget is going to register in supernode as a peer, it will send PeerCreateRequest to supernode.

Name Description Schema
IP
optional
IP address which peer client carries string (ipv4)
hostName
optional
host name of peer client node, as a valid RFC 1123 hostname.
Minimum length : 1
string (hostname)
port
optional
when registering, dfget will setup one uploader process.
This one acts as a server for peer pulling tasks.
This port is which this server listens on.
Minimum value : 15000
Maximum value : 65000
integer (int32)
version
optional
version number of dfget binary. string

PeerCreateResponse

ID of created peer.

Name Description Schema
ID
optional
Peer ID of the node which dfget locates on.
Every peer has a unique ID among peer network.
It is generated via host's hostname and IP address.
string

PeerInfo

The detailed information of a peer in supernode.

Name Description Schema
ID
optional
ID of peer string
IP
optional
IP address which peer client carries.
(TODO) make IP field contain more information, for example
WAN/LAN IP address for supernode to recognize.
string (ipv4)
created
optional
the time to join the P2P network string (date-time)
hostName
optional
host name of peer client node, as a valid RFC 1123 hostname.
Minimum length : 1
string (hostname)
port
optional
when registering, dfget will setup one uploader process.
This one acts as a server for peer pulling tasks.
This port is which this server listens on.
Minimum value : 15000
Maximum value : 65000
integer (int32)
version
optional
version number of dfget binary string

PieceErrorRequest

Peer's detailed information in supernode.

Name Description Schema
dstIP
optional
the peer ID of the target Peer. string
dstPid
optional
the peer ID of the target Peer. string
errorType
optional
the error type when failed to download from supernode that dfget will report to supernode enum (FILE_NOT_EXIST, FILE_MD5_NOT_MATCH)
expectedMd5
optional
the MD5 value of piece which returned by the supernode that
in order to verify the correctness of the piece content which
downloaded from the other peers.
string
range
optional
the range of specific piece in the task, example "0-45565". string
realMd5
optional
the MD5 information of piece which calculated by the piece content
which downloaded from the target peer.
string
srcCid
optional
the CID of the src Peer. string
taskId
optional
the taskID of the piece. string

PieceInfo

Peer's detailed information in supernode.

Name Description Schema
pID
optional
the peerID that dfget task should download from string
path
optional
The URL path to download the specific piece from the target peer's uploader. string
peerIP
optional
When dfget needs to download a piece from another peer. Supernode will return a PieceInfo
that contains a peerIP. This peerIP represents the IP of this dfget's target peer.
string
peerPort
optional
When dfget needs to download a piece from another peer. Supernode will return a PieceInfo
that contains a peerPort. This peerPort represents the port of this dfget's target peer's uploader.
integer (int32)
pieceMD5
optional
the MD5 information of piece which is generated by supernode when doing CDN cache.
This value will be returned to dfget in order to validate the piece's completeness.
string
pieceRange
optional
the range of specific piece in the task, example "0-45565". string
pieceSize
optional
The size of pieces which is calculated as per the following strategy
1. If file's total size is less than 200MB, then the piece size is 4MB by default.
2. Otherwise, it equals to the smaller value between totalSize/100MB + 2 MB and 15MB.
integer (int32)

PiecePullRequest

request used to pull pieces that have not been downloaded.

Name Description Schema
dfgetTaskStatus
optional
dfgetTaskStatus indicates whether the dfgetTask is running. enum (STARTED, RUNNING, FINISHED)
dstPID
optional
the uploader peerID string
pieceRange
optional
the range of specific piece in the task, example "0-45565". string
pieceResult
optional
pieceResult It indicates whether the dfgetTask successfully download the piece.
It's only useful when status is RUNNING.
enum (FAILED, SUCCESS, INVALID, SEMISUC)

PieceUpdateRequest

request used to update piece attributes.

Name Description Schema
clientID
optional
the downloader clientID string
dstPID
optional
the uploader peerID string
pieceStatus
optional
pieceStatus indicates whether the peer task successfully download the piece. enum (FAILED, SUCCESS, INVALID, SEMISUC)

PreheatCreateRequest

Request option of creating a preheat task in supernode.

Name Description Schema
filter
optional
URL may contains some changeful query parameters such as authentication parameters. Dragonfly will
filter these parameter via 'filter'. The usage of it is that different URL may generate the same
download taskID.
string
headers
optional
If there is any authentication step of the remote server, the headers should contains authenticated information.
Dragonfly will sent request taking the headers to remote server.
< string, string > map
identifier
optional
This field is used for generating new downloading taskID to identify different downloading task of remote URL. string
type
required
this must be image or file enum (image, file)
url
required
the image or file location
Minimum length : 3
string

PreheatCreateResponse

Response of a preheat creation request.

Name Schema
ID
optional
string

PreheatInfo

return detailed information of a preheat task in supernode. An image preheat task may contain multiple downloading task because that an image may have more than one layer.

Name Description Schema
ID
optional
ID of preheat task. string
finishTime
optional
the preheat task finish time string (date-time)
startTime
optional
the preheat task start time string (date-time)
status
optional
The status of preheat task.
WAITING -----> RUNNING -----> SUCCESS
|--> FAILED
The initial status of a created preheat task is WAITING.
It's finished when a preheat task's status is FAILED or SUCCESS.
A finished preheat task's information can be queried within 24 hours.
PreheatStatus

PreheatStatus

The status of preheat task. WAITING -----> RUNNING -----> SUCCESS |--> FAILED The initial status of a created preheat task is WAITING. It's finished when a preheat task's status is FAILED or SUCCESS. A finished preheat task's information can be queried within 24 hours.

Type : enum (WAITING, RUNNING, FAILED, SUCCESS)

ResultInfo

The returned information from supernode.

Name Description Schema
code
optional
the result code integer (int32)
data
optional
the result data object
msg
optional
the result msg string

TaskCreateRequest

Name Description Schema
cID
optional
CID means the client ID. It maps to the specific dfget process.
When user wishes to download an image/file, user would start a dfget process to do this.
This dfget is treated a client and carries a client ID.
Thus, multiple dfget processes on the same peer have different CIDs.
string
callSystem
optional
This attribute represents where the dfget requests come from. Dfget will pass
this field to supernode and supernode can do some checking and filtering via
black/white list mechanism to guarantee security, or some other purposes like debugging.
Minimum length : 1
string
dfdaemon
optional
tells whether it is a call from dfdaemon. dfdaemon is a long running
process which works for container engines. It translates the image
pulling request into raw requests into those dfget recognizes.
boolean
fileLength
optional
This attribute represents the length of resource, dfdaemon or dfget catches and calculates
this parameter from the headers of request URL. If fileLength is vaild, the supernode need
not get the length of resource by accessing the rawURL.
integer (int64)
filter
optional
filter is used to filter request queries in URL.
For example, when a user wants to start to download a task which has a remote URL of
a.b.com/fileA?user=xxx&auth=yyy, user can add a filter parameter ["user", "auth"]
to filter the url to a.b.com/fileA. Then this parameter can potentially avoid repeatable
downloads, if there is already a task a.b.com/fileA.
< string > array
headers
optional
extra HTTP headers sent to the rawURL.
This field is carried with the request to supernode.
Supernode will extract these HTTP headers, and set them in HTTP downloading requests
from source server as user's wish.
< string, string > map
identifier
optional
special attribute of remote source file. This field is used with taskURL to generate new taskID to
identify different downloading task of remote source file. For example, if user A and user B uses
the same taskURL and taskID to download file, A and B will share the same peer network to distribute files.
If user A additionally adds an identifier with taskURL, while user B still carries only taskURL, then A's
generated taskID is different from B, and the result is that two users use different peer networks.
string
md5
optional
md5 checksum for the resource to distribute. dfget catches this parameter from dfget's CLI
and passes it to supernode. When supernode finishes downloading file/image from the source location,
it will validate the source file with this md5 value to check whether this is a valid file.
string
path
optional
path is used in one peer A for uploading functionality. When peer B hopes
to get piece C from peer A, B must provide a URL for piece C.
Then when creating a task in supernode, peer A must provide this URL in request.
string
peerID
optional
PeerID is used to uniquely identifies a peer which will be used to create a dfgetTask.
The value must be the value in the response after registering a peer.
string
rawURL
optional
The is the resource's URL which user uses dfget to download. The location of URL can be anywhere, LAN or WAN.
For image distribution, this is image layer's URL in image registry.
The resource url is provided by command line parameter.
string
supernodeIP
optional
IP address of supernode which the peer connects to string
taskId
optional
This attribute represents the digest of resource, dfdaemon or dfget catches this parameter
from the headers of request URL. The digest will be considered as the taskID if not null.
string
taskURL
optional
taskURL is generated from rawURL. rawURL may contains some queries or parameter, dfget will filter some queries via
--filter parameter of dfget. The usage of it is that different rawURL may generate the same taskID.
string

TaskCreateResponse

response get from task creation request.

Name Description Schema
ID
optional
ID of the created task. string
cdnSource
optional
CdnSource
fileLength
optional
The length of the file dfget requests to download in bytes. integer (int64)
pieceSize
optional
The size of pieces which is calculated as per the following strategy
1. If file's total size is less than 200MB, then the piece size is 4MB by default.
2. Otherwise, it equals to the smaller value between totalSize/100MB + 2 MB and 15MB.
integer (int32)

TaskFetchInfo

It shows the task info and pieces info.

Name Description Schema
pieces
optional
The pieces which should belong to the peer node < PieceInfo > array
task
optional
TaskInfo

TaskInfo

detailed information about task in supernode.

Name Description Schema
ID
optional
ID of the task. string
asSeed
optional
This attribute represents the node as a seed node for the taskURL. boolean
cdnStatus
optional
The status of the created task related to CDN functionality. enum (WAITING, RUNNING, FAILED, SUCCESS, SOURCE_ERROR)
fileLength
optional
The length of the file dfget requests to download in bytes
which including the header and the trailer of each piece.
integer (int64)
headers
optional
extra HTTP headers sent to the rawURL.
This field is carried with the request to supernode.
Supernode will extract these HTTP headers, and set them in HTTP downloading requests
from source server as user's wish.
< string, string > map
httpFileLength
optional
The length of the source file in bytes. integer (int64)
identifier
optional
special attribute of remote source file. This field is used with taskURL to generate new taskID to
identify different downloading task of remote source file. For example, if user A and user B uses
the same taskURL and taskID to download file, A and B will share the same peer network to distribute files.
If user A additionally adds an identifier with taskURL, while user B still carries only taskURL, then A's
generated taskID is different from B, and the result is that two users use different peer networks.
string
md5
optional
md5 checksum for the resource to distribute. dfget catches this parameter from dfget's CLI
and passes it to supernode. When supernode finishes downloading file/image from the source location,
it will validate the source file with this md5 value to check whether this is a valid file.
string
pieceSize
optional
The size of pieces which is calculated as per the following strategy
1. If file's total size is less than 200MB, then the piece size is 4MB by default.
2. Otherwise, it equals to the smaller value between totalSize/100MB + 2 MB and 15MB.
integer (int32)
pieceTotal
optional
integer (int32)
rawURL
optional
The is the resource's URL which user uses dfget to download. The location of URL can be anywhere, LAN or WAN.
For image distribution, this is image layer's URL in image registry.
The resource url is provided by command line parameter.
string
realMd5
optional
when supernode finishes downloading file/image from the source location,
the md5 sum of the source file will be calculated as the value of the realMd5.
And it will be used to compare with md5 value to check whether this is a valid file.
string
taskURL
optional
taskURL is generated from rawURL. rawURL may contains some queries or parameter, dfget will filter some queries via
--filter parameter of dfget. The usage of it is that different rawURL may generate the same taskID.
string

TaskMetricsRequest

Name Description Schema
IP
optional
IP address which peer client carries string (string)
backsourceReason
optional
when registering, dfget will setup one uploader process.
This one acts as a server for peer pulling tasks.
This port is which this server listens on.
string
cID
optional
CID means the client ID. It maps to the specific dfget process.
When user wishes to download an image/file, user would start a dfget process to do this.
This dfget is treated a client and carries a client ID.
Thus, multiple dfget processes on the same peer have different CIDs.
string
callSystem
optional
This attribute represents where the dfget requests come from. Dfget will pass
this field to supernode and supernode can do some checking and filtering via
black/white list mechanism to guarantee security, or some other purposes like debugging.
Minimum length : 1
string
duration
optional
Duration for dfget task. number (float64)
fileLength
optional
The length of the file dfget requests to download in bytes. integer (int64)
port
optional
when registering, dfget will setup one uploader process.
This one acts as a server for peer pulling tasks.
This port is which this server listens on.
Minimum value : 15000
Maximum value : 65000
integer (int32)
success
optional
whether the download task success or not boolean
taskId
optional
IP address which peer client carries string (string)

TaskRegisterRequest

Name Description Schema
IP
optional
IP address which peer client carries string (ipv4)
asSeed
optional
This attribute represents the node as a seed node for the taskURL. boolean
cID
optional
CID means the client ID. It maps to the specific dfget process.
When user wishes to download an image/file, user would start a dfget process to do this.
This dfget is treated a client and carries a client ID.
Thus, multiple dfget processes on the same peer have different CIDs.
string
callSystem
optional
This attribute represents where the dfget requests come from. Dfget will pass
this field to supernode and supernode can do some checking and filtering via
black/white list mechanism to guarantee security, or some other purposes like debugging.
Minimum length : 1
string
dfdaemon
optional
tells whether it is a call from dfdaemon. dfdaemon is a long running
process which works for container engines. It translates the image
pulling request into raw requests into those dfget recognizes.
boolean
fileLength
optional
This attribute represents the length of resource, dfdaemon or dfget catches and calculates
this parameter from the headers of request URL. If fileLength is vaild, the supernode need
not get the length of resource by accessing the rawURL.
integer (int64)
headers
optional
extra HTTP headers sent to the rawURL.
This field is carried with the request to supernode.
Supernode will extract these HTTP headers, and set them in HTTP downloading requests
from source server as user's wish.
< string > array
hostName
optional
host name of peer client node.
Minimum length : 1
string
identifier
optional
special attribute of remote source file. This field is used with taskURL to generate new taskID to
identify different downloading task of remote source file. For example, if user A and user B uses
the same taskURL and taskID to download file, A and B will share the same peer network to distribute files.
If user A additionally adds an identifier with taskURL, while user B still carries only taskURL, then A's
generated taskID is different from B, and the result is that two users use different peer networks.
string
insecure
optional
tells whether skip secure verify when supernode download the remote source file. boolean
md5
optional
md5 checksum for the resource to distribute. dfget catches this parameter from dfget's CLI
and passes it to supernode. When supernode finishes downloading file/image from the source location,
it will validate the source file with this md5 value to check whether this is a valid file.
string
path
optional
path is used in one peer A for uploading functionality. When peer B hopes
to get piece C from peer A, B must provide a URL for piece C.
Then when creating a task in supernode, peer A must provide this URL in request.
string
port
optional
when registering, dfget will setup one uploader process.
This one acts as a server for peer pulling tasks.
This port is which this server listens on.
Minimum value : 15000
Maximum value : 65000
integer (int32)
rawURL
optional
The is the resource's URL which user uses dfget to download. The location of URL can be anywhere, LAN or WAN.
For image distribution, this is image layer's URL in image registry.
The resource url is provided by command line parameter.
string
rootCAs
optional
The root ca cert from client used to download the remote source file. < string (byte) > array
superNodeIp
optional
The address of supernode that the client can connect to string
taskId
optional
Dfdaemon or dfget could specific the taskID which will represents the key of this resource
in supernode.
string
taskURL
optional
taskURL is generated from rawURL. rawURL may contains some queries or parameter, dfget will filter some queries via
--filter parameter of dfget. The usage of it is that different rawURL may generate the same taskID.
string
version
optional
version number of dfget binary. string

TaskUpdateRequest

request used to update task attributes.

Name Description Schema
peerID
optional
ID of the peer which has finished to download the whole task. string