Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

spec: update roundup #482

Merged
merged 4 commits into from
Jun 16, 2020
Merged

spec: update roundup #482

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f5b4022
timestamp -> tserver and explain relation to locktime
buck54321 Jun 7, 2020
89d4250
Table of contents update and trade suspension
buck54321 Jun 15, 2020
8419a61
fix links and typos. admin API. trade resumption
buck54321 Jun 15, 2020
4f03310
add ban endpoint, admin api entry in toc, other stuff
buck54321 Jun 16, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions dex/msgjson/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -370,7 +370,7 @@ type Match struct {
Signature
OrderID Bytes `json:"orderid"`
MatchID Bytes `json:"matchid"`
Quantity uint64 `json:"quantity"`
Quantity uint64 `json:"qty"`
Rate uint64 `json:"rate"`
Address string `json:"address"`
ServerTime uint64 `json:"tserver"`
Expand Down Expand Up @@ -661,7 +661,7 @@ type OrderNote struct {
// limit or market order.
type TradeNote struct {
Side uint8 `json:"side,omitempty"`
Quantity uint64 `json:"osize,omitempty"`
Quantity uint64 `json:"qty,omitempty"`
Rate uint64 `json:"rate,omitempty"`
TiF uint8 `json:"tif,omitempty"`
Time uint64 `json:"time,omitempty"`
Expand Down
142 changes: 72 additions & 70 deletions spec/README.mediawiki
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,124 +1,123 @@
=Decred DEX=

This is the repository for the Decred Distributed Exchange (DEX), for which this
document provides the '''DRAFT''' specification.
=Decred DEX Specification=

__TOC__

==Abstract==

We propose here a decentralized, non-custodial, trustless, and intermediary-free
cryptocurrency exchange platform.

The primary service provided by the decentralized exchange (DEX) is to match
clients who would like to trade an asset of one type for an asset of another
type.
Trades are facilitated by a trustless atomic swap negotiation process carried
out with all the cryptographic security guarantees of the respective blockchains.

A custom order matching procedure is offered to mitigate common abusive trading
patterns associated with high-frequency trading (HFT) algorithms and other
bot-driven trading.
Elimination of per-trade fees disincentivizes deceptive bookkeeping practices
commonly observed in centralized exchanges.
==Introduction==

To encourage open and active development, descriptions of critical protocol
components and a full client-server API definition are provided.
Decred DEX is a decentralized, non-custodial, trustless, and intermediary-free
cryptocurrency exchange platform. Decred DEX aims to solve a number of issues
with existing systems of exchange.

==Introduction==
Currently, the majority of cryptocurrency trading is performed via centralized,
custodial exchanges.
These exchanges have filled an important role in the blockchain ecosystem, but
are in many ways antithetical to the spirit of blockchain.
Blockchain is decentralized, trustless, and enables self-custody, but when a
user must send their coin to a third party in order to trade, all of those
benefits are lost.

Existing cryptocurrency exchange platforms fall into 4 overlapping categories:
custodial corporate entities, services, tokens, and blockchains.
The vast majority of cryptocurrency trading is currently performed via
centralized, custodial exchanges.
Since trades at these exchanges take place off-chain, they generally settle
instantly to make funds available for further trading.
This is a nice feature, but it practically guarantees that the market
will be a target of HFT and algorithmic traders, which can be unpredictable and
malicious.
Additionally, such exchanges earn revenue by collecting trading fees.
Such a model creates incentives for an exchange to artificially inflate their
volume by wash trading [1].
By one study, it has been estimated that more than 95% of reported exchange
volume is faked
[https://www.sec.gov/comments/sr-nysearca-2019-01/srnysearca201901-5164833-183434.pdf [2]].

Several DEX projects have been created to address some of these issues by
Several projects have attempted to address this misalignment by
replacing the exchange with a blockchain or a token, and they have met with
varying degrees of success.
While they remove the trusted third party (TTP), they insert their own products
as a means to capture the trading fees, which replaces the TTP friction with a
as a means to capture trading fees, which replaces the TTP friction with a
new platform friction.
The simple act of collecting trading fees serves to act as an incentive to
centralize on a given solution, which runs counter to a system of open voluntary
exchange.
While a chain or token serves to remove the TTP, it also creates challenges with
order matching, which typically occurs via the intermediate chain or token.

The DEX described in this document is based on atomic swaps [https://github.com/decred/atomicswap [3]], signed and
transmitted by the clients. Thus, the exchange of funds is trustless and secure.
The purpose of the DEX is to facilitate such peer-to-peer trades, while
disincentivizing manipulative or abusive trading patterns, and utilizing an
independently verifiable order matching algorithm that is difficult to game by
traders and operators alike.

A more robust solution is based on atomic swap technology [https://github.com/decred/atomicswap [2]],
which enables trustless exchange directly between wallets.
Until recently, few blockchains supported atomic swaps, but the past few years
have seen widespread adoption, and such an exchange is now possible.

Decred DEX is the first exchange built on atomic swaps that doesn't collect
trading fees. The primary service offered by DEX is to match traders through a
familiar interface based on markets and orders. Decred DEX's novel matching
algorithm disincintevizes manipulative, abusive trading practices commonly seen
on centralized exchanges. The epoch-based match cycle is difficult to game by
the client, while for the server, cryptographic order commitments make
censorship and unfair matching difficult.

While trades are fully trustless, steps are taken to discourage malicious
clients from hindering normal trade activity.
Registered clients are bound by the
All clients pay a non-refundable registration fee.
Registered clients are then bound to the
[[community.mediawiki|rules of community conduct]], violation of which typically
results in loss of trading privileges.
Serious violations may result in a permanent ban, in which case the affected user
must pay another registration fee to continue using the DEX.

The following list of non-negotiable, fundamental, DEX properties served as a
basis for design.

# Non-custodial. Assets must be traded directly between clients, without the exchange being in control of assets at any time.

# Intermediary-free. Trades must be executed in a secure and trustless manner.
results in loss of trading privileges up to and including a permanent ban.

# Resistant to malicious or manipulative trading. Deliberately disruptive behavior should be disincentivized, and volume manipulation mitigated by pseudo-random epoch-based order matching.
In the interest of maintaining active, open-source, community-driven
development, this specification document describes the protocols necessary for
implementation of both DEX clients and DEX servers.

==Sections==
==Contents==

The remainder of this document details the design of the proposed DEX.
The remaining content details the Decred DEX protocol.
It is intended as a first resource when implementing servers and clients.
Continuing client-server interoperability will likely depend on subtle aspects
of the specification laid out in this document, especially the following
critical sections.
of the specification laid out in this document.

'''[1] [[comm.mediawiki|Communication Protocols]]''' describes the
messaging protocols and communication layer technologies that are to be used
for the DEX API.

* [[comm.mediawiki#WebSockets|WebSockets]]
* [[comm.mediawiki/#Encoding|Data Encodings]]
** [[comm.mediawiki/#Timestamps|Timestamps]]
** [[comm.mediawiki/#Rate_Encoding|Rate Encoding]]
** [[comm.mediawiki/#Coin_ID|Coin ID]]
* [[comm.mediawiki/#Message_Protocol|Message Protocol]]
* [[comm.mediawiki/#Session_Authentication|Session Authentication]]
* [[comm.mediawiki/#HTTP|HTTP]]

'''[2] [[fundamentals.mediawiki|Distributed Exchange Design Fundamentals]]'''
describes the notable design aspects that facilitate an exchange service with
the features described above.

* [[fundamentals.mediawiki/#Exchange_Variables|Exchange Variables]]
** [[fundamentals.mediawiki/#Global_Variables|Global Variables]]
** [[fundamentals.mediawiki/#Asset_Variables|Asset Variables]]
** [[fundamentals.mediawiki/#Market_Variables|Market Variables]]
** [[fundamentals.mediawiki/#Configuration_Data_Request|Configuration Data Request]]
* [[fundamentals.mediawiki/#Epochbased_Order_Matching|Epoch-based Order Matching]]
** [[fundamentals.mediawiki/#Epoch_Time|Epoch Time]]
** [[fundamentals.mediawiki/#Pseudorandom_Order_Matching|Pseudorandom Order Matching]]
* [[fundamentals.mediawiki/#Identities_based_on_Public_Key_Infrastructure_PKI_Key_Pairs|Identification]]
* [[fundamentals.mediawiki/#Blockchain_Interaction|Blockchain Interaction]]
* [[fundamentals.mediawiki/#Adding_New_Assets|Adding New Assets]]

'''[3] [[admin.mediawiki|Distributed Exchange Administration]]''' describes
the tasks required to administer the exchange.

* [[admin.mediawiki/#Exchange_Variables|Exchange Variables]]
* [[admin.mediawiki/#Perasset_Variables|Per-asset Variables]]
* [[admin.mediawiki/#Administration_API|Administration API]]

buck54321 marked this conversation as resolved.
Show resolved Hide resolved
'''[4] [[accounts.mediawiki|Client Accounts]]''' details account creation.

* [[accounts.mediawiki/#Step_1_Registration|Registration]]
* [[accounts.mediawiki/#Step_2_Fee_Notification|Fee Notification]]

'''[5] [[orders.mediawiki|Client Order Management]]''' details the different
order types and the client/server workflows required to synchronize the order
book and place orders.

* [[orders.mediawiki/#Connection_Persistence|Connection Persistence]]
* [[orders.mediawiki/#Order_Book_Subscriptions|Order Book Subscriptions]]
* [[orders.mediawiki/#Order_Preparation|Order Preparation]]
** [[orders.mediawiki/#Calculating_Transaction_Fees|Calculating Transaction Fees]]
** [[orders.mediawiki/#Coin_Preparation|Coin Preparation]]
** [[orders.mediawiki/#Order_Commitment|Order Commitment]]
** [[orders.mediawiki/#Order_Signing|Order Signing]]
** [[orders.mediawiki/#Order_ID|Order ID]]
* [[orders.mediawiki/#Order_Types|Order Types]]
** [[orders.mediawiki/#Limit_Order|Limit Order]]
** [[orders.mediawiki/#Market_Order|Market Order]]
*** [[orders.mediawiki/#Market_Buy_Orders|Market Buy Orders]]
** [[orders.mediawiki/#Cancel_Order|Cancel Order]]
* [[orders.mediawiki/#Preimage_Reveal|Preimage Handling]]
* [[orders.mediawiki/#Match_Revocation|Match Revocation]]
* [[orders.mediawiki/#Match_negotiation|Match Negotiation]]
* [[orders.mediawiki/#Trade_Suspension|Trade Suspension]]

'''[6] [[api.mediawiki| Data API]]''' defines http and websocket APIs to browse
'''[6] [[api.mediawiki| Data API]]''' defines http and WebSocket APIs to browse
trade history.

'''[7] [[atomic.mediawiki|Atomic Settlement]]''' walks through the settlement
Expand All @@ -127,5 +126,8 @@ process with a couple of examples to provide a high-level overview.
'''[8] [[community.mediawiki|Community Conduct]]''' describes the system of rules
to which clients interacting with the DEX must adhere.

* [[community.mediawiki/#Rules_of_Community_Conduct|Rules of Community Conduct]]
* [[community.mediawiki/#Penalties|Penalties]]

'''[9] [[references.mediawiki|References]]''' lists references used in the development
of the specification.
2 changes: 1 addition & 1 deletion spec/accounts.mediawiki
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ string in API messages.

==Step 1: Registration==

The user creates a websocket connection and sends their
The user creates a WebSocket connection and sends their
[[fundamentals.mediawiki/#Identities_based_on_Public_Key_Infrastructure_PKI_Key_Pairs|public account key]].
The message is signed with the corresponding private account key. The response
includes the server's public key. The server's public key will also be
Expand Down
31 changes: 27 additions & 4 deletions spec/admin.mediawiki
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ That said, changes to exchange or asset variables will often entail revocation
of all existing orders on a market, so should be done as infrequently as
possible.

'''Exchange Variables'''
==Exchange Variables==

{|
! variable !! relevant section !! units || default
Expand All @@ -27,14 +27,14 @@ possible.
| broadcast timeout || [[fundamentals.mediawiki/#Exchange_Variables|Exchange Variables]] & [[community.mediawiki/#rule-1-clients-must-respond-to-all-preimage-requests|Rule 1]] || milliseconds || 60,000
|}

'''Per-asset Variables'''
==Per-asset Variables==

{|
! variable !! units !! description
|-
| lot size || atoms || the minimum order quantity and the order quantity increment when an asset is the base asset
|-
| rate step || atoms || The minimum price rate and the price rate increment when an asset is the quote asset. [[orders.mediawiki/#Rate_Encoding|message-rate encoding]]
| rate step || atoms || the minimum price rate and the price rate increment when an asset is the quote asset. [[orders.mediawiki/#Rate_Encoding|message-rate encoding]]
|-
| fee rate || atoms/byte || the minimum fee rate for swap transactions
|-
Expand All @@ -45,4 +45,27 @@ possible.

See also [[fundamentals.mediawiki/#Exchange_Variables|Exchange Variables]].

...
==Administration API==

The server will provide an HTTP API for performing various adminstrative tasks.

'''API Endpoints'''
buck54321 marked this conversation as resolved.
Show resolved Hide resolved
{|
! path !! description
|-
| /config || the current DEX configuration. See [[fundamentals.mediawiki/#Configuration_Data_Request|Configuration Data Request]]
|-
| /accounts || lists information about all known accounts
|-
| /accounts/{accountID} || list information about a specific account
|-
| /markets || display status information for all markets
|-
| /market/{marketID} || display status information for a specific market
|-
| /market/{marketID}/suspend || schedule a market suspension at the end of the current epoch
|-
| /unban/{accountID} || clear an account's penalties and re-enable trading
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feelings about the /ban API endpoint? This relates to the previously discussed penalization notification, which should probably be signed by the server and include details of the infraction(s).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was gonna chat with you and @JoeGruffins about that. I see the question is still sort of in the air at #469 (comment). My gut says that we shouldn't even have the option. Is there a good reason?

Copy link
Member

@chappjc chappjc Jun 15, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unforeseen mischief, including hack attempts, etc. There are already a myriad of ways an operator can DoS a user, so making this seamless just makes sense to me. If the issue is sorted out, the unban function exists.

Copy link
Member

@chappjc chappjc Jun 15, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, the ban/unban will probably need to be renamed to suspend/reinstate or similar (eventually, not now), or at least the ban function will need a duration, maybe even option to disallow connection, when we figure out more advanced scoring and temporary suspension mechanics.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. Will just add /ban for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think with the current rules, ban is not so useful. Maybe only to re-ban someone you accidentally unbanned. But new rules may be added in the future, that can't be done programmatically, or that need to be retroactively imposed.

However, it could be abused by the operator. A normal exchange works on trading fees, so banning isn't in the best interest of the operator. But since the DEX is based on an initial fee, unless a user is vocal about it, banning has no impact on the operator. I don't know if this is an actual problem. I think we will find out when the DEX goes into production. Of course, even without this endpoint, an operator can just switch a value in the db and have the same effect. One possible solution is to try to make ban data public, if it is indeed a problem.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

banning has no impact on the operator

This may not be true. I guess having low volume would not get you new users. So there may be enough incentive for this to not be abused. If I were banned for nothing, I certainly wouldn't pay the fee again, I'd find another DEX.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with all of that. My sort of half-baked philosophy on this is that if we need to manually ban someone, then our software isn't doing it's job. But I also recognize that there are bugs, and a learning curve, and that the ability to ban someone who skirts the rules might be critical at some point. Right now, I'm feeling like we should leave it, and just make it one of our goals that nobody ever has to use it.

Copy link
Member

@chappjc chappjc Jun 16, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have it in the code and document it. It could be hacked into the code anyway, the DB could be fiddled with, and firewalls employed for banning, so it should be a supported function. I don't think it make senses to neuter the operator on principle, especially since there will be legit reasons to close an account like this. Consider that a programmatic ban according to the rules also potentially creates a burden for the operator to demonstrate the broken rules. Little changes with a ban function IMO. If the operator closes accounts, their exchange is going to be on everyone's shit list.

|-
| /ban/{accountID}/{ruleNumber} || ban an account for violating [[community.mediawiki/#Rules_of_Community_Conduct|a rule]]
|}
2 changes: 1 addition & 1 deletion spec/api.mediawiki
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

__TOC__

Trade history will be made available to both websocket and HTTP clients.
Trade history will be made available to both WebSocket and HTTP clients.
23 changes: 14 additions & 9 deletions spec/comm.mediawiki
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,15 @@ Match notification via HTTP polling or other request interval-based methods are
thus not suitable for the DEX system.
Persistent, full-duplex communication is critical to minimizing communication
latencies and wasted bandwidth.
Websockets ([https://tools.ietf.org/html/rfc6455 [4]]) are chosen as the
WebSockets ([https://tools.ietf.org/html/rfc6455 [3]]) are chosen as the
default and preferred communications protocol for
the DEX exchange API.
In addition to fulfilling the aforementioned needs, Websockets are now a
well-established technology with client software available for integration in
virtually all popular programming languages.

Websocket messages are secured by encryption on Transport Layer
Security (TLS) [https://tools.ietf.org/html/rfc8446 [5]] connections.
WebSocket messages are secured by encryption on Transport Layer
Security (TLS) [https://tools.ietf.org/html/rfc8446 [4]] connections.

==Encoding==

Expand All @@ -30,6 +30,9 @@ units of time are milliseconds unless otherwise specified. Location-independent
timestamps are encoded as milliseconds since the UNIX epoch (Jan 01 00:00:00
1970 UTC).

If a timestamp must be converted to seconds, e.g. for encoding as a locktime in
a swap contract, the timestamp should be rounded down.

===Rate Encoding===

Because the rate assigned to a limit order is a quotient, the value is naturally
Expand Down Expand Up @@ -57,7 +60,7 @@ ID and output index (vout).

In an effort to stay blockchain-protocol agnostic, the DEX accepts
and recognizes the locating information as a single byte-array called the
'''''coin ID''''', with the term '''''coin''''' being defined here as a some
'''''coin ID''''', with the term '''''coin''''' being defined here as some
amount of spendable value that is verifiable on the blockchain.
It is up to backend and wallet developers to decide on how to properly encode the
identifier as a coin ID. As an example, Bitcoin implements
Expand All @@ -66,7 +69,7 @@ big-endian encoded output index as the last 4 bytes.

==Message Protocol==

DEX messaging is JSON-formatted [https://tools.ietf.org/html/rfc8259 [6]].
DEX messaging is JSON-formatted [https://tools.ietf.org/html/rfc8259 [5]].
All messages, regardless of originating party, use a common top-level
structure called a '''Message'''.

Expand Down Expand Up @@ -145,7 +148,7 @@ The payload for a response has a structure that enables quick error checking.

==Session Authentication==

Many DEX messages must be sent on an authenticated connection. Once a websocket
Many DEX messages must be sent on an authenticated connection. Once a WebSocket
connection is established, the client will supply their account ID and signature.

'''Request route:''' <code>connect</code>, '''originator: ''' client
Expand Down Expand Up @@ -182,15 +185,17 @@ the client's absence. A list of any pending matches is included in the response.

<code>result</code>
{|
! field !! type !! description
! field !! type !! description
|-
| matches || &#91;object&#93; || list of [[orders.mediawiki/#Match_negotiation|Match objects]]
|-
| matches || &#91;object&#93; || list of [[orders.mediawiki/#Match_negotiation|Match objects]]
| sig || string || hex-encoded server's signature of the serialized connection data
|}

==HTTP==

An API using HTTP for message transport may be provided for basic account
management and server status queries, however websocket connections are to be
management and server status queries, however webWebSocketsocket connections are to be
the sole means for
[[orders.mediawiki/#Client_Order_Management|placing, monitoring, and executing orders]].
The primary reason for limiting the scope of the HTTP API is to eliminate client
Expand Down
Loading