Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

spec: update roundup #482

Merged
merged 4 commits into from
Jun 16, 2020
Merged

spec: update roundup #482

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f5b4022
timestamp -> tserver and explain relation to locktime
buck54321 Jun 7, 2020
89d4250
Table of contents update and trade suspension
buck54321 Jun 15, 2020
8419a61
fix links and typos. admin API. trade resumption
buck54321 Jun 15, 2020
4f03310
add ban endpoint, admin api entry in toc, other stuff
buck54321 Jun 16, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions dex/msgjson/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -370,7 +370,7 @@ type Match struct {
Signature
OrderID Bytes `json:"orderid"`
MatchID Bytes `json:"matchid"`
Quantity uint64 `json:"quantity"`
Quantity uint64 `json:"qty"`
Rate uint64 `json:"rate"`
Address string `json:"address"`
ServerTime uint64 `json:"tserver"`
Expand Down Expand Up @@ -661,7 +661,7 @@ type OrderNote struct {
// limit or market order.
type TradeNote struct {
Side uint8 `json:"side,omitempty"`
Quantity uint64 `json:"osize,omitempty"`
Quantity uint64 `json:"qty,omitempty"`
Rate uint64 `json:"rate,omitempty"`
TiF uint8 `json:"tif,omitempty"`
Time uint64 `json:"time,omitempty"`
Expand Down
139 changes: 70 additions & 69 deletions spec/README.mediawiki
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,122 +1,120 @@
=Decred DEX=

This is the repository for the Decred Distributed Exchange (DEX), for which this
document provides the '''DRAFT''' specification.
=Decred DEX Specification=

__TOC__

==Abstract==

We propose here a decentralized, non-custodial, trustless, and intermediary-free
cryptocurrency exchange platform.

The primary service provided by the decentralized exchange (DEX) is to match
clients who would like to trade an asset of one type for an asset of another
type.
Trades are facilitated by a trustless atomic swap negotiation process carried
out with all the cryptographic security guarantees of the respective blockchains.

A custom order matching procedure is offered to mitigate common abusive trading
patterns associated with high-frequency trading (HFT) algorithms and other
bot-driven trading.
Elimination of per-trade fees disincentivizes deceptive bookkeeping practices
commonly observed in centralized exchanges.
==Introduction==

To encourage open and active development, descriptions of critical protocol
components and a full client-server API definition are provided.
Decred DEX is a decentralized, non-custodial, trustless, and intermediary-free
cryptocurrency exchange platform. Decred DEX aims to solve a number of issues
with existing systems of exchange.

==Introduction==
Currently, the majority of cryptocurrency trading is performed via centralized,
custodial exchanges.
These exchanges have fill important role in the blockchain ecosystem, but are in
buck54321 marked this conversation as resolved.
Show resolved Hide resolved
many ways antithetical to the spirit of blockchain.
Blockchain is decentralized, trustless, and enables self-custody, but when a
user must send their coin to a third party in order to trade, all of those
benefits are lost.

Existing cryptocurrency exchange platforms fall into 4 overlapping categories:
custodial corporate entities, services, tokens, and blockchains.
The vast majority of cryptocurrency trading is currently performed via
centralized, custodial exchanges.
Since trades at these exchanges take place off-chain, they generally settle
instantly to make funds available for further trading.
This is a nice feature, but it practically guarantees that the market
will be a target of HFT and algorithmic traders, which can be unpredictable and
malicious.
Additionally, such exchanges earn revenue by collecting trading fees.
Such a model creates incentives for an exchange to artificially inflate their
volume by wash trading [1].
By one study, it has been estimated that more than 95% of reported exchange
volume is faked
[https://www.sec.gov/comments/sr-nysearca-2019-01/srnysearca201901-5164833-183434.pdf [2]].

Several DEX projects have been created to address some of these issues by
Several projects have attempted to address this misalignment by
replacing the exchange with a blockchain or a token, and they have met with
varying degrees of success.
While they remove the trusted third party (TTP), they insert their own products
as a means to capture the trading fees, which replaces the TTP friction with a
as a means to capture trading fees, which replaces the TTP friction with a
new platform friction.
The simple act of collecting trading fees serves to act as an incentive to
centralize on a given solution, which runs counter to a system of open voluntary
exchange.
While a chain or token serves to remove the TTP, it also creates challenges with
order matching, which typically occurs via the intermediate chain or token.

The DEX described in this document is based on atomic swaps [https://github.com/decred/atomicswap [3]], signed and
transmitted by the clients. Thus, the exchange of funds is trustless and secure.
The purpose of the DEX is to facilitate such peer-to-peer trades, while
disincentivizing manipulative or abusive trading patterns, and utilizing an
independently verifiable order matching algorithm that is difficult to game by
traders and operators alike.

A more robust solution is based on atomic swap technology [https://github.com/decred/atomicswap [2]],
which enables trustless exchange directly between wallets.
Until recently, few blockchains supported atomic swaps, but the past few years
have seen widespread adoption, and such an exchange is now possible.

Decred DEX is the first exchange built on atomic swaps that doesn't collect
trading fees. The primary service offered by DEX is to match traders through a
familiar interface based on markets and orders. Decred DEX's novel matching
algorithm disincintevizes manipulative, abusive trading practices commonly seen
on centralized exchanges. The epoch-based match cycle is difficult to game by
the client, while for the server, cryptographic order commitments make
censorship difficult.
buck54321 marked this conversation as resolved.
Show resolved Hide resolved

While trades are fully trustless, steps are taken to discourage malicious
clients from hindering normal trade activity.
Registered clients are bound by the
All clients pay a non-refundable registration fee.
Registered clients are then bound to the
[[community.mediawiki|rules of community conduct]], violation of which typically
results in loss of trading privileges.
Serious violations may result in a permanent ban, in which case the affected user
must pay another registration fee to continue using the DEX.

The following list of non-negotiable, fundamental, DEX properties served as a
basis for design.

# Non-custodial. Assets must be traded directly between clients, without the exchange being in control of assets at any time.

# Intermediary-free. Trades must be executed in a secure and trustless manner.
results in loss of trading privileges up to and including a permanent ban.

# Resistant to malicious or manipulative trading. Deliberately disruptive behavior should be disincentivized, and volume manipulation mitigated by pseudo-random epoch-based order matching.
In the interest of maintaining active, open-source, community-driven
development, this specification document describes the protocols necessary for
implementation of both DEX clients and DEX servers.

==Sections==
==Contents==

The remainder of this document details the design of the proposed DEX.
The remaining content details the Decred DEX protocol.
It is intended as a first resource when implementing servers and clients.
Continuing client-server interoperability will likely depend on subtle aspects
of the specification laid out in this document, especially the following
critical sections.
of the specification laid out in this document.

'''[1] [[comm.mediawiki|Communication Protocols]]''' describes the
messaging protocols and communication layer technologies that are to be used
for the DEX API.

* [[comm.mediawiki#WebSockets|Websockets]]
* [[comm.mediawiki/#Encoding|Data Encodings]]
** [[comm.mediawiki/#Timestamps|Timestamps]]
** [[comm.mediawiki/#Rate_Encoding|Rate Encoding]]
** [[comm.mediawiki/#Coin_ID|Coin ID]]
* [[comm.mediawiki/#Message_Protocol|Message Protocol]]
* [[comm.mediawiki/#Session_Authentication|Session Authentication]]
* [[comm.mediawiki/#HTTP|HTTP]]

'''[2] [[fundamentals.mediawiki|Distributed Exchange Design Fundamentals]]'''
describes the notable design aspects that facilitate an exchange service with
the features described above.

* [[fundamentals.mediawiki/#Exchange_Variables|Exchange Variables]]
** [[fundamentals.mediawiki/#Global_Variables|Global Variables]]
** [[fundamentals.mediawiki/#Asset_Variables|Asset Variables]]
** [[fundamentals.mediawiki/#Market_Variables|Market Variables]]
** [[fundamentals.mediawiki/#Configuration_Data_Request|Configuration Data Request]]
* [[fundamentals.mediawiki/#Epochbased_Order_Matching|Epoch-based Order Matching]]
** [[fundamentals.mediawiki/#Epoch_Time|Epoch Time]]
** [[fundamentals.mediawiki/#Pseudorandom_Order_Matching|Pseudorandom Order Matching]]
* [[fundamentals.mediawiki/#Identities_based_on_Public_Key_Infrastructure_PKI_Key_Pairs|Identification]]
* [[fundamentals.mediawiki/#Blockchain_Interaction|Blockchain Interaction]]
* [[fundamentals.mediawiki/#Adding_New_Assets|Adding New Assets]]

'''[3] [[admin.mediawiki|Distributed Exchange Administration]]''' describes
the tasks required to administer the exchange.

* [[admin.mediawiki/#Exchange_Variables|Exchange Variables]]
* [[admin.mediawiki/#Per_Asset_Variables|Per-asset Variables]]
buck54321 marked this conversation as resolved.
Show resolved Hide resolved

buck54321 marked this conversation as resolved.
Show resolved Hide resolved
'''[4] [[accounts.mediawiki|Client Accounts]]''' details account creation.

* [[accounts.mediawiki/#Step_1_Registration|Registration]]
* [[accounts.mediawiki/#Step_2_Fee_Notification|Fee Notification]]

'''[5] [[orders.mediawiki|Client Order Management]]''' details the different
order types and the client/server workflows required to synchronize the order
book and place orders.

* [[orders.mediawiki/#Connection_Persistence|Connection Persistence]]
* [[orders.mediawiki/#Order_Book_Subscriptions|Order Book Subscriptions]]
* [[orders.mediawiki/#Order_Preparation|Order Preparation]]
** [[orders.mediawiki/#Calculating_Transaction_Fees|Calculating Transaction Fees]]
** [[orders.mediawiki/#Coin_Preparation|Coin Preparation]]
** [[orders.mediawiki/#Order_Commitment|Order Commitment]]
** [[orders.mediawiki/#Order_Signing|Order Signing]]
** [[orders.mediawiki/#Order_ID|Order ID]]
* [[orders.mediawiki/#Order_Types|Order Types]]
** [[orders.mediawiki/#Limit_Order|Limit Order]]
** [[orders.mediawiki/#Market_Order|Market Order]]
*** [[orders.mediawiki/#Market_Buy_Orders|Market Buy Orders]]
** [[orders.mediawiki/#Cancel_Order|Cancel Order]]
* [[orders.mediawiki/#Preimage_Reveal|Preimage Handling]]
* [[orders.mediawiki/#Match_Revocation|Match Revocation]]
* [[orders.mediawiki/#Match_negotiation|Match Negotiation]]
* [[orders.mediawiki/#Trade_Suspension|Trade Suspension]]

'''[6] [[api.mediawiki| Data API]]''' defines http and websocket APIs to browse
trade history.
Expand All @@ -127,5 +125,8 @@ process with a couple of examples to provide a high-level overview.
'''[8] [[community.mediawiki|Community Conduct]]''' describes the system of rules
to which clients interacting with the DEX must adhere.

* [[community.mediawiki/#Rules_of_Community_Conduct|Rules of Community Conduct]]
* [[community.mediawiki/#Penalties|Penalties]]

'''[9] [[references.mediawiki|References]]''' lists references used in the development
of the specification.
29 changes: 25 additions & 4 deletions spec/admin.mediawiki
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ That said, changes to exchange or asset variables will often entail revocation
of all existing orders on a market, so should be done as infrequently as
possible.

'''Exchange Variables'''
==Exchange Variables==

{|
! variable !! relevant section !! units || default
Expand All @@ -27,14 +27,14 @@ possible.
| broadcast timeout || [[fundamentals.mediawiki/#Exchange_Variables|Exchange Variables]] & [[community.mediawiki/#rule-1-clients-must-respond-to-all-preimage-requests|Rule 1]] || milliseconds || 60,000
|}

'''Per-asset Variables'''
==Per-asset Variables==

{|
! variable !! units !! description
|-
| lot size || atoms || the minimum order quantity and the order quantity increment when an asset is the base asset
|-
| rate step || atoms || The minimum price rate and the price rate increment when an asset is the quote asset. [[orders.mediawiki/#Rate_Encoding|message-rate encoding]]
| rate step || atoms || the minimum price rate and the price rate increment when an asset is the quote asset. [[orders.mediawiki/#Rate_Encoding|message-rate encoding]]
|-
| fee rate || atoms/byte || the minimum fee rate for swap transactions
|-
Expand All @@ -45,4 +45,25 @@ possible.

See also [[fundamentals.mediawiki/#Exchange_Variables|Exchange Variables]].

...
==Administration API==

The server will provide an HTTP API for performing various adminstrative tasks.

'''API Endpoints'''
buck54321 marked this conversation as resolved.
Show resolved Hide resolved
{|
! path !! description
|-
| /config || the current DEX configuration. See [[fundamentals.mediawiki/#Configuration_Data_Request|Configuration Data Request]]
|-
| /accounts || lists information about all known accounts
| -
| /accounts/{accountID} || list information about a specific account
| -
| /markets || display status information for all markets
| -
| /market/{marketID} || display status information for a specific market
| -
| /market/{marketID}/suspend || schedule a market suspension at the end of the current epoch
| -
| /unban/{accountID} || clear an account's penalties and re-enable trading
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feelings about the /ban API endpoint? This relates to the previously discussed penalization notification, which should probably be signed by the server and include details of the infraction(s).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was gonna chat with you and @JoeGruffins about that. I see the question is still sort of in the air at #469 (comment). My gut says that we shouldn't even have the option. Is there a good reason?

Copy link
Member

@chappjc chappjc Jun 15, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unforeseen mischief, including hack attempts, etc. There are already a myriad of ways an operator can DoS a user, so making this seamless just makes sense to me. If the issue is sorted out, the unban function exists.

Copy link
Member

@chappjc chappjc Jun 15, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, the ban/unban will probably need to be renamed to suspend/reinstate or similar (eventually, not now), or at least the ban function will need a duration, maybe even option to disallow connection, when we figure out more advanced scoring and temporary suspension mechanics.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. Will just add /ban for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think with the current rules, ban is not so useful. Maybe only to re-ban someone you accidentally unbanned. But new rules may be added in the future, that can't be done programmatically, or that need to be retroactively imposed.

However, it could be abused by the operator. A normal exchange works on trading fees, so banning isn't in the best interest of the operator. But since the DEX is based on an initial fee, unless a user is vocal about it, banning has no impact on the operator. I don't know if this is an actual problem. I think we will find out when the DEX goes into production. Of course, even without this endpoint, an operator can just switch a value in the db and have the same effect. One possible solution is to try to make ban data public, if it is indeed a problem.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

banning has no impact on the operator

This may not be true. I guess having low volume would not get you new users. So there may be enough incentive for this to not be abused. If I were banned for nothing, I certainly wouldn't pay the fee again, I'd find another DEX.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with all of that. My sort of half-baked philosophy on this is that if we need to manually ban someone, then our software isn't doing it's job. But I also recognize that there are bugs, and a learning curve, and that the ability to ban someone who skirts the rules might be critical at some point. Right now, I'm feeling like we should leave it, and just make it one of our goals that nobody ever has to use it.

Copy link
Member

@chappjc chappjc Jun 16, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have it in the code and document it. It could be hacked into the code anyway, the DB could be fiddled with, and firewalls employed for banning, so it should be a supported function. I don't think it make senses to neuter the operator on principle, especially since there will be legit reasons to close an account like this. Consider that a programmatic ban according to the rules also potentially creates a burden for the operator to demonstrate the broken rules. Little changes with a ban function IMO. If the operator closes accounts, their exchange is going to be on everyone's shit list.

|}
8 changes: 4 additions & 4 deletions spec/comm.mediawiki
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,15 @@ Match notification via HTTP polling or other request interval-based methods are
thus not suitable for the DEX system.
Persistent, full-duplex communication is critical to minimizing communication
latencies and wasted bandwidth.
Websockets ([https://tools.ietf.org/html/rfc6455 [4]]) are chosen as the
Websockets ([https://tools.ietf.org/html/rfc6455 [3]]) are chosen as the
buck54321 marked this conversation as resolved.
Show resolved Hide resolved
default and preferred communications protocol for
the DEX exchange API.
In addition to fulfilling the aforementioned needs, Websockets are now a
well-established technology with client software available for integration in
virtually all popular programming languages.

Websocket messages are secured by encryption on Transport Layer
Security (TLS) [https://tools.ietf.org/html/rfc8446 [5]] connections.
Security (TLS) [https://tools.ietf.org/html/rfc8446 [4]] connections.

==Encoding==

Expand Down Expand Up @@ -57,7 +57,7 @@ ID and output index (vout).

In an effort to stay blockchain-protocol agnostic, the DEX accepts
and recognizes the locating information as a single byte-array called the
'''''coin ID''''', with the term '''''coin''''' being defined here as a some
'''''coin ID''''', with the term '''''coin''''' being defined here as some
amount of spendable value that is verifiable on the blockchain.
It is up to backend and wallet developers to decide on how to properly encode the
identifier as a coin ID. As an example, Bitcoin implements
Expand All @@ -66,7 +66,7 @@ big-endian encoded output index as the last 4 bytes.

==Message Protocol==

DEX messaging is JSON-formatted [https://tools.ietf.org/html/rfc8259 [6]].
DEX messaging is JSON-formatted [https://tools.ietf.org/html/rfc8259 [5]].
All messages, regardless of originating party, use a common top-level
structure called a '''Message'''.

Expand Down
49 changes: 28 additions & 21 deletions spec/fundamentals.mediawiki
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ In particular, for ''lot size'' '''''l''''', the requested ''order quantity'',
Every asset is assigned a unique integer ID that will be used to identify the
asset in serialized structures. Whenever possible, the ID is the same as the
BIP-0044 registered coin type index
[https://github.com/satoshilabs/slips/blob/master/slip-0044.md [7]].
[https://github.com/satoshilabs/slips/blob/master/slip-0044.md [6]].

When the asset is the quote asset, a '''price increment''' is enforced.
The '''rate''', '''''r''''', of a limit order must be an integer multiple of the
Expand Down Expand Up @@ -104,7 +104,7 @@ respond with its current configuration.
|-
| btimeout || int || the broadcast timeout
|-
| regfee || int || registration fee (Decred atoms)
| fee || int || registration fee (Decred atoms)
|-
| regfeeconfirms || int || required confirmations for the registration fee payment transaction
|-
Expand Down Expand Up @@ -138,19 +138,23 @@ respond with its current configuration.
'''Market object'''

{|
! field !! type !! description
! field !! type !! description
|-
| name || string || market name
| name || string || market name
|-
| base || int || base asset ID
| base || int || base asset ID
|-
| quote || int || quote asset ID
| quote || int || quote asset ID
|-
| epochlen || int || the [[#Epochbased_Order_Matching|epoch duration]] (milliseconds)
| epochlen || int || the [[#Epochbased_Order_Matching|epoch duration]] (milliseconds)
|-
| startepoch || int || the epoch number at which trading did or will commence. May be in the future e.g. [[orders.mediawiki/#Trade_Suspension|after maintenance]]
| buybuffer || float || the [[orders.mediawiki/#Market_Buy_Orders|market buy buffer]]
|-
| buybuffer || float || the [[orders.mediawiki/#Market_Buy_Orders|market buy buffer]]
| startepoch || int || the epoch number at which trading did or will commence. May be in the future e.g. [[orders.mediawiki/#Trade_Suspension|after maintenance]]
|-
| finalepoch || int || the epoch number at which trading will be suspended. Only present when a suspension is scheduled
|-
| persistbook || bool || whether or not booked orders will be persisted through a scheduled suspension. Only present when a suspension is scheduled
|}

==Fees==
Expand Down Expand Up @@ -314,17 +318,20 @@ broadcast transactions to each of the blockchain networks involved in
assets supported.
While operation via a surrogate blockchain data service such as a block explorer
is potentially feasible, it would entail significant security risks.
Initial development will require a direct connection to full nodes of each
asset's blockchain.

Adding support for an asset is accomplished by implementing a particular
[https://golang.org/ Go] interface.
The interface is exported and compiled with <code>-buildmode=plugin</code> to enable
dynamic importing by the exchange operator.
The operator will decide which plugins to import via configuration file.
As the interface is expected to evolve greatly during initial implementation,
the specifics of the interface structure will be released at a later time.

With the exception of Decred and Bitcoin, which will be implemented during
Initial development will require that the server have a direct connection to
full nodes of each asset's blockchain.

==Adding New Assets==

Adding support for an asset is accomplished by writing a
[https://golang.org/ Go] package with types that implement a particular set of
interfaces, defined [https://github.com/decred/dcrdex/blob/master/server/asset/common.go here]
and [https://github.com/decred/dcrdex/blob/master/server/asset/driver.go here].
There are then two ways to import the asset backend into the server software.

# The package is compiled with <code>-buildmode=plugin</code> and imported at runtime by specifying the plugin in the server configuration.
# The backend is added to the dcrdex repository, and imported directly at compile time.

With the exception of a small handful of assets which will be implemented during
initial phases of DEX development, it is expected that development communities
will release their own appropriately vetted plugins.
Loading