Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugable coins API #2168

Merged
merged 90 commits into from May 27, 2019
Merged
Show file tree
Hide file tree
Changes from 84 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
95f74f8
Peatio::Transaction & Peatio::Block
ysv Apr 9, 2019
d35b306
Peatio::BlockchainAPI & Peatio::Blockchain::Abstract
ysv Apr 10, 2019
6d107b3
BlockchainService2
ysv Apr 10, 2019
4411ca5
Minor refactoring of BlockchainService2
ysv Apr 11, 2019
d6fc589
Raname blockchain_service_2
ysv Apr 11, 2019
d6e8896
Prototype specs
ysv Apr 11, 2019
94ae110
Document Peatio::Blockchain::Abstract
ysv Apr 11, 2019
9c4ff51
FIRST WORKING SPEC111111111
Apr 11, 2019
5387b10
Beautify specs
ysv Apr 11, 2019
0f39eeb
Add specs for process block with deposits
Apr 11, 2019
d15a6f2
Refactor specs
ysv Apr 12, 2019
0207b59
Make constuctor abstract & add doc
ysv Apr 12, 2019
ce47f56
WIP
Apr 12, 2019
a6c57a1
Add withdraws specs
Apr 12, 2019
2114eb0
Finishing specs for blockchain_service
Apr 15, 2019
011b513
Call deposit_collection_fees for each deposit
Apr 15, 2019
35fcacb
Refactor specs for blockservice
Apr 15, 2019
16e3888
Add spec for several blocks
Apr 15, 2019
da2fd84
Finalize specs
Apr 16, 2019
78b5397
Implement Bitcoin::Blockchain & Bitcoin::Client (#2171)
ysv Apr 16, 2019
cbf464a
Rename specs
ysv Apr 16, 2019
b643121
Add todo
ysv Apr 16, 2019
d9fb8de
Fix blockchain2 specs
ysv Apr 16, 2019
73f4509
Add methods to BlockchainService2 for compitability with v1 Add #bloc…
ysv Apr 16, 2019
db15636
Remove legacy supports_cash_addr_format? && case_sensitive? from new …
ysv Apr 16, 2019
805e345
Fix specs after adding redis
Apr 17, 2019
1578fca
Finalize Abstract doc & Bitcoin specs
ysv Apr 17, 2019
9fca735
Add missing configure step to BlockchainService2
ysv Apr 17, 2019
60afd77
Implement Ethereum::Blockchain & Ethereum::Client (#2174)
Apr 23, 2019
946ec7a
Fix specs for BlockchainService2
Apr 23, 2019
ebe6cbe
[WIP] WalletService2 & Peatio::Wallet::Abstract design
ysv Apr 17, 2019
634782d
Implement collect_deposit in WalletService2
ysv Apr 18, 2019
51cb05d
Finalize WalletService2
ysv Apr 18, 2019
489ac15
Beautify
ysv Apr 19, 2019
db81ef6
[WIP] Wallet::Abstract doc
ysv Apr 19, 2019
ce0a5b6
Finalize Wallet::Abstract documentation. Move to using Peatio::Transa…
ysv Apr 22, 2019
ec50872
Add fake blockchain, currency & wallet to factories
ysv Apr 22, 2019
3bd0fdf
Symbolize gataway & client in Wallet & Blockchain Service v2
ysv Apr 22, 2019
5c60a1e
Beautify blockchain service 2 specs
ysv Apr 22, 2019
ac614bb
Start WalletService2 specs
ysv Apr 22, 2019
1c5439b
:spread_between_wallets testing
ysv Apr 22, 2019
03a6b2a
Minor Updates
ysv Apr 22, 2019
98bf356
Add supported settings to Wallet & create_transaction! note
ysv Apr 23, 2019
c848274
Bitcoin::Wallet implementation
ysv Apr 23, 2019
7d2077b
WIP Bitcoin::Wallet specs
ysv Apr 23, 2019
c095cb4
Finalize Bitcoin::Wallet & specs
ysv Apr 23, 2019
85f4fce
Fix case when all wallets are full and negative min_collection_amount
ysv Apr 23, 2019
416091c
WalletService2 specs
Apr 23, 2019
9990730
Fix min_collection issue
ysv Apr 24, 2019
8ed9d6b
Handle load_balance error
ysv Apr 24, 2019
23023a6
Add spread_deposit specs
ysv Apr 24, 2019
fc870be
Finalize load_balance stuff
ysv Apr 25, 2019
7b3c6c4
Revert Wallet GATEWAY validation
Apr 25, 2019
663916e
Define Exception hierarchy for Blockchain & Wallet (#2183)
ysv Apr 25, 2019
a2874bf
Remove BlockchainAddress, use Hash instead; Fix collection publish; …
Apr 26, 2019
85fcfcd
Add load_balance & load_balance_of_address (#2185)
ysv Apr 26, 2019
bb81ce0
Bitcoin::Blockchain load_balance_of_address & Bitcoin::Wallet load_ba…
ysv Apr 26, 2019
62b3769
Move to Blockchain & Wallet API v2 (#2187)
ysv Apr 26, 2019
9c352ea
Add Ethereum::Wallet specs (#2188)
Apr 30, 2019
930e947
Update Bitcoin::WalletClient (#2192)
May 2, 2019
acb46bb
Update blockchain daemon with BlockchainService2 (#2193)
May 2, 2019
f376e4d
Make daemons compatible with WalletService v2 (#2190)
ysv May 2, 2019
a74340c
Local daemons bugfixing and improvements (#2195)
ysv May 6, 2019
1873fbd
Remove Legacy source code & finalize TODOs (#2199)
ysv May 6, 2019
d5f3251
Move core code to peatio gem (#2201)
ysv May 6, 2019
450e6a7
Add step column to blockchain model, update Blockchain daemon (#2194)
May 7, 2019
38a24c9
Bugfixing and improvements for Blockchain/WaletService (#2200)
May 7, 2019
3182ef6
Add rescue for clients
May 7, 2019
e94597a
Skip empty payment_addresses on deposit filter
May 7, 2019
df30ea9
Rewrite rescue in ethereum/bitcoin client
May 7, 2019
646b015
Blockchain & Wallet final refactoring (#2203)
ysv May 7, 2019
1b93a86
Compact options before passing to Wallet & Blockchain
ysv May 7, 2019
29b9873
Fetch BlockchainAPI clients from Peatio::Registry
May 8, 2019
423fc96
Delete blockchain clients from view
May 8, 2019
e00db79
Beautify blockchain.rb
ysv May 8, 2019
48e7f1c
Update peatio gem
May 13, 2019
54ffdd5
Peatio-litecoin gem integration with minor updates. Coin plugin devel…
ysv May 16, 2019
378ff5e
Clear client state after configure in all Blockchain & Wallet
ysv May 16, 2019
6a2c5fc
Use success? & failed? for Peatio::Transaction
ysv May 16, 2019
257d5cf
Update peatio gem to 0.5.0 & peatio-litecoin to 0.1.1
ysv May 16, 2019
8504f14
Temporary switch to rubykube/peatio-litecoin
ysv May 16, 2019
6dd4325
Temporary switch to github peatio-core
ysv May 21, 2019
0d42e44
Fix management trades specs
ysv May 21, 2019
db88aec
Update peatio to 0.6.0
ysv May 21, 2019
5a1008e
Use only enabled currencies in BlockchainService
ysv May 23, 2019
faaf16c
Use peatio branch: transaction-string-status
ysv May 23, 2019
bca8026
Fix wallet_service
ysv May 23, 2019
f61721f
Fix other specs
ysv May 23, 2019
0392e00
Fix deposit_collection_fees
ysv May 23, 2019
7f0a23e
Bump peatio & peatio-litecoin version
ysv May 27, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions Dockerfile
Expand Up @@ -34,7 +34,7 @@ WORKDIR $APP_HOME
# Install dependencies defined in Gemfile.
COPY Gemfile Gemfile.lock $APP_HOME/
RUN mkdir -p /opt/vendor/bundle \
&& chown -R app:app /opt/vendor \
&& chown -R app:app /opt/vendor $APP_HOME \
&& su app -s /bin/bash -c "bundle install --path /opt/vendor/bundle"

# Copy application sources.
Expand Down Expand Up @@ -62,7 +62,7 @@ CMD ["bundle", "exec", "puma", "--config", "config/puma.rb"]
FROM base

# Copy Gemfile.plugin for installing plugins.
COPY Gemfile.plugin $APP_HOME
COPY Gemfile.plugin Gemfile.lock $APP_HOME/

# Install plugins.
RUN bundle install --path /opt/vendor/bundle
3 changes: 2 additions & 1 deletion Gemfile
Expand Up @@ -35,6 +35,7 @@ gem 'grape_logging', '~> 1.8.0'
gem 'rack-attack', '~> 5.4.2'
gem 'easy_table', '~> 0.0.10'
gem 'faraday', '~> 0.15.4'
gem 'better-faraday', '~> 1.0.5'
gem 'jwt', '~> 2.1.0'
gem 'email_validator', '~> 1.6.0'
gem 'validate_url', '~> 1.0.4'
Expand All @@ -51,7 +52,7 @@ gem 'jwt-multisig', '~> 1.0.0'
gem 'cash-addr', '~> 0.2.0', require: 'cash_addr'
gem 'digest-sha3', '~> 1.1.0'
gem 'scout_apm', '~> 2.4', require: false
gem 'peatio', '~> 0.4.5'
gem 'peatio', '~> 0.6.0'
gem 'rack-cors', '~> 1.0.2', require: false
gem 'env-tweaks', '~> 1.0.0'
gem 'vault', '~> 0.12', require: false
Expand Down
25 changes: 21 additions & 4 deletions Gemfile.lock
@@ -1,3 +1,14 @@
GIT
remote: https://github.com/rubykube/peatio-litecoin
revision: cfaccc9c8b98c439a62b2ebc961c66b0932582d3
specs:
peatio-litecoin (0.1.0)
activesupport (~> 5.2.3)
better-faraday (~> 1.0.5)
faraday (~> 0.15.4)
memoist (~> 0.16.0)
peatio (~> 0.6.0)

GEM
remote: https://rubygems.org/
specs:
Expand Down Expand Up @@ -69,6 +80,9 @@ GEM
ice_nine (~> 0.11.0)
thread_safe (~> 0.3, >= 0.3.1)
base58 (0.2.3)
better-faraday (1.0.5)
activesupport (>= 4.0, < 6.0)
faraday (~> 0.12)
bootsnap (1.4.2)
msgpack (~> 1.0)
bootstrap (4.3.1)
Expand All @@ -80,7 +94,7 @@ GEM
activesupport (>= 3.0.0)
uniform_notifier (~> 1.11)
bump (0.8.0)
bunny (2.14.1)
bunny (2.14.2)
amq-protocol (~> 2.3, >= 2.3.0)
byebug (11.0.0)
cancancan (2.3.0)
Expand Down Expand Up @@ -220,7 +234,7 @@ GEM
metaclass (~> 0.0.1)
msgpack (1.2.9)
multi_json (1.13.1)
multipart-post (2.0.0)
multipart-post (2.1.1)
mustermann (1.0.3)
mustermann-grape (1.0.0)
mustermann (~> 1.0.0)
Expand All @@ -232,7 +246,8 @@ GEM
parser (2.6.0.0)
ast (~> 2.4.0)
passgen (1.0.2)
peatio (0.4.5)
peatio (0.6.0)
activemodel (~> 5.2.3)
amqp
bunny
clamp
Expand Down Expand Up @@ -400,6 +415,7 @@ DEPENDENCIES
annotate (~> 2.7.4)
api-pagination (~> 4.8.2)
arel-is-blank (~> 1.0.0)
better-faraday (~> 1.0.5)
bootsnap (>= 1.1.0)
bootstrap (~> 4.3.1)
bullet (~> 5.9)
Expand Down Expand Up @@ -441,7 +457,8 @@ DEPENDENCIES
mocha (~> 1.8)
mysql2 (~> 0.5.2)
passgen (~> 1.0.2)
peatio (~> 0.4.5)
peatio (~> 0.6.0)
peatio-litecoin!
pry-byebug (~> 3.7)
puma (~> 3.12.0)
rack-attack (~> 5.4.2)
Expand Down
3 changes: 3 additions & 0 deletions Gemfile.plugin
Expand Up @@ -2,3 +2,6 @@ source 'https://rubygems.org'
git_source(:github) { |repo_slug| "https://github.com/#{repo_slug}" }

# List your plugins here.

gem 'peatio-litecoin', github: 'rubykube/peatio-litecoin'
# gem 'peatio-litecoin', '~> 0.1.1'
1 change: 1 addition & 0 deletions README.md
Expand Up @@ -44,6 +44,7 @@ You must know what you're doing, there's no shortcut. Please get prepared before
* Designed as high performance crypto currency exchange
* Built-in high performance matching-engine
* Built-in multiple wallet support (e.g. deposit, hot, warm and cold)
* Built-in [plugable coin API](docs/coins/development.md)
* Build-in Management API - server-to-server API with high privileges
* Build-in RabbitMQ Event API
* Usability and scalability
Expand Down
1 change: 1 addition & 0 deletions app/controllers/admin/blockchains_controller.rb
Expand Up @@ -49,6 +49,7 @@ def permitted_blockchain_attributes
client
server
height
step
min_confirmations
explorer_address
explorer_transaction
Expand Down
19 changes: 15 additions & 4 deletions app/models/blockchain.rb
Expand Up @@ -7,10 +7,20 @@ class Blockchain < ApplicationRecord

validates :key, :name, :client, presence: true
validates :status, inclusion: { in: %w[active disabled] }
validates :height, numericality: { greater_than_or_equal_to: 1, only_integer: true }
validates :min_confirmations, numericality: { greater_than_or_equal_to: 1, only_integer: true }
validates :height,
:min_confirmations,
:step,
numericality: { greater_than_or_equal_to: 1, only_integer: true }
validates :server, url: { allow_blank: true }

scope :active, -> { where(status: :active) }

class << self
def clients
Peatio::Blockchain.registry.adapters.keys
end
end

def explorer=(hash)
write_attribute(:explorer_address, hash.fetch('address'))
write_attribute(:explorer_transaction, hash.fetch('transaction'))
Expand All @@ -21,12 +31,12 @@ def status
end

def blockchain_api
BlockchainClient[key]
BlockchainService.new(self)
end
end

# == Schema Information
# Schema version: 20180708171446
# Schema version: 20190502103256
#
# Table name: blockchains
#
Expand All @@ -36,6 +46,7 @@ def blockchain_api
# client :string(255) not null
# server :string(255)
# height :integer not null
# step :integer default(6), not null
# explorer_address :string(255)
# explorer_transaction :string(255)
# min_confirmations :integer default(6), not null
Expand Down
11 changes: 10 additions & 1 deletion app/models/currency.rb
Expand Up @@ -83,7 +83,7 @@ def types
# Allows to dynamically check value of code:
#
# code.btc? # true if code equals to "btc".
# code.xrp? # true if code equals to "xrp".
# code.eth? # true if code equals to "eth".
#
def code
id&.inquiry
Expand All @@ -101,6 +101,15 @@ def as_json(*)
fiat: fiat? }
end

def to_blockchain_api_settings
# We pass options are available as top-level hash keys and via options for
# compatibility with Wallet#to_wallet_api_settings.
opt = options.compact.deep_symbolize_keys
opt.deep_symbolize_keys.merge(id: id,
base_factor: base_factor,
options: opt)
end

def summary
locked = Account.with_currency(code).sum(:locked)
balance = Account.with_currency(code).sum(:balance)
Expand Down
33 changes: 26 additions & 7 deletions app/models/deposit.rb
Expand Up @@ -4,6 +4,8 @@
class Deposit < ApplicationRecord
STATES = %i[submitted canceled rejected accepted collected].freeze

serialize :spread, Array

include AASM
include AASM::Locking
include BelongsToCurrency
Expand Down Expand Up @@ -50,6 +52,22 @@ class Deposit < ApplicationRecord
end
end

def spread_to_transactions
spread.map { |s| Peatio::Transaction.new(s) }
end

def spread_between_wallets!
return false if spread.present?

deposit_wallet = Wallet.active.deposit.find_by(currency_id: currency_id)
spread = WalletService.new(deposit_wallet).spread_deposit(self)
update!(spread: spread.map(&:as_json))
end

def spread
super.map(&:symbolize_keys)
end

def account
member&.ac(currency)
end
Expand Down Expand Up @@ -85,12 +103,12 @@ def plus_funds
end

def collect!(collect_fee = true)
if coin?
if currency.is_erc20? && collect_fee
AMQPQueue.enqueue(:deposit_collection_fees, id: id)
else
AMQPQueue.enqueue(:deposit_collection, id: id)
end
return unless coin?

if collect_fee
AMQPQueue.enqueue(:deposit_collection_fees, id: id)
else
AMQPQueue.enqueue(:deposit_collection, id: id)
end
end

Expand Down Expand Up @@ -126,7 +144,7 @@ def record_complete_operations!
end

# == Schema Information
# Schema version: 20180925123806
# Schema version: 20190426145506
#
# Table name: deposits
#
Expand All @@ -142,6 +160,7 @@ def record_complete_operations!
# block_number :integer
# type :string(30) not null
# tid :string(64) not null
# spread :string(1000)
# created_at :datetime not null
# updated_at :datetime not null
# completed_at :datetime
Expand Down
3 changes: 2 additions & 1 deletion app/models/deposits/coin.rb
Expand Up @@ -32,7 +32,7 @@ def as_json_for_event_api
end

# == Schema Information
# Schema version: 20180925123806
# Schema version: 20190426145506
#
# Table name: deposits
#
Expand All @@ -48,6 +48,7 @@ def as_json_for_event_api
# block_number :integer
# type :string(30) not null
# tid :string(64) not null
# spread :string(1000)
# created_at :datetime not null
# updated_at :datetime not null
# completed_at :datetime
Expand Down
3 changes: 2 additions & 1 deletion app/models/deposits/fiat.rb
Expand Up @@ -14,7 +14,7 @@ def charge!
end

# == Schema Information
# Schema version: 20180925123806
# Schema version: 20190426145506
#
# Table name: deposits
#
Expand All @@ -30,6 +30,7 @@ def charge!
# block_number :integer
# type :string(30) not null
# tid :string(64) not null
# spread :string(1000)
# created_at :datetime not null
# updated_at :datetime not null
# completed_at :datetime
Expand Down
20 changes: 18 additions & 2 deletions app/models/wallet.rb
Expand Up @@ -11,7 +11,6 @@ class Wallet < ApplicationRecord
ENUMERIZED_KINDS = { deposit: 100, fee: 200, hot: 310, warm: 320, cold: 330 }.freeze
enumerize :kind, in: ENUMERIZED_KINDS, scope: true

GATEWAYS = %w[bitcoind bitcoincashd litecoind parity geth dashd rippled bitgo].freeze
SETTING_ATTRIBUTES = %i[ uri
secret
bitgo_test_net
Expand All @@ -20,6 +19,8 @@ class Wallet < ApplicationRecord
bitgo_rest_api_root
bitgo_rest_api_access_token ].freeze

NOT_AVAILABLE = 'N/A'.freeze

include BelongsToCurrency

store :settings, accessors: SETTING_ATTRIBUTES, coder: JSON
Expand All @@ -30,7 +31,8 @@ class Wallet < ApplicationRecord
validates :address, presence: true

validates :status, inclusion: { in: %w[active disabled] }
validates :gateway, inclusion: { in: GATEWAYS }

validates :gateway, inclusion: { in: ->(_){ Wallet.gateways.map(&:to_s) } }

validates :nsig, numericality: { greater_than_or_equal_to: 1, only_integer: true }
validates :max_balance, numericality: { greater_than_or_equal_to: 0 }
Expand All @@ -48,6 +50,10 @@ class Wallet < ApplicationRecord
end

class << self
def gateways
Peatio::Wallet.registry.adapters.keys
end

def kinds(options={})
ENUMERIZED_KINDS
.yield_self do |kinds|
Expand Down Expand Up @@ -75,6 +81,16 @@ def kinds(options={})
end
end

def current_balance
WalletService.new(self).load_balance!
rescue BlockchainService::BalanceLoadError
NOT_AVAILABLE
end

def to_wallet_api_settings
settings.compact.deep_symbolize_keys.merge(address: address)
end

def wallet_url
blockchain.explorer_address.gsub('#{address}', address) if blockchain
end
Expand Down
18 changes: 10 additions & 8 deletions app/models/withdraws/coin.rb
Expand Up @@ -23,15 +23,17 @@ class Coin < Withdraw
end

def audit!
wallet = Wallet.active.deposit.find_by(currency_id: currency_id)
inspection = WalletClient[wallet].inspect_address!(rid)
# TODO: Revert this inspection with WalletService/BlockchainService v2.
# wallet = Wallet.active.deposit.find_by(currency_id: currency_id)
# inspection = WalletClient[wallet].inspect_address!(rid)

if inspection[:is_valid] == false
Rails.logger.info { "#{self.class.name}##{id} uses invalid address: #{rid.inspect}" }
reject!
else
super
end
# if inspection[:is_valid] == false
# Rails.logger.info { "#{self.class.name}##{id} uses invalid address: #{rid.inspect}" }
# reject!
# else
# super
# end
super
end

def as_json(*)
Expand Down