Bitfinex FIX Gateway
If one does not already exist, install a Go environment by following these instructions. If GOPATH is set to ~/go and the bfxfixgw repository is cloned to ~/go/src/github.com/bitfinexcom/bfxfixgw, then the following simple go command is all that is needed if issued from the ~/go directory. Both bfxfixgw and fix_client will be installed to ~/go/bin after issuing the command.
Obtain all additional sources, build, and install:
go get ./...
Building Source Changes
Local bfxfixgw source changes can be built by issuing the following commands from the ~/go/src/github.com/bitfinexcom/bfxfixgw directory. The install command will copy the latest version of bfxfixgw and fix_client to ~/go/bin.
go build ./...
And install binaries:
go install ./...
The FIX gateway can operate in order routing mode, market data mode, or both. Order routing & market data FIX endpoints must be separate with distinct session IDs.
DEBUG=1to enable debug logging
FIX_SETTINGS_DIRECTORY=./configto read the configs from the given directory,
./configis the default directory.
Sessions must be known to the FIX gateway prior to startup. A FIX gateway can manage any number of sessions. Each FIX session will create 1 websocket proxy connection.
The market data FIX service does not support resend requests or replaying FIX messages. When connecting to the market data FIX endpoint, a FIX initiator must have the correct sequence number, or a lower than expected sequence number.
The order routing service strictly tracks sequence numbers and does support message storage. A FIX initiator can send
ResetSeqNumFlag=Y on Logon to reset session sequence numbers.
FIX Configuration Examples
Example service FIX session configuration for a market data service:
[DEFAULT] SenderCompID=BFXFIX ResetOnLogon=Y ReconnectInterval=60 FileLogPath=tmp/md_service/log SocketAcceptPort=5001 StartTime=00:05:00 StartDay=Sun EndTime=00:00:00 EndDay=Sun [SESSION] TargetCompID=EXORG_MD BeginString=FIX.4.2 DefaultApplVerID=FIX.4.2 HeartBtInt=30
Example service FIX session configuration for an order routing service:
[DEFAULT] SenderCompID=BFXFIX ReconnectInterval=60 FileLogPath=tmp/ord_service/log FileStorePath=tmp/ord_service/data SocketAcceptPort=5002 StartTime=00:05:00 StartDay=Sun EndTime=00:00:00 EndDay=Sun [SESSION] TargetCompID=EXORG_ORD BeginString=FIX.4.2 DefaultApplVerID=FIX.4.2 HeartBtInt=30
To startup the gateway in verbose mode (-v) with both order routing and market data endpoints (staging configuration) run the following command:
FIX_SETTINGS_DIRECTORY=~/go/src/github.com/bitfinexcom/bfxfixgw/conf/integration_test/service ~/go/bin/bfxfixgw -v -orders -ordcfg "orders_fix42.cfg" -md -mdcfg "marketdata_fix42.cfg" -rest "https://api.bitfinex.com/v2/" -ws "wss://api.bitfinex.com/ws/2"
FIX session information must be obtained prior to a FIX client establishing a connection. The pre-determined TargetCompID, SenderCompID, and FIX version strings should be configured in the FIX client configuration.
Once sessions are configured, a FIX client can authenticate by adding the Bitfinex User ID, API key, and API secret into the FIX
35=A Logon message body:
|Field||FIX Tag #||Description|
|BfxApiKey||20000||User's API Key|
|BfxApiSecret||20001||User's API Secret|
|BfxUserID||20002||User's Bfx ID|
Optionally, a user can request all session-level orders be canceled when the FIX session is disconnected by setting the following FIX tag
|Field||FIX Tag #||Description|
|CancelOnDisconnect||8013||Cancel session orders on FIX disconnect|
Note: FIX clients should use separate API keys for market data and order routing FIX endpoints.
These tags are supported by the gateway's default data dictionary. An example staging logon message (
SOH replaced with
Testing using fix_client
The project includes a test client utility called fix_client. fix_client is a simple gateway client that demonstrates a subset of gateway functionality. The client currently supports sending and canceling orders via the gateway. Simply run the client, issue a root command (either nos or cxl), and then provide additional request parameters as prompted.
The fix_client takes a quickfix configuration file located here ~/go/src/github.com/bitfinexcom/bfxfixgw/conf/integration_test/client/orders_fix42.cfg. To successfully connect to BFX using the gateway, you must change the following parameters to match the API access of your account. Please note the security risk of adding credentials to this file in plain text. You may want to remove these values after you are done with testing using fix_client
ApiKey=[User's API Key] ApiSecret=[User's API Secret] BfxUserID=[User's Bfx ID]
Running fix_client Example
Here is an example of using fix_client to place a market order. All outgoing and incoming FIX traffic is logged. You can decode the FIX manually or using your favorite FIX parser. Please be careful to avoid sending FIX logs that contain private keys to online parsers. Some logging has been stripped out for clarity and user input is bold.
./fix_client -cfg ../src/github.com/bitfinexcom/bfxfixgw/conf/integration_test/client/orders_fix42.cfg
-> New Order Single
Enter ClOrdID (integer):
Enter order type:
Options? (hidden, postonly, fok): [FIX Client] MockFix.ToApp (outgoing): 8=FIX.4.2|9=126|35=D|34=547|49=EXORG_ORD|52=20180504-16:45:04.697|56=BFXFIX|11=1|21=3|38=0.0500|40=1|54=1|55=tBTCUSD|60=20180504-16:45:03.518|10=249|
[FIX Client] MockFix.FromApp (incoming): 8=FIX.4.2|9=221|35=8|34=550|49=BFXFIX|52=20180504-16:45:04.804|56=EXORG_ORD|1=connamara|6=0.00|11=1|14=0.0000|17=43a98b1d-0312-41cf-9678-33e268e9bff9|20=3|32=0.0000|37=1149703695|38=0.0500|39=0|40=1|54=1|55=tBTCUSD|59=1|150=0|151=0.0500|10=007|
<2018-05-04 16:45:05.823298994 +0000 UTC, FIX.4.2:EXORG_ORD->BFXFIX, incoming>
[FIX Client] MockFix.FromApp (incoming): 8=FIX.4.2|9=250|35=8|34=551|49=BFXFIX|52=20180504-16:45:05.818|56=EXORG_ORD|1=connamara|6=850.00|11=1|12=0.0001|13=3|14=0.0500|17=1cfb00fc-9ea6-4dca-8d9f-a5cce1cc0959|20=3|31=850.0000|32=0.0500|37=1149703695|38=0.0500|39=2|40=1|54=1|55=tBTCUSD|59=1|150=2|151=0.0000|10=182
Market Data Distribution
The FIX gateway service may be configured to distribute market data. Starting the process with
-md will enable market data distribution, configured by the
tBTCUSD top-of-book Precision0 updates:
tETHUSD full raw book updates at 25 price levels:
35=W book snapshot (for the first tBTCUSD request):
35=X market data incremental update (for the first tBTCUSD request):
35=X trade incremental update (for the first tBTCUSD request):
Order routing can be enabled with the
-ordcfg flags on startup.
The following table lists Bitfinex order type support in the FIX gateway:
|Order Type||FIX 4.2|
Below is a table of various Bitfinex order features and their FIX
35=D NewOrderSingle tags:
|Bitfinex Order Feature||FIX Tag||FIX Tag Value|
|Hidden||DisplayMethod (1084)||Undisclosed (4)|
|Post-Only*||ExecInst (18)||Participate don't initiate (6)|
|Fill or Kill||TimeInForce (59)||Fill or Kill (4)|
* Post-Only orders are considered to have a Good-till-Cancel time in force.
For a trailing stop order:
|Trailing Stop Feature||FIX Tag||FIX Tag Value|
|Order Type||OrdType (40)||Stop (3) or Stop Limit (4)|
|Execute as Trailing Peg||ExecInst (18)||Primary Peg (R)|
|Trailing Peg Value||PegOffsetValue (211)||Price*|
* The trailing stop price should be in the same units as a Price (44) or StopPx (99).
e.g. For a trailing stop order where the stop should trail the market by $5.25, the following FIX tags should be set:
Send limit new order single:
35=8 execution report for working order:
Send market new order single:
35=8 execution report for a partial fill:
Receive the alst FIX
35=8 execution report for a full fill:
Cancel working limit order:
35=8 pending cancel acknowledgement:
8=FIX.4.2|9=296|35=8|34=40|49=BFXFIX|52=20180417-22:29:11.290|56=EXORG_ORD|1=connamara|6=0.00|11=2000|14=0.0000|17=d067ff27-4182-454f-8c51-bd08c09b5730|20=3|37=1149698709|38=0.1000|39=6|40=2|44=20000.0000|54=2|55=tBTCUSD|58=Submitted for cancellation; waiting for confirmation (ID: 1149698709).|150=6|151=0.1000|10=112|
35=8 cancel acknowledgement:
Order State Details
When receiving a Bitfinex Order update object (on, ou, oc), the following tables demonstrate rules for mapping FIX tag
Order Update Status Mappings
|BFX Order State||FIX OrdStatus Code||Order Status|
|PARTIALLY FILLED||1||PARTIALLY FILLED|
Executions are received as
te TradeExecution messages and
tu TradeUpdate messages. TradeExecution messages come first, but generally omit the order type, original price, fee, and some other fields. The gateway processes
tu TradeUpdate messages as executions. When receiving a TradeUpdate, MsgType
8 ExecutionReports are generated following these rules:
- 1 TradeUpdate message will create 1 ExecutionReport
- Tag 37 OrderID is derived from the
- An order's CID is mapped to a tag 11 ClOrdID
- The gateway maintains an in-memory cache of ClOrdID -> order information, including:
- Original order details (symbol, account, price, quantity, type, side)
- Calculated state details (average fill price, total open quantity, total filled quantity)
- Executions related to the original order
- Cancel details related to a ClOrdID (original order ID, symbol, account, ClOrdID, and side)
- When receiving order state updates (rejection, fill, cancel acknowledgement), the cache must be referenced to provide FIX-required details
- When receiving a TradeUpdate, if cached details indicate the incoming TradeUpdate would fully fill the order, the gateway will publish an ExecutionReport with an OrdStatus of FILLED.
Synthetic Order State Message Mappings
on-req generally maps to PENDING NEW, with an exception for market orders, which do not receive subsequent
on ack working messages.
|BFX Order Type||Incoming BFX Message||FIX OrdStatus Code||Order Status||Notes|
|EXCHANGE MARKET||n (on-req)||0||NEW||Market orders do not receive
|EXCHANGE LIMIT||n (on-req)||0||PENDING NEW|
|EXCHANGE LIMIT||ou||Depends on status||Depends on status|
Below are a few common issues with simple procedures to resolve runtime problems:
FIX client won't log on
If the session has been rolled over or restarted, a FIX initiator may have a higher sequence number than its acceptor, which is an error condition. Simply reset the FIX initiator's sequence number (deleting the sequence store file in QuickFIX works) and the initiator should no longer disconnect on logon.
FIX logs on but does not process requests
Ensure the correct endpoint is configured for use. (i.e. MarketDataRequests should be sent to the FIX Market Data endpoint, and NewOrderSingle messages should be sent to the order routing endpoint).
Average prices & working quantities for out-of-session working orders
Orders placed outside of the FIX trading session may sometimes have incorrect filled quantities (FIX tag 14
CumQty), remaining quantities (FIX tag 151
LeavesQty), and/or average prices (FIX tag 6
AvgPx). The gateway uses an order's individual executions to calculate average prices, filled quantities, and remaining quantities on demand. The gateway does not receive working order execution details in order snapshots on client logon. Therefore, the gateway will attempt to go out-of-band, using the authenticated REST API, to fetch execution details for working orders. Out of band requests use the same API key as the orders websocket connection, and may suffer from internet routing race conditions. Occasionally the order trades endpoint will unexpectedly return an empty set, therefore execution details may not always be available to the gateway.
Internet routing race conditions could be solved with a future enhancement to the gateway to use yet another (third) API key for authenticated out-of-band REST requests.
Execution reports out of order
To preserve fee information,
tu API messages are used to populate execution reports. However, the API publishes
tu messages out of order, so corresponding ERs may also be out of order.
Immediate or Cancel collapses into Fill or Kill time in force
If an order is sent with an Immediate or Cancel time in force, the order will be mapped as a Bitfinex fill or kill limit order. Corresponding execution reports will indicate the order was placed as a fill or kill limit and not an IOC limit order.
Unsolicited trailing stop Execution Report missing trailing peg
If a trailing stop order was placed outside of the FIX session, a
39=0 NEW Execution Report will be missing the trailing stop peg price. The Bitfinex API currently does not return trailing stop peg prices on order new notification acknowledgements, but instead lists the calculated stop price, which is included in tag
99 StopPx on the
39=0 NEW ExecutionReport. Subsequent ExecutionReports related to the unsolicited trailing stop may also be missing the peg price until the gateway's cache is updated from the Bitfinex API.