[0.25 XMR Bounty] [Proposal] Control websocket for receiving a price feed and pushing logging events

[cimble-code]

I think it might be useful to allow the ASB operator to specify a custom control websocket URL. The ASB could then connect to this websocket to receive a custom price feed and send logging events (swap started, bitcoin was locked by peer, xmr was redeemed, ...). The custom webserver could then execute some action based on that information (for example calling an API to buy back the sold XMR) . I think it would greatly ease the integration of the ASB into existing infrastructure of exchanges.

[da-kami]

Thanks for the proposal, interesting brain stimulation!

To clarify a few things, you would like to achieve:

1. using a custom price feed
2. get notified about swap events in real-time

Correct?

For achieving (1) we need some standardized way to tell the ASB what price to use. Doable, but requires some more though.
You can already achieve (2) by piping the logs into a service (e.g. using https://vector.dev/ ) - but that can be improved as well.

Would be great if you could define your ideas in a bit more depth - we would also be very happy about contributions. Our team is currently busy with working on maker discovery.

[cimble-code]

Yes exactly. I want to use a custom price feed and be able to process the log output more efficiently. I think it would be enough for now to let the ASB listen for messages on a websocket, which then internally updates the current XMR / BTC price. For the sake of simplicity, I would suggest using a very simple JSON protocol.

Something like:

{
      price: 7128831
}

In order to be more future-proof and customizable it would be useful to request the exact amount of XMR that the provider is willing to trade for a given amount of BTC. This way, a provider could set higher fees for smaller trades and lower fees for larger trades, or set a fixed fee to be added to each trade. But I think that could be added at a later date and the very simple mechanism outlined above would suffice for now.
The logs would also just be sent over the ws as JSON.

I still have to get familiar with Rust as I haven't used it before, but I'll try to implement it myself. Of course, I wouldn't mind if someone else wanted to take it upon themselves.

[cimble-code]

We have decided to introduce a bounty of 0.25 XMR for this issue, for anyone who implements this before someone from the COMIT team or us implements it. Will be paid out after the pull request is approved. If you want to contribute to the growth of BTC and XMR atomic swaps and help the decentralization of Monero, start working on this and open a pull request once you are done.

[thomaseizinger]

First of, I want to state that I am very supportive of the idea of motivating other people through bounties to contribute to the implementation. However, as with all contributions, it is important that they fit well within the overall architecture, both feature-wise and also in how the implementation is carried out.

Hence, as @bonomat pointed out in #571 (comment), I think it is paramount that we take a step back first and discuss what problem is being solved here before we jump into solutions. Otherwise, we may end up with a situation where someone is keen on the bounty and submits a PR that is not at all inline with how we would like the codebase to evolve.

I hope that makes sense :)

Without that out the way, I am keen to hear more about what you are trying to achieve and design a solution together that a) solves your problem and b) fits well into the codebase.

[cimble-code]

I definitely don't want to force my ideas on you. If you favor a different approach that suits the existing codebase and your way of doing things better, I'm perfectly on board with that.
I have reiterated the problem I'm trying to solve here #571 (reply in thread)

[bonomat]

Hi @cimble-code ,

thanks for posting this feature request and adding the bounty, we would love to see contributions :)

When reading your initial description my initial first thought were that you are describing two different features:

1. allow for pushing XMR/BTC rates to the ASB (e.g. let the ASB subscribe to a custom price-feed)
2. get notified by the ASB about swap events (e.g. ASB publishes events on a websocket)

Your latest message made me think otherwise though. So instead of focusing on the solution I would prefer if you could describe the actual use case you would like to solve.

My current understanding is that you would like to have fine granular control over swap requests, e.g. it should be possible to react to a trader asking for a quote based on the amount she wants to swap. This would allow the ASB to have different rates for smaller or bigger amounts.
This is just my assumption though and you might have something else in mind.

[bonomat]

1. I need to be able to push prices to the ASB. We will most likely be using an exchange that does not have as much liquidity as others and therefore has some potential price differences compared to other larger exchanges. We also need to consider things like the depth of sell orders at a given price. Basically, we need to mitigate the effects of price fluctuations, so we need to be able to set our own price at a given time.

Gotya.
There is a relatively easy to implement (but temporary) solution for that. We could let the user configure the kraken websocket url via the ASB config. That way you can plug in your own service. The only requirement is that your service publishes prices updates in the same format as kraken, e.g. :

[
980,
    {
        "a":["0.00440700",7,"7.35318535"],
        "b":["0.00440200",7,"7.57416678"],
        "c":["0.00440700","0.22579000"],
        "v":["273.75489000","4049.91233351"],
        "p":["0.00446205","0.00441699"],
        "t":[123,1310],
        "l":["0.00439400","0.00429900"],
        "h":["0.00450000","0.00450000"],
        "o":["0.00449100","0.00433700"]
    },
    "ticker","XMR/XBT"
]

A more sophisticated and long term sustainable solution would need more thinking.

2. An easier way to handle events. For example, to place a buy order on an exchange when Bob locks his BTC.

I understand the convenience of websockets. Another solution would be to use the log files for this. As @da-kami proposed in #571 (comment), you can use tools like vector or logstash to parse the logs and emit events based on this.
Did you have a look at the log files in json format if they give you all the information you'd need? We can easily log more information if needed.

I haven't tried it but this sink looks like it can send parsed log events to a socket. That way you can have your business logic in a separate tool.

[thomaseizinger]

For what its worth, reading from a (web)-socket should be technically no different than reading from stderr. However, if your ASB instance is managed through something like systemd, then you'd just have to either interface with journald or pipe stderr into another tool. If all your processing happens on the same machine, I don't think you need something like vector at all.

[thomaseizinger]

If you are dead-set on websockets, combining something like: journalctl --user --output=json -efu asb.service and https://github.com/joewalnes/websocketd should give you all the logs via a websocket. You'd obviously have to tailor what kind of output you'd want to see from journald but you get the idea :)

[thomaseizinger]

I need to be able to push prices to the ASB. We will most likely be using an exchange that does not have as much liquidity as others and therefore has some potential price differences compared to other larger exchanges. We also need to consider things like the depth of sell orders at a given price. Basically, we need to mitigate the effects of price fluctuations, so we need to be able to set our own price at a given time.

From what I read, being able to write a plugin that is asked to produce a price whenever a new swap is set up would solve your issue?

[cimble-code]

There is a relatively easy to implement (but temporary) solution for that. We could let the user configure the kraken websocket url via the ASB config. That way you can plug in your own service. The only requirement is that your service publishes prices updates in the same format as kraken, e.g. :

I think that would be a good solution for now. It would solve my problem. I want to clarify that I don't need to specify the rate for every trade. I just think it would be reasonable to consider this when implementing a more robust mechanism for price control in the future.

For what its worth, reading from a (web)-socket should be technically no different than reading from stderr. However, if your ASB instance is managed through something like systemd, then you'd just have to either interface with journald or pipe stderr into another tool. If all your processing happens on the same machine, I don't think you need something like vector at all.

I understand the convenience of websockets. Another solution would be to use the log files for this. As @da-kami proposed in #571 (comment), you can use tools like vector or logstash to parse the logs and emit events based on this.

This is basically what I am doing right now and it works fine. Piping stdout/stderr into a file and then watching that file from another process. I agree with you that introducing another websocket just adds complexity. It is better to let people use existing tools to handle capturing logging.

Did you have a look at the log files in json format if they give you all the information you'd need? We can easily log more information if needed.

The JSON output is neat and easy to use. The only small improvement is that the amount to be swapped is never really reported. This log message could be extended to also report the amount of BTC locked (after fees).

{
  "timestamp": "2021-06-17 16:38:04",
  "level": "INFO",
  "fields": {
    "message": "Waiting for Bitcoin transaction finality",
    "txid": "ABC123",
    "required_confirmation": "2"
  },
  "span": {
    "id": "5b058dc4-4f6f-4c3e-b260-dbef49d16e31",
    "name": "swap"
  },
  "spans": [
    {
      "id": "5b058dc4-4f6f-4c3e-b260-dbef49d16e31",
      "name": "swap"
    }
  ]
}

Modified:

{
  "timestamp": "2021-06-17 16:38:04",
  "level": "INFO",
  "fields": {
    "message": "Waiting for Bitcoin transaction finality",
    "txid": "ABC123",
    "required_confirmation": "2",
    "amount": 0.001
  },
  "span": {
    "id": "5b058dc4-4f6f-4c3e-b260-dbef49d16e31",
    "name": "swap",
    "amount_after_fees": 0.00076,
    "rate": 0.008
  },
  "spans": [
    {
      "id": "5b058dc4-4f6f-4c3e-b260-dbef49d16e31",
      "name": "swap",
      "amount_after_fees": 0.00076,
      "rate": 0.008
    }
  ]
}