Support BSON in SignalR Core

[HappyNomad]

I know about MessagePack but my scenario involves Asp.Net Core receiving raw BSON from MongoDB, and then forwarding it onward to a .NET client app. I don't want to needlessly translate the data from one binary format (BSON) to another (MessagePack).

Is sending BSON over SignalR Core currently achievable? If so then how and how involved? Otherwise, please consider adding BSON support, or enabling any binary format. I know MessagePack is better and all, but Mongo uses BSON and I use Mongo.

The alternative that comes to mind is to use web APIs since HTTP handles any binary format, and Asp.Net Core lets me configure output formatters. What about configuring formatters in SignalR Core? I couldn't find any Formatters options. I want to stick to using SignalR Core rather than web API, due to other benefits of the former.

[highfield]

I personally would not transfer data straight from MongoDB, unless you're dealing a very specific tool for it (e.g. an inspector of the database or so). I always prefer to leave (at least one) abstraction layer in order to decouple the actual data-source (MongoDB) and the API: that gives a lot of benefits, but I believe you already know that.

However, if you really want to wire BSON over socket, I think you could try with the official MongoDB's library: https://github.com/mongodb/js-bson

Just in case, time ago I did a small library to convert back-and-forth JSON and BSON: what comes from MongoDB can be converter to a JSON, and viceversa. Might that helps, maybe?

[analogrelay]

We don't have plans for any new official protocols. I do agree with @highfield here that you should not be transferring data directly from any data store to SignalR, as you generally want an abstraction (and the added security of being careful about what you expose to your clients).

The SignalR protocol layer is pluggable though, so if you are interested in developing your own protocol (using BSON or any other format), you can do so by implementing the IHubProtocol interface and registering it in the DI container as a singleton. Clients will be able to request your protocol by name (whatever you return from IHubProtocol.Name). On the JavaScript client you'll similarly have to implement the TypeScript interface IHubProtocol.

Closing this as we don't plan to add an additional protocol at this time. If you have further questions about building a custom hub protocol, feel free to comment and we can help as best we can.

[HappyNomad]

Looks like IHubProtocol is to SignalR what formatters are to web API. Where's the official doc on implementing it? If there's none, would you help me get started? Please show how the below BsonOutputFormatter's functionality would fit into an IHubProtocol implementation.

using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc.Formatters;
using Microsoft.Net.Http.Headers;
using MongoDB.Bson;
using MongoDB.Bson.IO;
using MongoDB.Bson.Serialization;

public class BsonOutputFormatter : OutputFormatter
{
    public BsonOutputFormatter() =>
        SupportedMediaTypes.Add( MediaTypeHeaderValue.Parse( "application/bson" ) );

    protected override bool CanWriteType( Type type ) =>
        typeof( BsonDocument ).IsAssignableFrom( type );

    public override async Task WriteResponseBodyAsync( OutputFormatterWriteContext context )
    {
        if ( context == null )
            throw new ArgumentNullException( nameof( context ) );
        else if ( context.Object == null )
            return;

        var response = context.HttpContext.Response;
        using ( var memoryStream = new MemoryStream() )
        using ( var bsonWriter = new BsonBinaryWriter( memoryStream ) ) {
            BsonSerializer.Serialize( bsonWriter, (BsonDocument)context.Object );
            bsonWriter.Flush();
            memoryStream.Position = 0;
            await memoryStream.CopyToAsync( response.Body );
        }
    }
}

[analogrelay]

There's no docs for implementing it, as it's not a high priority use case for us. I don't have any specific suggestions for you on how to implement it other than to look at the existing JSON and MessagePack implementations.

Implementing a custom hub protocol is a significant amount of work and requires that your client and server be in sync. I don't yet understand the value it adds to your scenario. Do you have data showing that translating the BSON to some other format like JSON is causing a performance problem in your application? Tightly coupling your client to the BSON in your Mongo database seems like a very risky idea to me since it means you can't change the format without risking a break to all your client logic.

[HappyNomad]

In the long run, would you consider refactoring MessagePackHubProtocol to make it easy to inherit from and override a few methods for customizing the payload format without having to re-implement other protocol details? I'd like a developer experience more like implementing formatters for web API. It could be useful in a variety of scenarios and binary formats other than BSON.

I'll keep in mind your reasoning for sending to the client either JSON or MessagePack instead of BSON. For now at least, it seems like an unnecessary extra step without tangible benefit. My BSON-specific client-side code is confined to a single method - not difficult to swap out if the occasion arises. Also, MongoDB's .NET library makes it easy to deserialize only the pieces of BSON that are needed. That means the server-side code can see the parts it needs, and compose them with other parts, without deserializing everything.

[analogrelay]

In the long run, would you consider refactoring MessagePackHubProtocol to make it easy to inherit from and override a few methods for customizing the payload format without having to re-implement other protocol details?

I doubt we would do that. The hub protocols aren't designed for customization in this way.

[HappyNomad]

SignalR Core's MessagePack hub protocol might already be customizable via the MessagePack library. In MessagePackHubProtocol, I just spotted usage of IFormatterResolver. It has a single GetFormatter method that returns an IMessagePackFormatter. It has Serialize and Deserialize methods which look a lot like what I'm after.

[analogrelay]

Do remember that any change you make here also needs to be replicated in all the clients you intend to support. The .NET client shares this code with the server but the JS and Java clients use completely different code (another reason why I would strongly advise against customizing the Hub Protocol).

[highfield]

Also, MongoDB's .NET library makes it easy to deserialize only the pieces of BSON that are needed. That means the server-side code can see the parts it needs, and compose them with other parts, without deserializing everything.

@ChainReactive that's not part of the protocol, rather of the interface exposed by the driver. Moreover, you're asking to support BSON-over-websocket: to me that means the binary representation of the original data, not the DOM useful for manipulation.

That being said, why not make yourself a small test? take any BSON recordset coming out the DB, then copy its binary buffer to a signalR response. On the client side, you should able to deserialize th stream by the MongoDB official library. I'd first make an attempt using a "verbose" way using a base-64 codec over the text signalR: overhauled, but pretty easy as starting point. If successful, you might switch the MessagePack in order to transfer the binary as is.

[analogrelay]

@highfield makes a good point. There's no reason you couldn't just send the BSON "via MessagePack" as a byte array, using a model like this:

public class MongoResult
{
    public byte[] RawBson { get; set; }
}

If you send this object through SignalR, it will be appropriately serialized and deserialized and on the client you can access .RawBson and pass it to Mongo's deserializer.

You won't be "converting" the BSON to SignalR's format, you're just wrapping it in a SignalR message. In JSON, we'll base-64 encode it for you automatically, in MessagePack we'll send it as mostly raw binary (with a little bit of protocol goop wrapping it).