Large Data Channel Messages for Firefox

At the moment, you have to be careful about the maximum message size you can send over a WebRTC data channel. Especially if the data channel has been established between Firefox and Chrome. With help from Randell Jesup, Nils Ohlmeier and plenty of other helpful guys from the #media Mozilla IRC channel, I've added EOR handling to Firefox data channels (tracked in bug 979417) which will land in Firefox 57. So, what does this actually mean and what are the benefits?

EOR... what?

SCTP is the transport protocol used for exchanging WebRTC data channel messages. Its API provides an EOR flag which stands for end of record and tells us whether a message is partial or complete. Thus, every implementation needs to pay attention to this flag or message integrity will be violated. Which sadly is the case for all popular data channel implementations at the moment and results in harsh size limitations for data channel messages. For more technical information on this and how one can work around this, take a look at my previous blog posting. But do not worry, determining that size limitation will be a lot easier, soon.

Unfortunately, none of the browsers today support the SCTP ndata extension yet, so a single data channel message will monopolise the whole SCTP assocation. This phenomenom is also known as head-of-line blocking and makes sending large messages quite painful if other channels need to keep exchanging messages frequently, for example in a real-time chat with file-sending capability. All messages exchanged in the chat would arrive after the lengthy process of sending a gigantic file has been completed.

While the ndata extension is currently in the specification process, it's making good progress and there's no excuse for holding back proper EOR handling anymore. Personally, I believe this should have never been an excuse for not handling EOR as it has so many undesired side effects. Since I've gained experience in writing a standalone data channel implementation with proper EOR handling (called RAWRTC), it was a logical choice to apply this to at least one of the browsers.

Exchanging Large Messages

With EOR handling, one major benefit is of course being able to send and receive large messages. But hold on a second... sending large messages from Firefox to Firefox was possible before, wasn't it? Yep, that's true, with help of the deprecated PPID-based fragmentation/reassembly mechanism. However, no one else implemented this. A detection algorithm has been added, so Firefox will fall back to use PPID-based fragmentation/reassembly when needed: In case the maximum message size attribute in the remote description's SDP is missing and 256 incoming streams have been negotiated for the SCTP association, it is assumed that the other peer is an older Firefox browser. It is not the cleanest of solutions but it should be adequate and will be removed once Firefox ESR has EOR support as well.

Thanks to EOR handling, Firefox will support receiving up to 1 GiB messages. Why is there a limitation? Well, theoretically we can support much more but the limiting factor is available memory and the fact that the RTCDataChannel API of WebRTC is too high-level, meaning it's not possible to send or receive a message in chunks. Therefore, when you want to send a large file as a whole, there's an enormous backpressure due to the fact that the file's data needs to be in memory when calling the send method. Changing this requires changes to the W3C WebRTC specification.

Nevertheless, this size limitation may be increased in the future. But having a size limit requires knowledge about this limitation. More precisely, that value must be available before an application even tries to send a message or it would be a guessing game...

Maximum Message Size Indication

In the W3C WebRTC spec, there's an RTCSctpTransport object which has a field called maxMessageSize. It maps to the attribute from the SDP and is exactly what a user application needs to look for. So, the following example should work, right?

let pc = new RTCPeerConnection(...);
let dc = pc.createDataChannel('test', {
    negotiated: true,
    id: 1
});

dc.onopen = () => {
    let maximumMessageSize = pc.sctp.maxMessageSize;
    dc.send(new Uint8Array(maximumMessageSize));
};

// Create and exchange offer/answer here.
// ...

But sadly, it doesn't because Firefox and all other browsers currently do not support the object API that has been borrowed from the ORTC specification. So, what can be done to retrieve this information? The answer lies within the SDP blob. Check this out:

let pc = new RTCPeerConnection(...);
let maximumMessageSize;
let dc = pc.createDataChannel('test', {
    negotiated: true,
    id: 1
});

dc.onopen = () => {
    dc.send(new Uint8Array(maximumMessageSize));
};

mySignaling.onRemoteDescription = (description) => {
    // Parse a=max-message-size
    maximumMessageSize = 65535;
    const match = description.sdp.match(/a=max-message-size:\s*(\d+)/g);
    if (match !== null && match.length >= 2) {
        maximumMessageSize = parseInt(match[1]);
    }

    // Set remote description
    ...
};

// Create and exchange offer/answer here.
// ...

Now we know how much data we can send. In case an application sends a message that exceeds this limitation, the send method will raise an error as described in the next section.

But you definitely don’t want to do that ugly SDP inspection yourselves, so support in adapter.js is about to be added. I encourage you to update to the latest adapter version once that PR has been merged as it also sorts out various other incompatibility nits regarding the maximum message size for many browser interoperability cases.

Explicit Errors

These changes will not only make it possible to send and receive large data channel messages, but will also let you know if the other peer is capable of receiving a message that has been passed to the send method in form of a TypeError exception as specified (formerly, the data channel has been closed implicitly in case of such an error). This ensures backwards compatibility to implementations that do not handle EOR at the moment as all implementations are capable of handling 64 KiB sized messages.

Other Browsers

Now, it's your turn Chromium. Let's raise the limit - at least well beyond 64 KiB.

An edited version of this article has also been released on Mozilla's WebRTC blog.