This is another problem that was brought up by @gecko in the chat; and I feel like it is also one of the weaker parts of the spec at the moment.
The short version is: MLS requires everyone in a group to apply commits in the same order, and Cryptid’s dumb-pipe server design creates a problem with that because they are deliberately content-unaware and can’t act as a central authoritative queue. This can (and most likely will) lead to people bricking their own group states.
Here’s what I think might help, but it comes at a wee bit of cost.
We do a 3 layer approach with the following layers:
Layer 1 — Server side commit sequencing
We can add 2 fields to the message envelope for commit messages only:
- an opaque
group_token(Blake3 of the group ID) - a
commit_sequencenumber.
The server can use these to maintain per group commit ordering and buffer briefly if a later sequence number arrives before an earlier one while rejecting clear sequence violations.
The privacy cost of this would be that the server learns that an opaque token exists and can count commits associated with it. It learns nothing about group membership or whatever goes on in the group itself. I think this is a minimal and acceptable metadata leak, but I’d definitely like more feedback about this.
This does require leaking the fact that a message is a commit, which is in direct conflict with the current design goals where message types are fully hidden. I personally think that “this is a group state change” reveals far less than the social graph metadata the server already unavoidably sees from delivery patterns.
Layer 2 — Client-side epoch fencing
Clients should keep track of their current epoch and when they receive a commit for epoch N+K when sitting at epoch N, rather than erroring out, they should buffer the commit and request gap commits by sequence number from the server. So the request would look like “give me sequence numbers X through Y for group token Z”.
Layer 3 — External Commit Self healing
For situations where the gap commits are just fully gone, clients can self heal using MLS external commits. The flow might look like:
- Detect epoch gap, request missing commits by sequence number
- If unavailable, fetch current
GroupInfofrom server - Construct and send an External Commit
- Rejoin current epoch, back in sync
From the user’s perspective this could look like a “resyncing…” indicator at worst and not a borked room that requires admin intervention to re-add them.
This requires servers to store the latest GroupInfo blob (opaque) per group, keyed on the opaque group token I mentioned earlier.
Quorum ACK Gate for Commit Proposals
On top of the 3 layers thing, we should probably make it so that new commits can only be proposed once a quorum of currently active members has ACKed the previous commit.
Critically, this would gate proposals and not commit applications. “Currently active” needs a proper definition though. Maybe members seen within the last N minutes? The gate should open regardless after a hard timeout so that offline members don’t deadlock commits forever.
I have questions
- How big/small should the activity window be for the quorum gate?
- Should the
group_tokenrotate with epochs or stay stable for the group lifetime? I feel like rotating it would be best for privacy, but it’ll require a lot of work to get right. - Is leaking the commit flag an acceptable choice? Alternatively, we could probably use a separate out-of-band sequencing channel so the main envelope stays fully opaque?
