Document ability to use websocket endpoint

[gmile]

Following the comment:

There is also a websocket in Mailpit which broadcasts new messages, although that is not part of the official API as it is intended for the web browser (so it is not documented).

Would you consider it possible to make usage of ws://mailpit/api/events endpoint part of the documentation? Or alternatively introduce a separate websocket endpoint such that could be promoted for public usage in the docs?

For context, here's my use-case. I am using mailpit in a test setup involving docker-compose and playwright. After noticing that Mailpit uses WS for its internal purposes I managed to simplify my "wait for an email to arrive" code to just this:

mailpit_baseurl = "ws://..." // or wss://...

export async function openMailbox(page) {
  await page.evaluate((baseurl) => { new WebSocket(`${baseurl}/api/events`) }, mailpit_baseurl);
  return page.waitForEvent("websocket");
}

export async function waitForEmail(webSocket) {
  const event = await webSocket.waitForEvent("framereceived");
  return JSON.parse(event.payload).Data;
}

I find this much cleaner (and potentially more robust) than polling the Mailpit API to see, if the email arrived. However, as it goes with any internal API, depending on it poses a risk this endpoint will be modified without a notice. So, it would be nice to have a "public" form of this endpoint.

[axllent]

That's a good question @gmile. The /api/events endpoint is where I can broadcast events relevant to the web UI in order to achieve a functional interface, as well as reflect changes made in another browser (two users browsing the same instance). As I mentioned, I am reluctant to add this to the official (documented) API because I don't want others relying on something that may change. It also handles far more than just new messages, it handles other types of events such as read/unread statuses, single & mass deletion, etc, and this is where things get tricky. This also leads to a deeper problem - how does one even begin to document a websocket which handles an array of different event types (currently I count 7)?

If I understand you correctly, all you are actually interested in is new messages, am I correct?

On the one hand a separate API endpoint makes sense, however the structure of those new messages is tied to structure in the message summary (same as the API), which is tied to the database. This is unlikely to change as it's the same structure as the API uses.

If I was (theoretically) to create a separate endpoint for just new message broadcasts, it would probably lead to a silly amount of code duplication within Mailpit to handle two separate pools of websocket connections, as well as only a subset of broadcast messages etc. In essence, I think this would horribly overcomplicate things in an attempt to simplify it, all of that to end up with the same "new message" events that the current websocket endpoint already handles exactly what we (you) need.

What is your opinion here? The /api/events is unlikely to change, at least with regards to new message notifications. Whilst I don't think it's physically possible for me to document this in the API docs (ie: the same endpoint return various JSON responses, so it's not so much a REST API), it could definitely be documented on the Mailpit website - expect how far would I need to go to explain what the websocket returns, when it's just a limited few "types" of information that one should be after?

I'll give this some more thought, but to be honest I have given this a lot of thought over the last year or so, and I keep coming up short.

[gmile]

Thank you for the extensive context, @axllent! I can see your perspective now, and how a matter of making the endpoint public is not as simple as I thought.

If I understand you correctly, all you are actually interested in is new messages, am I correct?

You're absolutely correct. My case is quite narrow: I am only interested in receiving notifications about new messages arriving in the inbox during an end-to-end test scenario. I strongly believe that designing test code to use event-driven notifications for new emails is superior to API polling approaches. This strategy not only improves efficiency but can also lead to higher quality test helpers and DSLs in the long term.

Commenting more broadly, I am thinking if the single websocket endpoint couldn't serve both public and private needs at the same time, while not making a mess. Perhaps the following strategy could work?

1. websocket endpoint comes out of shadow and gets highlighted on the website documentation (e.g. not in the API docs),
2. in the documentation it is made clear that currently websocket endpoint emits messages serving both external and internal needs,
3. message of type new is the only one considered consumable by public; it carries information about newly arrived email, and conforms to a well-defined JSON schema,
4. messages of any other type are considered private/internal (and need not necessarily to even be enumerated what exactly they are) and are not to be relied on (or could be relied on, as long as a risk of them disappearing or changing is acceptable),

This way of thinking about websocket endpoints shifts from being either a "public" or "private", to being merely a hose that produces messages which would be considered either public or private.

To sum up I think this could have the following benefits, maintenance-wise:

• no need to document all event types apart from public ones (this should address concerns about documentation),
• no need for refactoring or any changes to code today (this should address concerns about complicating current implementation unnecessarily),
• since events are just JSON payloads, potentially they could carry explicit info if they are "public" or "private" ("internal"),
• public events could evolve (just like DB schemas do) or be versioned / deprecated / removed over time, so making any event public doesn't mean you're locking yourself to support it forever

[axllent]

Thank you @gmile for your feedback, this is extremely useful. I agree with your proposal as I think this ticks almost all the boxes, both from the maintainer (my) perspective as well as a developer (in this case: you). I still need to give this more thought, as I'm not keen on documenting everything coming out of the "hose" (from an effort and document perspective), but by stating this, and clarifying the structure new data type I think it could go a long way.

[axllent]

I'm just noting that this is still on my TODO list - I just haven't had the time yet.

[gmile]

@axllent thank you for update. No rush on my end obviously, and happy to hear this remains on your radar 🙇

[axllent]

@gmile I have added a page with some documentation about the websocket here. Hopefully it makes sense, however if you believe it should be worded differently then please feel free to send a merge request (you can clone the site via the edit this page link at the bottom and then submit your change as a merge request). Thanks.

[gmile]

Took an initial look, this is wonderful, thank you so much @axllent!