Skip to content

feat: Devtools - A separate, lightweight event bus -- utilizing Dispatch (Sync)#950

Open
theVedanta wants to merge 17 commits into
uwdata:mainfrom
theVedanta:devtools-event-bus
Open

feat: Devtools - A separate, lightweight event bus -- utilizing Dispatch (Sync)#950
theVedanta wants to merge 17 commits into
uwdata:mainfrom
theVedanta:devtools-event-bus

Conversation

@theVedanta
Copy link
Copy Markdown
Contributor

@theVedanta theVedanta commented Dec 25, 2025

This commit resolves some tasks discussed in the meeting with @domoritz. In response to the issue #943
For an overview, these are some of the immediate action items discussed:

Action items

  • Eliminate existing logger implementation
  • Add a .observe method to a single place on the coordinator
    • Params: observer, Set of event types (not provided subscribes to all events)
  • Initial implementation only includes bare minimum events to replicate current logging experience
  • Put code into coordinator for now, may split it into a separate file later
  • Remove logQueries bool from coordinator
  • Add abstract class Dispatch
    • existing AsyncDispatch extends it
    • Add new ObserveDispatch also extends it
    • .observe routes to observe dispatch .addEventListener

The UI-related example, written in Solid JS, will be in a separate PR. The current goal is to switch out the logger in Coordinator and QueryManager to consume our event bus for uncertain event types (which will be crystallized later for devtools).

https://docs.google.com/document/d/1cyw2sv9G5-CzGG-JgeOEcISSu7zmEqdzsa-J2oh2F6c/edit?tab=t.0

Copy link
Copy Markdown
Member

@domoritz domoritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool start. looking good. Here are some comments.

Comment thread dev/index.html Outdated
function setQueryLog() {
vg.coordinator().manager.logQueries(qlogToggle.checked);
// logQueries is no longer in use
// vg.coordinator().manager.logQueries(qlogToggle.checked);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where we should add or remove our custom observer that redirects events to console.

* callback function to remove.
*/
override removeEventListener(type: string, callback: EventCallback<T>): void {
super.removeEventListener(type, callback);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need this override? Looks equivalent

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I had some extensibility in mind for later, but on at another glance, I think this feels unnecessary. Removing...

Comment thread packages/mosaic/core/src/Coordinator.ts Outdated
if (arguments.length) {
this._logger = logger || voidLogger();
this.manager.logger(this._logger);
// subscribe logger to events
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good as a stepping stone but I think we can go ahead and make the logger a separate observer that users need to explicitly add. We can remove the logger entirely now imo.

Comment thread packages/mosaic/core/src/Coordinator.ts Outdated
): Promise<unknown> {
client.queryPending();
return client._pending = this.query(query, { priority })
return client._pending = this.query(query, { priority, clientId: (client as any).id })
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as any is worrying me a bit here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking we get rid of the clientId for now?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do that

Comment thread packages/mosaic/core/src/Coordinator.ts Outdated
Comment thread packages/mosaic/core/src/EventBus.ts Outdated
Comment thread packages/mosaic/core/src/QueryManager.ts Outdated
Comment thread packages/mosaic/core/src/QueryManager.ts Outdated
Comment thread packages/mosaic/core/src/QueryManager.ts Outdated
Comment thread packages/mosaic/core/src/QueryManager.ts Outdated
Comment thread packages/mosaic/core/src/QueryManager.ts Outdated
Comment thread packages/mosaic/core/src/Coordinator.ts Outdated
Comment thread packages/mosaic/core/src/Events.ts Outdated
Comment thread packages/mosaic/core/src/Events.ts Outdated
Comment thread packages/mosaic/core/src/util/ObserveDispatch.ts Outdated
Comment thread packages/mosaic/core/src/Coordinator.ts Outdated
Comment thread .prettierrc.json Outdated
@theVedanta theVedanta changed the title feat: Devtools v2 -- Creates a separate, lightweight event bus feat: Devtools - A separate, lightweight event bus -- utilizing Dispatch (Sync) Jan 20, 2026
@theVedanta theVedanta marked this pull request as ready for review January 24, 2026 21:36
@theVedanta theVedanta requested a review from jheer as a code owner January 24, 2026 21:36
@theVedanta theVedanta requested a review from domoritz January 28, 2026 22:26
Comment thread dev/query/index.html
output.replaceChildren();
} else {
await coordinator().exec(
await vg.coordinator().exec(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to add vg.?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

honestly, I'm not really sure. this example was broken when I tested it; the coordinator is not accessible as an object here, but only as a property of vg

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, good catch. I'll fix it separately. Let's keep PRs focused on one thing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in #965. Let's remove this here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

Comment thread dev/query/index.html
enable();
});

function setQueryLog() {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this logic (which replicates the old logger, right?) be in mosaic core so that people can easily get the old logger behavior again?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh. we can definitely do that. My mistake, I felt that when we said 'remove the logger from core entirely,' we give the client the option to write that logic. I can put this function in core to give back a logger util to the user/client (perhaps we can discuss that idea some more (?))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC the idea was to remove the logger but give users an equivalent tool via the new bus.

Comment thread dev/index.html Outdated
vg.coordinator().manager.logQueries(qlogToggle.checked);
const bus = vg.coordinator().eventBus;

bus.observe("QueryStart", (event) =>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplicated logic?

Comment thread packages/mosaic/core/src/util/AsyncDispatch.ts Outdated
Comment thread packages/mosaic/core/src/util/AsyncDispatch.ts Outdated
if (cached) {
const data = await cached;
this._logger.debug('Cache');
const data = cached;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we not need to await anymore?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

await will have no effect here

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make a separate pull request. IIRC there was some reason where cached results can be promises but I am not 100% sure right now.

Comment on lines +119 to +121
this.eventBus?.emit(EventType.Error, {
message: err,
});
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this strictly needed to replicate the behavior of the old logger?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, all it's doing it just pipe whatever the message was (which logger emitted) into the bus, which is now consumable by anyone, including the logger. I don't know if Im missing something, did you have a simpler approach in mind?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just didn't see a log message (we removed) here before so I am wondering whether the emit is needed.

Comment thread packages/mosaic/core/src/Events.ts Outdated
Error = "error",
}

export type EventMap = {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've never seen this pattern before. Why is this needed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was mainly for type-safety (for us and the clients)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not do this and instead use the class approach I suggested?

Comment thread packages/mosaic/core/src/Events.ts Outdated
QueryStart = "query-start",
QueryEnd = "query-end",
ClientConnect = "client-connect",
ClientStateChange = "client-state-change",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this still be here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry? I don't understand

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see client-state-change used anywhere.

@@ -0,0 +1,43 @@
export enum EventType {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of having this enum here. Wouldn't it be easier to have classes for the events that can be instantiated with a constructor?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is probably true. It will get rid of this "broken"-ish type interface for the events. But I remember that we agreed on enums (denoted by strings instead of numbers) to provide the user a simple list of event types in autocomplete

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we can still use enums in the event types but in order to create an event object, we could use a class. Maybe it's cleaner to actually put the event type into the message so that we can easily type check a message in isolation. Let's do that.

@theVedanta
Copy link
Copy Markdown
Contributor Author

@domoritz let me know if the current approach is closer to the logger implementation you had in mind. If not, we can perhaps meet again and clear up any confusion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants