Custom renderer investigation

[ScreamZ]

Note

There is a lot of information below, sorry if this is a bit disordered, I try to update the issue everytime it gets clearer on my mind, I'll try to split it too.

I'm currently using the streaming custom adapter (to have the writing effect) while I'm received a single (full) message of text from my API.

Summary

• Typing issue of content prop. (See Custom renderer content prop is defined as an Array but is a string #75)
• Rendering difference between environments (SEE Implementing a DefaultStreamingRenderer component and remove `containerRef` #74)
• status: "streaming" never go complete

Code sample

Click to see my code

import { StreamResponseComponentProps } from "@nlux/react";
import React from "react";

export function MyResponseResponseRenderer<AiMsg = string>(props: StreamResponseComponentProps<AiMsg>) {
  console.log(props);
  return (
    <div className="flex flex-col">
      <div ref={props.containerRef} />
      <div className="grid grid-cols-3">
        <button onClick={() => console.log("I like it!")}>👍</button>
        <button onClick={() => console.log("I love it!")}>❤️</button>
        <button onClick={() => console.log("I hate it!")}>😵</button>
      </div>
    </div>
  );
}

import type { ChatAdapter, StreamingAdapterObserver } from "@nlux/core";
import { sendMessage } from "../server/actions/sendMessage";
import { parseResponseMessageParsing } from "../utils/message-parsers";

export const MyAdapterBuilder = (characterID: string, conversationID: string): ChatAdapter => ({
  streamText: async (message: string, observer: StreamingAdapterObserver) => {
   
    const result = await sendMessage({
      character: characterID,
      conversation: conversationID,
      userInputMessage: message,
    });

    const parseResult = parseResponseMessageParsing(result.data);

    if (!result.serverError && parseResult.status === "success") {
      observer.next(parseResult.data.message);
      observer.complete();
    } else {
      observer.error(new Error(result.serverError || "Could not generate message"));
    }
  },
});

Attachments

• Through initialConversation from NextJS React server component with use client directive.
  
• Through adapter (full client side)
  

Typing issue

• 1️⃣ Type is a string, but typed as an array for the content property.

Note

See #75

See the screenshot above.

type StreamResponseComponentProps<AiMsg> = {
    uid: string;
    dataTransferMode: 'stream';
    status: 'streaming' | 'complete';
    content?: AiMsg[];
    serverResponse?: unknown[];
    containerRef: RefObject<never>;
};

Custom response renderer.

• With the response renderer defined in sample code at the beginning of this post I get no content on NextJS SSR (use client). Which means the containerRef is not defined. Which seems OK because SSR. And if i use props.content (which is a string and not an array as we saw) I got the plain markdown content.

What can we do ?

Note

See #74

Tip

Here are some observations / suggestions of what we might do. Or you can tell me if you have some.

1. It's okay that server/initialConversation content is considered dataTransferMode: "batch" that way we can define a different render code for server/initialConversation. But we could need the default parser, otherwise, messages are not rendered as markdown but plain text ? Or should i use my own renderer for MD ?.
2. For client/adapter message with stream or whatever what can we do ? Maybe expose primitives and/or some components for streaming message instead of using the containerRef would be better.

💡 Briefly:
- Expose a DefaultStreamingRenderer component instead of containerRef to handle the streaming mode if someone use a custom renderer. (more flexibility) This component would receive content props which should be the same for streaming/batch.

- For client side (after receiving a message from the adapter), containerRef is defined but content is empty.

status: "streaming" never go complete

When using the above code, I never get status: "streaming", going to complete. for streaming messages

[salmenus]

The issue related to status property passed to custom renderers not being updated:
That's now fixed ✅

• Published in version 2.4

Example here:
https://codesandbox.io/p/sandbox/custom-renderer-status-9tt6zt?file=%2FApp.tsx

[salmenus]

• Issue related to content property fixed ✅
• content is now always an array
• Fixed in 2.5
• Give it a try and let me know @ScreamZ

[ScreamZ]

• Issue related to content property fixed ✅
• content is now always an array
• Fixed in 2.5
• Give it a try and let me know @ScreamZ

Looks good so far, now we can implement any renderer we want.

[salmenus]

One more item is still pending: Rendering difference between environments