@vsllabs/webgl-react

Installation:

To install the package, run:

Using npm:

$ npm install @vsllabs/webgl-react

Using yarn:

$ yarn add @vsllabs/webgl-react

Table of Contents

• Usage Example (Normal mode)
• Usage Example (Streaming mode)
• Documentation
• Required Parameters
• Returned Values
• Example Workflow

Usage Example (Normal mode):

Below is an example of how to use the useVslWebGL hook within a React component:

import { useState } from 'react'
import { useVslWebGL } from '@vsllabs/webgl-react';

const app = () => {
    // example text input state
    const [inputText, setInputText] = useState('')

    // invoke the useVslWebGL hook with your personal API_KEY and build URLs
    const { VSLWebGl, unityProvider, translateTextToASL, isUnityLoaded, isTranslating, replay } = useVslWebGL({
        API_KEY: 'Your API Key here',
        streaming_mode: false,    // Optional (false by default)
        // *We will provide these URLs for you
        loaderUrl: 'Unity build Url',
        dataUrl: 'Unity build Url',
        frameworkUrl: 'Unity build Url',
        codeUrl: 'Unity build Url',
     })

  return (
    <div
      style={{
        width: "400px",
        height: "700px",
        position: "relative",
        margin: "auto",
      }}
    >
      {!isUnityLoaded && (
        <div
          style={{
            position: "absolute",
            width: "100%",
            height: "100%",
            top: 0,
            left: 0,
            background: "#999",
            display: "grid",
            placeItems: "center",
          }}
        >
          {/* Your Loading spinner here */}
          Loading...
        </div>
      )}

      {/* The WebGL component for rendering, can be styled as needed */}
      <VSLWebGl
        id="unityWebGl"
        style={{ width: "100%", height: "100%" }}
        unityProvider={unityProvider}
      />

      {/* Example input for the text state */}
      <input
        type="text"
        placeholder="Enter Text..."
        value={inputText}
        onChange={(ev) => setInputText(ev.target.value)}
        style={{
          border: "1px solid black",
          borderRadius: "8px",
          minHeight: "30px",
          width: "100%",
          marginBottom: "16px",
          padding: "8px",
        }}
      />

      {/* Translate button, triggers translation when Unity is loaded and input is provided */}
      <button
        style={{
          border: "1px solid black",
          borderRadius: "4px",
          padding: "8px 16px",
          marginRight: "16px",
          backgroundColor: "green",
          color: "white",
        }}
        type="button"
        onClick={() => translateTextToASL(inputText)}
      >
        {isTranslating ? "Loading..." : "Translate"}
      </button>

      {/* Replay button, replays the last translation */}
      <button
        style={{
          border: "1px solid black",
          borderRadius: "4px",
          padding: "8px 16px",
        }}
        type="button"
        onClick={replay}
      >
        Replay
      </button>
    </div>
  );
}

Usage Example (Streaming mode):

Below is an example of the streaming mode with basic UI using the browser speech recognition for illustration:

import { useState, useEffect } from "react";
import { useVslWebGL } from "@vsllabs/webgl-react";
import "./App.css"; // Importing the styling CSS file (file content is below)

const App = () => {
  const [inputText, setInputText] = useState("");
  const [isStreaming, setIsStreaming] = useState(false);

  const {
    VSLWebGl,
    unityProvider,
    translateTextToASL,
    isUnityLoaded,
    isTranslating,
    replay,
  } = useVslWebGL({
        API_KEY: 'Your API Key here',
        streaming_mode: isStreaming,
        // *We will provide these URLs for you
        loaderUrl: 'Unity build Url',
        dataUrl: 'Unity build Url',
        frameworkUrl: 'Unity build Url',
        codeUrl: 'Unity build Url',
     });

  useEffect(() => {
    if (isStreaming) {
      const SpeechRecognition =
        window.SpeechRecognition || window.webkitSpeechRecognition;

      if (!SpeechRecognition) {
        alert("Speech Recognition is not supported in this browser.");
        return;
      }

      const recognition = new SpeechRecognition();
      recognition.continuous = true;
      recognition.interimResults = false;
      recognition.lang = "en-US";

      recognition.onresult = (event) => {
        const lastResult = event.results[event.results.length - 1];
        if (lastResult.isFinal) {
          const spokenText = lastResult[0].transcript.trim();
          setInputText(spokenText);
          translateTextToASL(spokenText);
        }
      };

      recognition.onerror = (event) => {
        console.error(
          "Speech recognition error:",
          event.error || event.message || event
        );
      };

      recognition.start();

      return () => recognition.stop();
    }
  }, [isStreaming]);

  return (
    <div id="appContainer">
      {!isUnityLoaded && (
        <div id="loadingOverlay">
          <div className="spinner" />
        </div>
      )}

      <div id="webglContainer">
        <VSLWebGl
          id="unityWebGl"
          style={{ width: "100%", height: "100%" }}
          unityProvider={unityProvider}
        />
      </div>

      <input
        id="textInput"
        type="text"
        value={inputText}
        onChange={(e) => setInputText(e.target.value)}
        placeholder="Enter text to translate"
      />

      <div id="buttonGroup">
        <button
          id="translateBtn"
          type="button"
          onClick={() => translateTextToASL(inputText)}
          disabled={!isUnityLoaded || isTranslating}
          className={isTranslating ? "btn translate disabled" : "btn translate"}
        >
          {isTranslating ? "Loading..." : "Translate"}
        </button>

        <button
          id="replayBtn"
          type="button"
          onClick={replay}
          className="btn replay"
          className={isStreaming ? "btn replay disabled" : "btn replay"}
          disabled={isStreaming}
        >
          Replay
        </button>

        <button
          id="streamBtn"
          type="button"
          onClick={() => setIsStreaming((prev) => !prev)}
          className={`btn stream ${isStreaming ? "stop" : "start"}`}
        >
          {isStreaming ? "Stop Streaming" : "Start Streaming"}
        </button>
      </div>
    </div>
  );
};

export default App;

Streaming mode app.css file:

#appContainer {
  width: 800px;
  height: 850px;
  margin: 0 auto;
  position: relative;
  display: flex;
  flex-direction: column;
  align-items: center;
  gap: 1rem;
  padding: 1rem;
  background-color: white;
  border-radius: 1rem;
  box-shadow: 0 10px 15px rgba(0, 0, 0, 0.1);
}

#loadingOverlay {
  position: absolute;
  top: 0;
  left: 0;
  width: 100%;
  height: 100%;
  background-color: #d1d5db;
  display: flex;
  align-items: center;
  justify-content: center;
  border-radius: 1rem;
  z-index: 10;
}

.spinner {
  animation: spin 1s linear infinite;
  border-top: 4px solid #3b82f6;
  border-radius: 9999px;
  width: 3rem;
  height: 3rem;
  border-right: 4px solid transparent;
}

@keyframes spin {
  to {
    transform: rotate(360deg);
  }
}

#webglContainer {
  width: 600px;
  height: 600px;
  position: relative;
  border-radius: 1rem;
  overflow: hidden;
}

#textInput {
  width: 100%;
  padding: 0.5rem 1rem;
  border: 1px solid #d1d5db;
  border-radius: 0.375rem;
  outline: none;
  transition: box-shadow 0.2s;
}

#textInput:focus {
  box-shadow: 0 0 0 2px #3b82f6;
}

#buttonGroup {
  display: flex;
  gap: 0.5rem;
  width: 100%;
}

.btn {
  flex: 1;
  padding: 0.5rem 1rem;
  border-radius: 0.375rem;
  color: white;
  transition: background-color 0.2s;
  cursor: pointer;
  border: none;
}

.btn.translate {
  background-color: #2563eb;
}

.btn.translate:hover {
  background-color: #1d4ed8;
}

.btn.translate.disabled {
  background-color: #93c5fd;
  cursor: not-allowed;
}

.btn.replay {
  background-color: #6b7280;
}

.btn.replay.disabled {
  background-color: #5c616b;
  cursor: not-allowed;
}

.btn.replay:hover {
  background-color: #4b5563;
}

.btn.stream.start {
  background-color: #16a34a;
}

.btn.stream.start:hover {
  background-color: #15803d;
}

.btn.stream.stop {
  background-color: #dc2626;
}

.btn.stream.stop:hover {
  background-color: #b91c1c;
}

Documentation

The useVslWebGL hook provides the necessary setup and functionality for integrating the VSL WebGL component within a React application. It returns an object with various properties and functions for rendering, controlling, and interacting with the WebGL component.

Required Parameters

• API_KEY: Your unique API key for accessing the VSL WebGL services.
• loaderUrl, dataUrl, frameworkUrl, codeUrl: URLs provided by VSL for accessing the Unity WebGL build. Each URL is necessary for loading the Unity environment properly.

Optional Parameters

• streaming_mode: Toggle streaming mode. (boolean)

Returned Values

The following values and functions are returned by useVslWebGL:

Value	Explanation
VSLWebGl	A JSX component for rendering the Unity WebGL. Can be styled and controlled within a parent component or container. Example: <VSLWebGl style={{ ... }} unityProvider={unityProvider} />
unityProvider	Required prop for the VSLWebGl component, provides the Unity instance. Pass this to the unityProvider prop of VSLWebGl to initialize the Unity environment.
translateTextToASL	Function to trigger text translation within the Unity WebGL. Arguments: Accepts a single argument (the text to translate). Example: translateTextToASL("Hello, world!")
isUnityLoaded	Indicates whether the Unity WebGL component has fully loaded. Useful for checking readiness to show loaders and before triggering translation. Example: Disabling the translate button until Unity is ready.
isTranslating	Represents the loading state during the translation process. Helpful for displaying loading indicators. Example: {isTranslating ? 'Translating...' : 'Translate'}
replay	Function to replay the last translated text within the Unity WebGL. Arguments: No arguments required.
changeBgColor	Changes the background color of the WebGL component. Arguments: Accepts a single argument, a string representing a hex color value (e.g., #FFFFFF for white). Example: changeBgColor('#FF5733') to set the background color to a shade of orange.
setAnimationSpeed	Controls the speed of animations within the WebGL environment. Arguments: Accepts a single string argument, which can be one of four options: "0" (for pause), "0.5", "1" (default), or "1.5". Example: setAnimationSpeed("1.5") to set the animation speed to 1.5x.
toggleCameraRotation	Toggles the rotation of the camera in the WebGL environment. Arguments: Accepts a single boolean argument to enable (true) or disable (false > default) camera rotation. Example: toggleCameraRotation(true) to enable camera rotation.
error	If any errors occur during loading or translation, this string provides an error message explaining the issue. Example: Display error in your UI if it’s not an empty string. Note: Errors are also logged in the console
streaming_mode	Set to true to use streaming mode. Example: When streaming mode is on, text provided to translateTextToASL function will be added to a queue where the animations will play in sequential manner. Note: When set to false and a new text is provided to translateTextToASL while an animation is playing, will override it and start animating the new sentence. Note: When streaming mode is on the reply function cannot be used, invoking it will do nothing

Example Workflow

1. Initialize the Hook: Call useVslWebGL with the required parameters to initialize the WebGL component.
2. Render the Component: Use in your component, styled to fit your layout.
3. Translate Text: Use the translateTextToASL function to translate input text when Unity is loaded (isUnityLoaded).
4. Replay Last Translation: Use the replay function to repeat the last translation as needed.
5. Handle Errors: Check the error value to catch and display any issues that occur during loading or translation.

License

MIT