A collection of open source libraries to help build your custom robot platform on top of Formant APIs.
With this repo's tools you can make custom web and 3D experiences on top of Formant using ThreeJS, get information about your fleet of devices, create realtime visualizations of your robots, and more.
- Embed your own application in Formant
- Create your own application that uses Formant's APIs and RealtimeSDK for data
- Create WebView's for mobile applications that show 3D visualization of your fleet
We understand you want complete control over your user experience, so Formant provides:
- A Data SDK to get realtime and historical data
- Simple 3D elements that you can import into your ThreeJS scene
- UI custom elements to make your application visually look similar to Formant's ecosystem
This depends on the type of app you're making:
Using user login credentials or service accounts, you can use the Data SDK to get access to device data.
import { Authentication, Fleet } from "@formant/data-sdk";
await Authentication.login("sam@robot.xyz", "12345");
// Get all devices
const allDevices = await Fleet.getDevices();
// find the device(s) you want
const device = allDevices.find((d) => d.name === "spot");
// Get data ...
const data = await device.getLatestTelemetry();The Data SDK will be able to figure out from url the authentication needed to access the current viewing device.
import { Authentication, Fleet } from "@formant/data-sdk";
await Authentication.waitTilAuthenticated();
// Get the context of a device is passed along as a query string
const device = await Fleet.getCurrentDevice();
// Get data ...
const data = await device.getLatestTelemetry();- install NodeJS
cd examples/teleopnpm installnpm run dev- Add the url indicated in terminal as a custom view with the right query parameters ( most likely "?device={device_id}&auth={auth}")
- Go the device page you want to see the custom view on
- Select the custom view from the views dropdown
For more detailed instruction on custom views, see our custom view readme.
This ThreeJS element will show a URDF of a robot using the zipped URDF uploaded for formant and control it's joint states in realtime.
This ThreeJS element will show a realtime video stream of your device as a plane.
If your just interested in using our APIs, there's two main steps
await fetch("https://api.formant.io/v1/admin/auth/login", {
method: "POST",
body: JSON.stringify({{email:"sam@robot.xyz", password:"12345"}}),
headers: {
"Content-Type": "application/json"
}
});curl -X POST "https://api.formant.io/v1/admin/auth/login" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-d '{"email":"sam@robot.xyz","password":"12345"}'This will return a JWT token.
{
"authentication":{
"accessToken": "abc......xyz",
...
},
...
}// the token from above
cons token = "abc......xyz"
await fetch("https://api.formant.io/v1/admin/device-details/query", {
method: "POST",
body: JSON.stringify({{enabled:true, type:"default"}}),
headers: {
"Content-Type": "application/json",
"Authorization": "Bearer "+token
}
});curl -X POST "https://api.formant.io/v1/admin/device-details/query" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer abc......xyz" \
-d '{"enabled":true,"type":"default"}'Yes, you can do it in two ways, first as an ES6 module:
<script type="module">
import {
Authentication,
Fleet,
} from "https://cdn.jsdelivr.net/npm/@formant/data-sdk/dist/data-sdk.es6.js";
await Authentication.waitTilAuthenticated();
const devices = await Fleet.getDevices();
window.document.body.innerHTML = devices.map((_) => _.name).join("<br>");
</script>or as a non-module:
<script src="https://cdn.jsdelivr.net/npm/@formant/data-sdk/dist/data-sdk.umd.js"></script>
<script type="module">
const { Authentication, Fleet } = window.FormantDataSDK;
await Authentication.waitTilAuthenticated();
const device = await Fleet.getCurrentDevice();
window.document.body.innerHTML = JSON.stringify(
await device.getLatestTelemetry()
);
</script>
To prepare a URDF for the web, you must collect the .urdf file from ROS. You will be consructing a zip that contains the:
- urdf in the root folder
- meshes referenced by the urdf
Currently formant only supports .dae models. You can easily convert models using ctmconv.
Install using ubuntu:
apt-get install openctm-tools
ctmconv leg.stl leg.dae
An example zip layout will look like
- my_arm.zip
- my_arm.urdf
- meshes/
- arm1.dae
- arm2.dae
It's important that your meshes be referenced by relative links. Remove absolute paths that start with "package://".
<mesh filename="meshes/base_link.dae" />
Yep, checkout the example.
await device.startRealtimeConnection();
const j = await device.createCustomDataChannel("string_command");
await j.waitTilReady();
j.send("konami");
j.addListener((message) => {
console.log(message);
});
def string_command_callback(message):
print(str(message))
if __name__ == "__main__":
c = FormantClient()
c.register_custom_data_channel_message_callback(
string_command_callback, channel_name_filter=["string_command"])
You can change your connection network reliability settings
const options = { ... };
const j = await device.createCustomDataChannel("joystick", options);
Here's some combinations we find useful
Reliable TCP-like: `{ ordered: true }`
- This is the default option `createCustomDataChannel` uses
High Speed UDP-like: `{ ordered: false, maxRetransmits: 0 }`
- Good for non-critical one shot messages
Time Constrained UDP-like: `{ ordered: false, maxPacketLifeTime: 300 }`
- Good for things like joysticks
When building an embedded app, you might want a quick way to get a website that looks like Formant. This css augments class-less semantic HTML making it incredibly easy to use.
<link
rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/formantio/toolkit/css/formant.css"
/>See an example here.
Yes, we even provide a simple component ui-sdk-login to handle authorization for you if you just want to focus on user experience.
See an example here
Our partners building on Formant.