Cameras and 3D

Filmed the animal from more than one angle? Put every camera in one project and keep them in sync.

1. All the videos in one folder? File → Open Folder of Videos… lists every video in it (picture size, frame rate, length): tick the ones to import and put them in camera order (Move up / Move down): camera 1 is the reference clock, and the order is the order of the cameras in calibrations, 3D and exports — keep it the same as your calibration's. They open as one project, each camera named after its file (files that share a name, like a GX010001.MP4 in each of eight camera folders, are named after their folders instead, or get a (2) — every camera name is unique, because offsets are matched by name), saved at once as <folder>/<folder>.kinetrace unless you untick that; if a project of that name is already there, you are asked before it is replaced. Or open the first camera's video as usual, then ＋ Add video in the CAMERAS panel (top of the right dock) — pick one file or several at once, then put the project's cameras and the new ones in their order (the new ones are marked new). Camera order… under the CAMERAS list reorders an open project at any time: each camera keeps its points, offset, frame rate, lens and calibration (offsets are re-measured from a new camera 1 — nothing moves in time); a 3D result is dropped. Each camera appears as its own view in a grid above the timeline, showing its own tracked points and silhouette. All cameras share one list of points: a point added in one camera appears (greyed, not placed yet) in every other camera's LAYERS tree, under the same animal; select it there, click it on the video and track it. Animals are shared by name too (an animal made in one camera appears, with no silhouette, in the others). Rename, delete and moving a point to another animal act on every camera at once (one Ctrl+Z step), a skeleton chosen in one camera is given to the others, and a camera added later gets the whole list. Switching cameras keeps the selected point selected. Up to 15 cameras. Adding a camera to a calibrated project keeps the calibration, but 3D needs every camera calibrated: calibrate again with all of them, or remove the new one with its × to get 3D back.
2. Line them up. The first camera you load is the reference — its offset is always 0 and is not editable. Every other camera carries a frame offset: the frame this camera shows when the reference is at its frame 0. A camera switched on N frames after the reference has offset −N; one switched on earlier has +N. That meaning does not change when you switch which camera you are working in. The quick way is 3D → Sync → Sync Cameras (Sound / Motion)…: from the cameras' sound tracks (claps, voices — with a noise filter; the usual way for hand-started cameras) or from how much the picture changes frame to frame, each camera gets an offset and a verdict (CLEAR / WEAK / NONE). The motion method needs a change that is large in the picture (a person or animal crossing, a flash, the wand close to a camera); a small object far away is too little of each frame to be seen, and its verdict is NONE — use sound there. It works on a stretch around the instant on screen — the motion method looks in exactly the window you ask for — so park the playhead near the clap first; Cancel stops a running search at once (also while sound is being read), and a silent track or a file that could not be read is named as such per camera; then press Apply the ticked offsets. The manual way (the reference camera's offset box and ◂ ▸ are greyed out: it is the clock): scrub to something every camera saw — a flash, a clap, the first contact — then nudge that camera's offset with ◂ ▸ (or type it) until the same moment is on screen in both views, or press Align here to take what you are looking at as the match (on the reference camera's row it retimes the camera you are working in, since the reference is the clock and never moves). The panel then tells you the overlap — the frames for which every camera has a picture.
3. Work one camera at a time. The camera you track is the one selected in the panel (or just click its view). Points, silhouette, skeleton, events, undo and the timeline are all that camera's; the others stay visible but view-only, scrolling along with the playhead through their offsets. Switching cameras keeps you on the same instant, so you never lose your place. A plain click on another camera's picture only switches to it; with N (Point) armed the click switches and places the point there in one go. A switch puts the segment tool down and drops a half-marked event (they belong to the camera you left), while the toggles — Follow, Auto-pause, ROI, Mask, the point model — stay as you set them. Every camera has the same list of points, in the same order; ＋ Point above the LAYERS tree makes one with no position yet in every camera, ready to be clicked in each. Track ▾ → Every camera (or Shift+T once) tracks the selected points — or all — in each camera that has them at this instant, ball markers included, all cameras at the same time, each live in its own view (the button reads "Track ▶ · 3 cams"; semi-automatic F steps a frame in every camera; X stops every camera at once; one Ctrl+Z undoes it in every camera; a point lost in one camera does not stop the others, and the playhead ends where it was lost). The cameras take turns on the graphics card a frame at a time: the memory of one run, and each camera's result is exactly what tracking it alone gives. The Track ▾ menu stays open while you tick, so the run mode, Every camera and the point model combine; if only the working camera has the point here, Track says which camera lacks it. While Every camera is ticked, every camera draws its own trails (View → Trails: Off / Last 10 frames / Custom…), during the run and while you scrub afterwards.
4. Everything is saved in the project (Ctrl+S): every camera's video path, tracks, silhouette and offset, and which one you were working in. Reopening a project reopens all the videos and asks you to locate any that moved. A camera whose video you cannot locate is left out of that session, and the project is then not overwritten (Save asks for a new name), so reopening once the drive is back gives you every camera again.
5. Export for 3D: choose ALL CAMERAS — DLTdv8 xypts… (Ctrl+E). It writes one file with pt1_cam1_X, pt1_cam1_Y, pt1_cam2_X, … columns, matching landmarks by name across cameras — the input DLTdv / Argus / a triangulation script expects. Row k is frame k of the reference (first) camera, counting from 0; the other cameras are sampled at the same instant through their offsets, and NaN fills every cell a camera has no data for. Pixels are DLTdv8's: top-left origin, first pixel = 1. The *_pointnames.csv sidecar names the points, the convention, the rows and which video is cam1, cam2, …. The shared point list keeps the names equal in every camera, so each landmark lines up. The ordinary 2D exports stay per camera and default to <project>_<camera>_tracks.csv, so one camera's files never overwrite another's. To SEE the cameras together, File → Export → Overlay Video → Views: All shown cameras together writes one synced video of the cameras shown in the window.

View → Other cameras (or Sync all in the CAMERAS panel) decides how the other cameras follow the playhead: Sync all views (the default — every camera shows the playhead's instant, and a single step shows every camera's new picture at once), Active view only (Ctrl+Shift+2 — only the working camera reads its video; the others stay on the picture they last showed, dimmed and captioned "not following", whatever you do, until Sync all views is back or you click one to work in it; the guides in the working camera keep working, because they come from the other cameras' tracks) or Only the working camera (Ctrl+2, or just M — the others are hidden and stop decoding, which gives scrubbing the whole disk back; M again puts every camera back in the same order and sizes, hidden ones still hidden). With many cameras, choose which are on screen: each camera is ONE line in CAMERAS (its number, name and offset; ▶ opens its controls — Align here, the offset box, frame rate, remove) with an eye that shows / hides its view (a hidden camera is not decoded; its tracks still count for 3D and the guides; Show all brings them back). With Track ▾ → Every camera, a hidden camera is tracked too, without being drawn — the Track button says "(n hidden)" and a warning names them when the run starts (click it to show them all). Drag a view by its title bar onto another to move it there — only the screen changes, the cameras keep their numbers; right-click a title bar to hide it or put the views back in camera order. The choice is kept with the project. The timeline always shows the working camera (outlined in blue); where only another camera has a body part, its row shows a thin line (hover it: which cameras), so a point tracked in another view never looks lost. Cameras may run at different frame rates and resolutions: a camera at twice the reference rate is marked ×2 in the panel, steps two of its frames per reference frame, and carries its offset in its own frames. Each camera's row shows the rate it recorded at (30 fps): high-speed footage is often saved for slow-motion playback, so the file says 30 while the camera filmed at 240 or 1000 — click the button and enter the real rate (marked *; offsets are kept, the ×n and every time in seconds follow). The same button fixes a single camera's speeds.

Keep a lens profile for later. File → Export → Lens Profile… saves any camera's lens profile at any time, however it was made (checkerboard, GoPro's lens model, a file): Kinetrace .klens.json, OpenCV .yml / .json, or an Argus .txt line for a standard lens. A .klens.json measured from a checkerboard names its board video (name and path) and the frame numbers of the boards it used, so you can go back to them. 3D → Calibrate → Load a Lens Profile for Cameras… attaches a saved one to every camera you tick, in one go (a rig of 5 + 5 or 4 + 4 + 2 identical cameras: one load per group). Each camera's row says what it records and what the file does to it: fits, fits turned (a camera filmed on its side), does not fit (another picture size, cannot be ticked) or replaces its profile (never ticked for you). Tick every camera it fits ticks the rest; untick cameras of another model, lens or mode - the same picture size does not prove the same lens. Use it for the same camera in another project, or for other cameras of the same model, lens, zoom and recording mode.

Identical cameras share one lens calibration. Ten cameras of the same model, lens, zoom and recording mode need only one checkerboard: the lens wizard's last page offers Also use it for the other cameras with … pictures, and the wand wizard's Cameras page has Use for all on each camera's lens row (a camera with a different profile of its own is replaced only after a yes). Units of one model still differ slightly, so the wand fine-tunes the focal length of every camera sharing a profile. On a test rig that made a shared profile as accurate as one per camera (0.34 % camera spacing error against 2.1 % with the focal lengths held).

GoPro footage (read only for videos straight from a GoPro HERO or Mission; a copy another program exported usually loses it). When a GoPro video opens, Kinetrace reads what the camera wrote into the file and warns once about what can spoil 3D: stabilisation left on (HyperSmooth / EIS re-warp every frame: no lens calibration holds), dropped frames, a camera that was knocked or moved during the recording (with the frame), and lens modes, frame rates or shutter speeds that differ between cameras. 3D → Cameras Overview… lists every camera (GoPro or not): what it records, its lens profile and whether it is used, whether the 3D calibration covers it, and for GoPro footage its settings, its tilt from the gravity sensor, dropped frames and when it moved (click a Moved cell to go there). The same at a glance: each CAMERAS row has a lens badge (lens ✓ / lens ✗ not used / no lens); hover it for the details, click it for the overview. Every GoPro video also carries GoPro's own lens model for its recording mode, valid over the whole picture: use it as it is (Use GoPro's lens model… in that window, or Use GoPro's lens without boards in the lens wizard; it is the lens design, so each unit differs by about 1 % in focal length and ~10 px in centre, which the wand partly refines), or, better, choose GoPro lens from the video + your boards in the lens wizard: GoPro's curve is kept and only the focal length and centre are measured from the checkerboard, so the board need not reach the corners. On a real HERO12 2.7K Wide video that matched the full checkerboard fit (0.68 px) without its run-away corners. With timecode in every camera, Sync Cameras starts its search from the start times (the camera clocks are set to about a second: a starting point, not the sync).

From cameras to 3D (the 3D menu)

6. 3D → Calibrate → Calibrate a Lens (checkerboard)… — for wide-angle / action cameras (needed) or when unsure (the report says in pixels whether it matters). Best with the project of that camera open, so the profile can be attached to it at the end; with nothing open the wizard still runs and ends by saving a lens file to attach later. Its first page saves the printable board (Save the checkerboard to print…). Print it, film it with that camera at the same picture size as its videos (another size is refused, naming both; a camera filmed on its side or upside down records the same size, its video only plays turned: film the board with the camera held normally and the profile is turned to fit, exactly; a Kinetrace lens file saved before the turn was recorded asks you to find its checkerboard video and reads the turn from it), and choose that video on the wizard's second page. The wizard finds the board, fits the lens (standard or fisheye model, chosen automatically: auto fits both to the same boards and says when it leaves the fisheye out because it runs away at the edges), shows a verdict with reasons and a before/after picture, and Attach to … gives the profile to the camera; I already have a lens file… takes a .klens.json or an Argus / DLTdv profile instead and skips to the last page. The wand wizard's Cameras page opens the same wizard per camera (Calibrate… / Load file…). The wand calibration then straightens that camera's points; with every camera profiled it takes their focal lengths from the checkerboards, with only some it refines the focal lengths from the wand (and, with lens distortion ticked, fits the distortion of the cameras without a profile). 3D → Calibrate → Calibrate Cameras with a Wand… — calibrate inside the app, no MATLAB: track the two ends of a wand of known length (Point ▾ → Ball marker on each ball; the same landmark names in every camera), optionally a dropped ball for the vertical and an independent scale check, and the wizard solves the cameras (sparse bundle adjustment, the easyWand principle) and reports a plain-language verdict — good / usable / not good enough — with the reasons and what to change. GOOD needs every camera to keep at least 30 usable observations (and at least half of its own) and at most 5 % of all clicks rejected as outliers; Find it automatically for the focal length works when only some cameras have a lens profile; Use this calibration makes it the project's. It opens with one camera too, to review the wand tracking, and says a second camera is needed before it can calibrate. Any of the cameras: the wand page has a tick per camera (Cameras to calibrate); the cameras that share at least 10 wand frames with another are ticked, a camera where the wand was never clicked is unticked and says so, and any camera can be left out by hand. A camera left out is not calibrated: 3D (Reconstruct, Volume, the guides, the 3D view) runs on the calibrated cameras — two are enough — and each step names the cameras it left out. Track the wand in them later and calibrate again with them ticked. Timing check: before solving, every pair of ticked cameras is tested for a whole-frame sync mistake (the share of wand clicks that fit one epipolar geometry, with one camera's offset moved up to 30 frames each way). When the pairs agree that a camera is a whole frame or more off, the page says by how much and with what evidence, and offers Apply these offsets and calibrate or Calibrate with the offsets as they are. A camera off by a fraction of a frame is named ("about +0.29"), never rounded into a whole one, and a camera it cannot time gets the reason (too few shared frames, or no clear peak). Offsets applied in the wizard wait for Finish: the page says what the calibration is made at (old → new), the button reads Use this calibration and the new offsets, and only then do the offsets and the calibration go into the project together (one Ctrl+Z takes both back); Cancel leaves the project as it was. Camera timing: after every calibration the report checks the timing with the wand — each camera's wand ends as the OTHER cameras put them (median px, good / ok / poor: a camera whose offset is off stands out), wand length against wand speed, a fixed-calibration estimate (an under-estimate), and the start / middle / end of the recording (a drift or a dropped frame is said, never corrected). Refine offsets with the wand (optional) calibrates, estimates the offsets with that calibration fixed, calibrates again, … until nothing moves (at most 4 rounds), so the calibration it offers is made at exactly the offsets it suggests; it catches a camera a frame or two off too. A table gives each camera's offset now → refined, the other cameras' prediction now → refined and a verdict (good rows ticked); Apply the ticked offsets then pends them as above. The first ticked camera is the run's clock. The 3D picture opens by itself after each calibration (Show in 3D brings it back). It draws the calibration: each camera's pose pyramid and its full field of view (the picture's border, lens included, turned into rays, to a flat far plane past the wand — Blender / Qualisys style — its four sides filled faintly in the camera's colour, denser where fields overlap), every wand position used as small dots, coloured good / ok / poor / rejected (hover: frame and error; zoom in to tell single positions apart), and the space seen by at least N cameras (translucent, drawn over the fields of view; counted by projecting a voxel grid through each camera, up to its far plane) with "the wand cloud lies x % inside it" and a verdict (≥ 90 % good, ≥ 70 % ok). Every layer is its own tick, and Cameras ▾ picks the cameras drawn and counted. File → Export → Calibration… writes a self-describing .kcal.json for your animal projects, a _dltCoefs.csv (MATLAB 1-based pixels) for DLTdv users and the report as text; when a camera has a lens correction the csv is for lens-corrected pixels and a _dltCoefs_README.txt says so. File → Import → Calibration… — a .kcal.json, or the 11-parameter DLT that DLTdv / easyWand / Argus write (…dltCoefs.csv, one column per camera), an …easyWandData.mat, a DLTdv8 …dvProject.mat saved as MATLAB v7, which also carries DLTdv's lens undistortion (its local-weighted-mean transform is rebuilt from the control points stored in the file), or OpenCV-style cameras (a K matrix per camera plus R/t, as JSON or text; also an OpenCV .yml / .yaml / .xml / .json camera file). Map file columns to cameras (by picture size where the file records it) and state the pixel convention: a dltCoefs.csv or an easyWand file does not say, so the choice starts empty and OK waits for it (DLTdv8 / easyWand count from 1, y down — the first choice; a wrong one moves every 3D point by about a pixel with a normal-looking residual, so it is asked, never guessed); a .kcal.json, a DLTdv8 project or a K + R/t file states its own and the choice is locked. Inside Kinetrace every calibration is then kept 0-based, y down; the DLTdv / easyWand / MATLAB exports add the 1 back. A dltCoefs.csv records no picture sizes, so match its columns to your calibration software's camera order. Every camera can also be set to not calibrated (a file with fewer cameras than the project leaves the rest so); the exports then hold the calibrated cameras under their own names, and a dltCoefs.csv of such a calibration comes with a README saying which camera each column is.
7. 3D → Sync → Estimate Sub-frame Offsets… — cameras that were not genlocked differ by a fraction of a frame; the fraction is found from the tracks themselves by minimising the triangulation residual (grid search, then a joint Powell refinement) and applied after you confirm. A card counts the tests and has a Cancel; when the landmarks do not move enough in the window to time anything, the verdict reads flat: the landmarks do not move enough to time it and it advises against applying. Each camera is judged on the landmarks it shares with the others (fewer than 30: at most weak). Whole-frame alignment by eye first, this afterwards. On the wand recording itself use Refine offsets with the wand instead: it calibrates again at every step.
8. 3D → Reconstruct 3D Landmarks (Ctrl+3) — every landmark seen by ≥ 2 cameras at an instant is triangulated (linear DLT), with DLTdv's residual sqrt(Σe² / (2n−3)) per position. Tracks are interpolated at the fractional local frame of each camera. The 3D view (3D → Show 3D View, Ctrl+5) opens and follows the playhead (with a calibration it also has the wand calibration layers: a click on a wand end jumps to its frame; after a reopen the wand is rebuilt from its two landmarks - without them, the cameras and the space they cover only), and a verdict comes with the numbers (GOOD / USABLE, with care / NOT TRUSTWORTHY, the worst landmark named). Where one camera disagrees with the others, a magenta band marks the stretch on that camera's timeline lanes. Epipolar guides (switched on whenever a calibration is loaded or made; View → Epipolar guides from the other cameras) draw, in every camera on screen, the line the selected landmark must lie on given where the other cameras see it — place a point in camera 1 and camera 2 shows its line at once. Each line is computed exactly and runs to the picture's edges (or ends at the vanishing point / the other camera when that is in the picture; dotted where a lens model is only guessing). Once two cameras have the landmark, each of them shows its 3D rmse beside it (the reconstruction residual in px, DLTdv's definition: green good, yellow usable, red poor; also in the status bar) and the lines turn faint in every camera. With only two cameras the 3D rmse never reads better than yellow ("ok"): an error along the line joining the two views is invisible to two cameras. Once two other cameras have it, a ◇ shows where they put it: A, or a click on the ◇, places it exactly there; in a camera that already has it, the ◇ says how many px apart the placements are; no ◇ when those cameras disagree or see it along one line (the label says why). Alt+click draws where any spot can be in the other cameras, editing nothing (Esc clears it); a plain click with nothing selected places nothing and says so. A camera added after calibrating gets no guides; the calibrated ones keep theirs. Right-click a landmark → Snap to the other cameras' rays here puts it there (one undo step); 3D → Re-track Disagreeing Stretches… does that at the start of every magenta stretch, re-tracks it, reconstructs again and asks Keep / Undo with a before / after verdict (X or Space stops the whole queue).
9. 3D → Carve Volume at This Frame (Ctrl+4) — the visual hull: the space every camera's silhouette agrees on, carved on a voxel grid, meshed (marching tetrahedra on the signed distance), with its volume in the caption. It needs the animal segmented in at least three cameras at that instant (two silhouettes carve a long sliver, not the animal, and the app says so). A camera that frames the whole animal also votes against the space outside its picture, so a tightly framed camera no longer leaves a two-camera sliver; where a camera cuts the animal at its picture edge it cannot, and the app warns (Volume partly checked by two cameras only) when more than about a tenth of the volume was checked by fewer than three cameras. File → Export → Mesh of This Frame… writes OBJ/PLY. 3D → Set World Axes… (needs a 3D result) lets you choose the world yourself: pick the landmark for the origin, one that +X points toward and one that +Y points toward, either at the frame on screen or as each landmark's median position over the whole result (for markers that do not move); +Y is made perpendicular to X and +Z follows the right-hand rule, which also gives a mirrored (left-handed) easyWand / DLTdv world a proper right-handed one. The calibration and the 3D result are re-expressed in that frame; the pictures and every camera's pixels are unchanged. A volume carved earlier is cleared (carve again), and the calibration and 3D points should be exported again. Ctrl+E → 3D landmarks writes the xyz CSV + residual sidecar, Ctrl+E → 3D kinematics smoothed velocities and accelerations. Exported 3D points are always in the same world as exported cameras. A DLTdv xyzpts file keeps frames with no 3D position as empty frames, so later frames stay on their own rows. Calibration, offsets and 3D live in the project. Most 3D entries stay clickable with one camera or no calibration yet, and say what is still missing.

You can start tracking from any frame of interest — tracking always begins at the frame you are looking at. Points that have no position at that frame are skipped (their existing tracks are kept untouched).

ROI zoom (the ROI toggle in the control bar, on by default): when your points sit in a small part of a high-resolution frame — say three balls on a stick somewhere in a 4K shot — the tracker automatically works inside a crop around them, so the model sees the objects with several times more detail. It engages only when it clearly helps and follows the points by transparently re-seeding when they near the crop edge.

Appearance lock: right-click a point — either its marker on the video or its entry in the LAYERS panel — and choose "Lock to seed appearance". For a distinct, rigid feature this snaps the track back whenever the point's original appearance is found nearby, catching slow drift. Off by default because it changes the plain "track the placed location" semantics. It applies to plain points only: a circled region doesn't need it (its center is already stabilized by the member fit), so the menu shows it grayed out there.

Checking a lens calibration

A calibration is a number you cannot eyeball, so the lens wizard shows its evidence (boardreview.py). After the scan you get every board it found, in a scrollable gallery: the detected corners, the first row drawn as a line with corner 0 ringed (so a flipped or rotated detection is obvious), the board's own X/Y/Z axes from solvePnP, and that view's reprojection error colour-coded. Click a board to open the full-resolution frame and drag a corner onto where it belongs — the drop is cornerSubPix-refined, so a rough drag still lands sub-pixel. Tick or untick any image to choose what the fit uses, with Best spread / All / None / Drop the worst as shortcuts (All fits every ticked board). The board size is locked while the scan runs, holding the board still counts as one pose (distinct poses are what make a view count), the lens verdicts' pixel limits use the picture's longer side (a portrait video is judged like a landscape one), and the printable board saves into folders with accented names. After unticking a board or moving a corner, press Fit the lens from the ticked boards: Next waits for that refit, so the profile you attach is the one fitted from what you see. The last page's coverage map lights the boards the fit actually used.

The automatic choice (lens.auto_select) runs twice: a diverse spread over the picture, near and far, tilted and square-on (a fit from one pose is degenerate however many frames it has), then a provisional fit and the removal of views that reproject worse than 3× the median — unless that would leave too few. It returns the reason in words, and the page shows it.

---

← Segments, silhouettes & skeletons  |  Home  |  Human bodies →