Motion Studio

Track points on high-speed footage and get scientific results in real units, each with its uncertainty: speeds, accelerations, strain, impact and rebound, natural frequencies, RPM, projectile velocity and joint angles. Pick the application, place or auto-pick points, track, and read the results; export them to Excel or a PDF report. Open it from AI > Motion Studio ().

1. Start: what are you measuring?

Opening the studio pauses the workflow, hides the toolbox and closes the editor and the viewer; closing it brings back the toolbox and playback. On first open it asks what you are measuring. Each tile is a customer application, not a tracking technique: it opens its own studio, with its own name, colour, named points, tracking settings, result cards and charts. Custom lets you choose everything yourself. The button in the top bar brings this page back to switch application.

The launcher. Each tile sets up points, tracking method, group model, filters, analyses and charts for that application.

After you pick the application, the studio opens on its first step, Source. The studio's state (application, clip, points, keys, scale, settings) is saved in the workflow file, so a saved workflow reopens the studio exactly as you left it.

2. The screen

Everything fits on one screen: the frame on the left with the timeline, the tracking buttons and one lane per point; the current step's pane on the right. The four steps, Source, Points, Track and Results, are in the middle of the top bar; you can move between them at any time. On the right of the top bar: Reset all, this guide, and ✕ to close.

Crash Analysis Studio on the Points step. Lanes: ■ sure, ■ uncertain, ■ placed by the group, ■ lost. The faint line is the whole path, the solid part the last 30 frames; the white arrow is the motion vector.

3. Step 1, Source

The Source step starts with one choice: One camera (2D), a clip or a reader in the workflow (below), or Multi-camera (3D), a rig saved in Camera Studio whose clips open side by side, synced (see Multi-camera (3D)). Switching back to one camera brings your one-camera session back as you left it.

Pick the clip to measure. The list shows the workflow's Movie Reader and Photron Movie Reader nodes; click one to use its clip. Open... opens IW's file browser to pick any other MP4 or Photron MRAW (Assets, Downloads, Documents); it is listed under Opened. The clip is read where it is, never copied or uploaded. The card below shows its size, frame count and playback rate. Next: points goes on; clicking on the frame also goes straight to placing points.

The Source step: the workflow's movie readers, Open... for any other clip, and the clip's facts.

4. Step 2, Points

Most applications name their points, and the pane asks for them in order: e.g. Gauge top and Gauge bottom for material testing, Object and Second point for a drop, Hip, Knee, Ankle, Toe, Trunk for biomechanics, Centre and Rim mark for rotating machinery, Projectile for ballistics. Click each on the frame, then drag to fine-tune with the loupe.

• Auto-pick best features places the most trackable spots (strong corners, no repeating patterns, spread over the frame). How many points sets the count.
• Auto points in box (): drag a rectangle on the frame; its most trackable features join the active body. Keep the box on the object only.
• Add grid points (Material testing) keeps the gauge points and adds distinctive features in between: the grid behind the strain map.
• Bodies (Crash): every point belongs to a body (Head, Chest, Pelvis, Sled reference or one you add with New body). A body holds any number of points; its position and rotation come from all of its sure points together. Click a body chip to make it active; select a point to rename it or move it to another body.
• Tracking settings come from the application and can be changed: Method (Auto, the default, picks one at each Track; or template matching, optical flow, marker centroid and, when their packages are installed, the AI box trackers: see Trackers), Group model (none, auto, similarity, affine, perspective: points that move together help each other through blur and occlusion), and Template / search size in pixels.

Time & scale

The Time & scale card decides what the numbers mean. Both choices show as chips in the top bar and are saved with the studio.

• Capture rate. A Photron file states it (From the Photron file). An MP4 only gives its playback rate, so enter the camera's capture rate; otherwise times, speeds and accelerations are in playback time and the results say so. Vibration, Rotating machinery, Ballistics and Biomechanics refuse to give results without it.
• Scale:
  • The clip's calibration, made in Knowledge Studio > Describe > Calibrate (lens correction included);
  • Two points a known distance apart;
  • From gravity (Drop & impact): the object's free fall against 9.81 m/s², stated as an assumption;
  • From the projectile (Ballistics): its stated diameter, measured edge to edge on one frame with the ruler button and the loupe;
  • From a segment (Biomechanics): a stated segment length, e.g. the shank, over its median tracked length;
  • Pixels: no scale.
  The scale's own uncertainty is carried into every scaled value.
• Time zero at frame: the frame where t = 0, e.g. the moment of impact.

5. Step 3, Track

Track forward or back from the current frame, or one frame at a time. Stop at any time; fix a point by dragging it and track again from there.

While it tracks, the frame shows the newest tracked frame with its points and trails, the timeline follows it, and the lanes fill in. The preview skips frames when tracking is faster than the display; every frame is still tracked.

Every point gets a state on every frame, shown by its colour on the frame and in its lane:

• Sure: found and confirmed.
• Uncertain: found, but a look-alike was nearly as good, the position jumped against the motion, or the point has just come back from hiding and is not confirmed yet.
• Placed by the group: hidden or contradicting, so the other points of the group placed it.
• Lost: not found, or outside the image.

Results use sure frames only. Uncertain, placed and lost frames are gaps, so no peak comes from a doubted position.

Lanes after tracking from keys: tracking starts at the first key; the toe drops to uncertain where it passes the other leg, then is re-keyed.

The trackers: how they work, what they are good at

The Tracker card on the Track step picks the method. New sessions start on Auto (recommended): when you press Track, the studio takes a quick look (well under a second) at the placed points over the next 10 frames and picks the method, then says why in one line, e.g. Auto chose Template: textured targets, steady look. Printed dots on a plain background get Marker centroid; points whose look changes in a short template trial (rotation, blur, cover) get NanoTrack when its package is installed (up to 12 points; otherwise Template with a looser look-alike margin); large moves between frames, or many points on smooth texture, get Optical flow; anything else keeps the application's own method. The choice is made again at every Track. Choosing a method yourself in the list overrides Auto; sessions saved before Auto existed keep their method.

Each application's own method suits it; you rarely need to change it. The speed tag is measured end to end, decoding included (28 points on a 1280 × 674 clip): Fast ≥ 200 fps, Medium 60–200, Slow < 60.

Use an AI box method only when the target turns over, tumbles or changes its look so much that template and optical flow lose it; for measurement on a target that keeps its look, template and optical flow are over ten times more accurate (under 0.1 px on the synthetic tests). Both run on the GPU following the FP16 setting (without a GPU NanoTrack uses every CPU core, 29 fps with 28 points). Each appears in the Tracker card only when its optional feature package (NanoTrack, DaSiamRPN) is installed; a saved session that used a missing one opens with Template.

Group model: points of one body move together. Similarity (move, turn, scale), affine (plus shear) or perspective fit all the body's points each frame; a point that disagrees is rejected, a hidden point is placed by the others (shown as placed by the group), and the fit's rotation and scale turn each patch so matching keeps working. Each point alone uses no fit (independent points, e.g. droplets).

Advanced values (changes apply to the next track run; Defaults restores the application's values):

• Template (px): patch size, odd. Bigger is steadier; smaller follows fine detail. For optical flow it is the flow window.
• Search (px): area searched around the predicted position. Must cover the largest move between two frames; bigger is slower and finds more look-alikes.
• Min confidence: below this match score (1 = identical patch) a point is uncertain and its frame is left out of the results.
• Template update: 0 always matches the first patch (no drift). Above 0 the patch slowly adapts to a target whose look changes (a burning or deforming part), but long tracks can drift.
• Predict motion: search where constant velocity predicts the point, not where it was. Keep it on for anything that moves fast.
• Optical flow only: Pyramid levels (coarse-to-fine; each level doubles how far a point may move between frames, default 3), Back-check (px) (each point is tracked forward and back; one that does not return within this distance loses confidence, default 1 px), Iterations (refinement steps per level, default 30).
• AI box only: Box (px) (the box each network follows, at least 16 px: cover the part of the object, a bigger box is steadier) and Min confidence (the network's score); search, template update, look-alike margin and predict do not apply.
• Template only: Look-alike margin (a second match within this score of the best makes the point uncertain, default 0.2; 0 turns the check off).

6. Step 4, Results

Results shows the application's own cards and four charts, synced with the frame: the crosshair on the charts follows the current frame. Every value carries its uncertainty (± one standard deviation unless it says 95 %), and charts draw a ±1σ band. The fit window is chosen from the data (the longest window whose residuals still match the tracking noise), and the cards show Computing... until the server's analysis arrives.

Results in the Custom application: cards with ±, four synced charts, and the three exports.

7. Multi-camera (3D)

Two to eight synchronized cameras see the same points from different directions. Camera Studio calibrates them as a 3D rig (each camera's lens, position and time offset, from a wand; see Camera Studio's 3D rig mode) and saves a .rig file. Motion Studio opens that rig, shows its clips side by side on one timeline, and keeps every point paired across the views. Results come in 3D: x, y, z in mm with ± for every point seen by two or more cameras, or per view in image pixels.

Open a rig

1. On the Source step pick Multi-camera (3D). The list shows the rigs saved in Camera Studio (${assets}/rigs, with each rig's wand error) and two rendered demos with an exact calibration: the Demo 3D rig (an LED flash, a wand and a thrown ball) and the Demo: drill drop test (3 cameras), a photoreal test bay at 1,000 fps where a cordless drill with four printed bullseye targets falls 1 m onto concrete, bounces and tumbles (impact at 4.43 m/s; camera A is level, so −y is up and a bounce reads straight off the y chart). Camera Studio's 3D rig mode offers the same two demos for the wand calibration. Open rig... opens a .rig from anywhere in Assets; Camera Studio takes you there to calibrate a new one.
2. The views appear in a grid (the layout that shows them largest), each with its camera letter and its time offset to camera A. Scrubbing, the arrow keys and Play move every view together: camera B's frame is camera A's frame plus B's offset. The table lists each camera's clip, frames and offset; − / + nudge an offset by one frame if a view looks early or late (the tag turns amber, and its tooltip keeps the rig's value). Check the sync by eye on an event every camera sees (the Demo's LED flash lights all three views on the same frame).
3. The card shows the capture rate (from the rig), the common frames every clip covers (the timeline's range), and the calibration.

Points in a 3-camera rig: the point placed in A, its epipolar lines in B and C, and the free cell showing the selected point at native pixels in every view. Ring colours: on the line (snapped), found by Find match, proposed (dashed), white: placed by hand.

Place points in every view

1. Click the point in any view. It gets a name (the application's next point name, or Point 1, 2, ...; rename it in the pane). Its epipolar line appears in every other view as an amber dashed line labelled from A: the point must lie on that line there. The line comes from the rig's lens and pose, so with lens distortion it is a gentle curve, drawn exactly.
2. Find match (on by default after the first click) searches along each line for the same patch (normalised cross-correlation on the frame, a narrow band around the line, the patch taken at the exact mark, the peak fitted to a few hundredths of a pixel; about 40 ms) and proposes the mark: a dashed amber ring. Click it to accept, or click where the point really is. A view where no confident match exists says so instead of guessing.
3. A click within the snap band (6 px each side by default) lands on the line; with two marked views the lines cross at the one place the point can be, and a click near the crossing lands on it. Hold Alt to place freely. Drag a mark to fix it (a loupe shows the pixels; zoom the view for sub-pixel work); right-click removes that view's mark (the last mark removes the point).
4. The table lists every point with a chip per camera (its colour says how the mark was made) and its Reprojection error: the point is triangulated from all its marks and projected back into each view; the column shows the worst distance, in pixels, between a mark and that projection. Near 0 means the marks are one 3D point; a few pixels means one view has the wrong feature (the chip turns amber). A point marked in one camera only reads 1 view. The free grid cell shows the selected point at native pixels in every view, with its lines, for checking at pixel level.
5. When a point is in every view, click an empty spot (or ) to start the next one. Editing marks on a frame sets a key, as with one camera.

Track

Track runs the same tracker in every view, with the same settings (Auto included: Auto picks a method per view from a quick look at each), one frame of every view in turn, so all views fill in live. Points stay paired by name: a row always belongs to the same 3D point in every view. The Track pane shows, per view, the frames tracked, the states on this frame and the method used; per point, its chips on this frame and the reprojection error now and over the tracked frames (median / worst). A worst value of several pixels points to a frame where one view jumped to another feature. The lanes show one band per point with a thin row per view. Speed is shown as world frames per second with every view tracked.

A run only rewrites the rows of the points it tracks: a point placed later (say a part that comes into view on frame 500) keeps its own track when you track the others again from an earlier frame, and a key set on an earlier frame leaves a point that did not exist yet alone.

Results in 3D

Results opens on 3D (rig): on every frame, each point seen surely by two or more cameras is triangulated (a linear start, then the position that best fits all its marks). The left shows the 3D path: every point's path in mm, coloured by speed (scale on the right), the cameras as frustums (or as markers on the edge when they are off screen), camera A's x, y, z axes and a floor grid. Drag to orbit, Shift+drag to pan, the wheel zooms, double-click (or ) resets; the marker on each path is the current frame and follows the timeline and Play. Views switches the left back to the cameras. The right shows the usual cards and charts in mm: displacement, speed (m/s) and acceleration (g) in 3D, each with ±, and Distance 1-2 in 3D (on the Demo's wand it reads its 500 mm). The time and CFC settings work as with one camera. The chips A, B, C switch to one camera alone, in its image pixels.

Where the ± comes from (the info line and the 3D position ± card show it):

• Tracking noise, measured from the data: the marks of a point never meet exactly, and the spread of those reprojection residuals over all frames gives the noise in pixels (measured; with too few frames it falls back to 0.3 px and says assumed).
• Each camera's pose as the rig's calibration knows it (Camera Studio's report).
• The rig's scale: the wand's length sets the scale for every frame at once, so its tolerance (0.05 %) is one common factor: about 1.5 mm at 3 m from camera A. It moves absolute positions, but hardly displacements or speeds.
• Where each mark was put: a click or a drop is only as precise as your aim, about a third of a screen pixel, so its uncertainty in image pixels is that divided by the view's zoom (zoom in to place a mark and it shrinks); a Find match proposal is good to about 0.05 px. A mark the tracker never re-centres keeps that offset for the whole track, and an offset along the epipolar line does not even show in the reprojection error, so it moves the point in depth: this part is propagated into the ± (not into speeds). The marker-centroid tracker re-centres every frame, so it adds none.

Checked against the Demo 3D rig's truth: with the exact cameras, the tracked ball is within 0.12 mm RMS and 1σ covers 73 % of the frames (68 % is ideal). With a rig calibrated through Camera Studio's wand, the error is 0.45 mm RMS (a common scale of 0.015 %); the stated ± is cautious there, because it carries the wand's full stated tolerance. Placed by hand in the studio (zoomed in, clicks with a person's aim, Find match, snap), both wand ends come out at 0.6 mm RMS with 1σ covering 68 %. A point seen by only one camera has no 3D value: it is named in the info line, never guessed. Export CSV and Excel gain z and the ± per axis; the PDF report adds the 3D path view as shown.

8. The applications

Each application adds its own analysis on top of the tracked points. The numbers in the mockups are real results from the sample clips.

9. Export

The Results pane has three exports, all built from exactly what is on screen (sure frames, time zero, analysis window, capture rate and units):

• Export CSV: every frame and point.
• Export Excel: a Summary sheet (clip, application, capture and playback rate, scale, time zero, the cards with ±, limits), a Per frame sheet (position in px and real units, confidence, state, displacement, speed and acceleration with σ, plus the application's columns) and a Settings sheet. Applications add their own sheets (e.g. Drop, Frequencies, Spectrum, Rotation, Ballistics, Biomechanics).
• Report (PDF): the first frame with the points, the settings, the cards, a 2x2 chart grid in ms from time zero, the application's table, and a Limits line that says what the numbers rest on (sure frames only, pixels only, playback time, assumed g...).

The PDF report and the Excel workbook's per-frame sheet (illustrative values).

Video with overlays and strobe still

Video (next to the analysis frames) opens a dialog with two tabs.

• Video with overlays: the chosen frames as an MP4 with what the viewer shows. Options:
  • output rate in fps (the file rate, ¼ speed, or real time when the camera rate allows), with the slow-motion factor shown;
  • resolution (source, or a smaller long side) and trail length (off, 10, 30, 100 frames or the whole path);
  • point labels, a timestamp (ms from time zero, frame number, slow-motion factor), a scale bar (needs mm) and a speed readout per point.
  The engine decodes, draws and encodes, so it is fast (about 500 fps on a 322 x 648 clip). A progress bar shows frames and fps; Stop cancels and leaves no file. The file downloads and is also kept in the temp folder iw_motion_studio_exports. It is encoded as MPEG-4 (mp4v): it plays in desktop players (Windows Media Player, VLC) but not inside a browser page.
• Strobe still: one PNG of N evenly spaced exposures (or every k-th frame) on one background, with optional trajectory lines. Moving subject over the median background suits most clips; Lightest suits a bright subject on dark, Darkest a dark one on light.

Fit a model

Fit fits a model to one point or body over a frame interval: x or y position, displacement, speed or acceleration.

Each parameter has ±1σ from the fit covariance scaled by the residuals, plus R² and the residual RMS. The chart shows the data, the fit curve and the residuals. Keep for the export adds the fit to a Fits sheet in Excel and a Model fits table in the PDF.

Fit positions where you can. Speed and acceleration samples are smoothed, so their ± is a lower bound. With a scale from gravity, a free-fall fit cannot measure g: the studio says so.

Compare two trials

Compare opens trial A against trial B on one screen.

• Save as trial stores the current session (tracked points, time scale, names). Track the other clip, then compare it with the saved trial.
• Sync on time zero, the peak of the chosen quantity, or time zero + offset (ms).
• Both clips play side by side (or overlaid) at the synced time. One chart shows the chosen quantity of both.
• The cards give each peak with its time, the peak difference, the RMS difference after sync and the shift applied to B.

10. Set up with a prompt

AI setup (the button in the top bar) opens a dialog that takes a plain sentence, e.g. "head acceleration of the dummy in g"; suggestion chips show typical requests for the application, including the crash templates. The dialog shows the proposed setup as a table (application, method, group, template / search, start frame, filter, smoothing, units) and nothing changes until Apply. Later requests refine it ("use optical flow", "smooth more", "it tumbles"). With an empty prompt, Set up from the clip proposes a setup from what Knowledge Studio knows about the clip: its notes, timed boxes, activity, detections and calibration. The AI only reads this text, never the frames. Without an AI, built-in rules are used (AI unavailable: rules used).

11. Reset all

Reset all in the top bar starts the studio again from scratch: the application, points, keys, tracking results, scale and settings are removed, and the studio goes back to the application picker and the Source step. It asks first:

The confirmation. Esc or a click outside cancels.

12. The Multi-Point Tracker node

The studio runs on the C++ node Multi-Point Tracker (cv.multi_point_tracker, Tracking category, CUDA and CPU builds), which you can also use in any live workflow:

• Methods: template matching (NCC, sub-pixel peak), pyramidal optical flow with a forward-backward check, marker centroid, and the AI box trackers NanoTrack (all points in one batched network pass) and DaSiamRPN (up to 8 points), each from its optional feature package.
• Group model: 2 points similarity, 3 affine, 4+ perspective; only sure points vote, and hidden points are placed by the group and re-acquired.
• World units: optional lens (a Camera Studio .calib) and world_homography inputs.
• Outputs: a preview (trails, state colours, motion vectors, group outline), the points matrix for the current frame, and history: every frame since the restart, so a slow reader (logger, Live Sheet, Python) never misses a frame. history_frames caps it (default 10,000, 0 = off).
• Studio mode: the frame and keys inputs let it track in either direction and restart from hand-set keys; with frame = -1 it runs live.

The tracker's fixed cost is under 1 ms a frame on both builds; the studio shows its tracking speed (fps best) after each run.

For box tracking in other workflows, Multi Box Tracker (cv.multi_box_tracker, NanoTrack feature package) follows N boxes (x, y, w, h) at once and outputs the boxes, a confidence per box and a preview; Start or new boxes restart it. Demo: demos/Inference/Multi Box Tracker.json.

13. Good results and limits

• Use the camera's real capture rate. Copies from the web are often slowed or have repeated frames; times are only as good as the frame rate.
• Choose points with texture or a marker, on the object only. Avoid repeating patterns, reflections and edges that slide (a turning helix, a limb's outline).
• Check the lanes: long uncertain or lost stretches mean the point needs a key, a bigger template, or a different method.
• A clip calibration is only correct on its plane: points in front of or behind it read too long or short.
• One camera measures in the image plane (use Multi-camera (3D) for depth). Out-of-plane motion shortens lengths and angles; Biomechanics puts this in the ±.
• A camera mounted on the moving vehicle or rig shakes: add a reference body (4-6 points on rigid structure, both sides of the frame) so its motion is subtracted. On NHTSA 11500 this cut the head's difference to the accelerometer from 15 g to under 6 g RMS.
• Acceleration needs a clear view: compare video with a sensor only while the point is visible (on 11500, before the airbag covers the head, about 45 ms).
• AI box trackers keep the point through rotation and blur but are box-level (about 2-3 px), not sub-pixel: prefer Template or markers for precise measurements.