AI Studios

User guide for the AI Labeling, Video Segmentation, Video Splitting, and Fine Tuning Studios in INFINIWORKFLOW.

1. AI Studios Overview

INFINIWORKFLOW provides four tightly integrated studios for dataset creation and machine learning preparation: the AI Labeling Studio and Video Segmentation Studio for building datasets, the Video Splitting Studio for breaking up long or high-speed sequences, and the Fine Tuning Studio for training models on the result.

What are the AI Studios?
Use the studios as a connected workflow: label and prepare data, split long sequences when needed, and fine-tune models from the resulting dataset.

2. AI Labeling Studio

The AI Labeling Studio is a specialized annotation environment for compiling high-quality ML dataset corpora. It combines an interactive vector/raster edit canvas, sequential frame propagation, and automated checkpoint management for pixel-accurate labeling.

Task Types Supported

Labeling, Tracking, & Navigation

Annotate frames by clicking the Edit button on an extracted frame directory. Once in the viewer, select your active class, use a path tool (such as Add Rectangle or polygon points), and click and drag to define your shape. Switch classes as needed, and save your progress.

Icon Action Type Description
Save Labels All Saves the current frame's annotation state to local storage and the active SQLite cache (Hotkey: s).
Timeline Navigation All Cycles forward or backward through the extracted image frames (Hotkeys: a for previous, d for next).
Calculator Jumper All Opens a numeric keypad overlay - enter a specific 6-digit frame index (e.g. 000045) to jump straight to that position on the timeline, without repeatedly clicking next/previous.
AI Smart Segment Detection / Segment Uses Segment Anything Model (SAM) prompt grids to automatically generate multi-object boundaries from single point clicks.
Sequential Propagation Image Segment Uses Segment Anything 2 (SAM2) tracking to propagate segmentations forward and backward, automatically labeling the keyframes between manual annotations.
Spot Check Preview Image Segment A fast playback slider for reviewing segment blending and overlay consistency across frames.
Vectorization Image Segment Deletes an automated outline segment to trigger vectorization, converting the rasterized mask into editable Bezier anchor nodes.
Manual Overrides Image Segment Modifying a frame already marked "AUTO" elevates it to "MANUAL", invalidating adjacent automatically tracked regions to avoid spatial drift.
Checkpoint Rollbacks All Cleans, prunes, or deletes prior automated frame segments and manual tags, or restores a previous workspace checkpoint.

Live Labels Summary & Report Verification

To keep your dataset's annotation status fully visible, the AI Labeling Studio adds a live, auto-updating Summary & Report column to the results list (right panel):

  • Interactive Class Summary: Each frame row shows a live summary of the labels/classes drawn on that frame (e.g., Apple, Banana), so you don't need to open a frame to see what's already annotated.
  • Auto-Segmentation Badge: Frames tracked automatically via SAM2 propagation display a distinct AUTO badge, marking them as machine-generated candidates pending review.
  • Error Reporting & Validation: If a frame's labels point to classes that no longer exist or are out of bounds, the summary column shows a prominent INVALID warning badge, so faulty data can't slip into training.

Example: the Summary & Report column - a normal frame, an automatically tracked one, and one flagged invalid

frame_00183.jpg Apple, Banana
frame_00184.jpg Apple, BananaAUTO
frame_00185.jpg INVALID

Dataset Fine-Tuning & Weights Compilation

After compiling labeled sequences, use the Fine Tuning controls to compile native models (e.g. YOLO, UNet) based on your custom annotations.

3. Video Segmentation Studio

The Video Segmentation Studio is the central control room for managing raw media sources, converting video into discrete frames, and preparing frame sequences. Extraction and preprocessing run asynchronously in the background, so large jobs don't block the interface.

Workspace Organization

Your video corpus is managed inside an interactive spreadsheet overview, providing clear indicators for thumbnail preview, sequence filename, and lifecycle actions:

Example: the video corpus table - a raw file awaiting processing, a processed sequence, and one of its preprocessed variations, each with its own set of actions

ThumbnailSequenceActions
warehouse_cam_04.mp4
Raw file - not yet processed
conveyor_belt_02
Processed - 1,840 frames
conveyor_belt_02 - Copy (blur variation)
Preprocessed variation - 1,840 frames

Video Actions and Workflow Icons

Each video entry displays a set of actions depending on whether it's a raw file, a processed sequence, or a preprocessed variation:

Icon Action Function & Backend Flow
Open Workspace Opens the frame-by-frame canvas editor for an already processed video directory.
Process Video Launches frame extraction for a raw video file. Videos longer than 1024 frames redirect automatically to the Video Splitting Studio to define target boundaries first.
Re-Extract / Refresh Clears the previous frames and re-runs extraction - useful when the source file changes or extraction parameters are adjusted.
Create Variation (Option B) Creates a lightweight copy of the entry, carrying over its annotations. If a copy already exists, you're prompted to Regenerate Variations instead, which resets and overwrites the existing folder rather than cluttering the directory with duplicates.
Apply Preprocessing Workflow Opens the variations setup interface to configure image preprocessing nodes that synthetically expand your dataset's variance.
Rename Safely renames directories, database entries, and dependent checkpoints across the filesystem.
Delete Permanently deletes extracted frames, caches, and database configurations.

Background Frame Tracking (SAM2)

To keep labeling pipelines interactive, long-sequence frame propagation using SAM2 can be offloaded to a background thread directly from the annotation editor:

  • Run in Background: Click the "Run in Background" button inside the tracking modal to close the editor and resume other workflow actions.
  • Live Progress Banner: A tracking banner appears at the top of the Video Segmentation Studio, showing a spinning gear, the sequence name, and a real-time progress bar with completion percentage.
  • Concurrent Operations: While background propagation runs, you can create variations, delete sequences, or perform metadata changes on other videos.

Dataset Preprocessing & Sequential Duplication

To expand your annotations synthetically, the Variations Panel offers single-click duplication and bulk propagation across the workspace:

  • Header-Level Controls: The "Generate" and "Back to Video Segmentation Studio" buttons sit in the header of the variations window, keeping the working area unobstructed.
  • Delete All Except Copy: Clicking "Delete All Variations" removes all augmented variations while keeping the primary - Copy directory intact on disk, letting you reset your augmentation pipeline without losing your annotations.
  • Sequential Duplication to ALL: The "Duplicate Variations to All" action copies the selected source video's configured variations sequentially across every other processed/labeled video in the active workspace.
  • Silent Fail-Over Guard: The duplication queue handles errors gracefully - unlabeled videos, empty mask states, or missing directories are skipped automatically, so the bulk queue completes without stopping on errors.
  • Dual-Progress Visualizer: A progress overlay appears during bulk duplication, showing the current task (e.g., Task 2 of 5) and completion percentage in real time.

Batch Processing & Automated Queue Management

To fast-track pipelines involving large video libraries, the Video Segmentation Studio integrates Batch Processing:

  • Wipe & Clean Rebuild: Toggle "Clean preprocessed data" to clear previously completed cookies and cache state, so the parallel processor reconstructs every target frame and augmented variation from scratch instead of trusting a stale cache.
  • Workload Parallelization: Set "Number of parallel batches" to control how many independent background Python processes are spawned, splitting the dataset across them to keep multiple processing pipelines busy.
  • Hardware Telemetry: The read-only "Number of devices (GPUs)" field shows how many graphics processing units are available on the host machine.
  • Chained Training Execution: Checking "Automatically start training after data expansion" chains the pipeline into an automated feedback loop, triggering model fine-tuning with the configured hyperparameters as soon as data expansion completes.
  • Graceful GUI Exit & Dedicated Terminal Spawn: Clicking "Do you want to EXIT and start Batch Preprocessing?" serializes the configuration, closes the local web server GUI, and spawns a dedicated, high-priority terminal window to run the processing workload. The browser window is then redirected to Photron's home page.
  • Distribute This Job: Check "Distribute this job" to hand the generated batch script to another machine on the network instead of exiting - INFINIWORKFLOW stays open and your local GPU is freed for other work. Checking the box looks up known peers (discovered the same way as Distributed Rendering's peer list); if none are found the checkbox disables itself with an explanation. Only real remote peers are offered - running the job on "this machine" wouldn't free anything, since it would still compete with INFINIWORKFLOW for the same GPU. The button label switches away from any "EXIT" wording the moment the box is checked, and no window closes - the job runs on the remote peer while you keep working.

Example: the Batch Preprocessing dialog, configured for a four-way parallel run with distribution enabled

Batch Preprocessing
Clean preprocessed data
Number of parallel batches
Number of devices (GPUs) 2 detected
Automatically start training after data expansion
Generates batch.py, then runs it on the picked peer instead - INFINIWORKFLOW stays open instead of exiting.

Video Studio Setup & Discord Integration

To customize frame handling and receive remote alerts during long batch runs, the Video Studio Settings modal provides the following options:

  • Interactive Setup Dialog: Accessible via the gear icon () in the Video Segmentation Studio control bar. Adjust settings on the fly and they're saved persistently.
  • Frame Deduplication Control: Set a threshold (range: 0 - 1000, default: 1) to filter out redundant, static, or identical frames during video processing, reducing annotation workload by eliminating duplicate image sequences.
  • Discord Notifications Integration: Toggle "Enable Discord Notifications" and supply a Discord Webhook URL. The backend monitors background batch preprocessing progress and training metrics, periodically posting plain-text status updates (and, for some events, the relevant log/output files) to your Discord channel via the webhook.
  • Export in Subfolder: When checked, fine-tuned models export to the project's root folder under the export name instead of assets/<export name> - useful when you want trained weights kept alongside the project rather than under the shared assets tree.
  • Persistent Cookie-State Retention: All settings (deduplication threshold, Discord toggles, webhook URL, and the export-subfolder choice) are saved in cookies, so your preferences persist across page reloads and future sessions.

4. Video Splitting Studio

The Video Splitting Studio is a dedicated interface for managing long or high-speed video sequences. Because annotating thousands of frames linearly is impractical, the Splitter lets you clip precise frame spans and compile them into shorter, manageable chunks.

Automated Redirection

You don't need to launch the Splitter manually. Inside the Video Segmentation Studio, clicking Process on any raw video file with more than 1024 frames automatically opens the full-screen Video Splitting Studio interface. (The Splitter's own default output chunk size, 240 frames, is a separate, independently adjustable setting - see Custom Chunking Size below.)

Features and Interactive Scrubber Controls

5. Fine Tuning

Fine Tuning provides a web-native interface to train, validate, and optimize state-of-the-art machine learning models directly on your custom-labeled datasets. Rather than exporting your data and writing boilerplate training scripts, you can start training in one click, monitor metrics in real time, and deploy the resulting weights immediately within your workspace.

1. Dataset Preparation & Rules

Before starting training, make sure your annotated dataset meets the system's requirements. Training is gated by the following validation checks:

2. Hyperparameter Configuration

Before clicking Train Now, the panel exposes the following fields (each is remembered between sessions, so you only need to set them once per project):

3. Initiating Training & Multi-Video Selection

For a model to generalize well, it should train on a variety of clips. The Fine Tuning Studio makes it easy to compile a multi-video dataset:

4. Batch Training & Standalone Execution

When you start a project-wide, multi-sequence training run in batch mode, the selection dialog turns into a standalone training setup panel:

Example: the Batch Training dialog - epoch staging, the standalone/distribute options, and the launch button

Batch Training
Epoch Stages (split epochs into stages)
Delete prior training checkpoint

5. Real-Time Training Monitoring

Once you trigger the run, the dashboard switches to a live telemetry and performance view:

Example: the live training monitor - progress, current metrics, and the streaming log

Epoch 84 / 150 ETA 6m 12s
train_loss 0.412 val_loss 0.487 mAP50 0.911 mAP50-95 0.734
Epoch 82/150: loss=0.431 val_loss=0.502 mAP50=0.903
Epoch 83/150: loss=0.418 val_loss=0.493 mAP50=0.908
Epoch 84/150: loss=0.412 val_loss=0.487 mAP50=0.911

6. Deployment and Canvas Integration

When training concludes, your optimized model is packaged and made available for immediate production:

AI Tracking & Mechanics Tips

  • Drift Prevention: Always verify your path after manually overriding a keyframe, as the system clears dependent "AUTO" segments to maintain sequence integrity.
  • Performance: Images are processed at 1024px using the FastSAM-s model for the best balance between speed and precision.
  • Dataset Integrity: All AI-assisted labels must be finalized or vectorized before the final training export, so the ML model trains on high-quality ground truth.

Offline & Custom Training

Need to run training on a dedicated GPU cluster or your own local machine? The platform provides full transparency and exportability:

  • Training Script Export: Once you start a fine-tuning job, open the Code output of the Fine Tuning node in the workflow.
  • Downloadable Assets: Download the complete training script and your finalized dataset package directly from the interface.
  • Run Anywhere: Run the exported script in any environment that supports the framework, so you can use your own hardware for heavy training workloads instead of being limited to this machine.