INFINIWORKFLOW is a node-based platform for building AI, ML, and Computer Vision pipelines, with real-time support for sensor acquisition and device control.
INFINIWORKFLOW runs entirely in your browser. Its main UI components are:
The HUB portal provides a comprehensive overview of the system's capabilities, hardware compatibility, and the latest software advancements. To open the information overlay, click the HUB icon located in the toolbox. The following functions are available at the HUB:
The App Menu is a floating toolbar docked to the top of the canvas, providing the following functionality:
Hover any icon above to see its name - the same tooltip behavior as the real toolbar. Gold icons open AI features, red icons are destructive (New Workflow/Exit), and the underlined icons on the left each expand into their own popout menu (File, Edit, Workflow Graph, Customize, Settings, Windows, Performance, Help) - listed in full below.
Hide and Show the Toolbox
Pauses all the selected playable nodes which is useful if you are tracking and want to position the tracker marker
The Toolbox is where you add new tools to your flowgraph as nodes.
The first tab shows every tool; the remaining tabs each show a subset, such as tools related to Computer Vision or ML. Hover over a tab icon to see a tooltip with its category name. Once you've selected a category tab, you can further narrow the list by typing keywords into the search field - useful for quickly finding a specific tool to insert into your workflow.
Tool tile colors indicate language: purple = C++, blue = Python, green = CUDA, orange = OpenCL, pink = OpenGL, gray = Widget. The highlighted tile shows a hover/selected state.
Hovering over a tool shows a tooltip describing it. You can insert a tool into the workflow using any of the following gestures:
Infiniworkflow has several hundred nodes available, which allows for many possibilities but also can be daunting for new users or can stand in the way of users who are only looking to build a specific application (if you are building a Data Science workflow, you probably don't need to see the many Color Correction nodes that are available). As such, Layouts have been added as a feature for controlling which categories and nodes show up in your Toolbox. They are straightforward to use, but by no means necessary to learn about if you do not wish to change how your Toolbox appears, so you may skip this section and not lose any critical information.
To begin changing how the categories / nodes appear within your Layout's Toolbox, click on the Layouts button and then click on the Layout you wish to change. Now that you're in, you may begin moving both categories and nodes as you like. The following operations are possible:
If you are unhappy with your Layout and want to start over, simply go to the Settings button and click on the "Reset Layout" button. Alternatively, you may delete the Layout all together by clicking on the "Delete Layout" button within Settings. NOTE: Make sure that you are IN the Layout that you want to reset / delete, or you may end up resetting / deleting the wrong Layout! Do this by clicking on the desired Layout after clicking the Layouts button before making any changes.
The flowgraph is where you construct your workflow, which comprises Nodes and Edges. Nodes represent functions that take input and generate outputs. You create nodes by dragging tools into your workflow from the Toolbox. A node's inputs and outputs have 'ports', which are where edges connect. An edge connects the output port of an upstream node to the input port of a downstream node. Any input port left unconnected can instead be set to a specific value using the Parameter Editor. A node's color indicates its execution language:
| C++ Nodes (can be executed on GPU or CPU) | |
| Python Nodes (can be executed on GPU or CPU) | |
| Cuda Kernels (always executed on the GPU) | |
| Widget nodes (executed on the CPU) |
The flowgraph has the following components:
Adding an edge connects the output of an upstream node to the input of a downstream node. In this example, we want to run Yolo Classification on the output of a Movie Reader, so we connect the Movie Reader node's output to the Yolo Classification node's input. Click the source port of the upstream node and drag to the destination port of the downstream node. A green line indicates the edge is allowed, based on the type of the two ports matching.
There are a few exceptions to allow different types to be connected to each other. For example, the image2D type, which represents a 2D image in system memory, can be connected to a type cuda2D, a 2D image in GPU memory and vice-versa. The exceptions are as follows:
| Output Type | Input Type |
|---|---|
| * | Any type |
| Any type | * |
| image2D | cuda2D |
| cuda2D | image2D |
| double, int, bool or numeric | double, int, bool or numeric |
| double, int, bool or numeric | double, int, bool or numeric |
| numeric2 | double2, int2 |
| double2, int2 | numeric2 |
| numeric3 | double3, int3 |
| double3, int3 | numeric3 |
| torch.nn.Module | torchvision.model |
| torchvision.model | torch.nn.Module |
Removing an edge disconnects the output of an upstream node from the input of a downstream node. In this example, we no longer want Yolo Classification run on the output of the Movie Reader. Hover over the edge - the cursor changes to show it can be deleted - then click the edge to delete it.
To delete a node can be done by clicking the delete key
To 'View & Edit' a node, double click the node, if there are multiple outputs a menu will allow you to select which output you wish to view
To View a specific output you can double click on the output port of the node
To zoom into the center of the flowgraph you can press the + hotkey
To zoom out of the flowgraph you can press the - hotkey
To zoom fit flowgraph, showing all nodes in viewport, you can press the f hotkey
Clicking the left mouse button over the node brings up the node context menu and also selects the node
Inspect and adjust
Node attribute
Performance
ML functions are available when ML nodes are selected
Experimental
Selection
When you bring up the context menu without a node selected, the flowgraphs viewport functions are shown:
The Parameter Editor allows you to edit the parameters of the currently edited node
The dialog is laid out as the tool icon and the name of the node being edited, followed by the list of that node's input parameters, and finally the dialog buttons. Clicking the node's name opens a webpage with the tool's full description. Only inputs that aren't already connected via the flowgraph show up as editable parameters here - a connected input's value comes from its upstream node instead.
Hovering over the parameter will show the description of the parameter:
The dialog buttons allow you to close the dialog - either you can accept the changes made by clicking OK, or reject any changes made to the parameters by clicking Cancel. A button also allows you to Reset All the parameters to the original default tool settings. The UI for each parameter input will be based on the type of the input, but all of them will have a reset icon that allows you to reset that particular parameter input back to its default value. The different types of parameter UI controls are as follows
Hover any icon above to see its name. The divider groups are: zoom/inspection, frame playback, wipe compare, color channels & adjustments, and the collapsible More Tools menu (Measurement, Keystone, Stabilize, Multi-View, plus the Path Editor tools).
The viewer allows you to view the outputs of the currently viewed node
The viewer's toolbar includes the following functions:
The viewer, on top of displaying images, has specific UI to display mutli-dimensional data:
Multi-View compares up to 4 image sources live, in one place - a wipe divider or side-by-side split for 2, a tiled 3-way or 4-way view for 3 or 4. Slot A always tracks whatever you're currently viewing in the main viewer; slots B, C and D are added and removed manually from the Multi-View dialog itself, so browsing to something else in the flowgraph never disturbs a comparison you've already built up. It never permanently adds anything to the flowgraph: opening Multi-View creates a temporary, hidden compare node behind the scenes, and closing Multi-View removes it immediately. Only image outputs can be compared this way (not data frames, tensors, or other non-image views).
Multi-View is only available from the main viewer - it isn't part of the AI Label Studio or Video Segmentation Studio viewing workflow. For comparing camera nodes specifically, see Live Cameras, which sets up a Multi-View comparison of every camera in the workflow in one click.
Live Cameras, in the floating Toolbox and also under the Windows menu, is a one-click way to see every camera in the current workflow at once. It automates the exact same steps as viewing a node and turning on Multi-View yourself - it doesn't add any special hidden mode of its own - so anything true of Multi-View elsewhere in this manual (up to 4 sources, temporary hidden compare node, image-only) is true here too.
Dataframes represent 2D tables and are implemented using the Pandas Python module. The viewer displays the DataFrame as a HTML table. Additional controls allow you to slice a set of the rows and columns, in the example below we slice rows [30-40). The icon allows different views of the table including showing the sliced rows and columns, with red cells represent missing data, a summary description of the statistics of each column, a line chart of the numerical columns and the description of the types of each column:
Numpy represents multidimensional numerical arrays and are implemented using the Numpy Python module. You can set a matrix using the Set Matrix tool. The viewer can display Numpy arrays in a variety of different visualizations where it will select the most useful visualization first and by clicking the allows you to visualize other representations of the arrays. Slicing controls are also available to reduce the tensor to a subset of its numerical data.
Tensors represent multi-dimensional numerical arrays and are implemented using the PyTorch Python module. The viewer can display tensors in a variety of different visualizations where it will select the most useful visualization first and by clicking the allows you to visualize other representations of the tensor. Slicing controls are also available to reduce the tensor to a subset of its numerical data.
Beyond displaying images and matrices, INFINIWORKFLOW has specific UI to display one-dimensional audio data:
The concept of the project resolution is meant to be used for spatial parameters (points and path parameters). These parameters will have defaults in percentages, for example a point parameter could have the deualt of (50%,50%), which corresponds to the center of the project resolution. After nodes are placed down the defaults are converted to pixel coordinates and the user will set pixel coordinates. However, you can change the project resolution and then also select to modify the spatial parameters to also be updated. This allows the user to work in different resolutions but have the workflow adjust to the new resolution if needed (this is also known as working in a "proxy" resolution, typically a lower resolution but then later swith to the full resoultion for final processing.)
The Path Editor allows you to draw, modify, and manage vector-based shapes and paths over the viewer. These tools are primarily used for measurements, defining regions of interest (ROI), or masking.
The Shape tool is used for creating both sharp-edged polygons and smooth, complex curves.
The Insert Freehand and Insert Lasso tools support an advanced Vector Brush / Stroke Mode. Instead of always generating solid, filled shapes, this mode allows you to draw strokes and outlines of a specific thickness following your hand path.
Managing multiple points is essential for complex adjustments.
Alt and Shift allows you to draw a selection marquee (box) around multiple points or click individual points to add them to your current selection group.When you have selected all points in a path (or a specific group of points), the editor enters Shape Mode.
Shift while resizing will maintain the aspect ratio.To quickly enter Shape Mode for a single object, you can often double-click a segment of the path to select all connected vertices instantly.
Ctrl + Z for Undo and Del for deleting selected points.INFINICAM and HAWK are high-speed streaming cameras capable of capturing and transferring 1.2-megapixel of image data to PC memory at 1,000fps via USB 3.1. Infiniworkflow, on top of all of its many other functionalities, is designed to be a platform for using INFINICAM(s) and HAWK(s) and saving their footage. There are certain differences between nodes related to INFINICAM/HAWK and most other nodes, so if you will be using an INFINICAM or HAWK, reading through this section is the fastest way to understand everything Infiniworkflow can do with your camera.
When an INFINICAM or HAWK is plugged in, a node called "Infinicam" will come up. This node allows you to view the camera feed, and also set the preroll and postroll frames (pre/postroll frames will be discussed later). If multiple cameras are connected, each one will show up as its own node (i.e. "Infinicam", "Infinicam #2", etc). Note that the camera may take a few seconds to open. Also note that this node only allows you to view the feed; saving is done separately.
Infiniworkflow has 2 ways of saving footage for each camera line — "Infinicam Save Movie" and "Infinicam Save Compressed" for INFINICAM cameras, and "Hawk Save Movie" and "Hawk Save Compressed" for HAWK cameras. These saving nodes will come up for each respective camera that is connected to your machine (in other words, if you have 2 INFINICAM cameras connected, "Infinicam Save Movie" and "Infinicam Save Compressed" save footage from the first, and "Infinicam Save Movie #2" and "Infinicam Save Compressed #2" save footage from the second — the same numbering convention applies to HAWK nodes). Note that these saving nodes do not need to be connected to the viewer node itself; all that is required is that the Trigger is clicked.
The "Save Movie" nodes ("Infinicam Save Movie" / "Hawk Save Movie"), upon hitting the Trigger, save footage from the selected camera in any file type (.MP4, .MDAT, etc.) and to any file location. The total number of frames that will be saved when the Trigger is clicked is based on your camera's pre-roll and post-roll number of frames. To explain what these terms mean, consider the following example: you wish to save footage whenever an object falls off a conveyor belt in a factory. You have a workflow that will set a Trigger to True as soon as it detects that an object has just begun to fall off the belt. To understand why objects sometimes fall off the belt, you want to save the 2000 frames of footage from before the moment the object begins falling, as well as 1000 frames of footage after that point for good measure. Thus, you will set your pre-roll to 2000, and your post-roll to 1000. When the Save Movie node is Triggered, a total of 3000 frames will be saved precisely as you want them to.
The "Save Movie" nodes tend to be slower, as they need to compress and decompress data on the fly. The "Save Compressed" nodes ("Infinicam Save Compressed" / "Hawk Save Compressed"), on the other hand, save out compressed images, which means footage gets saved to your computer faster and is more informationally dense (a single 2 second video can be a few hundred megabytes). Whereas the Save Movie nodes allow users to select the Codec and File format, the Save Compressed nodes hardcode both, so 2 files are always returned: a MDAT file of the footage itself and a CIH file of the footage metadata.
Note for all saving nodes: if the Trigger has already been pressed and you wish to stop saving (i.e. save a shorter clip of footage), you can simply click the Trigger again to immediately save out all frames already gathered to your machine.
If you wish to view footage saved from either "Save Compressed" node, use the "Infinicam Movie Reader" node, which reads MDAT/CIH files.
Important note: by default, when a "Save Compressed" node is Triggered, the number of frames saved will equal the camera's preroll plus postroll. However, if you wish to save footage continually, you can click the checkbox in the editing menu for "Constant Saving". When true, you may set the maximum file size for the saved footage. When the node is Triggered, footage will continue to save into the file you created until the maximum file size limit has been reached.
When 3 or more INFINICAM/HAWK devices are connected, additional nodes called "Infinicam Trio" and "Hawk Trio" become available in the catalog (both work identically - INFINICAM and HAWK devices are the same underlying camera pool in Infiniworkflow, so which one you see depends only on which brand name you drag onto the canvas). A Trio node drives three devices as one synchronized unit and packs them into a single 3-channel image - the Left camera becomes the Red channel, Middle becomes Green, and Right becomes Blue. All three cameras share one settings panel (fps, shutter, resolution, expose); there is no way to give the three cameras different acquisition settings, by design.
Which three physical cameras play the roles of Left/Middle/Right is chosen once, in a picker dialog, the moment you drag a Trio node onto the canvas - drag each detected camera tile into a Left/Middle/Right slot, then confirm. This assignment is then locked for the lifetime of that node: to point the node at different cameras, delete it and drag a new Trio node onto the canvas to pick again. This mirrors how a regular Infinicam/Hawk node's device cannot be changed after it is placed either.
Trio nodes require monochrome cameras - a device reporting a color sensor is refused with an error naming that device. The Left camera is automatically the synchronization (genlock) master; Middle and Right automatically slave to it. If the three cameras' frames ever drift out of alignment (a dropped frame, a disconnected cable), the node holds its last good composite and reports a sync-degraded status rather than silently showing a misaligned image.
Besides its main packed-RGB output, a Trio node has a second output called "strip", showing the three source frames laid out side by side (Left | Middle | Right) rather than merged into channels - useful for monitoring what each individual camera sees. This is only computed while something is actually connected to or viewing that output, so leaving it unused costs nothing.
Saving a Trio composite works the same way as a regular Infinicam/Hawk: connect (or simply enable) an "Infinicam Trio Save Movie" or "Hawk Trio Save Movie" node and click its Trigger. It saves the packed RGB composite as a normal color .avi/.mp4 (with the same preroll/postroll behavior described above for the single-camera Save Movie nodes), or as a single still .png image - there is no "Save Compressed"/MDAT equivalent for Trio, since a software-composited RGB frame has no camera-native compressed form to preserve.
If no INFINICAM/HAWK hardware is connected on a development machine, both the regular viewer nodes and the Trio nodes fall back to a simulated noise feed (clearly labeled "SIMULATED" on the image) so workflows remain buildable and testable without hardware attached; this simulated fallback never occurs on a production install.
Audio is represented by a list of numerical intensity values over time and they are implemented using the Numpy Python module. You can get audio using the Read Audio tool for audio files or the Input Audio tool for streaming input through the microphone. The viewers in Audio nodes can display audio arrays in two different ways: a waveform and a spectrogram. It will default to the waveform visualization first, which shows the loudness of the sound at every sample over time. By clicking the , it allows you to cycle through visualizations. The other visualization is a spectrogram, which is a colormap of frequencies over time where colors represent the volume of each frequency in decibels (dB).
A set of tools, called Widgets, are available that provide user interface controls directly in the flowgraph
These widgets are an easy way to modify the parameters without having to open the Parameter Editor - you can selectively decide which parameters are important enough to add as widgets to the flowgraph. For example, the following flowgraph has a number of widgets added: a "Filebrowser Widget", a "Selection List Widget" and a "Slider Widget" are added to the flowgraph as well as two "Output Widgets":
You can now modify those controls directly in the flowgraph. You can refine how each widget is shown by editing it in the Parameter Editor and setting its widget attributes. Widget attributes include the name which will show for each widget. Widgets such as Sliders allow you to set their specific attributes such as the minimum, maximum and step value for the Slider widget. All widgets have the common attributes of the name and description (used for tooltips) as well as layouts. The layouts allow you to specify an optional Tab that the widget will be placed in and also the order in which the control will be ordered in the UI (a lower order will allow the control at the top of the layout). An example of the Widget Slider's parameters are as follows:
See the reference section for the full list of Widget Tools
A set of tools, called Displays, are available that provide viewing displays directly in the flowgraph This allows you to constantly monitor the output of multiple nodes and avoid switching back and for using the Viewer.
See the reference section for the full list of Display Tools
You can create triggers to activate certain nodes that require the trigger to start execution. Typically, you can use the various boolean expression - for example, in the workflow below, the number of detected faces is applied to a "Numeric a>b" tool, this will yield a true value whenever the number of faces is greater than a certain amount. The output of this node is a "trigger" that is used to execute the "Text to Speech" node.
Loop Triggers is allows you to update a Trigger Variable based on when downstream Python nodes have executed and trigger an upstream Python node. This allows you to do "for" loops as the workflow graph is acyclic - meaning no edges can connect a downstream node to an upstream node, so loops are not allowed. However, with this feature you can make a trigger happen upstream when a downstream node is executed. You can add two new nodes, 'Loop Variable' and 'Loop Trigger':
The Loop Trigger when the source has changed (or you click the next trigegr), will use the referenced Loop Variable and will trigger the output of the Loop Variable. The Loop Variable can be placed upstream and flow back to the Loop Trigger, and thus this forms a loop cycle. You can use Loop triggers to perform simulations which may require multiple passes of the workflow nodes.
The Data Science tools are all under the Data Frame category. The implementation is based on Pandas, an open source data analysis and manipulation library. A DataFrame can be loaded with the "Read CSV" or "Read Excel" tools or created programmatically with the "Random Table" tool or converting from numpy or tensors. For many of the tools, they will use "Column" or "Columns" properties representing a choice of a single column or a subset of columns. Some of the tools also have an "arg" property which is a map parameter that allows you to pass in additional key/value pair optional arguments The Key/Value Dialog UI will show the corresponding Pandas function's documentation which is useful to determine the additional parameters you wish to set.
See the reference section for the full list of Data Science Tools
A number of tools are available to create charts for DataFrames. These tools are all under the Plot category. Each plot tool has parameters placed into two different tabs: Data and Layout. The Data parameters allow you to set the columns you wish to plot and the Layout parameters allow you to adjust the title of the chart etc.
An subset of the plot tool visualizations are as follows:
See the reference section for the full list of Plot Tools
The Machine Learning filters are based using scikit-learn Python module are all under the ML category. Each ML tool has 3 tabs: Train, Hyperparameters and Export. The Train parameters allow you to set the X and Y columns as well as a trigger to start the training. Since training can be slow, a trigger is used to start the process - though when running a grid search, that trigger is generated automatically.
The typical approach to building models involves splitting your training data into test and train splits. The following workflow illustrates the steps involved and the nodes required to implement the training:
The CSV file is read and then a test train split is done, the training table is then passed to the ML model. In this case the "Is Batch" Tool is used to set the "save" parameter which will automatically save the model for any Grid Search. The output of the model is then passed to a model predict and the predicted values can be compared against the ground truth to establish the accuracy of the model. In this scenario, we use a confusion matrix to plot the accuracy of the results. And a ML metric nodes such as "R2 Score" allow you to see the accuracy and it can be further used to initiate a Grid Search.
See the reference section for the full list of ML Tools
The AI Inference algorithms including using pretrained models tools are all under the AI category. The full list of display tools is as follows:
The Audio signal processing filters tools are all under the Audio category. The full list of display tools is as follows:
The Image Color Correction filters tools are all under the Color category. The full list of display tools is as follows:
The Combine and Split Images tools are all under the Composite category. The full list of display tools is as follows:
The Database processing such as reading tables, performing queries and writing to databases tools are all under the Database category. The full list of display tools is as follows:
The Datascience filters using the Pandas library tools are all under the Datascience category. The full list of display tools is as follows:
The Experimental Tools tools are all under the Experimental category. The full list of display tools is as follows:
The Image Processing Filters such as Blurs, Edge Detetors and Morphological Filters tools are all under the Image Processing category. The full list of display tools is as follows:
The Grayscale Filters that create single channel images tools are all under the Grayscale category. The full list of display tools is as follows:
The Source Inputs such as Webcameras or Reading Movies tools are all under the Inputs category. The full list of display tools is as follows:
The Logical functions for conditional processing, useful to create triggers tools are all under the Logic category. The full list of display tools is as follows:
The Mathematical and Scientific Operators tools are all under the Math category. The full list of display tools is as follows:
The Matrix operations using numpy arrays tools are all under the Matrix category. The full list of display tools is as follows:
The Machine Learning filters based using scikit-learn tools are all under the ML category. The full list of display tools is as follows:
The Display tools that allow you to visualize directly in the flowgraph tools are all under the Outputs category. The full list of display tools is as follows:
The Photron Camera and Processing Tools tools are all under the Photron category. The full list of display tools is as follows:
The Plotting tools using Matplotlib tools are all under the Plot category. The full list of display tools is as follows:
The PyTorch deep learning for building ML models tools are all under the Pytorch category. The full list of display tools is as follows:
The 3D tools to create 3D scenes and render tools are all under the Rendering category. The full list of display tools is as follows:
The Image Statistics tools such as mean, median and RMSE tools are all under the Statistics category. The full list of display tools is as follows:
The String tools such as string concatentation tools are all under the String category. The full list of display tools is as follows:
The Tracking features in images tools are all under the Tracking category. The full list of display tools is as follows:
The Image Transformation tools tools are all under the Transform category. The full list of display tools is as follows:
The Triggers tools which are useful to create signals for control tools are all under the Triggers category. You can create triggers to activate certain nodes that require a trigger to execute - review the section Creating Triggers. The full list of display tools is as follows:
The Python Utility tools tools are all under the Utilities category. The full list of display tools is as follows:
The Industry-specific tools built from the verticals proposal review (docs/verticals_todo_analysis.md): agriculture, drones, factory automation, infrastructure, logistics, maintenance, medical, quality inspection, R&D, robotics, safety management, security, smart city, sports analytics, startups, and teaching. Each node's tags list which vertical(s) it belongs to. tools are all under the Verticals category. The full list of display tools is as follows:
The User Interface Widgets which can be used for Publishing a simplified UI tools are all under the Widgets category. The full list of display tools is as follows:
The "System Performance" tool can be used to report performance metrics of your workflow. You can add it to your workflow and it will output a table of counters that will refresh regularly to show the performance for each node. The output table format is described below:
| Column | Description |
|---|---|
| name | Name of the node |
| Work(ms) | Average time in milliseconds to process one frame |
| #Wait | Number of times since last update the node waited because inputs were not ready or hadn't changed |
| #Render | Number of times since last update the node executed |
| Host to Host(MB) | Amount of System CPU Host memory copied in megabytes |
| Host to Device(MB) | Amount of System CPU Host memory uploaded to the GPU memory in megabytes |
| Device to Host(MB) | Amount of GPU memory to the System CPU Host memory in megabytes |
| Device to Device(MB) | Amount of GPU memory copied in megabytes |
| Peer to Peer(MB) | Amount of GPU memory in megabytes copied between different GPUs when multiple GPU are available on the system |
The GUI Designer compiles your workflow into a standalone application - a generated C++ or Python program with its own window, viewer, and parameter sidebar - instead of running inside INFINIWORKFLOW itself. It's opened from the same CodeGen entry point used to export OFX/PFV4 plugins and other generated-code targets. For the full walkthrough (Canvas, Views, Features, Style, and Build tabs, plus generating and running the output) see the dedicated CodeGen & GUI Designer guide.
Contours.json
AI_Welding.json
Color_Grade.json
Live_Camera.json
The Browser provides a sophisticated interface for managing, previewing, and selecting your workflows and assets.
The file browser manages files located in the ${assets}, ${demos}, and ${downloads} folders. Use the Root Folder Select dropdown at the top-left to toggle between these primary storage locations.
Breadcrumb Navigation:
The path bar below the header allows you to jump back to any parent directory by clicking on the folder names.
Example: $ / demos / Verticals
Custom Path Mapping:
You can define additional directory paths as prefixes using the -path argument when launching the application.
Example: ..\python.exe app.pyc -path "captures=C:\Users\imagi\CapturesFolder;media=C:\media"
Customize your browsing experience using the icons in the top-right toolbar:
When selecting a .json workflow, the browser provides an integrated preview window:
The system validates your hardware against the workflow requirements.
Note: If the requirement text turns red, your current system is missing the necessary GPU nodes (CUDA/PyTorch) required to run the selected workflow.
Hide or show the inputs and outputs for a selected node. This feature allows you to simplify the workflow to focus on the most relevant inputs and outputs. The Customize menu is also a concise way of seeing all descriptions for each input/output in one place. The following cutomizations are available in each node:
Grouping lets you take a set of connected nodes and combine them into a single compound group node. Grouping reduces the visual complexity of your flowgraph, and you can go further and export a group node as a fully-fledged tool of its own.
See also the section on Customize Nodes which in combination with Grouping allows you to create customized group nodes.
Profile is available from the toolbar's Performance menu. It measures each node's execution performance and briefly shows a colored FPS speed-dial badge on every active node (green/yellow/red for fast/medium/slow), fading out automatically after a few seconds. It has no effect on performance when not in use.
Synchronize is available from the toolbar's Performance menu when the selected node has 2 or more connected inputs (moved here from the node's right-click context menu, and renamed from "Balance Inputs"). Every node runs on its own free-running thread, so two branches feeding the same node can drift out of sync - one branch might be 3 nodes deep and slower to update than a branch feeding straight from the camera. This opens the Synchronize dialog:
cv.delay_balance node inline on every branch that's ahead of the slowest one,
each configured to hold its input back by the measured number of milliseconds - bringing all the branches back in sync with the slowest one.
The inserted cv.delay_balance nodes are ordinary nodes - you can select one afterward and hand-tune its delay_ms
parameter in the Parameter Editor if needed. It's a time-based sample-and-hold (holds one frame, only refreshing it once real wall-clock
time has actually elapsed), not a frame-count buffer, since a branch's own update count isn't a reliable stand-in for elapsed time. This
compensation only makes sense while the graph is genuinely free-running live/interactive, so it is automatically bypassed (acts as a
pass-through) during export and other batch/composite rendering, and during Stabilization frame tracking.
Merge Threads and Split Threads are available from the toolbar's Performance menu (moved here from the node's right-click context menu). By default each C++ node is executed in a different CPU thread for automatic pipeline acceleration, however, there may be circumstances where you want to finely control the order of stream processing:
You can set the hyperparameters using the node context menu and selecting 'Hyperparameters'. This brings up a dialog that allows you to select each input parameter and also set the range of values you want to have as part of the Grid Search. The dialog also includes the documentation for the model including the values expected for each hyperparameter argument.
Once you have created a ML model using the ML Toolsyou have refined your 'Hyperparameters' then you can start a Grid Search on a metric node you wish to maximize or minimize such as the "R2 Score" ML tool. Select the metric node and then bring up the context menu and select 'Grid Search':.
You can also click on the Import link to load the assets created during the Grid Search. Each ML model tool allows you to save the model to a file, by default models will not be saved but it is recommended whenever you have complex models that take time to execute. A common practice when doing a Grid Search is to connect the "Is Batch Tool" to the "save" input parameter of the model - this will always be true when a Grid Search is being done in the background batch process - thus, all the models will be saved during the Grid Search process. The import will then allow you to copy the model into your workflow folder:
When building models using the Torch nodes, the neural networks can get large with multiple nodes to generate the entire neural network. You can create a macro to create a tool that replaces all the nodes with a new tool which can be further used in the future and promotes sharing of models. To create a macro, select the node in your flowgraph that is a "Sequential" Torch Tool , then show the context menu for the node and select 'Create Macro'. The dialog allows you to name and set optional notes that will be associated with this new tool.
PyTorch is an open source machine learning framework that is excellent in performing Deep Learning.
You can find the properties of a tensor by using nodes such as Size, Dimension, Mean, Sum, Standard Deviation, and more. Further, you can combine two or more tensors together via either basic arithmetic (add, subtract, multiply, divide, etc.) or concatenation (concatenate, horizontal stack, vertical stack, etc.).
Additionally, you can convert tensors to and from DataFrames, NumpyArrays, and Images.
In PyTorch, a tensor can be one of many data types. In Infiniworkflow, all tensors are of data type torch.float32 by default (as this is the standard default within PyTorch as well). However, if you wish to change the data type of a tensor, simply drag in a Set Default Type node into the workflow and select one of 4 data types: torch.float32, torch.float64, torch.float16, and torch.bfloat16. This will change the data type of ALL tensors within the workflow. Note that this node doesn’t need to be connected to any other node to work; simply having it somewhere within the workflow is enough.
Neural Networks in Infiniworkflow can be Trained, Tested, and finally exported to a custom AI Inference node or exported to ONNX. The following sections will break down how to create a neural network, along with bringing in custom datasets and creating your own Inference Macros based on the neural nets you create.
The steps for creating a Neural Network, whether that be for Regression, Classification, or Segmentation, are more or less the same. The following section will describe in detail how to create a Neural Network for Regression, but most all steps can be copied for Classification or Segmentation. Exceptions and differences to note for creating Classification or Segmentation Neural Networks will be detailed at the end of this section.
To begin regression training (or any kind of training for that matter), we need 4 key inputs: a Neural Network Model, an Optimizer function, a Loss/Criterion Function, and the Data that the model will train on.
The Sequential node performs two actions behind the scenes. Firstly, it combines all machine learning modules that are provided as inputs (including nodes such as Linear, ReLU, Conv2D, MaxPooling2D, LogSoftmax, etc.) into a PyTorch Sequential container; to adjust the amount of input modules the Sequential node takes in, simply right-click on the Sequential node and click “Add Input” or “Remove Input”. Then, the Sequential node takes the Sequential Container and creates a neural network model out of it, with a base class of torch.nn.Module. The output of the Sequential node will thus be the “model” input of the Regression Train node.
Several optimizer functions are included in InfiniWorkflow. Most are intuitive (simply set the Neural Network Model as an input, set the Learning Rate and Weight Decay as needed, then set the output of the Optimizer node as an input to Regression Train), but the Per Parameter Optimizer is easy to misunderstand. The Per Parameter Optimizer node only works in tandem with another Optimizer node (such as Adam Optimizer), so make sure to connect the output of Per Parameter Optimizer as an input to the standard Optimizer node.
Using the Per Parameter Optimizer, specify the individual penalization weights you wish to set for specific parameter groups from your model; note that, if you wanted, you could set an individual penalization weight for each of your model’s parameter groups, but you would need to have a Per Parameter Optimizer node for each of these weights (additionally, you would need to Add Inputs to your standard Optimizer like an Adam Optimizer, and then feed each of your Per Parameter Optimizer nodes into your standard Optimizer). Any parameter groups that are not explicitly specified in any Per Parameter Optimizer nodes will take on the weights specified by the standard Optimizer node.
The output of the standard Optimizers is a torch.optim. Connect this as an input to the Regression Train node.
Several loss functions are included in InfiniWorkflow. Simply connect the one you would like to use as input to Regression Train.
In order to perform Regression, you need clean, numerical data. Assuming that your data is viable, set it as the input to the Train Test Split node. This will allow you to split data into Training data and Testing data. Set the Training data as an input to the Regression Train node.
Edit the Regression Train node and hit the Trigger button to initiate training. You can see the status of the training in real-time by hitting the [Render Status] icon in the application menu. If at any point you want to stop training, simply hit the Abort button within the Render Status Console. If you would like to save the output model once training is complete, click the “save state dict” box to enable saving, and specify where on your local machine you would like the output to be saved to.
With training complete, you can now begin testing your data, which you can do in 2 main ways. The first way is to have a Regression Test node in the same workflow as your Training, and connect the nodes appropriately. The second way is to use a Load Torch Model node, which you can only do if you saved the training output model to your local machine. Note that if you do want to use the Load Torch Model, you need to hit the Trigger in order to bring the data in from your local machine into Infiniworkflow. Furthermore, if you use this method you can have your training and testing in different workflows entirely. However, you would need to either recreate your model entirely (i.e. the Sequential node and all modules that feed into it in your Training workflow), or alternatively create a Macro on the Sequential node in the Training workflow such that the Macro can then be instantly brought into your Testing workflow (and any other workflow you want).
Your training and testing is now complete. The same steps can be repeated for performing Classification or Segmentation, with the biggest exception being the way that the datasets for Classification or Segmentation will appear in Infiniworkflow. An example from the CIFAR10 Dataset can be seen below. View and edit the node and set the “train” input to either Train, Test, or Validate (if Validate is an option).
Below is an example of a training workflow for a Convolutional Neural Network that performs Classification on the MNIST Dataset. Note the similarities between this and the Regression example seen above, with the principal exception being the number of layers that are fed into the Sequential node.
If you wish to create a Convolutional Neural Network (like the one depicted above) but do not want to immediately attempt creating the neural net from scratch, you can use the Convolutional Neural Net node instead to rapidly prototype your desired neural net.
The first three inputs relate to information on the input image data that this CNN will be trained on. The fourth input is how many classes the CNN will be trained to identify. All CNNs are composed of various convolution cycles followed by various fully connected layers. Since this node is meant for rapid prototyping, what is within each of these layers is already set. Each convolution layer is composed of a Convolution 2D, ReLU, and Max Pooling 2D node; each fully connected layer is composed of a Linear and ReLU node, apart from the last fully connected layer, which only has a Linear node. A Flatten node separates the convolution layers from the fully connected layers. Specifications for kernel size can be set in the convolution kernel size and pooling kernel size inputs. The final input is a boolean of whether the CNN ends with a LogSoftmax node at the end. Once again, this node is meant for primitive prototyping, and therefore is not fully robust; each fully connected layer only halves the number of filters until it gets to the desired number of classes the dataset identifies.
A few common datasets are already implemented in Infiniworkflow for classification and segmentation. These include CIFAR10, MNIST, FashionMNIST, and Cityscapes.
Bringing in custom datasets can be done in one of two ways. The first is via the Generic Dataset node; simply specify the naming convention of your inputs and outputs (X and Y), list all the classes, set the directory of where the dataset is coming from on your local machine, and set a colormap if one exists (for the purpose of segmentation).
The second (and probably more useful) approach is to create a plugin for your desired dataset. Refer to the Customizing Tools section on how to do so.
Once a model has been trained, users can then take their model and immediately begin using them within Infiniworkflow as a custom node for AI Inference. These nodes are called Inference Tools. (Alternatively, after a model is trained, the model and its weights can be exported to ONNX, a popular machine learning framework, using the Convert To Onnx node.)
To create an Inference Tool, simply right-click and select “Create Inference Tool” after your model has been trained. NOTE: The “Create Inference Tool” option will only appear under a Training Node (i.e. any node that is capable of training a model) after the model has been trained, not before.
Fill in the name of your Tool and any notes associated with it, and hit “Ok”. A prompt should inform you that “New tool has been added”, one which you can find in the toolbox alongside your other nodes. This node will now be able to perform AI Inference using the machine learning model you created and trained.
Pyrender is a Python library for physically-based rendering and visualization and useful for Synthetic Rendering.
There are three primary object types to know to render a Scene; these are Meshes, Lights, and Cameras.
A Mesh node is basically a wrapper of any number of primitive types. These primitive types represent the physical geometry that can be drawn to the screen. Infiniworkflow allows users to load meshes from existing Trimesh objects. In the assets folder, ensure all necessary files (including the object file, material file, and UV file) are included in order for the mesh to appear correctly when brought into a Scene, as seen below.
The output of a Mesh node is a 3DNode (in Pyrender, “Node” is the name of one of the most commonly-used classes when creating a Scene; in order to avoid confusion between Pyrender Nodes and Infiniworkflow’s Nodes, we have elected to denote Pyrender Nodes as a “3DNode”).
In addition to Meshes that come from existing Trimesh objects, you can also create your own basic 3D objects from scratch using the Trimesh Creator nodes. These basic objects include boxes, capsules, cones, icospheres, and toruses. The output of each of these Trimesh Creator nodes (such as “Trimesh Box” or “Trimesh Capsule”) is a 3DNode.
Pyrender supports 3 types of Light: PointLight, SpotLight, and DirectionalLight. The output of any of these 3 Light nodes is a 3DNode.
Pyrender supports 3 Camera types: PerspectiveCamera, IntrinsicsCamera, and OrthogonalCamera. The output of any of these 3 Camera nodes is a Camera (NOT a 3D Node).
To begin creating a Scene, bring in a Render Scene node from the Pyrender toolbox (marked with a 3D icon). The camera that you choose to view your scene with is the first input to the Render Scene node. The final input is for any 3DNodes you want to be present in your scene (i.e. Lights, Meshes, etc.); the Render Scene node allows users to add as many 3DNode inputs as they wish. The output of the Render Scene node is a Color (or Default) viewer, Depth viewer, and Segmentation viewer. Each of these viewers will be explained in further detail below.
If you have been following these steps so far, it is likely that your scene does not show anything. This is because you need to position your Camera and your 3DNodes where you want them. To do this, use a Transformation Matrix node or a LookAt Matrix (generally Transformation Matrices are used for nodes that output a 3DNode and LookAt Matrices are used for nodes that output a Camera, but any of these matrix nodes could be used in practice). Your final workflow might look something like this:
The following is a description of each of the three output views from the Render Scene node:
YOLOX is a version of the computer vision object detection model YOLO (You Only Look Once) that is better for fine-tuning.
Training a pretrained YOLOX model on custom data requires a dataset as well as some hyperparameters. YOLOX Train Custom Data
The image dataset must be in COCO or VOC format and labeled using Labelme or CVAT. The YOLOX Train node takes in the COCO/VOC dataset directory, train annotation/labels JSON, and validation annotation/labels JSON.
The hyperparameters include image size (416x416 for YOLOX light models, 640x640 for standard YOLOX models), the number of unique classes in the dataset, and the filepath to the checkpoint or pretrained PyTorch (.pth) model.
The experiment (exp) file contains all the other hyperparameters that can be adjusted, with args exposing a select few for ease of use.
The name and output directory determine the name of the fine-tuned model (.onnx) and the output directory where the log data is written to, as seen below:
The YOLOX Train node outputs a model that is trained on the input dataset, so it can classify things outside of the 80 COCO classes that it is originally trained on, like manholes.
Tensorboard is a web-based visualization tool for tracking machine learning training and validation errors.
It simply takes in the output directory of an ML training process and a localhost port to be hosted on.
If port is 6006, then triggering the node and opening localhost:6006 will show graphs like these updating live as the model trains:
ONNXRuntime is a high-performance engine tool for ONNX models.
INFINIWORKFLOW features an general-purpose ONNXRuntime node as well as a YoloxOnnxRuntime node.
Fortunately, the YOLOX Train node outputs an .onnx model, so the YoloxOnnxRuntime node can be used.
The relevant inputs for running inference on YOLOX are the model input, the list of class labels if they are not the standard COCO classes, and the input size (416x416 for YOLOX light models, 640x640 for standard YOLOX models).
The YoloxOnnxRuntime node outputs a preview of the detected objects as well as an output matrix.
Network Manager lets you move part of your workflow onto a different process on the same machine, or a different machine on the network entirely - useful for spreading heavy processing across more hardware, or offloading GPU-bound nodes onto a dedicated machine. It is opened from the toolbox's Performance menu's Network Manager button.
The dialog opens on a schematic view of your workflow (any nodes already selected on the live canvas are seeded into the
selection). Click a node, or drag a box to select several, to build the set of nodes to send - the selection must form a
single, contiguous region of the graph (one boundary edge in, one boundary edge out). A Profile button on this
screen runs a short live FPS pass over the schematic so you can see which nodes are slow before deciding what to send,
without committing to anything yet. The sidebar lists known peers (discovered automatically via UDP beacon from other
INFINIWORKFLOW instances started with -distributed-listen) to pick as the send target, plus the CUDA Device
on the target that will run the selected nodes.
Once a Profile pass has produced real timing data for the open workflow, a Suggest button appears alongside the schematic. It opens a small ranked list of candidate selections, each pre-checked against every rule the dialog itself would enforce - so anything offered here is guaranteed sendable. Candidates form a ladder from a single node up to a handful of nodes: the first entry is just the single slowest node in the workflow, the next adds whichever directly-adjacent node is next slowest, and so on - always one contiguous region, never a scattered pick. Each row shows a small preview of its nodes, the estimated time saved, and the estimated data cut across its new boundary. Already-distributed (bridge) nodes and display/viewer nodes are never offered, since neither can usefully be sent to a peer. Click Use this selection on any row to close the list and carry that candidate straight into the normal node selection, ready to pick a peer and device as usual.
Every image output the selection exposes at its boundary gets a real port on the local Distributed Source node - including one that isn't wired to anything yet, so there's always somewhere to connect something new later without having to undistribute first. A CUDA-accelerated node's output is exposed locally as a plain image type rather than a GPU buffer either way, since the actual pixels always cross the network as ordinary data - GPU memory itself is never shared between machines (or processes). Double-clicking the Distributed Source node shows its current output live, the same as any other image-producing node.
Clicking Next builds a manifest of the files referenced by the selected nodes' file inputs (models, LUTs, media, etc). For a remote target, each entry defaults to checked unless the target already appears to have a matching copy (shown with a "target already has this" note and left unchecked) - uncheck any others you don't want copied, then click Copy to actually transfer them. For a local target (this machine, whether in-thread or a separate process), the same list is shown as read-only "Referenced Files" instead - nothing is actually copied, since a local target already reads every file from the exact same filesystem the checklist itself was built from; just click Next to continue. Nodes that may not relocate safely are flagged here as warnings either way. If nothing is referenced at all, this step is skipped automatically.
Clicking Copy transfers the checked assets to the target with a live progress bar (cancellable). Once the copy finishes - or immediately, if nothing needed to be copied - the dialog reaches a Ready step; nothing has been sent to the graph yet. Only when you click Proceed do the selected nodes actually get replaced with a pair of bridge nodes, Distributed Sink and Distributed Source, and the job starts: the Sink sends data out from your system and the Source receives the processed data back in. This lets you review the asset checklist and back out before anything changes your workflow.
Multiple distributed jobs can run at the same time. A local target (this machine, used to test distribution without a real second machine) allows any number of concurrent jobs; each distinct remote peer is capped at one job at a time, but different peers can each run their own job concurrently. If a job's selection borders a node that's already part of another active job's boundary, the connection is simply redirected to that other job's live bridge port when needed - nothing is rejected or dropped.
While one or more jobs are active, reopening the dialog opens on a job picker: a tile per job with its target and status, plus per-job Resume/Delete buttons and Resume All/Delete All shortcuts. Picking a tile shows that job's live status table (refreshed every couple of seconds) with each bridge's role, target, health, frame count and what data it's carrying. Health is derived from a per-tick heartbeat: Connected while frames have arrived recently, Stalled if too much time has passed since the last one, or Waiting.../Not connected before a connection is established. Click Delete (Undistribute) to tear a job down and restore its original nodes.
A distributed job's bridge nodes are saved as part of the workflow, so the job survives closing and reopening the app (or loading a saved workflow) - but the underlying connection itself is not, so it needs to be told to reconnect. A group node whose job needs resuming shows a pulsing highlight on the flowgraph so it's easy to spot; opening the Distributed Rendering dialog shows a banner with a Resume button per job (or Resume All for every job at once) to reconnect it.
Render to Clip captures one node's output(s) straight to disk - video, or a CSV for a numeric/matrix value - useful for "freezing" an expensive or slow part of a workflow (a detector, an optical-flow pass, anything GPU-heavy) into plain files you can play back or read back instantly instead of recomputing every time. It's opened from the same Network Manager dialog, as one of two mutually-exclusive modes on the node-selection step: Stream (the dialog's normal continuous distribute/streaming behavior) or Render (this feature). Only one mode is ever active, and which ones are even selectable depends on the current selection - Render requires selecting exactly one node, so it's disabled the moment more than one is selected; Stream stays available for both single- and multi-node selections (whichever ones are valid for it in the first place - the same rules as always).
Unlike a normal distribute selection, Render always takes exactly one node - the one whose output(s) you want
recorded. Everything feeding it is discovered automatically by walking the graph backwards from that node, so there's
no need to also select its sources by hand. Once a single node is selected and Render is chosen, a picker appears
listing every capturable output on that node - image/CUDA outputs record to a video file (with a Format/Codec choice
per output), numeric (double/int) outputs record to a single-column CSV, and matrix2D outputs record to a CSV padded
out to the largest matrix seen across the recording. Only the first image/CUDA output is checked by default (or the
first output of any kind, if the node has no image output at all) - check or uncheck any others you also want
captured, and edit each one's filename if you don't want the auto-generated one. A Save to field (with a folder
browse button) lets you pick a destination other than the default clip_renders subfolder under your
project's assets. Set a start/end frame range, then click Next to proceed - exactly like Stream, Render still
goes through peer selection and (for a remote target) asset copying before the export actually starts.
A local Render always runs in a separate INFINIWORKFLOW process, never sharing this session's own live graph or threads - there's no "run in this session" choice the way a plain distribute job has, since interrupting your own interactive session to record a clip would defeat the point. Remote targets work the same way they do for Stream: pick a peer, and the export runs there instead.
Progress is shown live as the export proceeds, all the way up to 100% - not just a spinner - and the job appears as its own tile in the same picker as ordinary distribute and script jobs, with the usual Cancel option while it's running.
Once the export finishes, a dialog asks what to do with the recorded outputs - nothing is applied to the graph automatically. Click a card to see what it does, then Confirm:
Either way, a reader node added by Replace or Keep is named after its own output file rather than the generic tool name, so several added at once (one per captured output) are still easy to tell apart on the canvas.
If you close the dialog without choosing, the job stays available - open its tile again from the Distributed Rendering picker to make the same choice later; the exported files are on disk either way, regardless of what you eventually pick.
Remote Script Execution is a separate, lighter entry point from Network Manager above - opened from its own Remote Script Execution button in the Windows menu - for simply running a Python script on this machine or a peer, without touching your workflow's nodes at all.
Opening it browses straight to picking a .py file, or a folder - if you pick a folder, it runs
batch.py inside it (the fixed name Video Studio's export uses). Choose a target machine from the known
peers, then click Run. The dialog switches to a status view listing script jobs with their state (running,
finished, failed) and buttons to View, Cancel a running job, or Dismiss a finished one, plus a live
console pane streaming that job's output. Only jobs started from this machine are tracked and listed here - a script sent
to run on a remote peer runs there untracked by this dialog.
The same peer-dispatch mechanism also powers the "Distribute this job" checkbox in the AI Studios' Batch Preprocess and Batch Train dialogs - see the AI Studios manual - which sends a generated batch script to a remote peer instead of exiting INFINIWORKFLOW to run it locally.
ROS is a set of libraries used to communicate with robotic devices including robotic cars and arms.
Everything in a ROS system is a node, communicating with one another through topics, services, and actions.
Every ROS node in INFINIWORKFLOW corresponds to a ROS node in a ROS system.
Technically, INFINIWORKFLOW only supports ROS2, but they are referred to interchangeably in this manual.
ROS Publishers and Subscribers communicate by streaming data to and from a ROS topic.
The ROS2 Publisher node takes in a ROS topic name and a string message.
On trigger, it broadcasts the string message to the specified ROS topic once a second.
The ROS2 Subscriber node takes in a ROS topic name as input.
On trigger, it outputs received messages from the specified ROS topic.
Devices that support ROS manifest themselves as ROS servers, which typically contain both service servers and action servers.
Instead of getting continual updates like publishers and subscribers, services only provide data or take effect when requested to by a client.
Like services, actions are only executed when called by a client. Unlike services, actions typically involve sending a goal and the action server can provide feedback on its progress towards that goal.
The ROS2 Server node takes in a ROS package name and executable file.
On trigger, this launches a ROS server that includes service and action servers.
By default, the ROS2 Server node launches the turtlesim_node of the turtlesim package, but this can be robotic arms, cars, or any other ROS node.
The ROS2 Service Client node takes in a ROS service name, the input type of the service, and args representing the input.
On trigger, this queries the specified ROS service with the args input for some information or effect.
By default, the node calls the /spawn service of the turtlesim_node, which spawns a turtle at its default position in the bottom left corner.
The ROS2 Action Client node takes in a ROS action name, the input type of the action, args representing the input, and a feedback bool.
On trigger, this uses the args input to set an objective for the specified ROS action and outputs its progression to that goal if feedback is true.
By default, the node calls the /turtle1/rotate_absolute action of the turtlesim_node, which rotates turtle1 to a specified angle; in this case, {theta: 1.57}.
The service can be any supported service on any ROS node, so the ROS2 Service and Action Client nodes can interact with devices like robotic arms and cars.
To have a prompt console where you can actually "ask" the AI to build or update your workflow, you need a Client that connects to the SSE (Server-Sent Events) port we just opened in your app (Port 8000). AnythingLLM is the easiest desktop console for MCP.
See the MCP / AI Agent page for live server status, how to start the server, copy-button setup snippets for Claude Code / Claude Desktop / AnythingLLM, the full tools reference, and a large set of example prompts. The app's Windows menu also has a quick "MCP AI Agent" launcher that opens a terminal running your agent CLI directly.
Security note: the server listens on all network interfaces with no authentication, so anything on your network that can reach port 8000 can read and modify your workflow while it's running. Only use -mcp on a trusted machine/network.
Chat with an AI coding agent wired to InfiniWorkflow's own MCP server, so it can edit the live workflow graph, generate and compile C++ plugins, drive AI Studio, inspect state, and control playback, all from natural-language requests. Both Claude (Anthropic, via the claude/Claude Code CLI) and ChatGPT (OpenAI, via the codex CLI) are supported. For the full walkthrough (prerequisites, opening the chatbox, conversation history, and what happens when you send a message) see the dedicated Generative AI guide.
INFINIWORKFLOW runs on a modern PC with Windows 11 or higher, or macOS 12.6.2 or higher. It requires an Intel or AMD processor, at least 12GB of free disk space, and ideally an NVIDIA GPU with sufficient VRAM for your models. A multicore processor is highly recommended, since execution runs noticeably smoother with one.
On Windows, the software also runs on machines without an NVIDIA GPU, but that significantly reduces performance, especially for ML workflows. A package with no dependencies on CUDA or PyTorch is also available to download - it doesn't require an NVIDIA GPU to be present on your system and is substantially smaller in size, but it doesn't let you build deep learning models and is slower for AI inference.
You must also have a recent Google Chrome browser installed - version 131.0.6778.140 or higher.
The following are the full set of downloads packages:
| Operating System | CUDA Installation | Non-CUDA Installation |
|---|---|---|
| Windows | infiniworkflow | infiniworkflow_noncuda |
| MacOSX x86_64 | Not applicable | infiniworkflow_osx_x86_64 |
| MacOSX arm64 | Not applicable | infiniworkflow_osx_arm64 |
| Linux x86_64 (Ubuntu24.04 LTS) | infiniworkflow_linux | Not available |
| Nvidia Jetson | infiniworkflow_jetson | Not available |
| AWS Cloud | Contact Photron | Not available |
| Operating System | Link |
|---|---|
| INFINIWORKFLOW PATCH - Windows | infiniworkflow_patch |
| INFINIWORKFLOW PATCH - MacOSX x86_64 | infiniworkflow_osx_patch |
| INFINIWORKFLOW PATCH - MacOSX arm64 | infiniworkflow_osx_patch |
| INFINIWORKFLOW PATCH - Linux | infiniworkflow_linux_patch |
| INFINIWORKFLOW PATCH - Jetson | infiniworkflow_jetson_patch |
install.bat in the unzipped folder and grant Administrative privileges.
INFINIWORKFLOW.bat.First, verify your system architecture:
python3 -c "import platform; print(platform.processor())"
i386: Install the x86_64 version of INFINIWORKFLOW.arm: Install the arm64 version of INFINIWORKFLOW.sudo sh install.sh and enter your password.sh INFINIWORKFLOW.sh to start.sudo sh install.sh in the unzipped folder and provide your password.sh INFINIWORKFLOW.sh to start../install.sh in the unzipped folder and provide your password../INFINIWORKFLOW.sh to start.A webpage displayed in the Google Chrome browser should appear - if another browser shows up then change your default browser to Chrome and redo this step. The first thing that will be displayed in the browser is the INFINIWORKFLOW EULA which you must agree to. You will also see a Windows dialog that requests "Do you want to allow public and private networks to access the app?" for Python - you must allow access.
When you install INFINIWORKFLOW and run it the first time, you may see a Windows dialog that requests "Do you want to allow public and private networks to access the app?" for Python - you must allow access. If this dialog does not pop up and INFINWORKFLOW does not show images in the viewer then you have to manually grant access to allow INFINIWORKFLOW's Python installation to have access to public and private networks as follows:
If your Firewall is controlled by your anti-virus software then you will need to allow access of INFINIWORKFLOW's python.exe using the anti-virus software.
For an unattended Network Manager peer or any other headless render server, you generally want
INFINIWORKFLOW to come back on its own after a crash, an uncaught exception, or someone closing the console window - not sit down until
an admin notices. scripts/iw_watchdog.py is a small, cross-platform supervisor for exactly that: it starts INFINIWORKFLOW,
waits for it to exit, and restarts it, with a crash-loop guard (a rolling restart-rate limit plus a minimum-uptime check) so a broken
config can't tight-loop the machine instead of just backing off and trying again. It never gives up permanently - "keep restarting it" is
the whole point.
The watchdog does not itself register as a systemd unit, a launchd job, or a Windows service - it is a plain long-running script. Each platform's own service manager is used only to keep the watchdog alive at boot (it rarely crashes; it is a small restart loop), while the watchdog is what actually supervises INFINIWORKFLOW and restarts it, with the same behavior on every OS instead of three different native "restart on failure" mechanisms.
Run it manually to check it works before installing it as a service:
python iw_watchdog.py --dir "C:\InfiniWorkflow" --extra-args "-nobrowser -port 5000"
--dir is the INFINIWORKFLOW install root (the folder containing app/, and on Windows the bundled
python.exe) - it defaults to the current directory, so pass it explicitly for a service that won't start with a known working
directory. --extra-args forwards its value to app.py as a single shell-quoted string (-nobrowser is
added automatically if not already present). Restart-loop tuning defaults to 5 restarts per 300-second window before a 60-second backoff,
and any exit faster than 10 seconds is always treated as a crash-loop tick regardless of the rolling window; all of these, plus
--python (override the interpreter) and --log (defaults to <dir>/logs/iw_watchdog.log, shared
by both the watchdog's own restart events and INFINIWORKFLOW's stdout/stderr), are documented in python iw_watchdog.py --help.
For the Network Manager peer case specifically, --extra-args is where the
-distributed-listen/-distributed-token flags actually go, alongside whatever other startup flags you'd normally
pass to app.py:
python iw_watchdog.py --dir "C:\InfiniWorkflow" --extra-args "-nobrowser -distributed-listen -distributed-token my-shared-secret"
See Command line arguments for the full reference - -distributed-listen opts this instance in to
accepting jobs on the control channel (port 5099), and -distributed-token is the shared secret both the orchestrator and this
listening instance must agree on; without it, this instance refuses all incoming jobs regardless of -distributed-listen. Any
other flag documented there (-resolution, -port, etc.) can be added to the same --extra-args string.
On a clean stop (Ctrl+C, or a service manager's own stop signal) the watchdog exits without restarting. On POSIX, that stop signal is
forwarded to INFINIWORKFLOW as SIGTERM - exactly what app.py's own signal handler is registered for, so a clean stop also runs
the same distributed-renderer cleanup a normal exit does. On Windows, TerminateProcess cannot deliver a clean signal to
another process, so a stop there is abrupt by necessity; INFINIWORKFLOW's own startup sweep (sweep_stale_local_renderers())
cleans up anything that abrupt stop left behind the next time it starts.
Create /etc/systemd/system/infiniworkflow-watchdog.service:
[Unit] Description=INFINIWORKFLOW auto-restart watchdog After=network.target [Service] Type=simple ExecStart=/usr/bin/python3 /opt/infiniworkflow/scripts/iw_watchdog.py --dir /opt/infiniworkflow Restart=on-failure RestartSec=5 User=infiniworkflow [Install] WantedBy=multi-user.target
Then enable and start it:
sudo systemctl daemon-reload sudo systemctl enable --now infiniworkflow-watchdog
Create ~/Library/LaunchAgents/com.photron.infiniworkflow-watchdog.plist:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.photron.infiniworkflow-watchdog</string>
<key>ProgramArguments</key>
<array>
<string>/usr/bin/python3</string>
<string>/opt/infiniworkflow/scripts/iw_watchdog.py</string>
<string>--dir</string>
<string>/opt/infiniworkflow</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
</dict>
</plist>
Then load it:
launchctl load ~/Library/LaunchAgents/com.photron.infiniworkflow-watchdog.plist
Windows has no built-in way to run a plain script as a service, so use NSSM (the Non-Sucking Service Manager) as a thin wrapper:
nssm install InfiniWorkflowWatchdog "C:\InfiniWorkflow\python.exe" "C:\InfiniWorkflow\scripts\iw_watchdog.py --dir C:\InfiniWorkflow" nssm set InfiniWorkflowWatchdog AppDirectory "C:\InfiniWorkflow" nssm set InfiniWorkflowWatchdog Start SERVICE_AUTO_START nssm start InfiniWorkflowWatchdog
A Scheduled Task ("Run whether user is logged on or not", triggered "At startup") is a lighter-weight alternative to NSSM if you'd rather not install a third-party tool - it won't restart the watchdog itself if it dies, but the watchdog process itself is small and stable enough that this is rarely a practical concern.
| Name | OS | Description | Link |
|---|---|---|---|
| INFINIWORKFLOW SDK | SDK to allow you to write your own Python and C++ Plugins for INFINIWORKFLOW | infiniworkflow_sdk | |
| Code Generation Python | Codegen package for Python GUI | codegenpy | |
| OpenCV Barcode Detection Inference | WeChat QRCode including CNN models for `wechat_qrcode` module, including the detector model and the super scale model | barcode | |
| Labeling and Finetuning | Finetuning and Label | labeling_finetuning | |
| Medical | Blood Cell Detection | medical | |
| Aviation | Plane Detection | aviation | |
| Road Infrastructure | Road Crack Detection and Segmentation | road_infrastructure | |
| Warehouse | Warehouse Detection | warehouse | |
| Satellite | Satellite Imagery Building Detection | satellite | |
| Emergency | Fire and Smoke Detection for Emergency Response | emergency | |
| Cityscapes Segmentation Training and Testing | Semantic Understanding of Urban Street Scenes | cityscapes | |
| Colorization Inference | Colorful Image Colorization | colorization | |
| Tracking Inference, DaSiamRPN | Formulates the task of visual tracking as a task of localization and identification simultaneously using DaSiamRPN algorithm | dasiamrpn | |
| Tracking Inference, Nano | Formulates the task of visual tracking as a task of localization and identification simultaneously using Nano Tracker algorithm | nano | |
| Holistic Edge Inference | Code for holistic edge detection using pretrained hed model(caffe) | edge | |
| DexiNed Edge Inference | Code for edge detection using a model(ONNX) using a Convolutional Neural Network (CNN) | dexined | |
| Nano Font | Noto Sans JP — Japanese font from the Noto font family | nano_font | |
| Face Detect Inference using Haarcascades | Face Detect Inference using Haarcascades using OpenCV | haarcascades | |
| Human Face Segmentation | Human Face Segmentation | human | |
| Mask Segmentation Inference | Mask Segmentation | mask_rccn | |
| Person Reidentification Inference | Person REID Inference | personReiD | |
| MiDaS Depth Inference | MiDaS computes relative inverse depth from a single image | midas | |
| Hand and Body Pose Inference | OpenCV Hand and Body Pose Inference | pose | |
| Segmentation Inference | A Deep Neural Network Architecture for Real-Time Semantic Segmentation | segmentation | |
| Human Segmentation Inference | A Deep Neural Network Architecture for Real-Time Segmentation on Humans Specifically | human_seg_pp | |
| OpenCV Text Spotting Detection Inference | An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition | text_spotting | |
| Text Detection PP-OCRv3 Inference | Text Detection using PP-OCRv3 Model. | text_detect_ppocr | |
| YuNET Face Tracking and Facial Expressions Recognition Inference | A Light-weight, Fast, and Accurate face Detection Model, with Ability to Track Faces and Points on a Face and Perform Facial Expression Recognition | yunet | |
| UTKFace Dataset | UTKFace dataset is a large-scale face dataset with long age span | utkface | |
| YOLO4 Darknet Object Detection Inference | A computer vision model that uses YOLO4 deep learning using Darknet to detect objects in images and videos | yolodarknet4 | |
| YOLO4 Tiny Darknet Object Detection Inference | A computer vision model that uses YOLO4 Tiny deep learning using Darknet to detect objects in images and videos | yolodarknet4tiny | |
| YOLO7 Darknet Object Detection Inference | A computer vision model that uses YOLO7 deep learning using Darknet to detect objects in images and videos | yolodarknet7 | |
| YOLO7 Tiny Darknet Object Detection Inference | A computer vision model that uses YOLO7 Tiny deep learning using Darknet to detect objects in images and videos | yolodarknet7tiny | |
| YOLO NAS Object Detection Inference | A computer vision model that uses YOLO7 deep learning to detect objects in images and videos | yolonas | |
| Custom YOLO3 using Darknet |
A computer vision model using YOLO3 using Darknet that allows you to customize and train as well as do inference
on the trained models
Photron does not distribute YOLO3 as part of INFINIWORKFLOW, if you wish to use YOLO3, you must download it separately and agree to the license terms on your usage: After installing 1. Create a new directory in the assets folder: INFINIWORKFLOW_PATH/assets/yolov3 1. Download https://github.com/patrick013/Object-Detection---Yolov3/blob/master/model/yolov3.weights 2. Unzip file and copy yolov3.weights to: INFINIWORKFLOW_PATH/assets/yolov3/yolov3.weights 3. Download https://github.com/pjreddie/darknet/blob/master/cfg/yolov3.cfg 4. Copy yolov3.cfg to: INFINIWORKFLOW_PATH/assets/yolov3/yolov3.cfg |
custom_yolo3 | |
| YOLO5 Ultralytics Object Detection Inference |
A computer vision model that uses YOLO5 deep learning to detect objects in images and videos
Photron does not distribute YOLO5 as part of INFINIWORKFLOW, if you wish to use YOLO5, you must download it separately and agree to the license terms on your usage: LICENSE Additional steps: After installing the patch 1. Create a new directory in the assets folder: INFINIWORKFLOW_PATH/assets/yolo5 1. Download https://github.com/RsGoksel/Cpp-Object-Detection-Yolov5-OpenCV/releases/download/ONNX/yolov5s.onnx 2. Copy yolov5s.onnx to: INFINIWORKFLOW_PATH/assets/yolo5/yolov5s.onnx 3. Download https://github.com/RsGoksel/Cpp-Object-Detection-Yolov5-OpenCV/blob/main/Yolov5_Image_Object_Detection/Models/classes.txt 4. Copy classes.txt to: INFINIWORKFLOW_PATH/assets/yolo5/classes.txt |
yolo | |
| YOLO7 Ultralytics Object Detection Inference |
A computer vision model that uses YOLO7 deep learning to detect objects in images and videos
Additional steps: After installing the patch 1. Create a new directory in the assets folder: INFINIWORKFLOW_PATH/assets/yolo7 1. Download https://dl.opencv.org/models/yolov7/yolov7.onnx 2. Copy yolov7.onnx to: INFINIWORKFLOW_PATH/assets/yolo7/yolov7.onnx |
yolo7 | |
| YOLO8 Ultralytics Object Detection Inference |
A computer vision model that uses YOLO8 deep learning to detect objects in images and videos
Additional steps: After installing the patch 1. Create a new directory in the assets folder: INFINIWORKFLOW_PATH/assets/yolo8 1. Download https://dl.opencv.org/models/yolov8/yolov8n.onnx 2. Copy yolov8n.onnx to: INFINIWORKFLOW_PATH/assets/yolo8/yolov8n.onnx |
yolo8 | |
| YOLO9 Ultralytics Object Detection Inference |
A computer vision model that uses YOLO9 deep learning to detect objects in images and videos
Additional steps: After installing the patch 1. Create a new directory in the assets folder: INFINIWORKFLOW_PATH/assets/yolo9 1. Download https://dl.opencv.org/models/yolov9/yolov9t.onnx 2. Copy yolov9t.onnx to: INFINIWORKFLOW_PATH/assets/yolo9/yolov9t.onnx |
yolo9 | |
| YOLO10 Ultralytics Object Detection Inference |
A computer vision model that uses YOLO10 deep learning to detect objects in images and videos
Additional steps: After installing the patch 1. Create a new directory in the assets folder: INFINIWORKFLOW_PATH/assets/yolo10 1. Download https://dl.opencv.org/models/yolov10/yolov10s.onnx 2. Copy yolov10s.onnx to: INFINIWORKFLOW_PATH/assets/yolo10/yolov10s.onnx |
yolo10 | |
| YOLOX Inference | YOLOX is a high-performing object detector based on the YOLO series | yolox_inference | |
| Yahoo Finance API | Realtime Yahoo Finance quotes | yfinance | |
| Philips Hue | Trigger Philips Hue Lights | philips_hue | |
| Geo API | Geo and Geo Reverse | geopy | |
| Slack | Send Slack Message | slack_utility | |
| Send Email | Send Email | send_email | |
| Blink1 | Blink1 LED Light | blink | |
| Upload Video | Upload Video | upload_video | |
| Google Drive | Google Drive Upload and Download | google_drive | |
| AWS S3 | AWS S3 Upload and Download | aws_s3 | |
| Azure | Azure Upload and Download | azure | |
| Live Stream | Live Stream | live_stream | |
| PyRender | PyRender - 3D Rendering | pyrender | |
| YouTube Reader | YouTube Reader | youtube_reader | |
| Livestream Chat | Livestream Chat | livestream_chat | |
| OpenNI Depth Sensor | OpenNI Depth Sensor | openni_depth_sensor_windows | |
| Mask 2 Former | A unified framework for panoptic, instance and semantic segmentation | mask2former | |
| ONNX Runtime | Accelrated C++ Inference engine for running ONNX models | onnx_runtime_windows | |
| YOLOX Train | YOLOX Train | yolox_train | |
| Audio |
Audio
Additional steps: After installing the patch 1. Download FFMPEG: https://ffmpeg.org/download.html 2. Place the ffmpeg executable in your path, or in external/bin folder |
audio | |
| Audio Classify | Audio Classify | audio_classify | |
| Database | Database | database | |
| Robot Operating System (ROS) |
A set of libraries that communicates data and actions across sensors and robotic devices.
Photron does not distribute ROS as part of INFINIWORKFLOW, if you wish to use ROS, you must download it separately and agree to the license terms on your usage: ROS2 LICENSE Additional steps: 1. Install ROS2: https://docs.ros.org/en/jazzy/Installation/Ubuntu-Install-Debs.html 2. Make sure you source the ROS environment: source /opt/ros/jazzy/setup.bash 3. Start INFINIWORKFLOW using LD_LIBRARY_PATH=$LD_LIBRARY_PATH:../external/bin/onnxruntime/:../external/bin/opencv2/:../external/bin/ python3 app.py |
ros2 | |
| RTCBot | RTCBot | rtcbot | |
| Serial | Serial | serial | |
| OCR | OCR | ocr | |
| Text to Speech | Speech Synthesis | text2speech | |
| MQTT | MQTT Publish and Subscribe for factory-floor/IoT messaging | mqtt | |
| Modbus | Modbus TCP Read and Write for PLCs and factory equipment | modbus | |
| QR Code and Barcode Generator | Generate QR Code and Barcode images from text | qr_barcode_generator |
There are three flavours of patches - one is a patch to the application, one is a SDK patch and the others are feature patches to allow installation of different demos and packages. See the Downloadable Packages for the full list. The application patch is meant to patch your exisiting installation with a smaller set of files and thus has a substantially smaller download size. It is expected that the application patch will be frequently updated when bugs are fixed and small features added. To patch your exisiting installation you should download the patch, and then open a Windows Powershell prompt and then change the directory to your existing INFINIWORKFLOW installation (i.e. where you unzipped your original installation). Then type the following where you should replace /folder/to/ to the actual folder you downloaded the application patch:
.\patch.bat /folder/to/infiniworkflow_patch_v1_0.zip
sh patch.sh /folder/to/infiniworkflow_patch_v1_0.zip
The packages (feature patches) are meant to allow new functionaility including demos and assets such as the ML model. For example you can download the YOLO classifier feature patch, yolo_v1_0.zip then install by open a Windows Powershell prompt and then change the directory to your existing INFINIWORKFLOW installation (i.e. where you unzipped your original installation) and type the following where you should replace /folder/to/ to the actual folder you downloaded the feature patch:
.\patch.bat /folder/to/yolo_v1_0.zip
sh patch.sh /folder/to/yolo_v1_0.zip
To ensure security, INFINIWORKFLOW provides the ability to check User Authentication before users may access the application. This is not enabled by default, but can be enabled easily by your administrator by running the following python script to create a INFINIWORKFLOW superuser:
cd /path/to/INFINIWORKFLOW/app
python3 create_superuser_script.py
With a superuser created, User Authentication is enabled. Now, when starting INFINIWORKFLOW, users will reach the login page (as seen below) and have to enter their login information to continue to INFINIWORKFLOW.
When a user is finished using Infiniworkflow, they may click "Yes" or "Save and Exit" to exit the application, or they may click "Logout", which saves the current workflow and logs out of the session, returning back to the login screen.
As an Admin, you will have extended permissions that regular users won't, which includes making and removing users and groups. To access these controls, click on the Admin link and enter your admin username and password to continue. You will then reach a page like this, with all of the Admin controls available.
You can start INFINIWORKFLOW from the command line. To successfully execute you need to change the current directory to the app folder located under INFINIWORKFLOW and then run the command:
Note, on OSX and Linux instead of using "..\python.exe", you should use "python3"
..\python.exe app.pyc
[-help]
[-url ...]
[-port ...]
[-device #]
[-cuda ...]
[-batch]
[-final node name or node uuid]
[-override json]
[-nobrowser]
[-path ...]
[-source ...]
[-resolution ...]
[-test ...]
[-fps ...]
[-mcp]
[-label_variations directory]
[-movie movie_prefix]
[-clean]
[-studio]
[-distributed-listen]
[-distributed-token ...]
[-clip_uuid ...]
[-clip_start ...]
[-clip_end ...]
[-clip_result ...]
[workflow]
| Argument name | Description | Default Value | Example |
|---|---|---|---|
| -url | url the server will start on, if not specified starts on localhost | 127.0.0.1 | python app.py -url 192.168.5.52 |
| -device | allows you to set the default GPU device used | 0 | ..\python.exe app.pyc -device 1 |
| -cuda | enable or disable cuda system will test system again to determine if cuda is available auto will use prior system check options: [auto|on|off|system] |
auto | ..\python.exe app.pyc -cuda off |
| -batch | batch mode will not show UI | ..\python.exe app.pyc -batch |
|
| -final | if in batch mode you can set the final node that you wish to execute before you exit. Either pass in node name or node uuid | ..\python.exe app.pyc -final R2Score |
|
| -override | a json string that allows you to override attributes in the workflow you pass | ..\python.exe app.pyc -final "{'Set Int.value' : '5', 'Set Int1.value' : '11' }" |
|
| -help | shows a help message | ..\python.exe app.pyc -help" |
|
| -port | specify which network port to use, if none specified then 5000 is used | 5000 | ..\python.exe app.pyc -port 8888" |
| -nobrowser | does not automatically open a browser | ..\python.exe app.pyc -nobrowser" |
|
| -path | establishes paths that can be used as a prefix | ..\python.exe app.pyc -path "captures=C:\Users\imagi\CapturesFolder;media=C:\media" |
|
| -source | creates a workflow with a movie reader with this media | ..\python.exe app.pyc -path "media=C:\media_folder" -source media:movie.mp4 |
|
| -test | Sets the project resolution | ..\python.exe app.pyc -resolution 1920x1080 |
|
| -test | See Automated Testing | ..\python.exe app.pyc -test yolo |
|
| -label | Allow labelling of workflows used for training LLM | ..\python.exe app.pyc -label |
|
| -fps | Set python rendering fps | 60 | ..\python.exe app.pyc -fps 100 |
| -mcp | Starts MCP server. See MCP Server | ..\python.exe app.pyc -mcp |
|
| -label_variations | Runs batch variation preprocessing for the specified directory (e.g., assets folder). | ..\python.exe app.py -batch -label_variations ..\assets\AI_Welding |
|
| -movie | Filter variation subdirectories under the .variations folder when running -label_variations. If specified, only folders starting with the given name/prefix (e.g., "bighole" matching "bighole - Copy", "bighole - Variation #1") are searched, cleaned, and processed. | ..\python.exe app.py -batch -label_variations ..\assets\AI_Welding -movie bighole |
|
| -devices | (Used with run_batch_preprocess.py) Number of devices (e.g. GPUs) to distribute batches over. If set to 2, it creates 2x more batches, passing -device 0 to the first half and -device 1 to the second half. | 1 | ..\python.exe run_batch_preprocess.py -label_variations ..\assets\AI_Welding -devices 2 |
| -clean | Force full recomputation of variations when used with -label_variations by deleting existing completion cookies and images first. Supports -movie filtering. | ..\python.exe app.py -batch -clean -label_variations ..\assets\AI_Welding |
|
| -resolution | Sets the initial viewer resolution, as WIDTHxHEIGHT | ..\python.exe app.pyc -resolution 1920x1080 |
|
| -studio | Opens directly into Studio mode instead of the default editor view | ..\python.exe app.pyc -studio |
|
| -distributed-listen | Accept Network Manager jobs pushed from another INFINIWORKFLOW instance on the control channel (port 5099) and run them here - opt-in, requires -distributed-token to actually accept anything |
..\python.exe app.pyc -distributed-listen -distributed-token my-shared-secret |
|
| -distributed-token | Shared secret required on both ends of a Network Manager job dispatch (the orchestrator sending it and the -distributed-listen instance receiving it) - without it, a -distributed-listen instance refuses all incoming jobs and an orchestrator refuses to send to a remote target |
..\python.exe app.pyc -distributed-token my-shared-secret |
|
| -clip_uuid | (Batch mode, internal - Render to Clip remote dispatch) uuid of the cv.video_writer node to bound-export. Not intended to be set by hand - the orchestrating instance passes this automatically when dispatching a clip render to a remote peer. |
||
| -clip_start | (Batch mode, internal) First frame of the -clip_uuid bounded export |
||
| -clip_end | (Batch mode, internal) Last frame of the -clip_uuid bounded export |
||
| -clip_result | (Batch mode, internal) Path to write the clip render result JSON to before exiting | ||
| workflow | The final argument is the workflow json file | ..\python.exe app.pyc ..\demos\Untitled\Untitiled.json" |
Plugins can be implemented in C++ or Python and both will require a JSON file. The JSON Schema specifies the input and output parameters as well as the name and description of the plugin amongst other things.
To start creating your own plugin, it is recommended you base your code on the Canny2 plugin that is provided upon installation. For Windows users, you can immeadiately run the Canny2 plugin via the Visual Studios solution. For Mac and Linux users, the process of creating a plugin will require a few more steps. These steps are detailed below in the section "Creating Plugins for Mac/Linux"
A simple way to make your own tool without writing Python or C++ code is to simply use an existing tool and customize its parameters. You simply create an updated JSON for the tool and place it in the extensions folder. You can get the JSONs for the exisiting tools in the subfolders in the app/catalog folder. For example, say we want to customize the "Lift" Tool to create a new "Red Lift" Tool - this tool would allow the lift color correction but the default value for the red parameter would be higher. The steps are as follows:
"identifier": "red_lift:cv.color_lift",
"category": {
"id": "Color"
},
"icon": "bi-palette",
The category is in the "id" attribute in the app/catalog/toc.json. This will place the "Red Lift" tool in the exisiting Color tab of the
Toolbox.
Alternatively, you can create a new category "id" and that will show up in the Toolbox in a new tab.
All plugins must have an accompanying JSON file. The JSON file specifies the input and output parameters as well as the name and description of the plugin amongst other things. The specification of the schema is as follows:
| Attribute name | Mandatory | Default Value | Description | Example |
|---|---|---|---|---|
| title | The UI name of the plugin |
"title" : "Canny Edge Detector"Or you can specify a localized set "title": {
"en_US": "Canny Edge Detector",
"ja-JP": "キャニーエッジ検出器",
"es-ES": "Detector de bordes Canny",
"de_DE": "Canny Kantendetektor",
"zh_CN": "Canny 边缘检测器"
}
|
||
| identifier | The name of the plugin file |
Python:
"identifier": "day_of_week.py"C++: "identifier": "Canny2.plugin" |
||
| description | The description that explains the purpose of this plugin which will be shown in the UI |
"description" : "Canny Edge Detection is a popular edge detection algorithm"Or you can specify a localized set "description": {
"en_US": "Canny Edge Detection is a popular edge detection algorithm",
"ja-JP": "Canny Edge Detectionは人気のエッジ検出アルゴリズムです",
"es-ES": "Canny Edge Detection es un popular algoritmo de detección de bordes",
"de_DE": "Canny Edge Detection ist ein beliebter Kantenerkennungsalgorithmus",
"zh_CN": "Canny 边缘检测是一种流行的边缘检测算法"
}
|
||
| url | www.photron.com | A URL that is shown in the UI to have more information about the plugin |
"url": "https://docs.opencv.org/3.4/dd/d1a/group__imgproc__feature.html#ga04723e007ed888ddf11d9ba04e2232de" |
|
| tags | A list of tags that is associated with the plugin |
"tags": ["opencv", "edges", "canny"]Or you can specify a localized set: "tags": {
"en_US": ["opencv", "edges", "canny"],
"ja-JP": ["オープンCV", "エッジ", "賢い"],
"es-ES": ["abrircv", "bordes", "astuto"],
"de_DE": ["OpenCV", "Kanten", "schlau"],
"zh_CN": ["opencv", "边缘", "精明的" ]
}
|
||
| icon | "icon": "bi-heart-fill" |
A bootstrap icon that represents the plugin in the UI |
"icon": "bi-star" |
|
| category | The category the plugin will be placed in the Toolbox |
If you want to specify your own new category
"category": {
"id": "python_scripts",
"description": "User defined python scripts",
"icon": "bi-filetype-py"
}
If you want to place it in an exisiting category
"category": {
"id": "Photron"
}
|
||
| language | Must be either python, c++ or cuda |
"language": "c++" |
||
| gpu | "gpu": false |
Informs if CUDA GPU is recommended for execution |
"gpu": true |
|
| os | "os": ["windows", "osx", "linux"] |
Specifies if the plugin wants to limit which Operating Systems the plugin will be available |
"os": ["osx"] |
|
| supervise | "supervise": false |
Specifies if the plugin wants to handle supervion callbacks to enable/disable or hide/show parameters dynamically Note you must also set one of the input parameters to have a supervise attribute to be true which are the parameters that cause other parameters to change visibility |
"supervise": true |
|
| supervise_values | "supervise_values": [] |
Specifies if the plugin wants to handle supervion callbacks to dynamically set values, specify the list of identifiers you want to update their values on a supervised change |
"supervise_values": ["color", "opacity"] |
|
| inputs | "inputs": [] |
An array of input objects that specifies each input of the plugin - see inputs schema |
"inputs": [
{
"name": "source",
"type": "image2D",
"mandatory": true,
"description":"Input image",
"identifier": "source"
},
{
"name": "threshold1",
"type": "double",
"default": "100.0",
"mandatory": true,
"description": "First threshold for the hysteresis procedure",
"identifier": "threshold1"
},
...
]
|
|
| outputs | "outputs": [] |
An array of output objects that specifies each output of the plugin - see outputs schema |
"outputs": [
{
"name": "out",
"type": "image2D",
"description": "Output edge map; single channels 8-bit image, which has the same size as image",
"identifier": "out"
}
]
|
|
| overlay | "overlay": [] |
An array of svg elements that are drawn in the viewer when the node is viewed, the markup has a special inputs attribute to specify the list of input parameters |
"overlay": [
"<polygon stroke='yellow' opacity='0.5' stroke-width='2' fill='none' inputs='src[0],src[1],src[2]' />",
"<polygon stroke='limegreen' opacity='0.5' stroke-width='2' fill='none' inputs='dst[0],dst[1],dst[2]' />"
]
|
| Attribute name | Mandatory | Default Value | Description | Example |
|---|---|---|---|---|
| name | The UI name of the input parameter |
"name" : "out"Or you can specify a localized set "name": {
"en_US": "out",
"ja-JP": "出力",
"es-ES": "afuera",
"de_DE": "aus",
"zh_CN": "出去"
|
||
| identifier | The unique identifier for this input parameter |
"identifier": "out" |
||
| description | The description that explains the purpose of this output |
"description" : "Second threshold for the hysteresis procedure"Or you can specify a localized set "description": {
"en_US": "Second threshold for the hysteresis procedure",
"ja-JP": "ヒステリシス手順の2番目の閾値",
"es-ES": "Segundo umbral para el procedimiento de histéresis",
"de_DE": "Zweiter Grenzwert für das Hystereseverfahren",
"zh_CN": "滞后过程的第二个阈值"
}
|
||
| type | The type of the parameter which which include the standard types: int, double, int2, double2, bool, string, numeric, image2D, cuda2D. Or you can define your own type name. |
"type": "double" |
||
| mandatory | "mandatory": false |
Specifies if the input parameter is mandatory and must be set by the user. |
"mandatory": true |
|
| default | The default value of the input parameter which is must be enclosed in a string. No default values should be needed for types that are not set directly by the user e.g. image2D and cuda2D |
"default": "200.0" |
||
| min | Only for numeric types such as int or double. The minimum value the input value can be set to |
"min": 5.0 |
||
| max | Only for numeric types such as int/int2/int3 or double/double2/double3. The maximum value the input value can be set to |
"max": 10.0 |
||
| softmin | "softmin": false |
Only for numeric types such as int/int2/int3 or double/double2/double3 that are not sliders but textfields. If softmin is true then the limit is only via the dragging of the UI, if you enter manually in the textfield the limit does not apply |
"softmin": true |
|
| softmax | "softmax": false |
Only for numeric types such as int/int2/int3 or double/double2/double3 that are not sliders but textfields. If softmax is true then the limit is only via the dragging of the UI, if you enter manually in the textfield the limit does not apply |
"softmax": true |
|
| step | Only for numeric types such as int or double. The step value the increments of the parameter UI will jump up and down |
"step": 1.0 |
||
| permitted | Only for int or string types. An array of strings that will be in the selection UI menu or the tag selection UI | |||
| private | "private": false |
Will not show the parameter in the UI |
"private": true |
|
| editable | "editable": true |
If the parameter can be editable or not, if not editable it will be disabled in the UI |
"editable": false |
|
| hidden | "hidden": false |
If the input is shown in the flowgraph |
"hidden": true |
|
| editor_hidden | "editor_hidden": false |
If the parameter is shown in the editor UI |
"editor_hidden": true |
|
| multiple | "multiple": false |
Only for string types that will allow multiple values to entered in the tag UI |
"multiple": true |
|
| userOptionAllowed | "userOptionAllowed": false |
Only for string types that allow user defined strings to be entered in the tag UI |
"userOptionAllowed": true |
|
| look |
A hint to indicate how the UI should be represented instead of the default look
int types: button, slider int2, double2, numeric2: point string: map, filebrowser, curve, path, table, html, week, month, time, date, datetime-local |
"look": "button" |
||
| icon | "icon": "bi-fire" |
The icon for parameters that have a button look |
"icon": "bi-robot" |
|
| maxrows | "maxrows": 100 |
Only for table looks to set the maximum number of rows for the table |
"maxrows": 2 |
|
| maxcols | "maxcols": 100 |
Only for table looks to set the maximum number of columns for the table |
"maxcols": 2 |
|
| random | "random": false |
Only for color, int, int2, int3, double, double2, double3 types that ignores the default value and sets a random value instead |
"random": true |
|
| ganged | Only for int2, double2, numeric2 types that allows both the dimensions to be ganged and set to the same value |
"ganged": "button" |
||
| multiline | "multiline": false |
Only for string types that indicate if the UI should have a text or a single textfield widget |
"multiline": true |
|
| rows | Only for string types with a multiline set to true, indicates the number of rows of the text widget |
"rows": 5 |
||
| cols | Only for string types with a multiline set to true, indicates the number of columns of the text widget |
"cols": 10 |
||
| password | Only for string types to make the text now show when you enter text in the widget |
"password": true |
||
| supervise | "supervise": false |
Specifies when this parameter changes if you want to handle supervion callbacks to enable/disable or hide/show parameters dynamically. Note you must also set the supervise attribute of the main JSON object to true as well |
"supervise": true |
|
| smooth | "smooth": true |
Only for path types that indicate if the path can have bezier smooth control handles |
"smooth": false |
|
| moveonly | "moveonly": false |
Only for path types that indicate if the path can only have points moved (no insertion or deletion of any points) |
"moveonly": true |
| Attribute name | Mandatory | Default Value | Description | Example |
|---|---|---|---|---|
| name | The UI name of the output parameter |
"name" : "threshold2"Or you can specify a localized set "name": "name": {
"en_US": "threshold2",
"ja-JP": "閾値2",
"es-ES": "umbral2",
"de_DE": "Schwelle2",
"zh_CN": "阈值2"
}
|
||
| identifier | The unique identifier for this output parameter |
"identifier": "threshold2" |
||
| description | The description that explains the purpose of this output |
"description" : "Output edge map; single channels 8-bit image, which has the same size as image"Or you can specify a localized set "description": {
"en_US": "Output edge map; single channels 8-bit image, which has the same size as image",
"ja-JP": "出力エッジマップ。画像と同じサイズの単一チャネル8ビット画像。",
"es-ES": "Mapa de borde de salida; imagen de 8 bits de canales individuales, que tiene el mismo tamaño que la imagen",
"de_DE": "Ausgabekantenkarte; Einzelkanal-8-Bit-Bild, das die gleiche Größe wie das Bild hat",
"zh_CN": "输出边缘图；单通道8位图像，与图像大小相同"
}
|
||
| type | The type of the parameter which which include the standard types: int, double, int2, double2, bool, string, map, numeric, image2D, cuda2D. Or you can define your own type name. |
"type": "image2D" |
||
| column_names | For matrix2D, the list of column names used in the viewer when showing this output |
"column_names": ["x", "y"] |
||
| graph | "graph": false |
For matrix2D types, visualize by default as graph chart |
"graph": true |
|
| look | For string types, show as a DataFrame |
"look": "DataFrame" |
||
| hidden | "hidden": false |
If the output is shown in the flowgraph |
"hidden": true |
The Python SDK uses the PythonNode base class and at a minimum you need to define a new instance which you should return in the result variable. The final plugin will be the python script and should be placed in the Extensions folder together with its JSON file. A simple example is the days_of_weeks.py sample plugin provided. The python code is as follows
from python_node import PythonNode
import datetime
class DayOfWeekNode(PythonNode):
def __init__(self):
super().__init__()
self.value = None
def execute(self, host):
if not host.is_enabled():
self.value = False
else:
year = host.get_input_int_value(0)
month = host.get_input_int_value(1)
day = host.get_input_int_value(2)
self.value = datetime.datetime.strptime(str(day) + "/" + str(month+1) + "/" + str(year), "%d/%m/%Y").strftime('%A')
host.set_output_value(0, self.value)
....
def copy(self, host):
return DayOfWeekNode()
result = DayOfWeekNode()
If you wish to instead use an existing Node but with a different JSON, e.g. you want to use the GenericDataset tool but set the parameters and hide them, then no code is needed any instead the result variable should return the identifier of the existing tool:
result = "torch.generic_loader"
However, using the Python API provides you full capability as long as you override the PluginApi class which requires 3 at least methods to be implemented: copy, execute and view_html. The following methods should be overriden by your derived class of PluginApi
| Instance method | Mandatory | Arguments | Return type | Purpose |
|---|---|---|---|---|
| copy | self, host : PluginHost |
instance of this plugin class |
This method will be called when INFINWORKFLOW requests a copy of an instance of this class which should return a deep copy. | |
| execute | self, host : PluginHost |
None |
The method called when the plugin is executed usually when some input parameters have changed. You can call the host to get input values, e.g. host.get_input_int_value(...), and finally set the output value. If the execution was unsuccessful then you can call host.set_error_message with the error message If you want to have the node to be executed you can call host.set_dirty(True) otherwise the node will only get re-executed when input parameters have been modified | |
| view_html | self, host : PluginHost, nth_output : int |
string |
The method is called after the execute method, when the output of node is viewed The view_html should return a html string that represents the output of the nth output. Typically, in the execute method you can compute output values and store them in instance variables of the class and then later in the view_html you can use those values to establish what HTML string you will pass back. | |
| has_dynamic_inputs | self, host : PluginHost |
bool |
Returns if the plugin has dynamic inputs, defaults to False | |
| has_dynamic_outputs | self, host : PluginHost |
bool |
Returns if the plugin has dynamic outputs, defaults to False | |
| allows_inference_macro | self, host : PluginHost |
bool |
Returns if if the node allows inference macros to be created | |
| update_inference_macro_json | self, host : PluginHost, tool_json : dict |
None |
Updated the Tool JSON for the inference macro | |
| get_macro_identifier | self, host : PluginHost |
str |
Gets the base macro for the inference tool generation | |
| reset_trigger_counters | self, host : PluginHost, nth_index : int |
None |
The trigger at the nth index should reset any internal state that you maintain | |
| get_adornment | self, host : PluginHost, output_port_num : int, output_type : str |
None |
Returns the adornment in the UI, returning "1" adds the slicing adornment | |
| supervise_inputs | self, host : PluginHost, inputs_flags : ctypes.c_int_array |
False |
Called to dynamically enable/disable or hide/show parameters. Modify inputs_flags using SUPERVISE_FLAG_HIDDEN (1) or SUPERVISE_FLAG_DISABLED (2). | |
| supervise_value | self, host : PluginHost, nth_index : int |
None |
Called for indices listed in update_values in the JSON. Return a string (e.g., "0.0,0.0,1.0") to force a new value in the UI. |
The methods for PluginApi have a instance of PluginHost, the host, which is a helper class that allows you to call INFINIWORKFLOW related functions. The execute method should for example call the methods to get input values (e.g. get_input_int_value) and set the output value (i.e. set_output_value). The following methods should can be called on the PluginHost
| Instance method | Purpose | Example |
|---|---|---|
| get_input_value | During exeuction you can get the value of an input to the plugin, where you pass the order of the parameter E.g. pass 0 for the first input parameter | value = host.get_input_value(3) |
| get_input_bool_value | A helper method that calls get_input_value and returns the value as a bool Python type | value = host.get_input_bool_value(3) |
| get_input_int_value | A helper method that calls get_input_value and returns the value as a int Pythontype | value = host.get_input_int_value(3) |
| get_input_numeric_value | A helper method that calls get_input_value and returns a float, int or bool Python type | value = host.get_input_numeric_value(3) |
| get_input_string_value | A helper method that calls get_input_value and returns a str Python type | value = host.get_input_string_value(3) |
| get_input_filename_value | A helper method that calls get_input_value and returns a str Python type and resolves the path (replacing the ${assets} with the correct path) | value = host.get_input_filename_value(3) |
| get_input_map_value | A helper method that calls get_input_value and returns a dict Python type | value = host.get_input_map_value(3) |
| get_input_bool_list_value | A helper method that calls get_input_value and returns a list of bool Python type | value = host.get_input_bool_list_value(3) |
| get_input_numeric_list_value | A helper method that calls get_input_value and returns a list of float, int or bool Python type | value = host.get_input_numeric_list_value(3) |
| get_input_string_list_value | A helper method that calls get_input_value and returns a list of str Python type | value = host.get_input_string_list_value(3) |
| set_output_value | During exeuction you can set the value of an output to the plugin, where you pass the order of the parameter and the value E.g. pass 0 for the first input parameter This also will set the dirty flag to False (see set_dirty) | host.set_output_value(0, result) |
| set_dirty | During exeuction you can set if the node has been executed by setting dirty flag to False. This is automatically set when you set the outputs but you can set it to True if you want to have the node get executed again | host.set_dirty(True) |
| get_num_inputs | Returns the number of inputs that plugin has | value = host.get_num_inputs() |
| get_num_outputs | Returns the number of outputs that plugin has | value = host.get_num_outputs() |
| set_error_message | Sets an error message that will be shown in the UI | host.error_message("Something bad happened") |
| is_enabled | Returns if the node is enabled | value = host.is_enabled() |
| is_cancel_render | Returns true if the user has pressed cancel during the execution, in which case you should return from execution | value = host.is_cancel_render() |
| convert_filepath_to_relative_path | Returns the argument path from an absolute path to relative path i.e. will prefix the path with ${assets} as appropriate | updated_path = host.convert_filepath_to_relative_path(path) |
| get_source_time | Returns the source time depending on where the source originated from upstream e.g. if the source is from a Movie it will be frame number or if it is a web camera then it will be the epoch time | source_time = host.get_source_time(path) |
| set_source_time | Sets the source time, all further downstream nodes will inherit this time | host.get_source_time(source_time) |
| is_triggered | Returns if the trigger parameter at the nth input has been triggered or not | host.is_triggered(counter, nth_index) |
| get_view_slice | Returns the view slice value in viewer | host.get_view_slice() |
The C++ SDK is based on compiling a DLL using some standard headers and libraries provided in the INFINIWORFLOW SDK patch package. You can use the exisiting Canny2 example as a starting point and rename all the files to your plugin name. The final plugin will be a DLL but prefixed with the .plugin extension and should be placed in the Extensions folder together with its JSON file. The SDK is based on Microsoft Visual Studio 2022 and only supports x86_64 builds.
The following methods should be overriden by your derived class of PluginApi
| Instance method | signature | Purpose |
|---|---|---|
| setup | bool setup(PluginHost * host); |
Setup a Plugin and will be called anytime the thread to run the plugin is started |
| update | bool update(PluginHost * host, BlobHandle blob, int inputIndex); |
Update the Plugin instance based on a change of the input blob This will be called anytime the user changes a property, the inputIndex is the index into the "inputs" array in the JSON representing the parameters Typically you can copy the value of the contents of the Blob instance (using the host API such as getAsDouble) and then copy this into your plugin instance |
| superviseInputs | bool superviseInputs(PluginHost * host, int* inputsFlags); |
Called if the plugin sets "supervise" in the JSON and will allow you to enable/disable and/or hide or show parameters The inputsFlags is an array that is the size of the number of inputs and you are responsible to set the values You can set the flags SUPERVISE_FLAG_NORMAL (0) to have it shown regularly Or set it to the flag SUPERVISE_FLAG_HIDDEN (1) to have it hidden Or set it to the flag SUPERVISE_FLAG_DISABLED (2) to have it disabled |
| superviseValue | bool superviseValue(PluginHost* host, int index, char* value); |
Called for indices listed in the "update_values" array in the node JSON. The value parameter is a 512-byte buffer. Return true and write a string to the buffer (e.g., a hex color or status text) to force a new value into the UI widget. |
| execute | bool execute(PluginHost * host); |
Execute a Plugin - you can call the pluginGetOutputBlob to get the output blob Return true if success or false otherwise |
| isCached | bool isCached(PluginHost * host); |
Asks node if it has cached any blob values inits instance variables If so then the host may call flushCache |
| flushCache | bool flushCache(PluginHost * host); |
Asks node to flush its cache - anything it has stored must be released For example, if you have cached the image as a Mat then release it |
| teardown | bool teardown(PluginHost * host); |
Teardown called anytime the thread to run the plugin is stopped |
| destroy | bool destroy(PluginHost * host); |
Destroy a Plugin which is you can destroy your plugin instance data |
The following methods should can be called on the PluginHost that is passed into the API methods of PluginApi
| Signiture | Purpose |
|---|---|
| BlobHandle getOutputBlob(int outputNum) | Get the Output Blob handle which can be callied during the executePlugin call, the outputNum is the index into the "outputs" array in the JSON representing the output |
| void setNumOutputs(int numOutputs) | Sets the number of outputs the blob supports and can be called during makePlugin |
| void setErrorMessage(const char *message) | Notifies an error has occurred which will be shown in the UI |
| void isEnabled() | Checks if the node is enabled, if not then the plugin should usually just copy source to output |
| bool hasOutputObservers(int outputIndex) | Returns if the output is currently connected - only render the output if it has observers |
| cv::Mat& getAsImage2D(BlobHandle blob) | From the blob handle get the reference to a two dimensional image represented by OpenCV matrix |
| cv::Mat& getAsMatrix2D(BlobHandle blob) | From the blob handle get the reference to a two dimensional matrix represented by OpenCV matrix |
| double &getAsDouble(BlobHandle blob) | From the blob handle get the double value it represents |
| int &getAsInt(BlobHandle blob) | From the blob handle get the int value it represents |
| bool &getAsBool(BlobHandle blob) | From the blob handle get the bool value it represents |
| std::string &getAsString(BlobHandle blob) | From the blob handle get the string value it represents |
| std::string &getAsFilename(BlobHandle blob) | From the blob handle get the filename value it represents, resolving the prefix ${assets} |
| double *getAsDouble2(BlobHandle blob) | From the blob handle get the 2D double point value it represents |
| int *getAsInt2(BlobHandle blob) | From the blob handle get the 2D integer point value it represents |
| float *getAsColor3f(BlobHandle blob) | From the blob handle get the color RGB value it represents |
| void cloneFromImage(BlobHandle blob, cv::Mat& dest) | From the blob handle gets a cloned copy (which may be opencv mat or cuda memory) |
| CudaSuite *getCudaSuite() | Get Cuda Suite - may return nullptr if no cuda supported on host |
The following methods should can be called on the CudaSuite
| Signiture | Purpose |
|---|---|
| bool allocateCudaImage(int width, int height, int type, int rowbytes, int numChannels, CudaImage& cudaImage) | Allocate a Cuda Image on the device |
| void freeCudaImage(CudaImage& cudaImage) | Free a Cuda Image |
| void setCudaImage(BlobHandle blob, const CudaImage& cudaImage) | Set the Cuda Image to the output Blob |
| void cloneCudaImageFromBlob(BlobHandle blob, CudaImage& cudaImage) | Copy and clone the cuda image from the input blob |
| void upload(const cv::Mat mat, CudaImage& cudaImage) | Utility method to upload CPU image to Cuda device Image |
| cv::Mat download(CudaImage& cudaImage) | Utility method to download to CPU image from Cuda device Image |
As with the C++ SDK instructions, you can use the exisiting Canny2 example as a starting point and rename all the files to your plugin name. See the section above for instructions on how to do this.
With your JSON now set up, you will need to construct the CMakeLists.txt file for your plugin. Open Canny2's CMakeLists.txt for reference. In your plugin's CMakeLists.txt file, rename any instance of "canny2" to the name of your plugin. Everything else should be kept the same.
In your console, "cd" into the folder containing your plugin. For Canny2, this is in "infini-workflow/sdk/examples/Canny2". Once you are in this folder in your console, enter the following command to build x86_64 architecture:
cmake -B build . -G "Unix Makefiles"For arm64 architecture enter the following command to build x86_64 architecture:
cmake -DCMAKE_VS_PLATFORM_NAME="arm64" -B build . -G "Unix Makefiles"This will create a folder called "build" within the directory you are currently in. Then enter the following commands:
cd build makeThis should output a few lines of text, with the last one being "[100%] Built target {NAME_OF_PLUGIN}". Now when you run INFINIWORKFLOW on a Mac or Linux device, the plugin will be available and usable.
The Shader SDK is based on OpenGL GLSL shader language. You can use the exisiting "ShaderExample" in the SDK as a starting point and rename all the files to your plugin. The final plugin will be a combination of a JSON file and a GLSL shader file and both should be placed in the Extensions folder. Additonally, the GLSL shader file should also be placed in the assets/shader folder as well. The shader names will match the names of the input parameters in the JSON file, an example GLSL file is as follows:
#version 150 core
in vec3 Color;
in vec2 Texcoord;
out vec4 outColor;
uniform sampler2D source;
uniform float brightness;
uniform float redBrightness;
uniform float greenBrightness;
uniform float blueBrightness;
void main()
{
vec4 pixel = texture(source, Texcoord);
outColor = pixel * vec4(brightness + redBrightness, brightness + greenBrightness, brightness + blueBrightness, 1.0);
}
The JSON file should have the name of the identifier the same name as the GLSL file. The first input as a string which defaults to the location of the shader file in the assets shaders directory. Next, the JSON file should have a inputs for each uniform, where the sampler2D input will be of type "texture2D". Finally, the JSON should have one output will be of type "texture2D" The example JSON is as follows for this GLSL file:
{
"identifier": "example_shader.glsl",
"description": {
"en_US": "Example GPU GLSL Shader"
},
"url": "https://www.khronos.org/opengl/wiki/OpenGL_Shading_Language",
"icon": "bi-plugin",
"tags": [
"opengl",
"GLSL",
"gain",
"texture"
]
},
"language": "opengl",
"inputs": [
{
"name": "shader",
"type": "string",
"private": true,
"default": "${assets}/shaders/example_shader.glsl",
"description": "Shader Source",
"identifier": "shader"
},
{
"name": "source",
"type": "texture2D",
"mandatory": true,
"description": "Input texture",
"identifier": "source"
},
{
"name": "brightness",
"type": "double",
"default": "0.0",
"min": "-1.0",
"max": "4.0",
"look": "slider",
"mandatory": false,
"description": "Overall brightness amount",
"identifier": "brightness"
},
{
"name": "redBrightness",
"type": "double",
"default": "1.0",
"min": "0.0",
"max": "4.0",
"look": "slider",
"mandatory": false,
"description": "Red brightness amount",
"identifier": "redBrightness"
},
{
"name": "greenBrightness",
"type": "double",
"default": "1.0",
"min": "0.0",
"max": "4.0",
"look": "slider",
"mandatory": false,
"description": "Green brightness amount",
"identifier": "greenBrightness"
},
{
"name": "blueBrightness",
"type": "double",
"default": "1.0",
"min": "0.0",
"max": "4.0",
"look": "slider",
"mandatory": false,
"description": "Blue brightness amount",
"identifier": "blueBrightness"
}
],
"outputs": [
{
"name": "out",
"type": "texture2D",
"description": "Output texture",
"identifier": "out"
}
],
"title": {
"en_US": "Example GLSL Shader"
}
}
To perform automated testing you will need to create a directory "tests" and in this directory place a json test script called "tasks.json". For example, if your test was called "assembly_line", then create a folder "tests/assembly_line" under INFINIWORKFLOW main installation folder. Then place any expected results in the folder "tests/assembly_line/expected_results" that will be used in the "assertEquals" task (see JSON testing schema below for details) To start testing, open a terminal/powershell and change the current directory to the app folder located under INFINIWORKFLOW and then run the command:
..\python.exe app.pyc -test assembly_line
Note, on OSX and Linux instead of using "..\python.exe", you should use "python3"
The terminal will show the results of the test. For example:
assert: YOLO5 Classification:numDetects expected 11 but got 10 assert: YOLO5 Classification:preview expected rmse 0.1 bit got 0.16262965760694073 exit_tests... test summary: 0 out of 2 pass
Each assert you do in your test will result in saving a file to a subfolder "actual_results", for example, in the assembly_line example described above, the actual result files will be in the subfolder "tests/assembly_line/actual_results"
The format of the JSON schema will is as an array of tasks, for example:
{
"tasks": [
{
"name": "showMessage",
"message": {
"en_US": "First, start with the Yolo demo"
},
"delay": 1000
},
{
"name": "assertEquals",
"node_name": "YOLO5 Classification",
"output_port": "numDetects",
"expected_value": 10
},
{
"name": "assertEquals",
"node_name": "YOLO5 Classification",
"output_port": "preview",
"expected_rmse": 0.1
},
{
"name": "exitTests",
"delay": 0
}
],
"workflow": "${demos}/Inference/Yolov5/yolov5.json",
"on_error": "exit",
"verbose": true
}
If you want to start the test by loading a workflow then include the "workflow" attribute that should be set to the workflow that will be loaded at the start of testing
If you want to exit testing if an assertion error occurs, then set the "on_error" attribute to "exit", otherwise set it to "continue" and it will continue further processing of the test even after an error has occurred.
Set the verbose attribute to true to see the results of the outputs of the testing in the terminal.
All testing should end with the task "exitTasks". The tasks allow you to do all the functionality you as the user can do with your mouse and keyboard - instead it is driven by your tasks in your script. The tasks are described as follows
| Name | Description | Other Attributes | Example | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| showMessage | Shows a message in the bottom tooltip window |
|
{
"name": "showMessage",
"message": "First, start with the Yolo demo",
"delay": 1000
}
|
||||||||||||||||||
| hideMessage | Hides a message in the bottom tooltip window |
|
{
"name": "hideMessage",
"delay": 1000
}
|
||||||||||||||||||
| printOutput | Prints the value of a node's output to the info dialog |
|
{
"name": "printOutput",
"node_name": "Yolo",
"output_port" : "out",
"delay": 1000
}
|
||||||||||||||||||
| assertEquals | Asserts a node output value will be expected to be some value The actual result will be saved in the actual_results folder and compared against the file in the expected_results |
|
{
"name": "assertEquals",
"node_name": "Classification",
"output_port" : "numDetects",
"delay": 1000
}
In this case the file comparison will be "Classification_numDetects.png" For matrix2D output types, the file is saved in csv format otherwise all other cases it is saved in a text (.txt) file" If you specify the "prefix" attribute, then the file name is prefixed with this value |
||||||||||||||||||
| saveOutputImage | Saves the output image for a node's output |
|
{
"name": "saveOutputImage",
"node_name": "Yolo",
"output_port" : "preview",
"path" : "output.png",
"delay": 1000
}
|
||||||||||||||||||
| saveOutputMatrix | Saves the output matrix for a node's output |
|
{
"name": "saveOutputMatrix",
"node_name": "Yolo",
"output_port" : "out",
"path" : "output.png",
"delay": 1000
}
|
||||||||||||||||||
| exitTests | Exits INFINIWORKFLOW and prints the test summary in the console |
|
{
"name": "exitTests"
}
|
||||||||||||||||||
| addNode | Adds a node from the Toolbox to the workflow |
|
{
"name": "addNode",
"tool_id": "cv.movie_reader",
"mandatory_params": [
"$\{assets\}/city.mp4"
],
"node_name": "Movie Reader",
"duration": 180,
"delay": 1000
}
|
||||||||||||||||||
| delay | Wait a delay before proceeding to next task |
|
{
"name": "delay",
"amount": 180,
"delay": 1000
}
|
||||||||||||||||||
| addLink | Adds a link from the output of one node to the input of another node |
|
{
"name": "addLink",
"from_name": "Movie Reader",
"from_port": "out",
"to_name": "Yolo Classifier",
"to_port": "source",
"duration": 300,
"delay": 1000
}
|
||||||||||||||||||
| removeLink | Remove a link from the output of one node to the input of another node |
|
{
"name": "removeLink",
"from_name": "Movie Reader",
"from_port": "out",
"to_name": "Yolo Classifier",
"to_port": "source",
"duration": 300,
"delay": 1000
}
|
||||||||||||||||||
| openViewer | Displays the node's output in the viewer |
|
{
"name": "openViewer",
"node_name": "Movie Reader",
"output_name": "out",
"duration": 300,
"delay": 1000
}
|
||||||||||||||||||
| closeViewer | Closes the viewer |
|
{
"name": "closeViewer",
"delay": 1000
}
|
||||||||||||||||||
| insertInput | Inserts an input to a node |
|
{
"name": "insertInput",
"node_name": "And",
"duration": 300,
"delay": 1000
}
|
||||||||||||||||||
| removeInput | Removes an input to a node |
|
{
"name": "removeInput",
"node_name": "And",
"duration": 300,
"delay": 1000
}
|
||||||||||||||||||
| openEditor | Displays the node's output in the editor |
|
{
"name": "openEditor",
"node_name": "Movie Reader",
"duration": 300,
"delay": 1000
}
|
||||||||||||||||||
| closeEditor | Closes the editor |
|
{
"name": "closeEditor",
"delay": 1000
}
|
||||||||||||||||||
| editParameter | Changes the value of the input of an edited node |
|
{
"name": "editParameter",
"node_name": "Yolo Classifier",
"input_name": "filter",
"value": "car",
"duration": 300,
"delay": 5000
}
|
||||||||||||||||||
| openPointOverlay | Opens the edited input parameter point in the overlay |
|
{
"name": "openPointOverlay",
"node_name": "Tracker Inference",
"input_name": "center",
"duration": 300,
"delay": 1000
}
|
||||||||||||||||||
| clickTrigger | Clicks the trigger button of the edited input parameter |
|
{
"name": "clickTrigger",
"node_name": "Tracker Inference",
"input_name": "start_stop",
"duration": 200,
"delay": 1000
}
|
||||||||||||||||||
| openRenderStatus | Opens the render status window |
|
{
"name": "openRenderStatus",
"duration": 200,
"delay": 1000
}
|
||||||||||||||||||
| closeRenderStatus | Closes the render status window |
|
{
"name": "closeRenderStatus",
"duration": 200,
"delay": 1000
}
|
||||||||||||||||||
| abortRenderStatus | Aborts the render in the render status window |
|
{
"name": "abortRenderStatus",
"duration": 200,
"delay": 1000
}
|
||||||||||||||||||
| nextVisualization | Goes to the next visualization of the matrix in the viewer |
|
{
"name": "nextVisualization",
"duration": 200,
"delay": 1000
}
|
||||||||||||||||||
| importWorkflow | Goes to the next visualization of the matrix in the viewer |
|
{
"name": "importWorkflow",
"workflow": "${demos}/PyTorch/CIFAR Classification/CIFAR Test/cifar test.json",
"delay": 1000
}
|
||||||||||||||||||
| clearWorkflow | Clears the workflow |
|
{
"name": "clearWorkflow",
"delay": 1000
}
|
||||||||||||||||||
| zoomFit | Zooms the workflow viewport around the selected nodes |
|
{
"name": "zoomFit",
"delay": 1000
}
|
||||||||||||||||||
| selectNode | Selects a node in the workflow |
|
{
"name": "selectNode",
"node_name" : "Add",
"delay": 1000
}
|
||||||||||||||||||
| togglePlay | Toggles the playback between paused and playing |
|
{
"name": "togglePlay",
"duration": 1000,
"delay": 2000
}
|
||||||||||||||||||
| firstFrame | Goes to the first frame |
|
{
"name": "firstFrame",
"duration": 1000,
"delay": 2000
}
|
||||||||||||||||||
| previousFrame | Goes to the prior frame |
|
{
"name": "previousFrame",
"duration": 1000,
"delay": 2000
}
|
||||||||||||||||||
| nextFrame | Goes to the next frame |
|
{
"name": "previousFrame",
"duration": 1000,
"delay": 2000
}
|
||||||||||||||||||
| lastFrame | Goes to the last frame |
|
{
"name": "lastFrame",
"duration": 1000,
"delay": 2000
}
|
||||||||||||||||||
| setCurrentFrame | Sets the current frame |
|
{
"name": "setCurrentFrame",
"frame": 10",
"duration": 1000,
"delay": 2000
}
|
||||||||||||||||||
| click | Clicks on a HTML element |
|
{
"name": "click",
"id": "#publish-workflow",
"duration": 1000,
"delay": 2000
}
|
||||||||||||||||||
| pan | Pans the viewport of the workflow |
|
{
"name": "pan",
"dx": 0,
"dy": 200,
"delay": 100
}
|
||||||||||||||||||
| openHyperparameters | Opens the Hyperparameter dialog for a node |
|
{
"name": "openHyperparameters",
"node_name" : "Logistic Regression",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| closeHyperparameters | Closes the Hyperparameter dialog |
|
{
"name": "closeHyperparameters",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| startGridSearch | Starts a grid search for a node |
|
{
"name": "startGridSearch",
"node_name" : "R2 Score",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| openGridSearch | Opens the Grid Search dialog |
|
{
"name": "openGridSearch",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| closeGridSearch | Closes the Grid Search dialog |
|
{
"name": "closeGridSearch",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| optimizeGridSearch | Selects to optimize in the Grid Search dialog |
|
{
"name": "optimizeGridSearch",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| openCreateMacro | Opens the Create Macro Dialog |
|
{
"name": "openCreateMacro",
"node_name" : "Sequential",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| setMacroName | Sets the Macro name in the Create Macro Dialog |
|
{
"name": "macroName",
"macroName" : "MY CIFAR",
"delay": 100
}
|
||||||||||||||||||
| closeCreateMacro | Closes the Create Macro Dialog and creates the macro |
|
{
"name": "closeCreateMacro",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| mergeCpuThreads | Merges the nodes into same CPU thread |
|
{
"name": "mergeCpuThreads",
"node_name" : "Canny",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| splitCpuThreads | Splits the nodes into into different CPU threads |
|
{
"name": "splitCpuThreads",
"node_name" : "Canny",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| setCudaDevice | Sets the CUDA device for a node |
|
{
"name": "setCudaDevice",
"node_name" : "Brightness",
"duration": 200,
"delay": 100
}
|
||||||||||||||||||
| escapeToClose | Press the escape key to close any open dialog |
|
{
"name": "escapeToClose",
"delay": 100
}
|
||||||||||||||||||
| showImageUrl | Shows an image in a popup window |
|
{
"name": "showImageUrl",
"url": "https://www.mdpi.com/sensors/sensors-19-04933/article_deploy/html/images/sensors-19-04933-g001.png",
"duration": 4000,
"width": 800,
"height": 800,
"delay": 1000
}
|
||||||||||||||||||
| showWebpage | Shows an webpage in a popup iframe |
|
{
"name": "showWebpage",
"url": "https://docs.opencv.org/4.x/da/d22/tutorial_py_canny.html",
"duration": 4000,
"width": 800,
"height": 800,
"delay": 1000
}
|
Ensure lens and tripod are mounted, then connect the INFINICAM via high-quality USB 3.1 Type-C cable.
Download the INFINICAM SDK, unzip, right-click phrtcam.inf, and select Install.
Confirm "Photron USB3.0 Real Time Camera" is detected in Device Manager.
Get infiniworkflow_v1_0.zip from the Photron dev portal and unzip to your Documents folder.
Execute install.bat (Windows) or sudo sh install.sh (Linux/OSX) with admin privileges.
Allow the installer to finish configuring Python modules and Firewall access (5–30 min).
Run INFINIWORKFLOW.bat or sh INFINIWORKFLOW.sh to start the local server.
Use Chrome (131.0+) to open the local interface URL.
Click the HUB icon to view system diagnostics and available tools.
Select a "Live Tutorial" from the Help menu to begin.
Open the Photron category and drag the INFINICAM node onto your workspace.
Search for Canny Edge in the toolbox and connect its input to the INFINICAM node's output port.
Drop View and Edit nodes onto the canvas to visualize and refine your pipeline.
Check the full documentation for deep dives into detection parameters and data saving configurations.
View Infinicam Section →