# onnxinference

This element can apply an ONNX model to video buffers. It attaches the tensor output to the buffer as a ref GstTensorMeta.

To install ONNX on your system, follow the instructions in the README.md in with this plugin.

## Example launch command:

Test image file, model file (SSD) and label file can be found here : https://gitlab.collabora.com/gstreamer/onnx-models

GST_DEBUG=ssdobjectdetector:5
gst-launch-1.0 filesrc location=onnx-models/images/bus.jpg !
jpegdec ! videoconvert ! onnxinference execution-provider=cpu model-file=onnx-models/models/ssd_mobilenet_v1_coco.onnx !
ssdobjectdetector label-file=onnx-models/labels/COCO_classes.txt ! videoconvert ! imagefreeze ! autovideosink

Note: in order for downstream tensor decoders to correctly parse the tensor data in the GstTensorMeta, meta data must be attached to the ONNX model assigning a unique string id to each output layer. These unique string ids and corresponding GQuark ids are currently stored in the tensor decoder's header file, in this case gstssdobjectdetector.h. If the meta data is absent, the pipeline will fail.

As a convenience, there is a python script currently stored at https://gitlab.collabora.com/gstreamer/onnx-models/-/blob/master/scripts/modify_onnx_metadata.py to enable users to easily add and remove meta data from json files. It can also dump the names of all output layers, which can then be used to craft the json meta data file.

## Hierarchy

    GObject
        ╰──GInitiallyUnowned
            ╰──GstObject
                ╰──GstElement
                    ╰──GstBaseTransform
                        ╰──onnxinference

## Factory details

Authors: – Aaron Boxer

Classification: – `Filter/Effect/Video`

Rank – primary

Plugin – onnx

Package – GStreamer Bad Plug-ins

## Pad Templates

### `sink`

    video/x-raw:
             format: { RGB, RGBA, BGR, BGRA }
              width: [ 1, 2147483647 ]
             height: [ 1, 2147483647 ]
          framerate: [ 0/1, 2147483647/1 ]

Presence – always

Direction – sink

Object type – GstPad

### `src`

    video/x-raw:
             format: { RGB, RGBA, BGR, BGRA }
              width: [ 1, 2147483647 ]
             height: [ 1, 2147483647 ]
          framerate: [ 0/1, 2147483647/1 ]

Presence – always

Direction – src

Object type – GstPad

## Properties

### execution-provider

    “execution-provider” GstOnnxExecutionProvider *

ONNX execution provider

Flags : Read / Write

Default value : cpu (0)

Since : 1.24

### input-image-format

    “input-image-format” GstMlInputImageFormat *

Model input image format

Flags : Read / Write

Default value : hwc (0)

Since : 1.24

### input-tensor-offset

    “input-tensor-offset” gfloat

offset each tensor value by this value

Flags : Read / Write

Default value : 0

### input-tensor-scale

    “input-tensor-scale” gfloat

Divide each tensor value by this value

Flags : Read / Write

Default value : 1

### model-file

    “model-file” gchararray

ONNX model file

Flags : Read / Write

Default value : NULL

Since : 1.24

### optimization-level

    “optimization-level” GstOnnxOptimizationLevel *

ONNX optimization level

Flags : Read / Write

Default value : disable-all (0)

Since : 1.24

## Named constants

### GstMlInputImageFormat

GST_ML_INPUT_IMAGE_FORMAT_HWC Height Width Channel (a.k.a. interleaved) format GST_ML_INPUT_IMAGE_FORMAT_CHW Channel Height Width (a.k.a. planar) format

#### Members

`hwc` (0) – Height Width Channel (HWC) a.k.a. interleaved image data format
`chw` (1) – Channel Height Width (CHW) a.k.a. planar image data format

Since : 1.20

### GstOnnxExecutionProvider

#### Members

`cpu` (0) – CPU execution provider
`cuda` (1) – CUDA execution provider

Since : 1.20

### GstOnnxOptimizationLevel

#### Members

`disable-all` (0) – Disable all optimization
`enable-basic` (1) – Enable basic optimizations (redundant node removals))
`enable-extended` (2) – Enable extended optimizations (redundant node removals + node fusions)
`enable-all` (3) – Enable all possible optimizations

Since : 1.20

The results of the search are