# Media Types and Properties

There is a very large set of possible media types that may be used to pass data between elements. Indeed, each new element that is defined may use a new data format (though unless at least one other element recognises that format, it will be useless since nothing will be able to link with it).

In order for media types to be useful, and for systems like autopluggers to work, it is necessary that all elements agree on the media type definitions, and which properties are required for each media type. The GStreamer framework itself simply provides the ability to define media types and parameters, but does not fix the meaning of media types and parameters, and does not enforce standards on the creation of new media types. This is a matter for a policy to decide, not technical systems to enforce.

For now, the policy is simple:

- Do not create a new media type if you could use one which already exists.

- If creating a new media type, discuss it first with the other GStreamer developers, on at least one of: IRC, mailing lists.

- Try to ensure that the name for a new format does not conflict with anything else created already, and is not a more generalised name than it should be. For example: "audio/compressed" would be too generalised a name to represent audio data compressed with an mp3 codec. Instead "audio/mp3" might be an appropriate name, or "audio/compressed" could exist and have a property indicating the type of compression used.

- Ensure that, when you do create a new media type, you specify it clearly, and get it added to the list of known media types so that other developers can use the media type correctly when writing their elements.

## Building a Simple Format for Testing

If you need a new format that has not yet been defined in our List of Defined Types, you will want to have some general guidelines on media type naming, properties and such. A media type would ideally be equivalent to the Mime-type defined by IANA; else, it should be in the form type/x-name, where type is the sort of data this media type handles (audio, video, ...) and name should be something specific for this specific type. Audio and video media types should try to support the general audio/video properties (see the list), and can use their own properties, too. To get an idea of what properties we think are useful, see (again) the list.

Take your time to find the right set of properties for your type. There is no reason to hurry. Also, experimenting with this is generally a good idea. Experience learns that theoretically thought-out types are good, but they still need practical use to assure that they serve their needs. Make sure that your property names do not clash with similar properties used in other types. If they match, make sure they mean the same thing; properties with different types but the same names are not allowed.

## Typefind Functions and Autoplugging

With only defining the types, we're not yet there. In order for a random data file to be recognized and played back as such, we need a way of recognizing their type out of the blue. For this purpose, “typefinding” was introduced. Typefinding is the process of detecting the type of a data stream. Typefinding consists of two separate parts: first, there's an unlimited number of functions that we call typefind functions, which are each able to recognize one or more types from an input stream. Then, secondly, there's a small engine which registers and calls each of those functions. This is the typefind core. On top of this typefind core, you would normally write an autoplugger, which is able to use this type detection system to dynamically build a pipeline around an input stream. Here, we will focus only on typefind functions.

A typefind function usually lives in `gst-plugins-base/gst/typefind/gsttypefindfunctions.c`, unless there's a good reason (like library dependencies) to put it elsewhere. The reason for this centralization is to reduce the number of plugins that need to be loaded in order to detect a stream's type. Below is an example that will recognize AVI files, which start with a “RIFF” tag, then the size of the file and then an “AVI” tag:

    static GstStaticCaps avi_caps = GST_STATIC_CAPS ("video/x-msvideo");
    #define AVI_CAPS gst_static_caps_get(&avi_caps)
    static void
    gst_avi_typefind_function (GstTypeFind *tf,
                  gpointer     pointer)
    {
      guint8 *data = gst_type_find_peek (tf, 0, 12);

      if (data &&
          GUINT32_FROM_LE (&((guint32 *) data)[0]) == GST_MAKE_FOURCC ('R','I','F','F') &&
          GUINT32_FROM_LE (&((guint32 *) data)[2]) == GST_MAKE_FOURCC ('A','V','I',' ')) {
        gst_type_find_suggest (tf, GST_TYPE_FIND_MAXIMUM, AVI_CAPS);
      }
    }

    GST_TYPE_FIND_REGISTER_DEFINE(avi, "video/x-msvideo", GST_RANK_PRIMARY,
        gst_avi_typefind_function, "avi", AVI_CAPS, NULL, NULL);

    static gboolean
    plugin_init (GstPlugin *plugin)
    {
      return GST_TYPEFIND_REGISTER(avi, plugin);
    }

Note that `gst-plugins/gst/typefind/gsttypefindfunctions.c` has some simplification macros to decrease the amount of code. Make good use of those if you want to submit typefinding patches with new typefind functions.

Autoplugging has been discussed in great detail in the Application Development Manual.

## List of Defined Types

Below is a list of all the defined types in GStreamer. They are split up in separate tables for audio, video, container, subtitle and other types, for the sake of readability. Below each table might follow a list of notes that apply to that table. In the definition of each type, we try to follow the types and rules as defined by IANA for as far as possible.

Jump directly to a specific table:

- Table of Audio Types

- Table of Video Types

- Table of Container Types

- Table of Subtitle Types

- Table of Other Types

Note that many of the properties are not required, but rather optional properties. This means that most of these properties can be extracted from the container header, but that - in case the container header does not provide these - they can also be extracted by parsing the stream header or the stream content. The policy is that your element should provide the data that it knows about by only parsing its own content, not another element's content. Example: the AVI header provides samplerate of the contained audio stream in the header. MPEG system streams don't. This means that an AVI stream demuxer would provide samplerate as a property for MPEG audio streams, whereas an MPEG demuxer would not. A decoder needing this data would require a stream parser in between to extract this from the header or calculate it from the stream.

### Table of Audio Types

Table of Audio Types
Media Type | Description
--- | ---
All audio types.
audio/* | All audio types
channels | integer
channel-mask | bitmask
format | string
layout | string
All raw audio types.
audio/x-raw | Unstructured and uncompressed raw audio data.
All encoded audio types.
audio/x-ac3 | AC-3 or A52 audio streams.
audio/x-adpcm | ADPCM Audio streams.
block_align | integer
audio/x-cinepak | Audio as provided in a Cinepak (Quicktime) stream.
audio/x-dv | Audio as provided in a Digital Video stream.
audio/x-flac | Free Lossless Audio codec (FLAC).
audio/x-gsm | Data encoded by the GSM codec.
audio/x-alaw | A-Law Audio.
audio/x-mulaw | Mu-Law Audio.
audio/x-mace | MACE Audio (used in Quicktime).
audio/mpeg | Audio data compressed using the MPEG audio encoding scheme.
framed | boolean
layer | integer
bitrate | integer
audio/x-qdm2 | Data encoded by the QDM version 2 codec.
audio/x-pn-realaudio | Realmedia Audio data.
audio/x-speex | Data encoded by the Speex audio codec
audio/x-vorbis | Vorbis audio data
audio/x-wma | Windows Media Audio
audio/x-paris | Ensoniq PARIS audio
audio/x-svx | Amiga IFF / SVX8 / SV16 audio
audio/x-nist | Sphere NIST audio
audio/x-voc | Sound Blaster VOC audio
audio/x-ircam | Berkeley/IRCAM/CARL audio
audio/x-w64 | Sonic Foundry's 64 bit RIFF/WAV

### Table of Video Types

Table of Video Types
Media Type | Description
--- | ---
All video types.
video/* | All video types
height | integer
framerate | fraction
max-framerate | fraction
views | integer
interlace-mode | string
chroma-site | string
colorimetry | string
pixel-aspect-ratio | fraction
format | string
All raw video types.
video/x-raw | Unstructured and uncompressed raw video data.
All encoded video types.
video/x-3ivx | 3ivx video.
video/x-divx | DivX video.
video/x-dv | Digital Video.
video/x-ffv | FFMpeg video.
video/x-h263 | H-263 video.
h263version | string
video/x-h264 | H-264 video.
video/x-huffyuv | Huffyuv video.
video/x-indeo | Indeo video.
video/x-intel-h263 | H-263 video.
video/x-jpeg | Motion-JPEG video.
video/mpeg | MPEG video.
systemstream | boolean
video/x-msmpeg | Microsoft MPEG-4 video deviations.
video/x-msvideocodec | Microsoft Video 1 (oldish codec).
video/x-pn-realvideo | Realmedia video.
video/x-rle | RLE animation format.
depth | integer
palette_data | GstBuffer
video/x-svq | Sorensen Video.
video/x-tarkin | Tarkin video.
video/x-theora | Theora video.
video/x-vp3 | VP-3 video.
video/x-wmv | Windows Media Video
video/x-xvid | XviD video.
All image types.
image/gif | Graphics Interchange Format.
image/jpeg | Joint Picture Expert Group Image.
image/png | Portable Network Graphics Image.
image/tiff | Tagged Image File Format.

### Table of Container Types

Table of Container Types
Media Type | Description
--- | ---
video/x-ms-asf | Advanced Streaming Format (ASF).
video/x-msvideo | AVI.
video/x-dv | Digital Video.
video/x-matroska | Matroska.
video/mpeg | Motion Pictures Expert Group System Stream.
application/ogg | Ogg.
video/quicktime | Quicktime.
application/vnd.rn-realmedia | RealMedia.
audio/x-wav | WAV.

### Table of Subtitle Types

Table of Subtitle Types
Media Type | Description
--- | ---

### Table of Other Types

Table of Other Types
Media Type | Description
--- | ---

The results of the search are