Media Handle Management
PUBLISHED
Your application can manage various media handles with media tools.
The main features of the Media Tool API include:
-
Creating the media format handle
You can set and get video or audio information using the
media_format_h
handle. -
Creating the media packet handle
You can set and get metadata using the created
media_packet_h
handle. Somemedia_packet_h
metadata are NOT filled after creating themedia_packet_h
handle: pts, dts, duration, extradata, codec data, codec data size, and buffer flags.
The media_format_h
handle is created by the caller, who can set and get the video or audio information. The media_format_h
handle creates the media_packet_h
handle and allocates the buffer. The caller can set and get the metadata with the media_packet_h
handle.
The media_format_h
handle has a specific design for increasing and decreasing its reference count.
Prerequisites
To enable your application to use the media tool functionality:
-
To use the media format handle of the Media Tool API (in mobile [1] and wearable [2] applications), include the
<media_format.h>
header file in your application:#include <media_format.h>
-
To use the media packet handle of the Media Tool API, include the
<media_packet.h>
header file in your application:#include <media_packet.h>
Managing the Media Format Handle
To manage the media format handle:
-
Define a
media_format_h
variable for the media format handle, and pass the variable to themedia_format_create()
function, which returns the handle.To set the video information when creating the handle:
media_format_h format; if (media_format_create(&format) == MEDIA_FORMAT_ERROR_NONE) { media_format_set_video_mime(format, MEDIA_FORMAT_H264_HP); media_format_set_video_width(format, 640); media_format_set_video_height(format, 480); media_format_set_video_avg_bps(format, 10000000); media_format_set_video_max_bps(format, 15000000); } else { printf("media_format_create() failed!"); }
To set the audio information, replace the
media_format_set_video_XXX()
functions with the relevantmedia_format_set_audio_XXX()
functions in the above example code. -
To retrieve the video format information, use the
media_format_get_video_info()
function:media_format_h fmt; media_format_mimetype_e mime; int w; int h; int avg_bps; int max_bps; if (media_format_get_video_info(fmt, &mimetype, &w, &h, &avg_bps, &max_bps) == MEDIA_PACKET_ERROR_NONE) printf("media_format_get_video_info success! width = %d, height = %d", w, h); else print("media_format_get_video failed...");
To retrieve the audio format information, use the
media_format_get_audio_info()
function.
Managing the Media Packet Handle
To manage the media packet handle:
-
Define a
media_packet_h
variable for the media packet handle, and pass the variable to the appropriatemedia_packet_create_XXX()
function (as the last parameter) with an existing media format handle (as the first parameter).After creating the media packet handle, call the
media_format_unref()
function for the media format handle. Because all functions that create a media packet handle increase the reference count of the media format handle, you must decrease the count.To reuse the media packet handle even after the
media_packet_destroy()
function is called, define a callback (the third last parameter of themedia_packet_create_XXX()
function), which is called when the handle is destroyed. Set the callback to returnMEDIA_PACKET_REUSE
.The following example codes show the different ways you create the media packet handle:
-
To create the handle and allocate a buffer into the heap or TBM surface, use the
media_packet_create_alloc()
function:{ media_format_h fmt; media_packet_h packet; media_format_create(&fmt); media_format_set_video_mime(format, MEDIA_FORMAT_H264_HP); media_format_set_video_width(format, 640); media_format_set_video_height(format, 480); media_format_set_video_avg_bps(format, 10000000); media_format_set_video_max_bps(format, 15000000); /* MEDIA_FORMAT_H264_HP data type is MEDIA_FORMAT_ENCODED and the buffer is allocated into the heap If the data type is MEDIA_FORMAT_RAW, the buffer is allocated into the TBM surface */ media_packet_create_alloc(fmt, _finalize_callback, fcb_data, &packet); media_format_unref(fmt); media_packet_destroy(packet); } int _finalize_callback(media_packet_h packet, int err, void* userdata) { return MEDIA_PACKET_REUSE; }
-
To create only the handle, use the
media_packet_create()
function:{ media_format_h fmt; media_packet_h packet; media_format_create(&fmt); media_format_set_video_mime(format, MEDIA_FORMAT_H264_HP); media_format_set_video_width(format, 640); media_format_set_video_height(format, 480); media_format_set_video_avg_bps(format, 10000000); media_format_set_video_max_bps(format, 15000000); /* Only create the handle, do not allocate a buffer */ media_packet_create(fmt, _finalize_callback, fcb_data, &packet); media_format_unref(fmt); } int _finalize_callback(media_packet_h packet, int err, void* userdata) { return MEDIA_PACKET_FINALIZE; }
-
To create the handle and store the TBM surface data, use the
media_packet_create_from_tbm_surface()
function:{ media_format_h fmt; media_packet_h packet; media_format_create(&fmt); media_format_set_video_mime(format, MEDIA_FORMAT_RGBA); media_format_set_video_width(format, 128); media_format_set_video_height(format, 128); media_format_set_video_avg_bps(format, 10000000); media_format_set_video_avg_bps(format, 15000000); media_packet_create_from_tbm_surface(fmt, surface, _finalize_callback, fcb_data, &packet); media_format_unref(fmt); } int _finalize_callback(media_packet_h packet, int err, void* userdata) { return MEDIA_PACKET_FINALIZE; }
-
To create the handle with an already allocated external buffer, use the
media_packet_create_from_external_memory()
function:{ media_format_h fmt; media_packet_h packet; media_format_create(&fmt); media_format_set_video_mime(format, MEDIA_FORMAT_H264_HP); media_format_set_video_width(format, 640); media_format_set_video_height(format, 480); media_format_set_video_avg_bps(format, 10000000); media_format_set_video_avg_bps(format, 15000000); media_packet_create_from_external_memory(fmt, mem_ptr, size, _finalize_callback, fcb_data, &packet); media_format_unref(fmt); } int _finalize_callback(media_packet_h packet, int err, void* userdata) { /* Do something */ return MEDIA_PACKET_FINALIZE; }
-
-
Set and get the metadata with the media packet handle:
int ret = MEDIA_PACKET_ERROR_NONE; /* format1 already exists */ media_packet_create_alloc(format1, NULL, NULL, &packet); ret = media_packet_set_duration(packet, duration); /* After media_packet_get_format(), use media_format_unref() */ media_format_h tmp; media_packet_get_format(packet, &tmp); media_format_unref(tmp); /* Set previously created format2 to packet */ /* After media_packet_set_format(), use media_format_unref() */ media_packet_set_format(packet, format2); /* Packet format is format2 If format2 ref_count is 1, format2 is free If format2 ref_count is bigger than 1, it is not free */ media_packet_destroy(packet);
Reference Count Design
The following table describes the reference count design of the media_format_h
handle.
Table: Media format handle reference count design
Function | Reference count number | Description |
---|---|---|
media_format_h fmt1, fmt2, tmp; media_packet_h pkt1, pkt2; media_format_create(&fmt1); |
fmt1 : 1 |
Define the media_format_h and media_packet_h handles.
Create the |
media_packet_create(&pkt1, fmt1); |
fmt1 : 2 |
After the media_packet_create() function, you must use the media_format_unref() function, because the media_packet_create() function increases the media_format_h reference count. |
media_format_unref(fmt1); |
fmt1 : 1 |
If the ref_count is 1, the fmt1 is currently owned by the pkt1 only. |
media_packet_copy(pkt1, &pkt2); |
fmt1 : 2 |
Copy the pkt1 metadata to pkt2 , except the allocated buffer and buffer size. pkt2 refers to fmt1 , and fmt1 ref_count is increased. |
media_packet_get_format(pkt1, &tmp); |
fmt1 : 3 |
fmt1 ref_count is increased by the media_packet_get_format() function. |
media_format_set_video_mime(tmp, ...); |
fmt1 : 3 |
If you try to modify the fmt1 data (call the media_format_set_video_mime() function) for fmt1 (=tmp ), the ref_count is bigger than 1, and fmt1 cannot be modified.
To modify the |
media_format_make_writable(tmp, &fmt2); |
fmt1 : 2fmt2 : 1 |
If called, the tmp (fmt1 ) ref_count is decreased. Creates the fmt2 handle and copies the fmt1 data to fmt2 . |
media_format_set_video_mime(fmt2, ...); |
fmt1 : 2fmt2 : 1 |
fmt2 ref_count is 1, which means that you can modify the fmt2 data. |
media_packet_set_format(pkt2, fmt2); |
fmt1 : 2fmt2 : 2 |
Set the modified fmt2 to the pkt2 handle. You must call the media_format_unref(fmt2) function. |
media_format_unref(tmp); |
fmt1 : 1fmt2 : 2 |
You must call this function because of the media_packet_get_format(pkt1, &tmp) function call. |
media_format_unref(fmt2); |
fmt1 : 1fmt2 : 1 |
You must call this function because of the media_packet_set_format(pkt2, fmt2) function call. |
media_packet_destroy(pkt1); |
fmt1 : 1 > finalizefmt2 : 1 |
If you destroy the pkt1 handle, the fmt1 ref_count is decreased. If the ref_count becomes 0, fmt1 is freed. |
media_packet_destroy(pkt2); |
fmt1 : 1 > finalizefmt2 : 0 > finalize |
If you destroy the pkt2 handle, the fmt2 ref_count is decreased. If the ref_count becomes 0, fmt2 is freed. |