Media Codec: Encoding and Decoding Media Files

This tutorial demonstrates how you can encode and decode media files using media codecs.

Warm-up

Become familiar with the Media Codec API basics by learning about:

Initializing Media Codecs
Initialize a media codec for use.
Filling the Media Packet with Data
Fill the media_packet with data.
Running the Media Codec
Run the media codec in a loop.

Initializing Media Codecs

To initialize a media codec for use:

To use the functions and data types of the Media Codec API (in mobile and wearable applications), include the <media_codec.h> header file in your application:
```
#include <media_codec.h>
```
Create a handle for the media codec using the mediacodec_create() function:
```
mediacodec_h *mediacodec;
ret = mediacodec_create(&mediacodec);
```
The handle must be passed to all other Media Codec APIs.
Set the codec using the mediacodec_set_codec() function:
```
ret = mediacodec_set_codec(mediacodec, (mediacodec_codec_type_e)codecid, flag);
```
The mediacodec_codec_type_e (in mobile and wearable applications) and mediacodec_support_type_e (in mobile and wearable applications) enumerations define the media codec type and support type (second and third parameters).

To configure the video and audio encoder and decoder:

// Video encoder
ret = mediacodec_set_venc_info(mediacodec, width, height, fps, target_bits);
// Video decoder
ret = mediacodec_set_vdec_info(mediacodec, width, height);
// Audio encoder
ret = mediacodec_set_aenc_info(mediacodec, samplerate, channel, bit, bitrate);
// Audio decoder
ret = mediacodec_set_adec_info(mediacodec, samplerate, channel, bit);

To set callbacks for the input and output buffers:
1. To receive notifications when the input buffers are used, register a callback using the mediacodec_set_input_buffer_used_cb() function. The callback is invoked when the input buffers are queued to the codec.
```
ret = mediacodec_set_input_buffer_used_cb(mediacodec, _input_buffer_used_cb, NULL);
```
  If a media_packet is used, it must be destroyed when the callback is invoked:
```
static void
_input_buffer_used_cb(media_packet_h pkt, void *user_data)
{
   media_packet_destroy(pkt);

   return;
}
```
2. To receive notifications when the output buffers are dequeued, register a callback using the mediacodec_set_output_buffer_available_cb() function. The callback is invoked when the output buffers are dequeued.
```
ret = mediacodec_set_output_buffer_available_cb(mediacodec, _output_buffer_available_cb, mediacodec);
```
  If the media_packet is dequeued from the codec, this callback is invoked.
  
  Retrieve the output packet using the mediacodec_get_output() function inside the callback:
```
static void
_output_buffer_available_cb(media_packet_h pkt, void *user_data)
{
   media_packet_h out_pkt;
   mediacodec_h mediacodec = (mediacodec_h)user_data;

   if (pkt != NULL)
   {
      mediacodec_get_output(mediacodec, &out_pkt, 0);
      media_packet_destroy(out_pkt);
   }

   return;
}
```

Filling the Media Packet with Data

After the media_packet is allocated with corresponding codec MIME types, fill it with data:

Retrieve the data pointer from the media_packet, and set the buffer size on the preallocated packet:

unsigned char nal[48] = {0x00, 0x00, 0x00, 0x01, 0x67, 0x4D, 0x40, 0x33,
                         0x9A, 0x73, 0x80, 0xA0, 0x08, 0xB4, 0x20, 0x00,
                         0x32, 0xE9, 0xE0, 0x09, 0x89, 0x68, 0x11, 0xE3,
                         0x06, 0x23, 0xC0, 0x00, 0x00, 0x00, 0x01, 0x68,
                         0xEE, 0x3C, 0x80, 0x00, 0x00, 0x00, 0x01, 0x65,
                         0x88, 0x80, 0x01, 0x48, 0x00, 0x06, 0x57, 0xFD};

media_format_h fmt;
media_packet_h pkt;
void *data;

ret = media_format_set_video_mime(fmt, MEDIA_FORMAT_H264_SP);
ret = media_format_set_video_width(fmt, 1280);
ret = media_format_set_video_height(fmt, 544);
ret = media_packet_create_alloc(fmt, NULL, NULL, &pkt);

ret = media_packet_get_buffer_data_ptr(pkt, &data);
memcpy(data, nal, 48);
ret = media_packet_set_buffer_size(pkt, 48);

If the memory buffer contains extra padding bytes after each pixel row, check whether the stride in the uncompressed video frame is the same as the video width. If it is not, make a strided copy:

void
_fill_buffer(media_packet_h pkt, unsigned char *yuv, int width, int height)
{
   int i;

   // Y plane
   media_packet_get_video_stride_width(pkt, 0, &stride);
   media_packet_get_video_plane_data_ptr(pkt, 0, &data);

   for (i = 0; i < height; i++)
   {
      memcpy(data, yuv, width);
      data += stride;
   }
}

Running the Media Codec

After initializing the medic codec and filling the media_packet with data, run the media codec in the following loop:

When an input buffer is ready, read a chunk of input and copy it into the buffer to be encoded or decoded.
When an output buffer is ready, copy the encoded or decoded output from the buffer.

To run the media codec loop:

Prepare the media codec using the mediacodec_prepare() function:
```
ret = mediacodec_prepare(mediacodec);
```
Set the media_packet flags using the media_packet_set_flags() function:
- If the media_packet contains codec data, such as SPS or PPS in case of H.264, set the codec config flag:
```
ret = media_packet_set_flags(pkt, MEDIA_PACKET_CODEC_CONFIG);
```
- If the media_packet contains the end of the stream, set the end-of-stream (eos) flag:
```
ret = media_packet_set_flags(pkt, MEDIA_PACKET_END_OF_STREAM);
```
  The eos callback is invoked if the eos packet is decoded or encoded and the eos callback is registered with the mediacodec_set_eos_cb() function.
Note

You must set the flags before calling the mediacodec_process_input() function.

Note
You must set the flags before calling the mediacodec_process_input() function.

Start the media codec loop using the mediacodec_process_input() and mediacodec_get_output() functions:

media_packet_h in_buf = NULL;
ret = mediacodec_process_input(mediacodec, in_buf, 0);

media_packet_h output_buf = NULL;
ret = mediacodec_get_output(mediacodec, &output_buf, 0);

After calling the mediacodec_get_output() function, check the frame using the media_packet.

To check whether the media_packet contains key frame or codec data:
```
bool keyframe;
bool codec_config;

ret = media_packet_is_sync_frame(pkt, &keyframe);
ret = media_packet_is_codec_config(pkt, &codec_config);
```
After the loop is over and you have finished working with the media codec, reset the codec and destroy the codec handle using the mediacodec_unprepare() and mediacodec_destroy() functions:
```
ret = mediacodec_unprepare(mediacodec);

ret = mediacodec_destroy(mediacodec);
```
The media codec state changes to MEDIACODEC_STATE_NONE.