Audio I/O: Recording from the Audio Device and Playing Raw Audio Data

This tutorial demonstrates how you can control audio input and output to record, and play an audio sample.

Warm-up

Become familiar with the Audio I/O and Sound Manager API basics by learning about:

Initializing the Audio Devices
Initialize the audio input and output devices for use.
Managing Simple Recording and Playback
Record an audio sample from the audio input device, play the recorded audio sample through the audio output device, and modify the volume in the recorded audio sample using a simple API.
Managing Asynchronous Recording and Playback
Record an audio sample from the audio input device, play the recorded audio sample through the audio output device, and modify the volume in the recorded audio sample using a low-latency API.
Releasing Resources
Finish all audio device operations.

Initializing the Audio Devices

To initialize the audio devices for use:

To use the functions and data types of the Audio I/O (in mobile and wearable applications) and Sound Manager (in mobile and wearable applications) APIs, include the <audio_io.h> and <sound_manager.h> header files in your application:
```
#include <audio_io.h>
#include <sound_manager.h>
```
To initialize the audio input and output devices, use the audio_in_create() and audio_out_create() functions:
```
// Define the sample rate for the recording
#define SAMPLE_RATE 44100

// Declare the variable used for checking function result
audio_io_error_e error_code;

// Initialize the audio input device
audio_in_h input;

error_code = audio_in_create(SAMPLE_RATE, AUDIO_CHANNEL_MONO, AUDIO_SAMPLE_TYPE_S16_LE, &input);

// Initialize the audio output device
audio_out_h output;

error_code = audio_out_create(SAMPLE_RATE, AUDIO_CHANNEL_MONO, AUDIO_SAMPLE_TYPE_S16_LE, SOUND_TYPE_SYSTEM, &output);
```
The audio input and output devices support the channel types defined in the audio_channel_e enumeration (in mobile and wearable applications), and the sample types defined in the audio_sample_type_e enumeration (in mobile and wearable applications). For playing the recorded audio, set the same channel and sample type for both devices.

The sound types are defined in the audio_channel_e enumeration. You can select the sound type according to the audio sample type.

Managing Simple Recording and Playback

Creating an Audio Buffer

Before starting the synchronous recording/playback process, you need to allocate the buffer that is used for storing the captured audio data. The buffer is allocated with the malloc() function, so the desired buffer size must be known first. You can get that size by using one of the following options:

Get the buffer size recommended by the Audio I/O API:
1. Use the audio_in_get_buffer_size() function to get the buffer size recommended by the sound server (such as PulseAudio):
```
int buffer_size;

error_code = audio_in_get_buffer_size(input, &buffer_size);
```
  The buffer_size parameter returns the recommended size of the buffer based on the specified audio parameters.
2. The size returned by the audio_in_get_buffer_size() function depends on the device (it can be different for TV, mobile, and wearable). The synchronous recording process ends when the buffer is full. To determine the duration of the recording, set the buffer size corresponding to the desired duration.
  
  For example, this function returns the size for around 100 milliseconds for the device used for creation of this tutorial. In such case, multiply the obtained buffer size by 10 (to change the units to seconds) and by the number of seconds that the recording lasts:
```
#define RECORDING_SEC 5

buffer_size *= 10 * RECORDING_SEC;
```
Explicitly calculate the buffer size:
1. Retrieve the audio channel type information using the audio_in_get_channel() function:
```
audio_channel_e channel;

error_code = audio_in_get_channel(input, &channel);
```
2. Retrieve the audio sample type information using the audio_in_get_sample_type() function:
```
error_code = audio_in_get_sample_type(input, &sample_type);
```
3. Calculate the buffer size based on the retrieved information:
```
int buffer_size = SAMPLE_RATE * (channel == AUDIO_CHANNEL_STEREO ? 2 : 1) * (sample_type == AUDIO_SAMPLE_TYPE_S16_LE ? 2 : 1);
```
4. Multiply the buffer size by the number of seconds that the recording takes:
```
buffer_size *= RECORDING_SEC;
```

After determining the buffer size, allocate the memory for the buffer using the malloc() function:

void *buffer = malloc(buffer_size);

Now, when the local buffer for recording/playback is created, you can start the recording/playback process.

Recording and Playing an Audio Sample

The synchronous recording/playback process is blocking. It means that launching a recording or playback process from the main thread of the application can make it unresponsive. To prevent this, launch the recording/playback in a different thread. This use case uses the ecore_thread_run() function for that purpose.

ecore_thread_run(synchronous_playback, NULL, NULL, NULL);

Now, inside the synchronous_playback() function, you can safely run the recording/playback process:

To start capturing the audio from the H/W device, prepare the audio input device. After using the audio_in_prepare() function, the device starts buffering (storing in an internal buffer) the captured audio.
```
error_code = audio_in_prepare(input);
```
To obtain the captured audio data from the internal buffer, use the audio_in_read() function:
```
// Copy recorded data from the input buffer to the local buffer
int bytes_number = audio_in_read(input, buffer, buffer_size);
```
The returned value must be the number of bytes read from the input buffer. However, if the value is negative, it represents an error code.

This function can behaves in 2 different ways:
- When it is called immediately after calling the audio_in_prepare() function, it blocks the thread it is launched from until the buffer is fulfilled with the captured audio data.
- When it is called with a delay long enough to let the internal buffer store more data than the buffer size indicates, the function executes immediately, without blocking its thread.
The audio_in_read() function fulfills the local buffer by copying the data from the internal buffer. If not enough audio data was captured yet (not enough data is stored in the internal buffer), the function waits until enough data is captured to fulfill the whole local buffer. It is important to understand this mechanism. If you want to start recording audio after clicking a button, call the audio_in_prepare() function just before the audio_in_read() function (in the same button callback). Otherwise, if you prepare the device earlier and run only the reading function inside the button callback, your recording buffer is fulfilled with the audio data recorded earlier (before the button was clicked).
To stop the hardware recording process (buffering) after reading the data, use the audio_in_unprepare() function:
```
error_code = audio_in_unprepare(input);
```
Playing the recorded audio sample follows the same rules as the recording process.

To play the recorded audio sample from the buffer, prepare the audio output device. After using the audio_out_prepare() function, the device prepares its internal output buffer for the playback.
```
error_code = audio_out_prepare(output);
```
The audio sample playback process is launched by using the audio_out_write() function:
```
// Copy the recorded data from the local buffer to the output buffer
int bytes_number = audio_out_write(output, buffer, buffer_size);
```
The returned value must be the number of bytes read from the input buffer. However, if the value is negative, it represents an error code.
To stop the hardware playback process (buffering) after writing the data to the output buffer, use the audio_out_unprepare() function:
```
error_code = audio_out_unprepare(output);
```

Modifying the Audio Sample Volume

To modify the volume of the audio sample stored in the local buffer:

#define MIN_2BYTES_SIGNED (−32768)
#define MAX_2BYTES_SIGNED 32767

void modify_sound()
{
   // Get the sample type of the input
   audio_sample_type_e sample_type;

   int error_code = audio_in_get_sample_type(input, &sample_type);
   if (error_code != AUDIO_IO_ERROR_NONE)
   {
      dlog_print(DLOG_ERROR, LOG_TAG, "audio_in_get_sample_type() failed! Error code = %d", error_code);

      return;
   }

   uint8_t *index = (uint8_t*)buffer;
   while (index < (((uint8_t*)buffer)+buffer_size))
   {
      if (AUDIO_SAMPLE_TYPE_S16_LE == sample_type)
      {
         // Use int16_t type, because it is 2 bytes long
         int16_t *value = (int16_t*)index;

         // Make the sample louder
         int32_t tmp = (*value) * 8; // Why not 8 times louder? (on dB scale even much louder)
         if (tmp > MAX_2BYTES_SIGNED) tmp = MAX_2BYTES_SIGNED;
         if (tmp < MIN_2BYTES_SIGNED) tmp = MIN_2BYTES_SIGNED;
         (*value) = tmp;
      } 
      else 
      {
         // Use uint8_t type, because it is 1 byte long
         uint8_t *value = (uint8_t*)index;

         // Make the sample louder
         uint16_t tmp = (*value) * 8; // Why not 8 times louder? (on dB scale even much louder)
         if (tmp > 255) tmp = 255;
         (*value) = tmp;
      }

      // Go to the next sample
      index += sample_type == AUDIO_SAMPLE_TYPE_S16_LE? 2 : 1;
   }

   dlog_print(DLOG_DEBUG, LOG_TAG, "Volume of the synchronous recording increased.");
}

In this example, the volume is significantly increased. You can also make other modifications to the audio sample playback.

Managing Asynchronous Recording and Playback

Starting the Asynchronous Recording

The asynchronous recording process uses callback functions for capturing audio samples.

After initialization of the audio input device, proper callback function must be set using the audio_in_set_stream_cb() function. Use this function before calling the audio_in_prepare() function, because otherwise the callback function is not be called.

// Set a callback function that is called asynchronously for every single part of the captured voice data
error_code = audio_in_set_stream_cb(input, _audio_io_stream_read_cb, NULL);

After this step, call the audio_in_prepare() function and start capturing the audio. However, in this case, to store the recording in a file instead of using local buffer, you must prepare (create and open) a file the recording is stored in:

#include <storage.h>

// Prepare a file, where the recorded data is stored
char io_stream_w_path[200];
char *storage_path;
// storage_id can be found using the storage_foreach_device_supported() function
int error = storage_get_directory(storage_id, STORAGE_DIRECTORY_SOUNDS, &storage_path);  
snprintf(io_stream_w_path, 200, "%s/%s", storage_path, "pcm_w.raw");
free(storage_path);

FILE* fp_w = fopen(io_stream_w_path, "w");
if (!fp_w) 
{
   dlog_print(DLOG_ERROR, LOG_TAG, "fopen() function failed while opening %s file!", io_stream_w_path);
}

To obtain the storage path, use the Storage API (in mobile and wearable applications) instead of direct access.

The file is ready and the callback function is set. Proceed with the asynchronous recording by calling the audio_in_prepare() function to initiate the hardware recording process:

error_code = audio_in_prepare(input);

From now on, the callback function is invoked respectively for every captured audio sample pack.

Inside the callback function, you retrieve a handle to the input device and the number of captured audio sample bytes using the audio_in_peek() function. This information can be used to extract the data from the input internal buffer and store it in the file. The audio data stored in the obtained buffer is written to the recording file (the one that was prepared earlier) with the fwrite() function, and after storing the received audio sample pack in the file, it can be removed from the internal buffer using the audio_in_drop() function.

void _audio_io_stream_read_cb(audio_in_h handle, size_t nbytes, void *userdata)
{
   const void * buffer = NULL;

   if (nbytes > 0)
   {
      // Retrieve buffer pointer from audio input buffer
      int error_code = audio_in_peek(handle, &buffer, &nbytes);
      if (error_code != AUDIO_IO_ERROR_NONE)
      {
         dlog_print(DLOG_ERROR, LOG_TAG, "audio_in_peek() failed! Error code = %d", error_code);

         return;
      }

      // Store the recorded part in the file
      fwrite(buffer, sizeof(char), nbytes, fp_w);

      // Remove the obtained audio input data from the actual stream buffer
      error_code = audio_in_drop(handle);
      if (error_code != AUDIO_IO_ERROR_NONE)
      {
         dlog_print(DLOG_ERROR, LOG_TAG, "audio_in_drop() failed! Error code = %d", error_code);
      }
   }
}

Stopping the Asynchronous Recording

It is possible to stop the asynchronous recording process manually:

The audio_in_unprepare() function stops the hardware recording process, so the callback for asynchronous recording is no longer called:
```
error_code = audio_in_unprepare(input);
```
If the asynchronous recording is no longer used, or if the audio_in_set_stream_cb() function is called each time before starting the asynchronous recording, you can unset the callback function with the audio_in_unset_stream_cb() function:
```
error_code = audio_in_unset_stream_cb(input);
```
The file used for recording is still opened, so if the recording was stopped, the file can be closed:
```
error_code = fclose(fp_w);
```

Starting the Asynchronous Playback

Playing the audio sample asynchronously is similar to the recording:

The callback function used for asynchronous playback is set with the audio_out_set_stream_cb() function:

// Set a callback function that is called asynchronously for every single part of the stored audio data
error_code = audio_out_set_stream_cb(output, _audio_io_stream_write_cb, NULL);

After this step, you can call the audio_out_prepare() function and start playing the audio. However, in this case, to play the recording from a file instead of using local buffer, you must prepare (open for reading) a file the recording is stored in:

#include <storage.h>

char io_stream_r_path[200];
char *storage_path;
// storage_id can be found using the storage_foreach_device_supported() function
int error = storage_get_directory(storage_id, STORAGE_DIRECTORY_SOUNDS, &storage_path);  
snprintf(io_stream_r_path, 200, "%s/%s", storage_path, "pcm_w.raw");
free(storage_path);

FILE* fp_r = fopen(io_stream_r_path, "r");

The file is ready and the callback function is set. Proceed with the asynchronous playback by calling the audio_out_prepare() function to initiate the hardware playback process:
```
error_code = audio_out_prepare(output);
```

From now on, the callback function is invoked respectively for every audio sample pack retrieved from the file.

Inside the callback, you retrieve a handle to the output internal buffer and the number of audio sample bytes that can be written to the output buffer using the malloc() and memset() functions. Read the pack of the audio sample from the file and store it in the local buffer using the fread() function, and copy the audio sample data from the local buffer to the output internal buffer to start the playback with the audio_out_write() function.

void _audio_io_stream_write_cb(audio_out_h handle, size_t nbytes, void *userdata)
{
   char * buffer = NULL;

   if (nbytes > 0)
   {
      buffer = malloc(nbytes);
      memset(buffer, 0, nbytes);

      // Play the following part of the recording
      fread(buffer, sizeof(char), nbytes, fp_r);

      // Copy the recorded data from the buffer to the output buffer
      int data_size = audio_out_write(handle, buffer, nbytes);

      if (data_size < 0)
      {
         dlog_print(DLOG_ERROR, LOG_TAG, "audio_out_write() failed! Error code = %d", data_size);
      }

      free (buffer);
   }
}

Stopping the Asynchronous Playback

you can stop the asynchronous playback process manually:

The audio_out_unprepare() function stops the hardware playback process, so the callback for asynchronous playback is no longer called:
```
error_code = audio_out_unprepare(output);
```
If the asynchronous playback is no longer used, or if the audio_out_set_stream_cb() function is called each time before starting the asynchronous playback, you can unset the callback function with the audio_out_unset_stream_cb() function when the playback is stopped:
```
error_code = audio_out_unset_stream_cb(output);
```
The file used for playback is still opened, so if the playback was stopped, the file can be closed:
```
error_code = fclose(fp_r);
```

Releasing Resources

To destroy the audio handles and release the allocated resources after you have finished working with the audio input and output devices:

Release the memory allocated to the local buffer using the free() function:
```
free(buffer);
```

Unprepare the audio input and output devices using the audio_in_unprepare() and audio_out_unprepare() functions:

// Stop the hardware recording process
error_code = audio_in_unprepare(input);

// Stop the hardware playback process
error_code = audio_out_unprepare(output);

Unset the callback functions used for asynchronous recording/playback, if they were not unset yet, using the audio_in_unset_stream_cb() and audio_out_unset_stream_cb() functions:

// Unset the callback function used for asynchronous recording process
audio_in_unset_stream_cb(input);

// Unset the callback function used for asynchronous playback process
audio_out_unset_stream_cb(output);

Destroy the audio input and output handles using the audio_in_destroy() and audio_out_destroy() functions:

// Deinitialize audio input device
error_code = audio_in_destroy(input);

// Deinitialize audio output device
error_code = audio_out_destroy(output);

Close opened files, if they were not closed yet, using the fclose() function:
```
fclose(fp_w);
fp_w = NULL;

fclose(fp_r);
fp_r = NULL;
```