Voice control elementary: Controlling UI Elements with Voice

This tutorial demonstrates how you can register voice commands for EFL-supported UI components to perform component-related actions, such as button clicks.

This feature is supported in mobile applications only.

Warm-up

Become familiar with the Voice control elementary API basics by learning about:

Initializing the Voice Control
Initialize the Voice control elementary library and create a handle.
Monitoring Language Changes
Set and unset a callback to get notifications when the language changes.
Retrieving Voice Control Information
Get information about supported languages, widgets, and actions.
Managing Commands and Hints
Set and unset commands and hints, and define the hint direction and offset.

Initializing the Voice Control

To initialize the Voice control elementary library and create a handle:

To use the functions and data types of the Voice control elementary API, include the <voice_control_elm.h> header file in your application:
```
#include <voice_control_elm.h>
```

Initialize the Voice control elementary library with the vc_elm_initialize() function:

void 
initialize_voice_control_elm()
{
   int ret;
   ret = vc_elm_initialize();
   if (VC_ELM_ERROR_NONE != ret)
   {
      // Error handling
   }
}

Create a handle for each UI component for which you want to set a command and hint. Use the vc_elm_create_object() function to create a handle for Evas_Objects, and the vc_elm_create_item() function for Elm_Object_Items.

void 
create_vc_elm_handler()
{
   int ret;
   vc_elm_h handler_object;
   vc_elm_h handler_item;
	
   // Creating Evas_Object button
   Evas_Object *button = elm_button_add(win);
   ret = vc_elm_create_object(button, &handler_object);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
   // Creating Elm_Object_Item list
   Elm_Object_Item *item = elm_list_item_append(...);
   ret = vc_elm_create_item(item, &handler_item);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

When voice control is no longer needed, destroy the handle and deinitialize the library:

void 
destroy_vc_elm_handler(vc_elm_h &handler)
{
   int ret;
   ret = vc_elm_destroy(handler);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

void 
deinitialize_voice_control_elm()
{
   int ret;
   ret = vc_elm_deinitialize();
   if (VC_ELM_ERROR_NONE != ret)
   {
      // Error handling
   }
}

Monitoring Language Changes

To monitor when the system or application language changes:

Set the language change callback to be invoked when the system or application language changes.
It is important to track the current language, since you must set the commands and hints accordingly to ensure that the voice commands are recognized correctly.
```
void 
set_language_changed_cb()
{
   int ret;
   ret = vc_elm_set_current_language_changed_cb(language_changed_cb, NULL);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}
```

Define the callback to set the commands and hints according to the current language:

void 
language_changed_cb(const char *previous, const char *current, void* user_data)
{ 
   if ("en_US" == current) 
   {
      vc_elm_set_command(handler, "button");
      vc_elm_set_command_hint(handler, "button");
   }
   else if ("ko_KR" = current) 
   {
      vc_elm_set_command(handler, "버튼");
      vc_elm_set_command_hint(handler, "버튼");
   }
}

When no longer needed, unset the callback:

void 
unset_state_changed_cb()
{
   int ret;
   ret = vc_elm_unset_current_language_changed_cb();
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

Retrieving Voice Control Information

To get information about supported languages, widgets, and actions:

Get the current Voice control elementary language using the vc_elm_get_current_language() function:

void 
get_language()
{
   char *current_language;
   int ret;
   ret = vc_elm_get_current_language(&current_language);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

Get the languages supported by Voice control elementary library with a callback function that runs for each language separately.

The callback cannot be unset: as long as it returns true, it continues to loop over the supported languages.

bool 
supported_language_cb(const char *language, void* user_data)
{
   return true; // To continue to get the next language

   return false; // To stop
}

void 
set_default_language_changed_cb()
{
   int ret;
   ret = vc_elm_foreach_supported_languages(supported_language_cb, NULL);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

Get the UI components supported by Voice control elementary library with a callback function that runs for each UI component separately.

The callback cannot be unset: as long as it returns true, it continues to loop over the supported UI components.

void 
widget_cb(const char *widget, void *user_data)
{
   return true; // To continue to get the next UI component 

   return false; // To stop
}

void 
set_supported_widget_cb()
{
   int ret;
   ret = vc_elm_foreach_supported_widgets(widget_cb, NULL);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

Get the actions supported by Voice control elementary library for specific UI components with a callback function that runs for each action separately.

The callback cannot be unset: as long as it returns true, it continues to loop over the supported actions.

To translate an actions to corresponding command parameters, use the vc_elm_get_action_command() function.

void 
action_cb(const char *action, void *user_data)
{
   char *command_param;
   vc_elm_get_action_command(action, &command_param);

   return true; // To continue to get the next action

   return false; // To stop
}

void 
set_supported_action_cb()
{
   int ret;
   // Elm_Button is an example, this can be obtained using vc_elm_foreach_supported_widgets callback
   ret = vc_elm_foreach_supported_actions("Elm_Button", action_cb, NULL);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

Managing Commands and Hints

To set and unset commands and hints, and define the hint direction and offset:

Use the handle created for a specific UI component to set a command the user must speak in order to perform actions on the UI component:

void 
set_command(vc_elm_h &handler)
{
   int ret;
   ret = vc_elm_set_command(handler, "Command"); 
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

To show the command on the screen as a tooltip, set a hint (tooltip) for the UI component:

void 
set_command_hint(vc_elm_h &handler)
{
   int ret;
   ret = vc_elm_set_command_hint(handler, "Hint");
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

Set the hint direction and offset (X and Y coordinates).

By default, the hint tooltip is shown so that it does not cover the underlying UI component and does not intersect with other hints.

void 
set_hint_direction(vc_elm_h &handler)
{
   int ret; 
   ret = vc_elm_set_command_hint_direction(handler, VC_ELM_DIRECTION_RIGHT);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

void 
set_hint_offset(vc_elm_h &handler)
{
   int ret; 
   ret = vc_elm_set_command_hint_offset(handler, 100, 100);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

You can get the current hint location using the vc_elm_get_command_hint_direction() and vc_elm_get_command_hint_offset() functions:

void 
get_hint_direction(vc_elm_h &handler)
{
   int ret;
   vc_elm_direction_e direction;
   ret = vc_elm_get_command_hint_direction(handler, &direction);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

void 
get_hint_offset(vc_elm_h &handler)
{
   int ret;
   int x, y;
   ret = vc_elm_get_command_hint_offset(handler, &x, &y);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

When no longer needed, unset the command and hint:

void 
unset_command(vc_elm_h &handler)
{
   int ret;
   ret = vc_elm_unset_command(handler);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}

void 
unset_command_hint(vc_elm_h &handler)
{
   int ret;
   ret = vc_elm_unset_command_hint(handler);
   if (VC_ELM_ERROR_NONE != ret) 
   {
      // Error handling
   }
}