Wakeword and keyword spotting integration

The wakeword and keyword spotting (WW/KW) pipeline lets a Matter application capture audio from a digital microphone, detect a wakeword, recognize spoken keywords, and dispatch commands to application logic. Reusable inference, state machine, and DMIC handling are located in samples/common. Your application supplies exported Edge AI models, keyword class definitions, Matter actions, and board-specific setup.

Integration prerequisites 

Before you integrate the WW/KW pipeline into a Matter application, make sure that the following prerequisites are completed:

Requirements and setup for the Matter Edge AI add-on, including an nRF Connect SDK workspace with the Matter Edge AI west manifest.
A supported development kit with a PDM microphone (see the Matter Edge AI generic switch sample for a supported board target).
A Matter application that you can extend with sample-specific source files and Kconfig options.
Wakeword and keyword spotting models exported from Nordic Edge AI Lab. Each export provides the generated C sources and headers described in Adding exported models.

Solution architecture 

The WW/KW pipeline runs as a dedicated Zephyr thread (EdgeAITask) in samples/common/nrf_edgeai_ww_kw_task.cpp. The thread captures PCM audio from the DMIC, runs wakeword inference until a wakeword is detected, opens a keyword detection window (duration set by the CONFIG_MATTER_EDGEAI_KEYWORD_DETECTION_TIMEOUT_S option), and runs keyword spotting inference until a command is confirmed.

Generic code in samples/common handles DMIC capture, model inference loops, and the wakeword-to-keyword state machine. Your application provides:

Exported Edge AI models (wakeword and keyword spotting).
Keyword class definitions and per-class detection tuning.
Matter actions and board-specific setup (PDM gain, LED feedback, and similar).

The following diagram shows the runtime data flow:

DMIC capture
     │
     ▼
EdgeAITask thread (common)
     │
     ├─► ww_process() ──► wakeword_app_model_get() ──► generated WW model
     │         │
     │         └─► wakeword detected ──► EdgeAIWwKwOnWakewordDetected()
     │
     └─► kw_process() ──► keyword_app_model_get() ──► generated KWS model
               │
               └─► command confirmed ──► EdgeAIWwKwHandleKeyword(class)

Matter-facing work must be scheduled on the Matter thread (for example with SystemLayer().ScheduleLambda(...)), as shown in the reference nrf_edgeai_ww_kw_impl.cpp.

Integration steps 

The WW/KW integration consists of the following steps:

Adding exported models
Binding the wakeword model
Defining keyword classes
Implementing keyword application hooks
Implementing task callbacks
Enabling Kconfig options

These steps are described in the following sections. The reference implementation is the Matter Edge AI generic switch sample (samples/generic_switch).

Adding exported models 

Each application that uses WW/KW ships two models exported from Nordic Edge AI Lab. Place the generated files under your sample source tree (for example src/models/wakeword/ and src/models/kws/).

Each export includes the following files:

nrf_edgeai_user_model.h
nrf_edgeai_user_model.c
nrf_edgeai_user_model_axon.h
nrf_edgeai_user_types.h

Add the generated sources to your application CMakeLists.txt and include the generated headers only from the application binding files (wakeword_impl.c and keyword_impl.c), not from samples/common.

Binding the wakeword model 

The common wakeword.c module does not call the generated functions directly. It calls the application hook wakeword_app_model_get(), which you implement in wakeword_impl.c.

Your generated wakeword header must expose at least:

nrf_edgeai_t *nrf_edgeai_user_model_ww(void);
uint32_t nrf_edgeai_user_model_neuton_size_ww(void);

The reference wakeword_impl.c returns the model pointer:

struct nrf_edgeai *wakeword_app_model_get(void)
{
    return nrf_edgeai_user_model_ww();
}

Note

If your Lab export uses different function names (not nrf_edgeai_user_model_ww), update wakeword_impl.c accordingly. Only wakeword_impl.c needs to know the generated function names.

Defining keyword classes 

Define the keyword class enum in keyword_app.h. Each enumerator index must match the output class index of your keyword spotting model exported from Nordic Edge AI Lab.

Example from the generic switch sample:

typedef enum keyword_labels_e {
    KEYWORD_LIGHT = 0,
    KEYWORD_OFF = 1,
    KEYWORD_ON = 2,
    KEYWORD_OTHER = 3,
    KEYWORD_SCENE_FOUR = 4,
    KEYWORD_SCENE_ONE = 5,
    KEYWORD_SCENE_THREE = 6,
    KEYWORD_SCENE_TWO = 7,
    KEYWORD_SILENCE = 8,
    KEYWORD_TOGGLE_LIGHT = 9,
    KEYWORDS_cnt
} keyword_labels_t;

Use keyword_app.h anywhere you need readable class constants (for example in nrf_edgeai_ww_kw_impl.cpp).

Implementing keyword application hooks 

Similarly, keyword.c calls keyword_app_model_get() from your keyword_impl.c.

The generated keyword spotting header must expose at least:

nrf_edgeai_t *nrf_edgeai_user_model_kws(void);
uint32_t nrf_edgeai_user_model_neuton_size_kws(void);

Reference binding:

struct nrf_edgeai *keyword_app_model_get(void)
{
    return nrf_edgeai_user_model_kws();
}

Implement all keyword_app_() hooks in keyword_impl.c. Typical contents:

static const keyword_class_cfg_t s_keyword_classes_cfg[] — detection tuning per class:
- name — debug label.
- count_needed — consecutive stable frames required before accepting the class.
- threshold_percent — minimum average probability (0 = disabled).
Model getter calling the generated nrf_edgeai_user_model_kws().
Rules for keyword_app_is_command_component() and keyword_app_is_actionable().

Only classes for which keyword_app_is_actionable() returns true can produce a kw_process() return value of 1.

Implementing task callbacks 

Implement the four callbacks declared in nrf_edgeai_ww_kw_task.h in nrf_edgeai_ww_kw_impl.cpp:

Callback	When called	Typical application work
`EdgeAIWwKwHandleKeyword()`	Keyword spotting command confirmed	Map `keyword_class` to Matter actions
`EdgeAIWwKwOnWakewordDetected()`	Wakeword accepted, keyword window opens	Start LED dimming or other user feedback
`EdgeAIWwKwOnKeywordSessionEnd()`	Keyword window closes after a command	Stop feedback effects
`EdgeAIWwKwAppInit()`	After DMIC trigger, before audio loop	PDM gain, dimming module init, board setup

Enabling Kconfig options 

Enable the following Kconfig options in your application project:

CONFIG_FPU — Enable Floating Point Unit.
CONFIG_NEWLIB_LIBC — Enable Newlib C library.
CONFIG_MATTER_EDGEAI — Enable Matter Edge AI.
CONFIG_MATTER_EDGEAI_DMIC — Enable Matter Edge AI DMIC.

For the full list of add-on and sample options, see Configuration options.

Configuration 

This section describes the configuration options for the sample.

See Configuring and building in the nRF Connect SDK documentation for information about how to permanently or temporarily change the configuration.

Optional modules 

The following optional modules in samples/common extend the WW/KW pipeline:

CONFIG_MATTER_EDGEAI_DIMMING — Enable dimming support once a keyword is detected.

Commonly tuned options 

The following options are commonly tuned in the application project:

CONFIG_MATTER_EDGEAI_KEYWORD_DETECTION_TIMEOUT_S — Timeout for keyword detection in seconds after recognizing the wakeword.
CONFIG_MATTER_EDGEAI_TASK_THREAD_STACK_SIZE — Stack size for the Edge AI wakeword/keyword task thread.
CONFIG_MATTER_EDGEAI_TASK_THREAD_PRIORITY — Priority for the Edge AI wakeword/keyword task thread.
CONFIG_MATTER_EDGEAI_WW_KW_LOG_LEVEL — Log level for the Edge AI wakeword/keyword task and model processing modules.

Wakeword tuning options 

CONFIG_MATTER_EDGEAI_WAKEWORD_PROBABILITY_THRESHOLD — Per-frame probability threshold (0–1000; default 990 corresponds to 0.99).
CONFIG_MATTER_EDGEAI_WAKEWORD_HISTORY_SIZE — Sliding window length for detections.
CONFIG_MATTER_EDGEAI_WAKEWORD_COUNT_THRESHOLD — Number of above-threshold frames required to confirm a wakeword.

Applications and samples 

The following sample demonstrates the WW/KW integration in the Matter Edge AI add-on:

Matter Edge AI generic switch — Voice-controlled Matter generic switch using wakeword and keyword spotting (samples/generic_switch).

Library support 

The WW/KW pipeline is implemented by shared sources under samples/common:

nrf_edgeai_ww_kw_task.cpp / nrf_edgeai_ww_kw_task.h — EdgeAITask thread, DMIC trigger, and application callback dispatch.
wakeword.c / wakeword.h — wakeword inference loop and detection state machine.
keyword.c / keyword.h — keyword spotting inference and class confirmation.
dmic.c / dmic.h — DMIC capture for the Edge AI audio pipeline.
dimming_effect.cpp / dimming_effect.h — optional LED dimming feedback (requires CONFIG_MATTER_EDGEAI_DIMMING).

Include matter_edge_ai_common.cmake from your sample CMakeLists.txt to add these sources to the build when CONFIG_MATTER_EDGEAI is enabled.

Dependencies 

The WW/KW integration depends on the following:

Edge AI add-on — nRF Edge AI library and Axon-based inference used by the generated models.
Nordic Edge AI Lab — training and export of wakeword and keyword spotting models.
Matter support in the nRF Connect SDK — the host Matter application and threading model (see Matter architecture).