Respeaker Core v2 and Snips

[cinettoa]

Hi,
I want to use Respeaker Core v2 and Snips.
I can find on-line all the info to install Snips in Respeaker Core v2 and run the tutorial and examples correctly.
Now, I would like to put a sound processing pipeline made with librespeaker between Respeaker and Snips in order to feed Snips with an improved audio stream.
Since I am learning from librespeaker examples, I was thinking to redirect the audio chunks in output from librespeaker pipeline (which in the librespeaker example are sent to a wav file) to a Linux device which then can be recognized by pulse. Then set up snips to use this pulse new source as an input.
Is it the most convenient way to reach my target or any simpler way is available?
any tutorial or guidance about this?

Thanks for any hint

[chdyejiewei]

Hi,

Sorry for the late reply. For now, this way is recommended to feed Snips:

Code: Select all

snips-audio-server --nomike --hijack 0.0.0.0:1234 &

Then you can stream mono audio data to this port in your code.

[cinettoa]

Thanks, I've found this option in Snips documentation and I am trying it out.

In the meanwhile do you have any example of code using the librespeaker class SnipsDoaKwsNode?

I've found it in the documentation but cannot find any related usage example.

Could you also advice whether it is possible to use this module within the respeakerd server?

Thanks!

[chdyejiewei]

Hi,

The class SnipsDoaKwsNode is located in librespeaker2.1.0, which is not released yet. A huge update of librespeaker is still developing, and we may release librespeaker3.0.0 directly in late Nov.

Thanks for your attention!

[cinettoa]

Hi and thanks for the spoiler

in the meanwhile I am trying to link the librespeaker and snips in this way:

1- I use librespeakerd server to run in pulse mode and output the audio in a new pulse source called respeakerd-output (I've followed the instructions in the GitHub repository).

Code: Select all

4. PulseAudio mode
4.1 PulseAudio configuratin
$ sudo vim /etc/pulse/default.pa
Add the following line to the end of the file:

load-module module-pipe-source source_name="respeakerd_output" rate=16000 channels=1
set-default-source respeakerd_output
4.2 Start respeakerd in PulseAudio mode
$ cd PROJECT-ROOT/build
$ ./respeakerd -mode=pulse -source="alsa_input.platform-sound_0.seeed-8ch" -debug -snowboy_model_path="./resources/snowboy.umdl" -snowboy_res_path="./resources/common.res" -snowboy_sensitivity="0.4"
Specify other options if you need. Please note that if no application's consuming the audio stream from respeakerd_output source, respeakerd will get blocked. But this is not a big deal. Now let's move on to the setup of Alexa C++ SDK example App [TODO]

2- I configure Snips in order to read from the aforementioned source (instruction found on snips dev center):

Code: Select all

The Respeaker core v2 is using pulsaudio. We need to allow the snips service user to access the pulse audio service:

sudo usermod -a -G pulse-access _snips
For a better sound experience you need to modify the default pulse audio daemon configuration: 

sudo vi /etc/pulse/daemon.conf
this file need to contains the lines: 

default-fragments = 5
default-fragment-size-msec = 2 

3- first I run respeakerd then I run snips-audio-server.
The output at this point ends like this:

Code: Select all

respeaker@v2:~$ snips-audio-server
[10:12:46.053030] INFO :snips_broadcast_mqtt_server: Starting broadcasting audio on hermes as site "default"
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_dmix.c:1052:(snd_pcm_dmix_open) unable to open slave
[10:12:46.406812] INFO :snips_audio_portaudio      : Use input device: "default"
[10:12:51.068839] ERROR:snips_audio_server         : Can not read a stopped source

Do you have any experience about what could be wrong?

Thanks!