How to use the SNIPS.ai voice service with a XMOS smart microphone?

[Thomas]

I would like to use the SNIPS.ai voice service with a VocalFusion™ Stereo Dev Kit
How can I set this up?

[Thomas]

Here are some simple steps to setup the SNIPS.ai voice service with the VocalFusion™ Stereo Dev Kit

1. Change the audio setup on Raspberry Pi to work with USB Audio.
See Appendix 1
Reboot RPi

2. Connect Stereo Loudspeakers to the VocalFusion™ Stereo Dev Kit

3. Connect VocalFusion™ Stereo Dev Kit to Raspberry Pi via USB
You can verify that the VocalFusion Kit is now enumerated as a sound card by executing aplay -l on the RPi. It should output something like:

Code: Select all

**** Liste der Hardware-Geräte (PLAYBACK) ****
Karte 0: StUAC10 [XMOS VocalFusion St (UAC1.0)], Gerät 0: USB Audio [USB Audio]
  Sub-Geräte: 1/1
  Sub-Gerät #0: subdevice #0

4. Install SNIPS.ai according to their documentation: https://snips.gitbook.io/getting-started/

5. Run the demo: https://snips.gitbook.io/getting-starte ... -assistant

When I say:

Hey Snips!
What will be the weather like in Madagascar in two days?

then it responds:

You asked for the weather in Madagascar in two days.

Note:
Running "sam watch” on the PC is very interesting.
It watches the ASR engine running on the RPi over the network and prints status infos including the output of SST.
See Appendix 2.


Appendix 1:
Instructions to change the audio setup on Raspberry Pi to work with USB Audio.
Make sure to reboot afterwards!

1. create /etc/modprobe.d/alsa-base.conf with this content
options snd slots=snd_usb_audio
options snd_usb_audio index=0

2. modify /usr/share/alsa/alsa.conf and comment out "~/.asoundrc"

Code: Select all

@hooks [
   {
      func load
      files [
         {
            @func concat
            strings [
               { @func datadir }
               "/alsa.conf.d/"
            ]
         }
         "/etc/asound.conf"
#         "~/.asoundrc"   
      ]
      errors false
   }
]

3. modify /boot/config.txt
#Enable audio (loads snd_bcm2835)
dtparam=audio=off

Appendix 2:

Code: Select all

[15:11:02] [Asr] was asked to listen on site default
[15:11:07] [Asr] captured text "what will the weather be in madagascar in two days" in 4.0s
[15:11:07] [Asr] was asked to stop listening on site default
[15:11:07] [AudioServer] was asked to play a wav of 93.1 kB with id 'df3a6c2b-688a-4e11-8c9c-5acc79b51037' on site default
[15:11:08] [AudioServer] finished playing wav with id 'df3a6c2b-688a-4e11-8c9c-5acc79b51037'
[15:11:08] [Nlu] was asked to parse input "what will the weather be in madagascar in two days"
[15:11:08] [Nlu] detected intent searchWeatherForecast with probability 0.972 for input "what will the weather be in madagascar in two days"
              Slots ->
                        forecast_country -> Madagascar
                 forecast_start_datetime -> 2018-09-29 00:00:00 +02:00
[15:11:08] [Dialogue] New intent detected searchWeatherForecast with probability 0.972
              Slots ->
                        forecast_country -> Madagascar
                 forecast_start_datetime -> 2018-09-29 00:00:00 +02:00
[15:11:08] [Dialogue] was ask to end session with id 027e2fa0-2c75-43ef-bbcc-01a39f60cb23 by saying 'You asked for the weather in Madagascar in two days'
[15:11:08] [Tts] was asked to say "You asked for the weather in Madagascar in two days"
[15:11:08] [AudioServer] was asked to play a wav of 87.5 kB with id '28bae0a5-6546-4815-9010-033010017067' on site default
[15:11:11] [AudioServer] finished playing wav with id '28bae0a5-6546-4815-9010-033010017067'
[15:11:11] [Tts] finished speaking with id 'f9d61439-868f-4335-8e97-e24559266342'
[15:11:11] [Dialogue] session with id '027e2fa0-2c75-43ef-bbcc-01a39f60cb23' was ended on site default. The session ended as expected

[Thomas]

Here are some details for using SNIPS.ai on RPI together with the XK-VF3510-L71 Kit.

The Kit doesn't have an audio output .
The reason is that it was designed for Smart TV and Set Top Box applications where the audio output is located elsewhere.
Therefore the loudspeaker has to be connected to the audio output of the RPi.
But the same audio output signal has to be routed to the XVF3510 as AEC reference.

One solution is duplicating the stereo output signal sending it to the RPi Analog output (loudspeaker) and also over USB to the XVF3510.

Steps:
- Make sure your XK-VF3510-L71 is running the USB firmware binary.
- Plug the XK-VF3510-L71 into the RPi with a USB cable.
- Select the Analog output on the RPi (right click on the loudspeaker symbol in the top right corner)
- Create a ~/.asoundrc file with the content shown below
- Select the "stereomulti" device as your output device for SNIPS.ai

After that the audio output signal should be played through a loudspeaker connected to the RPi Analog output.
The same audio signal will be sent over USB Audio to the XVF3510 as AEC reference.

Code: Select all

pcm.multi2boards {
 
    type multi;
 
    slaves.a.pcm "sysdefault:0";
    slaves.a.channels 2;
    slaves.b.pcm "sysdefault:XVF3510UAC10";
    slaves.b.channels 2;
 
    bindings.0.slave a;
    bindings.0.channel 0;
    bindings.1.slave a;
    bindings.1.channel 1;
 
    bindings.2.slave b;
    bindings.2.channel 0;
    bindings.3.slave b;
    bindings.3.channel 1;
 
}
 
# produce 2 to 4 channel output
 
pcm.stereomulti{
    type route
    slave.pcm "multi2boards"
    ttable.0.0 1
    ttable.1.1 1
    ttable.0.2 1
    ttable.1.3 1
 
}
 
pcm.stereoout_analog_plus_usb{
    type plug
    slave.pcm "stereomulti"
}