Pleas give me a small shove...

info about ReSpeaker series products

Moderators: lily.li, violet, seth.welday

Post Reply
c.bickmeier
Pre-kindergarten
Pre-kindergarten
Posts: 2
Joined: Mon Sep 17, 2018 8:45 pm

Pleas give me a small shove...

Post by c.bickmeier » Mon Sep 02, 2019 7:36 pm

Hi all,

I've build a voice control solution for my smart home based on Respeaker Core V2. But now I realized that all the fancy voice algorithms are not active when I just use the normal microphone API and I want to port my code over to a respeakerd based version.

Unfortunately my linux / python skills are very poor (I´ve more than twenty years experience on Windows and .net development, but never worked with linux / python). So I need a little help from someone who knows what he`s doin :-) ).

The samples provided are all bound to alexa, but I want to build my own solution to prevent the use of any cloud services. So all I want is:
1. Look for a hotwork
2. Record while someone is talking
3. Use all the fancy algorithms (Noice Cancelling, Beamforming etc.)
4. Dump the recorded data to disk (later I´ll send the data to my own service)
5. Get back to hotword listening mode
6. Be able to quit the application without terminating the respeakerd service

Here is my code so far:

Code: Select all

import os
import sys
import time
import signal
import logging
import mraa
import Queue as queue
import ptvsd

ptvsd.enable_attach()

sys.path.append('/home/respeaker/respeakerd/clients/Python')

from respeakerd_source import RespeakerdSource

class BoardComputer(object):
    def __init__(self):
        self.listening = False
        self.audioQueue = queue.Queue(maxsize=1000)
        pass

    def start(self):
        print('Start')

    def stop(self):
        print('Stop')

    def run(self):
        print( 'Run')

    def put(self, audio):
        if self.listening:
            self.audioQueue.put(audio)

    def listen(self, dialog=None, intiator=None, timeout=10000):
        if self.listening:
            return

        self.audioQueue.queue.clear()
        self.listening = True

        def getAudio():
            timeElapsed = 0

            print('getAudio')

            while self.listening and timeElapsed <= timeout:
                print ('in loop')

                try:
                    chunk = self.audioQueue.get(timeout=1.0)

                    print('chunk read')
                except queue.Empty:
                    break

                yield chunk
                timeElapsed += len(chunk) * 1000 / (2 * 16000)
                print(timeElapsed)
            self.listening = False

        ccc = getAudio()

        print('Listen')

def onDetected(dir, index):
    print('Detected')
    boardComputer.listen()

def onVad():
    #print ('VAD')
    pass

def onSilence():
    #print ('Silent')
    pass

def onDoa(dir):
    #print "DOA"
    #print dir
    pass

logging.basicConfig(level=logging.DEBUG)
logging.getLogger('hpack.hpack').setLevel(logging.INFO)

en = mraa.Gpio(12)

if os.geteuid() != 0:
    time.sleep(1)

en.dir(mraa.DIR_OUT)
en.write(0)
                        
boardComputer = BoardComputer()

src = RespeakerdSource()
src.set_callback(onDetected)
src.set_doa_callback(onDoa)
src.set_vad_callback(onVad)
src.set_silence_callback(onSilence)

src.link(boardComputer)

src.recursive_start()
src.on_cloud_ready()

try:
    while True:
        time.sleep(1)
except KeyboardInterrupt:
    print('Stopping')
finally:
    #src.on_disconnected()
    src.recursive_stop()
    #src.stop()

    en.write(1)
Questions:
1. Am I on the right track (correct way in unsing respeakerd)?
2. Listensing for the hot word does work, but it always fires "VAD" and nearly never "Silent"... why?
3. Why does he never go into "getAudio"?
4. When I exit the applicaiton with Ctrl+C the respeakerd service is terminated and has to be restartet every time I want to start my application... why?
5. Can anyone provide my a sample for using respeakerd WITHOUT alexa... just detect wakwork... start recording... stop recording when timout arrived or no VAD and dump the recorded data to file?

Some more info about the "restart problem":
When I run my script for the first time:

respeaker@E3BC1:~/BoardComputer$ python RespeakerdTest.py
Start
INFO:respeakerd_client:Start to connect to the socket: /tmp/respeakerd.sock ...
INFO:respeakerd_client:Connected to socket
^CStopping
INFO:respeakerd_client:going to close the socket...
Stop

Everything works fine... but when I start my script again...

respeaker@E3BC1:~/BoardComputer$ python RespeakerdTest.py
Start
INFO:respeakerd_client:Start to connect to the socket: /tmp/respeakerd.sock ...
INFO:respeakerd_client:Connected to socket
ERROR:respeakerd_client:Other socket error when recv data: #104 Connection reset by peer
INFO:respeakerd_client:going to close the socket...
INFO:respeakerd_client:Start to connect to the socket: /tmp/respeakerd.sock ...
ERROR:respeakerd_client:Error when connect to the socket: [Errno 111] Connection refused
INFO:respeakerd_client:Start to connect to the socket: /tmp/respeakerd.sock ...
ERROR:respeakerd_client:Error when connect to the socket: [Errno 111] Connection refused
INFO:respeakerd_client:Start to connect to the socket: /tmp/respeakerd.sock ...
ERROR:respeakerd_client:Error when connect to the socket: [Errno 111] Connection refused
INFO:respeakerd_client:Start to connect to the socket: /tmp/respeakerd.sock ...
ERROR:respeakerd_client:Error when connect to the socket: [Errno 111] Connection refused
INFO:respeakerd_client:Start to connect to the socket: /tmp/respeakerd.sock ...
ERROR:respeakerd_client:Error when connect to the socket: [Errno 111] Connection refused
^CStopping
INFO:respeakerd_client:going to close the socket...
Stop
respeaker@E3BC1:~/BoardComputer$

...it seems that the respeakered service has stopped (when I restart it manually, I can start my script again)... but why?


Thanks you very much for your help.

Carl

Post Reply