Please note that all Mycroft Documentation has now moved here.

Mycroft commands from Android or other RTP streams

If you are interested in control mycroft from an Android or any other device able to stream RTP audio, following steps could be useful.

Note: following tests has been done with an Android phone and a Linux Mint computer host of mycroft software.

1. Install and configure a RTP program in your device

App "rtpmic" (see here) has been installed from Google play (disclaimer: absolutely no relation with its authors).

Configure the app with following data:

• Target IP: the IP of the host where mycroft is running
• Port: 5555
• Media protocol: G.711u

2. Prepare your host computer

a) Install "snd-aloop" module. You can use a temporary command as "modprobe snd-aloop" or create/modify system configuration file with:

echo 'snd-aloop' >> /etc/modules-load.d/alsa.conf

(restart is need if configuration file is changed)

b) Configure your iptables to accept incoming UDP at port 5555. This configuration is very specific and, as all security related issues, must be done with care. A typical way to open this path is with command:

sudo iptables -A INPUT -p udp --dport 5555 -j ACCEPT

c) install "gstreamer" software:

apt install gstreamer1.0-tools

3. Configure mycroft input device:

a) "aplay -l" allows list of all audio inputs. Verify number for loopback "card" (on this test, it is number 0).

b) translate the device name to an integer device index to be used in mycroft config. The following python script show the list of device with its names and identifiers:

import pyaudio
p = pyaudio.PyAudio()
info = p.get_host_api_info_by_index(0)
numdevices = info.get('deviceCount')
for i in range(0, numdevices):
   di=p.get_device_info_by_host_api_device_index(0, i)
   maxc=di.get('maxInputChannels')
   if maxc  > 0:
       print "Input Device id ", i, " - ", di.get('name'), ": rate=",di.get('defaultSampleRate'), "channels=", maxc

c) Configure in mycroft.conf, section listener, item "device_index" to the integer that identifies the audio device.

4. Run it

a) In a terminal, execute command (and let it running, in background if you prefer):

gst-launch-1.0 udpsrc port=5555 caps="application/x-rtp,media=audio" ! rtppcmudepay ! mulawdec ! audioconvert ! audioresample ! 'audio/x-raw, rate=16000, channels=1, format=S16LE' ! alsasink device='hw:0,0'

to forward audio from port 5555 to alsa loopback devices (hw:0,0 - hw:0,1).

b) start mycroft and it is done.

Trobleshooting.

In case of problems, following commands could be useful:

a) check your audio output, play a wav file:

 gst-launch-1.0 filesrc location=some_wav_file.wav ! wavparse ! alsasink

(change to alsasink device=plughw:xx,0 with xx your speakers card number if necessary).

b) check rtp incoming audio, forward rtp audio to your speakers:

gst-launch-1.0 udpsrc port=5555 caps="application/x-rtp,media=audio"  ! rtppcmudepay ! mulawdec ! audioconvert ! alsasink

c) command

tcpdump -i any "port 5555" 

must show an udp package each 20 ms (aprox.). If not, check your phone configuration.

Next to do

Allow multiple audio sources in mycroft. On going ...