Introducing TT – the talking Linux Penguin

I love working with the kids at CoderDojo and Techgirlz and am always on the lookout for new projects and technologies. Looking through a Raspberry PI magazine, I spotted a Do-It-Yourself AI kit from Google and since it was only $10, why not give it a try?

Like all tech projects, I had a few frustrations, but I ended up with a fully functioning Google Assistant inside a cardboard box, with a big red button on top, and a Raspberry PI, speaker and microphone inside. I wasn’t sure quite what to do next – would kids growing up with Siri, Alexa, etc actually be interested in this cardboard box? Then, downstairs comes my 13 year old son who proceeds to spend the next 15 minutes playing with the cardboard box to see if he can get Siri on his phone and Google assistant in the box to converse with each other…

Sizing up

While he was busy at that, my mind wandered. Humm, the size of that cardboard box is about the same size as the head of the stuffed Linux penguin I made (yes, there is an open source stuffed Linux penguin pattern!). I wondered, could I somehow take the parts out of the cardboard box and put them inside a penguin – and have a talking Linux penguin?

 

Creating Tux the Terminator!

OK, perhaps there aren’t too many people out there like me, but on a cold winter night, boy did I have fun making a second stuffed penguin. During assembly (before I did the great cover-up), my husband looked at the penguin-like creature with a speaker in the middle of his front, a Raspberry Pi sticking out the back, and wires everywhere and said it looked like the terminator, so, I proudly introduce you to “TT” – short for Tux the Terminator!

Designing for easy access

TT is a lot of fur and stuffing covering up the components of a Google AIY 1.0 voice kit and a Raspberry PI. The Raspberry Pi is in his back, with wires running up to the red button in his head, and through the front to a speaker and microphone I hid inside his beak. I had to make the beak abnormally wide to cover them up, which I like to think of as giving him extra character. I also have a power cable and display cable running out the bottom so I can hook up a battery and a monitor. I have a USB dongle for my wireless keyboard/mouse in the PI. I use velcro for easy access to the Raspberry Pi – just in case I need to get to it.

TT is programmed in Python and uses the Google Cloud Platform Google Assistant and Cloud Speech APIs. I currently run two programs on TT. First, I use the Google Assistant sample program “as-is” which turns TT into a full-fledged Google Assistant. As you can see in this video, the kids love asking the penguin the most outrageous questions.

Customising with code

I also did a little custom programming of TT starting with the sample program cloudspeech_demo.py to turn on and off the light. I added voice interactions to respond to keywords including “red hat”, “hello tux”, and “what’s inside you”, as you can watch in this video. These were straightforward extensions, with the main trick to debug what the program is actually hearing and add some “or”s. For example, “hello tux” often was interpreted as “hello tax”, so we ended up coding a response to either “hello tux” or “hello tax”. As you can see on the video – there is a slight delay before TT responds to questions, but that is understandable as TT needs to access the internet Google Cloud Platform for every interaction.

Code to “Turn Light on”

Connecting on the move

TT needs to be on the internet, so I purchased a small monitor and wireless keyboard so I can get him on the internet when TT travels. Once TT is on the internet, TT can operate with no visible wires or external devices connected – he is just a stuffed talking penguin! I do find a remote desktop connection into TT very useful for monitoring and troubleshooting as the programs display what TT “hears” which helps in debugging.

 

My son got tired of my asking for him to bring up the remote desktop, so he automated it for me. His Raspberry Pi Remote Desktop application (currently Mac only) includes a UI for me to input the penguin’s IP address, user id and password, and does the undercover magic for the SSH and VNC (even installing the packages) to bring up the Remote Desktop UI.

To make the job easy, the application searches your local network for devices, and gives you the list of IP addresses (list may take a bit of time to appear). This helps me find the penguins IP quickly (it search fails on some networks, so sometimes I need to use TT’s monitor to find the IP, and then use the program to connect). You are welcome to try it out to connect to your Raspberry Pi (suggestions and improvements are welcome). It’s been super handy for me!

 

Hiccups along the way

I would be remiss if I didn’t point out a few of the gotchas I hit along the way. I used the older Google AIY 1.0 kit with an existing Raspberry Pi. The booklet directions for assembling the hardware part of the kit were great. There have been some software changes, so I had to go on-line and improvise a little between the 1.0 and 2.0 directions to get everything running. I am impressed with how well voice is picked up by the microphone and the volume of the speaker – even when inside the beak of a penguin. My biggest issues so far are:

1. The mechanical voice

The Google Assistant voice is very natural sounding, but if you program your own application using the Speech APIs the voice is not natural. You’ll notice the two different voices between the videos. We’ve checked forums and played with settings and haven’t found any good solution yet. I also had to tailor my questions and the answers the penguin gives since the Speech API has difficulty with some tech words like “Linux” and “API”. I wanted to ask the Penguin what operating system it was running, but after trying “Linux”, “Raspbian”, and “Debian” without success, I put in the sarcastic answer “Isn’t it obvious”.

2. Paid API

The Google Cloud Speech API is a billable API and requires a credit card. The Google Assistant API is free, but to actually program a custom interaction, which is what I want the kids to be able to experience, requires the billable API. Just a small amount of usage puts me over the free monthly limit and eating into the trial credit. It would be nice if there were a no credit card way to demo and teach with this feature. Prepaid cards aren’t accepted. In the EU, the API can only be used for business purposes.

3. Corrupted SD card

The kit comes with a warning “Power off carefully”. I did corrupt an SD card so be sure to shutdown when possible (from the remote desktop, you need to type “sudo shutdown now” into terminal). I now carry an extra SD card for TT just in case this happens again.

 

Kids adding their own interactions

Overall, it’s hard to complain when I’m having such a blast with this project! The kids are starting to program their own interactions. Last week, the girls programming TT’s favorite color – RED of course! I can’t wait to take Tux with me to more spring STEM events. Who can resist a stuffed, talking Penguin?!

 

Thank yous

My sincere thanks to all who support me in continuing to pursue my diverse passions during retirement. With special thanks to IBM, whose community grant program allows me to buy gadgets like Raspberry Pis and Micro:bits for the kids, and provides their facilities for STEM events. I’d also like to thank both CoderDojo and Techgirlz – fantastic STEM organizations!

 

Special thanks to CoderDojo champions Pete Gegen, Zoryana Tischenko, Niambh Scullion, and Brendon Murray. Pete, Zory, Niambh, and Brendon inspired and encouraged me down the volunteer path –  “Just start a Dojo Ruth – you’ll never regret it

Ruth Willenborg, a retired IBM Distinguished Engineer, is the founder and champion of IBM CoderDojo RTP and a Triangle Techgirlz volunteer instructor.

Keep up to date with our newsletter

You can unsubscribe at any time.