The Tomoni project was a custom built voice assistant for a power plant. It’s very similar to a voice assistant such as Alexa or Google Assistant. In fact, I often told my friends I was working on “Alexa for power plants”.
Tomoni was designed and built by a small team of very talented people. I was very lucky to get the chance to work on such a cool product. The following is my own experience, and does not represent any other person’s opinions or experiences other than my own.
This project was done while I was working for Vectorform, a small-sized software/hardware consultancy. I was hired 2 weeks before covid and sent home to work remotely the rest of my time.
The project brought together a few different technologies:
- An Android tablet
- A custom in-house built tablet dock, to allow for extra microphones and speakers
- Voice Integration with Wake word
- Artificial Intelligence (AI) – used to translate voice commands into report queries
- General Intelligence commands (Wolfram Alpha)
- … and more!
Workflow
The general “flow” of a command goes as follows:
- Wake word (aka “hey tomoni”)
- Speak your command: “What’s the temperature of turbine 1A”
- App transcribes command to text, sends to AI service
- AI is trained to look for keywords, returns those keywords: “temperature”, “turbine”, “1A”
- App looks at results, does it have enough keywords to make a query? Answer: Yes
- App queries legacy system for the temperature of turbine 1A
- App displays result on screen, reading aloud the answer
My contributions
I think my favorite parts were the ones where I got to help with the architecture. For example: We had 4-5 possible responses to questions: such as temperature, status, joke, information, etc. We utilized interfaces, abstract classes, and sealed classes to standardize our responses. This made the app much more robust and organized.
I’m happy to report we successfully delivered the product on time. After presenting it to the client, they immediately asked us to provide a way to answer more general questions (i.e. “What is Basketball”). That responsibility fell onto me. I found that Wolfram Alpha (WA) had a natural language API – you send it a question and it tries to respond with a 1-sentence answer. It was a good fit for our needs.
At our project retrospective, I got the opportunity to influence the team in the next direction. We had a ton of logic in our Android app, we could move several parts of it into a cloud service. This would make the Android app a lot more robust and move the logic that changes most frequently to a platform that is designed to handle such changes.
Time to move on
I voluntarily left the project to start working on a new project for client that needed an app for a fragrance device. We also had a new junior developer that I thought would be a good fit for the next phase. I was a little sad to leave, but it was for the best. I was really looking forward to seeing the pull request that removed thousands of lines of code 🙂
I have worked at other places that build projects like this, and one of the best things you can do for yourself and the company is to move around. If you are on the same project for too long, you start building a “tower of knowledge” that cannot be easily transferred to other people.
You hear this advice all the time in management books and blog posts, but it’s often not a very popular decision in real life. No one, especially management, likes change. The only advice I could give is to make change the culture, so that when you aren’t making changes it makes people feel uncomfortable. It also encourages good documentation.
Funny moments
I always had the tablet running on my desk at home. My kids still talk about hearing me say “Hey Tomoni” thousands of times.
The funniest moment came during a phone call with our pediatrician. We had set up the service with a few jokes and easter eggs. One of them was if you asked it’s relationship status.
One day our pediatrician called us about test results. We were talking about a future appointment when suddenly the tablet said (very loudly) “I think it’s best if we stick to a professional relationship”
There was silence on the line.
I spent the next few minutes explaining my career and what I was working on. I think she bought it, although we agreed she should contact my wife for scheduling 🙂
