Designing with Voice

Technology Voice

Designing with Voice

a look at where we are headed from the machine-learning
authority Alex Capecelatro

By Alex Capecelatro

Designing with Voice

Steve Jobs, in 2003 while discussing the iPod, insightfully explained, “Design is not just what it looks like and feels like. Design is how it works.” In the luxury smart home, we believe the same holds true for technology. How does the end user experience it? How do they turn it on? How do they adjust it? How can we keep things simple, preserve the architect and designer’s vision, and enable awe-inspiring moments?

I spend a lot of time speaking with designers about their vision, frustrations, and thoughts on the end user experience. How do their clients want to interact with their home and how can we work together to heighten that experience? That’s why I truly believe we’re building a design-first company with Our goal is to design the ultimate user experience in today’s luxury residential sector.

Imagine walking into a beautiful living room and not being able to open the shades. Or having to fumble around with an app on your phone to dim the lights. Or trying to figure out which switch on which keypad turns the music off. In fact, I believe that even having to think about how to control your smart home kills the ambiance and downgrades the experience. It should all be simplistically intuitive.

In 2015 my business partner, Tim Gill, and I co-founded because we had that kind of bad experience. We were both remodeling relatively large homes, and at the time I didn’t know a lot about smart home technology. I was pitched expensive, complicated, often outdated interfaces and it felt like there had to be a better way. My home is decorated with works from Richard Serra and Ellsworth Kelly – geniuses with a modern appetite – and I couldn’t imagine cluttering the walls with screens and panels.

So three-plus years into Josh, let me bring those of you in the design-build space up to speed on where designing with voice currently is at and where we are headed. I want to also touch on privacy issues. And most importantly, where machine learning elevates the user experience.

First and foremost, voice control’s number one goal should be to simplify everything. This is particularly difficult when parsing text because we tend to leave out a lot of context when we speak. For example, consider the command, “Turn that off.” Was “that” referring to the music you have on, the lights that were recently dimmed, the TV that plays Netflix’s next season hit, or maybe you’re pointing at a particular device you want to shut off. Voice assistants struggle with disambiguating these simple commands. But that’s where we’re focused with It’s still early days and there’s a lot of growth in store, but we’re excited about building truly smart systems to accommodate natural speech.





Today you can ask Josh to play any artist, album, song or genre, watch anything that streams, turn on or off lights and switches, open or close the shades, run scenes, control the thermostat, and even operate your pool, garage, and motorized doors. You can string together commands, issuing an order such as, “Dim the lights in the kitchen, living room, and dining room, play Frank Sinatra, shut the shades just a little, and make it warmer in here.” Or you can set up a scene and simply say, “It’s dinner time” to orchestrate the exact same scenario. The goal is to make it simple and intuitive.

At Josh, our system starts with Josh Micro, a beautiful piece of hardware with a far-field microphone array, full smart home processor, built-in security and contextual sensors. Josh Micro is about the size of a Nest thermostat, designed to be wall mounted or placed on a side table, and typically installed in every major room of the home.

Each Micro attempts to process commands locally, avoiding going to the cloud when possible to increase security and processing time. This tiny device handles natural language processing, machine learning, building configuration, and pattern recognition. Each command by default is stored locally on the device so the system can improve over time. That said, from a privacy perspective, the end user can always choose to turn off this recording so commands are given, thrown away, and there’s no history. Some are calling this the Snapchat of voice assistants. From another privacy point-of-view, the system only listens for a maximum of 20 seconds after a wake word is issued (“Okay Josh, play the Beatles”), while Josh is listening the LEDs light up so you know it’s listening, and you can always see what was heard from the Josh app. Don’t like what you see? You can always delete commands individually or erase your entire history of commands. We believe the end user should always be in charge of their personal data.

Further, Josh has an industry leading app design focused on ease-of-use, simplicity, and complete control. You can view and control your home remotely, even if you’re halfway around the world, checking on security cameras and utilizing the built-in microphone in your phone to give voice commands if not near a Micro. We believe voice is great some of the time, but it’s not the only way to control your home.

To guarantee the product and installation deliver a luxury experience, is only sold through certified technology integrators that meet our standards and have been properly trained. Every client is unique, every home has its own demands, and this is a boutique business where we want to ensure the client gets the customization they deserve. To further guarantee the experience, we have recommended network configurations and device integrations where we know reliability and robustness will be at peak performance.

From my point of view, we want homeowners to experience living in their home without headaches. They shouldn’t have to think about or learn the technology embedded behind the walls. Simply tell Josh to dim the lights, put on some music, watch a show, or shut everything down. It should be that easy.

Alex Capecelatro
DENVER, CO 80206