Cortana Office Assistant

Exploring the relationship between PCs and assistive devices

This unnamed project played with the idea of a smart speaker for your office desk, that also serves as a companion device for your PC. Notifications would be handled by the device, allowing you to focus more on your work. Using computer vision, we could use the camera, along with other signals, to do things like lock your PC when you are not seen. While away, the device could be used by others to leave messages or find out you will be back.

Hardware

This was one of my first forays into industrial/hardware design, but I started it like I do any other design project--by sketching. To keep it simple, I focused on simple shapes and forms.

Eventually, I settled on the simplest shapes I came up with--a cylinder and a cube. I brought it into Adobe Illustrator to flesh out the concept a bit more. The idea was starting to take shape, but trying to design a 3D object in a 2D space is limiting.

Then, I thought to myself, “what if I learned 3D?”

I discovered Blender and immersed myself in tutorials and guides. Within a week, I managed to create a decent looking render. However, it still felt a bit rough, and as much as I enjoyed learning 3D, I can only learn and do so much on my own.

We needed help.

Collaboration

We reached out to a 3D artist on our team, who took my designs and really went wild with them. He created some gorgeous, hyper-realistic, renders, which helped secure approval and resources to continue the project

Alongside our hardware explorations, we also worked closely with a group of computer vision experts, who helped develop Microsoft's Hololens. While we were designing the hardware, they were working on a software prototype to explore what facial and gaze recognition could do for users. How can we help users if we know who they are, who is around them, and what they are focused on?

Prototyping

I've always wanted to try 3D printing, and this project was the perfect excuse to do so. I invested in a 3D printer, started learning Fusion 360, and began designing the outer shell of the prototype.

For the electronics, the shell itself was designed off the form factor of a Raspberry Pi 3 B+, as well as microphone, LED array, touchscreen, and speakers.

Results

From knowing nearly nothing about 3D software, 3D printing, or industrial design, we were able to create a functioning hardware prototype to explore the concept of Cortana in the workplace. In the process, we learned a lot of things and leveled up some new skills, but the most important we learned was computer vision is an amazing technology, but can also be invasive and raises many privacy and security concerns.