Sous Chef

Introducing the Sous Chef, second in command for all your cooking needs, a voice activated kitchen assistant which provides users with voice and visual assistance during the cooking process.

Screen Shot 2020-12-08 at 12.39.15



“The global smart kitchen appliance market size is poised to grow by USD 6.16 billion during 2020-2024…”


Adults tend to prioritize efficiency and convenience in the kitchen, yet there lacks a personalized guide to aid them through the process.  


There is an opportunity to create a more personalized, hands-free tool to help adults generate, execute and share recipes at their own pace


To inform the design process, our team conducted a formative study and used experience prototype research methods. This user research helped us uncover key insights relating to the needs and goals of our target audience in the realm of cooking, which we used to define design requirements. 


Through this method, we learned more about the target audience’s actions and attitudes in the context of cooking and meal prep. We wanted to gain insight into pain points and positive experiences in the cooking process, as well as motivations and tools used. We studied six participants between ages 20 and 30 who answered questions after each meal.


We performed surveys to learn about the target audience’s attitudes, behaviors, and characteristics when cooking. We included 23 questions about frequency of meals cooked, confidence in cooking skills, and positives and negatives when cooking. We received 76 responses from family, friends, and members of Facebook and Reddit groups.


We performed user enactments to observe how users interact with a voice assistant (VA) when cooking. We performed five user enactments with five participants. This allowed us to see how a VA enhances or detracts from the cooking experience, the proper level of personalization, and how a VA supports social connection.

Key Insights


Based on our diary studies, we learned that target users have five main needs when cooking.


Valued the ease and efficiency at which they consume food


Struggled to balance low-cost meals with time and effort required

Health & Nutrition

Motivated to cook by preparing well-balanced meals

Time & Effort

Frustrated that meal prep requires more effort and time than expected

Social Engagement

Enjoyed sharing cooking experiences with others


Based on our survey results, the majority of potential users prioritize taste and time when cooking a meal and lack confidence in their cooking skills. We also discovered themes as to why our target audience finds cooking to be either a positive or negative experience. In our design process, we aimed to create features that met users' priorities and expertise, as well as minimize pain points and promote positives when cooking.

44 % 

of individuals prioritize taste when cooking a meal

28.8 % 

of individuals prioritize time when cooking a meal

50 % 

of individuals are unconfident in their cooking skills


Therapeutic, calming, stress-reliever; social activity, brings people together


Deciding what to cook, accessing ingredients, using tools efficiently


We learned more about how a voice assistant enhances the cooking process and when users become confused with its responses and feedback. Particularly, users found the VA to be helpful since it provided clear, straightforward responses, personalization, and social opportunities. Conversely, they found the VA to be challenging when recipes lacked variety and detail, they lost control in voice interactions, and were confused by feedback. We used these insights to reduce user confusion and include helpful affordances in our design by defining requirements that met this criteria.


Based on our research results, we were able to validate the need for an innovative cooking assistant and make several key design decisions to build this product. Three key design opportunities are as follows:


Our design address conflicting user needs by personalizing interactions and recipe suggestions when cooking

Social Engagement

Our design supports individuals’ goals to use cooking as a means of social connection, bringing people together

Voice Interaction

Our design provides hands-free guidance through voice interaction makes cooking efficient, easy, and convenient

Final System Concept​


Generate recipe ideas

The Sous Chef takes into account the users’ tastes, time, ingredients available, number of guests the meal is being prepared for and generates personalized results based on the user preferences.








Audio & Visual Guidance

The cooking assistant offers audio & visual guidance. The blue light lets the user know that the sous chef is processing the users request, and the yellow light indicates that the system is  waiting for the user to speak.










Social engagement 

The cooking assistant has the ability to share recipes with loved ones. Users can send their favorite recipes to family and friends with a personalized message.


Sofia is looking to cook dinner using the ingredients in her fridge. She gives Sous Chef 4-5 ingredients in her fridge, and is presented with a list of recipes she can choose from. Based on the ratings, Sofia decides to make the penne ala vodka, but realizes she is out of shallots. Sous Chef presents her with substitutes, and the next steps in the cooking process. The visual feedback from Sous Chef helps her know whether Sous Chef is processing her request, or waiting for a command. When Sofia is done cooking, she is impressed by the end results. She decides to share the recipe with her friend, Orly. Orly receives the recipe from Sofia, and decides to make the same dish. Sous Chef adjusts the measurements of the ingredients based on the number of people Orly is cooking for, and gives her step by step instructions. She is impressed with the end results and shares the picture of her dish on her instagram. Anandita sees Orlys instagram post, and decides to buy the Sous Chef. She starts setting up the Sous Chef by stating her preferences, and dietary restrictions.


Prototype Shots and Key Features

Below are images of our prototyping process. We began by sketching out our design to represent how the audio, visual and light feedback would come together to create Sous Chef. We chose to use common household products and class materials to accommodate our need to create four nearly identical prototypes.



We built our physical prototype by setting up our breadboard and securing our Neopixel into a paper towel roll wrapped in tinfoil. This serves as the speaker and microphone for the voice assistant while the Neopixel represents the light feedback within the system. We attached this to a tablet that displayed recipes using screens in Figma.  

Screen Shot 2020-12-02 at 11.40.13
Screen Shot 2020-12-02 at 11.40.19

Lastly, we wrote the proper code to power the Neopixel that would be serving as our light feedback and we pre-recorded the audio for the Sous Chef’s voice interactions.

Screen Shot 2020-12-02 at 11.40.26

Demo Key Features Explained

Key elements the demo captured of the intended experiences include: onboarding, recipe suggestions, guiding instructions, and social affordances. The system gives audio, visual, and light feedback when interacting with users, helping them better leverage the tool while cooking. The demo captures different light displays based on interactions such as, when the device is listening (yellow light), processing input (white/blue light), device response (solid blue light), standby (solid purple light). The device also has a digital display that provides additional visual output to the user such as recipe suggestions, and additional instructions and prompts to help personalize the users experience that are displayed during the demonstration. Primarily the demo features voice activated interactions with the user and provides a step-by-step guide to users throughout the cooking process, allowing for a hands-free digital assistant in the kitchen. This is shown by having the user interact and ask clarifying questions to the kitchen assistant.  

Ideal System Proposal

We are proposing a voice assistant that will aid users in the kitchen regardless of expertise level.  The Sous Chef helps users select recipes to cook based on available ingredients, allergies and preferences, creating a more personalized experience than competitors.  The voice assistant guides users through the cooking process by calling out and displaying steps throughout the recipe, answering clarifying questions about ingredients or quantities, and providing substitutions when needed. The system includes a social feature allowing users to easily share recipes with friends and family and a save feature to quickly access favorite recipes.  

As seen in the diagram below, our system consists of features that interact with the user from both the user-facing side of the product and the internal backend. The physical product, serving as the user-facing side features a visual display, a light and a dual speaker microphone.  The visual display presents the user with recipe results, recipe detail pages and recipe reviews. The speaker serves as a voice assistant to provide the user with audio feedback including ingredient substitutions and recipe instructions. This same feature also serves as a microphone for the user to interact with the assistant. Lastly, there is a circular light in the center of the speaker as indicated by the blue circle in the diagram. This light serves as visual feedback for the user to understand the current state of the device: standby, processing user input, speaking and waiting for a user response. This helps guide the user through voice interactions with the product. 

On the backend, our product uses artificial intelligence to recognize and respond to a user’s voice commands. It utilizes user input to learn more about their common habits, substitutions, allergies and preferences to better assist the user in a personalized cooking process. The internal architecture of the product includes a recipe database that serves as a search engine to generate recipe ideas for the user. Finally, the backend utilizes Bluetooth, allowing the user to easily connect their cooking assistant to Wifi, pair with their smartphone and contacts, and share recipes instantly.

system arch.png



One limitation of our design is the personalization required for Sous Chef to operate. We designed Sous Chef to accumulate personal information about the user and evolve its knowledge with the use of Artificial Intelligence. However, it is important to consider privacy concerns involved with collecting this information. Some users may not want Sous Chef to have certain information. Despite this, we believe that most users will opt into sharing this personal information since they intended to use Sous Chef as a personalized guide throughout the cooking process.


The next step our team would take to realize Sous Chef would be to create a higher fidelity product, such as improving the visual display. We would then perform user testing for common interactions that users have with the visual display when cooking with Sous Chef. It would also be helpful to perform more user testing on voice interactions to see how users respond to the visual light feedback. We would want to confirm that the light feedback helps guide users throughout voice interactions with Sous Chef, improving the intuitiveness of our product's design. 


Throughout the prototyping and demo process, our team uncovered insights that validated several design decisions. One important insight is the importance of designing multiple forms of feedback from Sous Chef to the user. While preparing the demo, we confirmed the need to include audio, visual, and light feedback within our assistant to meet standard conventions of voice assistants and create a more intuitive experience. Through the demo, we also learned the importance of ensuring that the user feels in control when interacting with Sous Chef. This confirmed design decision that we include to empower the user and provide more freedom in interactions. For example, we chose to build-in several comments, such as "repeat that step" and "slow down", which increase user control