Voice assistants or virtual assistants use voice recognition and artificial intelligence to understand the verbal commands or questions of the user and provide appropriate answers or feedback. Google, Microsoft, Apple, and Amazon are the frontrunners in this field with millions of connected devices all around the world. MIT Technology Review has a compilation of 56 stories about how Voice Assistants can useful and sometimes be spooky.

We screened 12 potential participants and selected 6 participants based on their digital literacy and competency with using any voice assistant. We had equal representation of gender and their proficiency to speak in English. The only shortcoming of this demographic is that all our participants were students between the age group 23 to 35. 

We screened 12 potential participants and selected 6 participants based on their digital literacy and competency with using any voice assistant. We had equal representation of gender and their proficiency to speak in English. The only shortcoming of this demographic is that all our participants were students between the age group 23 to 35. 

We screened 12 potential participants and selected 6 participants based on their digital literacy and competency with using any voice assistant. We had equal representation of gender and their proficiency to speak in English. The only shortcoming of this demographic is that all our participants were students between the age group 23 to 35. 

The Problem statement for our semester-long project is to evaluate a design system that is being used widely on a day-to-day basis. The rubrics to evaluate the system are Consistency, efficiency and End-user satisfaction.

The Problem statement for our semester-long project is to evaluate a design system that is being used widely on a day-to-day basis. The rubrics to evaluate the system are Consistency, efficiency and End-user satisfaction.

The Problem statement for our semester-long project is to evaluate a design system that is being used widely on a day-to-day basis. The rubrics to evaluate the system are Consistency, efficiency and End-user satisfaction.

The Problem statement for our semester-long project is to evaluate a design system that is being used widely on a day-to-day basis. The rubrics to evaluate the system are Consistency, efficiency and End-user satisfaction.

The goal of the study is to critique the usability of Google's Voice Assistant - The Google Assistant, which is widely available in over a billion smartphone devices. We realized that each user's needs and expectations of the assistant differ, hence we concentrated on end-user satisfaction. 

The Problem statement for our semester-long project is to evaluate a design system that is being used widely on a day-to-day basis. The rubrics to evaluate the system are Consistency, efficiency and End-user satisfaction.

The Problem statement for our semester-long project is to evaluate a design system that is being used widely on a day-to-day basis. The rubrics to evaluate the system are Consistency, efficiency and End-user satisfaction.

The Problem statement for our semester-long project is to evaluate a design system that is being used widely on a day-to-day basis. The rubrics to evaluate the system are Consistency, efficiency and End-user satisfaction.

• Analyzed the verbal comments

• Facial expressions

• Reactions

• Analyzed the verbal comments
• Facial expressions
• Reactions

• Analyzed the verbal comments
• Facial expressions
• Reactions

• Analyzed the verbal comments
• Facial expressions
• Reactions

• Completion success rate

• Time-taken to complete

• Likert satisfaction scale

• Completion success rate
• Time-taken to complete
• Likert satisfaction scale

• Completion success rate
• Time-taken to complete
• Likert satisfaction scale

• Completion success rate
• Time-taken to complete
• Likert satisfaction scale

1. Ask Google Assistant to read unread messages from your phone.
2. Send a message to a contact in your phone.
3. Ask the Assistant to show you movies playing around you.
4. Ask the Assistant to book a ticket for a movie that is playing around you.
5. Search the latest news about a celebrity.
6. Add a calendar event for a meeting with your boss tomorrow at 8 am.
7. Add a reminder to check for flights to San Francisco for Saturday in the morning.
8. Ask the Assistant to order food online.
9. Ask the Assistant to detect and translate a foreign language into English for conversation.

1. Ask Google Assistant to read unread messages from your phone.
2. Send a message to a contact in your phone.
3. Ask the Assistant to show you movies playing around you.
4. Ask the Assistant to book a ticket for a movie that is playing around you.
5. Search the latest news about a celebrity.
6. Add a calendar event for a meeting with your boss tomorrow at 8 am.
7. Add a reminder to check for flights to San Francisco for Saturday in the morning.
8. Ask the Assistant to order food online.
9. Ask the Assistant to detect and translate a foreign language into English for conversation.

1. Ask Google Assistant to read unread messages from your phone.
2. Send a message to a contact in your phone.
3. Ask the Assistant to show you movies playing around you.
4. Ask the Assistant to book a ticket for a movie that is playing around you.
5. Search the latest news about a celebrity.
6. Add a calendar event for a meeting with your boss tomorrow at 8 am.
7. Add a reminder to check for flights to San Francisco for Saturday in the morning.
8. Ask the Assistant to order food online.
9. Ask the Assistant to detect and translate a foreign language into English for conversation.

1. Ask Google Assistant to read unread messages from your phone.
2. Send a message to a contact in your phone.
3. Ask the Assistant to show you movies playing around you.
4. Ask the Assistant to book a ticket for a movie that is playing around you.
5. Search the latest news about a celebrity.
6. Add a calendar event for a meeting with your boss tomorrow at 8 am.
7. Add a reminder to check for flights to San Francisco for Saturday in the morning.
8. Ask the Assistant to order food online.
9. Ask the Assistant to detect and translate a foreign language into English for conversation.

We conducted a pilot study on 2 pilot participants to evaluate and improve our study design for better usability experience for our participants and to get better data. The following changes were made to our study design after the pilot study.

1. Simplified the tasks so that participants were able to comprehend the task.
2. Optimized the study time from 45 minutes to 30 minutes.
3. Make sure that the tasks were tailor-made to the scenarios in which they were grouped.

We conducted a pilot study with 2 participants to evaluate and improve our study design for better usability experience for our participants and to get better data. The following changes were made to our study design after the pilot study.

1. Simplified the tasks so that participants were able to comprehend the task.
2. Optimized the study time from 45 minutes to 30 minutes.
3. Make sure that the tasks were tailor-made to the scenarios in which they were grouped.

We conducted a pilot study on 2 pilot participants to evaluate and improve our study design for better usability experience for our participants and to get better data. The following changes were made to our study design after the pilot study.

1. Simplified the tasks so that participants were able to comprehend the task.
2. Optimized the study time from 45 minutes to 30 minutes.
3. Make sure that the tasks were tailor-made to the scenarios in which they were grouped.

We conducted a pilot study on 2 pilot participants to evaluate and improve our study design for better usability experience for our participants and to get better data. The following changes were made to our study design after the pilot study.

1. Simplified the tasks so that participants were able to comprehend the task.
2. Optimized the study time from 45 minutes to 30 minutes.
3. Make sure that the tasks were tailor-made to the scenarios in which they were grouped.

We screened 12 potential participants and selected 6 participants based on their digital literacy and competency with using any voice assistant. We had equal representation of gender and their proficiency to speak in English. The only shortcoming of this demographic is that all our participants were students between the age groups 23 to 35.

We screened 12 potential participants and selected 6 participants based on their digital literacy and competency with using any voice assistant. We had equal representation of gender and their proficiency to speak in English. The only shortcoming of this demographic is that all our participants were students between the age group 23 to 35. 

We screened 12 potential participants and selected 6 participants based on their digital literacy and competency with using any voice assistant. We had equal representation of gender and their proficiency to speak in English. The only shortcoming of this demographic is that all our participants were students between the age group 23 to 35. 

We screened 12 potential participants and selected 6 participants based on their digital literacy and competency with using any voice assistant. We had equal representation of gender and their proficiency to speak in English. The only shortcoming of this demographic is that all our participants were students between the age group 23 to 35. 

In the following carousal,

• The tasks performed by the user are listed.
  
• The various rubrics we used like Average Time-taken, Average user rating, and Average User Emotion.
• The various issues listed or observed by the users are also listed down.
• The dots represent the number of participants with the color indicating if it is successful or not. 
  

In the following carousal,

• The tasks performed by the user are listed.
  
• The various rubrics we used like Average Time-taken, Average user rating, and Average User Emotion.
• The various issues listed or observed by the users are also listed down.
• The dots represent the number of participants with the color indicating if it is successful or not. 
  

In the following carousal,

• The tasks performed by the user are listed.
  
• The various rubrics we used like Average Time-taken, Average user rating, and Average User Emotion.
• The various issues listed or observed by the users are also listed down.
• The dots represent the number of participants with the color indicating if it is successful or not. 
  

In the following carousal,

• The tasks performed by the user are listed.
  
• The various rubrics we used like Average Time-taken, Average user rating, and Average User Emotion.
• The various issues listed or observed by the users are also listed down.
• The dots represent the number of participants with the color indicating if it is successful or not. 
  

To get a better picture of how our users performed, I visualized the quantitative data using Tableau - a data visualization software. Below are the visualizations to help better understand the data we collected.

To get a better picture of how our users performed, I visualized the quantitative data using Tableau - a data visualization software. Below are the visualizations to help better understand the data we collected.

To get a better picture of how our users performed, I visualized the quantitative data using Tableau - a data visualization software. Below are the visualizations to help better understand the data we collected.

To get a better picture of how our users performed, I visualized the quantitative data using Tableau - a data visualization software. Below are the visualizations to help better understand the data we collected.

In most instances, beginners and intermediate users spent more time to perform the task.

Some of the power users spent more time in a certain task because their expectation was different from the assistant's response. So they tried framing the questions with different keywords.   

The above visualization is an animated bar chart relating the time taken to complete the task vs the tasks. The user's time taken for that task are stacked on top of each other. It can be clearly seen that in most instances, beginners and intermediate users spent more time to perform the task.

Some of the power user spent more time in certain task because their expectation was different from the assistant's response. So they tried framing the questions with different keywords.   

The above visualization is an animated bar chart relating the time taken to complete the task vs the tasks. The user's time taken for that task are stacked on top of each other. It can be clearly seen that in most instances, beginners and intermediate users spent more time to perform the task.

Some of the power user spent more time in certain task because their expectation was different from the assistant's response. So they tried framing the questions with different keywords.   

The above visualization is an animated bar chart relating the time taken to complete the task vs the tasks. The user's time taken for that task are stacked on top of each other. It can be clearly seen that in most instances, beginners and intermediate users spent more time to perform the task.

Some of the power user spent more time in certain task because their expectation was different from the assistant's response. So they tried framing the questions with different keywords.   

It was strange to see that irrespective of the user's competency, the rating (y-axis) was almost the same for the tasks (x-axis). This prompted the team to probe more into this interesting finding. The participants gave a unifying answer that they are okay if the voice assistant does not provide accurate answers since they had low expectations.

It was strange to see that irrespective of the user's compentancy, the rating (y-axis) was almost the same for the tasks (x-axis). This prompted the team to probe more into this interesting finding. The participants gave a unifying answer that they are okay if the voice assistant does not provide accurate answers and that they have low expectations.

It was strange to see that irrespective of the user's compentancy, the rating (y-axis) was almost the same for the tasks (x-axis). This prompted the team to probe more into this interesting finding. The participants gave a unifying answer that they are okay if the voice assistant does not provide accurate answers and that they have low expectations.

It was strange to see that irrespective of the user's compentancy, the rating (y-axis) was almost the same for the tasks (x-axis). This prompted the team to probe more into this interesting finding. The participants gave a unifying answer that they are okay if the voice assistant does not provide accurate answers and that they have low expectations.

We investigated the various issues our participants reported and found a reoccurring pattern both within the task and across tasks. This chart shows the Frequency of issues (y-axis) and the tasks (x-axis).

The highlighted section contains the issues that occur more than once per task or across tasks. 

We investigated the various issues our participant reported and found a reoccuring pattern both within the task and across tasks. The above chart shows the Frequency of issues (y-axis) and the tasks (x-axis).

We investigated the various issues our participant reported and found a reoccuring pattern both within the task and across tasks. The above chart shows the Frequency of issues (y-axis) and the tasks (x-axis).

We investigated the various issues our participant reported and found a reoccuring pattern both within the task and across tasks. The above chart shows the Frequency of issues (y-axis) and the tasks (x-axis).

• Train the assistant on languages so that it can recognize the user’s pronunciation and colloquial language more accurately.
• Enhance the ability for continued conversation by incorporating more trigger words for a follow-up response. (Like yes, no, stop).
• Understand the context and react to the user’s request in a smart and informed way.
• Give the user the freedom to choose just voice interaction or visual interaction.
• Add a function that automatically provides some choices of what it thinks the user is trying to convey and let the user pick from those options if the users fail to ask the question more than 2-3 times.
• A clear distinction between “Events” and “Reminders”.
• Improve the food ordering mechanism.

1. Train the assistant on languages so that it can recognize the user’s pronunciation and colloquial language more accurately.
2. Enhance the ability for continued conversation by incorporating more trigger words for a follow up response. (Like yes, no, stop).
3. Understand the context and react to the user’s request in a smart and informed way.
4. Give the user freedom to choose just voice interaction or visual interaction.
5. Add a function that automatically provides some choices of what it thinks the user is trying to convey and let the user pick from those options, if the users fails to ask the question more than 2-3 times.
6. Clear distinction between “Events” and “Reminders”.
7. Improve the food ordering mechanism.

1. Train the assistant on languages so that it can recognize the user’s pronunciation and colloquial language more accurately.
2. Enhance the ability for continued conversation by incorporating more trigger words for a follow up response. (Like yes, no, stop).
3. Understand the context and react to the user’s request in a smart and informed way.
4. Give the user freedom to choose just voice interaction or visual interaction.
5. Add a function that automatically provides some choices of what it thinks the user is trying to convey and let the user pick from those options, if the users fails to ask the question more than 2-3 times.
6. Clear distinction between “Events” and “Reminders”.
7. Improve the food ordering mechanism.

1. Train the assistant on languages so that it can recognize the user’s pronunciation and colloquial language more accurately.
2. Enhance the ability for continued conversation by incorporating more trigger words for a follow up response. (Like yes, no, stop).
3. Understand the context and react to the user’s request in a smart and informed way.
4. Give the user freedom to choose just voice interaction or visual interaction.
5. Add a function that automatically provides some choices of what it thinks the user is trying to convey and let the user pick from those options, if the users fails to ask the question more than 2-3 times.
6. Clear distinction between “Events” and “Reminders”.
7. Improve the food ordering mechanism.

• Privacy and security concerns - “I don’t want Google to read my messages”
• Language support - “It doesn’t always get what I am saying”
• Discovery and retention - “I didn’t know that this feature exists. It's cool”

1.Privacy and security concerns - “I don’t want Google to read my messages”
2.Language support - “It doesn’t always get what I am saying”
3.Discovery and retention - “I didn’t know that this feature exists. Its cool ”

1.Privacy and security concerns - “I don’t want Google to read my messages”
2.Language support - “It doesn’t always get what I am saying”
3.Discovery and retention - “I didn’t know that this feature exists. Its cool ”

1.Privacy and security concerns - “I don’t want Google to read my messages”
2.Language support - “It doesn’t always get what I am saying"
3.Discovery and retention - “I didn’t know that this feature exists. Its cool ”

• The test was conducted on a small set of users.
• Google Assistant faced trouble to recognize voice profiles for different people and tailor the responses accordingly as they were not using their own device for the study.
• The users were biased while rating each task based on the feature they discovered.

• The test was conducted on a small set of users.
• Google Assistant faced trouble to recognise voice profiles for different people, and tailor the responses accordingly as they were not using their own device for the study
• The users were biased while rating each task based on the feature they discovered

• The test was conducted on a small set of users.
• Google Assistant faced trouble to recognise voice profiles for different people, and tailor the responses accordingly as they were not using their own device for the study
• The users were biased while rating each task based on the feature they discovered

• The test was conducted on a small set of users.
• Google Assistant faced trouble to recognise voice profiles for different people, and tailor the responses accordingly as they were not using their own device for the study
• The users were biased while rating each task based on the feature they discovered

For the widespread implementation of VUI, machines should enhance the user experience by,

• Comprehending context.
• Understanding the user’s tone, attitude, and intention.
• Personalize the user experience based on historical data and the observation of previous patterns of behavior.

For widespread implementation of VUI, machines should enhance the user experience by

• Comprehending context
• Understanding the user’s tone, attitude, and intention
• Personalize the user experience based on historical data and the observation of previous patterns of behaviour

For widespread implementation of VUI, machines should enhance the user experience by

• Comprehending context
• Understanding the user’s tone, attitude, and intention
• Personalize the user experience based on historical data and the observation of previous patterns of behaviour

For widespread implementation of VUI, machines should enhance the user experience by

• Comprehending context
• Understanding the user’s tone, attitude, and intention
• Personalize the user experience based on historical data and the observation of previous patterns of behaviour