Visit Harford is a part of Maryland's Destination Marketing Organization. Harford County has seen increasing year-over-year growth in tourism sales and revenue through brick and motor tourism commerce. However, analytics show that the user engagement of their digital platform has not contributed much to their tourism economy. Mr. Gregory Pizzuto, Executive Director, Visit Harford was excited to collaborate with us to understand more about its current audience and increase user engagement of Visit Harford.

This is a group project for my Advanced Usability Testing Course work. I was part of a 3 member team and we wanted to evaluate a system that is very subjective and unexplored. 

This is a group project for my Advanced Usability Testing Course work. I was part of a 3 member team and we wanted to evaluate a system that is very subjective and unexplored. 

This is a group project for my Advanced Usability Testing Course work. I was part of a 3 member team and we wanted to evaluate a system that is very subjective and unexplored. 

I worked along with 3 other UX researchers who were involved in contextual inquiry, recruitment and screening participants and data analysis.  

The Problem statement for our semester-long project is to evaluate a design system that is being used widely on a day-to-day basis. The rubrics to evaluate the system are Consistency, efficiency and End-user satisfaction.

The Problem statement for our semester-long project is to evaluate a design system that is being used widely on a day-to-day basis. The rubrics to evaluate the system are Consistency, efficiency and End-user satisfaction.

The Problem statement for our semester-long project is to evaluate a design system that is being used widely on a day-to-day basis. The rubrics to evaluate the system are Consistency, efficiency and End-user satisfaction.

Mr. Gregory Pizzuto and his team met with my team in the first week of October to discuss their needs and their understanding of the problem. The key concern raised by them were,

• Negligible traffic on their Web platform.
• Convert Business visitors to vacation tourists.
• Harford County is primarily seen as a tourist spot by the visitors who are 35 and above.
• To cater to all age groups.
• Involving local businesses to boost tourism and attractions.

The goal of the study is to critique the usability of Google's Voice Assistant - The Google Assistant, which is widely available in over a billion smartphone devices. We realized that each user's need and expectation of the assistant differ, hence we concentrated on end-user satisfaction. 

The goal of the study is to critique the usability of Google's Voice Assistant - The Google Assistant, which is widely available in over a billion smartphone devices. We realized that each user's need and expectation of the assistant differ, hence we concentrated on end-user satisfaction. 

The goal of the study is to critique the usability of Google's Voice Assistant - The Google Assistant, which is widely available in over a billion smartphone devices. We realized that each user's need and expectation of the assistant differ, hence we concentrated on end-user satisfaction. 

Based on our kick-off meeting, client needs, we came up with 2 main research questions.

1. How do people plan their vacation? 
2. What do they expect out of their vacation?

We screened 12 potential participants and selected 6 participants based on their digital literacy and competency with using any voice assistant. We had equal representation of gender and their proficiency to speak in English. The only shortcoming of this demographic is that all our participants were students between the age group 23 to 35. 

We screened 12 potential participants and selected 6 participants based on their digital literacy and competency with using any voice assistant. We had equal representation of gender and their proficiency to speak in English. The only shortcoming of this demographic is that all our participants were students between the age group 23 to 35. 

We screened 12 potential participants and selected 6 participants based on their digital literacy and competency with using any voice assistant. We had equal representation of gender and their proficiency to speak in English. The only shortcoming of this demographic is that all our participants were students between the age group 23 to 35. 

The Identity model helped to understand how the users view themselves, who they were and how they planned their vacation

The Identity model helped focus on collecting data that uncovers the sources of pride, self-expression, and core values. This model reflects data relevant to one’s Identity. It reveals the key identity elements in the target population that a product might touch. We used this model to understand the view of the users’ world. Identity model helped us in the later stage during product concepts since we designed the solution based on the user’s identity because a user will use the product with which he/she relates him/herself.

To get a better picture of how our users performed, I visualized the quantitative data using Tableau - a data visualization software. Below are the visualizations to help better understand the data we collected.

To get a better picture of how our users performed, I visualized the quantitative data using Tableau - a data visualization software. Below are the visualizations to help better understand the data we collected.

To get a better picture of how our users performed, I visualized the quantitative data using Tableau - a data visualization software. Below are the visualizations to help better understand the data we collected.

The above visualization is an animated bar chart relating the time taken to complete the task vs the tasks. The user's time taken for that task are stacked on top of each other. It can be clearly seen that in most instances, beginners and intermediate users spent more time to perform the task.

Some of the power user spent more time in certain task because their expectation was different from the assistant's response. So they tried framing the questions with different keywords.   

The above visualization is an animated bar chart relating the time taken to complete the task vs the tasks. The user's time taken for that task are stacked on top of each other. It can be clearly seen that in most instances, beginners and intermediate users spent more time to perform the task.

Some of the power user spent more time in certain task because their expectation was different from the assistant's response. So they tried framing the questions with different keywords.   

The above visualization is an animated bar chart relating the time taken to complete the task vs the tasks. The user's time taken for that task are stacked on top of each other. It can be clearly seen that in most instances, beginners and intermediate users spent more time to perform the task.

Some of the power user spent more time in certain task because their expectation was different from the assistant's response. So they tried framing the questions with different keywords.   

Decision Point Model

We investigated the various issues our participant reported and found a reoccuring pattern both within the task and across tasks. The above chart shows the Frequency of issues (y-axis) and the tasks (x-axis).

We investigated the various issues our participant reported and found a reoccuring pattern both within the task and across tasks. The above chart shows the Frequency of issues (y-axis) and the tasks (x-axis).

We investigated the various issues our participant reported and found a reoccuring pattern both within the task and across tasks. The above chart shows the Frequency of issues (y-axis) and the tasks (x-axis).

The decision point model helps designers to understand what influences the user to rely on a service or a product. Designers can leverage these mental models to design intuitive and user-friendly products.

We went ahead with Decision point model because we were interested in knowing what influences the behavior of our user. Decision Point model brings exactly that out clearly. These served as a checkpoint during the conceptualization process.

1. Train the assistant on languages so that it can recognize the user’s pronunciation and colloquial language more accurately.
2. Enhance the ability for continued conversation by incorporating more trigger words for a follow up response. (Like yes, no, stop).
3. Understand the context and react to the user’s request in a smart and informed way.
4. Give the user freedom to choose just voice interaction or visual interaction.
5. Add a function that automatically provides some choices of what it thinks the user is trying to convey and let the user pick from those options, if the users fails to ask the question more than 2-3 times.
6. Clear distinction between “Events” and “Reminders”.
7. Improve the food ordering mechanism.

1. Train the assistant on languages so that it can recognize the user’s pronunciation and colloquial language more accurately.
2. Enhance the ability for continued conversation by incorporating more trigger words for a follow up response. (Like yes, no, stop).
3. Understand the context and react to the user’s request in a smart and informed way.
4. Give the user freedom to choose just voice interaction or visual interaction.
5. Add a function that automatically provides some choices of what it thinks the user is trying to convey and let the user pick from those options, if the users fails to ask the question more than 2-3 times.
6. Clear distinction between “Events” and “Reminders”.
7. Improve the food ordering mechanism.

1. Train the assistant on languages so that it can recognize the user’s pronunciation and colloquial language more accurately.
2. Enhance the ability for continued conversation by incorporating more trigger words for a follow up response. (Like yes, no, stop).
3. Understand the context and react to the user’s request in a smart and informed way.
4. Give the user freedom to choose just voice interaction or visual interaction.
5. Add a function that automatically provides some choices of what it thinks the user is trying to convey and let the user pick from those options, if the users fails to ask the question more than 2-3 times.
6. Clear distinction between “Events” and “Reminders”.
7. Improve the food ordering mechanism.

“The Wall Walk is managed like a visit to an art museum—each person experiences the data silently, having their own experience with it, and writing up design ideas as they go.”

— Karen Holtzblatt in Contextual Design

Wall walks gives the team a chance to review and think about what this story means for the product we hope to design. It’s the team’s first chance to see the whole scope of data together and consider how to respond with a coherent design solution.

As we went around our affinity maps, we came up with a Design Ideas (DI). Once we collated our Design Ideas, we took a vote on these ideas. We call them the Hot Ideas. These will be used in our visioning process.

1.Privacy and security concerns - “I don’t want Google to read my messages”
2.Language support - “It doesn’t always get what I am saying”
3.Discovery and retention - “I didn’t know that this feature exists. Its cool ”

1.Privacy and security concerns - “I don’t want Google to read my messages”
2.Language support - “It doesn’t always get what I am saying”
3.Discovery and retention - “I didn’t know that this feature exists. Its cool ”

1.Privacy and security concerns - “I don’t want Google to read my messages”
2.Language support - “It doesn’t always get what I am saying"
3.Discovery and retention - “I didn’t know that this feature exists. Its cool ”

Keeping in mind all the Hot Ideas we came up with during the wall walk, we collectively went through the various scenarios that a user could encounter. We storyboarded various scenarios to see if our products will serve our users.

• The test was conducted on a small set of users.
• Google Assistant faced trouble to recognise voice profiles for different people, and tailor the responses accordingly as they were not using their own device for the study
• The users were biased while rating each task based on the feature they discovered

• The test was conducted on a small set of users.
• Google Assistant faced trouble to recognise voice profiles for different people, and tailor the responses accordingly as they were not using their own device for the study
• The users were biased while rating each task based on the feature they discovered

• The test was conducted on a small set of users.
• Google Assistant faced trouble to recognise voice profiles for different people, and tailor the responses accordingly as they were not using their own device for the study
• The users were biased while rating each task based on the feature they discovered

These findings were based on our interviews and models. We also came up with design recommendations and suggestions for our client. These are part of the slides we presented during the final presentation.

Use the arrow keys to go through the findings in the carousel.

For widespread implementation of VUI, machines should enhance the user experience by

• Comprehending context
• Understanding the user’s tone, attitude, and intention
• Personalize the user experience based on historical data and the observation of previous patterns of behaviour

For widespread implementation of VUI, machines should enhance the user experience by

• Comprehending context
• Understanding the user’s tone, attitude, and intention
• Personalize the user experience based on historical data and the observation of previous patterns of behaviour

For widespread implementation of VUI, machines should enhance the user experience by

• Comprehending context
• Understanding the user’s tone, attitude, and intention
• Personalize the user experience based on historical data and the observation of previous patterns of behaviour