Usability Testing

The goal of usability testing is to get feedback from real users primarily by having users complete a set of tasks within your prototype using a script that you will provide.  This feedback is to ensure that they can understand and use your application and feature as intended as well as to gather suggestions for improvements to the product or features being tested.  If they can’t then it is during this time that you’re keying in to where they got stuck or had questions.

For your script you’re going to want to start with a quick introduction, i.e., “Thank you for coming in today and for giving some feedback.  Today we’d like you to go through this prototype doing a few set tasks to test the product.  We’re not testing you, but the product itself.  If you would please think aloud as you’re completing these tasks so we can gather as much info from you as possible.  You’re not offending me with any commentary you provide.”  We do this so that our participants feel very comfortable and they give us as much honesty as possible so that we aren’t led down the wrong path in developing our products and features. 

After the initial intro you’ll want to get some context around the user’s existing behaviors and workflows.  You’ll want to ask them open-ended questions like who, what, where, when, and why.  These questions will tell you how your user behaves and will structure and adapt the tasks you’re about to give them so that it fits whatever they were doing before.  A few examples of questions you’d want to ask would be:

• When was the last time you did X?
• Can you give me a specific example of what you were doing during that time?

After your intro and introductory questions you’ll then show them the rough prototype.  You’ll explain that the screens you’ve done are just mock ups and they aren’t all perfectly functional and that I’ll point out the times I know something isn’t functional.  From here you’ll make sure to take copious notes, check to make sure they actually completed the task, and you’ll want to understand contextual points.  This could be things like seeing the user get stuck at a certain point, seeing them click somewhere expecting to be taken somewhere, but didn’t.  

Be sure to ask follow up questions.  Asking, “What did you think was going to happen when you X?”  or “Is there any other way you might have tried to do that?” or “What did you expect would happen when you clicked on that button?”.  Commentary from them after these would allow you to take the feedback to improve your product. 

When you’re wrapping up you’re going to want to ask debrief questions.  Questions you could ask would be, “What did you like or dislike about this current prototype?” or “What are some features or functionality you would’ve liked in this prototype?” or “What parts of this page were very important for you?”

An example of a usability test script I’ve created follows here:

Usability Test Script Template

1. Introduction

Take some time to make your participant feel comfortable and use this opportunity to set

some expectations for the usability test.

Items to mention include:

“Thanks for coming in today; we really appreciate your time and your frank feedback. This is an

informal session where we’ll be going through a prototype and we’re not testing you, we’re

testing the product.”

“Please think aloud as you complete these tasks.”

“I didn’t design this product so please be as honest as possible and don’t worry about hurting

my feelings or offending me.”

2. User Context Exploration

Ask open ended questions to get to know your participant and better understand their current

workflows & behaviors in relation to the prototype that you’d like to test. A few examples of

questions include:

“When was the last time you did X?”

“Can you give me a specific example of what you were doing at that time?”

3. Tasks

Here is where you have a series of tasks laid out for your participant to complete. Again, you

want to set expectations that your prototype is not going to be 100% functional. Remember to

take detailed notes when your participant is going through these tasks!

Task 1:  Login as a production manager and create a new project.

Task 2:  Login as a crew member to see how far along John S., a customer, and his project is.  Clock in and then complete the project as a crew member.

Task 3:  Login as a customer.  Check the existing comments and then also add one of your own.

Also remember to follow-up with relevant questions as your participant is going through tasks,

I.e.:

“I noticed you tried to do [x], what did you think was going to happen there?”

“Is there any other way you would have tried to do that?”

“What did you expect would happen when you clicked on that button?”

3. De-brief / Wrap-up

Use the last few minutes to ask high level debriefing questions to get further insights. You

might want to ask questions like:

“What did you like/dislike about this prototype?”

“What are some features or functionality you wish you had here?”

“Which parts of this page were very important to you?”

Wireframing

Wireframing is a cost and time efficient way to showcase your product or features and what it will look like to users, development, and designers so they can get a feel for what to expect.  It is a good idea to start by using the tried and true pen and paper.  This helps to avoid getting stuck on the details when you’re first starting out, i.e. not freaking out about aligning the text, all the boxes are the same width, whether it is responsive on mobile devices, etc.  It also helps to flush out creative ideas for the product. 

When you’re starting create a list of information you want to get across to your users.  What kind of screens will be involved and what type of information will be on those screens that you want the user to know about or act upon.  You’ll want to do this for every single page on your app, i.e. home page, account creation/login page, dashboard page, etc.

Doing this will lead you to an information architecture.  This will help you think about how your end user is going to navigate through your product.  To that end folding a piece of paper twice so that when unfolded you have four boxes you can then draw out quickly four different versions of each screen you may have for your product.  Don’t worry about being perfect with these because the team and designers are likely going to change and or throw out a lot of what you came up with anyways in order to increase aesthetics and speed.  When doing these though aim to create something where you don’t have to make your users think.  Less = More. 

Wireframing tools that you can use would be balsamic or azure.  Interactive prototypes would be like envision app.  After you’ve completed your wireframing and uploaded it into an interactive prototype program or application the next step would be to then sit down with a user to see if your wireframe turned prototype is actually user friendly.  So get to it and see what your users think!

How to Create a User Persona or Avatar

Your company is rockin and rollin.  You’ve got good people in place, your branding is on point, and your core product is sky high.  You want to expand and compliment your team and product with another dynamic product in a new segment that you see opportunity in.  Where do you start?  For starters you need to truly understand your customer and creating a persona or avatar is one way of doing that.  What follows is a structure I use to discover what unique problems your business solves for your customer.  We’ll have a better idea of the kind of story could you tell to help you connect with your customer and what things your products or service do to solve these fears and frustrations.

Now we’re going to build your customer avatar or persona…..

Who is your customer?

Age?  

Gender?

Socioeconomic status = how much money they make?

Spending Habits?   

Personality?  

Name?  

Photo?  

Where does your customer hang out?

Newspapers, magazines, websites?  

Physical locations, conferences, retreats?  

Other businesses they frequent?  

Pains & Problems?

Customer fears?

Pains?

Frustrations?

Hidden costs?  

Financial?

Physical?

Emotional?

Opportunities?

Financial?

Physical?

Emotional?

Small tweaks?

What is a Spec?

A spec is a document that is written to allow people to understand a product’s purpose, functionality, and or features.  It should be comprehensive and understandable so that anyone without any prior context could be able to properly understand the product or feature that is outlined.  A detailed spec won’t solve every possible problem, but it should be written so that you can think through as many possible problems as you can and minimize any surprise situations that could come up.  Once the team starts work you may find that some or many of the things outlined in the spec aren’t able to be implemented and so you need to be flexible for what it will actually turn out to be.

The following is a spec I use in my work as a Product Owner.

Spec Template

Product / Feature Name: What is this product / feature called?

Painting Project Progress

Author Name(s): Who wrote this spec?  

Brock Norton

Team Members: Who are the stakeholders involved in this product / feature? This provides context for other teams reading your spec so they know who they might want to reach out to for further details.

Brock Norton, Renee Norton, Tammy Lasman

Summary / Background:  What is the context of this product / feature? For example, if this is a user-facing product/feature, what is the current situation for your users and at a high level, what will this product / feature functionally do?

This will be a web application that production managers, crew members, and customers can log in to to clock in and out on a specific project,check the progress of a project being completed so that scheduling from the PM and customer standpoint can occur without having to call or text people to see where things are at with the project.

Terminology:  List any terms that you plan to newly introduce with this product / feature. Explain in concise language what these terms mean in relation to this product / feature. If you can avoid it, try not to introduce too many new terms that might confuse other team members.

N/A

Goals: What are the goals of launching this product / feature?

Track project progress for scheduling and customer service purposes.

Success Metrics: How will you be measuring success of this product / feature? You may want to list quantitative / qualitative criteria that you will be evaluating. Every product / feature you release should have some sort of measurable success metric!

Eliminate text or call check ins by painters to production manager and subsequently production manager to customer.  Will be able to see if a project came under/over budget timewise with the customer facing portion and budget wise with the production manager as well as with time budget.

Target Audience: Who are the target users for this product / feature? Provide some context as to what they are currently doing or any existing workflows / behaviors. If you have any existing research on user personas, make sure to include that context here. If you need to make any assumptions about your target audience, make note of that here.

Production managers, crew members, customers.  When a painter is assigned a project they clock in under that project, but it doesn’t give them, the production manager, or the customer, any indication of how far along in the project the crew member is.  If they want to find that out the production manager needs to tally up the hours, compare it to the project’s budget, then update the customer accordingly.  Additionally this updating of roadblocks or impediments may be hard to communicate due to phone tag issues between the crew member, PM, and customer whereas there would be a place within the app to upload a picture or string of text to give a status update on the project or any issues within the project.

Requirements:

#	User Need	User Stories	Launch Phase
1	User needs are one of, if not the most important portion of any product / Feature. Prior to even worrying about your user stories or solutions, you should have clearly defined user needs (remember as a PM, you’re a champion for your users and you should be completely in-tune with the needs that your team is building for!) A user need can follow a template like: needs a way to	User stories are short descriptions of a feature told from the perspective of the user who desires the new Capability. A user story generally follows a template like: As a , I want so that Sometimes, a user story may cover a large amount of functionality. In these cases, these stories may be referred to as ‘epics’ and may need to broken out into individual, smaller user stories.	While it’s important to be thinking of every possible user need /story from the get-go, the reality of development is that without unlimited resources, you’ll generally be time-boxed and therefore will need to prioritize which user stories are important for an initial MVP launch vs. a V2 launch. This column section will specify which phase this story should launch in.
2	Crew member/production manager/customer needs a way to quickly and easily see how long till a project is completed for scheduling purposes.	As a crew member, I want to see what my hours for the day did to impact the overall progress on the project so that I can see if I’m over or under budget to earn bonuses for coming under budget on a project.	MVP
3	needs a way to	As a production manager, I want to see end of day progress on a project and any comments on issues for the day so that I can resolve issues and update the customer if needed without having to play phone tag via call or text with the crew member to gather said information.	MVP
4		As a customer, I want to log in to the app to see how far along the project is to being completed so that I can plan for my next project, vacation, family visiting, etc.	MVP

User Story Details:

1. User Story

As mentioned in the previous section, sometimes a user story may cover a large

amount of functionality. In these cases, these stories may be referred to as ‘epics’ and may need to broken out into individual, smaller user stories.

You may want to include this details section so that you can take the time to provide any extra context / details for the user story and/or use this section to break the epic into smaller individual stories.

Along with the extra context / details, you can also include “user story conditions” that explain scenarios you want to ensure the user story is covering. For example, if your user story was “As a user, I want a credit card drop-down option so that I can select which credit card I want to pay with,” then you might have user story conditions that say something like: “The credit card drop-down options should include: Amex,Mastercard, Visa.”

Additionally, you may want to include any screenshots / wireframes / design

presentations that can help provide visual context for the user story. This helps guide the reader into better understanding the functional use of this particular story.

User can log in via Facebook, twitter, or google.

      2.  User Story

Write details about User Story 2.

Crew member can clock in and out on a project where the production manager has set the hours budgeted for a project.

      3.  User Story

            Write details about User Story 3.

           

Customer can login to see project progress at the end of each day.

Designs: If you plan on handing off this spec for a designer, you’ll want to consider adding a timeline of what deliverables you expect to receive from the designer. If you have final designs / prototype already for the product / feature, feel free to include or link to them in this section.

Sprint one design should have wireframing and prototype design completed.  Web portal design should be completed by sprint two.

Legal: If there are any potential legal issues / implications, you’ll want to include any relevant context / details in this section. I.e. perhaps your product / feature involves collecting sensitive user data and your legal team has specified that this data needs to be held on record in a database for a minimum time period.

N/A

Launch / Roll-Out Plan & Timeline: What are the logistics and timeline of this launch? You may want to have a separate launch checklist that you link to in this section. If you only plan to roll-out this product / feature to a select group of users, make sure to detail that in this section.

One month from initial sprint planning all wireframing, prototyping should be completed.  Users should be able to login to clock in and out during this initial sprint.  Sprint two should have the ability for the production manager to set the hours on a project and crew members can clock in and out on said project.  Sprint three will have a spot for the crew member to add comments or pictures of issues on the project.