Goals for 2020 and 2019 recap

Last year Ashlee and I made one goal really that I can remember because we talked about it and looking through my old blog posts I couldn't find anything about goals for 2019.  The one goal we did have was that we wanted to make $200,000 in 2019.  We came up just short at $175,408.40.  A good chunk of that came from my excellent job, but we also did a tax free cash out refi of our home, and then of course we've been doing flips of houses for the last year to make up the rest.  Pretty cool that we got that close and unless we change the doing of flips from a hobby/business to a legit business I don't see that repeating itself in that exact same format.

A buddy of mine and I are going to try and really see if we can't make doing multifamily investing here in SLC work.  It'd be soooooo awesome if we could so we're going to start working on that.  We need to build up our team, i.e. property manager, banker, brokers for off market deals, and contractors.  We're going to start looking for them and inviting them to present at our meetup we've been holding for the past year.

My brother introduced me to process goals instead of goals several years ago and so I'm not really sure I have firm goals for the coming year, but want to continue the meetup, mastermind groups, and reading on real estate.  We're going to keep doing flips.

On the flip front we want to work on getting a heloc setup on my mom's properties so that when we do flips we aren't paying so much interest on the hard money and that'll save us/make us thousands more on the exact same deals as compared to last year working on those homes with hard money.  We're also going to partner with my mom and have her get her realtor's license so that we can do more upfront quick analysis on comps of potential flips in the future to help mitigate picking up one that might not sell for what we were hoping for in the end and beginning.

This coming year Ash and I do have the goal to have a great room remodeled in the back of our house.  So we gotta start saving for that and begin clearing out the existing space and figure out where to put what we have in there elsewhere.

I think that is most everything.  Next year we want to take the kids to disneyland so we're gonna work on that then :)

Big user stories

Sometimes as a product owner you run into the problem of having to deliver a large piece of a project.  As mentioned on a previous post you want to avoid going to technical and constraining a dev team to follow what you’ve written when instead a story is the beginning of a conversation and isn’t a commitment to exactly how something should be done.  With a user story though you don’t want it to be too big and so you need to break it down into manageable parts. 

If for example you need your team to create a new micro-service to process requests for information you don’t want to write a story that doesn’t create any value once completed.  An example of this would be writing a story that addresses the need to create a database for a user to send, receive, and update information from.  Instead you’d write a story that starts perhaps with the first part of the process, i.e. send data.  With this broken out into its own story it should be discussed with the dev team that while we know that doesn’t encompass creating a database, receiving info, or updating info they will inherently know or be explicitly told that a database needs to be created in order to complete this story.  It should be noted that the very first story will also be front loaded with initial set up and configuration of other things and so it may be larger than subsequent stories.  That is ok though because we want a story to be able to stand on its own in terms of value being delivered even if it is only one portion of data being worked on, in this case sending information. 

What to do and not do when writing a user story

When writing your story don’t include the “How” it should be done.  That is up to the dev team to figure out the best way to do that and then create tasks/sub tasks to complete those things.  Dipping your toe in the “How” waters can lead to constraining a story to follow what you have written.  You also run the risk of getting too involved with the details of how the story and value is to be delivered and that can lead you to spending time on things that don’t need your attention, which then detracts from continuing to gather requirements and prioritize which valuable things need to be done first.

When writing acceptance criteria, it is important to write them so that you tell the developer and QA resources what you want them to test against.  All acceptance criteria should be testable.  A bad example would be, “X technology passes a flag to X other technology so that a customer doesn’t have tax withheld from their order”.  Instead a better example would be, “Demonstrate that a customer can click a button to then have taxes withheld from their order.”  The first example is too specific and limiting.  The second lets the dev team decide how to make something happen but is easy to demonstrate that it works or not, i.e. if it was done right this would be the result.

5 Whys

As I dive deeper into my role as a product owner, I am constantly looking for ways to improve to make things easier for my stake holders, coworkers, and dev team.  One of the things I’ve picked up when writing a user story is to utilize what is called the “5 Whys”. 

To help clarify a description of a story and the value to be delivered, which is then subsequently distilled down to acceptance criteria that a dev codes to and a QA resource tests against, asking “Why” leads the writer of the story to really drill down to what the value is that needs to be delivered.  It helps clarify unknown requirements at times and makes a story concise and easy to understand.

Professional business and real estate experience that has prepared and qualified me to be an apartment syndicator

I wanted to write out a post about my career in business and investing to recap my background and experience that fits well with moving into apartment syndication. Enjoy!

I’ve successfully partnered on two separate painting businesses. Both became licensed despite the first owner not securing a license for over 40 years and the second not having a journeyman painter in the company to start.  The first business hadn’t ever realized it's potential until we partnered and re-positioned it. It hadn’t ever done more than $60,000 in revenue and by the end of the first year of partnership we’d done over $113,000 in revenue. The following year we were a few hundred dollars short of $500,000 in gross revenue.  Unfortunately the owner of that first partnership passed away and so the business closed down. I started my own painting business in another partnership and despite not having a 40 year track history and customer base to work off of we did over $300,000 in revenue in our first year while also landing big commercial clients like Costco, Walmart, and the VA, which we hadn’t done previously in the other company.  We also implemented an exhaustive intake and retention marketing campaign that utilizes email marketing campaigns, live answering services, and videos that introduce us to our customers. This has been systematized and would be similar to how we’d operate updating investors on the status and progress of an investment as well as revamp the intake process for new tenants seeking to rent from us. I’ve since trained up the right resources and put systems in place to turn that business into passive income where it takes very little of my time now, but still spits off reliable income each and every month.

During college for my internship I managed an 82 unit apartment complex.  While initially there was a learning curve my key take away from my experience there was that it wasn’t as hard and daunting as I first thought it would be. It is also about providing good customer service, and I learned that it is about systems and just following those systems and being firm, but fair when enforcing rules, policies, and evictions.  In fact the times that the owner was lenient on late payments or continued noise problems more often than not those tenants and units ended up being the most trashed and payments weren’t collected or delayed in collecting through collections services. That experience was a huge confidence builder in demystifying what it takes to run a top notch property, that was large, run well, and provided great housing for people.  It gave me experience and the direct knowledge of good and bad practices when it comes to owning an apartment complex. I knew after finishing my internship that this was something I was going to do later and be more successful at than my competition because of my experience there.

I’ve also consistently shown results in my professional career and been promoted several times in the tech industry despite not having a degree in computer science.  I did that through hard work, boot camps, courses, earning my MBA, ongoing meetups, and certifications. Currently as a Product Owner I help the team know what to work on next, resolve roadblocks, find ways to improve the team and output, and communicate directly to my stakeholders on the progress of projects and any issues we’re having.  If they need to be resolved I am forecasting those ahead of time in order to bring on additional personnel resources or software/hardware resources. This of course is similar to communicating with investors on the progress of projects, forecasting capital and human resources to turn over vacant properties and upgrade them as well as giving progress reports monthly or quarterly to investors via email newsletter.  

Separate from apartment syndication I’ve helped raise over a quarter million dollars for doing flips.  This is partially from my own funds, but also through lines of credit for businesses, HELOCs, credit cards, and three outside partners.  The very first flip that we completed was hands on, but profited well above projections at a gross profit of $34,000 before taxes.  During the first flip we realized that we wanted to scale the business and so we rolled profits from that first flip into the second. We hired a general contractor that we vetted after interviewing several qualified general contractors.  He has been great to work with, but as we projected he is faster and ahead of schedule, cheaper, and doing better quality work than what we did on our first flip. This has largely taken the partnership out of the day to day and I am almost completely passive in this venture now as well.

I also run my own real estate mastermind group once a week.  In this meeting we follow a format to update the group on what we’re working on, any problems we have that the group can resolve, and set goals and hold each other accountable for said goals.  This group has been insightful for different niches of the real estate investing sector and learning things about said niches. Being accountable and taking action is the core of the meeting and will be beneficial to myself and others going forward.

I am enrolled in a syndication school, am voraciously reading blogs, books, and listening to podcasts that are strictly related to syndication all of which will lead to me acting as General Partner on an apartment syndication.

Product Design Question

“Walk me through how you would design X product.”  This could be asked with, “Walk me through how you might design an alarm clock for someone who is blind.  Walk me through how you might design a better wallet.  Walk me through how you might design an elevator for handicapped individuals.”  

This is a question you could be asked by a client, internal or external, in your journey to bringing a new product to market and we will discuss how you’d answer it in a concise, but productive way.  What you’ll want to do is have a framework in mind.  The framework would be as follows.

1. Ask clarifying questions, i.e. who might the wallet be used by? And what does better mean in the context of a wallet?
2. Communicate your answer outline, i.e. “Now that I understand the scope of this project I’d like to layout how I’d approach the design of this project.”  

          a.  First I’m going to reiterate what my business goals are. i.e. revenue, adoption, better conversion, etc.

          b.  Second I’ll identify my customer base and their use cases.

          c.  Third I’m going to brainstorm some features and evaluate the features against the business goals listed.  

          d.  Lastly I’ll discuss tradeoffs and summarize my recommendation.  

3. Identify the users and their use cases.  Could draw on whiteboard two columns.  Users on left hand side and use cases on right.  After listing the various users you’d ask which of the users the interviewer would like to focus on.  
4. Identify gaps in their use cases or room for improvement.  Could create a third column to list these. 
5. Brainstorm features and improvements.  Brainstorm solutions/features that address the gaps.  Ask the client if you’re on the right track or not. 
6. Prioritize and identify trade-offs of each feature/improvement.  Could be revenue generating vs time and cost to develop it.  Or by customer satisfaction.  Walk through pros and cons of each feature/improvement so you show you’re thinking about all facets of product.  
7. Summarize your recommendation.  

This is a brief, but effective way to quickly walk a client through how you’d design something for a product they need released to the market.

Create a New Product

If you are a Product Owner chances are you're not the one that will be coming up with a new product that the company is going to create.  Hypothetically though say you are at a startup and they want you to create additional revenue streams and so you’ll need to identify and validate a problem.  I’d recommend using a whiteboard to map out a structure or outline of how you’d find users, what questions you’d ask, and how I’d document their needs.  What follows is a framework you could use to do so.

1. Discuss prioritization methods.  What prioritization frameworks have you used in the past, i.e. rev vs time and cost to develop vs cust satisfaction or adoption.
2. Walk through how you would document requirements with users interviews and feedback and provide wireframes with tools like Lucidchart, Balsamiq, Axure, etc.  If you write specs take some time to walk through how you write them. 
3. Explain how you’d work with other teams to build the product.  Explain using agile (MVP and quick iteration) vs waterfall (planning for all possible outcomes, costs, features, etc up front) for the product being developed.  Walk through execution process and how you’d work with teams like engineering and design.  Walk through how you’d engage with the design team.  Do you like to provide more or less guidance?  How often do you check in with designers?  Explain how you’d work with engineering to cost and assess the technical designs that are proposed as well as how you would test the product to ensure product quality.  
4. Discuss launch plan and how you would track success.  Would you be conducting a beta test with a limited number of users?  Define which metrics you’d be tracking to measure success. 

This is a brief, but effective way to quickly walk a client through how you’d create a new product they need released to the market.

Improve an Existing Product

A core responsibility of a Product Owner is to always be improving and iterating on a product by adding value to it and its users.  If a client asks you how you’d improve an existing product here is a simple framework you could follow.

1. Discuss the product’s objective - keep it high level.
2. Figure out what the product needs to reach its objectives.  Here are a list of questions you’d start with and then drill down from there based on the answers you received.

           a.  What have you noticed about the product on its way to reaching its objective?  What has it been successful or not successful at doing? 

                 i.  Think acquisition funnel = acquisition, activation, retention, revenue, and referral.  

           b.  Does the product seem to have a hard time acquiring or activating users?  Does the product have features that encourage users to stick around for a long time?  Does the product have enough drivers that could incentivize users to convert into paying customers?  Is there a strong reason for the product to have users refer or recommend it to other users? 

3. Brainstorm improvements for these needs.  Which of the needs is the highest priority for the product and come up with ideas using that priority.  Or ask which need the interviewer needs you to focus on.  Explain your own prioritization first.  With ideas for improvements come up with short and long term implementations.  
4. Lay out your execution plan and success criteria.  With each improvement discuss the pros and cons or costs and benefits of each.  Explore how you’d roll out the execution plan in the short, mid, and long term with your improvement ideas and how you’d launch them.  Short = increasing revenue by improving the completion rate of purchases then briefly discuss how you might conduct split tests around the shopping cart experience.  Explain success metrics for each idea, i.e. for shopping cart is a 10% increase in purchase completion.  Additionally you’ll want to list out the metrics you plan to track in order to confirm the success of this improvement.

This is a brief, but effective way to quickly walk a client through how you’d improve a product they need released to the market.

Favorite Product Question Walkthrough

If you’re interviewing for Product Owner type positions it won’t be uncommon to be asked what your favorite product is.  Resist the urge to pick the latest iPhone or app if you can because the interviewer will already have heard that answer a half dozen times up to that point of calling through a list of 12 candidates over the phone or perhaps even in person.  Instead try and think of something that is unique and that you can actually speak to because you’ve used it.  Bonus points would be to use something that is relevant to the company you’re interviewing with.  What follows is a general framework of how you’d answer that question and how I’d answer it.

1. What is the product and what does it do?  The product is called PEP, Paint Estimating Program.  Initially in my painting business we were guessing on the cost of each project and then we were trying to come up with our own rates for production.  Then we upgraded to this software that is automated, mobile, and uses the PDCA rates of production when doing measurements.  It also integrates with our quickbooks, time clock software, and scheduling applications.  
2. Who are its competitors and what makes this product better?  Estimate rocket, The Paint Estimator, One-Step Estimating, Planswift, and many online spreadsheets from various contractors.  PEP is better because it already has PDCA standards loaded in, everything is stored in the cloud, it has automated follow up after estimates, syncs with quickbooks, customizable presentations to customers, gives feedback timewise on how long what content was viewed on the bid and in what order for AB testing opportunities, as well as workflow management and scheduling capabilities.  
3. How did you first hear about this product and what keeps you continually using it?  At a painting conference I went to learn more about the industry and how to set myself apart from my competitors and what others around the country who were successful were doing.  I keep using it because the price point is affordable, great support and training was and is excellent, and the product is really easy to use and intuitive once you have the basics down.  It makes us much faster by giving estimates the same day, makes us more professional looking, and takes out the guesswork for various rates of production you might not have exposure to prior to showing up for a certain job.
4. Product features/product strategy - Despite how amazing this product is its ability to integrate with our time clock software and assign crews/painters to projects created.  It needs to add the ability to plan and schedule more than one crew at a time.  That would be a short term implementation that would make a big impact.  Long term letting customers check in on the status of a project via their application would be helpful as well.  

This is a brief, but effective way to quickly walk an interviewer through what your favorite product is right now.

Pricing a Product

So you’ve gone through the rigamarole of coming up with a product that solves a need in the marketplace and you’ve validated your solution with potential users, but now you need to figure out how much to charge people to use said product.  What follows is a framework you could use to try and solve this conundrum internally with your team or by yourself.

1. Ask clarifying questions.  Ask for background information on internal costs and pricing or competitor prices and costs.  
2. Customer’s willingness to pay.  What is the maximum a customer is willing to pay?  BATNA - Best alternative to a negotiated agreement, i.e. if Enterprise charges $40/day to rent a car then say our car rental app is the first of its kind with automated driverless cars coming then we’d price around that.
3. Competitive pricing.  Use alamo, rent-a-car, hertz, etc.  Write out advantages and disadvantages of my product and why it skews my pricing one way or the other.  
4. Cost-based pricing.  Takes into account the cost of the product and adds markup afterwards to come up with the price.  You would tally up the unit cost, literal cost of the item, as well as the overhead cost.  May include rent, shipping, or stocking fees as well.  Unit cost of a rental car would be $15/day.  Overhead costs would be parking, maintenance, insurance add another $10/day for a total cost of $25/day.  Add 40% markup results in a price to users of $35/day for car rental.
5. Maximizing profits is typically the first priority, but others may present themselves as more important.  You may want to maximize market share by lowering the price, which could produce network effects, i.e. product value goes up as more people use that product.  May price product higher to give the product a higher perceived value. 
6. Could use supply/demand curves to also help in coming up with a price.  

This is a brief, but effective way to quickly determine what price you could charge a paying client for using your product.

Product Success Metrics

Your company is considering the release/launch of a new product/feature and they need to know what success looks like with it.  In this walkthrough we’ll use Facebook Stickers as our example.  What follows is a framework you could use to assess the success of a product or feature launch.  

1. What metrics would you look at to evaluate the success of the Facebook Stickers launch?  Start with the AARRR framework.

• A: Acquisition - how you are getting users to your product
• A: Activation - getting users to sign up for service/product, i.e. number of conversions from a reader to someone who actually creates an account
• R: Retention - getting users to engage with product multiple times over a long period of time, i.e. day 1, day 7, day 14, day 60 retention rates
• R: Referral - Measures the virality of the product, how would the user refer this product to anyone else.  I.e. dropbox gave free space to referrers for referring the service to other users. 
• R: Revenue - endpoint of funnel at which user converts into a paying customer.

2. Pick out relevant metrics and why these metrics make more sense to track than others.  Last three of AARRR would be most relevant.  AA wouldn’t be measured cause it doesn’t necessarily affect the signup process or first interacting with fb messenger.  It isn’t necessarily why someone starts to use fb messenger in the first place.  It does keep people around and may generate revenue because they could pay for premium or alternate stickers.      
3. Focus on the trade-offs.  Sign ups may go up by 3%, but session times decrease by 4%.  Ask the company what the current strategic goals are right now?  Launching stickers perhaps made 7 day retention go up, avg. # of messages sent went up, revenue went up by 1%, but referral invites to fb down 2%.  With FB already capturing market share we don’t need more users, but we want more engaged users instead of more referrals.

This is a brief, but effective way to quickly determine what success metrics to use and track for a new product or feature release.

Identifying Characteristics of a Good Developer

As you work on teams of diverse backgrounds and expertise one that stands out is that of developers.  As a product owner it may be that you have to sit in with interviews to see if a developer coming on board would be a good fit or not.  I’ll cover a few things you’re going to want to look for in one.

First would be hustle.  Development often is a practice of creating something out of nothing.  Sometimes the projects or problems won’t have a solution known beforehand.  You’ll want to ask your developer about a time they encountered a problem and how they worked to solve it.  You’ll want to ask:

• What was the challenge?
• What was their approach?
• How quickly did they solve the challenge?
• Did their solution work well?
• How did you prioritize?

That last one is important because while being as perfect as possible is important, having that get in the way of actually delivering a MVP or feature that needs to be released into dev or test environments for validation purposes can obviously be a problem.  You’ll also need to ask them how they work with teams.  Sometimes they may need to code review someone else’s work or contribute during a sprint retrospective.  If they’re a unicorn, i.e. a developer who really can work and create anything from start to finish (front and back end combined as a full stack developer), but don’t like helping others to progress and grow the overall team that may be a problem.  

Lastly you’ll want to assess their code quality.  Ideally their code is cleanly written and so it is easy to update, processes faster, is modular, and has plenty of commenting in it so that someone else could step right in and know what the code and functions are doing without needing to rely on someone else to explain it and then fix it.

Front End vs Back End & the Tech Stack

Front end (client side) refers to the content that someone sees when visiting a site through a browser or a native application.  The primary languages used on the front end are HTML, CSS, and Javascript.  Yes there are many variations on this in terms of languages used, same goes for back end languages, but these are some of the most popular with many additional languages, libraries, and compilers that assist in the speed of either end development.  Front end is typically the bridge between designers and back end developers and is what displays a user content.

Back end (server side) refers to the data storage and manipulation of said data.  It is comprised of working with the web server, database, and application.  Say you want to buy a ticket to a concert.  You navigate to a website and plug in your info (on the front end).  This data is stored in a database on a server.  On the day of the concert you want to print out your ticket, when you do so it is retrieved from the database on the server, fed into the application where you’d then have the option to print it out.

The tech stack is any combination of operating system, web server, database, programming language, and a web framework or native application.

Inbound and Outbound Marketing

Inbound and outbound marketing are two types of marketing that you can use to bring users to you.  Inbound is where your tactics and methods bring the users to you whereas outbound revolves more around buying ads, buying spots on email lists, buying sales leads and then reaching out to those people.  Both are effective at reaching your target audience and we’ll discuss briefly both.

Inbound marketing is here to stay and often comes across as more genuine.  One method for inbound marketing is content marketing.  This doesn’t mean needing to buy an ad, but would require time spent on behalf of the marketing person writing content say for a blog or paying someone to write the content.  This written content would be things like seo, email newsletters, blog posts, whitepapers, and case studies.  This customer progression would look something like: Read some articles/blog posts -> Sign up as a subscriber -> Purchase product.  

Outbound marketing is more of a traditional approach to marketing.  Typical examples of this would be buying ads on Google, Facebook, Twitter, LinkedIn, etc.  Another option would be using affiliate marketing where someone works to sell your product for you and they take a cut of each sale sent to you.  The goal with either channel of marketing, but especially outbound marketing is to make sure that the cost of acquiring the customer is less than the lifetime value of the customer else it doesn’t make sense to continue that paid service.  If you spend $1 to get a customer you want to earn more than that $1 over the life of working with that customer.

Product Roadmap

A product roadmap is a strategic document that helps you communicate your broader product vision and goals to your team, other stakeholders, and other users.  It can help open up a dialogue about the direction the business is being taken in and gather info from customers about what they want and need without giving away top secret details should you choose to publish your roadmap to the world and competitors.

Your product roadmap will not be an exact timeline of development.  It is quite hard to predict releases and launch dates in advance.  Rather than putting down specific dates putting down broader dates in terms of a range, i.e. Q4 of 2019 will see the release of X when you’re listing elements within the product roadmap.  Also labeling or color coding similar product roadmap items will help to paint a broader pictures of the type of things being worked on now, soon, and in the future. 

What follows is an example of a product roadmap I like from ProdPad that illustrates many of the points I made above.

Product Backlog

In a nutshell a product backlog is a prioritized feature list that contains short descriptions of all the functionality desired in that particular feature.  While the PO is in charge of the product backlog not all of the ideas should be flowing from you.  As the PO you’ll be getting ideas from everywhere and it is your responsibility to triage, organize, and prioritize them.  Ideas may come from user feedback, making feature lists around what competitors are doing, or you’ll get feature requests from your own internal teams. 

Users may give ideas to you, but they may not be your customers.  Users may submit ideas via Zendesk tickets, web submission forms, forums like Reddit and others, internal teams/employees or social media.  These submissions may be complaints about the product or processes within it or what they wish they had.  You could build things with word clouds.  

Customers are different from users.  Say your product lies in the educational games arena.  While children are the users the customers end up being the parents buying the product.  

Metrics could also be a way to gather ideas to be added to your product backlog.  This can be any kind of quantitative data that you can get in or from your product.  

It may be important to also list any bugs or any technical work within the backlog.  Prioritizing the list will require consistent communication and interaction with relevant stakeholders so that you are bringing forth the most relevant and valuable features to your product.  

When maintaining a backlog there are many tools you can use, but for those just starting out an easy tool to use is google spreadsheets.  The first column you’d want to add is Date Added.  This will show you when something was added and allow easy sorting for that.  The next column would be Feature Idea Name.  Then after that you’d have the description, i.e. what is overview and desired functionality of this suggested feature. 

If you’re backlog contains a bug there are a few things you’ll want to include when adding it to the backlog.  One you only want one bug report in a story and be sure it has a clear title and proper grammar in the report.  You need simple and repeatable reproduction steps.  Include expected results and observed results.  Screenshots or other pictures of bug in action when user interfaces are involved.  You’ll also want to include the build/version/release where bug was found as not all customers are always on the same release.  

Prioritization criteria will vary by team and company, but it is on the product owner to come to a consensus on how this should look.  Three pretty standard criteria that can be used are impact, urgency, and engineering cost. 

Impact or importance is a way to define in a subjective manner whether or not this particular feature idea would move an important metric.  A high numerical score here using a scale of 1 - 5, with 5 being equal to high importance vs a 1, which would indicate low importance. 

Urgency would be asking the question of how important it is to get this feature out now.  1 being not urgent and 5 being very urgent.  So even though something may be impactful doesn’t necessarily translate into something being urgent.  Maybe users would be saying that something is really affecting their workflow and cutting down on their efficiency and is something that needs to get worked on right away even though this feature might not have the same impact as something else score wise.  

Engineering cost is typically where you’d want to involve someone else, i.e. an engineering manager or a high level developer you’re very close with if you don’t have the technical expertise yourself.  Scoring these stories on a scale of 1 - 5 would have 1 representing something as slow to build (1 month) with 5 being quick to build (1 day). 

Other is another column you could have in your backlog.  There are other potential prioritization criteria to consider outside of the ones already mentioned.  

You’ll want to have a total and status set of columns in your backlog as well.  Total would be a sum of any of the prioritization criteria you gave values to earlier.  Status would detail where the feature is at the moment.  It would include things like whether it needs more definition or to be reviewed even further or to be slotted into a development cycle or sprint.  The target launch date would also be listed in your status column. 

Holding periodic check-ins with relevant stakeholders to prioritize your backlog will ensure you are releasing the most valuable things for them and the company.  

Usability Testing

The goal of usability testing is to get feedback from real users primarily by having users complete a set of tasks within your prototype using a script that you will provide.  This feedback is to ensure that they can understand and use your application and feature as intended as well as to gather suggestions for improvements to the product or features being tested.  If they can’t then it is during this time that you’re keying in to where they got stuck or had questions.

For your script you’re going to want to start with a quick introduction, i.e., “Thank you for coming in today and for giving some feedback.  Today we’d like you to go through this prototype doing a few set tasks to test the product.  We’re not testing you, but the product itself.  If you would please think aloud as you’re completing these tasks so we can gather as much info from you as possible.  You’re not offending me with any commentary you provide.”  We do this so that our participants feel very comfortable and they give us as much honesty as possible so that we aren’t led down the wrong path in developing our products and features. 

After the initial intro you’ll want to get some context around the user’s existing behaviors and workflows.  You’ll want to ask them open-ended questions like who, what, where, when, and why.  These questions will tell you how your user behaves and will structure and adapt the tasks you’re about to give them so that it fits whatever they were doing before.  A few examples of questions you’d want to ask would be:

• When was the last time you did X?
• Can you give me a specific example of what you were doing during that time?

After your intro and introductory questions you’ll then show them the rough prototype.  You’ll explain that the screens you’ve done are just mock ups and they aren’t all perfectly functional and that I’ll point out the times I know something isn’t functional.  From here you’ll make sure to take copious notes, check to make sure they actually completed the task, and you’ll want to understand contextual points.  This could be things like seeing the user get stuck at a certain point, seeing them click somewhere expecting to be taken somewhere, but didn’t.  

Be sure to ask follow up questions.  Asking, “What did you think was going to happen when you X?”  or “Is there any other way you might have tried to do that?” or “What did you expect would happen when you clicked on that button?”.  Commentary from them after these would allow you to take the feedback to improve your product. 

When you’re wrapping up you’re going to want to ask debrief questions.  Questions you could ask would be, “What did you like or dislike about this current prototype?” or “What are some features or functionality you would’ve liked in this prototype?” or “What parts of this page were very important for you?”

An example of a usability test script I’ve created follows here:

Usability Test Script Template

1. Introduction

Take some time to make your participant feel comfortable and use this opportunity to set

some expectations for the usability test.

Items to mention include:

“Thanks for coming in today; we really appreciate your time and your frank feedback. This is an

informal session where we’ll be going through a prototype and we’re not testing you, we’re

testing the product.”

“Please think aloud as you complete these tasks.”

“I didn’t design this product so please be as honest as possible and don’t worry about hurting

my feelings or offending me.”

2. User Context Exploration

Ask open ended questions to get to know your participant and better understand their current

workflows & behaviors in relation to the prototype that you’d like to test. A few examples of

questions include:

“When was the last time you did X?”

“Can you give me a specific example of what you were doing at that time?”

3. Tasks

Here is where you have a series of tasks laid out for your participant to complete. Again, you

want to set expectations that your prototype is not going to be 100% functional. Remember to

take detailed notes when your participant is going through these tasks!

Task 1:  Login as a production manager and create a new project.

Task 2:  Login as a crew member to see how far along John S., a customer, and his project is.  Clock in and then complete the project as a crew member.

Task 3:  Login as a customer.  Check the existing comments and then also add one of your own.

Also remember to follow-up with relevant questions as your participant is going through tasks,

I.e.:

“I noticed you tried to do [x], what did you think was going to happen there?”

“Is there any other way you would have tried to do that?”

“What did you expect would happen when you clicked on that button?”

3. De-brief / Wrap-up

Use the last few minutes to ask high level debriefing questions to get further insights. You

might want to ask questions like:

“What did you like/dislike about this prototype?”

“What are some features or functionality you wish you had here?”

“Which parts of this page were very important to you?”