Here's a basic form you can test. After you're done, you can scroll down and compare it to
the list of bugs and usability issues further below.
Gear & Gadgets Wish List
What fun things are you wanting these days?
Item Listing
Qty
Price
Description
Feel free to delete the examples above.
Which bugs did you find?
I hope you found at least a few issues. Here are some you might have noticed with normal functional testing. (BTW, this is not intended to be a complete list and doesn't include details that you would normally enter in an issue-tracking system.)
Validation messages not cleared out The validation messages don't go away after the correction is made, even after the user successfully
submits the item. This will likely frustrate and confuse the user. (The only workaround is to refresh the
page.) Here are two approaches to take:
a) Ideally it would be helpful to clear the message as soon as the user corrects the problem.
b) If that's not workable, it would still help a lot if the message was cleared out after a successful
submission.
(TEST TIP: Even though "A" is preferable, you might need to go with "B" if this issue is entered late.
This is where clear mockups and requirements can really help get the best results.)
Quantity selector blank after first use The Quantity dropdown's default value stops showing after the first time you submit an entry. Although
it's easy to work around, it requires an extra step and makes it more likely the user will forget to set
it.
Fixing this problem would also take care of the misspelling in the empty quantity validation message
(see further below).
Price validation message gives wrong reason The Price validation message is misleading for numbers with 11 digits or more (and in some cases 10 digits). The message incorrectly says "The price must be a whole number" even
though it's actually because of the long number.
It would be simpler to put a maxlength limit in the field (to avoid the problem) or change the validation message to something like "The price cannot exceed 10 characters" (to make it more accurate).
TEST TIP: Giving 1 example demonstrates there's a problem. But giving 2 examples provides clues to the developer about the nature of the problem. And giving a third example speeds up the process of fully verifying the bug fix.
Need asterisk for Description field The Description field should have a red asterisk (*) to indicate it's required.
Description's placeholder text says "Max 30" instead of 25 The Description field's gray placeholder text says "Max 30 characters" whereas the input limit is
actually 25 characters. Not sure which one you want to fix:
a) Change the placeholder text to say "Max 25 characters"
b) Change the input field limit to be 30
Typo in Description message: "necesssary" In the Description validation message, it says "necesssary" instead of "necessary".
Full msg: "It is necesssary to enter a description."
Typo "quanity" in Quantity validation message In the Quantity validation message, it says "quanity" instead of "quantity".
Full msg: "Please enter a valid quanity."
TEST TIP: I try to enter the full error message for a couple reasons. It can make it easier for the
developer to find it in the code. Also, it makes it easier to verify the fix (because once the typo is
fixed, it can sometimes be difficult to know for sure which message it was in.)
Need Price field in currency format Would like the Price field to allow decimals and be shown in currency format. There would be 3 changes
to make for this:
1) Change the Price input field to allow numbers with or without a decimal (such as 0, .5, 1.05, 3,
etc.).
2) Convert the prices to the thousandths place (2 places to the right of the decimal). For example,
4.5 would get converted to $4.50, and 4.555 would get converted to $4.56.
3) Change the validation message so it doesn't say a whole number is required (something like "The
price must be a positive number or 0.").
Dollar sign for Price column In the Price column, it would be helpful to show a dollar sign ($) for each of the prices.
Right alignment for Qty and Price columns Would like the Qty and Price columns to be right-aligned. For example, in the Price column, this would
make the decimals line up.
JavaScript error when typing in Description field Consistently getting a JavaScript error when typing in the Description field.
Error: Uncaught ReferenceError: useTitleCase is not defined
at HTMLInputElement.onkeyup
TEST TIP: You can see this by looking in the Console (Ctrl+Shift+I in Chrome and choose Console).
Number 42 missing in Quantity selector The Quantity dropdown is missing the number "42". (Sorry, I couldn't resist.) ;)
Wrong item deleted if two entries have the same description If you have two entries (line items) with the same description and you try deleting the second one, it will actually delete the first one instead. For example, let's say you have two items with the description "Test":
- First item: Qty: 1, Price: $4, Description: Test
- Second item: Qty: 3, Price: $5, Description: Test
If you try to delete the second one, the first one is actually deleted.
TEST TIP: Developers really appreciate getting more detail on issues like this. Although it takes a little more time to enter the issue, it saves time for the developers who have to repro and fix it, for the POs who have to prioritize it, and for you (or other QA) who need to verify the fix. For this issue, I would take video or screenshots (marked up with circles and arrows if needed). Another way to dig deeper is to look under the hood (such as in the database or logs). In this example, if you have Console open (Ctrl+Shift+I in Chrome) you can see the "id" of each item added, and then id of the item that was actually deleted.
When entering bugs into an issue-tracking system, such as Jira, it's important to include more detail, such as the priority, severity, steps to reproduce, browsers, versions, screenshots, etc.
It helps to include what the desired behavior is with specific examples. It's sometimes necessary to get input from the PM if the desired behavior is unclear/subjective or if it's extensive and has multiple ways to fix it. Also, if there are a lot of open issues, it can be helpful to make them more searchable by including synonyms (such as "typos" and "misspellings", "dropdown" and "selector",
as well as "asterisk" and "*").
Usability Issues and Suggestions
Besides the bugs listed above, you also might have noticed some usability issues. The QA team is in a good
position to look for and offer suggestions for usability problems. If caught early enough, they can sometimes
be implemented in the current sprint. But the key to this is to provide the input as early as possible, such
as by scrutinizing any mockups you're given access to (which you need to do anyway to plan different use
cases). Also, it's a very collaborative process, so make sure to keep an open mind and positive approach.
Everyone is doing their best, and it's incredibly difficult to anticipate every potential problem.
Usability Suggestions
Show validation messages for each field
When time allows, it would be helpful to the user if the validations could be changed so they show
the messages for each field individually. The problem for the user is that only one message is shown
at a time.
Example: If the user leaves both an invalid price and invalid description, they only get a
message for the invalid price. They don't get the msg for the description until they try again. (If
the expected behavior/design of the validation error messaging wasn't captured in detail in the
mockups or user stories, then you might save this as a suggestion for the next version.)
Additional fields (totals, etc.)
For the second iteration of this feature, it would be helpful to show some totals, etc. to help with
budgeting:
- Total
- Line-item subtotals
- Input field for entering a budget
TEST TIP: While you're testing this first iteration, the product owner could be planning the next
version and might be open to feedback (as long as it doesn't put this first iteration at risk). If
you're not sure, you could ask beforehand to see if they're open to that kind of input. (That probably
depends on your relationship with the product owner and whether your company is short on product
owners, designers, etc.)
Maybe Not Bug-Worthy
Consistency tweaks
Although consistency is important for usability, the best time to catch problems is before they're
coded. This is very subjective and depends on the PO (product owner), whether we had clarification in
the mockups/requirements ("user stories"), and whether we're short-handed, etc.
For example, whereas I have a bug for the typo in the empty Description message, I didn't enter a
bug for the awkward wording. (It would have been simpler and more consistent to say, "Please enter a
description.") Here's my general approach:
- If the desired messaging was in the mockup/requirements, I'll enter it as a bug.
- If the messaging wasn't in the mockup/requirements, I'll try to get clarification from the PO
before the work is started.
- Depending on the feedback from the PO, some minor tweaks can be deferred until a future
iteration or ignored altogether.
Padding of the Description field is slightly different than Price field
This is an example of a layout quirk that's in a gray area. I would probably enter an issue if: 1)
The design of this page is critical, 2) The front-end engineer is all caught up and looking for
something to do, or 3) The PO, other stakeholders, or front-end engineer want that level of detail.
Since it does look a little odd, I went ahead and fixed it on the login form. But I left it "as
is" here as an example of one that could be ignored for low-visibility pages or entered as a very
low-priority tweak that the PO could bump to another sprint.
Other "look and feel" tweaks
Since this scenario is for a v1 (version 1, first iteration), I would probably leave the remaining
design limitations for now. Most of that depends on what the mockups looked like and what the
priorities are for the product owner.
By the way, when pointing out problems, it's important to use language that's not blame-oriented. We're all
on the same team, so it's helpful to stay solution-focused. It seems like the best way to keep the
collaborative spirit is to avoid metrics on bugs and lean towards using the word issue at least for usability
problems.
As you can guess, there's no perfect way to rate those. But giving an initial rating saves the PO time and
helps developers focus on the important ones without having to sift through a bunch of minor issues.
Here's a link to access a fake login page (which I'm using for other purposes).