Lo-fi Prototype and Usability Testing
Introduction
Radr is a location-based, mobile social networking site that allows members to interact with their social network more completely and practically through the use of location. Designed to take advantage of new mobile web devices and platforms like those of the iPhone, Radr allows members to find out friend’s and stranger’s proximity and location, mapping the member’s social network in real time. Customers may also monitor and fine tune privacy and visibility settings to the preferences of their choice. With privacy as a main concern, Radr hopes to provide the easiest and most advanced privacy and visibility systems to make our members feel safe and protected business, academic, and social contexts. We conducted usability testing in hopes of fixing and validating our design choices that have been previously based solely on survey data and our intuition.
Prototype
The lo-fi prototype of our system was constructed primarily by using 3 x 5 inch index cards, including thirteen full-size index cards and ten small movable pieces. Additionally, we made a fake iPhone using a printed picture and a Crayola Markers cardboard box that the index cards could slide into for testing. The ratio of the width and height of the screens are roughly the same as the screen on iPhone. The prototype only contained necessary screens to completing the demo and the three tasks. Because there were multiple ways to complete a task, we prepared as many possible ways for the participants to choose from as we could think of.
The design of the interface is almost identical to the original sketches we made; however, we simulated more extensive functionality by adding popup prompts, drop down menus, and transparent windows that were sketched out and cut into smaller pieces. This also included a popup window that indicated that a function was not implemented. We tried to make all of the buttons look like three-dimensional icons to mimic the look real buttons. At the bottom of each screen, we sketched a toolbar with standard iPhone browser buttons like “back” and “forward.”
During the first and the second tests, we encountered some problems: the participants could not locate certain functions such as the Visibility setting and therefore did not know what to do for the third task. Therefore, we made some changes to the interface for the last participant. First of all, we changed the label on the top left corner from the currently set Visibility “School” to simply say “Visibility.” On the drop down window triggered by this link, we added a large “Done” button and made the original “New” button smaller. Additionally, we added sort of tool tip at the bottom of the screen visibility-editing screen to help users determine what to do next. These subtle UI changes radically improved the usability of our prototype, and helped the third tester complete the tasks with fewer problems.
Method
For our user interface testing, we had a group of participants work with our paper prototype and asked them to complete three tasks, ranging from easy to difficult. Our participants were three potential customers with varying knowledge and experience with computing and technology. Each participant that used our paper prototype could be placed into our target customer group.
For the usability test, we used one of the small private meeting spaces in the basement of the Allen Center. The room we used was just large enough for our team and the participants, and the lack of windows added additional privacy so that the participants could be comfortable throughout the process.
When the participant arrived, one member of our group would greet him or her, and introduce the other members of the team. One member of our team acted as the computer and would change the interface based on the customer’s actions, and the other members acted as observers, noting interesting events. The person who introduced the team also introduced the purpose of our study and explained that the paper prototyping process was an effective way of testing an interface prior to production and implementation, and that although it may seem awkward, is quite informative. The participant was then asked to read and sign the consent form, and was informed that participation is optional and that he or she could stop at any time, for any reason. After asking the participant to remember to think aloud, another member of the team gave a brief demonstration of working with the system. Text Box: The participant’s view of the experiment. The iPhone prototype is on the left, the tasks are laid out in the middle, and consent form is on the right.The demonstration was scripted and covered non-task related features (see Appendix for demo script).
After the demonstration, each task was read to the participant and they were given a chance to ask questions. Our three tasks included finding a friend on a map, looking up information about the people around them (both friends and strangers), and finally, to changing their visibility settings to only allow a select few number of friends view their location. We chose to give our tasks in the order of easiest to hardest, so that the participant would be able to quickly gain confidence with the process and be initially frustrated (see Appendix for tasks).
We specifically avoided using words that appeared on our interface as we attempted to find out how intuitive our design was. After reading the tasks aloud, we provided a written version on note cards for the user to reference, making sure that we did not guide the participant into a specific action through our word choice.
At the end of the test, we would ask follow up questions and answer any questions that the participants had for us. We also asked the participants to give feedback about potential changes to the prototype.
Since none of the participants had seen our system before and the concept is rather new, we wanted to see how intuitive our interface is to a new customer. If the participant struggled in one area or task, it might suggest that there isn’t an intuitive, logical next step, which could be could warrant modifying the paper prototype. By looking at the amount of time it would take a participant to accomplish a task and the order in which he or she would click links, we could determine the confusing, or intuitive, portions of the interface.
Results
Our first task was supposed to be the “easy” tasks and we found that it did indeed take the least amount of time for each participant to complete. Participant 1 was at first confused by the interface and thought the iPhone navigation buttons were a part of our design. After this was clarified for him, he completed the task as we had intended (first looking up the profile of the friend he was trying to locate, then clicking on “Map,” which was linked to from the profile). The second participant had a more crippling misunderstanding with our paper medium that caused her not recognize that the word “Map” was a link; this prevented her from finishing the task without extensive help from us. The last participant chose a different approach to locating his friends by first clicking on “Locations” tab, then by finding his friend by zooming out.
The second task took longer for all participants to complete. Since finding out information about people around was a more unique task than the first one, the participants were forced to explore the system more. The first two participants tried to click on the “Home” tab, which was unimplemented. All three participants tried to click on the “Locations” tab, although only the third participant explored the list of friends that appeared on the screen to complete the task. Interestingly, both of the first two participants began to go down the right path—clicking on the “Friends” tab to find friends nearby—then correctly noticed the “Show Strangers” button, but neglected to push it. When asked afterwards why they didn’t push it even though they recognized it was there, neither could provide conclusive answers. In the end, none of the partipants completed the task the way we had originally intended.
We knew that the last task was going to be very difficult due to the complicated nature of the Visibility functionality. Because of this, we found that each participant needed a bit of a jump start, so the facilitator encouraged them to experiment and really voice their thoughts. Both of the first two partipants had difficulty identifying where to even begin. At the time, they had to click on a drop down that read “School,” the current Visibility. This was so problematic that both the first and second participant eventually had to be told where to click. After that, the first participant understood how the pop up menu functioned and explored the rest of the Visibility functionality in a natural, explorative manner. However, the second participant didn’t quite understand how the pop up menu worked, thereby misunderstanding some elements of the Visibility functionality. Both of the first two participants also did not immediately understand what a region was due to our paper representation of it. The third participant benefitted from some modifications we made between the first and second participants. We changed the word “School” to “Visibility,” created a new representation of a region that we hoped better conveyed its meaning, and added some helpful hints on the Visibility screen to explain functionality. He figured out where to click initially, understood how the pop up menu functioned, and recognized the region and how to manipulate it by asking very few questions.
Discussion
Using a prototype proved to be predictably challenging for us as testers and misleading for our participants. Although most examples of this weren’t particularly crippling (not seeing a scroll bar, not seeing a button to maximize, misreading an arrow as the numeral “7”), there were a few instances that we believe severely hindered our participants and prevented them from completing a task. For instance, we can do little to account for when a participant cannot tell that the crucial link she needs to press is in fact a link.
We also found that participants tended to go to the “Locations” tab for any kind of task that involved finding people, even if they could locate friends more easily using the “Friends” tab. Unfortunately, there was little we could do in the middle of our testing to address this confusion short of re-designing the entire interface. We hope to address this issue by either shifting around some of the contents of our program or by renaming some of our functionality to be more constrictive.
There was a lot of confusion over the text we displayed to indicate the Visibility. We originally thought that it made sense for the current Visibility to be displayed at all times. Unfortunately, the beginner customer didn’t understand what this indicated at all and therefore didn’t even think to click on it to alter their privacy settings. Of course, all participants did think to click on “Settings,” which led us to decide that we need to link to Visibility somewhere inside the “Settings” page.
After seeing some critical problems with the Visibility for the first two participants, we altered some things for our last participant. We changed the displayed text to simply say “Visibility” at all times rather than the current Visibility. We also added a tool tip-like hint on the Visibility editing screen so that customers will know what each of the tools (pointer, hand, or pen) should be used for. We also improved our paper prototype so that a region was more easily identified as being clickable. We believe that these three modifications for our last participant were the driving factors behind his marked improvement over the first two participants.