

PROJECT OVERVIEW
The University of Tennessee's official app aims to help students, faculty, staff, and campus guests stay connected and informed about any relevant information they might need while at Rocky Top. The usability test goal was to monitor the efficiency and effectiveness of the UTK official app while also surveying user satisfaction. This was done by creating two task-based scenarios that users would have to perform and completing a pre-test and a post-test questionnaire. This helped us gain insight into the user's familiarity with the app and satisfaction and confidence in completing the tasks.
USABILITY TESTING
The University of Tennessee's official app aims to help students, faculty, staff, and campus guests stay connected and informed about any relevant information they might need while at Rocky Top. The usability test goal was to monitor the efficiency and effectiveness of the UTK official app while also surveying user satisfaction. This was done by creating two task-based scenarios that users would have to perform and completing a pre-test and a post-test questionnaire. This helped us gain insight into the user's familiarity with the app and satisfaction and confidence in completing the tasks.

PROBLEM
The University of Tennessee wants to increase the number of students who download and use the school's official app. While the app's idea was good, several usability and functionality problems prevented users from using it successfully. The app is challenging to navigate, and most of the time, users are either unable to find what they're looking for or the process takes much longer than necessary. By understanding the areas that are more prone to usability issues, the University of Tennessee's official app can make changes accordingly.
PURPOSE
This study aimed to monitor the efficiency and effectiveness of the UTK official app. Surveys and task-based scenarios were used to gain insight and analyze the data on student users' efficiency, effectiveness, satisfaction, and confidence with the app.
INTERVIEW STYLE
One-on-one interviews were conducted with volunteer participants. This process took approximately a week to gather participants, schedule, and conduct the interviews.
All participants filled out a consent form to participate in the study.
USABILITY TESTING PROCEDURE
-
Greet participant
-
Have the participant fill out the Pre-Test Questionnaire
-
Reaffirm video consent
-
Begin recording
-
Introductions and breakdown of test
-
Begin usability test
-
Ask first task-based scenario
-
Ask second task-based scenario
-
Ask if the participant has any questions or concerns
-
Thank the participant for participating in the study
-
End recording
-
Have the participant fill out the Post-Test Questionnaire
PARTICIPANTS

Participant 1
Task 1:
Task 2:
45 seconds
39 seconds
21-year-old
Female
Student
Single

Participant 2
Task 1:
Task 2:
115 seconds
43 seconds
21-year-old
Male
Student
Single

Participant 3
Task 1:
Task 2:
35 seconds
29 seconds
22-year-old
Female
Student
Single

Participant 4
Task 1:
Task 2:
55 seconds
36 seconds
22-year-old
Female
Student
Single

Participant 5
Task 1:
Task 2:
28 seconds
29 seconds
22-year-old
Female
Student
Single

Participant 6
Task 1:
Task 2:
30 seconds
25 seconds
22-year-old
Male
Student
Single
TASK-BASED SCENARIO 1:
You are a UTK student, and you want to know more about the group classes and TREC hours. How would you go about doing this on the UTK app?
TASK-BASED SCENARIO 2:
You are a UTK student and want to figure out where your classes are on the campus map. How would you go about doing this on the UTK app?
TASK-BASED SCENARIOS RESULTS
The results from the quantitative data showed that users took longer to complete Task 1 compared to Task 2, with Task 1 averaging 51.34 seconds for participants to complete versus Task 2, which averaged users 33.5 seconds to complete the task. All participants completed the tasks successfully.
POST-TEST QUESTIONNAIRE
Question 1:
Rate the level of difficulty it was for you to complete the task. (1=easy and 5=difficult)
1
Easy
Difficult
1
Easy
Difficult
Easy
2
Difficult
Easy
2
Difficult
Easy
2
Difficult
Easy
2
Difficult
Question 2:
Rate your confidence level in completing the tasks successfully. (1=not confident and 5=very confident)
Not confident
5
Very confident
Not confident
5
Very confident
Not confident
5
Very confident
Not confident
5
Very confident
Not confident
4
Very confident
2
Not confident
Very confident
Question 3:
Rate the likelihood of you using the "Current Students" feature in the app. (1=Very unlikely and 5= Very likely)
Very unlikely
5
Very likely
Very unlikely
5
Very likely
Very unlikely
4
Very likely
Very unlikely
4
Very likely
2
Very unlikely
Very likely
2
Very unlikely
Very likely
Another result was from the participants' ratings on the difficulty level of completing the task and how successful they felt in completing it. For the first task, participants averaged 1.67 on a scale of 1 to 5, with 1 being not difficult at all and 5 being very difficult. Participants' confidence in completing the first task averaged 4.83 on a scale of 1 to 5, with 1 being not confident and 5 being very confident. The second task had similar results, with the first question averaging 1.83 on the level of difficulty. The participant's confidence in completing the second task was slightly lower than the first, with an average of 4.67. Participants also averaged a 3.67 on the likelihood of using the app in the future on a scale from 1 to 5, with 1 being not at all likely and 5 being very likely.