A. High-Fidelity Prototype:
1. The Prototype:
2a. User Story - Instant Messaging:
"As a concerned and confused student, I would like to message a classmate, within a short time frame, to ask about an assignment that is due in the near future."
With the original instant messaging feature on eLC, we found that many students were unaware of how best to navigate around and would often miss notifications. When trying to quickly contact classmates through the chat, it is crucial for users to intuitively find their messages from any page in the site. To alleviate this pain point within the existing system, we expanded the messages button to display instant messages, and filter them apart from class and email notifications. By utilizing a sliding filter switch, we hoped to sort out the combined list of alerts so that the users could have distinct sections to find exactly what they are searching for. In the context of instant messaging, this could prevent the student from losing their chat messages in a sea of notifications, therefore reducing the time it takes to respond. Many students currently flock to alternative messaging applications such as GroupMe for the ability to quickly view and respond to alerts, allowing for an effective way to instantly contact classmates. With this in mind, we believe the reorganization of the notifications will increase engagement with the eLC instant messaging feature, and provide a more complete communication experience for UGA students.
With the original instant messaging feature on eLC, we found that many students were unaware of how best to navigate around and would often miss notifications. When trying to quickly contact classmates through the chat, it is crucial for users to intuitively find their messages from any page in the site. To alleviate this pain point within the existing system, we expanded the messages button to display instant messages, and filter them apart from class and email notifications. By utilizing a sliding filter switch, we hoped to sort out the combined list of alerts so that the users could have distinct sections to find exactly what they are searching for. In the context of instant messaging, this could prevent the student from losing their chat messages in a sea of notifications, therefore reducing the time it takes to respond. Many students currently flock to alternative messaging applications such as GroupMe for the ability to quickly view and respond to alerts, allowing for an effective way to instantly contact classmates. With this in mind, we believe the reorganization of the notifications will increase engagement with the eLC instant messaging feature, and provide a more complete communication experience for UGA students.
2b. User Story - Personalization & Settings:
"As a student who would like a more personalized visual experience on eLC, I would like to customize my user and display settings so that I can more easily intepret and respond to messages sent through eLC."
One feature we believed was lacking from the present layout of eLC was the option to customize settings relevant to the appearance of instant messages and the general screen display. With the many hours that UGA students spend searching through notifications, our group proposed that we offer extended personalization options for each user to select a view that supports their visual preferences. Within this user story, one primary element was the introduction of a dark mode; with external messaging platforms such as GroupMe and Discord, many users opt for a dark theme for the sole purpose of comfortable viewing. We figured that this would be a simple feature to implement into eLC that could help support a fairly popular preference. Aside from the colors displayed on screen, we also integrated a list of fonts for the user to choose from alongside the already supported font size customization. We hope that the layering of these features will create several combinations of font appearances for the user to best interpret messages on screen. Additionally, we reorganized the notifications tab of the settings menu for a more logical navigation through the page. By implementing buttons that clearly change colors when selected, we hope that users will have an easier time understanding their current notification settings for each section of alerts (ex: Assignments, Content, Discussions, etc.). Our group believes that these changes will give student higher confidence in their ability to control their visual experience on eLC in a way that does not compromise the standard rules of accessibility. In the context of instant messaging, altering the appearance could allow the user to more easily interpret text messages, and arranging the notification settings could allow the user to better adjust which alerts they receive via text and email.
One feature we believed was lacking from the present layout of eLC was the option to customize settings relevant to the appearance of instant messages and the general screen display. With the many hours that UGA students spend searching through notifications, our group proposed that we offer extended personalization options for each user to select a view that supports their visual preferences. Within this user story, one primary element was the introduction of a dark mode; with external messaging platforms such as GroupMe and Discord, many users opt for a dark theme for the sole purpose of comfortable viewing. We figured that this would be a simple feature to implement into eLC that could help support a fairly popular preference. Aside from the colors displayed on screen, we also integrated a list of fonts for the user to choose from alongside the already supported font size customization. We hope that the layering of these features will create several combinations of font appearances for the user to best interpret messages on screen. Additionally, we reorganized the notifications tab of the settings menu for a more logical navigation through the page. By implementing buttons that clearly change colors when selected, we hope that users will have an easier time understanding their current notification settings for each section of alerts (ex: Assignments, Content, Discussions, etc.). Our group believes that these changes will give student higher confidence in their ability to control their visual experience on eLC in a way that does not compromise the standard rules of accessibility. In the context of instant messaging, altering the appearance could allow the user to more easily interpret text messages, and arranging the notification settings could allow the user to better adjust which alerts they receive via text and email.
2c. User Story - Pinning & Marking Messages:
"As one of many students within a class, I would like to easily navigate to pinned and popular messages within a group chat, and respond to individual messages with reactions and replies."
As it currently stands, instant messaging on eLC does not offer many of the standard communication methods that students frequently use on alternative platforms. One such interaction is the ability to pin messages and then easily trace back to those messages; within a class group chat, students should be able to mark certain messages with higher importance to the group. One example of this could be a classmate sharing a link to a communal quizlet, in which the members of the group would appreciate being able to quickly navigate back to the message without scrolling through messages of lesser importance. With this user story, we implemented the feature of pinning messages and a way to filter out only pinned messages within a chat history. Alongside this, our group agreed that users should be able to either react or directly reply to a chat message; with a like feature, students can send their appreciation for specific messages without always crafting an extended response. Each message can then be associated with the number of likes it received, supporting the ability to sort by the most liked messages. Since the current version of eLc does not comfortably support direct replies to chats, we designed a more intuitive instant messaging experience for users, complete with a sidebar that displays all active chats. We implemented a user interface that resembles the external communication apps that UGA students currently use, such as Slack and Discord, as we knew users could easily navigate through those menus. We condensed and reorganized the features that were previously offered to eLC messenger users, integrated standard reactions, and increased the ways in which the user can sort through chat history. Through these improvements, we hoped for the user to spend less time searching for previous messages or crafting a reply, allowing for more timely and effective communication.
As it currently stands, instant messaging on eLC does not offer many of the standard communication methods that students frequently use on alternative platforms. One such interaction is the ability to pin messages and then easily trace back to those messages; within a class group chat, students should be able to mark certain messages with higher importance to the group. One example of this could be a classmate sharing a link to a communal quizlet, in which the members of the group would appreciate being able to quickly navigate back to the message without scrolling through messages of lesser importance. With this user story, we implemented the feature of pinning messages and a way to filter out only pinned messages within a chat history. Alongside this, our group agreed that users should be able to either react or directly reply to a chat message; with a like feature, students can send their appreciation for specific messages without always crafting an extended response. Each message can then be associated with the number of likes it received, supporting the ability to sort by the most liked messages. Since the current version of eLc does not comfortably support direct replies to chats, we designed a more intuitive instant messaging experience for users, complete with a sidebar that displays all active chats. We implemented a user interface that resembles the external communication apps that UGA students currently use, such as Slack and Discord, as we knew users could easily navigate through those menus. We condensed and reorganized the features that were previously offered to eLC messenger users, integrated standard reactions, and increased the ways in which the user can sort through chat history. Through these improvements, we hoped for the user to spend less time searching for previous messages or crafting a reply, allowing for more timely and effective communication.
2d. User Story - Communicating with the Instructors:
"As a course instructor, I would like for my students to be able to more easily schedule office hours and communicate with instructors and peers alike to clear up confusions in a centralized place."
Our main objective with this project was to alleviate the pain points associated with communicating over eLC; in considering a professor’s point of view, we found that much confusion can arise from the professor not understanding the context of a student’s message. As students at UGA, it is not unusual for instructors to specifically request the class section, student ID number, and information relating to the assignment the student needs assistance with. Our group recognized that this information is directly available from certain pages on eLC, and thus decided to implement a button for students to reach out to an instructor from an assignment’s submission page. We also added a discussion forum for each assignment to lessen the need for communication with the instructor directly, especially when other classmates are able to quickly find and respond to these same questions. In designing the functionality for this user story, we implemented the “Ask the Instructor” button that allows for instant messaging with the instructor, where the chat history is specific to the assignment under discussion. The “Class Chat” button allows for instant messaging between a student and their classlist, including the professor, any teaching assistants, and the other students enrolled in the course. Separating communication methods into these pathways allows for a one-on-one lifeline to the professor if needed, or a forum where more generalized questions can be answered pertaining to a singular assignment; we believed this would help clear up confusion about a question’s context for the professor, and also reduce redundant questions for confused students. Finally, we integrated a more intuitive way for students to schedule a meeting with their instructor during provided office hours; the professor can display their availability through a calendar interface, and students can navigate through the “Ask the Instructor” page to the “Schedule Meeting” button. Although our group members have more personal experience with the frustrations of the student view in eLC, we believe that simplifying the action of scheduling a meeting during office hours could equally benefit the instructor by providing more information directly through the instant messaging platform.
Our main objective with this project was to alleviate the pain points associated with communicating over eLC; in considering a professor’s point of view, we found that much confusion can arise from the professor not understanding the context of a student’s message. As students at UGA, it is not unusual for instructors to specifically request the class section, student ID number, and information relating to the assignment the student needs assistance with. Our group recognized that this information is directly available from certain pages on eLC, and thus decided to implement a button for students to reach out to an instructor from an assignment’s submission page. We also added a discussion forum for each assignment to lessen the need for communication with the instructor directly, especially when other classmates are able to quickly find and respond to these same questions. In designing the functionality for this user story, we implemented the “Ask the Instructor” button that allows for instant messaging with the instructor, where the chat history is specific to the assignment under discussion. The “Class Chat” button allows for instant messaging between a student and their classlist, including the professor, any teaching assistants, and the other students enrolled in the course. Separating communication methods into these pathways allows for a one-on-one lifeline to the professor if needed, or a forum where more generalized questions can be answered pertaining to a singular assignment; we believed this would help clear up confusion about a question’s context for the professor, and also reduce redundant questions for confused students. Finally, we integrated a more intuitive way for students to schedule a meeting with their instructor during provided office hours; the professor can display their availability through a calendar interface, and students can navigate through the “Ask the Instructor” page to the “Schedule Meeting” button. Although our group members have more personal experience with the frustrations of the student view in eLC, we believe that simplifying the action of scheduling a meeting during office hours could equally benefit the instructor by providing more information directly through the instant messaging platform.
B. Testing Protocol:
Reaserch Question: Does the new eLC messaging platform make communication more efficient for students?
Testing Protocol:
Our team decided on testing the new interface using beta testing. This method will be effective because the users will have an opportunity to use the site voluntarily. Testing will be available for everyone to participate in, and we can collect useful survey information straight from the users at the end of the beta testing period. We would release the new interface on the main website at the beginning of the semester. When students access the website, a pop-up will ask them if they would like to try the new eLC experience. Once the user navigates to the new version of the site, there will be a link to the old version of the site on the home page. Our aim is to collect information on how many new users we get to the new site, how many users stay, and how many go back to the old version. Two months after the release, they will be prompted to fill out a survey when they go to the site by a pop-up containing a link. The survey site will tell them the information that we are collecting and request permission to collect this information. Regardless if the user gives permission for the information to be collected, the user will be allowed to take the survey.
Getting test subjects:
We are hoping to avoid any type of bias in the sample we chose by making the new experience available to all users that navigate to the site and making participation voluntary and not compensated with money or academic advantages.
Informed consent:
As required by the guidelines set forth by the University of Georgia’s Institutional Review Board and the Bellmont Report, an article that established guidelines for how we now conduct human research, we will get the users’ informed consent to gather and use personal information. According to the Belmont Report, the researchers typically disclose certain items to test subjects to cover most bases of informed consent. “These items generally include: the research procedure, their purposes, risks and anticipated benefits,” and an opportunity to withdraw from the research (Part C: Applications). Before taking the survey that is given to the users two months after the release, the user will be told that all information that is collected will not be information that can be reasonably connected to them.
Data to be collected and how will it be organized:
When the user decides to go to the new experience, the user will be asked permission for the site to follow how many times the user accesses eLC and if the user is using the old or new experience and will be connected with the user’s UGA ID until the end of the beta testing. The personal information that is collected in the survey will be used to determine whether or not our implementation of the new eLC features are being affected by convenience bias. The personal information that will be taken is age, sex, ethnicity, major, year standing, and grade point average. We will be examining which types of people are benefiting or suffering more than others. We will also be determining if the new experience was significantly helpful in aiding users in communication. The survey questions will ask the user to rate their experience with settings and personalization, group and personal messaging, pinning, marking, liking, and responding to messages, notifications, and usability on a scale from very unsatisfied, unsatisfied, neutral or N/A, satisfied, to very satisfied. Users will be asked if they used third-party messaging alternatives instead of using the new eLC experience during the testing period.
Safe testing in pandemic and other anticipated issues:
Testing will be done completely online with no reason for users to leave the comfort of their home. There are no specific times or places that the user must be to interact with the site. However, there are other problems we have to look out for. A paper written on a study about traditional software testing methodologies by Barta and Shelly addresses the fall backs of beta testing. Barta and Shelly write about beta testing that “testing conditions are far from controlled,” and “may not be able to reproduce the conditions under which the users are operating, if those conditions are even known (4. User Testing).” The people that stay on the old eLC or try the new one only a couple of times before going back may be a result of users being very used to the old eLC and not wanting to learn how to use a new system. A lot of the people who use the new experience may be underclassmen. In order to avoid biasing in our testing, as we are collecting data we will go over it to make sure we aren’t neglecting blocks of users or collecting too much from another. There are other reasons for bias to look out for in our study like people in certain majors or colleges being more prone to use the new or old eLC.
Evaluation:
We are going to use the site tracking information to find out who is using the new experience consistently, used it once or a couple times then left, or didn’t start using it at all to find out if users are liking the changes. We will be evaluating the usefulness of our site based off user ratings from the surveys. The personal information will help us find out which users are benefiting the most and if there are any users that are suffering.
Testing Protocol:
Our team decided on testing the new interface using beta testing. This method will be effective because the users will have an opportunity to use the site voluntarily. Testing will be available for everyone to participate in, and we can collect useful survey information straight from the users at the end of the beta testing period. We would release the new interface on the main website at the beginning of the semester. When students access the website, a pop-up will ask them if they would like to try the new eLC experience. Once the user navigates to the new version of the site, there will be a link to the old version of the site on the home page. Our aim is to collect information on how many new users we get to the new site, how many users stay, and how many go back to the old version. Two months after the release, they will be prompted to fill out a survey when they go to the site by a pop-up containing a link. The survey site will tell them the information that we are collecting and request permission to collect this information. Regardless if the user gives permission for the information to be collected, the user will be allowed to take the survey.
Getting test subjects:
We are hoping to avoid any type of bias in the sample we chose by making the new experience available to all users that navigate to the site and making participation voluntary and not compensated with money or academic advantages.
Informed consent:
As required by the guidelines set forth by the University of Georgia’s Institutional Review Board and the Bellmont Report, an article that established guidelines for how we now conduct human research, we will get the users’ informed consent to gather and use personal information. According to the Belmont Report, the researchers typically disclose certain items to test subjects to cover most bases of informed consent. “These items generally include: the research procedure, their purposes, risks and anticipated benefits,” and an opportunity to withdraw from the research (Part C: Applications). Before taking the survey that is given to the users two months after the release, the user will be told that all information that is collected will not be information that can be reasonably connected to them.
Data to be collected and how will it be organized:
When the user decides to go to the new experience, the user will be asked permission for the site to follow how many times the user accesses eLC and if the user is using the old or new experience and will be connected with the user’s UGA ID until the end of the beta testing. The personal information that is collected in the survey will be used to determine whether or not our implementation of the new eLC features are being affected by convenience bias. The personal information that will be taken is age, sex, ethnicity, major, year standing, and grade point average. We will be examining which types of people are benefiting or suffering more than others. We will also be determining if the new experience was significantly helpful in aiding users in communication. The survey questions will ask the user to rate their experience with settings and personalization, group and personal messaging, pinning, marking, liking, and responding to messages, notifications, and usability on a scale from very unsatisfied, unsatisfied, neutral or N/A, satisfied, to very satisfied. Users will be asked if they used third-party messaging alternatives instead of using the new eLC experience during the testing period.
Safe testing in pandemic and other anticipated issues:
Testing will be done completely online with no reason for users to leave the comfort of their home. There are no specific times or places that the user must be to interact with the site. However, there are other problems we have to look out for. A paper written on a study about traditional software testing methodologies by Barta and Shelly addresses the fall backs of beta testing. Barta and Shelly write about beta testing that “testing conditions are far from controlled,” and “may not be able to reproduce the conditions under which the users are operating, if those conditions are even known (4. User Testing).” The people that stay on the old eLC or try the new one only a couple of times before going back may be a result of users being very used to the old eLC and not wanting to learn how to use a new system. A lot of the people who use the new experience may be underclassmen. In order to avoid biasing in our testing, as we are collecting data we will go over it to make sure we aren’t neglecting blocks of users or collecting too much from another. There are other reasons for bias to look out for in our study like people in certain majors or colleges being more prone to use the new or old eLC.
Evaluation:
We are going to use the site tracking information to find out who is using the new experience consistently, used it once or a couple times then left, or didn’t start using it at all to find out if users are liking the changes. We will be evaluating the usefulness of our site based off user ratings from the surveys. The personal information will help us find out which users are benefiting the most and if there are any users that are suffering.
C. Final Summary Video:
Citations:
Application of Traditional Software Testing Methodologies to Web Accessibilty W4A '10: Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)April 2010 Article No.: 11Pages 1–4 https://doi.org/10.1145/1805986.1806002
The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research. [Bethesda, Md.]: The Commission, 1978.
The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research. [Bethesda, Md.]: The Commission, 1978.