The UX Factor is an online grading platform that allows users to anonymously view others' submissions and give their feedback. The goal of this is to allow students to get as much feedback as possible, while still maintaining the anonymity of each submission. In order to quantitatively analyze each students' submission (which allows us to translate feedback into a grade), we ask pointed questions and ask users to determine which submission, given a pair of two submissions, better answers the task of the question. In this way, we can not only give helpful feedback to students based on their peers' responses, but also a quantitative grade that is, ideally, less biased than a grade that would be given solely by one reviewer.
Beginning with just a basic concept, I began the process of building this grading platform by reading papers, doing research on pre-existing peer review softwares, and gathering opinions and advice from peers. Following this, I created a variety of sketches to help me visualize and brainstorm what the multitude of information I had gathered could look like in this grading platform.
After experimenting with a variety of setups, we decided to create a login page and a comparison page. Deciding that it would be simpler for users to have one page per submission pair, this setup would help simplify and clarify the different evaluation tasks that we were asking users to complete.
It was decided to conduct two rounds of user testing over the summer, due to time constraints on having this platform ready for the fall semester. A sample of the first user testing document is shown here, where we asked users to perform specific tasks, and fill out a google feedback form with their notes, comments, and critiques.
Following user testing, we ended up making a variety of changes, including but not limited to:
Information about the latest version of the prototype and how you can install it can be found by following the link below. We are currently working on adding functionality to weigh TA feedback more, allow students to see which submissions scored better than theirs in certain categories, and fully automating our current grading script (which translates the information gathered from the UX Factor into grades).
2020 All images and content are property of Sarah Bawabe.