RTutorAI UX Evaluation
RTutorAI UX Evaluation
2025
UX Research | Data Analysis | AI Product Design
2025
UX Research | Data Analysis | AI Product Design
ABOUT
ABOUT
I conducted UX research on an AI-powered tool with hundreds of users. Introduction to Statistics at UIUC requires students to use RTutorAI, a website combining R with the power of ChatGPT, to complete their coursework.
However, RTutorAI wasn't created with education in mind. In other words, it's meant for PhDs, not freshmen.
My role:
➢ Evaluate RTutorAI by conducting UX research and assist in improving the UI/UX
➢ Survey and analyze data from 700+ students
➢ Inform design changes for RTutorAI and future AI learning tools
My Mentor: V.N. Vimal Rao
I conducted UX research on an AI-powered tool with hundreds of users. Introduction to Statistics at UIUC requires students to use RTutorAI, a website combining R with the power of ChatGPT, to complete their coursework.
However, RTutorAI wasn't created with education in mind. In other words, it's meant for PhDs, not freshmen.
My role:
➢ Evaluate RTutorAI by conducting UX research and assist in improving the UI/UX
➢ Survey and analyze data from 700+ students
➢ Inform design changes for RTutorAI and future AI learning tools
My Mentor: V.N. Vimal Rao
PROCESS
PROCESS
We want to know:
➢ How does RTutorAI impact students' success in the class?
➢ What pain points are in the UI/UX?
➢ Do students find RTutorAI valuable?
➢ Would they use RTutorAI to create and analyze graphs in the future?
We want to know:
➢ How does RTutorAI impact students' success in the class?
➢ What pain points are in the UI/UX?
➢ Do students find RTutorAI valuable?
➢ Would they use RTutorAI to create and analyze graphs in the future?
Participants: 726 STAT 100 students.
Demographics: 69% freshmen, 73% women, with a diverse ethnic and academic background.
Methods: I created a voluntary survey, incorporating the UEQ, meCUE, and SUS questionnaires and using a 6-point Likert scale. Students also had the option to write in feedback.
(Excerpts from the UEQ and meCUE results below)
Participants: 726 STAT 100 students.
Demographics: 69% freshmen, 73% women, with a diverse ethnic and academic background.
Methods: I created a voluntary survey, incorporating the UEQ, meCUE, and SUS questionnaires and using a 6-point Likert scale. Students also had the option to write in feedback.
(Excerpts from the UEQ and meCUE results below)
RESULTS
RESULTS
My research is ongoing. Check back later for updates!
Preliminary Findings:
➢ Performance issues (crashes, slow runtime) were the main factors for a negative perception
➢ Students are split on whether the website is easy to understand
➢ Despite negative emotional responses, students consistently rank RTutorAI as practical and valuable
Potential Design Improvements:
➢ Improve performance (crashes, runtime) to handle high website traffic
➢ Redesign the "Submit" and "Reset" buttons
➢ Display past prompts, along with output history
➢ Redirect users when they get errors or null responses
➢ Incorporate more instructions or add a tutorial
➢ Allow students to ask questions about the R code or graphs
➢ Add the ability to manually modify code
What's Next:
➢ Analyze and organize the data for my professor's paper
➢ Act as a UI/UX consultant, deciding which changes could be implemented
➢ Design mockups to hand off to developers
➢ Conduct future surveys and UX research to ensure the changes are effective
This project has shown me how many factors go into a product's success. These data-driven changes have the potential to make a real impact on student learning and motivation. This is an incredible responsibility and I can't wait to keep working!
Thank you to V.N. Vimal Rao for overseeing and mentoring this project.
My research is ongoing. Check back later for updates!
Preliminary Findings:
➢ Performance issues (crashes, slow runtime) were the main factors for a negative perception
➢ Students are split on whether the website is easy to understand
➢ Despite negative emotional responses, students consistently rank RTutorAI as practical and valuable
Potential Design Improvements:
➢ Improve performance (crashes, runtime) to handle high website traffic
➢ Redesign the "Submit" and "Reset" buttons
➢ Display past prompts, along with output history
➢ Redirect users when they get errors or null responses
➢ Incorporate more instructions or add a tutorial
➢ Allow students to ask questions about the R code or graphs
➢ Add the ability to manually modify code
What's Next:
➢ Analyze and organize the data for my professor's paper
➢ Act as a UI/UX consultant, deciding which changes could be implemented
➢ Design mockups to hand off to developers
➢ Conduct future surveys and UX research to ensure the changes are effective
This project has shown me how many factors go into a product's success. These data-driven changes have the potential to make a real impact on student learning and motivation. This is an incredible responsibility and I can't wait to keep working!
Thank you to V.N. Vimal Rao for overseeing and mentoring this project.
Every experience is a chance to learn something new. I'm ready for the next one.
San Francisco, CA / Champaign, IL
Every experience is a chance to learn something new. I'm ready for the next one.
San Francisco, CA / Champaign, IL