Upon realizing that students enrolled in online programming courses often lack immediate support for their questions and desire a more personalized learning experience, Lang was motivated to address these unique challenges by developing a chatbot for his course ‘Data Analytics BAN-663.’

Harnessing his knowledge and expertise, Lang developed the course chatbot from scratch as a Python web application using the Streamlit platform, with LlamaIndex for retrieval augmentation and GPT-4 for generating responses. The chatbot is designed to ingest course-specific material including lectures, code and exercises to align its responses with the content and style of the course. Through this strategic development, Lang ensured that the chatbot could reference exercises, explain R programming concepts and stay consistent with the teaching style. Additionally, the chatbot is designed to clarify lecture topics, help with advanced queries beyond lecture material and troubleshoot coding errors in assignments.

“It’s designed to explain rather than provide direct answers, thus supporting students in understanding R programming and statistical concepts as part of the course,” said Lang. “AI complements traditional teaching by offering immediate, personalized support outside of live interactions. The chatbot provides support on demand, while video lectures and assignments reinforce core learning in an online course. This hybrid approach ensures that students have the support they need while maintaining structured, instructor-led instruction.”

Since its integration into the curriculum, Lang has launched a survey to investigate student engagement and satisfaction. The survey indicated that students with higher prior knowledge of R programming tended to use the chatbot more, especially for advanced topics. The survey responses resulted in high satisfaction, with 87% of students feeling that the chatbot enhanced their learning experience.

“Feedback was overwhelmingly positive,” said Lang. “Many students found the chatbot’s explanations helpful, appreciated its alignment with course materials and noted its ease of use. A large majority expressed a desire to see similar chatbots in other courses.” 

Although the chatbot garnered student support, Lang has worked diligently to address any challenges while implementing the AI tool. 

“One challenge was crafting a system prompt that ensures the chatbot could effectively respond within the boundaries of R programming and statistical questions,” he said. “Another was the initial setup and alignment with the course materials. Both were addressed by carefully designing the chatbot's retrieval and generation prompts and iteratively refining them.”

Considering the success of this AI tool, Lang advocates for the potential for emphasizing personalized support across a broader curriculum. Emphasizing his support and experience, Lang conducted an empirical study on the use of chatbots, which has been peer-reviewed and accepted at the ISCAP Conference.

“Given the positive reception, there is significant potential for implementing similar chatbots in other courses and departments, particularly those requiring programming or other technical skill,” said Lang. “Future improvements may include expanding the chatbot’s capabilities to address a wider range of technical questions and refining its responses through student feedback. Additionally, exploring different features to support varying skill levels could enhance its adaptability and relevance.”