Augmented Math: Authoring AR-Based Explorable Explanations by Augmenting Static Math Textbooks

Neil Chulpongsatorn, Mille Skovhus Lunding, Nishan Soni, Ryo Suzuki  

The ACM Symposium on User Interface Software and Technology (UIST 2023)

Links:     PDF   Poster   Slide   Video   30s Video   GitHub   ACM DL   IEEE   arXiv   Talk

Abstract

We introduce Augmented Math, a machine learning-based approach to authoring AR explorable explanations by augmenting static math textbooks without programming. To augment a static document, our system first extracts mathematical formulas and figures from a given document using optical character recognition (OCR) and computer vision. By binding and manipulating these extracted contents, the user can see the interactive animation overlaid onto the document through mobile AR interfaces. This empowers non-technical users, such as teachers or students, to transform existing math textbooks and handouts into on-demand and personalized explorable explanations. To design our system, we first analyzed existing explorable math explanations to identify common design strategies. Based on the findings, we developed a set of augmentation techniques that can be automatically generated based on the extracted content, which are 1) dynamic values, 2) interactive figures, 3) relationship highlights, 4) concrete examples, and 5) step-by-step hints. To evaluate our system, we conduct two user studies: preliminary user testing and expert interviews. The study results confirm that our system allows more engaging experiences for learning math concepts.

Publication

Slide