RealityCanvas: Augmented Reality Sketching for Embedded and Responsive Scribble Animation Effects

Zhijie Xia, Kyzyl Monteiro, Kevin Van, Ryo Suzuki  

The ACM Symposium on User Interface Software and Technology (UIST 2023)

Links:     PDF   Poster   Slide   Video   30s Video   GitHub   ACM DL   IEEE   arXiv   Talk

Abstract

We introduce RealityCanvas, a mobile AR sketching tool that can easily augment real-world physical motion with responsive hand-drawn animation. Recent research in AR sketching tools has enabled users to not only embed static drawings into the real world but also dynamically animate them with physical motion. However, existing tools often lack the flexibility and expressiveness of possible animations, as they primarily support simple line-based geometry. To address this limitation, we explore both expressive and improvisational AR sketched animation by introducing a set of responsive scribble animation techniques that can be directly embedded through sketching interactions: 1) object binding, 2) flip-book animation, 3) action trigger, 4) particle effects, 5) motion trajectory, and 6) contour highlight. These six animation effects were derived from the analysis of 172 existing video-edited scribble animations. We showcase these techniques through various applications, such as video creation, augmented education, storytelling, and AR prototyping. The results of our user study and expert interviews confirm that our tool can lower the barrier to creating AR-based sketched animation, while allowing creative, expressive, and improvisational AR sketching experiences.

Publication

Slide