Seungjae Oh

VibEye

Vibration-Mediated Object Recognition for Tangible Interactive Applications
(ACM CHI 2019)

Teasor

Abstract

We present VibEye: a vibration-mediated recognition system of objects for tangible interaction. A user holds an object between two fingers wearing VibEye. VibEye triggers a vibration from one finger, and the vibration that has propagated through the object is sensed at the other finger. This vibration includes information about the object’s identity, and we represent it using a spectrogram.

Collecting the spectrograms of many objects, we formulate the object recognition problem to a classical classification problem among the images. This simple method, when tested with 20 users, shows 92.5% accuracy for 16 objects of the same shape with various materials. This material-based classifier is also extended to the recognition of everyday objects.

Lastly, we demonstrate several tangible applications where VibEye provides the needed functionality while enhancing user experiences. VibEye is particularly effective for recognizing objects made of different materials, which is difficult to distinguish by other means such as light and sound.

Publication

Figure

Application

HW Hardware design of VibEye.

Dataprocessing The overview of computational procedure for signal processing and object recognition.

SampleSTFT Spectrograms of 16 standard objects. Objects are marked with material properties
(R: rigid, E: elastic, V: viscous, P: plastic, and S: stacked).

Possibility The further sensing possibiliteis of VibEye. The four sets of everyday objects and their PCA results (20 rep. each): (A) Liquid body products in soft tubes, (B) Candies in hard containers, © Stacked papers, and (D) A spray bottle.

Conclusion

In this work, we have presented a system for vibration-mediated object recognition. VibEye is simple: its hardware requires only a vibration emitter and a sensor, and its software processes the data using well-defined image-based methods. Essentially, VibEye transforms the object recognition problem to an image classification problem. We have validated the effectiveness of VibEye in several ways, using the cross-validation results for the standard objects of the same shape but different materials, and recognition performance for other users’ data and other unseen various everyday objects’ data. We envision tightly-coupled virtual and real environments that are seamlessly controlled by tangible objects. We hope that the concepts embodied by VibEye could pave the way.

Contribution

  • Seungjae Oh: Idea, Implementation (HW&SW), Experiment, Analysis, Writing, and Graphics
  • Others: Application Development, Graphics, and Video
  • Seungmoon Choi: Director