Inproceedings,

PredicTouch: A System to Reduce Touchscreen Latency using Neural Networks and Inertial Measurement Units

, , , and .
Proceedings of the ACM International Conference on Interactive Surfaces and Spaces (ISS), page 230-239. ACM, (2017)
DOI: 10.1145/3132272.3134138

Abstract

Touchscreens are the dominant input mechanism for a variety of devices. One of the main limitations of touchscreens is the latency to receive input, refresh, and respond. This latency is easily perceivable and reduces users' performance. Previous work proposed to reduce latency by extrapolating finger movements to identify future movements - albeit with limited success. In this paper, we propose PredicTouch, a system that improves this extrapolation using inertial measurement units (IMUs). We combine IMU data with users' touch trajectories to train a multi-layer feedforward neural network that predicts future trajectories. We found that this hybrid approach (software: prediction, and hardware: IMU) can significantly reduce the prediction error, reducing latency effects. We show that using a wrist-worn IMU increases the throughput by 15% for finger input and 17% for a stylus.

Tags

Users

  • @hlawatml
  • @sfbtrr161
  • @mueller
  • @leonkokkoliadis
  • @dblp
  • @tinabarthelmes
  • @visus

Comments and Reviews