Abstract

Capacitive touch screens are the most prevalent type of touch screens used in modern smartphones. They work based on the electrical properties of the human body. When we touch the screen with our fingers, it creates a disruption in the screen's electrostatic field. The smartphone detects this change and registers the touch. More precisely, mobile operating systems such as Android and iOS merely provide the 2D touch coordinates of the touch. However, in reality, the touch controllers sense much more information. Let us illustrate this by an example. The capacitive touchscreens of commodity smartphones comprise around 400 to 600 electrodes (e.g., 15×27 electrodes with each being 4.1×4.1mm on an LG Nexus 5). Having said that, a touch controller measures the change of coupling capacitance between two orthogonal electrodes, i.e., intersections of row and column pairs. These measurements, when visualized as a 2D image, result in a low-resolution finger imprint. Such raw measurements of capacitive touchscreens are often referred to as capacitive images. Eventually, the touch controller translates the raw measurements into a 2D coordinate which is then provided to the operating system. This project aims to study the feasibility of mutual capacitance-based touch screens as a bioimpedance sensor for health sensing (e.g., estimation of vitals, etc.). That is, we are interested in acquiring the raw measurements of capacitive touchscreens, i.e., capacitive images, in order to design and develop novel bioimpedance-based health sensing techniques. We note that it is indeed possible to access the capacitive images in Android applications, with some developer-level programming effort (e.g., by modifying the Android kernel). Once the acquisition of the capacitive images is done, the immediate next goal would be to construct a dataset by asking 20+ volunteers to touch the smartphone screen for a duration of 1 minute, while simultaneously recording their vitals by means of a pulse-oximeter. This labeled dataset will be pre-processed, segmented, and passed on to a number of machine, deep, and transfer learning methods in order to infer the body vitals.

Deliverables

• Either a demo or a research paper/poster/report.