Safeguarding Private Visual Information

Overview

People who are blind capture images and videos of their surroundings and share them with visual assistance technologies to learn about their immediate surroundings, as well as the content and purpose of the visual media. However, the images and videos people who are blind share can contain private personal data. A challenge is that people who are blind can not independently validate the content of their images prior to receiving visual assistance.

I conduct visual content analysis, interviews, policy analysis, and participatory and co-design research with blind people to develop visual assistance technologies guided by the principle of privacy by design.

This research is funded through a NSF/CRA Computing Innovation Fellowship, CREATE Funding, and a UT-Austin Good Systems Grant

Please see all related projects below.

Keywords

blind, data regulation, camera-based devices, contextual integrity, computer vision, low vision, privacy by design, privacy policy, private visual content, remote sighted assistance, vision to language, visual assistance, visual privacy, visual personal data, visually impaired, voice user interaction.

VizWiz Dataset and Grand Challenge

This image contains 12 image thumbnails and associated visual questions asked by people who are blind, and answers provided by visual assistants. — With visual assistance applications, blind people can share images and visual questions, and receive answers from sighted assistants. These are examples.

The VizWiz dataset contains images and visual questions shared by people who are blind. These data are used to train algorithms that generate image descriptions and visual question answers. The VizWiz Grand Challenged engages developers in to increase accuracy of these algorithms. Read our CVPR 19′ paper here.

VizWiz-Priv Image Dataset and Development

This image shows thumbnail images of 19 types of private information that were found in images taken by people who are blind. Some of the information is text-based, while other information can be classified as objects. (Face, Pregnancy Test, Tattoo, License Plate, Credit Card, Computer Screen, Pill Bottle, Letter with Address. — We all take and share images. People who are blind take images to learn about their surroundings. Some of their images contain private information.

Dr. Danna Gurari observed that some of the images in the VizWiz dataset contain personal visual data. We analyzed the 40,000 images and found 19 types of personally identifiable data. We removed these data from the VizWiz Dataset to protect the blind photographers’ privacy. Read our CVPR 19′ paper here.

User-Centered Visual Privacy Taxonomy

This is a screenshot of a table showing that financial and medical information are of most concern to people who are blind. — People who are blind rated the types of personal data they considered to be the most private when using human-powered and AI-powered visual assistance technologies.

We interviewed 18 blind people to learn about the visual information they consider to be private and their visual privacy concerns when using human and AI-powered visual assistance technologies. We contribute a taxonomy of visual information blind people consider to be private that is used to guide the development of privacy-protective algorithms. Read our ASSETS 20′ paper here.

Visual Personal Data Collection and Processing

Image of a typewriter with the text "Privacy" written on the paper — Privacy policies are notoriously hard to read, yet they are currently the primary location where users can learn if and how a company collects and processes their data.

One of the privacy concerns blind people have relates to how visual assistance companies collect and process their data. Our analysis of visual assistance technology companies’ privacy policies revealed that many companies do not provide notice and choice regarding the handling of visual data specifically. This work is in progress.

Visual Assistance through Privacy By Design

A diagram showing the stages of privacy by design: Transparency, Minimization, Security Safeguards, Disclosure to third parties, Accountability, Lawful and Fair — Contemporary data regulation requires companies to protect users’ privacy according to privacy by design standards, which are summarized in this graphic.

User-centered design is a method used to design technologies that are guided by the end-users’ interests and needs. In the case of this research, we focus on visual assistance technology users’ understanding of the principles of privacy by design, and the agency over how their visual private data is collected and processed when using visual assistance technologies. This work is in progress.

Contextual Integrity of Visual Assistance

This is a complex diagram of showing the flow of information between the visual assistance technology user, and the companies who handle their data. — Contextual integrity involves an analysis of the flow of data between the data subject and the companies handling their data to identify contextually relevant privacy risks.

Privacy is contextual. In this research, I investigate the social norms and transmission principles that influence visual assistance technology users’ privacy preferences in the context of visual assistance technologies. This work is in progress.