Exploiting unintended feature leakage in collaborative learning

Luca Melis; Congzheng Song; Emiliano De Cristofaro; Vitaly Shmatikov

doi:10.1109/SP.2019.00029

Back

Exploiting unintended feature leakage in collaborative learning

Conference paper

Open access

Exploiting unintended feature leakage in collaborative learning

Luca Melis, Congzheng Song, Emiliano De Cristofaro and Vitaly Shmatikov

Proceedings - IEEE Symposium on Security and Privacy

Institute of Electrical and Electronics Engineers Inc.

2019

DOI: https://doi.org/10.1109/SP.2019.00029

Abstract

Data privacy

Security of data

Collaborative learning

Inference attacks

Joint modeling

Model updates

Security

Specific location

Training data

Training dataset

Deep learning

Collaborative machine learning and related techniques such as federated learning allow multiple participants, each with his own training dataset, to build a joint model by training locally and periodically exchanging model updates. We demonstrate that these updates leak unintended information about participants' training data and develop passive and active inference attacks to exploit this leakage. First, we show that an adversarial participant can infer the presence of exact data points - for example, specific locations - in others' training data (i.e., membership inference). Then, we show how this adversary can infer properties that hold only for a subset of the training data and are independent of the properties that the joint model aims to capture. For example, he can infer when a specific person first appears in the photos used to train a binary gender classifier. We evaluate our attacks on a variety of tasks, datasets, and learning configurations, analyze their limitations, and discuss possible defenses. © 2019 IEEE.

Files and links (1)

url

https://doi.org/10.1109/SP.2019.00029View

Published (Version of record) Open

Metrics

1 Record Views

Details

Title: Exploiting unintended feature leakage in collaborative learning
Creators - without role: Luca Melis - University College London
Congzheng Song - Cornell University
Emiliano De Cristofaro - University College London
Vitaly Shmatikov - Cornell University
Publication Details: Proceedings - IEEE Symposium on Security and Privacy
Publisher: Institute of Electrical and Electronics Engineers Inc.
Grant note: Our results suggest that leakage of unintended features exposes collaborative learning to powerful inference attacks. We also showed that defenses such as selective gradient sharing, reducing dimensionality, and dropout are not effective. This should motivate future work on better defenses. For instance, techniques that learn only the features relevant to a given task [15, 42, 43] can potentially serve as the basis for “least-privilege” collaboratively trained models. Further, it may be possible to detect active attacks that manipulate the model into learning extra features. Finally, it remains an open question if participant-level differential privacy mechanisms can produce accurate models when collaborative learning involves relatively few participants. Acknowledgments. This research was supported in part by the NSF grants 1611770 and 1704296, the generosity of Eric and Wendy Schmidt by recommendation of the Schmidt Futures program, the Alan Turing Institute under the EPSRC grant EP/N510129/1, and a grant by Nokia Bell Labs.
Identifiers: 9920191009548
Academic Unit: Alan Turing Institute
Language: English
Resource Type: Conference paper
Date published: 2019

Exploiting unintended feature leakage in collaborative learning

Abstract

Files and links (1)

Metrics

Details

Alan Turing Institute Social media