We developed classifiers to predict privacy labels based on the text of privacy policies. Our study suggests that our technique achieves an F1 score of over 98%. We then proceeded to analyze a little over 350,000 apps in the iOS app store. Results suggest that discrepancies are quite common. In particular we find that nearly 30% of mobile apps seem to have privacy labels that disclose practices not disclosed in their privacy policies. In our view, while it is easy to blame app developers/publishers, app stores, as the more sophisticated and powerful actors, also bear some responsibility and should do a better job helping app publishers create more accurate privacy labels and policies. This includes in particular creating stricter standards for third party libraries, which, based on our earlier research, are often the source of inaccuracies.

‍

Askahth Jain, David Rodriguez, Jose del Alamo, and Norman Sadeh, "ATLAS: Automatically Detecting Discrepancies between Privacy Policies and Privacy Labels", 2023 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), Amsterdam, June 2023.

Aerin Zhang, "Undertanding People's Diverse Privacy Attitudes: Notification, Control and Regulatory Implications", Language Technologies Institute PhD Program, School of Computer Science, Carnegie Mellon University. March 17, 2023

‍

Back in 2013, we reported on research that showed how privacy labels could help app store users make better informed decisions. Our paper was published at CHI. In December 2020 Apple introduced privacy labels in its app store, crediting our earlier research for influencing their decision. Six months later the Google Play Store followed with its privacy (or "safety") labels. A year ago, we published a study at PoPETS, showing that unfortunately in their current form iOS privacy labels fall short. In a new paper published at USEC this month, we estimate to what extent iOS privacy labels (in their current form) answer those privacy questions people actually care about. Our study suggests that the answer might be "less than 50% of the time." Follow the link below to read our USEC2023 paper "Do Privacy Labels Answer People's Privacy Questions"".

‍

Just gave a keynote at the 1st International Workshop on "Privacy Algorithms in Systems."  My talk focused on "Privacy in the Age of AI and the Internet of Things".

Here's a CyLab press release summarizing the result of a study conducted with my PhD student, Aerin Zhang,  Yuanyuan Feng, Yaxing Yao and Lorrie Cranor on the usability of iOS Privacy Labels in their current form.

This research was presented at PoPETS 2022 earlier this month.

The full article is available here

This research is taking place under the umbrella of our Usable Privacy Policy Project

Helping Mobile App Developers Create Accurate Privacy Labels. by Jack Gardner, Akshath Jain, Yuanyuan Feng, Kayla Reiman, Zhi Lin and Norman Sadeh. In this work we discuss the design and evaluation of a tool to help iOS developers generate privacy labels. The tool combines static code analysis to identify likely data collection and use practices with interactive functionality designed to prompt developers to elucidate analysis results and carefully reflect on their applications’ data practices. We conducted semi-structured interviews with iOS developers as they used an initial version of the tool. We discuss how these results motivated us to develop an enhanced software tool, Privacy Label Wiz, that more closely resembles interactions developers reported to be most useful in our semi-structured interviews. We present findings from our interviews and the enhanced tool motivated by our study. We also outline future directions for software tools to better assist developers communicating their mobile app’s data practices to different audiences.

Honored to see our work on Livehoods, mining public social media data to understand the dynamics of cities, selected for the test of time award at AAAI's 16th International Conference on Web and Social Media.

Here's CMU's School of Computer Science press release.

Stop the Spread: A Contextual Integrity Perspective on the Appropriateness of COVID-19 Vaccination Certificates

Shikun Zhang, Yan Shvartzshnaider, Yuanyuan Feng, Helen Nissenbaum, and Norman Sadeh

‍

We present an empirical study exploring how privacy influences the acceptance of vaccination certificate (VC) deployments across different realistic usage scenarios. The study employed the privacy framework of Contextual Integrity, which has been shown to be particularly effective to capture people's privacy expectations across different contexts. We use a vignette methodology, where we selectively manipulate salient contextual parameters understood to potentially impact people's attitudes towards VCs. We surveyed 890 participants from a demographically-stratified sample of the US population to gauge the acceptance and overall attitudes towards possible VC deployments to enforce vaccination mandates and the different information flows VCs might entail. Analysis of results collected at part of this study are used to derive general normative observations about different possible VC practices and to provide guidance for the possible VC deployments in different contexts.

Here's a video of Aerin's presentation

Gave a keynote at Cyburgh 2022 on "Privacy as a New Tech Sector".

Panelist at Privacy Symposium in Venice.

My talk focused on "Making Privacy Humanly Tractable".

Justin Cranshaw, "Depicting Places in Information Systems: Closing the Gap Between Representation and Experience", PhD Dissertation, School of Computer Science technical report CMU-ISR-22-106, May 2022.

Interviewed by Pulitzer-winner Kathleen Gallagher as she launches her "Midwest Moxie' podcast, which features interviews with entrepreneurs.

My ICISSP keynote focused on "Why Usability Has Become Privacy's Biggest Challenge and What We Can Do About It".

See below for a copy of my slides.

Our paper about people’s perceptions of advanced video analytics has been selected to receive the prestigious Future of Privacy Forum’s annual Privacy Papers for Policymakers Award.

The paper titled “‘Did you know this camera tracks your mood? Understanding Privacy Expectations and Preferences in the Age of Video Analytics,” was originally published and presented at the 2021 Privacy Enhancing Technologies Symposium.

Click on the link below for the CyLab/S3D press release.

‍

Reflections on the limitations and strengths of human and artificial intelligence and how our work in user-oriented security and privacy aims to develop practical solutions where both forms of intelligence are deployed to best complement one another. The presentation builds on our anti-phishing work, including work at Wombat Security Technologies, as well as research at CMU in the context of the Usable Privacy Policy Project and the Personalized Privacy Assistant Project.

In response to increased demand from industry, we launched a new certificate program in privacy engineering earlier this Fall semester. The program is remote but involves live lectures and interactions with our faculty as well as both individual and group exercises. The program is delivered over 4 consecutive weekends. We will be running 4 cohorts per year.

Survey of the students who completed the program indicate they all felt they learned a ton and really enjoyed the live interactions and group exercises.

Following our 2013 proposing the adoption of mobile app privacy labels, both Apple (Fall 2020) and Google (Spring 2021) have now announced the introduction of such labels in their app stores - see our 2013 CHI article with Patrick Gage Kelley and Lorrie Cranor here.

It appears however that in the form in which they have been introduced, these labels are not fully delivering on their promises - with anecdotal evidence suggesting that both developers and end-users struggle to fully understand what these labels mean and how to use them. As part of this project, we will be conducting an in-depth study of the usability of current iOS mobile app privacy labels (joint work with my PhD student, Aerin Zhang, and with Lorrie Cranor) and will also be developing tools to help mobile app developers create more accurate privacy labels.

D. Smullen, "Informing the Design and Refinement of Privacy and Security Controls", Ph.D. Thesis, Software Engineering PhD Program, Technical Report CMU-ISR-21-111, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA. September 2021.

A few quotes related to our work on a privacy infrastructure for the Internet of Things. Our infrastructure is now hosting descriptions of about 150,000 IoT data collection systems and devices.

Abhilasha Ravichander presents our research at the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL/IJCNLP). Here's the abstract:

Privacy plays a crucial role in preserving democratic ideals and personal autonomy. The dominant legal approach to privacy in many jurisdictions is the “Notice and Choice” paradigm, where privacy policies are the primary instrument used to convey information to users. However, privacy policies are long and complex documents that are difficult for users to read and comprehend. We discuss how language technologies can play an important role in addressing this information gap, reporting on initial progress towards helping three specific categories of stakeholders take advantage of digital privacy policies: consumers, enterprises, and regulators. Our goal is to provide a roadmap for the development and use of language technologies to empower users to reclaim control over their privacy, limit privacy harms, and rally research efforts from the community towards addressing an issue with large social impact. We highlight many remaining opportunities to develop language technologies that are more precise or nuanced in the way in which they use the text of privacy policies.

S Zhang, Y Feng, L Bauer, LF Cranor, A Das, and N Sadeh, “Did you know this camera tracks your mood?”: Understanding Privacy Expectations and Preferences in the Age of Video Analytics Proceedings on Privacy Enhancing Technologies, 2, 1, Apr 2021.

Cameras are everywhere, and are increasingly coupled with video analytics software that can identify our face, track our mood, recognize what we are doing, and more. We present the results of a 10-day in-situ study designed to understand how people feel aboutt hese capabilities, looking both at the extent to which they expect to encounter them as part of their everyday activities and at how comfortable they are with the presence of such technologies across a range of realistics cenarios. Results indicate that while some widespread deployments are expected by many (e.g., surveillance in public spaces), others are not, with some making people feel particularly uncomfortable. Our results further show that individuals’ privacy preferences and expectations are complicated and vary with a number of factors such as the purpose for which footage is captured and analyzed, the particular venue where it is captured, and whom it is shared with. Finally, we discuss the implications of people’s rich and diverse preferences on opt-in or opt-out rights for the collection and use (including sharing) of data associated with these video analytics scenarios as mandated by regulations. Because of the user burden associated with the large number of privacy decisions people could be faced with, we discuss how new types of privacy assistants could possibly be configured to help people manage these decisions.

‍

People hold a myriad of misconceptions about the tools meant to help them protect their privacy and security. Here is a recent CyLab article on research that was presented by Peter Story at this week’s Privacy Enhancing Technologies Symposium.

Peter Story, Daniel Smullen, Yaxing Yao, Alessandro Acquisti, Lorrie Faith Cranor, Norman Sadeh, and Florian Schaub, Awareness, Adoption, and Misconceptions of Web Privacy Tools. Proceedings on Privacy Enhancing Technologies Symposium (PoPETS 2021), 3, Jul 2021

‍

Presentation of our research by my PhD student, Daniel Smullen, at PoPETS2021:

"Browser users encounter a broad array of potentially intrusive practices: from behavioral profiling, to crypto-mining, fingerprinting, and more. We study people’s perception, awareness, understanding, and preferences to opt out of those practices..."

‍

P. Story, "Design and Evaluation of Security and Privacy Nudges: From Protection Motivation Theory to Implementation Intentions", CMU-ISR-21-107, School of Computer Science, Carnegie Mellon University. August 2021.

Y Feng, Y Yao, N Sadeh, "A Design Space for Privacy Choices: Towards Meaningful Privacy Control in the Internet of Things." Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, May 2021.

“Notice and choice” is the predominant approach for data privacy protection today. There is considerable user-centered research on providing effective privacy notices but not enough guidance on designing privacy choices. Recent data privacy regulations worldwide established new requirements for privacy choices, but system practitioners struggle to implement legally compliant privacy choices that also provide users meaningful privacy control. We construct a design space for privacy choices based on a user-centered analysis of how people exercise privacy choices in real-world systems. This work contributes a conceptual framework that considers privacy choice as a user-centered process as well as a taxonomy for practitioners to design meaningful privacy choices in their systems. We also present a use case of how we leverage the design space to finalize the design decisions for a real-world privacy choice platform, the Internet of Things (IoT) Assistant, to provide meaningful privacy control in the IoT.

‍