CIS 7000: Empirical Security and Privacy, for Humans

Date	Topic / Speaker	Readings & handouts
Aug 26	Introduction and syllabus	S. Keshav. How to Read a Paper. ACM SIGCOMM Computer Communication Review, Volume 37, Number 3, July 2007. Lecture slides (PDF)
Aug 28	Economic view of cybersecurity - Alex Gantman, VP Security Engineering, Qualcomm	Ross Anderson. Why information security is hard - an economic perspective. Seventeenth annual computer security applications conference (ACSAC). IEEE, 2001. Ian Griggs, The Market for Silver Bullets, 2008. Lecture slides
Sep 2	End users and cybersecurity	Ho, Grant, et al. Understanding the efficacy of phishing training in practice. 2025 IEEE Symposium on Security and Privacy (SP). IEEE, 2025. Lorrie Faith Cranor. A Framework for Reasoning About the Human in the Loop. In Proceedings of UPSEC 2008.
Sep 4	Cybersecurity as a scientific pursuit - Cormac Herley, Principal Researcher, Microsoft	Cormac Herley and P.C. van Oorschot. SoK: Science, Security, and the Elusive Goal of Security as a Scientific Pursuit, IEEE S&P 2017.
Sep 9	Cybersecurity and risk assessment	Andrew Simpson. Into the unknown: The need to reframe risk analysis. Journal of Cybersecurity, Volume 10, Issue 1, 2024. John Downer. The Aviation Paradox: Why We Can 'Know' Jetliners But Not Reactors. Minerva 55.2 (2017): 229-248. Lecture slides
Sep 11	Passwords	Blase Ur, Jonathan Bees, Sean M. Segreti, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor. Do Users' Perceptions of Password Security Match Reality? In Proceedings of CHI 2016. Optional: Nisenoff, Alexandra, Maximilian Golla, Miranda Wei, Juliette Hainline, Hayley Szymanek, Annika Braun, Annika Hildebrandt, Blair Christensen, David Langenberg, and Blase Ur. A Two-Decade Retrospective Analysis of a University's Vulnerability to Attacks Exploiting Reused Passwords. In USENIX Security 2023. Lecture slides
Sep 16	The business of attacks, and paying attackers for defense	Huang, Keman, Michael Siegel, and Stuart Madnick. Systematically understanding the cyber attack business: A survey. ACM Computing Surveys (CSUR) 51.4 (2018): 1-36. Dirk Goehmann. Vulnerability Reward Program: 2024 in Review, Google Security Blog, March 2025. See also: Matthew Finifter, Devdatta Akhawe, and David Wagner. An Empirical Study of Vulnerability Rewards Programs. USENIX Security 2013. Lecture slides
Sep 18	LLMs and their impact on cyberattacks	Nicholas Carlini, Milad Nasr, Edoardo Debenedetti, Barry Wang, Christopher A. Choquette-Choo, Daphne Ippolito, Florian Tramer, Matthew Jagielski. LLMs unlock new paths to monetizing exploits. arXiv preprint arXiv:2505.11449 (2025). Oege de Moor and Albert Ziegler. XBOW Unleashes GPT-5's Hidden Hacking Power, Doubling Performance. XBOW blog, August 15, 2025. (On August 18, XBOW became #1 ranked, better than all human teams, on HackerOne.) Lecture slides
Sep 23	Measuring secure software development practices	Ruef, Andrew, Michael Hicks, James Parker, Dave Levin, Michelle L. Mazurek, and Piotr Mardziel. Build it, break it, fix it: Contesting secure development. CCS 2016. Long version (with an additional contest, fuller descriptions) in ACM TOPS 2020. Votipka, Daniel, Kelsey R. Fulton, James Parker, Matthew Hou, Michelle L. Mazurek, and Michael Hicks. Understanding security mistakes developers make: Qualitative analysis from build it, break it, fix it. USENIX Security 2020. Lecture slides
Sep 25	Project pitches	Individual students, or groups who wish to work together, should prepare project pitches. See the syllabus for details.
Sep 30	Empirical evaluations: Fuzz testing	Klees, G., Ruef, A., Cooper, B., Wei, S. and Hicks, M., 2018, October. Evaluating fuzz testing. In CCS 2018. Schloegel, Moritz, et al. Sok: Prudent evaluation practices for fuzzing. In S&P 2024. Optional/reference: Blackburn, Steve, et al. SIGPLAN Empirical Evaluation guidelines. 2018. Lectures slides
Oct 2	Statistical tests: Pitfalls	Jenny Tang, Lujo Bauer, and Nicolas Christin. Misuse, Misreporting, Misinterpretation of Statistical Methods in Usable Privacy and Security Papers. In SOUPS 25. Jim Frost, What is P hacking: Methods & Best Practices, 2024. Optional: Stephanie M. Lee. Here's How Cornell Scientist Brian Wansink Turned Shoddy Data Into Viral Studies About How We Eat. BuzzFeed News, Feb 2018. Lecture slides
Oct 7	Threat modeling - Adam Shostack, Shostack Associates	Adam Shostack. Experiences Threat Modeling at Microsoft. MODSEC@ MoDELS 2008 (2008): 35. Adam Shostack. Nothing Is Good Enough: Fast and Cheap Are Undervalued as Influencers of Security Tool Adoption. IEEE S&P magazine, Jan-Feb 2023. Optional: Kaur, Harjot, et al. "Threat modeling is very formal, it's very technical, and also very hard to do correctly": Investigating Threat Modeling Practices in Open-Source Software Projects. USENIX Security 2025.
Oct 9	Fall break, no class	Project proposals due. See the syllabus for details.
Oct 14	Password managers, ethics of human studies	Sanam Ghorbani Lyastani, Michael Schilling, Sascha Fahl, Michael Backes and Sven Bugiel. Better managed than memorized? Studying the Impact of Managers on Password Strength and Reuse. In SOUPS 2018. Tadayoshi Kohno, Yasemin Acar, Wulf Loh. Ethical Frameworks and Computer Security Trolley Problems: Foundations for Conversations (will discuss the full PDF). In USENIX Security 2023.
Oct 16	What's Still Missing in Static Analysis? A Decade-Long Journey - Mayur Naik, Prof of CIS @ UPenn	Guest lecture about static analysis technology (including for finding security bugs), the influence of LLMs on it, and how we measure progress.
Oct 21	Student presentations begin
Oct 23	Student: Arya Sanjary
Oct 28	Student: Sydnie-Shea Cohen
Oct 30	Student: Noopur Bhatt
Nov 4	Student: Jiayi Xin
Nov 6	Student: Bharath Namboothiry
Nov 11	Student: Davis Brown
Nov 13	Student: Lucia Kulzer
Nov 18	Student: Ali Shirzad
Nov 20	Student: Zhiyao Tang

Empirical Security & Privacy, for Humans

SCHEDULE