Week 6: Penetration Testing and Fuzz Testing

Last week we looked at how automated tools could be used to assist developers and testers in finding important security bugs. We focused in particular on static analysis and symbolic execution as technologies.

This week, we look at the broader practice of penetration testing of which tools using these technologies form some part, but other practices and techniques are of interest too. We will focus in particular on fuzz testing, a technique that attempts to find potentially security-relevant software failures.

Learning Objectives

After the completion of this week’s material, you will:

• Understand what penetration testing is and what it achieves
• Know the basics of several state-of-the-art penetration testing tools
• Understand fuzz testing techniques and how they compare
• Be aware of several state-of-the-art fuzz testing tools

Video Lectures

• Penetration Testing: Introduction (10:11)
• Penetration Testing: Techniques and Tools (14:28)
• Fuzz Testing: Techniques and Tools (15:13)

Break out: Interview with Eric Eames

In September 2014, Mike interviewed Eric Eames, at the time a Principal Security Consultant at FusionX. In this interview we discussed principles and practice of penetration testing. The interview is required from an assessment perspective – some quiz material will be drawn from this interview’s content.

Mike Hicks interviews Eric Eames (31:46). Highlights, indexed by time:

• start - Introduction and background
• 1:33 - Penetration testing: what is it?
• 5:04 - Tools and techniques used in penetration testing
• 9:43 - Common technical and human mistakes in engagements
• 15:05 - Defining an engagement; pen testers as outsiders or insiders
• 17:50 - Surprising discoveries
• 19:33 - What else, in addition to penetration testing can help ensure security
• 23:05 - Undergrad education – what should we do?
• 26:39 - Prognosis for security, looking ahead

Break out: Interview with Patrice Godefroid

In September 2014, Mike had the pleasure of interviewing Patrice Godefroid, who is a Partner Researcher at Microsoft Research. In this interview we discussed principles and practice of fuzz testing in general, and whitebox fuzz testing in particular, especially as it has come to be used within Microsoft. The interview is optional from an assessment perspective, but recommended – there will no quiz questions on it per se, but it might help provide context about material from last week and this week.

Mike Hicks interviews Patrice Godefroid (35:06). Highlights, indexed by time:

• start - Introduction and background
• 1:08 - The state of the art in automated vulnerability detection
• 5:50 - What drove your interest in working on model checking/analysis?
• 10:13 - Comparing different fuzz testing techniques
• 19:21 - The story of deployment of fuzzing at Microsoft for whitebox fuzzing
• 24:57 - Trends in the use of automated analysis tools
• 31:06 - Open problems in automated testing tool development

In 2020, Patrice wrote a nice review article about fuzzing.

Supplemental Links

Here we present links to supplemental material, in case you are interested to read it (none is required for assessment).

• Ware report - introduced the idea of penetration testing, as well as many other foundational ideas in systems security
• CPT (pen testing) certification - establish your credentials as a pen tester
• Defcon CTF contest - be the first to find vulnerabilities in other competitors’ systems and patch them in your own

Penetration testing tools

These tools are all free, or have free versions.

• NMAP - “network mapper” scans network to find what’s connected to it
• Zap - web proxy and automatic vulnerability scanner
• Burp suite - Several pen testing tools (some versions are free)
• Metasploit - customizable platform for developing, testing, and using exploit code.
• Kali - Linux distribution with pre-installed pen testing tools.

Fuzz testing tools

Again, these tools are all free, or have free versions.

• American Fuzzy Lop (AFL) - popular, mutation-based, gray-box fuzzer
• Libfuzzer - a fuzzer that is similar in spirit to AFL, developed at Google
• OSSFuzz - a Google service for fuzzing open-source software projects; uses AFL, Libfuzzer, and Honggfuzz
• Radamsa - mutation-based black-box fuzzer
• SPIKE - network fuzzing framework

Quiz

The quiz for this week covers all of the material for this week.

Project

There is no new project for this week. All outstanding projects and assessments are due by 8am ET the week after the course ends.