Archive for fuzz testing

Fuzzing Definitions for QA

Posted in fuzzing with tags , , , on 2009-07-20 by crashtesting

A fuzz test is difficult to place in a regular test specification. I decided to write a few notes to help you in integrating fuzzing.

A fuzz test comes in two basic modes:

  • Deterministic and systematic model-based fuzzing, where each test is carefully built from the grammar and syntax of a protocol.
  • Semi-random and indefinite mutation-based fuzzing, where a template of somekind is used as a basis of mutations.

A fuzz process typically starts with the identification of the SUT (system under test), and the actual test planning process can be seen to start from that analysis. Some people see this as the phase where the attack surface is mapped. Some just call it system scanning. Basically, the SUT can consist of numerous devices, each with numerous communication interfaces, each with numerous protocols in layers but also independent of each other.

Definitions that are needed to map the above into a test plan include:

  • SUT: System Under Test, the actual target system.
  • DUT: Device Under Test, subset of the SUT, or sometimes same as SUT (in stand-alone device test environment).
  • Interface: Physical or logical interface between two or more DUTs.
  • Protocol: A specific layer of communications in an interface.
  • Fuzz test plan: A specific test setup, goals, and configurations for testing a protocol, on one interface, in one device.
  • Test group: A set on individual inputs that aim to finding one type of a flaw. Can consist of either generated or mutated tests.
  • Test case: One specific sequence of messages potentially triggering a flaw.

In most cases, the starting point for a test engineer is a test requirement where SUT, DUT, Interface, and Protocol are clearly defined. The goal in such case is to build a test plan, define its structure, test purpose, used metrics, test grouping (if any). Finally the easy phase is building the tests, and reporting the results.

From that on, it is someone else’s problem. 😉


What Is Fuzzing

Posted in fuzzing with tags , , , on 2008-11-17 by crashtesting

Fuzz testing, or fuzzing, originally meant a simple testing technique for feeding random input to applications (see Fuzz by University of Wisconsin, 1990). Today, it is much more optimized. Model-based fuzzing tools have been available since 1999, from research teams such as PROTOS. Fuzzing techniques can basically be divided in four different categories:

  1. Random fuzzing: has close to zero awareness of the tested interface.
  2. Capture-replay fuzzing: learns the protocol from templates such as traffic captures or files.
  3. Block-based fuzzing: breaks the syntax of the tested interface into blocks of data, which it semi-randomly mutates.
  4. Model-based fuzzing: builds an executable model of the protocol based on protocol specification, which it then uses for generating systematic non-random test cases.

In short, fuzzing is about negative testing, generation on non-conformant messages in order to crash software. The failures (crashes, hangs, busy-loops, …) are studied from risk analysis perspective to see if they are something that need to be fixed. Most discoveries can also be identified as software vulnerabilities.