Archive for users

Fuzzing Definitions for QA

Posted in fuzzing with tags , , , on 2009-07-20 by crashtesting

A fuzz test is difficult to place in a regular test specification. I decided to write a few notes to help you in integrating fuzzing.

A fuzz test comes in two basic modes:

  • Deterministic and systematic model-based fuzzing, where each test is carefully built from the grammar and syntax of a protocol.
  • Semi-random and indefinite mutation-based fuzzing, where a template of somekind is used as a basis of mutations.

A fuzz process typically starts with the identification of the SUT (system under test), and the actual test planning process can be seen to start from that analysis. Some people see this as the phase where the attack surface is mapped. Some just call it system scanning. Basically, the SUT can consist of numerous devices, each with numerous communication interfaces, each with numerous protocols in layers but also independent of each other.

Definitions that are needed to map the above into a test plan include:

  • SUT: System Under Test, the actual target system.
  • DUT: Device Under Test, subset of the SUT, or sometimes same as SUT (in stand-alone device test environment).
  • Interface: Physical or logical interface between two or more DUTs.
  • Protocol: A specific layer of communications in an interface.
  • Fuzz test plan: A specific test setup, goals, and configurations for testing a protocol, on one interface, in one device.
  • Test group: A set on individual inputs that aim to finding one type of a flaw. Can consist of either generated or mutated tests.
  • Test case: One specific sequence of messages potentially triggering a flaw.

In most cases, the starting point for a test engineer is a test requirement where SUT, DUT, Interface, and Protocol are clearly defined. The goal in such case is to build a test plan, define its structure, test purpose, used metrics, test grouping (if any). Finally the easy phase is building the tests, and reporting the results.

From that on, it is someone else’s problem. 😉


Two Uses Of Fuzzing, Or Six, Or Something In-Between

Posted in fuzzing with tags , on 2009-01-16 by crashtesting

The people I know that talk actively about fuzzing seem to mix two different views where fuzzing is used. The first view is the “Role” of the user, and the second is the “Phase” of the test in the software life-cycle (note: NOT _development_ life-cycle, but the entire life-cycle).

The role of the tester is sometimes divided between Quality Assurance (QA) and Vulnerability Assurance (VA), but maybe an another view would be to look at the role from the test purpose perspective. Three roles can be easily identified:

  1. “Dev-fuzzer” is testing something that he or his team has built.
  2. “Test-fuzzer” is testing something that someone else has built.
  3. “Admin-fuzzer” is testing something that he or his team is planning to use or buy.

The roles given above place some limitations on which tools or techniques are most useful to the person conducting the tests. A developer will have access to the source code, and can also easily instrument the code for better tests. A system administrator (or any enterprise security or QA specialist) on the other hand has almost zero visibility to the internals of the software.

Even simpler categorization is the binary view created by one major event in the software life-cycle: the release, or launch date of the software. Pre-release fuzzing aims at finding as many problems as possible in software before it is released. Post-release fuzzing on the other hand aims at finding (and often disclosing) vulnerabilities in software that is often already widely used.

Third categorization can be based on who actually is doing the tests, whether it is the software development organization (or vendor), a researcher (hacker) or the end-user organization (enterprise).

All these categories can be mixed. I.e. you can have an enterprise user fuzzing a beta-release of a software, at a test-lab environment such as interoperability lab for a financial organization. Or a hacker that also will take part in the development of an open source project to fix the findings in the code for a future release of the same software.

Nobody said it was a simple categorization, but understanding that everyone has a different view into fuzzing can help you understand the problematics that your collegue is explaining to you. And we are here to learn, aren’t we.