Topic: Fuzz Testing

Message

   Fuzz testing is a software testing technique. The basic idea is to
attach the inputs of a program to a source of random data. If the
program fails (for example, by crashing, or by failing in-built code
assertions), then there are defects to correct.

The great
advantage of fuzz testing is that the test design is extremely simple,
and free of preconceptions about system behavior.

Uses

Fuzz
testing is often used in large software development projects that
perform black box testing. These usually have a budget to develop test
tools, and fuzz testing is one of the techniques which offers a high
benefit:cost ratio.

Fuzz testing is also used as a gross
measurement of a large software system's quality. The advantage here is
that the cost of generating the tests is relatively low. For example,
third party testers have used fuzz testing to evaluate the relative
merits of different operating systems and application programs.

Fuzz
testing is thought to enhance software security and software safety
because it often finds odd oversights and defects which human testers
would fail to find, and even careful human test designers would fail to
create tests for.

However, fuzz testing is not a substitute for
exhaustive testing or formal methods: it can only provide a random
sample of the system's behavior, and in many cases passing a fuzz test
may only demonstrate that a piece of software handles exceptions
without crashing, rather than behaving correctly. Thus, fuzz testing
can only be regarded as a proxy for program correctness, rather than a
direct measure, with fuzz test failures actually being more useful as a
bug-finding tool than fuzz test passes as an assurance of quality.

Fuzz testing methods

As
a practical matter, developers need to reproduce errors in order to fix
them. For this reason, almost all fuzz testing makes a record of the
data it manufactures, usually before applying it to the software, so
that if the computer fails dramatically, the test data is preserved.

Modern software has several different types of inputs:

* Event driven inputs are usually from a graphical user interface, or possibly from a mechanism in an embedded system.

* Character driven inputs are from files, or data streams.

* Database inputs are from tabular data, such as relational databases.

There are at least two different forms of fuzz testing:

* Valid fuzz attempts to assure that the random input is reasonable, or conforms to actual production data.

* Simple fuzz usually uses a pseudo random number generator to provide input.

* An combined approach uses valid test data with some proportion of totally random input injected.

By
using all of these techniques in combination, fuzz-generated randomness
can test the un-designed behavior surrounding a wider range of designed
system states.

Fuzz testing may use tools to simulate all of these domains.

Event-driven fuzz

Normally this is provided as a queue of datastructures. The queue is filled with data structures that have random values.

The
most common problem with an event-driven program is that it will often
simply use the data in the queue, without even crude validation. To
succeed in a fuzz-tested environment, software must validate all fields
of every queue entry, decode every possible binary value, and then
ignore impossible requests.

One of the more interesting issues
with real-time event handling is that if error reporting is too
verbose, simply providing error status can cause resource problems or a
crash. Robust error detection systems will report only the most
significant, or most recent error over a period of time.

Character-driven fuzz

Normally this is provided as a stream of random data. The classic source in UNIX is the random data generator.

One
common problem with a character driven program is a buffer overrun,
when the character data exceeds the available buffer space. This
problem tends to recur in every instance in which a string or number is
parsed from the data stream and placed in a limited-size area.

Another is that decode tables or logic may be incomplete, not handling every possible binary value.

Database fuzz

The
standard database scheme is usually filled with fuzz that is random
data of random sizes. Some IT shops use software tools to migrate and
manipulate such databases. Often the same schema descriptions can be
used to automatically generate fuzz databases.

Database fuzz is
controversial, because input and comparison constraints reduce the
invalid data in a database. However, often the database is more
tolerant of odd data than its client software, and a general-purpose
interface is available to users. Since major customer and enterprise
management software is starting to be open-source, database-based
security attacks are becoming more credible.

A common problem
with fuzz databases is buffer overrun. A common data dictionary, with
some form of automated enforcement is quite helpful and entirely
possible. To enforce this, normally all the database clients need to be
recompiled and retested at the same time. Another common problem is
that database clients may not enderstand the binary possibilities of
the database field type, or, legacy software might have been ported to
a new database system with different possible binary values. A normal,
inexpensive solution is to have each program validate database inputs
in the same fashion as user inputs. The normal way to achieve this is
to periodically "clean" production databases with automated verifiers.

Author	Message
tanushree Senior Member Joined: 04Apr2007 Online Status: Offline Posts: 2160	Topic: Fuzz Testing Posted: 15Oct2007 at 4:45am
	Fuzz testing is a software testing technique. The basic idea is to attach the inputs of a program to a source of random data. If the program fails (for example, by crashing, or by failing in-built code assertions), then there are defects to correct. The great advantage of fuzz testing is that the test design is extremely simple, and free of preconceptions about system behavior. Uses Fuzz testing is often used in large software development projects that perform black box testing. These usually have a budget to develop test tools, and fuzz testing is one of the techniques which offers a high benefit:cost ratio. Fuzz testing is also used as a gross measurement of a large software system's quality. The advantage here is that the cost of generating the tests is relatively low. For example, third party testers have used fuzz testing to evaluate the relative merits of different operating systems and application programs. Fuzz testing is thought to enhance software security and software safety because it often finds odd oversights and defects which human testers would fail to find, and even careful human test designers would fail to create tests for. However, fuzz testing is not a substitute for exhaustive testing or formal methods: it can only provide a random sample of the system's behavior, and in many cases passing a fuzz test may only demonstrate that a piece of software handles exceptions without crashing, rather than behaving correctly. Thus, fuzz testing can only be regarded as a proxy for program correctness, rather than a direct measure, with fuzz test failures actually being more useful as a bug-finding tool than fuzz test passes as an assurance of quality. Fuzz testing methods As a practical matter, developers need to reproduce errors in order to fix them. For this reason, almost all fuzz testing makes a record of the data it manufactures, usually before applying it to the software, so that if the computer fails dramatically, the test data is preserved. Modern software has several different types of inputs: * Event driven inputs are usually from a graphical user interface, or possibly from a mechanism in an embedded system. * Character driven inputs are from files, or data streams. * Database inputs are from tabular data, such as relational databases. There are at least two different forms of fuzz testing: * Valid fuzz attempts to assure that the random input is reasonable, or conforms to actual production data. * Simple fuzz usually uses a pseudo random number generator to provide input. * An combined approach uses valid test data with some proportion of totally random input injected. By using all of these techniques in combination, fuzz-generated randomness can test the un-designed behavior surrounding a wider range of designed system states. Fuzz testing may use tools to simulate all of these domains. Event-driven fuzz Normally this is provided as a queue of datastructures. The queue is filled with data structures that have random values. The most common problem with an event-driven program is that it will often simply use the data in the queue, without even crude validation. To succeed in a fuzz-tested environment, software must validate all fields of every queue entry, decode every possible binary value, and then ignore impossible requests. One of the more interesting issues with real-time event handling is that if error reporting is too verbose, simply providing error status can cause resource problems or a crash. Robust error detection systems will report only the most significant, or most recent error over a period of time. Character-driven fuzz Normally this is provided as a stream of random data. The classic source in UNIX is the random data generator. One common problem with a character driven program is a buffer overrun, when the character data exceeds the available buffer space. This problem tends to recur in every instance in which a string or number is parsed from the data stream and placed in a limited-size area. Another is that decode tables or logic may be incomplete, not handling every possible binary value. Database fuzz The standard database scheme is usually filled with fuzz that is random data of random sizes. Some IT shops use software tools to migrate and manipulate such databases. Often the same schema descriptions can be used to automatically generate fuzz databases. Database fuzz is controversial, because input and comparison constraints reduce the invalid data in a database. However, often the database is more tolerant of odd data than its client software, and a general-purpose interface is available to users. Since major customer and enterprise management software is starting to be open-source, database-based security attacks are becoming more credible. A common problem with fuzz databases is buffer overrun. A common data dictionary, with some form of automated enforcement is quite helpful and entirely possible. To enforce this, normally all the database clients need to be recompiled and retested at the same time. Another common problem is that database clients may not enderstand the binary possibilities of the database field type, or, legacy software might have been ported to a new database system with different possible binary values. A normal, inexpensive solution is to have each program validate database inputs in the same fashion as user inputs. The normal way to achieve this is to periodically "clean" production databases with automated verifiers. Post Resume: Click here to Upload your Resume & Apply for Jobs

	IP Logged


One Stop Testing Forum : Types Of Software Testing @ OneStopTesting : Performance & Load Testing @ OneStopTesting