|
|
There have been several papers written on ODC that
describe the concepts, define the ODC scheme, discuss
the relationships among the ODC attributes, and
include results from pilot studies. For this reason,
we include only a short description of the ODC details
in this paper. ODC methodology provides both
a classification scheme for software defects and a set
of concepts that provides guidance in the analysis of
the classified aggregate defect data. "Orthogonal"
refers to the nonredundant nature of information
captured by the defect attributes and their values that
are used to classify defects. Much like the Cartesian
coordinates in geometry, nearly a decade of research
has shown that these attributes (and their values)
are adequate to "span" the interesting part of the
defect information space for most software development
issues. Unlike other defect analysis tools
used primarily by management, ODC targets the
technical team-developers, testers, and service personnel.
As a result, it is the technical team that classifies
defects, takes an active part in assessment of
data, and decides on the actions to be implemented.
ODC deployment process. The process for deploying
ODC has evolved over the last 10 years. However,
the following basic steps are critical in order for the
ODC deployment to be successful:
- Management must make a commitment to the deployment
of ODC and the implementation of actions
resulting from the ODC assessments.
- The defect data must be classified by the technical
teams and stored in an easily accessible database.
- The classified defects are then validated on a regular
basis to ensure the consistency and correctness
of the classification.
- Once validation has occurred, assessment of the
data must be performed on a regular basis. Typically,
the assessment is done by a technical person
who is familiar with the project and has the
interest and skills for analyzing data. A userfriendly
tool for visualizing data is needed.
- Regular feedback of the validation and assessment
results to the technical teams is important. It improves
the quality of the classification. It also provides
the teams with the necessary information so
that they can determine the appropriate actions
for improvement. This feedback is also important
in obtaining the necessary commitment from the
technical teams. Once they see the objective, quantified
data, and the reasonable and feasible actions
that result, commitment of the teams to the ODC
process usually increases.
- Once the feedback is given to the teams, they can
then identify and prioritize actions to be implemented.
When this ODC process has been integrated into the
process of the organization, the full range of benefits
can be realized. The development process and
the resulting product can be monitored and improved
on an ongoing basis so that product quality is built
in from the earliest stages of development.
Classification and validation of defects. The classification
of the defects occurs at two different points
in time. When a defect is first detected, or submitted,
the ODC submittor attributes of activity, trigger,
and impact are classified.
- Activity refers to the actual process step (code inspection,
function test, etc.) that was being performed
at the time the defect was discovered.
- Trigger describes the environment or condition that
had to exist to expose the defects.
- Impact refers to either perceived or actual impact
on the customer.
When the defect has been fixed, or responded to,
the ODC responder attributes, which are target, defect
type, qualifier, source, and age, can be classified.
- Target represents the high-level identity (design,
code, etc.) of the entity that was fixed.
- Defect type refers to the specific nature of the defect
fix.
- Qualifier specifies whether the fix that was made
was due to missing, incorrect, or extraneous code
or information.
- Source indicates whether the defect was found in
code written in house, reused from a library, ported
from one platform to another, or outsourced to a
vendor.
- Age specifies whether the defect was found in new,
old (base), rewritten, or refixed code.
Definitions of the attributes and a more detailed discussion
with examples can be found at http://www.research.ibm.com/softeng.
Typically, the ODC attributes are captured in the
same tool that is used to collect other defect information
with minor enhancements. Two methods are
used to validate data. The individually classified defects
can be reviewed for errors by a person with the
appropriate skills. This may be needed only until the
team members become comfortable with the classification
and its use. It is also possible to use an aggregate
analysis of data to help with validation. Although
this method of validation is quicker, it does
require skills beyond classification. In order to perform
a validation using this method, the validator
reviews the distribution of defect attributes. If there
are internal inconsistencies in the information contained
in the data or with the process used, it points
to potential problems in the quality of the data, which
can be addressed by a more detailed review of the
subset of defects under question. Even in cases where
there is a misunderstanding by a person in the classification
step, it is typically limited to one or two
specific aspects, which can be clarified easily. Once
the team understands the basic concepts and their
use, data quality is no longer a problem.
Data assessment. Once the data have been validated,
the data are then ready for assessment. When doing
an assessment, the concern is not with a single
defect as is done with causal analysis. Rather, trends
and patterns in the aggregate data are studied. Data
assessment of ODC classified data is based on the relationships
of the ODC attributes to one another and
to non-ODC attributes such as component, severity,
and defect open date. For example, to evaluate product
stability, the relationships among the attributes
of defect type, qualifier, open date, and severity of
defects might be considered. A trend of increasing
"missing function" defect type or increasing highseverity
defects may indicate that the product stability
is decreasing.
The following three case studies use ODC to measure
test effectiveness. In each case, ODC highlighted
exposures in the testing process that needed to be
addressed. The assessments provided the required
information for the final step in the ODC process-
selecting the appropriate actions for improvements.
The case studies include background information on
the products, how their ODC process was implemented,
and the details of the assessments, including
actions that resulted.
from paper: H. Munro M. Butcher and T. Kratschmer. Improving software testing via ODC:
Three case studies. IBM System Journal Software Testing and Verification, 41(1), 2002.
|
|
Forgot your password?
New user?
Up
Overview of Orthogonal Defect Classification
web site migration
