|
SoftFocus was developed at the Institute for
Reliability and Risk Analysis, and forms part of a suite of
programs designed to work in the Microsoft Windows environment.
OVERVIEW
Software for a Shot-Noise
Reliability Model
The research that motivates this software is described by Al-Mutairi, Chen
and Singpurwalla, [1]. The motivation for the research was software
reliability modeling for the test-debug stage of software testing. It is
also applicable to any process that experiences reliability growth or
decay, provided that the data takes the form of inter-failure times. In Lynn, [2], it is shown that the
predictive performance of the model is superior to two competing models.
The details of the probability model are described in [1]. Other features
of the model are described in [2]. However, it is possible to use this
software without reference to either of those papers. In the following we
briefly will outline features of the software.
MAIN WINDOW
Figure 1. Main Window
The starting window of the software is shown in
Figure 1. A title bar runs across the top of the window, with a number of
categories: File, Edit, Parameters, Run, View and Help. The major options
are also represented on the toolbar that lies below the title bar. In order
these buttons represent ``New File,'' ``Open File,'' ``Save File,''
``Specify Parameters,'' ``Perform Analysis,'' ``View Prior/Posterior
distributions,'' ``View Probabilities'' and ``View Forecasts.''
In what follows, we discuss a typical use of the software.
DATA INPUT
Figure 2. Elicitation of Prior Beliefs
To start a new analysis, select the ``New'' option from the ``File'' menu.
This will clear any existing analysis from memory, prompting you to save
the current analysis if it is not saved. Alternatively, to continue analyzing
pre-existing data, select the ``Open...'' option. This will prompt you to
give a filename of an existing analysis.
Supposing that you have chosen to perform a new analysis, there are a
number of things you must do prior to performing the analysis. The first of
these is to input your prior opinions regarding the nature of the data. To
do this, select the ``Specify...'' option from the ``Parameters'' menu.
This will bring up a dialog box as shown in Figure 2. This dialog box can
be used in two ways. For the experienced user, the reliability growth and
decay parameters can be specified directly, although it is best to consult
the paper, [1], before choosing these values. The recommended approach for
inexperienced users is to choose from a number of pre-defined priors
('prior' means your initial beliefs). If you believe that the data should
exhibit reliability growth, meaning that the inter-failure times are
expected to increase, choose that option. If you expect the inter-failure
times to decrease, choose a reliability decay option. If you have little
idea regarding the failure process, choose the neutral prior. Note that the
specification of these beliefs should not be influenced by the data that
you wish to analyze.
One can also define the prior (initial) belief about the probability of
reliability decay, growth or neither. Again, this is best left to the
experienced user. To specify these probabilities, specify a figure for the
probability of reliability growth and a figure for the probability of reliability
decay. These are probabilities and are therefore restricted to lie between
0 and 1. Also, the sum of the three probabilities must equal 1, which
enables the software to automatically calculate the probability of neither
growth nor decay. If this figure appears in red, it means that the
specification is not allowable since the sum of the first two probabilities
exceeds one.
To view the prior distribution, click on the button labeled ``View
Distribution.'' This image represents the prior probability density of the
parameters of the model. The bigger the dots, the higher the
probability density at that point. Although this image is mainly of use to
the experience user, it can also be interpreted by any user. The red line
represents the density of the parameters, given that there is neither
reliability growth nor decay. The area above the line represents that
probability density given that there is reliability decay; the area below
shows the density when there is reliability growth. More mass close to the
red line represents neutral beliefs; mass shifted away from the diagonal
red line represents more definite beliefs.
When you are happy with the choice of prior distribution, select ``OK'' to
finish. If you wish to cancel, select ``Cancel.''
FAILURE DATA
To enter the data, simply double-click the mouse button on the failure
number and enter the relevant figure in the entry-box provided. You may
also use clipboard facilities on this data. To cut or copy, simply
highlight the required data points, and select ``Cut'' or ``Copy.'' To
paste a column of data (from, perhaps, a spreadsheet), simply select
``Paste.'' You will be warned if the data is of an incorrect form.
ANALYSIS
Once data has been entered and prior beliefs have been specified, analysis
can be performed. However, prior to performing an analysis, you may wish to
change some program options. To do this, select ``Options...'' from the
``Run'' menu. This gives a number of program options. In the ``Calculate''
box, you may specify what you want the program to calculate. Typically, you
will want 1-step ahead predictions after each successive data point
considered. The program will also forecast future failures (if so desired),
where the forecast horizon (number of future forecasts) may be specified as
required. The ``Samples'' box allows you to specify the number of samples
required at each stage. The calculations are performed by sampling values
and the higher the number of samples, the higher the accuracy of the
forecasts. However, there is a time overhead associated with taking a large
number of samples. The default value, 1000, is a good compromise. For
quick, inaccurate results, consider a value like 200; for very accurate
results, try 5000. The actual number will be a function of personal judgment,
the number of data points or forecasts required and the speed of your
machine. Feel free to experiment, but bear in mind that small values (less
that 200) will give poor forecasts.
Once the program options have been set, you may start the analysis. To do
this, select the ``Start'' option in the ``Run'' menu or press the circular
button on the toolbar. A dialog box will appear, informing you of the
status of the analysis. The bar shows what percentage of the samples has
been taken for each failure in turn. If you wish to terminate the analysis,
press the ``Stop'' button. This will stop the analysis at the stage
indicated. The results can still be displayed, but will be limited to the
analysis performed.
RESULTS
You are now in a position to view the results of the analysis. The
available options are: Prior and Posterior Distributions
|
Figure 3. Prior and Posterior Distributions
of Parameters
|
|
The
first option, obtained via the ``Parameters'' option of the ``View''
menu, enables you to view the prior and posterior distributions of the
model parameters. Here, prior means your views before seeing the data, posterior
means your revised views subsequent to seeing the data. Figure 3 shows
the results of a typical analysis, where the prior distribution was the
``Neutral prior.'' in this example, most of the mass has shifted to the
bottom, indicating strong evidence of reliability growth. Note though
that these plots are of little interest to the user unfamiliar with the
technical aspects of the model.
|
Posterior Probabilities
Figure 4. Posterior Probabilities
Of more relevance to the typical user of the software is the
``Probabilities'' option of the ``View'' menu. Selecting this option will
produce a graphic indicating the probabilities of reliability growth, decay
or neither subsequent to each data point considered (see 4). The red bar is
the probability of reliability decay, which in this example is shrinking,
indicating less and less evidence for decay. The blue bar is the
probability of neither growth nor decay and is again shrinking. The green
bar represents the probability of reliability growth at each stage. Note
that this bar is increasing in height throughout the analysis, indicating
that the data gives quite strong evidence of reliability growth.
This graphic is of good use to people with little understanding of the
technical aspects. For instance, suppose that a software product is being
de-bugged, and the data represents successive inter-failure times. If the
green bar becomes quite short, or the blue one quite large, there is
evidence that the de-bugging process is having little effect, and this
might provoke a management decision to call off the effort or change
strategies.
PREDICTIONS
Figure 5. Observed and Predicted Values
The final display option is to display the observed and 1-step ahead
predicted inter-failure times (see Figure 5). This graphic is useful for
two reasons. Not only does it display the future predictions, but it gives
a visual indication of how well the software is performing in modeling the
data. Note that typically the plot of forecast values will tend to be
smoother than the actual values, since the actual values often show a high
degree of variability. Note also that the forecast values plotted are the
expected value of the next inter-failure time, rather than a probability
distribution.
To save the analysis, select the ``Save'' option from the ``File'' menu.
Alternatively, to continue with another analysis select the ``New'' option.
BIBLIOGRAPHY
Al-Mutairi, D., Chen, Y. and Singpurwalla, N.D. (1996). 'Software Reliability
Models whose Concatenated Failure Rates are ``Shot-Noise'' Processes.' Technical
Report TR-96-3/GWU/IRRA. Under review.
Lynn, N.J. (1996). 'New Models for
Software Reliability: An Appraisal.' Technical Report TR-96-5/GWU/IRRA.
Under review.
|
