Test Process Improvement - Random Testing to Scripted Testing

In any test project, many a times the test engineer is involved in random testing. Here what I mean by Random Testing is testing for which the script is not written. Many defects are identified as this type of random testing.

There are advantages and disadvantages for random testing - The advantages of random testing is that it is flexible based on the test engineer's desire and he is free to walk randomly in the application to serach the defect. No need for priorly written script.

But the disadvantage of random testing is that, the defect found due to random testing is found randomly only ie no script is written which catches this defect. Thus the challenge lies in converting the defects found due to random testing should have been found based on the scripted testing. Thus the defects found due to random testing needs to be analyzed and reasons should have been found why the script is not written which can catch this defect. This is one of the way for test process improvement.

Why QC is Inevitable

First let me briefly define QA and QC.

QA – Activities done to prevent the defects
QC – Activities done to detect the defect

It has been argued a lot many times that it is always cheaper, economical and justifiable to prevent the defect rather than correcting the defect later time. Here I am considering this assumption as valid.

I would like to delve further that why we haven’t reached a stage where QC is not required? Why we cant have robust processes through which we can always prevent the defects? Some one may like to answer that IT industry has not yet reached to that level of maturity. But then what about the other industries? IT may be considered relatively new industry but manufacturing is far older and mature industry then what IT industry is at today.

But even the manufacturing company has not reached to the stage of Nirvana where they can close their QC department. The reason probably lies in how our brain is functioning. Our decisions are based on our feelings, prejudices, past experiences etc. and these factors deviate us to take the rational and objective decision many times. Apart from this our mood varies a lot and because of that we lack the consistency. And these kinds of factors lead to the defect in the work product.

Thus our brain is programmed to make mistakes and that is the reason we can probably never close the QC activities. What we can do at the most is we can reduce the defects through better processes.

Software Development - Engineering or Art

The software community has intrinsic desire to compare it with the other engineering discipline ( By assuming the software development as an engineering discipline) I have come across a comparative statement like this -If civil engineers were building the bridge as the way software engineers are developing the software, then half of the bridge would have collapsed by now.

As per last Standish Group Report only 16% of the IT projects are completed on time within budget. The software development process is quite a lot chaotic in nature today.Are we really doing very bad at software development? Are software engineers not as competent as the engineers in the other streams?Not really - According to me, it is not perfectly justifiable to apply engineering principles to software development.Let us consider typical engineering discipline and software development. In a typical engineering discipline the same product is manufactured number of times using the same process ie if the product is a table, the same table is manufactured in a similar way for many times which is not the case with the software development. If we want to make the copy of the an existing software, we can have exact copy very easily without allowing any error in it but most of the time we are developing something completely new or modifying something that we havent done before. And this is significant deviation from the typical engineering discipline.And thats why we dont have commonly known engineering characterisitcs for our discipline like standard unit of measurement, conversion ratio among various units of measurement, precise estimation methodology etc.Software development today is somewhere between the art and engineering and hence applying only engineering principles of software development leads to inappropriate conclusion and misinterpreatation of the situation. Though the software development process is definately proceeding towards engineering discipline. But even in future there will be definitely some elements of art is going to remain in the software development process.

The need for considering the software development process as engineering discipline is derived from the business requirement of the software end user. The market always wants the software to be developed on time within budget and the software development community will be really happy if they can satisfy this market need. But the fact is that the software process itself is not a truly engineering process.

We need to understand this distinctive feature of software development so that we can better appreciate and probably better manage the software process.