Performance: Still the Cinderella of the Testing World?
Writing by Ralph Decker on Monday, 12 of July, 2010 at 4:24 am
The importance of software testing has grown enormously over the last decade or so, which is reflected in the massive investment in people, training, methodologies, and tools. While it has never been more important to ensure that the end user experience is a positive one, there is one key area that is still often neglected: Performance Testing.
This is particularly surprising given the greater risk and prominence of failure. Software is still developed against vague or even absent performance requirements. Where requirements do exist, little or no time has been allocated in the project plan for testing them and a similar story applies to the budget set aside for the people and tools and test environments required.
The key to performance testing is getting the requirements right and defining them early in the project. In our experience, this is often made more difficult as new systems and technologies have no predecessor on which to base expectations. This is where an experienced test partner such as AppLabs can help early in the life-cycle by ensuring, relevant stakeholders are involved in identifying and agreeing the performance requirements for the system; requirements are captured in a specification document and signed-off by all parties concerned; requirements are defined in an appropriate manner; and requirements to take into account the competitive nature of your market and users expectations of continually improving performance.
In addition, an experienced test partner such as AppLabs ensures that opportunities are assessed for performance testing early in the life-cycle. It is often beneficial to performance test key sub-systems or components before they are integrated into the overall system.
While vast majority of projects will plan development and testing activities early in the project, relatively little attention is paid to non-functional testing, especially performance testing. Despite the fact that performance testing tools have been around for over 10 years now, there is still relatively little practical experience in today’s IT world.
Performance testing also occurs very late in the life-cycle and so the time allocated to it is often squeezed. However, unless adequate time and resources are allocated for performance testing, the risk of delivering a “broken swing” is much more probable.
In our experience, a performance test typically requires, 4-6 weeks (sometimes longer) in order to define, develop, and execute various test scenarios; an experienced performance test consultant; investment in a suitable tool that has been proven to work with your application (rental can be the most cost-effective route); 1,000’s or even 100,000’s of pieces of test data in order to feed the scenarios which may run for several hours; a dedicated test environment that is representative of production in terms of scale and specification; and contingency if performance problems are discovered.
AppLabs advocates the use of performance monitoring technology to complement the use of pre-production performance testing. This will alert users of any issues and may in some cases allow some pre-emptive action. Provided the right type of reporting is available, performance monitoring tools can also provide intelligence on usage patterns in the production environment which can be fed back into performance tests of later versions of the system.
Category: Offshore Software Testing, Performance Testing, Software Testing
- Add this post to
- Del.icio.us -
- Meneame -
- Digg
No comments yet.