The Test Maturity Model Integration (TMMi) is a framework that is developed to complement the existing Capability Maturity Model Integration (CMMI). It provides a structured presentation of maturity levels, allowing for standard TMMi assessments and certification. It enables a consistent deployment of the standards and collection of industry metrics. The TMMi has a rapidly growing uptake across Europe, Asia and the USA and owes its popularity to being the only independent test process measurement method.

The independent TMMi Foundation initiative has been established with the sole intent of developing the TMMi standard. The model it promotes can be used in isolation or in support of other process improvement models. It provides an independently managed data repository to support TMMi assessment method accreditation. It also provides Assessment Method Accreditation/Audit Framework for TMMi in accordance with ISO15504 and the process to certify commercial assessment methods against the standard model. Further it provides the certification and training/examination process, procedures and standards for formal, public accreditation of Assessors and Lead Assessors and the on-going management.

As with the CMMI staged representation, TMMi has a staged architecture for process improvement. It contains stages/levels from 1 to 5 through which an organization passes as its testing process evolves from one that is ad-hoc and unmanaged, to one that is managed, defined, measured, and optimized.

In September 2008 AppLabs became a supporter of the TMMi Foundation, after having ensured that the objectives of the TMMi Foundation were compatible with our views and objectives. This initiative was valuable to us as a company, to the testing industry as a whole, something which AppLabs could add a valuable contribution to and help advance. As AppLabs was already a CMMI Level 5 company, the TMMi alignment with CMMI proved beneficial.

On the whole, these industry metrics and information are sorely missing from the software quality management industry and AppLabs would like to be a key driver in rectifying this process outage. The decision was not taken lightly though, as there were and indeed still are some negatives. The TMMi Foundation is still busy defining the TMMi Levels, at the time of AppLabs decision to support the Foundation, only Level 2 was available. Level 3 has since moved on, but there is still significant work to be done on Levels 4 and 5. There is no guarantee that the ‘Industry’ will recognize this model, despite these negatives, AppLabs still felt it was a valuable investment and could reap rewards for all sponsors of the TMMi Foundation, including AppLabs, but more importantly, for the testing industry as a whole.

AppLabs is currently in the process of accrediting our TMMi model and our experienced Test Consultants with the TMMi Foundation. AppLabs will be offering TMMi reviews as part of our Consultancy Service line in the near future.

Warranty Statement:

The TMMi Foundation makes no warranties of any kind, either expressed or implied, as to any matter included, but not limited to, warranty of fitness for purpose or merchantability, exclusivity, or results obtained from use of the material. The TMMi Foundation does not make any warranty of any kind with respect to freedom from patent, trademark or copyright infringement.

The published material is produced with the permission of the TMMi Foundation. Contents should not be reproduced without the permission of AppLabs and the TMMi Foundation. TMMi® is a registered trademark of TMMi Foundation.

The resources for cloud computing are generally made available through applications with web interfaces that provide functionality to allocate, configure, and maintain the resources. Hence cloud computing adds some ambiguity to the systems hosting your applications causing some concerns around performance and security. And for this reason any cloud integration should have a comprehensive performance testing plan.

Firstly have an end to end understanding of the application performance; ranging from knowing the requirements to understanding the impact on the systems. And with cloud some additional unknowns that need to be considered and planned for are response times, impact on the systems and most importantly the shared resources.

Once deployed into the Cloud, the only practical testing model is the external testing model – that is driving load into the application from a secondary location over the internet. This can easily provide information about response time compared to traffic or load. There will be a few hurdles to work out with the impact on the systems, but you can get a pretty good indication if carefully planned and coordinated. To measure and understand the impact of traffic on computer systems you must understand the environment and the infrastructure, generate realistic traffic, analyze the capabilities and the limitation of the system under test, and find the remedy for performance related defects.

The cloud is a shared environment. This must be taken into consideration for performance aspects of your applications that you deploy into the cloud. Few considerations in this regard are, who you are sharing the resources with, how much of resource is being consumed, how it is impacting the performance of your application, and is usage and performance consistent.

Performance monitoring is going to be essential to assure that your application remains responsive. Constant and on-going monitoring is a good way to verify that the performance continues to remain stable. For this you need to monitor over time, watch for degradation, do changes in performance correspond to changes in your application. Unexplained changes in performance may be attributed to, shared resources and cloud provisioning tool

Several factors that can lead to inadequate performance testing are, lack of requirements, lack of understanding the fundamentals of performance testing, misinformation provided by tool and simulator vendors, limited budget or funding and enormity of the task.

The application system evolved as per the organizational and end user need. It eventually became an integral part of business, banking, trading, shopping, and even entertainment, demanding perfect performance. Testing in the cloud adds testing capabilities that were not available in the past like multiple geographic locations, lots of unique real IP addresses, to formulate traffic patterns. Most performance testing clouds are built out in controlled cloud environments. To perform a controlled cloud testing, access to the full systems, know where the load originates- IP Address, geographic locations, control the distribution of the load, use same test system/systems for retesting, perfect coordination of the user traffic, and correlate the load with end-user response times and the impact of the systems under test.

The environment and global warming pose a disastrous threat, hence organizations world wide focus on reducing energy wastage, CO2 and other greenhouse gas pollution. The IT world is no exception; testing and test professionals too play a pivotal role in reducing the impact of this threat.

Unfortunately due to timeline issues testing gets squeezed leading to inefficient and ineffective testing and this in return leads to resource wastage, ranging from people, money, hardware and ultimately energy. Poorly defined roles and responsibilities leading to diluted lines of accountability, the chaotic development lifecycle leading to testing chaos, defect/bug management that needs structure, no supporting configuration management processes, 100+ Unix environments that are under utilized, replicating overnight process that utilizes huge CPU resource, lead to energy wastage, delays, resource misuse and environmental impact.

Certain testing approaches to reduce the impact are:

• Pertaining to the project necessities, travel only when necessary, utilize communications technology; video-/tele-conferencing, minimize paper trails, use electronic media and define risk criteria.
• When handling Test Management, commission to sustainable, reusable environments, liaise closely with development community and be focused; perform impact analysis of change and direct testing accordingly.
• When dealing with the Test Process conduct an environmental analysis and design, order work methodically by maximizing efficiencies, and adhere to structured testing methodology (Static Testing, Business Process Testing, V-Model and so on)
• To ensure environmental concerns the existing testing roles should be evolved. The roles should be captured during requirements gathering phase, they should be prioritized accordingly, alongside functionality and other business requirements, suitably tested (throughout development lifecycle), outstanding concerns are properly assessed, prioritized, risk managed and scheduled for future release.

Having a more energy efficient IT infrastructure and highly focused and driven people will minimize the organizations exposure to energy price increases and subsequent limitations imposed on the business.