You are here: Home » Archives for testing
Writing by Rich on Saturday, 18 of April, 2009 at 10:18 am
Gall’s Law: “A simple system may or may not work. A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.” - John Gall, Systemantics
This is a true principle that holds true to sports, education, family life, computer systems, and even software development and testing. If we don’t have a working simple system in any of these areas—if we haven’t mastered the basics, the core—then the difficult, challenging, and more complex challenges will be beyond our capability. We can’t go from zero to complex without paying the price beforehand to master the simple basics. It is not saying that we seek complexity. I believe that in any situation we should seek the minimum level of complexity that meets our needs.
It especially applies to software development and testing. When developing a web application, it is common practice to start with a simple working and tested core of functionality and grow in complexity from there. As we go, we test functionality, performance, and security along the way. We add a bit here and a bit there as we go, until we are satisfied with the final product. We don’t start with the complex and save our testing to the end. We release early and often, testing as we go.
The implications of this principle on software development and testing are profound. Start by creating a simple system or process that works for you and then add to it incrementally until it is just complex enough to meet all of your goals.
Category: Software Testing
Writing by Rajesh Patil on Sunday, 22 of March, 2009 at 11:51 am
Algorithms are used to detect trading opportunities within the market. It is, ‘placing a buy or sell order of a defined quantity into a quantitative model that automatically generates the timing and size of orders based on the goals specified by the parameters and constraints of an algorithm”. An algorithm describes a sequence of steps by which patterns in real-time market data can be recognized through various statistical analyses and responded to in order to detect trading opportunities in the market. In times to go it is expected that 40% of the trading volume in the US equities markets will be contributed by algorithmic trading. The algorithms hence must be entirely reliable to maximize the opportunities.
But, before taking a plunge into algorithm trading, the system needs to go through a thorough testing process. Ensure the reliability of a particular trading algorithm before using it in a live market, the software infrastructure that helps implement a trading algorithm should be reliable and the trading algorithm needs to be tuned for speed of execution.
One of the common methods of testing algorithmic trading is ‘backtesting’. Testing algorithmic trading requires continuous data flow such as LTP, LTQ and market depth. Here a simulator is used to replicate the past data, trade price, traded volume and market depth. Backtesting uses the historical intraday data to identify how the strategies would work under different situations.
Algorithm strategies can be classified based on the complexity of the business functionality. Higher complexity will lead to more risk on performance and the profitability. As algorithmic trading involves different permutations and combinations of market movements, testing these algorithms would also be very complex as each scenario has to be tested. To handle the complex functionality, DMA Strategies, Quantitative Algorithms, and Investment strategies are the algorithms strategies that need to be scrupulously tested.
Algorithms have expanded the capabilities of the trader, making each more productive. Algorithmic trading is speed oriented and highly automated which needs a high level of efficiency in identifying the opportunities to be profitable. Hence the importance for testing algorithmic trading strategies is increasing. AppLabs, with its extensive experience of functional and performance testing of mission critical systems in capital markets has proven committed to ensuring software reliability.
Category: Software Testing, Strategy, Trading Applications
Writing by AppLabs on Tuesday, 17 of March, 2009 at 11:35 am
BASEL II is a very comprehensive and complex document, which will have a large impact on systems in many cases. The new Accord will enable some firms to use their own internal risk-management methodology to calculate the capital they require as opposed to a prescribed regulatory calculation. However, this will require them to amass and process a considerable amount of historical-loss data. These databases will have to be built and integrated with the banks processes. Data must be available to the banks and their subsidiaries across all geographical locations.
The solutions that will be delivered to cater for the BASEL II regulation changes must supply several key points of functionality. They must be able to accept and record external and internal ratings data; they must keep risk evaluation data for the required period (which at the time of writing is a rolling five years), and provide access to historical data on command from any relevant area of the business; the systems must be able to support a suitable number of users, allowing for future growth; transactions must be tested end-to-end to check the different levels of hardware and software involved; a risk-based testing approach should be employed to ensure the most critical areas of the systems receive the lion’s share of the testing; the new hardware/software (and all business critical processes) must be recoverable in the event of a disaster situation causing systems outage. Not only should the full business critical list of systems and processes be tested, but also individual sub-systems should be tested; a regression test suite must be constructed for use when testing that existing functionality remains intact when system enhancements are added; and finally, the new systems must be integrated smoothly into the banks’ processes and systems;
Apart from the present Basel II Accord, the further amendments will surely be needed upon completion of the review period adding to its complexity and ultimate implementation throughout the industry. The next few years will hence be strenuous for finance organizations who, as well as implementing changes for the BASEL II Accord, will also potentially be faced with large-scale programs such as the Euro, Straight Through Processing / T+1 and other regulatory changes.
For the compliance of the same, the organizations should look at ways in which testing can become more standardized and more cost effective. AppLabs’ advice to organizations is to ensure appropriate testing strategies and plans are in place to mitigate the inherent risk of changes to IT systems. Organizations must also ensure that their testing programs provide sufficient coverage and appropriate prioritization of tests and testing.
Category: BASEL II, Compliance Testing
Writing by Chakri on Tuesday, 17 of February, 2009 at 11:55 am
Exposing systems to the internet increases the risk that security weaknesses in those systems will be leveraged to compromise the system or the underlying data. Further, the organizations risk direct financial loss, loss of reputation and legal repercussions. It is therefore necessary to examine the actual business risks this brings, understand the basic difficulties in implementing “secure systems”, and adequately test internet applications for security, as well as functionality and load performance, before they are exposed to the net.
Testing, the security testing in particular is a solution to mitigate these risks. Security testing of internet solutions provides two fundamental services, it allows cost-effective selection of security controls at all stages of the project cycle, allowing proper integration of security measures (procedural and technical) into the final solution and gives the Management a firm evidence of the level of security provided, showing that, in the event of a security breach, “due diligence” was exercised, which may limit damages claims or criminal liability.
Testing a system involves a number of separate checks. The infrastructure design should be implemented to allow secure operation, site functionality should be examined to ensure that access to sensitive information and administrative functions is protected appropriately, services necessary for the business process should be running on web-facing servers (the more different systems, the greater the likelihood of a serious flaw), network traffic should be monitored to check for plain text transmission of user names and passwords (whether related to site users or to back-office functions such as databases).
If flaws are found, detailed analysis should follow, which will attempt to identify software patches, replace service daemons or applications, or additional technical issues.
Category: Software Testing, Web
Writing by Ganesh on Tuesday, 17 of February, 2009 at 11:44 am
The new Regulation National Market System (Reg NMS) has brought in major changes in the structure of the equities markets in the USA. As the original NMS framework dates back almost three decades, it is obvious that the old framework cannot fit into the intricate trading systems of the present era. The present modernized and strengthened structure has brought in four major rules, The Order Protection Rule, The Access Rule, The Sub-Penny Rule, and The Market Data Rule, which necessitate the organizations to place quality at the forefront of the implementation. To achieve the desired quality, organizations must adopt a meticulous and rigorous testing approach across the program, and this in return will provide two significant benefits:
- Compliance with the new regulations for Reg NMS
- An application/infrastructure that is robust.
IT spending on Reg NMS compliance technology is estimated to be more than a hundred million USD ($100mn). This has clearly revealed the significant impact the new regulations are having on the trading companies. However, as with most initiatives driven by compliance, it requires organizations to invest wisely – both financially and in time – to implement a quality system that will provide additional business benefits. In making these significant changes, it is imperative that the organization places quality at the forefront of the implementation.
AppLabs believes that quality products are the key factor to assess the level of readiness in terms of compliance criteria and achieving the substantial business benefits of a robust system. Organizations should be very careful when implementing the new trading system; by understanding the detailed requirements of Reg NMS one will be able to formulate a comprehensive testing approach to meet the required business objectives.
Category: Reg NMS
Writing by Durga on Tuesday, 17 of February, 2009 at 11:42 am
To increase competition and consumer protection in investment services, there emerged MiFID (Marketing in Financial Instruments Directive), a highly developed version of ISD (Investment Services Directive), a harmonized regulatory regime for investment services, intended to create a more open, competitive, and transparent market for financial services across the European Union. With the deadline for MiFID looming large, many organizations operating in the financial markets are compelled to think about the systems they will need in place for them to adhere to this new legislation.
With technology underpinning MiFID concept, it stands a mandate that companies fall well within its compliance limits.
However, in implementations of this nature, many often fall short due to inefficiencies in quality assurance and testing. And if the loopholes appear in MiFID strategies due to inadequate testing procedures, serious problems like financial penalties, damage to the company’s reputation, the start of a problematic relationship with the FSA and above all, failure to adhere to this legislation could result in legal consequences for the business.
For the MiFID compliance, AppLabs ensures that the coverage and prioritization of testing meets the needs of the business and reduces the inherent risks involved in making high-impact changes on IT systems. AppLabs analyzes, re-engineers, upgrades or even replaces the key applications. AppLabs provides the new levels of regulatory reporting and transparency, timely provisioning and testing of such systems, which ensures a definite edge in the marketplace. In addition, AppLabs performs interface and integration testing for matters such as publishing ‘firm prices’, getting data for the best execution and transaction reporting, audit trail and so on. Performing of comprehensive regression test suites, automates the extra information required for the new Suitability and Appropriate Tests, customer categorization information, changes to the underlying database structure and so on. This provides tremendous benefit to an organization introducing new functionality in a faster pace.
AppLabs is significantly enhanced to satisfy the new demands of MiFID to support the Best Execution directive. To avoid further risks, AppLabs, in addition to the comprehensive regression test on existing functionality, conducts performance testing, as at this stage it is fundamentally important since Best Execution relies on market data being available in real-time with trading systems and market feeds potentially under heavy load. Capacity planning and so-called database bulking are also the important parts of this testing process.
Compliance with MiFID hence implies major rework and enhancement of a variety of IT systems within many investment companies across the EU, and above all the pressure of the deadline. It has therefore never been more important that affected companies plan and execute thorough regression and performance tests, while also deploying monitoring technology to ensure compliance with MiFID once new or enhanced systems go live.
Category: Compliance Testing, MiFID
Writing by Chakri on Monday, 16 of February, 2009 at 11:28 am
The exponential growth of online transactions with credit and debit cards, though has facilitated the process, but has made itself susceptible to insecurity; it has opened the gateway to greater and devastating security risks. Thus emerged the need of curbing this issue with a set of security standards which is known as the PCCI DSS, the Payment Card Industry Data Security Standard (PCI DSS), created by the major credit card companies, intending to protect their customers from increasing identity theft and security breaches.
Virtually all businesses, regardless of their size, need to understand the scope of PCI DSS, and ways to implement network security that is compliant with PCI DSS guidelines. In doing so, they will avoid penalties or the possibility of having their merchant status revoked and potentially being banned from accepting or processing credit cards.
AppLabs, an independent software company, is such a service provider, which is compliant with PCI DSS guidelines and satisfies the PCI DSS requirements, which include security management, policies, procedures, network architecture, software design and other critical protective measures. This comprehensive standard is intended to help organizations to proactively protect customer account data.
For the compliance process of PCI DSS, a multifaceted security standard, AppLabs conducts an annual onsite PCI and sometimes a Self-Assessment Questionnaire (SAQ), which is filled in to validate the compliance. In addition to this, AppLabs scans the network perimeter by an Approved Scanning Vendor (ASV) every quarter, submits the report and hence highlights the compliance status, network vulnerabilities and vulnerable services classified as per the scoring pattern and severities prescribed by PCI DSS. The evidences of these and the application and network penetration tests are shared with card brands, hence proving that that AppLabs’ practices sound patch management and vulnerability management processes.
Category: Compliance Testing, PCI Data Security Standard