Chaos Testing or Chaos Engineering is the discipline of experimenting on a system in order to build confidence in the system’s capability to withstand turbulent conditions in production.
What does this mean for software testing? And how can chaos engineering be implemented?
Traditionally most of the research and principles of chaos engineering and testing have been on physical chaos – network disruptions, over-consumption of memory, CPU, etc.; we believe there is much more to chaos testing and engineering.
Qyrus has always focused on reusability and with many of our newer features, we are also focusing on repurposing. For example, repurposing a functional web test into a load-generating performance test, “repurpose a virtualized API into a chaos inducing agent.”
Chaos Testing with Qyrus API Process Testing
An ecommerce API process test on Qyrus
In the image above, you can see an ecommerce API process test on Qyrus which represents a set of orchestrated API calls that are made when an order is placed. As you can see, to successfully place an order after the user submits it from the UI, the system has to check for payment status, inventory, shipment dates, CRM, etc. All of these APIs must function as expected, in time, and in correct order for this transaction to be successful.
If you are responsible for testing the above system, here are just a few of the possible things that can go wrong:
The individual APIs don’t respond in time or in other words, they don’t stick to the SLA.
Any one or multiple APIs violate the contract. For example, the OrderID was supposed to be an integer which is 9 digits long and one of the APIs sends an alphanumeric 9 digits.
An API Schema is changed without notice.
In a well-orchestrated system with hundreds of APIs working in sync, one API going rogue by having any one or more of the 3 issues mentioned above can induce enough functionalchaos in the system. As a result, this affects many other processes than just placing an order. How well is this system designed to handle such conditions gracefully?
Qyrus SV + API Process Testing = Functional Chaos Testing
The critical question here is, how are testing teams equipped today to test for the above situations? At Qyrus, we have solved this exact problem through a combination of API Process Testing and Service Virtualization (test doubles).
Building an API process test is simple – just reuse functional tests and create connections to transfer data between endpoints to represent a real-world workflow; any of the APIs in this test can be virtualized by building and hosting it through our Service Virtualization module.
Service Virtualization in Qyrus, unlike a standard approach, gives freedom to teams using it to build their virtualized APIs with no code and have complete control of request – response pairs, persistence of data, delay induction, codeless schema design, and synthetic data generation.
The virtual API in the process test can be programmed to go rogue by delaying the response, violating the contract based on a certain request from other real APIs in the system, or passing incoherent data. By doing so, teams can perform a “What if?” on all the 3 conditions above and ensure the system handles the situations gracefully with proper error codes and statuses.
As traditional industry and business lines blur in the rapid pursuit of customer-centric ecosystems, there is mounting pressure on developers to release new capabilities, features, and changes quickly. The overall quality of the software is more critical than ever before. If these integrated, inter-dependent ecosystems are to survive and eventually thrive, they may want to look to automated software testing.
Common Misconceptions About Test Automation
Although software drives big ideas and massive changes across numerous business models, quality and testing continue to be the last innovators. Often, organizations view software testing as a necessary evil. Instead, they should embrace it as a strategic driver in a competitive world with limited second chances. In those instances, where organizations see testing and automated testing as an essential element of the overall solution and market strategy, they often succumb to continuous challenges that typically don’t lend themsleves to flexible and straightforward testing processes.
Consider this statement, “Well implemented automated software testing improves development output and speed to market.” 65% of organizations saw a reduction in test cycle time with automated testing and 62% saw reductions in test costs with automated testing (Sogeti, World Quality Report 2020-21). Speed to market through rapid product release cycles is a basic expectation in today’s market. Although this is true, users won’t forgive or, more importantly, are quick to punish buggy and poor-quality software. These issues caused by poor or inadequate automated testing and technical/skill challenges can have a debilitating impact on delivering software.
Key benefits of automated testing:
Shorten development cycles: Automated testing saves time through faster processes and increases the pace of high-quality development.
Improved testing efficiency: The reusability introduced by automation improves overall testing and development efficiencies. It also means more test coverage where required, which is impossible in manual mode.
Implement CI/CD and DevOps: Automation testing is an essential element of Continuous Delivery and DevOps practices. It shortens the loop between developers and test results, allowing for faster feedback on quality. Automated testing can eliminate the burden of automating routine, labor-intensive tests that take up a tester’s time. The time saved can be spent automating more tests, designing, and monitoring test execution.
Reduce cost: There is a prevalent view that automated testing is cost-intensive. On the contrary, unlike manual testing methods, automated tests can be executed more frequently, faster, and allow testers to increase coverage. Test automation implemented correctly will significantly reduce costs.
How to Select the Right Automation Platform?
The importance of platform selection cannot be over-emphasized. It must fit the environment, skills, and objectives of future product roadmaps and is the essential core of your automated testing practice. Furthermore, its value is further enhanced by coordinating test execution with other delivery tools to create a software delivery pipeline best suited for your organization’s needs.
Consider these points while selecting a platform:
Current Environment: Align the platform to your environment. How will it integrate with the ecosystem that’s already in place?
Type: Is the platform SaaS or on-premise? Will it test your future enterprise solution?
Integrated: Is the platform an integrated testing suite? Is it capable of testing web apps, APIs, business processes, and mobile device? Is there a need for an integrated device and browser farm?
No Code or Coding Model: Align with your team’s skillsets but also future-proof to the extent possible. Do you want to invest in a multitude of coding models?
Support: Is there product onboarding and testing process support?
Types of Automated Testing:
Modular Testing
Define and manage your tests as small, functional-based modules tested independently or efficiently grouped into functional or end-to-end business process test suites. Start automating small, simple test cases and increase the complexity of your scripts by identifying tests prone to human error, that use multiple data sets, and that require repetitive test runs on numerous builds.
Self-Healing Tests
Automation scripts are often flaky and brittle when the configurations, objects, and test data change since the last test execution. Even when scripts are maintained – which is often not the case – the changes are not continuously updated in the test script. This can cause erroneous test results. In many cases, it is impossible to fix scripts in a timely, cost-effective manner manually. Self-healing tests identify and remedy these issues, which often can be as simple as a button that has moved somewhere else on the page.
Mock and Stub
Reduce variability around test failures. Tests should fail when there is a bug. You can design and implement mocks and stubs until real system components are available for end-to-end testing.
Govern Automated Testing
Create a governance structure for automation testing. Make transparency a core element of the automation strategy. That way, all stakeholders – developers, database teams, business analysts – can contribute to the success and take advantage of the value provided by the platform.
What is Qyrus and how it can help you:
Qyrus is an on-demand SaaS codeless automated testing platform that deploys machine learning algorithms, an intuitive user experience, and a collaborative codeless approach to test automation. It not only empowers developers and testers with automation testing, but, just as important, the business team. When you partner with Qyrus, you can expect an exceptional user experience and greater productivity from your development and testing teams.
Jerin Mathew M M is a seasoned professional currently serving as a Content Manager at Qyrus. He possesses over 10 years of experience in content writing and editing, primarily within the international business and technology sectors. Prior to his current role, he worked as a Content Manager at Tookitaki Technologies, leading corporate and marketing communications. His background includes significant tenures as a Senior Copy Editor at The Economic Times and a Correspondent for the International Business Times UK. Jerin is skilled in digital marketing trends, SEO management, and crafting analytical, research-backed content.
