About/Overview The following is a case study about a regulated, leading UK bank and its partnership with Qyrus. A leader in innovative financial solutions, this bank serves an affluent community and offers multiple competitive savings accounts and lifestyle services.
Qyrus, on the other hand, is a scalable, one-stop-shop SaaS testing platform for all digital applications with innovative AI solutions. Its capabilities help enhance testing operations and overall quality.
The bank’s stellar services, along with what Qyrus offers, allow for a successful strategic partnership. This case study explores the partnership between the two. The challenges, the successes, the awards, and the quest for betterment. Another section has highlighted the use of AI in banking applications.
Challenges in the Banking Sector: Like every industry, the banking sector too has challenges. These challenges significantly affect bank agility and growth. Some of these are highlighted below, followed by a section that explores how Qyrus has helped the bank navigate these issues.
Traditional methods, such as Selenium Java for web testing and XCode for mobile testing, are known to pose significant challenges. These challenges have led to longer scripting and debugging periods.
Traditional methods, which banks usually use and generally require much manual effort, lack comprehensive test coverage. This limitation prevents the bank from maintaining consistent quality assurance across platforms.
In order to help the bank with these problems, Qyrus’ capabilities seemed an excellent addition.
Life with Qyrus The following section highlights the bank’s collaboration with Qyrus and how it helped the institution achieve greater efficiency overall.
Reduction of Test execution times: Qyrus allows test execution at lightning speed. This was evidenced by the fact that using the service (Qyrus) significantly reduced test execution times for the bank. Processes that initially required 150 steps needed to take 2 hours and sometimes 2 days, now needed 9 minutes to execute.
Qyrus’ low code/no code approach simplified script creation and execution. The total training time needed was just a week, signifying ease of use. Qyrus’ clients gave the service glowing reviews, with comfort rated at an impressive 7/10. This further allowed broader coverage across banking applications, increasing defect detection and improving application quality.
Automated test scripts were executed for different areas, such as:
Mobility (iOS + Android OS: 88 scripts),
Components (iOS + Android OS: 160 scripts),
Web-Workflow Permissions (85 scripts),
Web – Lending/CS Ops (84 scripts),
Component – Lending (88 scripts)
BTL + Doc Upload (36 scripts).
3. Qyrus (linked with Slack to notify execution results) identified 28 defects in Release 17.0 and 7 defects in Release 17.1. The bank used Qyrus’ advanced reporting capabilities to help detect errors in the executions.
Achievements Through the Partnership: The progress with Qyrus has not gone unrewarded. The partnership has received recognition in the distinguished Transformation category of the 2023 ISG Paragon Awards™ EMEA. This award celebrated Quinnox’s innovative capabilities in driving the bank’s digital-first goals. The bank and Qyrus were also finalists in the ‘Functional Category’ in the ‘Best Test Automation Project. They were also recognized in the ‘Best Cultural Transformation’ category at the 2023 Digital Transformation Awards.
Future Aspirations Even after the success of the partnership, Qyrus and the bank are not complacent. Together, they’re striving to achieve even greater things. Some of these are listed below:
Together, the bank and Qyrus are working towards creating reusable functionsin the Component service. This will allow for mirroring Web service capabilities, enhancing the user experience.
New features for visual testing in Mobility and Component services are also being worked on to improve quality assurance.
A Slack Notification timer feature is being introduced across projects to improve team communication.
Overall Results and Outcomes As mentioned above, the bank has achieved significant reductions in testing time and increased test case throughput. This has allowed for operational efficiencies and cost savings. This in turn has led to better financial planning and resource allocations, allowing for the better provision of financial services.
Given the AI wave, this study is incomplete without an analysis of Artificial Intelligence in Banking.
AI use via Qyrus’ testing solutions has improved the bank’s approach to quality assurance. These tools have enhanced the detection of defects and improved testing processes, allowing greater accuracy and efficiency.
AI-driven insights into the bank’s operations have allowed strategic decision-making and improved overall service delivery.
Conclusion Innovative testing solutions help achieve a bank success story, and Qyrus’ success story is indeed a testament to that fact. It is simple to use, scalable to one’s testing needs, and implements smart features to help augment the testing process. It’s also entirely secure. A senior official from the bank praised the Quinnox team and Qyrus for being a ‘great addition’ to their original delivery features. He emphasized that within a few months, with the help of Qyrus, the bank was able to create an impressive test suite that consisted of complex scenarios across multiple platforms and channels. The fact that the partnership between the bank and Qyrus has been successful is evident from the numerous industry awards they have received. Together, they will continue redefining product experience benchmarks, a collaboration set to improve time-to-market features and the banking experience for customers. This is bound to help create the path for a wonderful feature in finance.
Annie Leonard, Executive Director of Greenpeace USA, famously said, “If we throw something away, it has to go somewhere.” Waste management companies work day in and day out to make that happen. Many companies in the USA use advanced software to manage tasks efficiently. These tasks include collections, waste disposal, logistics, inventory, and supply chain distribution. Bugs, when present, can cause this intricate web of systems to completely break down, impacting both customer service and the health and safety of communities.
With over 50,000 employees and 15.2 billion in revenue, the client is a Fortune 500 North American leader in garbage collection, recycling pickup, and waste disposal services with extensive requirements in field service monitoring and organization (collection trucks and drivers), inventory management (garbage & recycling centers), and supply chain distribution and logistics. The company adopted a digital transformation dependent on API technology to make their workflows seamless.
This waste management case study is the success story of how the company leveraged automated API software testing to help with it and the corresponding test automation ROIs.
The problems Behind the success story of every company is a long list of problems the company addressed. This one had three major software testing issues across two of its applications – an e-commerce application and an internal business application.
Outdated manual testing approach Revamping both the e-commerce application and the internal business process application meant that there would be a net increase in the total number of API endpoints. Furthermore, since the company was using manual testing processes, it would have to spend extensive time and testers to validate the applications.
Poor test coverage Given the time it would take to test the application APIs individually, the company was forced to limit test case coverage to reduce the resource spend and deliver critical releases on time.
API defects Lack of automation coupled with poor test coverage meant that bugs and inconsistencies appeared throughout applications across different environments. Rectifying these bugs was not only costly but also reduced the speed to market.
The waste management company needed an intuitive, codeless, and automated solution to simplify testing in the initial stages of the development process and facilitate effective sprint cycles to speed up time to market and lower costs. In the next section of this waste management case study, we explore how the company effectively solved all these problems through a test automation solution.
The solution The company improved its software testing process for its e-commerce application and internal business processes application by implementing Qyrus’ automation testing software, a simple, smart, and scalable testing solution. Through implementing the tool, the company witnessed:
Expanded test coverage With Qyrus, the waste management company developed tests for validating the applications end-to-end quickly across browsers, devices, and OS combinations. Furthermore, they facilitated mobile testing using a large variety of custom devices as well.
The company also monitored the success of these tests through robust visual and data-driven reporting and facilitated an inside-out service journey by enabling natural progression. Through these efforts, the company managed to increase its test coverage by 10x.
Improved test execution time The company successfully built hundreds of API-led workflows focused on dynamic data transfer and validation. It also automated the testing of regression suites through Jenkins, further improving test execution time and repurposing functional tests into performance tests. The effect? Test automation ROIs like 4,500+ tests in less than 5 minutes and a 50% decrease in execution time.
Enhanced release quality The company made it a point to rigorously test their applications before release, effectively testing 80+ applications and validating 400+ endpoints. This is how it was able to deliver applications with zero bugs over a 24-month period with Qyrus.
The Impact This waste management case study is the success story of how the company efficiently automated its testing practices with Qyrus.
The test automation ROIs for the company were as follows:
50% decrease in execution time
10x increase in coverage
Zero API bugs leaked into QA and Production over 24 months
Through Qyrus, the company was able to successfully expand its test coverage and improve its testing processes. This translated to a better user experience for customers and company executives.
Global Banks are at risk of losing one in five (20%) of their customers due to poor customer experience, according to 10X banking. This is particularly relevant for mega retail banks, which can’t afford to jeopardize their customer base due to lackluster web and mobile app interactions.
With over 4,500 employees, our client is a retail bank with over 290 branches and a trusted customer brand that offers full-service banking to more than five million customers. They operate on a modern banking platform and serve customers through digital channels, native IOS and Android applications, and in branches across the UK offering personal and business banking options with 1.33 billion in revenue.
That said, they recognized the pitfalls of manual testing—resulting in strained resources, sluggish release cycles, underwhelming web and mobile app performance, and subsequent revenue loss—one retail bank embarked on a mission to overhaul its full-service banking through digital banking testing infrastructure. Their goal? Delivering unmatched user experiences.
The Problems In the context of this banking case study example, the bank sought to establish a dedicated process aimed at improving customer experience across their web and mobile applications. In the process, they encountered obstacles such as:
Inefficient manual testing The manual testing infrastructure led to redundancies in testing, with testers having to re-write test scripts, choking their bandwidth and as a result bottlenecks in product release cycles. The outcome? Inefficient testing, complex bug detection, and resolution.
Additionally, the manual approach impeded the creation of automated CI/CD pipelines, given its inherent requirement for manual intervention.
Direct loss of customers due to inferior apps With inefficient testing came substandard applications, resulting in a less-than-optimal user experience. This directly impacted the bank’s business with a noticeable loss of customers and revenue.
In an effort to put an end to losing customers to competition, the retail bank turned to Qyrus to completely automate its testing framework. The next section of this banking case study example highlights how Qyrus helped achieve this and the magnitude of the transformation.
The Solutions The bank was able to ramp up speed to market by automation resulting in the expansion of test coverage using Qyrus’ testing solutions. End result was a consistent customer experience across its full-service banking infrastructure. Here’s how:
The transition from full manual to automated testing Qyrus prioritized application testing, starting with automated test scripts for mobile and web. Employing cloud-hosted reports featuring step-by-step screenshots and videos of test executions, a foundation for robust, data-driven reporting was established.
The endeavor resulted in a substantial increase in the scale of testing with the bank achieving more than 2,500 automated test executions each week, coupled with over 10,000 manual and object spy sessions per week.
Comprehensive test coverage With Qyrus’ automation came expanded test coverage – from web and mobile to API and end-to-end customer journey test cases. As a consequence, testers could repurpose test scripts, eliminating the need to create scripts anew for scenarios ranging from logging in to verifying balances and other transactions. For perspective, the bank accomplished a test coverage rating of 90%, wherein 100% of the tests generated were automated and reusable.
Leveraging Qyrus’ diverse device farm, the retail bank scaled testing from 30 to over 50 real devices, spanning Android and Apple tablets and mobiles, as well as various browsers (Chrome, FF, Edge, Safari, etc.) and operating systems (Windows, Linux, Mac).
Improved testing and QA through collaboration Qyrus’ testing solution made test sharing among cross-functional teams a breeze with as many as 50 testers, developers, and analysts building automated tests. Furthermore, test de-bugging was simplified owing to live test executions on the web and mobile.
Qyrus’ solution also enabled direct issue logging in Jira,profiling performance metrics for gauging system responsiveness, and visual regression for preventing unintended visual discrepancies across different software builds. All this meant efficient testing, fewer release delays, and faster bug-free releases.
The Impact The retail bank’s success story in this banking case study example exemplifies how testing automation can translate to an unparalleled customer experience, the outcome translated to:
2,500+ automated executions per week
10,000+ manual and object spy sessions per week
90% test case coverage (Web and Mobile)
Expanded usage from 30 to more than 50 devices
Conclusion The banking sector is recognized for its customer-centric approach, and this principle applies equally to digital banking. Battling with the challenges in manual testing and subpar user experiences, the bank benefited from Qyrus’ scalable and secure testing automation solution. The outcome? A remarkable 90% test coverage streamlined operations across web and mobile platforms, and ultimately elevated customer experience to a class of its own.
Developing new banking products quickly while offering best-in-class service is difficult for most banks in the UK, given there’s no dearth of competitors and there’s a surge in demanding customers. Needless to say, banks have to struggle harder to retain their customer base in the UK.
One Yobota survey revealed that 77% of UK banks and financial services firms surveyed began to engage with new customer segments in recent years. Moreover, 80% of them were regularly seeking feedback from customers, and 71% were already implementing them by introducing new products. While many others are focused on modernizing their digital channels since over 50% of the UK population is expected to use mobile phone banking services by 2026.
Improving the customer experience significantly while introducing new products is even harder for specialist banks with lower budgets, given the higher web application development and testing costs. This case report highlights how Shawbrook Bank, despite lower budgets and higher development and technological costs, managed to improve the customer experience significantly. Let’s get right into it.
The Challenge Every bank’s success story for digitization begins with a challenge, usually web application development. But for this bank, it was testing.
When this UK bank began developing new digital banking features to woo its existing customers, they were faced with the following challenges:
Inefficient testing They had a hard time validating the customer experience across various browsers. Since the bank did not have parallel testing measures in place post-web application development, they were wasting over 2500+ person days and significant funds to simply validate changes and test their product.
High tester turnover The tester turnover was unbearably high, presumably due to redundant and inefficient testing processes, leading to poor maintenance of the testing ecosystem. At one point, the bank was spending over $400,000 solely on testing.
Poor customer experience Given the inefficient testing practices and high tester turnover, validation efforts were delayed. So, the bank had to release the software to production prematurely. This, in turn, meant that customers were at the receiving end of a buggy software experience.
To put an end to all these problems, the bank decided to automate testing and accelerate the software development and feedback times. And for that, they turned to Quinnox for help. In the next section of this case report, we go over how the UK bank solved its software testing problems with Qyrus.
The Solution Quinnox used Qyrus, its proprietary AI-powered automation platform, to implement a shift-left quality strategy, through which the specialist bank was able to release software updates quickly without compromising the stability of business processes. Here’s how.
Automated tests The Qyrus team successfully automated a range of tests for the bank’s software through CI/CD pipeline integrations and Qyrus-native plugins and streamlined crucial processes like savings account activities, customer due diligence, and customer onboarding in its savings applications, lending applications, and customer apps.
Through Qyrus CI/CD automation, tests were executed immediately after the build, increasing testing efficiency. X-ray was used for test management and reporting was simplified as well, given reports got added directly to the pipeline.
Such automation lowered the number of hours spent by the UK bank’s testing team and reduced the number of manual errors dramatically. For perspective, such automation prevented the release of over 2500+ bugs into production.
Repurposed functional tests for performance testing To reduce the testing time, the bank repurposed functional tests for performance testing with Qyrus. They executed tests for over 1000+ users simultaneously, increasing test reusability and improving testing efficiency. Furthermore, thanks to repurposing tests, they were able to ensure greater consistency and run reliable tests, given there was no scope for introducing errors manually.
This meant that their savings applications, lending applications, and customer-facing applications were bug-free.
Scalable infrastructure and coverage With Qyrus’ help, the bank was also able to expand its test coverage across multiple platforms, browsers, devices, and operating systems, quickly. This ensured the bank’s customers had a consistent user experience across platforms, no matter which platform they used, savings applications, lending applications, or others.
The Impact The Shawbrook Bank success story is an excellent example of how companies can follow through with their promise of providing its customers the best experience possible, with better testing.
After all, with Qyrus’ AI-based testing automation platform, the UK bank was able to streamline testing considerably.
For the purpose of this case report, it translated to:
60% reduction in manual efforts with Qyrus automation
85% effort reduction across testing and QA lifecycle
200% ROI within first 12 months
“Quinnox is a key partner of ours, not only supporting our technology delivery teams but also with tools like Qyrus. Test automation is a critical capability for any engineering team, and Qyrus supports the testing of our web, API, and mobile components as a part of our CI/CD processes, and also in ongoing regression testing across our partner ecosystem. The real power of Qyrus is that we have this extremely broad testing capability in one tool, run in the Cloud, and reusable across all our development teams.” – Russ Thornton, CTO, Shawbrook Bank
The client is a London-based, consumer-facing fintech company founded in 2016, providing its customers with free digital-only accounts for personal finances.
The neobank startup set out with the goal to revolutionize social banking and make it as frictionless as possible through innovative products and enjoyable customer experiences. This is a case study of how the said challenger Neobank achieved this feat, more so on a tight budget.
The problem The hero of this case study, the challenger bank, was keen on automating their mobile testing for Payments, Split Transactions, Settlements, User onboarding, and User experience with a small testing budget. That meant the bank was burdened with several testing problems, including:
1. Insufficient QA testers The neobank had 4 manual testers for testing payments, and other software modules, given their low revenue and budget constraints. This meant that their software testing efforts took additional time and effort, wasting both time and resources.
2. Inefficient QA testing In addition to having a small number of testers, the bank was carrying out all its software testing efforts manually. This meant that the testing was prone to errors and cost several more employee hours.
3. High testing costs Perhaps most significantly, the bank wanted to test multiple client-facing applications across web, mobile, and business processes but did not have the budget to cover the costs. Had they not scaled up their testing rapidly, they would have faced multiple dissatisfied customers. This is precisely why they turned to Qyrus for help.
The solution After thoroughly screening multiple automation platforms on various critical parameters, the company selected Qyrus, the intelligent,codeless automation platform for Mobile and API licenses as well as devices on the Device Farm for both Android and iOS.
The challenger bank transformed its software testing for payments and other modules by training its QA testers to operate the Qyrus test automation platform for 3 weeks. With testers proficient with the platform and having a 9.5/10 comfort level using the tool, the bank moved on to improve its testing processes. Here’s how Qyrus helped with it.
1. Expanded test coverage Qyrus expanded the bank’s testing process and started off with 100+ Android and 200+ iOS tests built within the first few months, with an average test build time of 10 minutes. Besides that, Qyrus also scaled mobile and business process testing to increase instances, browsers, and devices, including 5 end-to-end business process tests. This made for a comprehensive testing system. In fact, the Neobank further expanded coverage by creating over 706+ Android tests and 716+ iOS tests with an average execution time of 5 minutes.
2. Fast tracked testing Through its AI-enabled plugin, Qyrus helped the bank startup deploy automated tests right after development, reducing the time spent on manually building and deploying tests. For perspective, the bank observed a 50% decrease in test build time through cloning (8 minutes per script) and close to a 5-minute execution time.
Furthermore, given the reports were uploaded directly onto the pipeline, Qyrus simplified reporting as well. Thanks to this, the challenger bank could validate end-to-end functionality in less than 2 hours post-release, allowing for quicker error resolution.
3. Improved testing practices With Qyrus, the challenger bank enabled data-driven testing through parameterization for the first time. They not only optimized organization and maintenance through Qyrus action types but also helped testers automate tests effortlessly. For instance, the bank’s testers developed 8 automated tests that would get triggered and executed immediately after the build.
The Impact This case study is a timely example of how neobanks can successfully validate the functionality of its software applications with Qyrus, no matter their budget constraints. For the social banking startup, in this case study, the subscription with Qyrus translated to:
~50% decrease in test build time through cloning
1500 total executions
Zero bugs leaked into QA & production
In 10 weeks of subscribing to Qyrus, the Neobank startup, achieved 90% test coverage across mobile, APIs, and end-to-end scenarios using two resources. Besides that, the bank successfully managed to scale its testing efforts across various platforms, despite its budget, and offer its customers a high-quality social banking experience.
Kumar Vivek
Development
Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim.
