Qyrus Named a Leader in The Forrester Wave™: Autonomous Testing Platforms, Q4 2025 – Read More

Book a Demo
Sign In
Book a Demo
Start Free trial

As more enterprises leverage and monetize APIs, it has become increasingly critical to ensure that they are working at maximum efficiency. That is where Automated API testing comes in. Leveraging API testing automation platforms, such as the one Qyrus offers, has multiple benefits.

‍

Improving Testing Outcomes with API Testing Automation 

Discover how three companies generated significant benefits—across multiple dimensions—by capitalizing on Qyrus’ API testing automation platform.

‍

Case 1

The Business and Its Challenge 

Headquartered in the United Kingdom, this bank specializes in savings and lending products. It handles over £6 billion in customer deposits and over £7 billion in loans, resulting in a complex business environment, utilizing a number of internal and customer‐facing platforms, which depend heavily on APIs. 

On average, the bank spent $400,000 a year on testing and over 2,500 manhours—but despite these investments, its testing objectives were not being met. Its challenges included less‐than‐optimal coverage of mobile applications, and tests related to newer browsers, devices, and operating systems. 

This resulted in a poorer customer experience for customers and employees and partners, which, in turn, gave rise to high customer churn. Operationally, too, the cost of its testing practices for APIs had a high price,, especially in terms of test efficiency and poor maintenance.

Success with Qyrus

The Qyrus team automated a range of tests including those related to Savings Account Activities, Customer Due Diligence, and the bank’s Onboarding Process. 

Soon the bank’s testing team began to witness benefits from a more rapid and robust feedback mechanism, underpinned by more visual and data‐driven reporting. The implementation of Qyrus’ solutions also resulted in broader test coverage, and increased efficiency with repeatable, recordable, and scalable tests. 

Within the first 12 months, the bank achieved a 200% return on its investment, prevented over 2,500 bugs from entering production, and lowered testing cost and effort by 75% and 85%, respectively. 

The testing platform also increased automation using CI/CD pipeline integrations, which ensured that tests could be executed immediately after every build, and that reports were generated directly within the CI/CD pipeline.

The Business  

Winning with Qyrus  

‍

Case 2

The Business and Its Challenge

The Fortune 500 company was building two major applications: One that was customer‐facing and would schedule, track, and manage orders; and the other an internal business process solution. 

As development progressed, the number of API endpoints increased. This, combined with the organization’s manual testing processes resulted in a demand for additional testers, and tester time. It also limited test case coverage and introduced bugs and inconsistencies, which proved costly, and increased the company’s time‐to‐market. 

The client needed a solution to manage and automate testing APIs across multiple applications, guarantee application functionality, and ensure reporting was integrated into existing CI pipelines.

Success with Qyrus

The client turned to Qyrus to strategize and implement test automation for APIs. 

The client utilized API‐led workflows focused on dynamic data transfer and validation. A robust set of 4,500‐plus test scenarios were built in under four months, ensuring smooth data exchange across 80 enterprise applications, both on‐premises and in the cloud. The Qyrus platform validated over 400 endpoints— including SOAP and REST APIs—using unique, and randomly generated data to ensure application functionality across multiple scenarios and environments. 

By introducing reporting—including automated testing metrics—into testing environments, Qyrus enabled fast and easy identification of bugs and inconsistencies. This led to the discovery and resolution of defects, which were resulting in delays, causing timeouts and data mismanagement, and access and validation errors. 

As part of a shift‐left strategy, Qyrus supported aggressive release times with seamless integration to existing CI pipelines, enhancing application quality without compromising the stability of existing business processes. 

Qyrus’ AI‐powered test automation platform enhanced software quality across internal and customer‐facing applications, leading to improved field service monitoring, inventory and supply chain management, and logistics. 

With Qyrus, the client achieved a 10X increase in test case coverage, and a 40% increase in test execution speed. Now tests are executed on each build cycle and are triggered through a Jenkins pipeline, thanks to Qyrus’ native integration with Jenkins, and other major CI pipelines. 

The organization continued to expand its API testing capabilities with Qyrus to include another internal trucking application. It is also integrating Qyrus into Mobile and Web application testing stacks, enhancing driver and service center applications that leverage customized tablets. 

The Business

Winning with Qyrus  

‍

Case 3

The Business and Its Challenge  

Over the last few years—accelerated by the pandemic—insurers have digitalized greater swathes of their businesses and continue to push their digital transformation agendas forward. This is especially true for customer-facing applications because consumers increasingly expect to be able to buy or renew insurance online quickly and frictionlessly. This necessitates a seamless onboarding experience, and the digitalization of complex underwriting processes, among other activities.  

With over $13 billion in revenue, this Fortune 500 (North America) leader in insurance, needed to bolster its testing practices so that it could introduce a steady stream of new features and products. But it was challenged by outdated manual testing processes, which delivered a one-two punch: Slow testing resulted in delayed releases, and poor test-building and efficiency also hurt test coverage.  

These challenges could potentially have a cascading effect on a number of business performance indicators including customer experience, policy renewals, and new sales.  

Success with Qyrus  

The insurer partnered with Qyrus to solve its testing challenges. By leveraging the Qyrus platform, it was able to increase testing speed, expand testing coverage, and lower testing costs.  

Within short order, the insurer’s testing group was able to create 50-plus automated test cases, testing 10 business services. More impressively, it could test over 90% of the scenarios. The Qyrus platform also enabled them to generate complex test cases in as little as 30 minutes, as well as shrink execution time by 90%. Today, its average test execution time is four minutes.  

Perhaps most importantly, it facilitated the migration of applications smoothly and improved its Customer-Insurance-Quote journey with rapid and comprehensive functionality and UX testing.  

It achieved these results and spent $40,000 less than it originally estimated. 

‍

The Business  

Winning with Qyrus  

‍

Modernize API Testing with Qyrus 

Qyrus offers codeless API application testing that improves product quality, increases development cycle efficiency, and expands test case coverage. 

With Qyrus API Testing Automation platform, enterprises can leverage service virtualization and execute functional, performance, and process tests as well as monitor the health and status of APIs.  

Qyrus can help your organization build tests more rapidly, while shrinking costs and execution time, and increasing both quality and coverage. Try it now! 

‍

Automated testing is becoming an integral part of the software development lifecycle (SDLC) and is gaining importance as time goes on. In Agile development, releases entail executing a multitude of test cases and other downstream actions. However, for projects that require a rapid release cycle, setting up test environments, allocating resources, creating test scripts, and numerous other factors can cause a time-intensive process all while trying to keep pace with development. To add to the problem, existing tools for test automation typically include a steep learning curve often involving advanced programming proficiency. This, however, defeats the whole purpose of automation – creating less of a burden on testers and shortening the test phase in the SDLC. Fortunately, codeless testing automation can effectively address these shortcomings.

How does codeless testing work?

Codeless automated testing means creating, executing, integrating, and maintaining tests and accompanying processes without writing any code. The basic idea behind it is to abstract these processes through an intuitive UI coupled with a keyword/taxonomy-driven methodology. The codeless model promotes broader participation across tester, developer, and business analyst personas. Among the tactical benefits?

However, the usefulness of the platform erodes if they cannot handle complexity, cross-platform processes, omnichannel requirements, easy CI/CD integrations, or scaling test infrastructure. Using the right product and processes, testers, developers, and business analysts can push efficiencies beyond the traditional scripting and maintenance activities. DevOps is still considered a challenge for many organizations, and more often than not if one follows the inefficiencies in the pipelines they are brought back to test coding, test flakiness, and/or script maintenance challenges. Reducing time spent on repetitive tasks and abstracting coding will yield time that is better spent automating the end-to-end processes.

Benefits of Codeless Testing

The codeless platform extends well beyond simple record and replay and taxonomy-driven automation. Self-healing tests address flakiness and brittleness of scripts and natural language voice and text interfaces have transformed test scripting into a conversation. Using these approaches, the timeline to design, script, execute, and maintain functional test has dramatically reduced. Add to that layers of performance and exploratory testing, and the codeless platform becomes an all-inclusive one. What used to be days and hours are often minute and seconds, and because the testing participants and reusability have increased, code coverage and test coverage also increase.

Qyrus is your safe bet.

Qyrus solves sluggish testing – the Achilles heel of digital transformation – where quality is not in step with organizational agility.

Qyrus is an on-demand SaaS codeless automated testing platform that deploys machine learning algorithms, an intuitive user experience, and a collaborative codeless approach to test automation.

What makes Qyrus different?

To summarize, Qyrus enables development and testing teams to produce higher quality products and an exceptional user experience all done quickly and efficiently. It helps maximize human ingenuity and minimize unnecessary dependencies across the SDLC, limit human error all while stimulating collaborative working models. It increases in scale when the need arises, decreases operating and testing costs, and enables teams to focus on high-value strategic tasks and growth.

If there is a need for a simple, smart, secure, and scalable platform for testing, consider Qyrus.

To learn more, visit us at https://qyrus.com/ | marketing@quinnox.com.

‍

Shoes, clothing, and water bottles, with almost all consumer items, there is often a negotiation between quality and price. As brands flourish on being high quality and best in class, consumers can often find a similar money-conscious option available. But as with most decisions, it is important to be critical and note that if it sounds too good to be true, it just might be. These ideals remain true across a range of industries, including development and quality assurance. This week’s feature Friday discusses manual and automated testing using the Qyrus device farm, and some of the benefits of testing applications using real Android and iOS mobile phones and tablets located across the globe. Let’s dive in as we interviewed Jorell and Raoul to learn more.

 

Tell us more about the testing process using the Qyrus device farm, its use cases, and its impact on testing and QA processes?   

Jorell:
Though very powerful the Qyrus device farm is actually simple. Real physical Android and iOS devices hosted within the cloud to support manual and automated testing of native applications and web applications.  

Raoul:
Exactly, all the functionality, each test has videos, screenshots, performance profiling metrics, and device logs. This means that if a defect is found – the developer has as many pieces of evidence to comprehend, recreate, and remedy the issue – Qyrus has native configurations to Jira which means an issue can be filed directly from the Qyrus test report!

 

Does the same or similar functionality exist without Qyrus, and how do competitors address similar problems?

Raoul:
Prior to device farm capabilities and during the testing process, testers would have to set up and maintain all required devices for testing purposes. Missing out on a range of features and functionalities, the process also requires more resources and time.  

Jorell:
And though other device farms exist, Qyrus differentiates itself through experience and capabilities. Feature-rich including the ability to toggle between network options, extensive reporting with screenshots and videos, all the way through offering beta and previous versions of Android and iOS to test on.

 

What is the overall impact on the testing process when using the Qyrus device farm?

Jorell:
When considering the testing process as a whole, the device farms impacts are clear. From the beginning, device setup, and maintenance are mitigated. That’s a heap of time and resources, and as requirements grow, the cloud-native solution grows as well, offering custom and private device options.  

Raoul:
Furthermore, during the testing process devices are easily accessible at all points in time. Teams can share devices, monitor runs across devices, and test in parallel, with extensive reports and collaborative features making the entire testing and QA process more efficient.

 

How might the Qyrus device farm testing options help testers, developers, and business technologists? What value can this feature bring?

Raoul:
Very broadly – you can test any scenarios, applications, and devices through this feature. Anything that the tester needs from manual testing to automated testing is both available, maintained, and ready to scale up or down as requirements change with no testing or developmental resources or overhead from the client side. This increases test case coverage allowing testers a range of devices on hand to test application functionality.  

Jorell:
We often find developers uploading beta versions of Android and iOS applications to test within the developmental process. Furthermore, with out-of-the-box Jira integrations, communication between developers and testers becomes seamless. Promoting high-quality applications.

Raoul:
Furthermore, coupled with an amazing UI and rich visual reporting, Qyrus creates ease of access for business analysts to access application information. Providing screenshots, videos, and the ability to email and download reports, Qyrus provides the necessary application information and collaborative capabilities for business analysts to integrate themselves into the testing process and QA lifestyle.

 

How do you see the Qyrus device farm impacting day-to-day operations across the organization?

Jorell:
The day-to-day impacts of this feature cannot be downplayed. Teams have access to any and all required devices for testing purposes. By enabling efficient testing utilizing parallel testing features, testers, and developers can establish functionality across a multitude of both Android and iOS devices simultaneously.

Raoul:
Furthermore, by offering both private devices and a public device pool, clients can utilize devices across the globe. Test case coverage is no longer a concern. Furthermore, with options to scale up and down as seen fit, clients can ramp up devices during release and scale down accordingly to prevent excess spending as requirements change.

As applications dominate the forefront of business, there are certain things that are worth going the extra mile for. High-quality mobile applications are functional regardless of the device. But building out device-agnostic applications and testing them across a range of Android and iOS devices to promote the best user experience can be a daunting task. With a multitude of devices and mobile operating systems at your fingertips, and the ability to upload and test beta versions of applications, within a powerful solution providing robust visual and data-driven reporting, Qyrus’ Device farm aims to mitigate all aforementioned headaches by increasing testing efficiency, coverage, and enabling high-quality application production.

We’re excited to share that Quinnox has been recognized as a “strong performer” in the Forrester Wave™ for Modern Application Development, Q3 2022.

According to the report, Quinnox shines across most of the current offering criteria. These include modern development and delivery services with agile, product development, DevOps, SRE and automation, architecture API and microservices, continuous testing, cloud-native, AI and data, case history examples, metrics, methodologies, frameworks and software, and vertical industry knowledge” (All good things in one place)

Autumn is just around the corner, and with that the warm winds leave the Northern Hemisphere. Hopefully, if you’re from the Southern Hemisphere, you’re gearing up for the summer heat already! And hopefully, if you’re a tester, you’re thinking about gearing up for better and more efficient testing strategies! Today on this Feature Friday, we are joined by Daniel and Amy who are going to talk to us more about our exciting new feature, Test Data Management!

Tell us more about Test Data Management offered by Qyrus and its use cases.

Dan:
Well, Test Data Management is a feature that manages your test data in one place. It helps eliminate the tedious steps of importing data from external sources like an Excel file. It also has an added benefit of allowing users to synthetically generate data within the Test Data Management system itself for usage during runtime.

Amy:
So, when you import data from a database, you need only to establish a connection one time. Afterwards, Test Data Management will retrieve the data anytime a test is run that uses that data. Furthermore, you can even grab data from an API call for usage during runtime, as well.

Interesting… it seems like a tool that can cover a lot of use cases under one umbrella. This must mean it has a significant impact on testing processes throughout.

What is this feature’s overall impact on the testing process?

Dan:
The areas in which we see the largest impact would probably by test building and execution. It has a direct benefit in terms of test coverage, allowing users to quickly build tests that use data dynamically and during runtime.

Amy:
Instead of having to incorporate some 3rd party tools to generate data, it’s all done from our Test Data Management system. Connections to databases and pulling data from API calls during runtime helps enhance the testing capabilities that Qyrus already provides to its users.

Fragmented environments are the worst. They cause problems and if no problems exist today they will months or a couple years down the line. The ability for users to have a centralized system to handle all of their needs is important. How might other users, besides testers, utilize this feature?

How might Test Data Management help testers, developers, and business technologists? What value can this feature bring? 

Amy:
Well, a tester would utilize this feature heavily. Again, it has a large impact on the test building and execution process, and would empower testers to manage their test data more efficiently. This also helps in terms of maintenance from their perspective.

Dan:
And developers can use the Test Data Management system to generate synthetic data to quickly build sample tests or unit tests even. This could help them then catch bugs before it even makes it to the testing phase.

Amy:
And the cool thing about the system is that it can generate data similar to what you provide to it. So, in that aspect, a business technologist could provide a sample set of data and use the tool to expand that sample set.

So, with it being useful for a wide arrange of users and personas, we might see this making waves in the ebb and flow of normal business operations. Let’s learn more!

How do you see using Test Data Management impacting day-to-day operations across organizations?

Amy:
Well, it helps increase test coverage at a faster pace, so day-to-day operations might be sped up to a degree. Data imports are faster, and AI is utilized to help generate sample data. All of these help speed up the testing process.

Dan:
On top of that, from a maintenance standpoint, it all becomes manageable from one spot. It’s super easy to take care of. Connect to your database once, and never worry about it again.

Does the same or similar functionality exist without Qyrus, and how do competitors address similar problems?

Dan:
From a competitor standpoint, not everything is integrated and combined in one place like we have in our Test Data Management system. From being able to connect and import data from databases, API calls, and synthetically generate similar data, we have it all in one spot.

Amy:
And before Qyrus, testers would have to have studied and built up a large amount of coding knowledge. That’s if they want to replicate it from the ground up. However, they could use a fragmented suite of tools to help achieve the same functionality.

Enjoy the last remnants of summer while you can! Before it gets chilly up here for us northern folk, we’re out to enjoy one last hurrah before the bell tolls, signifying the end of our summer. We hope you learned a lot about our Test Data Management system and how it can bring value to a tester’s day-to-day life. Hopefully we’ve convinced you to give Qyrus out a try. Until next time, thanks for joining us for this Feature Friday!‍

Quinnox, a global provider of full-spectrum IT and digital solutions, is a material outsourcer to Shawbrook, and a key enabler of their technology transformation journey.  For Shawbrook, this recent tie-up with Qyrus is another step towards testing at the speed of digital, re-invigorating their relationship with customers, improving customer satisfaction and loyalty and accelerating business outcomes.

Quinnox uses Qyrus, its proprietary AI-powered test automation platform, to develop new systems and change existing processes and systems for Shawbrook, a new type of financial services company in the UK that combines the relentless focus on customer service and innovation  that would be expected from a fintech with the expertise and certainty of a bank.

As part of their ‘shift-left’ quality strategy, Qyrus supports Shawbrook’s ambitious delivery schedule of new product features to customers as well as puts its focus on automation, through automated regression testing across the breadth of the Shawbrook Platform.

Qyrus helped Shawbrook transform its Platform to seamlessly automate savings account opening & servicing, real estate & SME lending origination as well as customer due diligence onboarding processes.

Shawbrook was able to deliver releases in a much shorter time using the Web, API, and Business Process testing capabilities of Qyrus. The outcomes this drove were faster time to feedback, increased velocity of deployments, better quality of releases with significantly less effort.

Full details on Shawbrook’s strategy to develop its next generation banking platform and 2021 commercial performance can be found in their 2021 Annual Report.

Russ Thornton, CTO of Shawbrook

“Quinnox is a key partner of ours, not only supporting our technology delivery teams, but with tools like Qyrus. Test automation is a critical capability for any engineering team, and Qyrus supports the testing of our web, API and mobile components as a part of our CI/CD processes, but also in on-going regression testing across our partner ecosystem. The real power of Qyrus is that we have this extremely broad testing capability in one tool, run in the Cloud, and reusable across all our development teams”

Somnath Chakraborty, SVP, Head of Quinnox Europe

“We are delighted to have Shawbrook as one of our strategic clients here in the UK and this partnership has grown strength to strength, as together we intend to drive positive change in the Banking industry and are proud that Qyrus is able to help accelerate it.”

About Shawbrook Bank:

Banking for the real world: Over the last decade we have built a new type of bank that is designed to accommodate individuality, diversity and the dynamics of the modern world.  Our ability to provide highly personalized finance solutions to help our customers achieve their immediate ambitions is enabled by our unique platform, which combines and integrates modular technology with human expertise, judgment and ingenuity. We power up ingenuity to create opportunity, every single day.

About Quinnox:

Quinnox is your agile, business-results-driven digital technology partner. With the power of human and applied intelligence, we simplify business processes, improve customer experiences and create exceptional business value for forward-thinking enterprises. Our data-driven digital solutions unlock the hidden potential of your business across your digital value chain, helping to accelerate success, today and tomorrow. We are headquartered in Chicago, Illinois, with a presence across geographies including UK, Germany, UAE and India.

About Qyrus:

Qyrus is a codeless, intelligent, and collaborative test automation platform for all things digital. It efficiently tests Web, Mobile, and APIs along with end-to-end business processes to drive quality, speed and frequency to market. As an on-demand AI-powered SaaS testing platform, it can scale with your needs enabling teams to deliver higher quality software faster and more efficiently thereby ensuring a seamless customer experience across all stages of the digital journey.

‍

Over the years, we have seen many examples of large ecosystems and species harmed by changes in predator and prey relationships caused by natural selection.

Is it so naive to say today’s technological era is accustomed to rules that may not be so different? The consumer chose movement and reach, and thus laptops rendered towers and workstations near obsolete.

The Kindle Fire as a primary reading utensil was dominated by Surface Pros, iPads, and other powerful hybrid devices which have since transitioned into handheld mobile devices with hundreds of gigabytes of memory and multi-core processors capable of running laptop-equivalent functionality.

This continual necessity for simpler and more practical technology has led to an influx of devices running on an array of differing operating systems connecting to multiple browser versions amongst more.

The options seem limitless and, often, overwhelming for those who aim to create or maintain applications of any caliber.

When looking at these challenges in business processes, it’s important to ensure that the software works well on different browsers, devices, and operating systems. This might mean using a device farm.

What is a Device Farm?

Device farms are dedicated hosting locations for a multitude of devices, used for testing application functionality in real-time.

But, with industry leading names like Facebook, Instagram, Tinder, and Uber topping the list of companies with the highest percentages of personal data collected, it can be daunting choosing a platform or provider for your entire business’ devices and testing with the confidence that your data is secure. 

Qyrus’ dedicated, cloud-based mobile device farm utilizes three prominent business fundamentals: power, performance, and privacy to maximize all facets of testing.

What Makes Qyrus’ Device Farm Different?

The Qyrus device farm is one of the prominent AWS device farm alternatives. It contains a multitude of different tablets and mobile devices to use for device farm testing including but not limited to iPads, various Android tablets, iPhones, and a multitude of Android-based mobile devices with device customizations available to fit specific client needs.

Alongside this, with the addition of parallel testing features, Qyrus allows multiple tests to be executed on multiple devices and operating systems simultaneously. The versatility provided by an option pool covering all facets of devices and testing synergized by the efficiency of parallel test execution comprise the power of the Qyrus platform, a platform that welcomes unique testing challenges.

The Qyrus device farm goes even a step further. With performance profiling metrics, further understand the application being tested. Learn how the app affects battery consumption, CPU and memory usage, and other device vitals. Screenshots can be captured during a session for reporting and device logs are also available for further debugging.

Qyrus’ device farm was built on a business centric interface that allows a user the freedom to connect and test from any device, in any location, and at any given time, within clicks. Emergencies are eminent and business never sleeps, therefore, your testing solution shouldn’t either.

Run suites of tests through the night using Qyrus’ scheduled run feature and wake up to a confirmation that your applications are properly functional. Furthermore, with a detailed and easily sharable analytics and reports tab, being in touch with your past and present business processes has never been easier.

Twenty-four seven coverage and testing capabilities with full scale reports and analytics, accessible from any location, built for a collaborative mentality, Qyrus specializes in increasing efficiency throughout all testing-based business, and development processes.

Qyrus’ device farm goes a step further in capturing useful data and metrics. With performance profiling metrics and

Qyrus Takes Your Data Security Very Seriously

Data security is becoming necessary at an alarming rate. As technology becomes more prevalent, the same can be said about cyber-attacks and data theft. Qyrus leverages private instances with each of their clients to allow maximum security through testing processes.

That’s to say, regardless of whether you are sharing, or have private devices, your instance of Qyrus and all company-related data hosted, by, or on Qyrus is private to you, and inaccessible by any other Qyrus users.

Qyrus also allows clients the option to essentially build their own device farm, providing private devices, instances, and specifications as required for optimal security.

Furthermore, Qyrus is empowered with many security features including single-sign-on options, and security testing features such as web token implementation and testing, alongside certificate-based authentication within API testing to ensure your web apps, mobile apps, and API’s are stable and secure.

As a recently certified SOC level II compliant company, Qyrus excels at all five pillars of the certification standard including security, availability, processing integrity, confidentiality, and privacy, standardizing a level of credibility and professionalism throughout client and business interactions.

Maximum coverage is the obvious combatant to ‘technological selection’. Keeping it simple, the more people you can reach the higher your chances of survival, and in this age of endless options, device farms are essential to ensure functionality and user retention effectively on arrays of different browsers, devices, and operating systems.

Technology can connect any corner of the world, and instead of fire or the wheel, it is device farms, and the simple, smart, and scalable solution in Qyrus, that stands as evolutionary assistance through unpredictable times.

Elevate Your Testing with Qyrus Device Farm

In today’s fast-paced digital landscape, ensuring that your applications perform seamlessly across various devices and platforms is crucial. The Qyrus Device Farm stands out as a premier solution that offers unparalleled testing capabilities, enhanced data security, and the flexibility to adapt to unique testing requirements.

With features like parallel testing and performance profiling, Qyrus not only simplifies the testing process but also empowers businesses to optimize their applications effectively.

Are you ready to elevate your testing strategy and experience the difference for yourself? Sign up today for a free trial of Qyrus Device Farm and transform the way you conduct application testing!

‍

In this discussion’s simplest form, we pose a question, “If there was a tsunami en route, are you positioned on the shore?” The days of garage launches and the necessity of brick and mortar are becoming lost to the sands of time. The Technological Era through its comfort and practicality has become a silent killer. With Victoria’s Secret, Bed Bath & Beyond, Gamestop, and Men’s Warehouse topping the list of chains permanently closing the most stores in 2020, the movement inland must come with swift technological adaptations.

But this concept in and of itself is both foreign and daunting to most. Meanwhile, noticeably forcing the hand of industry leaders, it becomes clear that the transformation is imminent but how to go about it is less transparent. With differing standards including new and pre-existing business management, transitioning to online or handheld effectively and efficiently often requires a large checkbook and developmental nightmares on the road to functionality. But as the times indicate the only alternative seems to be going under.

Importance of Automation is Software Testing

As in every proverbial dark tunnel, the developments in testing, specifically, automated testing technology, stands to be the light. Providing the ability to build test cases around individual elements, pages, and sites while storing and running them at your convenience, testing technology has taken the initiative in paving a simpler path through the development and deployment cycle. Build, test, save, repeat: a simple process that allows you to manage functionality with constant feedback, regulation, and iteration.

Qyrus – Benefits of an All-in-One Testing Platform

In this realm of testing technology lies the answer to the question, “Why Qyrus?” Qyrus is an inclusive platform providing Web, Mobile, API, Artificial Intelligence, and Infrastructure testing. With a business-first structure, Qyrus allows you to begin with web testing and transition through the platform with your business progression. Meaning as your business goes handheld so does Qyrus, providing you with comfort through the troubling tides of various development and deployment cycles. Hosting international device farms based with the addition of client-specific private instances, the platform provides power and security, but even more so a solid foundation that as your business scales your testing solution remains reliable and comprehensive.

Beyond comfort and business progression there stands an area that Qyrus has re-defined; the testing market. Qyrus’ “low-code, no-code” platform fused with a user-first interface allows full-scale tests to be built in minutes using a series of clicks. Reports immediately provide screenshots and video feedback of the live test upon run completion. But we didn’t stop there, with platform-specific features and AI such as Hawkeye, Healer, QyrusBot, Sage, and Rover behind the simplicity, Qyrus has the power to meet a wide range of testing requirements and needs increasing in levels of complexity. All placed behind a series of clicks, Qyrus has the power to map and record every click of your mobile application, the intuition to fix brittle and defective test scripts through AI implementation, and future additions including the use of Native language commands to build and test. As a field dominated by software developers and engineers, testing has now been simplified to the likes of high school students thanks to Qyrus.

The Future of Software Testing

The motion towards web and mobile applications was once inevitable but is now upon us. With the global number of mobile contactless payment users growing to 760 million, including roughly 30% of the North American population shifting to mobile wallets, it becomes as simple as following the money. But with the synergy of power and simplicity, and the streamlining of extensive test coverage attainable by professionals of any background, Qyrus simplifies the web and mobile transition, paving the path inward for businesses in wake of the eminent technological tidal wave.

‍

Ahhh, Spring. What a wonderful time to open our windows and doors and let the fresh air into our homes. As the cover of winter slowly lifts, trees begin to bud, birds begin to sing, and the scent of fresh flowers tickle the nose.

But let’s not forget the dangers that may lie just around the corner. Securing our applications, much like our houses, is of critical importance. Sensitive data and information can be accessed and stolen, similar to a burglar breaking into your house. Many applications require tokens or keys to access data. But, how exactly can those processes be tested?

This week’s Feature Friday covers just that. A prerequisite API test enables the testing of data-dependent APIs and API processes. To learn more, we interviewed Daniel and Steve on the Qyrus team:

Tell us more about the prerequisite API testing feature offered by Qyrus, its use cases, and its impact on testing and QA processes. 

Steve:
Just like the locks on a door, applications often work the same way, especially when they contain sensitive user and organization data. To test or even enter these applications, users require a unique key or token. This is a prominent use case for prerequisite API tests. Simply set up the prerequisite API, and it will execute prior to the API test, providing the required data to continue the testing process.

Daniel:
More than just security tokens, prerequisite APIs come in handy anytime initialization is required. Often, a particular API is dependent upon input data from a different source, most likely another API.  In order to truly test the behavior of the data-dependent API, it is nice to completely mimic this behavior.  A simple prerequisite API can be used instead of building an API process of two items, saving time but accomplishing the required behavior and data transfer process.

Data-dependent APIs are fundamental in application creation. Therefore, it’s important to gain some knowledge and background on how these use cases are tested today. So, we asked:  

Does the same or similar functionality exist without Qyrus, and how do competitors address similar problems?

Steve:
Other competitors often require extensive setup, usually utilizing code, to run the prerequisite API, enable data transfer, and trigger the accompanying API test.  Though, at the heart of it, the process boils down to calling concurrent API tests, Qyrus simplifies script building and management, mitigates coding requirements, and enables easy environment setup and dynamic data transfer all in one intuitive feature.

Daniel:
Prerequisite APIs on Qyrus differ from our competitors because we provide rich reporting, including pass and fail indicators, execution time, and response headers and body. Furthermore, we offer the ability to dynamically use the responses, building them into dynamic variables that can be implemented as desired across test scripts in the project.

What is the overall impact on the testing process when using prerequisite API testing with Qyrus?

Daniel:
We no longer need to manually enter in pre-request credentials for every pre-request API call. With the prerequisite API feature, the tester enters the pre-request credentials once, and the prerequisite API will always run before a request execution takes place.

Steve:
Additionally, building these interdependent tests could be a little complicated, as the asynchronous behavior of APIs can be conceptually difficult for newer developers to understand. This feature makes testing easier by eliminating the coding barrier required to build these scripts.

Daniel:
Furthermore, a user is able to dynamically use the response body from the prerequisite API into the request execution. In terms of data-dependent API testing, this is a simple, smart, and comprehensive option.

The prerequisite API test provides a lot of utility to users, that’s for sure. Newer developers have less of a hurdle to jump over, as just mentioned. The leap forward cannot be understated. However, does this mean that API testing will no longer be dependent on those with a high level of programming knowledge?

How might prerequisite API testing help testers, developers, and business technologists? What value can this feature bring?

Steve:
A great question. We find a handful of personas are able to use this feature to test required data-dependent APIs. Testers use this feature to enable an efficient and high-level data transfer process. To be specific, testers build out comprehensive prerequisite API tests across complex use cases through simple form-filling functionality. They can quickly set required data as dynamic variables and implement them directly into the API test for steadfast test building, execution, and data transfer processes. Testers often use this feature to speed up the testing process of production-level APIs.

Daniel:
That’s correct. Furthermore, we find developers using this feature to establish API functionality during development. Creating prerequisite APIs allows for the mapping and pre-development of API processes. Given the capability through prerequisite API testing, developers can pre-determine requirements and design their APIs to target specific use cases. Upon creation, a prerequisite API test can be created and conducted to validate the functionality of both the APIs and the data transfer process. 

Steve:
Also, the codeless nature of prerequisite API testing makes it easy for business analysts to build and execute prerequisite API tests. In fact, this feature could help provide more domain knowledge for business technologists. With form-filling functionality, business analysts could further understand the interdependent relationships between the APIs created by the business.

A unique feature for all personas within testing teams, we felt the need to further implore its usefulness to testing as a whole:

How do you see CI pipelines impacting day-to-day operations across organizations?

Daniel:
The main objective of prerequisite API testing is to provide a functional solution to data-driven API tests where requirements are contained in a logical fashion without removing the modularity of the API tests.

Steve:
Exactly, you can run functional tests on the prerequisite API.  Once you have confirmed that it is working, you can then apply the same process across all APIs. Also, after a Prerequisite API is created and executed, a user is able to dynamically reuse the response body across the entire API project.

Daniel:
And with a bright future roadmap, the prerequisite API feature aims to allow users to add logic or code snippets to the prerequisite request. This can be useful if a request header expects a unique and specific input.

That concludes this week’s Feature Friday! As the week concludes, let’s all take a breath of fresh air in anticipation of even better days ahead. Take a walk, stop and smell the flowers, and maybe have a picnic. Whatever it might be, we hope everyone enjoys the coming Spring!

They’re everywhere! Now that summer is rolling through the Northern Hemisphere, BUGS are everywhere! Yes, I’m talking about real, physical bugs, not the ones we find in our software. But, I suppose those are abundant everywhere, as well. Similarly, BOTS are now everywhere. From crawlers to transactional and informational bots and even Conversational Agents – or chatbots – the focus of this Feature Friday! Today, we’re here to introduce you to our new tool to test and train AI chatbots – BotMetrics.

What do you mean?

Chatbots are becoming more and more intelligent with advances in the area of Natural Language Processing and Understanding. But still, organizations and industries are hesitant and skeptical to adopt these agents. Not because chatbots are hard to build or develop — one can develop a chatbot easily using Google DialogFlow or Amazon Lex — but they are hard to test and maintain. Now, what if we say: we have a bot (we have named it – BotMetrics) that can connect to your Conversational Agent/chatbot, have a conversation with regard to your bot’s functionality, and rate your bot’s performance. Just like AI Gilfoyle having a chat with AI Dinesh, but without breaking anything. You might be thinking, “That’s sick!” It doesn’t end here; BotMetrics also helps your bot’s performance by providing it with data related to all the failed conversations they had.

What is BotMetrics?

BotMetrics is our Conversational Testing Platform, the very first of its kind. Currently, it’s under wraps, so we will try to divulge as much as we can to get you excited. We wanted to develop a Super Agent Generalized on Everything, and we started developing BotMetrics. Currently, it works with the English language, and there is a long way for it to go. BotMetrics can be seamlessly integrated with chatbot development platforms like Google’s DialogFlow, Meta’s Wit, and Amazon’s Lex to test and maintain your Conversational Agent/chatbot. SAGE tests the agent’s capability to understand and reply to phrases/utterances by creating hundreds to thousands of virtual conversations with your agent in just minutes.

Why BotMetrics?

We already have a lot of test automation functionalities to test APIs, Web Applications, and Native Mobile Applications in Qyrus, but we wanted to develop (still developing) something new which has never been developed or even imagined.

With the advancements in Natural Language Processing and Understanding, we thought, why don’t we create a Super Agent, which can ”test” other Conversational Agents/chatbots? Moreover, it’s hard to benchmark/test AI, and there is a scarcity of good platforms that can aide in testing and maintaining chatbots.

What does BotMetrics solve?

In the field of Machine Learning and AI, the models are benchmarked using a set of test data, and the model’s effectiveness is decided based on its performance on that data. Let’s take an example of a conversational agent who is trained to provide customers with details regarding their credit and debit accounts, transactions, purchases, and deposits. In this case, the agent needs to learn what type of sentences or phrases point to which of the above-mentioned category.

Moreover, the same thing can be asked in many different ways. Let’s say I want to know the outstanding amount on my credit card; I can ask the agent in a multitude of ways — what’s my credit card bill for this month, or what’s the due credit, or what’s the minimum due for this month? Still, these are normal phrases but if a non-native speaker is trying to interact with this bot and if they make a spelling mistake or the structure of the sentence is not correct — for example, “What account balance?” instead of, ”What’s my account balance?” — the bot won’t understand it and fail to provide an appropriate response. Not all of these test cases will be covered in the training, validation, or test datasets. Only when the user gets their hands on the agent, the developer comes to know about all of the shortcomings.

We have added different testing modes in BotMetrics which helps the developer check how their agent is performing with badly written phrases, phrases with spelling and grammatical errors, phrases with different slang, phrases with non-trivial words, etc. Plus, we provide a data upload integration, so the developer just needs to upload those phrases which their agent couldn’t understand and re-train in just a few clicks. These testing features and modes help the developers to add more and more features to their agents while continuously testing, benchmarking, and validating the new version using BotMetrics.

How does BotMetrics work?

Chatbots are everywhere, like it or not. We use chatbot assistants like Siri, Alexa, or Google to get different things done from time to time. Some might use it to set reminders or timers, some might use it to check facts, some might have a casual chat with it, or some might even use it to control household appliances and electronics.

Similar to these assistants, there can be a conversational agent for banks wherein a user can check their balance or transfer money by just chatting with the agent, or a conversational agent to track all your appointments and meetings, or a conversational agent for a restaurant support system. Applications like these, which are customer-facing for support and engagement are moving towards such conversational agents to minimize cost and human intervention for servicing and support. Platforms like DialogFlow and Wit are also boosting the adoption of Conversational Agents, but testing and maintenance remain an unsolved problem.

This is where BotMetrics comes in and alleviates the chatbot testing and maintenance issues.

With BotMetrics we offer two types of testing abilities for chatbots:

  1. Intent Testing
  2. Entity Testing

What is Intent Testing?

Every chatbot must understand what the user is saying and act accordingly. The process of understanding what the user is trying to say by typing or speaking is called Intent detection. The output of the actions (i.e. typing or speaking) is called utterance. The chatbot first needs to understand this utterance and map it to a pre-defined intent.

 If the bot fails at this very first step (and trust us, many bots fail) the conversation won’t move forward, and the user might close the application and move on.

Using BotMetrics, you can avoid this. You need only to add your pre-defined intents into BotMetrics and provide your bot credentials, and it connects to your bot and generates hundreds to thousands of conversations with the bot based on the pre-defined intents. Thus, testing your bot’s ability to understand intent.

What is Entity Testing?

When you ask Siri to “Set a timer for ten minutes…” it understands the intent first and then extracts the words “ten” and “minutes” from the utterance and sets the timer based on that. These words are what we call entities. One could use different units of time to set an alarm or timer. Similarly, there can be other cases where different words can be used for the same entity type.Based on the intent and the type of entity the chatbot supports, BotMetrics creates a lot of words of the same type and converses with the chatbot by putting these generated words into the conversation to check the bot’s capability of detecting entities.

We hope you like what you learned about BotMetrics. Again, this is just a sneak peek at what’s to come! BotMetrics is still being developed, but we have seen amazing leaps and bounds thus far with what we already have. Join us again in future postings to learn more about the amazing abilities of BotMetrics. For now, close your laptops or shut down your machines and enjoy the weekend! Stay cool, stay safe!