As the marketplace gets more competitive, the pressure to excel in providing innovative digital experiences in every software and product release is paramount. To remain competitive, organizations are doing whatever they can to push their products to the market before their competition does. Because of this urgency, development and testing cycles are being squeezed. Delivery teams need to innovate and augment their development and testing methods with the adoption of next-generation test automation and continuous testing processes to accelerate releases while still keeping an eye on quality.

Continuous Testing Explained

Continuous Testing (CT), is the repetitive process of testing in a continuous manner to deliver superior quality applications and software. This process is part of the SDLC (Software Development Lifecycle) and begins during the creation of the initial level of development plan through the release of the final product. Continuous testing facilitates the rapid feedback of any bugs or risks associated with the release/module/product that is being developed or upgraded. This process involves the execution of automated tests as a part of the software delivery pipeline which runs separately from unit tests.

This image* shows a continuous delivery cycle, involving development, source commit, deployment and monitoring. This process, as depicted in the image, allows the commitment of new code modules as they are ready, enabling the development team to quickly push updated code for testing. The code then triggers a Continuous Integration (CI) process which deploys the code into the target environment, initiating a set of automated tests to analyze if the newly committed code is impacting any existing functionality. The results of these tests are then passed on to the development team to decide next steps. Continuous testing can create immense value for businesses through a high level of integration and automation of testing and deployment.

5 Benefits of Continuous Testing (CT)

1. Accelerates Release Cycles: The automated testing & deployment helps organizations to better respond to market changes. With CT and test automation, updates are developed quicker and released more frequently. With the help of automated testing, developed code (existing & new) is tested rigorously and constantly. It stresses rapid error resolution to ensure clean code delivery which allows for integrations and accelerated launch of the application on a consistent basis with speed.

2. Early feedback to stakeholders: Deployment of CT in the SDLC ensures early feedback to the development team about any issues the code may cause to existing features. CT provides frequent actionable feedback at each development stage which helps to expedite the release of software applications into production with a lesser number of defects. This early feedback also helps to analyze business risk coverage and helps in achieving faster time to market.

3. Improves Test Coverage: Since CT begins at the initial development stage of implementing newly written code, it assesses all features and potential errors. With proper implementation of CT, testers and developers are assured that all critical errors in the application are identified very early - before integration.

4. Reduces Cost of Defects: CT plays a crucial part in ensuring error-free outputs. When code undergoes change, errors are expected to occur. CT ensures a faster turnaround time when dealing with identifying and eliminating these errors early in the development lifecycle. This reduces the overall cost of resolving defects. The cost of change is typically less when the change is identified at an early stage of the development cycle. This is where CT helps immensely.

5. Easy to implement: This is one area which almost all organizations considering CT have concerns. Rest assured that CT processes can be easily integrated with existing CI/CD pipeline without requiring any new tools. In cases where an organization doesn’t have an existing CI/CD implementation, they should consider starting it immediately to reap quantifiable benefits. Start by mapping out a DevOps strategy with continuous testing in mind.

Out of the 5 aspects mentioned above, the cost of poor-quality hurts organizations the most, as can be seen in a recent report* from CISQ (Consortium for IT Software Quality). The cost of poor-quality software in the US in 2018 was approximately $2.84 trillion, and close to 17% of that cost was the time and effort required to find and fix defects.

The report also cites some of the recent examples of how software failures have impacted brands:

• 400 mortgage loan customers of Wells Fargo Bank were accidentally foreclosed due to a calculation error in their accounting software, and the bank had to set aside $8m to address the matter.
• The Airline industry seems to be one of the worst hit from software failures. In June 2018, around 3000 flights of PSA Airlines were canceled due to issues with crew scheduling software.
• In March 2018, a fully autonomous Uber SUV test truck struck and killed a passenger due to its software not responding as expected when the pedestrian was detected.
• Another crash of a Tesla Model X in March 2018, was caused again by issues with its management software. This caused Tesla's stock price to drop by 3.3% the next day.

There are numerous instances across industries including states, banks, and health services etc. suffering losses due to software failures, most of which could have been detected and addressed or avoided through better software quality assurance processes.

Conclusion:

Continuous testing (CT) enables cost-effective and rapid releases by detecting issues early in the development cycle to optimize development and reduce overall project costs. With CT, software delivery becomes more agile with improved quality and faster product releases. Contact us if you would like to explore how CT can help you expedite your application release cycle.

References

*Image reference: http://www.professionalqa.com/continuous-testing
https://www.it-cisq.org/the-cost-of-poor-quality-software-in-the-us-a-2018-report/The-Cost-of-Poor-Quality-Software-in-the-US-2018-Report.pdf
https://www.atlassian.com/continuous-delivery/business-value
https://www.cigniti.com/blog/why-businesses-must-not-miss-out-continuous-testing/
https://blog.csscorp.com/why-do-you-need-to-implement-continuous-testing
https://dzone.com/articles/faster-to-market-attainable-through-continuous-tes
https://saucelabs.com/blog/shift-left-continuous-testing-eliminating-risk-through-agile-testing-practices