Behind every working app lies invisible innovation

SCIENCE NEWS FROM KRISTIANIA: Software testing

One fine morning, you wait in line for your morning coffee at a local café. It is your turn to pay with your credit card. However, after several attempts, nothing seems to work. Two pairs of puzzled eyes stare at the card machine, with no clue why it is not working. 

This situation can happen to anyone – some of us may have already experienced something similar. In this little story, billing software must work smoothly with financial service providers' systems and others to process payments. 

In July 2024, a faulty software update from the cybersecurity firm CrowdStrike caused a widespread service interruption, disrupting critical services worldwide, including air travel and financial systems. 

We need to test the systems that work together

Software is everywhere, from coffee machines to satellites, and our reliance on it is growing rapidly.

Over time, software has grown from simple programs into intricate systems. Consequently, the process of testing software has become increasingly difficult, especially with modern software architectures. Various systems from different vendors must work together seamlessly so that we can get our coffee on time, catch a flight on time, and even get the correct time. 

Testing these systems without a full understanding of their internal logics and external communications is challenging and tedious. The risk of failing to test unforeseen scenarios is extremely high. 

Recognising this challenge, we developed novel algorithms that enable us to automatically test such scenarios with zero modification to the software, without requiring knowledge of such external communications and without requiring manual human intervention. 

Mocking is cheap

In our study, as part of an ongoing project, we focused on developing search-based algorithms to automatically generate dynamic mock instances of external connections without requiring prior knowledge of the dependencies of these connections. 

Our research results demonstrate that these techniques can be used to automate the process and effectively detect potential failures in real-world systems. That way we enhance software reliability and minimize costly service interruptions. 

Furthermore, by building on automated mocking infrastructure, we are developing algorithms to automatically detect software vulnerabilities that involve external communications.

With our algorithms we generate tests that evaluate various software components. Such tests can be reused multiple times to ensure the software system remains intact while continuously adapting to changes in the code. 

This enables faster regression testing and reduces the need for manual human intervention. By minimizing human error and automating repetitive tasks, algorithms improve testing accuracy, efficiency, and scalability.

A small step towards full automation

Although these advancements make the process more efficient, they do not eliminate the fundamental need for testing. 

Even with the contributions from our research, our techniques are just a part of the beginning of a broader puzzle. The software engineering landscape is evolving rapidly. The way we write software is changing with new advancements, especially in automated code generation with artificial intelligence. 

At the same time, software testing has undergone significant evolution since the first software test was performed in the early 1950s. Transitioning from meticulously reviewing program instructions line by line to the current reality of automated software testing. Our contributions bring us one step closer to fully automated software testing in an ever-evolving industry. 

My coffee shop story is a work of fiction; however, there are many real-world examples of what can happen when software is not thoroughly tested. Whether it is a minor inconvenience or a major catastrophe, the consequences of software failure serve as a reminder that software testing is not an option; it is a necessity. It is the invisible element of our digital world, ensuring that our lives, and our coffee, can continue to flow smoothly. 

Text: Susruthan Seran, PhD candidate at School of Economics, Innovation and Technology, Kristiania

References:

Seran, S., Zhang, M., Duman, O. og Arcuri, A. (2025). Handling web service interactions in fuzzing with search-based mock-generation. ACM Transactions on Software Engineering and Methodology. 

Seran, S., Zhang, M. og Arcuri, A. (2023). Search-based mock generation of external web service interactions. I International Symposium on Search Based Software Engineering (s. 52–66). Cham: Springer Nature Switzerland. 

Arcuri, A., Zhang, M., Seran, S., Galeotti, J. P., Golmohammadi, A., Duman, O., ... og Ghianni, H. (2025). Tool report: EvoMaster—black and white box search-based fuzzing for REST, GraphQL and RPC APIs. Automated Software Engineering, 32(1), 4. 

Arcuri, A., Zhang, M., Belhadi, A., Marculescu, B., Golmohammadi, A., Galeotti, J. P. og Seran, S. (2023). Building an open-source system test generation tool: Lessons learned and empirical analyses with EvoMaster. Software Quality Journal, 31(3), 947–990.

We love hearing from you!

Send your comments and questions regarding this article by e-mail to kunnskap@kristiania.no.