fbpx
Select Page

Artificial Intelligence for Software Testing

It is interesting to find for some people that even software testing has this exposure to intelligence. So many manual testers are also focusing to learn this new and interesting concept of artificial intelligence in software testing. With the fast advance of AI technology and data-driven machine learning techniques, building high-quality AI-based software in different application domains has become a very hot research topic in both academic and industry communities. Many technologies have been developed to build intelligent application systems based on multimedia inputs to achieve intelligent functional features, such as how to do recommendation, how to do object detection and prediction, natural language processing and translation, and so much. This brings a strong demand for quality verification and assurance for AI software applications. The research work in the current scenario seldom discusses AI software testing questions, challenges, and validation with clear quality requirements and criteria. This article focuses on how AI software quality, including focuses, features, and process and potential testing approaches are working in the current scenario. Moreover, it presents a test process and a classification-allowed test modeling for AI classification functional testing. Finally, it will describe the challenges, issues, and importance of AI software testing.

Intuitively, AI software testing refers to diverse quality testing activities for AI-based software systems using well-defined quality validation models, methods, and tools. Its major objective is to validate system functions and features developed based on machine learning models, techniques and technologies. AI software testing includes the following primary goals: – Establish AI function quality testing requirements and assessment criteria – Detect AI function issues, limitations, and quantitative and quality problems – Gain the quality confidence of AI functional features developed based on AI techniques and machine learning models.  – Evaluate the AI system quality against well-established quality requirements and standards. 

The current AI-based software and applications are developed based on state-of-the-art machine learning models and techniques through large-scale data training to implement diverse artificial intelligent features and capabilities. Current AI-based system functions and features can be classified into the following categories: a) natural language processing (NLP) capability with language understanding and translation; b) detection and recognition function, for example, human face identification, voice recognition, and object detection; c) recommendation functional features in e-commerce and advertising; d) unman-controlled vehicles, robots, and UAVs; e) question and answer functions to assist users in messaging, phone calls, search, and smart home appliance control; f) object identification and classification; g) prediction and business decision-making, and so on.  A time will come when software testing will be changed by artificial Intelligence. Artificial intelligence in any field of software is to make computers think the way human thinks and thus produce intelligent software systems.

In the past two years, software quality testing professionals have conducted testing projects for numerous mobile apps powered with machine learning capabilities using conventional software testing and tools. We as software testers have encountered many questions during software testing. The major ones are listed below. – What is AI software testing?  

– How to test AI functions in a mobile app? – How to identify and establish well-defined test requirements for AI functions in a mobile app? – What and where are the cost-effective testing models and methods for testing AI functions? – What and where are the adequate quality assessment criteria for AI functions? – How to evaluate the training and test data sets? – Where are the automatic tools supporting AI software testing? 

For Free, Demo classes Call: 8237077325
Registration Link: Click Here!

In addition, we also found the following common facts and features of current AI mobile software apps after validating numerous ones. 

  1. Limited data training and validation  

– Most of our tested mobile apps with AI features are built based on machine learning models and techniques, and trained and validated with limited input data sets under ad-hoc contexts.  

  1. Data-driven learning features  

– Many mobile apps with learning features provide static and/or dynamic learning capabilities that affect the under-test software outcomes, results, and actions. 

  1. Uncertainty in system outputs, responses, or actions  

– Since existing AI-based models are dependent on statistics algorithms, this brings the outcome uncertainty of AI software. We experienced many mobile apps with AI functions generated inconsistent outcomes for the same input test data when context conditions are changed. Similarly, we also encountered accuracy issues when we changed the training test data sets and/or test data sets. 

This unique AI software features causes new difficulties and challenges in quality validation, hence, performing AI quality validation becomes a critical concern and a hot research subject for current and future AI software development and engineering. Although there have been numerous published papers addressing data quality and quality assurance in the past, seldom researches focus on validation for AI software from a function and feature view. As more and more intelligent software systems are developed, there will be a strong demand for quality testing and assurance services for AI software system.

What Is AI Software Testing?   

Intuitively, AI software testing refers to diverse quality testing activities for AI-based software systems using well-defined quality validation models, methods, and tools. Its major objective is to validate system functions and features developed based on machine learning models, techniques and technologies. AI software testing includes the following primary goals: – Establish AI function quality testing requirements and assessment criteria – Detect AI function issues, limitations, and quantitative and quality problems – Gain the quality confidence of AI functional features developed based on AI techniques and machine learning models.  – Evaluate the AI system quality against well-established quality requirements and standards. 

For Free, Demo classes Call: 8237077325
Registration Link: Click Here!

 Major Testing Focuses and Scope 

In the past two years, we have validated different types of AI mobile apps with diverse AI capabilities and features. Our students have tested numerous mobile apps powered with diverse machine learning models and AI algorithms. Here are some typical examples. 

 AI Software Testing Scope – Apple Siri – It is a built-in, voice-controlled personal assistant for Apple users. Calorie MAMA – It is a smart camera app that uses deep learning to track nutrition from food images. – Seeing AI – It is a free app that narrates the world around you. Designed for the blind and low vision community, this ongoing research project with powered AI techniques. Its goal is to open up the visual world for users by describing nearby people, text, and objects. 

– Check My Age – It is a biometric face detection and age estimation application. It uses the best technology for face recognition algorithms to find the age from the face itself

The major quality validation focuses on mobile apps with AI features that can be summarized below.  (a) Data quality validation: 

AI-powered functions/features are developed and trained based on selected/developed machine learning models using large-scale training data in a sample-based training approach. Many recent machine learning project experiences suggest that the quality of training data plays a critical role in AI function training and development. Hence, training data quality validation is not necessary but also critical. In our recent machine learning project and class experience, we encountered serious issues in training data quality checking for large-scale unstructured training data (such as images, audios, and videos). Three major issues are highlighted here. 

Issue #1 – Domain-specific training data quality checking could be very costly and time-consuming. For example, training data for medical machine learning projects require quality validation and confirmation from medical doctors. This is not only costly but very time-consuming. 

Issue #2 – There is a lack of automatic unstructured data quality validation tools. Although some existing tools (such as AAAA, BBB) are available for raw data quality checking, we could not find automatic tools for validating annotated rich-media training data (i.e. video, audio, and images). This becomes a serious issue in training data quality validation. 

Issue #3 – There is a lack of well-defined data quality evaluation models and assessment metrics for unstructured training data, including images, videos, and audios. (b) White-box testing focuses: 

For Free, Demo classes Call: 8237077325
Registration Link: Click Here!

In a conventional software system with AI functions/features, its program is structured with complex control structures to generate expected software behaviors and operations. A software program usually is written based on function-oriented logic algorithms, limited data parameters, and Boolean conditions for business rules and system constraints.  White-box testing for a conventional software system focuses on the detection of the program errors in its structures and logics, and limited data value settings, as well as Boolean conditions and logics. The typical white-box testing goal is to achieve predefined program logic structure coverage, such as code coverage, branch coverage, and condition coverage. 

Based on our recent teaching and research experience in AI software projects, we found that AI software development is highly based on selected machine learning models using statistic algorithms, and these models are trained using different learning approaches based on provided large-scale training and testing data. Using the current machine learning platform (such as TensorFlow), developed AI software programs usually include simple control program structures, limited business logic, basic system behavior conditions and constraints due to the nature of statistic model-based coding. This implies that the program based white-box testing becomes a trivial task.

We as software tester think that a time will come when software testing will be changed by artificial Intelligence. Artificial intelligence in any field of software is to make computers think the way human thinks and thus produce intelligent software systems. The remaining questions are addressed soon.

Author:

Priti Jha,
SevenMentor Pvt Ltd.

Call the Trainer and Book your free demo Class for now!!!

call icon

© Copyright 2019 | Sevenmentor Pvt Ltd.

 

Pin It on Pinterest

× How can I help you?