Published papers

Individual fairness testing evaluates a machine learning (ML) classifier by estimating its individual fairness ratio (IFr) and has become an important approach for assessing ML classifiers. However, existing studies on ML fairness testing focus exclusively on the Fairness Through Awareness (FTA) setting, where protected attributes are explicitly used. In this study, we introduce the first framework for individual fairness testing under Fairness Through Unawareness (FTU), where protected attributes are not explicitly available. The proposed framework, called IFTU_NAWARE, evaluates a classifier by estimating its IFr in the FTU setting. It extends individual fairness testing approaches developed for FTA by incorporating techniques, including protected attribute inference with confidence control, a generalized notion of individual similarity, and statistically guaranteed termination criteria. Our experiments show that IFTU_NAWARE can compute IFr estimates with statistical guarantees and demonstrate the effectiveness of both the generalized similarity notion and confidence-based protected attribute inference. These results highlight both the feasibility and the limitations of assessing individual fairness under FTU and provide a foundation for future methodological advances.

Recent efforts in GUI testing of mobile applications have seen the use of reinforcement learning. Such work have focused on single agents, but they have had issues such as repeating the same paths, resulting in it being difficult to increase coverage. On the other hand, the reinforcement learning research field has seen the use of multiple agents.We propose a multi-agent reinforcement learning based approach to Android testing called MultiQDroid. MultiQDroid has multiple agents exploring the state space, each with a different exploration strategy. Agents can access memory containing their own explored states, as well as a shared memory that contains states that have been explored by all agents. The reward function is defined so that new states are given higher reward. Preliminary evaluation shows that MultiQDroid achieve higher coverage compared to a single agent approach.

Testing web applications is challenging due to their complex user interfaces and dynamic behaviors. Achieving comprehensive state exploration requires generating appropriate input values that satisfy validation constraints. Existing approaches using random input generation or model-based techniques often struggle with form validation and contextual input requirements. This paper proposes a novel approach leveraging Large Language Models (LLMs) to understand web application context and generate meaningful inputs for effective state exploration. Our method extracts UI elements and contextual information from HTML, converts them into structured prompts for LLMs, and generates executable Selenium WebDriver code with appropriate inputs. The system then executes this code and continues exploration by detecting state transitions using normalized Levenshtein distance. Evaluation on multiple open-source web applications shows up to 30% higher code coverage and increased detection of client-side errors, confirming the effectiveness of our approach to LLM-assisted testing for web applications.

The CC2020 project was charged with two main objectives, to produce a comprehensive report and a visualization tool to provide global guidance in an evolving computing environment as it affects computing baccalaureate degree programs worldwide. T he report is now in draft form and has undergone four rounds of review. The other goal of the CC2020 project is to develop a set of visualization-based tools that will help users to explore questions they may have concerning curricular guidelines, as well as local computing curricula. A visualization tool has been scoped and a prototype built. This special session will unveil the prototype for the SIGCSE community who will have the opportunity to view and review the tool.

A significant part of the recently published ACM and IEEE-CS report "Computing Curricula 2020: Paradigms for Global Computing Education"[1] was the development of visualizations of any computing degree and the comparison with the ACM and IEEE-CS approved curricula. This visualization covers discipline areas that have approved curricula: Computer Engineering, Computer Science, Software Engineering, Information Systems, Information Technology and Cyber Security. The CC2020 report developed a "Landscape of Computing Knowledge"table (see table 1) aggregating all the topic areas of all six approved curricula into 34 topic areas and then assigned a minimum and maximum value for each of topic areas. As part of the visualizations project an online application has been developed (see Figure 1) where stakeholders can assign a minimum and maximum value to each of the topic areas that are required in their degree program and then that program can be matched against the current ACM/IEEE-CS approved curricula and other degree programs globally. This will have significant importance for educators and other stakeholders of computing degree programs.

Graphical User Interface (GUI) testing is crucial for ensuring that Android applications deliver a positive, consistent, and functional user experience. Nonetheless, most automated GUI testing techniques aim at maximizing code coverage or finding as many crashes as possible, not focusing much on the functional features of the app. Recently, test reuse based on semantic similarities between applications with similar functionalities has been utilized to reduce the cost of GUI testing. However, they still face many obstacles, such as limited text semantic information and ineffective semantic matching rules, resulting in generating inaccurate test cases. In this paper, we propose SMAPDroid, a framework for reusing test cases between Android applications with similar features. Our evaluation results show that SMAPDroid performs better than prior work in terms of precision and recall, and reduced human effort by 91% compared to writing test scripts from scratch.

With the internet’s evolution, consumers increasingly rely on online reviews for service or product choices, necessitating that businesses analyze extensive customer feedback to enhance their offerings. While machine learning-based sentiment classification shows promise in this realm, its technical complexity often bars small businesses and individuals from leveraging such advancements, which may end up making the competitive gap between small and large businesses even bigger in terms of improving customer satisfaction. This paper introduces an approach that integrates large language models (LLMs), specifically Generative Pre-trained Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT)based models, making it accessible to a wider audience. Our experiments across various datasets confirm that our approach retains high classification accuracy without the need for manual labeling, expert knowledge in tuning and data annotation, or substantial computational power. By significantly lowering the barriers to applying sentiment classification techniques, our methodology enhances competitiveness and paves the way for making machine learning technology accessible to a broader audience.

Machine learning–based classifiers have been used for text classification, such as sentiment analysis, news classification, and toxic comment classification. However, supervised machine learning models often require large amounts of labeled data for training, and manual annotation is both labor-intensive and requires domain-specific knowledge, leading to relatively high annotation costs. To address this issue, we propose an approach that integrates large language models (LLMs) into an active learning framework, achieving high cross-task text classification performance without the need for any manually labeled data. Furthermore, compared to directly applying GPT for classification tasks, our approach retains over 93% of its classification performance while requiring only approximately 6% of the computational time and monetary cost, effectively balancing performance and resource efficiency. These findings provide new insights into the efficient utilization of LLMs and active learning algorithms in text classification tasks, paving the way for their broader application.

Background: Individual fairness testing aims to identify individual discriminatory instances (IDIs) to improve the fairness of machine learning classifiers through retraining. While prior studies have primarily evaluated fairness testing algorithms based on efficiency and retraining performance, emerging metrics such as the diversity of identified IDIs have recently been proposed. However, these alternative metrics have not yet been established as standard evaluation criteria. Aims: This study investigates the significance of IDI diversity as a metric for evaluating fairness testing algorithms. Specifically, we aim to validate its utility as a core evaluation metric by examining its correlation with both fairness improvement and accuracy degradation in retrained classifiers. Method: We conduct an empirical study using a newly developed framework called Redi. This framework generates multiple IDI sets with controlled variations in diversity, enabling systematic evaluation of their impact on retrained classifiers. We apply Redi to analyze the correlations between IDI diversity and retraining outcomes. The validity of the framework is further supported through auxiliary empirical analyses. Results: Our experiments confirm that IDI diversity exhibits a moderate correlation with fairness improvement, while showing only a weak correlation with accuracy degradation. Additionally, our regression analysis indicates that the actual impact of diversity on fairness improvement is substantial, whereas its impact on accuracy degradation is relatively negligible. Conclusions: These results show that higher IDI diversity substantially enhances fairness with minimal accuracy loss, suggesting that it should be adopted as a meaningful proxy metric for evaluating fairness testing algorithms, complementing the established metrics.

© 2020 IEEE. Most state-of-the-art clone detection approaches are aimed at finding clones accurately and/or efficiently. Yet, whether a code fragment is a clone often varies according to different people's perspectives and different clone detection tools. In this paper, we present CPPCD (CP-based Potential Clone Detection), a novel token-based approach to detecting potential clones. It generates CP (clone probability) values and CP distribution graphs for developers to decide if a method is a clone. We have evaluated our approach on large-scale software projects written in Java. Our experiments suggest that the majority of clones have CP values greater than or equal to 0.75 and that CPPCD is an accurate (with respect to Type-1, Type-2, and Type-3 clones), efficient, and scalable approach to detecting potential clones.

The ever increasing usage of Android devices and apps has created a demand for faster and reliable testing techniques. While the quality of test cases can be summed up based on the amount of code they cover, fault detection in applications is one of the main objectives for testing. We introduce an Android app testing approach which uses multiobjective genetic algorithm with elitism which finds optimal test cases by minimizing their length, maximizes the code coverage and fault detection capability, and minimizes the whole test suite for re-usability. In addition to that, we also incorporate a progress indicator which checks for improvements in test suite quality after subsequent generations and use it as a stopping criterion. The effectiveness of our approach is shown in our evaluation where it is able to perform better than the existing state-of-The-Art tools.

Since the big boom of smartphone and consequently of mobile applications, developers nowadays have many tools to help them create applications easier and faster. However, efficient automated testing tools are still missing, especially for GUI testing. We propose an automated GUI testing tool for Android applications using Deep Q-Network and semantic analysis of the GUI. We identify the semantic meanings of GUI elements and use them as an input to a neural network, which through training, approximates the behavioral model of the application under test. The neural network is trained using the Q-Learning algorithm of Reinforcement Learning. It guides the testing tool to explore more often functionalities that can only be accessed through a specific sequence of actions. The tool does not require access to the source code of the application under test. It obtains higher code coverage and is better at fault detection in comparison to state-of-the-art testing tools.

In recent years, researchers have actively proposed tools to automate testing for Android applications. Their techniques, however, still encounter major difficulties. First is the difficulty of achieving high code coverage because applications usually have a large number of possible combinations of operations and transitions, which makes testing all possible scenarios time-consuming and ineffective for large systems. Second is the difficulty of achieving a wide range of application functionalities, because some functionalities can only be reached through a specific sequence of events. Therefore they are tested less often in random testing. Facing these problems, we apply a reinforcement learning algorithm called Q-learning to take advantage of both random and model-based testing. A Q-learning agent interacts with the Android application, builds a behavioral model gradually and generates test cases based on the model. The agent explores the application in an optimal way that reveals as much functionalities of the application as possible. The exploration using Q-learning improves code coverage in comparison to random and model-based testing and is able to detect faults in applications under test.

Software testing is a crucial task. Unlike conventional software, AI software that uses decision-making algorithms or classifiers needs to be tested for discrimination or bias. Such bias can cause discrimination towards certain individuals based on their protected attributes, such as race, gender or nationality. It is a major concern to have discrimination as an unintended behavior. Previous work tested for discrimination randomly, which has resulted in variations in the results for each test execution. These varying results indicate that, for each test execution, there is discrimination that is not found. Even though it is nearly impossible to find all discrimination unless we check all possible combinations in the system, it is important to detect as much discrimination as possible. We thus propose Coverage-Guided Fairness Testing (CGFT). CGFT leverages combinatorial testing to generate an evenly-distributed test suite. We evaluated CGFT with two different datasets, creating three models with each. The results show an improvement in the number of unfairness found using CGFT compared to previous work.

Multiple developers often take part in a software development project. Although these developers are collaborating towards the development within the same project, each developer creates code on their own. This may lead to duplicate or similar code appearing in different parts of the software. Such code should be removed to improve maintainability. This paper proposes an approach to automatically detect such code, which we shall call functionally similar code. The unit of detection is at the method level, and we focus on input/output and the method structure using program dependence graph. We show the results of applying our approach on open source software.

With the spread of smartphones, the importance of automated testing of mobile applications has increased. However, many current approaches are inadequate, as they are not able to test functions that are available only on hard-to-reach GUI, which is a screen that can be reached only through a specific sequence of input events. To solve this problem, there has been an increase in testing research based on reinforcement learning, specifically Q-learning. Each research uses different learning targets and reward function. Testing research has also been done using Deep Q-Network, which extends reinforcement learning in a “deep” way. Although each work has conducted their own evaluation, it is not clear how the combination of learning algorithm, learning target, and reward function affects the result. To bridge this gap, we have conducted an empirical study comparing eight possible combinations. Our study found that the combination of Deep Q-Network as the learning algorithm, component as the learning target, and GUI change ratio as the reward function had the highest test quality in terms of code coverage.

Today's software has a very complex structure with multiple components, making it difficult to identify the cause of a fault. The process of identifying the cause of a fault may include referring to the logs from the system if they exist. But large and complex systems may generate a huge amount of logs, making the task of finding the important log messages to be a tedious task. In the case of systems that require continuous operation, the cause of faults must be identified quickly in an efficient manner. In this paper, we propose a method that identifies the log messages that are key for finding faults in server-side applications that has a tiered structure, such as LAMP (Linux, Apache, MySQL, PHP), and outputs logs (including traces during operation). The key part of our proposed approach is the application of Spectrum-Based Fault Localization (SBFL) to log files.

With the increasing use of machine learning software in our daily life, software fairness has become a growing concern. In this paper, we propose an individual fairness testing technique called KOSEI. Individual fairness is one of the central concepts in software fairness. Testing individual fairness aims to detect individual discriminations included in the software. KOSEI is based on AEQUITAS by Udeshi et al., a landmark fairness testing technique featuring a two-step search strategy of global and local search. KOSEI improves the local search part of AEQUITAS, based on our insight to overcome the limitations of the local search of AEQUITAS. Our experiments show that KOSEI outperforms AEQUITAS by orders of magnitude. KOSEI, on average, detects 5,084.8% more discriminations than AEQUITAS, in just 7.5% of the execution time.

This special session will introduce the participants to the CC2020 Visualization Tool based on the Landscape of Computing Knowledge Table and its 34 topics areas. It will guide them through the process of assigning a minimum and maximum value to the discipline areas of their own degree programme, then enter them into the application and allow the participants to see where in the landscape of computing their own program fits. It will also allow participants to compare their own programme with any other programme entered into the application. We hope to populate the database with programmes from all over the world over the next few months to ensure a global reach for comparison. There has never been a tool to compare computing degrees and this tool not only allows for comparison with current approved curricula but also allows participants to compare their degree with each other. This tool also enables employers to develop a guideline for potential employees and compare with job definitions to help with alignment of qualifications and competencies.