论文标题

通过图像嵌入和增强学习的有效,独立于平台的GUI测试

Effective, Platform-Independent GUI Testing via Image Embedding and Reinforcement Learning

论文作者

Yu, Shengcheng, Fang, Chunrong, Li, Xin, Ling, Yuchen, Chen, Zhenyu, Su, Zhendong

论文摘要

软件应用程序在社会各个方面都起着越来越重要的作用。特别是,移动应用程序和Web应用程序在所有应用程序中最普遍,并且在各个行业以及人们的日常生活中都广泛使用。为了确保移动和网络应用程序质量,已经引入了许多方法,以通过自动探索​​来改善应用GUI测试。尽管付出了广泛的努力,但现有的方法仍在达到高码覆盖范围,构建高质量模型以及通常适用的情况下仍受到限制。基于增强学习的方法面临着艰难的挑战,包括有效的应用程序抽象,奖励功能设计等。此外,它们在很大程度上取决于特定的执行平台,从而导致不良的普遍性和无法适应不同平台。 我们提出了PIRRTEST,这是一种有效的与应用程序测试的无平台方法。它以一种新颖的协同方式利用计算机视觉和强化学习技术进行自动测试。它从GUI页面中提取GUI小部件,并表征相应的GUI布局,将GUI页面嵌入为状态。 App GUI状态结合了宏观的观点和微观的透视图,并从GUI图像中附加了关键的语义信息。这使得Pirltest可以独立于平台,并使测试方法通常适用于不同的平台。 Pirltest在好奇心驱动的策略的指导下探索了应用程序,该策略使用Q网络来估计特定的国家行动对的值,以鼓励在没有平台依赖性的未经覆盖的页面中进行更多探索。探索将以所有行动的奖励分配,这些操作是考虑到App GUI状态和具体小部件的设计,以帮助该框架探索更多未发现的页面。

Software applications have been playing an increasingly important role in various aspects of society. In particular, mobile apps and web apps are the most prevalent among all applications and are widely used in various industries as well as in people's daily lives. To help ensure mobile and web app quality, many approaches have been introduced to improve app GUI testing via automated exploration. Despite the extensive effort, existing approaches are still limited in reaching high code coverage, constructing high-quality models, and being generally applicable. Reinforcement learning-based approaches are faced with difficult challenges, including effective app state abstraction, reward function design, etc. Moreover, they heavily depend on the specific execution platforms, thus leading to poor generalizability and being unable to adapt to different platforms. We propose PIRLTest, an effective platform-independent approach for app testing. It utilizes computer vision and reinforcement learning techniques in a novel, synergistic manner for automated testing. It extracts the GUI widgets from GUI pages and characterizes the corresponding GUI layouts, embedding the GUI pages as states. The app GUI state combines the macroscopic perspective and the microscopic perspective, and attaches the critical semantic information from GUI images. This enables PIRLTest to be platform-independent and makes the testing approach generally applicable on different platforms. PIRLTest explores apps with the guidance of a curiosity-driven strategy, which uses a Q-network to estimate the values of specific state-action pairs to encourage more exploration in uncovered pages without platform dependency. The exploration will be assigned with rewards for all actions, which are designed considering both the app GUI states and the concrete widgets, to help the framework explore more uncovered pages.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源