Call for Papers

Software systems (e.g., smartphone apps, desktop applications, telecommunication infrastructures and enterprise systems, etc.) have strict requirements on software performance. Failing to meet these requirements may cause business losses, customer defection, brand damage, and other serious consequences. In addition to conventional functional testing, the performance of these systems must be verified through load testing or benchmarking to ensure quality service.

Load testing examines the behavior of a system by simulating hundreds or thousands of users performing tasks at the same time. Benchmarking compares the system's performance against other similar systems in the domain. The workshop is not limited by traditional load testing; it is open to any ideas of re-inventing and extending load testing, as well as any other way to ensure systems performance and resilience under load, including any kind of performance testing, resilience / reliability / high availability / stability testing, operational profile testing, stress testing, A/B and canary testing, volume testing, and chaos engineering.

Load testing and benchmarking software systems are difficult tasks that require a deep understanding of the system under test and customer behavior. Practitioners face many challenges such as tooling (choosing and implementing the testing tools), environments (software and hardware setup), and time (limited time to design, test, and analyze). Yet, little research is done in the software engineering domain concerning this topic.

Adjusting load testing to recent industry trends, such as cloud computing, agile / iterative development, continuous integration / delivery, micro-services, serverless computing, AI/ML services, and containers poses major challenges, which are not fully addressed yet.

This one-day workshop brings together software testing and software performance researchers, practitioners, and tool developers to discuss the challenges and opportunities of conducting research on load testing and benchmarking software systems. Our ultimate goal is to grow an active community around this important and practical research topic.

We solicit two tracks of submissions:

  1. Research or industry papers:
  2. Presentation track for industry or research talks:
Research/Industry papers should follow the standard ACM SIG proceedings format and need to be submitted electronically via EasyChair (LTB 2023 track). Extended abstracts for the presentation track need to be submitted as "abstract only'' submissions via EasyChair as well. Accepted papers will be published in the ICPE 2023 Companion Proceedings. Submissions can be research papers, position papers, case studies, or experience reports addressing issues including but not limited to the following:


Instructions for Authors from ACM

By submitting your article to an ACM Publication, you are hereby acknowledging that you and your co-authors are subject to all ACM Publications Policies, including ACM's new Publications Policy on Research Involving Human Participants and Subjects. Alleged violations of this policy or any ACM Publications Policy will be investigated by ACM and may result in a full retraction of your paper, in addition to other potential penalties, as per ACM Publications Policy.

Please ensure that you and your co-authors obtain an ORCID ID, so you can complete the publishing process for your accepted paper. ACM has been involved in ORCID from the start and we have recently made a commitment to collect ORCID IDs from all of our published authors. The collection process has started and will roll out as a requirement throughout 2022. We are committed to improve author discoverability, ensure proper attribution and contribute to ongoing community efforts around name normalization; your ORCID ID will help in these efforts.

Important Dates

Paper Track (research and industry papers):

Abstract submission: January 16, 2022, AOE January 23, 2023, AOE;
Paper submission: January 23, 2023, AOE;
Author notification: February 13, 2023;
Camera-ready version: February 20, 2023

Presentation Track:

Extended abstract submission: February 17, 2023, AOE;
Author notification: February 24, 2023, AOE;
Workshop date: April 16, 2023


Organization:

Chairs:

Alexander Podelko AWS, USA
Heng Li Polytechnique Montréal, Canada
Changyuan Lin York University, Canada


Program Committee:

Jinfu Chen Wuhan University, China
Lizhi Liao Concordia University, Canada
Diego Elias Costa UQAM, Canada
Michele Tucci Charles University, Czech Republic
Daniele Di Pompeo University of L'Aquila, Italy
Yiming Tang Concordia University, Canada
Sen He Augusta University, USA
Klaus-Dieter Lange HPE, USA
Gerson Sunyé University of Nantes, France


Past LTB Workshops: