Comparing GUI Automation Testing Tools for Dynamic Web Applications
Keywords:
Web application, GUI, Software testing, Agile testingAbstract
Automating software testing helps agile teams to release production-ready software as often as needed. There are so many automation testing tools with record/playback features available for test teams to automate functional tests against web applications. However, choosing the right tool can be vital to project success. This paper presents a survey on automation testing tools for web applications. These tools have been chosen from both open-source, namely Selenium and Sahi, and from commercial tools such as TestComplete and Microsoft Visual Studio 2010. These tools were compared based on the effectiveness of recorder/playback tools, handling of page waits, cross browser compatibility, technical support, and the number of different techniques available to programmatically locate elements on web pages. Results show that free and simple tools can be more powerful and time saving, compared to commercially sophisticated and expensive tools. This paper also summarizes best practices and guidelines when adopting automated GUI functional tests against web applications, drawing a road-map for test engineers to follow.
References
Afroz, S., Rani E. &Priyadarshini, N. (2011) “Web Application – A Study on Comparing Software Testing Toolsâ€, International Journal of Computer Science and Telecommunications, 2(3).
Business Internet Group of San Francisco (2003) The BIG-SF Report on Government Web ApplicationIntegrity. Available at: http://www.tealeaf.com/downloads/news/analyst_report/BIG-SF_Report_Gov2003-05.(Accessed: 9 May 2011).
Business Internet Group of San Francisco (2003) The Black Friday Report on Web Application Integrity. Available at: http://www.tealeaf.com/downloads/news/analyst_report/BIGSF_BlackFridayReport.pdf. (Accessed: 9 May 2011).
Brooks, P., Robinson, B., and Memon, A. M.(2009) “An initial characterization of industrial graphical user interface systems,†in ICST 2009: Proceedings of the 2nd IEEE International Conference on Software Testing, Verification and Validation. Washington, DC, USA: IEEE Computer Society, 2009.
Xun Yuan, Myra, B. Cohen, Atif M Memon, (2011) “GUI Interaction Testing: Incorporating Event Contextâ€, IEEE Transaction on Software Engineering, 37(4): 559-574.
Sommerville, I. (2004) Software Engineering, England: Addison Wesley.
Crispin, L. & Gregory, J. (2009) Agile Testing: A Practical Guide for Testers and Agile Teams, Boston: Addison Wesley.
Wikipedia (2012) Web Applications.Available at:http://en.wikipedia.org/wiki/Web_application (Accessed: 5 May 2012).
SmartBear Software (2011) Testing Dynamic Web Pages.Available at: http://smartbear.com/support/viewarticle/12725/
(Accessed 16 April 2011).
w3schools.com (2012) Learn HTML [Online]. Available at: http://www.w3schools.com/html/html_intro.asp (Accessed: 14 March 2012).
West, J. (2012) “Automated Testing in Agile Environmentsâ€, [Online]. Available at:http://support.smartbear.com/articles/testcomplete/automated-testing-agile-environment/ (Accessed: 10 May 2012).
Levinson, J. (2011) “Software Testing With Visual Studio 2010â€, Boston: Pearson, Inc.
Tyto Software Ltd (2012) Sahi[online]. Available at: http://sahi.co.in/w/sahi (Accessed 6 May 2012).
Riley, M. (2010) DevProConnections. Available at: http://www.devproconnections.com/article/software-testing/Review-SmartBear-Software-s-TestComplete-8-Enterprise (Accessed 15 April 2011).
SmartBear Software (2011) Top Reasons to Try TestComplete, [Online]. Available at:http://www.automatedqa.com/products/testcomplete/top-reasons-to-try/ (Accessed: 12 May 2011).
Microsoft (2012) Understanding Web Tests [Online]. Available at: http://msdn.microsoft.com/en-us/library/ms182537%28v=vs.90%29.aspx (Accessed 2 March 2012).
Selenium IDE (2012) Introduction to Selenium IDE [Online]. Available at: http://seleniumhq.org/docs/02_selenium_ide.html#script-syntax (Accessed 3 March 2012).
Java-Source.net (2012) Open Source Web Testing Tools in Java[Online]. Available at: http://java-source.net/open-source/web-testing-tools (Accessed 17 March 2012).
SmartBear Software (2011) Testing Dynamic Web Pages. Available at: http://smartbear.com/support/viewarticle/12725/
(Accessed 16 April 2011).
Stack Overflow (2012) Stack Overflow is a programming Q & A site that’s free. Available at: http://stackoverflow.com (Accessed 16 January 2012).
Zain, S., Elyan, D. (2012) ‘Automated User Interface Testing for Web Applications, and TestComplete’, CUBE 2012: International IT Conference and Exhibition. Pune, India 3-5 September. ACM, pp. 350-354.
Downloads
Published
How to Cite
Issue
Section
License
- Papers must be submitted on the understanding that they have not been published elsewhere (except in the form of an abstract or as part of a published lecture, review, or thesis) and are not currently under consideration by another journal published by any other publisher.
- It is also the authors responsibility to ensure that the articles emanating from a particular source are submitted with the necessary approval.
- The authors warrant that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required.
- The authors ensure that all the references carefully and they are accurate in the text as well as in the list of references (and vice versa).
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Attribution-NonCommercial 4.0 International that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
- The journal/publisher is not responsible for subsequent uses of the work. It is the author's responsibility to bring an infringement action if so desired by the author.