Automated User Interface Testing

Introduction

User interface (UI) testing is a process used to test if the application is functioning correctly. UI testing can be performed manually by a human tester, or it can be performed automatically with the use of a software program.

Automated UI testing is the automation of manual test tasks. Because manual test tasks can be time consuming and error prone, we implement automated UI testing as a more accurate, efficient, and reliable method. Over an extended period of time, automated UI testing becomes a cost-effective replacement to manual testing. However, it should be implemented after some part of the functionality is already finished (e.g. result of a sprint). Developing them in parallel with mainstream implementation might result in a headache while maintaining new functionality.

Furthermore, automated UI tests can run simultaneously on different platforms and on different browsers. However, they will not detect cases where the UI gets distorted. As soon as all DOM elements are in the document, the test will succeed. Automated UI tests become much more important when there are multiple people working on the same piece of code, for obvious reasons. Without good automated test coverage, complicated pieces of code quickly become one person's domain ("Don't touch that code without talking to Ted!").

One of the biggest benefits of automated UI testing is finding regression errors. They are supposed to ensure that new code you write doesn’t break or change any of the existing functionality. I found this statement that sums up the whole purpose of automated testing: “A well written unit test cannot find a bug... But it sure as heck can find a regression.”

Selenium Grid Cloud Solutions

There are multiple UI testing frameworks on the market, but one of the most commonly known and used is Selenium.

Selenium Grid allows scaling for a large number of tests, or for tests that must run in multiple environments. It also allows you to run different tests simultaneously on different remote machines. Utilizing Selenium Grid has two main advantages. First, if you have a large number of tests, or a slow-running test, you can boost its performance substantially by using Selenium Grid to run different tests at the same time using different machines. Second, if you’re running your tests on multiple environments you can have different remote machines supporting different browsers and operating systems. In each case, Selenium Grid greatly improves the time it takes to accomplish regression testing for a web project.

I had the opportunity to pilot two sites that offer cloud based Selenium Grid hub services: SauceLabs and BrowserStack. WebDriver was introduced for accessing the selenium hub service. WebDriver is designed to provide a programming interface to access browser functionality. It doesn’t matter if the browser instance is located in a local computer or remotely in the Selenium Grid hub.

Both of them offer environments to run automated UI tests based on the Selenium WebDriver, and both of them have similar functionalities. The key features are listed below:

Multiple browsers and platforms:

Platform and browser capabilities on SauceLabs

Automated and real time testing environments – both providers offer this functionality.

Mobile testing – both providers offer this functionality.

Parallelized testing – both providers offer this functionality.

Log views - SauceLabs has a very detailed log view. It even displays info from the browser console. This is very helpful in finding problems on a web site.

Live automated test video – both sites have this functionality. In SauceLabs, you can even take control of a live test and inspect a page or its elements.

Screenshots - both hubs have a screenshot functionality. BrowserStack even displays screenshots in the main test log.

Typically, I run tests in a SauceLabs environment. One of the SauceLabs features I have found very useful is the ability to take control of the running test and inspect the browser screen. It's a great tool to get additional debug information for a failed test.

Another Selenium Hub I have tried is BrowserStack. Though all tests offer the same results on both hubs, BrowserStack seems to lack some core functionality compared to SauceLabs. The biggest shortage I have noticed is that the history grid lacks important information. All you have here is the platform name and browser name with the version, so it can be very difficult to find particular tests in the list (search is not available here either).

But, one of the biggest advantages BrowserStack has over SauceLabs is that BrowserStack bills for user count and parallelism degree, while SauceLabs bills for the test run minutes.

UI tests for .NET projects

Tools

Let’s talk about the actual tools used to write automated tests. Selenium WebDriver UI tests can be implemented using any unit testing framework. In our daily unit testing we use NUnit for that purpose. To run tests locally, we use a Visual Studio ReSharper plugin. For continuous integration and automated tests runs – we use CruiseControl.Net continuous integration server.

UI tests run slowly and it makes sense to run them in parallel. However, it is a big challenge because it involves shared data synchronization. In order to run separate UI tests simultaneously, we create multiple parallel tasks for the NUnit Console in the continuous integration server. The easiest way to synchronize shared data is to divide all tests into separate isolated groups. Isolated groups means that a testing functionality from one group shouldn’t have shared data with another group.

Writing automated UI tests involves some recurring steps before each test can run. For example, you must initialize a connection to the Selenium hub, prepare data records in a database, execute test operations, send test status to the hub, and clear test data from the database. To avoid code duplication we have created SeleniumTestBase, an abstract class with all helper methods. Each test fixture should inherit it.

To use the SeleniumTestBase base class or other extension method install Devbridge.SeleniumTools package running the following command in the Package Manager Console:

PM> Install-Package Devbridge.SeleniumTools

Basic Automated UI Test

In this part I will show you an example of implementing basic UI tests. As I mentioned earlier, each test class must inherit the base helper class. This class holds initialization logic and some helper methods that I want to explain in detail. Here is a basic UI test based on a Selenium WebDriver:

namespace SeleniumExtensionTest
 {
   [TestFixture]
   public class TestCodingLove : SeleniumTestBase
   {
       [Test]
       public void TestFindCodingLove()
       {
            Driver.Navigate().GoToUrl( "http://www.google.com" );
            Console.WriteLine( "www.google.com opened." );
            WaitForPageLoad();

            var textInput = Driver.FindElement(By.Name( "q" ));
            textInput.SendKeys( "thecodinglove" );
            textInput.SendKeys( Keys .Enter);
            Console.WriteLine( "Searched for 'thecodinglove'." );

            var site = WaitUntilElementExists(By.XPath( "//a[@href='http://thecodinglove.com/']" ));
            Console.WriteLine( "Search results arrived." );
            site.Click();
            Console.WriteLine( "Clicked on thecodinglove.com link." );
            WaitForPageLoad();

            FindElement(By.LinkText( "the_coding_love();" ));
            Console.WriteLine( "thecodinglove.com opened." );
        }
     }
 }

In this test sample I want to find http://thecodinglove.com/ site with the Google search engine. First of all, we need to give the driver instructions to navigate to ‘www.google.com.’ Then, we must wait for the page to fully render.

Wait for page load

A WaitForPageLoad method is helpful when a site is refreshing and a test must wait until a page is completely rendered. This method checks if document.readyState == complete || document.readyState == loaded. It’s important to notice, that even if this function waits for the page to load, JavaScript logic can be executed later in the background. A WaitForPageLoad method does not detect if page elements are rendered with JavaScript.

Wait until element exists

In this example, a search action is triggered while typing and no button has to be pressed. In this situation, the WaitForPage method is not suitable. In such cases we need more wait methods to interact with a dynamic page. The first of them is WaitUntilElementExists. It loops and checks if an element is displayed in the DOM. If an element does not become visible within a timeout interval, an exception is thrown. In line 18 this function is used to wait while http://thecodinglove.com/ link appears on the webpage.

As soon as a link appears on the page, we can click on it and wait while the page is loaded. Then, we can check if the correct page was loaded by finding one of the known elements on the web site. If the correct page was loaded - the test was successful.

Wait until element does not exist

Sometimes we need to wait until an element disappears. For example, if we are testing a pager functionality and we are on the last page, we have to check if the pager`s Next button disappears. We can’t use WaitUntilElementExists inversion because at the very first check the element might still exist and the function might fail. We need to continue checking for a certain period of time, and check if the element disappears during that period. A WaitUntilElementNotExists method is implemented for such a check.

Preparing data for tests

Most of the time test data must be created prior to running a test. A set of test data can be created in the following ways:

Using automated UI tests with extra scenarios steps.
Adding test records directly into a database.
Data entries can be prepared manually through project UI. In this case, a database should be restored before each test run.

The first option is very time consuming and expensive. It’s only worth using when validating if an entity creation form works correctly. All other times when a data test is required, inserting data directly into a database is a better option. To make this easier, a fluent interface can be incorporated.

For example:

Database.CreateStaffMember(“Name”,”John“).WithTitle(“QA”)).Save();

Alternatively, there is a third option that can be used. Test data can be prepared manually with a website. In this option, the database should be backed up and restored before each test run. This option requires accessing website databases and changing them with new ones. Furthermore, it would be hard to maintain different versions of databases. Not to mention, running tests in a real environment may be impossible.

What's next

I would like to mention a Page Object design pattern. Its key concept is to create classes that reflect a page or a page part functionality. For example, we can implement GoogleSearchPage class with a method SearchFor(string text). Using this pattern, a test can be written without knowing about the page’s internal HTML structure and technologies. Everything you need to know is functionality carried out by the page - you could call GooglePage.SearchFor(“Test search”) and then check the results. Methods on the PageObject should return other PageObjects. This means that we can effectively model the user's journey through our application. The basic UI test can be rewritten using this technique:

namespace SeleniumExtensionTest
 {
     [TestFixture]
     public class TestCodingLove : SeleniumTestBase
     {
         [Test]
         public void TestFindCodingLove()
         {
              var googlePage = new GooglePage(Driver);

              googlePage.SearchFor("thecodinglove");

              var codingLovePage = googlePage.ClickOnResultWithLink("http://thecodinglove.com");
         }
     }
 }

Conclusions

In this blog post I have covered some practical aspects for writing UI tests, including personal experiences that I have gained from real project testing. Writing automated UI tests is not an easy job, because you have to be cognizant about how to do the following:

move through the application and enter data;
detect state changes and failure conditions. Stated bluntly, worst practice is to rely on pixel coordinates too much.