Testing views with HtmlUnit in Spring

Testing view part of web applications in Spring isn't a simple task. But to simplify it, you can use libraries as HtmlUnit, a kind of command-line browser written in Java. And Spring Test MVC HtmlUnit allows to integrate this library inside Spring test cases and to write more complex test cases.

In this article we'll write some test cases with Spring Test MVC HtmlUnit. But before that, we'll introduce the main component used to it, HtmlUnit. At the second part, we'll see how we can inspect view part with native Spring test tools. The third part of this article will show how to integrate HtmlUnit with Spring.

HtmlUnit presentation

HtmlUnit defines itself as "GUI-less browser for Java programs". It's like a Selenium test package where you can easily inspect given HTML page and check, for example, if given element is present at the right place of the DOM. Both allow us also to simulate user actions as clicks, forms sending. More, HtmlUnit even supports JavaScript.

The central working unit of HtmlUnit is com.gargoylesoftware.htmlunit.WebClient. As its name indicates, it's the class which simulates usual web browser. Thanks to its getPage method it allows to get an implementation of com.gargoylesoftware.htmlunit.Page interface. Further, this instance is used to represent content of the page visited by the WebClient. The Page's object can return various informations about read page: encoding, <head /> tag informations, <body /> elements or only some type of elements (for example: only <a /> tags). To observe the potential of HtmlUnit, take a look on following JUnit case in which we'll validate localhost page containing some links and texts:

public class LocalPageTest {
  @Test
  public void test() {
    WebClient client = new WebClient();
    client.getOptions().setJavaScriptEnabled(false);
    HtmlPage content = client.getPage("http://localhost/test.html");
    assertTrue("Content shouldn't be empty", 
      !content.asText().equals(""));
    HtmlAnchor defAnchor = 
      content.getAnchorByHref("/tested-anchor.html");
    assertTrue("Anchor 'tested-anchor.html' shouldn't be null", 
      defAnchor != null);
    HtmlPage defAnchorPage = defAnchor.click();
    String expectedFragment = "test fragment";
    assertTrue("Tested article should contain '"+expectedFragment+"' but it didn't", 
      defAnchorPage.asText().contains(expectedFragment));
  }

}

This example shows only the basic things which we can do with HtmlUnit. First, we download the page. After we get an anchor which href attribute equals to "tested-anchor.html". After, we click on retrieved anchor and get the content of next page. If it contains the fragment of the article about defensive programming, we consider the test as correct.

View testing with Spring native tools

Actually it's difficult to test the content of given page with Spring native test tools. Spring tests are based on mocking of real objects. Ie that it replaces real-world objects as HttpServletRequest or HttpServletResponse by corresponding objects adapted to test purposes (MockHttpServletRequest and MockHttpServletResponse). The mock object responsible for handling Spring MVC project is MockMvc. However, this mocking object doesn't know to work with technologies strongly dependeding on servlet containers. It means that it will correctly generate the output for all templating systems as Velocity, Freemarker, but it won't be able to do that for JSP which is a servlet container-depending technology.

To see that, let's make two web application context: one returning views with Velocity and second one with classical JSP. The content of generated page is:

<body><h1>Hello world !</h1><span>Test from ${contextVariable}</span></body>

First, Velocity configuration and test:

<bean id="velocityConfig" class="org.springframework.web.servlet.view.velocity.VelocityConfigurer">
  <property name="velocityProperties">
    <props>
      <prop key="resource.loader">file</prop>
      <prop key="file.resource.loader.class">
      org.apache.velocity.runtime.resource.loader.FileResourceLoader
      </prop>
      <prop key="file.resource.loader.path">/home/bartosz/tmp/templates/velocity</prop>
      <prop key="file.resource.loader.cache">false</prop>
    </props>
  </property>
</bean>
<bean id="viewResolver" class="org.springframework.web.servlet.view.velocity.VelocityViewResolver">
  <property name="prefix" value=""/>
  <property name="suffix" value=".vm"/>

  <!-- if you want to use the Spring Velocity macros, set this property to true -->
  <property name="exposeSpringMacroHelpers" value="true"/>

</bean>

And this is configuration used to test JSP rendering:

<bean id="viewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver"  p:prefix="/" p:suffix=".jsp" />

Test case is the same for both configurations:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations={"classpath:META-INF/applicationContext-test.xml"})
@WebAppConfiguration
public class TestViewsRendering {
  @Test
  public void testViewWithContextVar() {
    MvcResult result = this.mockMvc.perform(get("/viewTest")).andReturn();
    ModelAndView modelAndView = result.getModelAndView();
    String expectedResponse = "<body><h1>Hello world !</h1> <span>Test from contextVariable</span></body>";
    String response = result.getResponse().getContentAsString();
    assertTrue("Response '"+response+"' should be the same as expected '"+expectedResponse+"'", 
      expectedResponse.equals(response));
  }
} 

Set up Velocity configuration into application context file and launch the test. It should pass. It's not the case for JSP configuration. After set up this configuration, you should get following exception:

java.lang.AssertionError: Response '' should be the same as expected '<h1>Hello world !</h1> Test from contextVariable'
  at org.junit.Assert.fail(Assert.java:93)
  at org.junit.Assert.assertTrue(Assert.java:43)
  at com.mysite.test.TestViewsRendering.testViewWithContextVar(TestViewsRendering.java:48)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
  at java.lang.reflect.Method.invoke(Unknown Source)
  at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
  at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
  at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
  at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
  at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
  at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
  at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
  at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
  at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:232)
  at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:89)
  at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
  at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
  at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
  at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
  at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
  at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
  at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
  at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
  at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:175)

This test proves that we can't test JSPs rendering. Only views don't rendered by servlet container (as Tomcat) can be tested. So, let's return to the Velocity configuration and see which view elements we can test with Spring basic test package:

@Test
public void testViewWithContextVar() { 
  MvcResult result = this.mockMvc.perform(get("/viewTest"))
                  .andExpect(xpath("//h1").string("Hello world !"))
                  .andExpect(xpath("//h1").nodeCount(1))
                  .andExpect(xpath("//p").nodeCount(0))
                  .andReturn();
  ModelAndView modelAndView = result.getModelAndView(); 
}

This test case shows that with native Spring tools we can also inspect returned page (only for non-servlet container views) and check if expected elements are present. However, this way of testing isn't so elaborated as HtmlUnit.

Implement HtmlUnit with Spring

Before write som test cases with HtmlUnit, we need to add it into project's pom.xml:

<dependency>
  <groupId>org.springframework</groupId>
  <artifactId>spring-test-mvc-htmlunit</artifactId>
  <version>1.0.0.M1</version>
  <scope>test</scope>
</dependency>

The second step to do is to change the web application's context. It can't be equal to /. This is known issue of the project and should be treated in Github's issue tracker " Support context root of "/" #20". So, we need to change the context in web deployement file to, for example /test:

<servlet-mapping>
  <servlet-name>springDispatcherServlet</servlet-name>
  <url-pattern>/test</url-pattern>
</servlet-mapping>

After this quick setup, we can pass to define our test case. We want to test the same path as previously (/viewTest). But because the root context changed, we need to prepend it to the path. We receive now /test/viewTest. However, this path won't work because of no protocol defined in the path. We need to add it to construct a valid URL, as for example: http://mytestpage/test/viewTest. Note that mytestpage host can no exist in the host list defined in your system.

The test case written with Spring HtmlUnit project will cover following scenario. First, we want to check if one <h1 /> with content "Hello world !" is defined at the page. After, we want to click on <span /> tag. This click should provoke the generation of new <a /> element in JavaScript and appending it into <body />. After, we want to click on this element and check if the second page was correctly generated. Whole test case looks like:

@Test
public void testViewHtmlUnit() {
  WebConnection webConnection = new MockMvcWebConnection(mockMvc);
  WebClient webClient = new WebClient();
  webClient.setWebConnection(webConnection);
  HtmlPage testPage = webClient.getPage("http://mytestpage/test/viewTest");
    
  // check if <h1 /> exists
  HtmlElement h1 = (HtmlElement) testPage.getFirstByXPath("//h1");
  assertTrue("<h1 /> should be found but isn't", 
    h1 != null);
  assertTrue("<h1 /> should be equal to 'Hello world !' but is '"+h1.asText()+"'", 
    h1.asText().equals("Hello world !"));
    
  // check if <span /> with id="testSpan" exists and click on it
  HtmlElement clickableSpan = (HtmlElement) testPage.getElementById("testSpan");
  clickableSpan.click();
    
  // check if <a /> was correctly generated in JavaScript
  HtmlAnchor jsGeneratedLink = testPage.getAnchorByHref("/test/secondTestView");
  assertTrue("Anchor generated by JavaScript shouldn't be null", 
    jsGeneratedLink != null);
    
  // check the page opened by dynamically generated link
  HtmlPage secondTestPage = jsGeneratedLink.click();
  List<HtmlElement> welcomeSpan =  (List<HtmlElement>) secondTestPage.getByXPath("//span[text()='Welcome']");
  assertTrue("Welcome span can't be null", 
    welcomeSpan != null && welcomeSpan.size() > 0);
  assertTrue("Welcome span should be equal to 'Welcome' but is equal to '"+welcomeSpan.get(0).asText()+"'", 
    welcomeSpan.get(0).asText().equals("Welcome"));
}

Controller used to tests looks like:

@Controller
public class TestController {
    
  @RequestMapping(value = "/viewTest")
  public String testView(Model model) {
    model.addAttribute("contextVariable", "contextVariable");
    return "success";
  }
    
  @RequestMapping(value = "/secondTestView")
  public String testOtherView(Model model) {
    return "success2";
  }
}

Two generated views, success.vm and success2.vml, have following content:

# success.vm
<body>
  <h1>Hello world !</h1>
  <span id="testSpan">Test from ${contextVariable}</span>
  <script type="text/javascript">
  document.getElementById("testSpan").onclick = function() {
    var link = document.createElement('a');
    link.setAttribute('href', "/test/secondTestView");
    document.body.appendChild(link); 
  };
  </script>
</body>

# success2.vm
<span>Welcome</span>

This test case shows that HtmlUnit allows us to test more complex cases as native Spring test framework. We can not only check elements defined in the page at the moment of page loading, but also imitate user behaviour and launch dynamic actions in JavaScript. In additionally, we can, inside the same test case, navigate between several pages, exactly as final user do.

In this article we discovered the power of test cases written with HtmlUnit. Thanks to this "less-GUI browser" we can not only check elements on the page, but also imitate user actions as clicks, focuses or blurs (which are JavaScript events). The test writing is simplified with Spring Test MVC HtmlUnit project which provides, inter alia, mocked web connection object, needed by HtmlUnit. However, they remain one catch - JSP. As Spring test package, Test MVC HtmlUnit doesn't allow to test view part based on JSPs.


If you liked it, you should read:

📚 Newsletter Get new posts, recommended reading and other exclusive information every week. SPAM free - no 3rd party ads, only the information about waitingforcode!