Supercontexts

Q7, 11.06.2013 by Ivan Inozemtsev View Comments

Introduction

A test case in Q7 consists of two parts:

  • list of contexts, defining an application state
  • ECL script describing UI actions and validating that application behavior in given state is correct.

In some cases it might be required to execute same actions with just a few variances in initial state or arguments of script commands. Common examples are:

  • Test case verifies that a project can be built. An executed action is just a verification that Problems view is empty, but this should be done for several different projects. We can say that we have the same behavior for different states of workspace.
  • Test case verifies some form (like a validation of new Java Class dialog). We execute basically the same actions (type values in fields and check error message), but with different command arguments.

In Q7 1.3 we have added supercontexts which can be used to cover both of scenarios above. A supercontext is a list of contexts of the same type, which can be refered in test cases just as usual contexts, but during an execution, a test case will be executed several times with each of these contexts. In this blog post we are going to describe how to create test cases for two examples mentioned above.

First example – building Java projects

We are going to create a test case which verifies that Problems view contains no errors for the following states of a workspace:

  • empty workspace
  • empty Java project
  • “Hello, World!” Java project
  • two referencing Java projects

We start with a creation of a usual test case which verifies that Problems view contains no errors.

no-build-errors-test

Next, we create three more workspace contexts:

  • Empty Java Project
  • HelloWorld java Project
  • Two Java Projects

Then we create a new workspace supercontext (by selecting New -> Super Context) in a context menu in Q7 Explorer and add all our four workspace contexts into it:

new-super-context

Now we can get back to originally created test case and replace Emtpy Workspace context with Workspaces supercontext. If we execute this test case, an Execution view shows it has been executed four times, each time with different workspace context:

execution-view-workspaces

Second example – validating New Java Project dialog

For the next test case we are going to reuse Java Perspective workbench context and HelloWorld Java Project workspace context.

We record a filling values in New Java project and validating an error message:

get-view "Package Explorer" | get-tree | select helloWorld 
  | get-menu "New/Class" | click
with [get-window "New Java Class"] {
  get-editbox -after [get-label "Package:"] | set-text "org.example"
  get-editbox -after [get-label "Name:"] | set-text NewClass
  get-editbox -after [get-label "Java Class"] | get-property text 
    | equals "Create a new Java class." | verify-true
}

Now we need to parameterize it – at first we create Parameters context “Valid Input” with parameters we are going to change:

param-context

At second we replace string values in script with get-param paramName command and adding parameters context to a test case:

parameterized-test

And the rest is straightforward – create parameters context for each set of input values, put them into parameters supercontext and put this supercontext into testcase.

Once test case is executed, we can see that all inputs were iterated:

execution-view-params

Reference

A sample Q7 projects with test cases descibed above can be downloaded from Github – https://github.com/xored/q7.examples.supercontexts. To run these tests locally using Maven type the following in your console:

$ git clone git://github.com/xored/q7.examples.supercontexts.git
$ cd q7.examples.supercontexts
$ mvn clean install

More about supercontexts

If a test case refers to more that one supercontext, then all possible combinations of tuples from these supercontexts will be used. For instance, for our example above if we would like to check that all these Java projects can be built both with Java 1.6 and Java 1.5 compiler settings, we could create Preferences supercontext with two contexts 1.6 and 1.5, add it to a test case, so that it will be executed 8 times and Execution view would show a list like this:

  • No Build Errors (Empty Workspace, 1.5)
  • No Build Errors (Empty Workspace, 1.6)
  • No Build Errors (Empty Java Project, 1.5)
  • No Build Errors (Empty Java Project, 1.6)
  • No Build Errors (HelloWorld Java Project, 1.5)
  • No Build Errors (HelloWorld Java Project, 1.6)
  • No Build Errors (Two Java Projects, 1.5)
  • No Build Errors (Two Java Projects, 1.6)

In case we need to iterate over context pairs without this ‘cartesian product’ (like, workspace context and parameters context specific for this workspace), this can be done by creating a single group supercontext, referencing to several group contexts. For instance, if we create a test case which validates a build error count, then we can have a structure like this:

  • build errors (group supercontext)
    • one build error (group context)
      • workspace1 (workspace context)
      • params1 (params context)
    • two build errors (group context)
      • workspace2 (workspace context)
      • params2 (params context)

And a test case referencing this group supercontext will be executed twice:

  • Build Errors (one build error)
  • Build Errors (two build errors)

Conclusion

Supercontexts are powerful but might be hard for inexperienced users. In case when there are only a few variances of a test case which are unlikely to be changed, it might be much more easier and faster to create few separate test cases instead of dealing with parameterization.

blog comments powered by Disqus

Functional and UI Testing for Eclipse-based Applications

Test automation solution, handcrafted for Eclipse. Modern and easy to learn IDE.

Product Health Analytics and Support Automation Platform

End-user and product analytics solution that gives you rich insights into your application health & performance.

Eclipse-based IDE for the Fantom Programming Language

Full-featured Integrated Development Environment for emerging Fantom programming language.