Integrating Ranorex Automation in Jenkins CI Process

In software engineering, continuous integration means the continuous application of quality control processes — small units of effort, applied frequently.

Hudson and Jenkins are open source tools written in Java for just such continuous integration.

This blog post will describe how to set up a CI job with Hudson/Jenkins that automatically builds and executes your Ranorex automation and automatically sends out the generated test reports for every committed change in a subversion repository.

• Advantages of Continuous Integration
• Infrastructure
  • Continuous Integration Tool
  • Source Code Management
• Plugins
  • Install Plugins
  • Configure Plugins
• Add New Job
  • Configure Source Code Management
  • Add Build Steps
  • Add Post-Build Action
• Run Job
• Add Repository Hook
• Conclusion

Advantages of Continuous Integration

Continuous integration has many advantages:

• When tests fail or bugs emerge, developers might revert the codebase to a bug-free state without wasting time for debugging
• Developers detect and fix integration problems continuously – avoiding last-minute chaos at release dates
• Early warning of broken/incompatible code
• Early warning of conflicting changes
• Immediate testing of all changes
• Constant availability of a “current” build for testing, demo, or release purposes
• Immediate feedback to developers on the quality, functionality, or system-wide impact of code they are writing
• Frequent code check-in pushes developers to create modular, less complex code

Infrastructure

Continuous Integration Tool

A download link and an installation description for Hudson and Jenkins can be found at the following links:

• http://jenkins-ci.org/
• http://eclipse.org/hudson/

In this blog post we are going to use Jenkins as CI tool, but using Hudson instead should not make much difference.

As Jenkins is normally started as a Windows service, it does not have sufficient rights to start UI-applications.

So, make sure that the Jenkins service is not started automatically as a service. To do so, open the “Services” tool (which is part of the “Administrative Tools” in the control panel), choose “Jenkins” service, stop the service, and set the “Startup type” to disabled:

Jenkins can be started manually using following command (from the installation folder):

java -jar jenkins.war

After starting Jenkins, you can access the web interface on the machine you have installed it on using the following address:

http://localhost:8080/

You can configure your Jenkins installation according to your wishes navigating through the menu (“Manage Jenkins” -> “Configure System”):

Note: It is necessary to have the Ranorex main components – and a valid Ranorex license – installed on each machine on which you are going to build and execute Ranorex code.

Source Code Management

As mentioned before, we are going to use a subversion repository as base of our continuous integration process.

In this sample we have two solutions placed in our repository: the application we are going to test and the testing application.

To start the application under test from your test project, simply add a new “Run Application” action to your recorder action table which starts the application under test using a relative path to the repository root:

Plugins

 
As we want to build our code for each committed change within our repository, we need a subversion as well as a MS Build plugin for Jenkins. To send a mail holding the test report after each build, we additionally need a mail plugin.

Install Plugins

To install the mentioned plugins, open the “Manage Plugins” section (“Manage Jenkins” -> “Configure System”), choose the following plugins from the list of available plugins and install them using the button on the bottom:

• MSBuild Plugin
• Email Extension Plugin
• Subversion Plugin (should be installed automatically)

Configure Plugins

Next to installing the plugins, they have to be configured. Therefore open the “Manage System” section as described above and

• configure your “MSBuild” plugin by choosing the msbuild.exe installed on your machine,

• configure the “Extended E-Mail Notification” plugin by setting the recipients and altering the subject and content (adding the environment variable $BUILD_LOG to the content will add the whole console output of the build and the test to the sent mail)

• and configure the “E-mail Notification” plugin by setting the SMTP server specific configuration.

Add New Job

Now, as the system is configured, we can add a new Jenkins job which will update the checked out files from a SVN repository, build both, the application under test and the Ranorex automation project, execute the application under test as well as the automation code and send a mail with the report file attached.

First of all, create a new Job, choose “Build free-style software project” as job type and enter a name:

Configure Source Code Management

Next, we have to check out the source of both the application under test and our test automation project. Therefore choose Subversion as source code management tool, enter the repository holding your application under test as well as your test automation project and choose “Use ‘svn update’ as much as possible” as check out strategy:

With this configuration the application under test as well as the test automation project will be checked out and updated locally on the machine running Jenkins.

Add Build Steps

Now, as the source code management is configured, we can start with processing the updated files.
First of all, let’s add MSBuild steps for both projects:

Choose your configured MSBuild version and enter the path of the solution file relative to the repository root (which is the workspace folder of the Jenkins job) for both the automated and the automating project:

With adding these two build steps, the executables will be built automatically on the machine running Jenkins.

Now the newly built program should be tested.

This can be accomplished by adding a new “Windows batch command” starting the test suite executable (relative to the repository root which is the workspace folder root of the Jenkins job):

As you can see, some command line arguments are passed to the test suite executable.

In this sample the command line arguments “/zr”, which triggers the test suite executable to generate a zipped report file and “/zrf:.\Reports\Report-Build-%BUILD_NUMBER%.rxzlog” which defines the name and the location of the generated zipped report file, are used.

You can find a list of all available command line arguments in the section “Running Tests without Ranorex Studio” in our user guide.

The test suite executable returns “0” on success and “-1” on a failure. Based on this return value, Jenkins will mark the build as succeeded or failed.

Add Post-Build Action

After building and executing the application under test and the regarding test script, we will send an email which informs us about the success of the triggered build.

This email should include the zipped report file, mentioned before, as attachment.

Therefore add a new post-build action of type “Editable Email Notification”, choose the Report file location defined before as attachment, and add triggers for every job status you want to be informed about. In this sample an email will be sent if a job has failed or succeeded.

Run Job

After preforming these steps and saving your changes you can try out if everything works as expected by choosing “Build now”:

After running the generated job you will see all finished builds within the build hierarchy with icons indicating the status of the individual builds.
You can view the zipped report files of all builds by opening them via the local workspace (“Workspace/Reports”):

As configured before, an email will be sent to the specified email address(es), including the console output in the email text as well as the generated zipped report file as attachment.

Add Repository Hook

After performing the steps mentioned above, we are able to manually trigger a build. But as we are working with subversion, it would be beneficial to trigger the script for each commit.

Therefore you can add a server side repository hook, which automatically triggers Jenkins to start a new build for each change committed as described in the subversion plugin documentation.

Alternatively you can activate polling of the source code management system as build trigger in your Jenkins job configuration.

As shown in following picture, you can define the interval, the source code management will be invoked (e.g. 5 minutes after every full hour):

Conclusion

Following the steps above you will be able to easily setup a continuous integration process performing the automated test of the application you develop.

After every commit a test automation is triggered that leads to automated feedback (success/failure) for the developer via mail with the Ranorex Report attached.

Please feel free to share your thoughts on this topic in the comments section.