Recently, the more commonly used method for developing software has been shifting from traditional models like the waterfall or the v model to a more iterative model: the agile software development.
Agile development processes are implemented using continuous integration tools. To keep individual iterations speedy, it’s absolutely essential to bring automated testing into the continuous integration process… and likewise into the agile development process.
Ranorex can easily be integrated into continuous integration systems as Ranorex Studio creates an executable from of its Test Suite projects that can be triggered by almost any CI system.
This blog post explains how to integrate Ranorex automation into your existing continuous integration process.
- How it should work
- A closer look at the CI System
- Blog posts concerning integration into specific systems
- Conclusion
How it should work
The principle behind CI approach is to frequently promote code changes and rapidly get feedback about the impact these changes have on the existing system.
So basically every time a developer or a tester commits changes to the version control system, the version control system should inform the CI system that there are changes to the system under test.
This information triggers the CI system to build the application under test as well as the Ranorex automation project if necessary and executes the Ranorex automation project.
To evaluate the outcome of the test, the CI system usually examines the return value of the executable or its output text (e.g. “TEST FAILED” for failure). With Ranorex the return value ’0′ signals the successful execution of the test script and the return value ‘-1′ signals a failure.
Each involved team member should get notified about a finished build. The notification should include build logs as well as reports of the executed automation.
A closer look at the CI System
Let’s have a closer look at the CI System.
Note: It is necessary to have the Ranorex main components – and a valid Ranorex license – installed on each machine on which you are going to build and execute Ranorex code.
The whole build process will be triggered by committing changes to the version control system. For this reason the source repository as well as a mechanism for checking for changes (e.g. a checking interval) in the repository has to be configured.
After checking out the committed code from your version control system three steps in your build process have to be added.
One for building the application under test, one for building the Ranorex automation project and one for executing the Ranorex automation project.
Build Application Under Test
The first step should generate an executable of you application under test which will later be triggered from the Ranorex Test Suite project.
For this reason add a build step which will build your AUT (e.g. MSBuild build step, Ant build step, …).
Build Ranorex Test Suite
The second step should generate the executable automating your application under test. To do so, add a build step (MSBuild or Visual Studio) and choose the project file (*.csproj) of your Ranorex project which should be built.
Execute Ranorex Test Suite
The third step should execute the just created executable. Therefore simply add an execution step triggering the *.exe file and define some command line arguments if needed.
Basically, the test execution can be triggered on the same system as the projects were built. If the execution should be triggered on another system, make sure to deploy the built executable and all necessary files to that system.
Also make sure that the executing process is able to execute the AUT and the Ranorex Test Suite in a desktop session and not in a console session (don’t start as service).
With frequently applying code changes, it does not make sense to run the whole test suite with all it’s test cases but a subset of test cases regarding the affected area. For this reason, you can use so called run configurations which can be added and edited directly in the test suite editor (as you can see in the section ‘Running a Test Suite’ in our user guide).
The following example will illustrate how a run configuration can be triggered using command line arguments:
Executing following command line will run the test suite executable called ‘TestCIProject’, run the run configuration called ‘SmokeTest’ and generate a zipped report file called ‘Report.rxzlog’ stored in a folder called ‘/Reports/’:
TestCIProject.exe /rc:SmokeTest /zr /zrf:Reports/Report.rxzlog
You can find a list of all available command line arguments in the section ‘Running Tests without Ranorex Studio‘ in our user guide.
Provide Feedback
After finishing all build and execution steps, the responsible developers and tester should be informed about the success of the builds as well as the outcome of the test execution. To do so, add a post build action which sends a mail having the build log of the build steps as well as the generated zipped report file attached.
Blog posts concerning integration into specific systems
Have a look at the following blog posts which explain how to integrate into some specific systems in detail:
- Bamboo CI with Ranorex Test Automation
- Integrating Ranorex Automation in TeamCity CI Process
- Integrating Ranorex Automation in Jenkins CI Process
- Running Ranorex Automated Tests with HP Quality Center
- Running Ranorex Automated Tests with Microsoft Test Manager
Conclusion
As you can see, it’s easy to integrate Ranorex Automation in any system, simply by triggering the system to execute the Ranorex executable.
Please feel free to share your thoughts on this topic in the comments section.