How to Combine Ranorex and NeoLoad Tests

Let’s be honest: We rarely test the product functionality under load. But how can we be sure our end product works when our customers are using it? As we’ve described in our previous blog post “Combining Automated Functional and Load Testing”, it often makes sense to combine functional and non-functional tests. A functional test, which works fine in idle conditions, might fail when the back-end server is under load. Just like simply stressing a back-end system may not reveal functional issues, which can only be found by an automated functional test. If we want to find those errors that only occur under load, we have to combine automated functional tests and automated load tests.We’re happy to announce that you can now combine Ranorex and NeoLoad tests!

In this blog, we want to show you how you can set up the Ranorex-NeoLoad integration and what you can do with it. But first, let’s quickly cover the basics:

What is NeoLoad?

NeoLoad is an automated load and performance testing tool from Neotys.

NeoLoad offers a full-fledged REST API to either remote control the execution of a NeoLoad test or transmit timing values to NeoLoad. To enable integration with Ranorex, the REST API calls are wrapped with Ranorex functions and packaged into a NuGet package for easy deployment.

What do I need to enable the Ranorex-NeoLoad integration?

• Ranorex license – alternatively, you can download a free 30-day trial
• NeoLoad Professional license – alternatively, you can also use the free license

Now that you’re all set, we want to show you in detail how you can:

1. Set up the Ranorex-NeoLoad integration
2. Use the load testing modules available with the integration
3. Transmit navigation timing to a NeoLoad test
4. Update meta-information in cross-browser tests
5. Upgrade an existing Ranorex project with the Ranorex-NeoLoad NuGet package

Setting up the Ranorex – NeoLoad integration

First, we need to set up the integration:

Add the NuGet package to the Ranorex project

• Right-click on “References” in the Ranorex project view
• Select “Manage Packages…”
• Search for “Ranorex” and add the “Ranorex-NeoLoad integration” package

This will automatically add the necessary libraries to the Ranorex project. The following code modules will now appear in the module browser: 

You can now use the modules, which are included in the NuGet package, freely within the Ranorex test automation project.

Modules included in the Ranorex-NeoLoad NuGet package

The following modules, and their individual variables, are included in the Ranorex-NeoLoad NuGet package:

• NL_ConnectToDesignAPI
• NL_CreateProject
• NL_OpenProject
• NL_CloseProject
• NL_SaveProject
• NL_StartRecording
• NL_StartRecording_Extended
• NL_StopRecording
• NL_StopRecording_Extended
• NL_ConnectToRuntimeAPI
• NL_StartTest
• NL_StopTest
• NL_AddVirtualUsers
• NL_StopVirtualUsers
• NL_ConnectToDataExchangeAPI

For a full complete documentation of each module please refer to NeoLoad module definitions.

Transmit navigation timing data from any browser to NeoLoad

Opening a website is related to a certain latency. This latency depends on various factors, such as the network connection or the browser used. It can be measured with the “Navigation Timing” API, which is offered by all browsers. If you evaluate these timing values, especially when the website is under load, you can localize potential bottlenecks. Eliminating the identified bottlenecks will ultimately improve the user experience.

 The NuGet package offers a mechanism to calculate these timing values and transmit the results to NeoLoad. You can find a more detailed description of the navigation timing here. The timing values are calculated by the Ranorex/NeoLoad Nuget package:

Highlighted in green, you can see the timing values that are calculated by Ranorex and submitted to NeoLoad.

To transmit the timing values, you need to add a new user code action  into the action table in Ranorex Studio. Once the NuGet package is added to the Ranorex project, the additional user code action “SendTimingValues” will appear in your projects user code library.

The “SendNeoLoadTimingValues” action accepts the dom node from you repository and a “transaction name” as an argument. We recommend using the current page as a transaction name in the Ranorex action table. As soon as NeoLoad receives the timing values of this transaction, a tree with the root node containing the Ranorex test suite is automatically created. Another subfolder is automatically created for the respective transaction name. This folder contains the timing values transmitted from Ranorex.

Important:  Please make sure to initialize the module “ConnectToDataExchangeApi” before you use the module “SendNeoLoadTimingValues”. Otherwise, an error is thrown. 

You can drag the data series into the graph board in NeoLoad to visualize it. If you’ve provided meta-information, such as “Hardware”, “Software” or “Location” in the “ConnectToDataExchangeApi”, you can now use this information to filter timing values transmitted from Ranorex.

Update meta-information in cross-browser tests

If you execute the test in multiple browsers, you have to update the filter options in NeoLoad by calling the “ConnectToDataExchangeApi” module again. To do so, bind the data column, which specifies the browsers, with the “Software” argument from the “ConnectToDataExchangeApi” module. You can now compare timing values from different browsers.

Exemplary Ranorex project structure

In the screenshot below you can see an example of how you can use the modules provided in the NuGet package within a Ranorex test project:

As you can see, a connection to the runtime API is established in the global setup section. The login information take the form of global parameters. At the very beginning, “StartNeoLoadTest” starts the NeoLoad test scenario. The following test case is data driven and provides the number of virtual users that will be added to the test. These values are provided in “AddVirtualUsers”. The inner loop is a cross-browser loop. It defines the browsers in which the test will be executed.

Please note: The module “ConnectToDataExchangeApi” can be called multiple times to update the current browser with the filter feature in NeoLoad.

Upgrade an existing Ranorex project with the Ranorex/NeoLoad NuGet package

If you add the NuGet package to an existing Ranorex project, which already contains a Ranorex Object Repository with repository elements, the modules provided by the NuGet package are automatically available in the module browser. In this case, the “SendNeoLoadTimingValues” option won’t be available in the “Dynamic Actions” list for the already existing repository items. Perform the following steps to enable this option: 

1. Open the RanoreXPath editor

2. Switch to “Browser & Results”

3. Drag and drop the root element from the Ranorex Spy to the matching root element in the Ranorex Object Repository.

Now, the “SendNeoLoadTimingValues” will be available in the Dynamic Actions list for the repository root element that describes the website DOM.

Conclusion

In this blog, you’ve learned how you can combine Ranorex and NeoLoad tests. You’ve seen the modules and variables that are available with this integration and how you can transmit timing values to a NeoLoad performance tests. Now, you will be able to validate critical business cases under load conditions to ensure system stability under real usage conditions and identify potential bottle-necks across technology borders.

Further Resources

• Blog Combining Automated Functional and Load Testing: Read this blog to learn more about the benefits of combining functional and load testing.
• Webinar Combining Automated Functional and Load Testing Webinar: Watch this webinar for free and learn why and how to combine automated functional and load tests.
• Blog Ranorex-NeoLoad Webinar – Questions Answered: Get answers to questions asked during the Ranorex-NeoLoad webinar to fully understand how to set up the Ranorex-NeoLoad integration.

Watch the Ranorex-NeoLoad Webinar