So, how do I record BPM scripts for mobile monitoring

HP Business Process monitor leverage HP LoadRunner technology for monitoring.  HP LoadRunner is the industry-standard software for performance testing and HP BPM this technology.

Here is a great post how to use VuGen to record scripts for mobile.

How are mobile devices changing the way customers work?

A mobile device can be a smartphone, tablet, mini-tablet, phablet, or other small handheld computer.  Gartner says that by 2013 (yes, now!), there will be more mobile devices in use than traditional computers. This rise in the number of mobile devices is changing the way people use applications, and has the following implications:

•         Organizations must make their applications accessible to mobile users

•         Users expect a different experience when using their mobile device than with a PC:

•         The UI needs to be customized to fit the mobile device’s screen

•         The user expects fewer clicks to achieve their goal

•         Apps should respond quickly.  If they don’t, users are likely to abandon them.

•         For the best experience, organizations develop native apps to be downloaded onto the mobile device. This is in addition to the existing browser application, because infrequent users won’t want to download the dedicated app for occasional use. They’ll use their mobile device’s browser, but the application will need to respond differently when accessed by a mobile browser, to account for the different screen size etc.

What challenges do mobile devices present to app providers?

Here are some of the challenges that application providers need to address:

•         Both dedicated apps and browser-based applications need to support the plethora of mobile device types, models, platforms and platform versions

•         Mobile applications are typically updated pretty frequently - continuous delivery, anyone?

•         In order to provide mobile browser users with a mobile look and feel, applications are often redirected to a different server which serves up pages designed for mobile. Many organizations need to provide and support both native apps and browser-based applications.

•         Mobile devices use different networks (eg. 3G, 4G, LTE, etc.) with different bandwidths, and the application is expected to perform well on each of them. There are also network conditions that are common to mobile applications, such as latency, packet loss and dropped connections.

Yes, but how does that affect performance testing?

Let’s look at the ways performance testing for mobile applications differs from traditional performance testing

•         Mobile devices hold connections longer than regular devices. The implication is that there is a rise in the number of concurrent open connections. Your application, together with its backend servers and infrastructure, needs to cope with this rise to ensure that the connections aren’t exhausted and that other users aren’t denied service.

•         Data consumption from mobile devices is typically slower than regular services.  Applications need to ensure that this will not cause a drain on system resources.

•         Mobile devices communicate with cells, and as the device changes location, the cell it communicates with changes. This can result in a bandwidth change, and even complete drop-outs. This can affect the way the application behaves.

•         To record a script for performance testing, you need to be able to record the activity of users working on mobile devices.  But:

•         You can’t simulate it using a regular browser, because the network traffic sent from a mobile browser is different, both in the content of the UI that is served, and the HTTP headers.

•         Nor can you use a regular browser to simulate a native application hosted on the mobile device.

You can also check out this blog post from John Jeremiah which discusses how usage and deployment of mobile applications differ from that of traditional ones. He discusses how mobile applications raise new challenges, particularly around mobile performance testing.

I see.  So how do I record scripts for load testing then?

Performance testing software will take care of most of the hard work for you. HP LoadRunner is the industry-standard software for performance testing. It can test an application under the load of tens, hundreds or even thousands of potential users. HP LoadRunner load tests your application by emulating the environment where multiple users work concurrently. While the application is under load, LoadRunner accurately measures, monitors and analyzes a system’s performance and functionality.

How does LoadRunner work?

HP LoadRunner consists of three main components – Virtual User Generator (VuGen), Controller and Analysis.

The VuGen component is an Interactive Development Environment (IDE) that records and tunes real business processes that a user would perform in production. These can then be played back to emulate the real users. Using minimal hardware resources, the Controller component then emulates hundreds or thousands of concurrent virtual users to apply production workloads to almost any application platform or environment. HP LoadRunner stresses an application from end-to-end—applying consistent, measurable and repeatable loads—then uses the data to identify scalability issues that can affect real users in production.

As it drives load against the system, HP LoadRunner captures end-user response times for business processes and transactions.  It determines whether the application can meet the required service level agreements. Non-intrusive, real-time performance monitors obtain and display performance data from every tier, server and system component. It can also collect application-tier and code-level data from HP Diagnostics.

After the load test completes, the Analysis component provides a single view of end-user, system-level and code-level performance data. It includes a patented AutoCorrelation engine to scan end-user, system, and diagnostics data to provide the most likely causes of system slowdown. The data is correlated to quickly pinpoint problem areas and identify the root cause of performance bottlenecks. This helps your engineers quickly determine whether they have met their performance goals.  If they have not met these goals it helps them identify the reason and who owns the problem.

What support does LoadRunner offer for mobile applications?

HP LoadRunner 11.50 has two new protocols for helping to record mobile applications(*):

•         Mobile Application – HTTP/HTML, for recording scripts at the transport level for both browser-based mobile applications and native mobile applications, that communicate with their servers over HTTP

•         Mobile TruClient, for recording scripts for browser-based mobile applications through the browser-based user interface.  This protocol is based on Ajax TruClient, using a browser modified to emulate the mobile browser.

These protocols are independent of mobile operating systems, so will work on different versions of iOS, Android, Windows Mobile, WebOs (Palm), Blackberry, etc.

This table will help you choose the right protocol:

 

Application Type	Mobile TruClient	Mobile Application – HTTP/HTML
Browser-based app	Yes	Yes
Native app		Yes

 

 If your mobile app is browser-based, and supports Firefox(**), it’s a no-brainer. With the Mobile TruClient protocol, you select the type of mobile device you want the Firefox browser to emulate, and then perform the actions inside the browser just as if you were performing it on the mobile device.

There are some other considerations to take into account when using the Mobile Application-- HTTP/HTML protocol though.  Here’s another table which sums them up:

 

Scenario	If…	Then…
Proxy Recording	- The mobile device can be connected to the same network as the VuGen machine, and; - The app allows proxy configuration	You can configure your mobile app to use VuGen proxy(***) to record the traffic
Server-Side Recording	- You don’t want (or can’t) record from the actual device, and; - The device, server, and VuGen machine are in the same network, and; - You only need to record on one server	Install VuGen’s Mobile Sniffer Agent on the server.  You can then use the protocol’s ‘Record and Analyze Traffic’ feature, which will use the agent to record the traffic
Script from Network Capture	You already have a network capture file, such as WireShark’s ‘pcap’ file	Use the protocol’s ‘Analyze Traffic’ feature to turn it into a script
Device Emulator Recording	- You don’t want (or can’t) record from the actual device, and; - The mobile OS is Android, and; You have a device emulator available	Record from the emulator by using the protocol’s ‘Emulator Recording’ feature
On-Device Recording	- Your Android device is rooted (to allow access to the device’s system)	You can install the LoadRunner Mobile Recorder on the device (***), and use it to record the traffic as a standard ‘pcap’ file. You can then open the file on your VuGen machine to create a script.

Where do the network capture files come from, in the ‘Script from Network Capture’ scenario?

Recording application traffic to a capture file is effective when you are unable to record an application using VuGen, as is the case with mobile applications. A capture file is a trace file containing a log of all TCP traffic over the network (although you should note that only HTTP traffic will be included in the script generated from the ‘Analyze Traffic’ option). Using a sniffer application, such as WireShark, you can obtain a dump of all of the network traffic. The sniffer captures all of the events on the network and saves them to a capture file. To generate a smaller, more manageable script, you should try to capture the network traffic only for the time that you perform actions in your application.  To further reduce the script’s complexity, you can configure the ‘Analyze Traffic’ option to filter the data by the client’s or server’s IP.

Depending on your operating system and device, you have many options on where and how to record a capture file. See the ‘Recording Traffic into a Capture (Sniffer) File’ section of the LoadRunner documentation for more details.

Right.  You mentioned network speeds?

Different cellular networks have different speed settings, which can have an effect on the application. You should make sure the app can cope with a variety of networks. LoadRunner lets you simulate different network bandwidths for a large number of standard networks. If that’s not enough, you can configure your own values if you’re expecting unusual speeds from network providers in specific geographies.

You should think about how different groups of users will be affected by their networks, and configure your scenarios accordingly in LoadRunner’s Controller.

And what about things that can go wrong with the network?

You mean packet loss, latency, and things like that?  Yes, there are lots of network issues. You should know that delays in a mobile application will cause users to abandon the application.  The longer the delay, the more users you’re going to lose—and some of these users are never going to use your application again. Some of these delays might be a result of the application itself, but delays are also a ‘feature’ of the communications infrastructure that the mobile device itself is using.  Here are some examples:

•         Limited bandwidth – mobile networks typically have a smaller bandwidth than wireless networks, which means it takes data longer to get through

•         Network latency – the mobile device needs to communicate with a cellular tower, which in turn needs to send the data off to the server, and the response comes back through the tower to the mobile device.  All this takes time.

•         Packet loss – data packets being transmitted between any two points on the network can get lost, or corrupted. Although the network generally corrects these problems by itself, it will cause an additional delay

In order to accurately simulate your application’s behavior under these circumstances, LoadRunner is integrated withShunra’s Network Virtualization (NV), which provides the following capabilities:

•         Native integration with HP LoadRunner and HP Performance Center—because of the tight integration, no additional scripting requirements are needed

•         Robust network virtualization—each load generator can emulate a different location with a unique set of network conditions. This enables the simulation of multiple user populations, and more accurately reproduces actual conditions that can affect the end-user experience

•         Location-aware analytics—results are integrated with the load test results, which gives location-specific performance information, identification of poorly performing transactions, and root causes of performance issues

•         Automatic optimization recommendations—this is Shunra’s proprietary performance scorecard. It grades application performance and offers suggestions to improve performance.

For more details on Shunra NV, check out the datasheets and white paper from http://www.shunra.com/products/shunra-nv-hp-software.

 

Is there anything I should know about functional testing of mobile apps?

That’s a great question. Although we’ve been talking about performance testing, we should really throw in a few words about functional testing.

Your performance testing should be based around real scenarios that users are likely to perform. You need to ensure that your application behaves correctly in when it’s being used by an end-user. This is what we mean by functional testing.

Mobile applications are expected to meet the same quality standards as desktop applications and desktop browser-based applications—whether they are native applications or mobile browser-based applications. This is why functional testing of the mobile application should be integrated into your application lifecycle management infrastructure.

You’ve probably heard of HP Unified Functional Testing (UFT), HP’s flagship functional testing product. HP UFT Mobile extends UFT to enable full end-to-end testing of mobile applications on real devices. UFT Mobile lets you develop and run automated tests on multiple devices. It provides a dedicated cloud-based environment of mobile devices to support your testing needs, and makes them available to you wherever you are.

So together with HP LoadRunner and HP Performance Center, HP UFT and HP UFT Mobile can introduce real device application side testing into your mobile load and performance tests. This full, end-to-end testing allows you to create very realistic test scenarios that will closely match and predict what the end user will actually experience. Check out HP’s mobile testing suite here.

Summary

We’ve seen how to record and prepare scripts for representing real users on the mobile devices that your application will run on.  Once you’ve decided which network conditions that you want to emulate, build your load testing scenario in the same way as you’d build it for traditional load testing, and you continue to have the full power of LoadRunner at your fingertips.

When you run the load test, you can monitor and measure the metrics you’d normally measure in a load test. For example these metrics include: server load and utilization, database and backend server performance, etc.  And you can see exactly how your emulated network behaves by looking at the relevant network emulation monitors. When the load test has finished running, you have access to all of the data collected during the execution of the load test. You can correlate data, analyze how specific virtual users, or scripts, affected system performance, and how network conditions impact the behavior of the system, and ultimately, the end-user’s experience.

This should help you answer some of the questions that mobile applications raise:

•         With so many users using so many different devices and versions of operating systems, how can I validate the performance of my application?

•         How can I be sure my customers can reach the application’s servers?

•         What’s the end-to-end user experience like?

•         How does the addition of a mobile interface affect my existing users?

You’ll find a fully BSM - BPM functional trial version at http://goo.gl/0UBBtP

Here is link to the LR blog http://h30499.www3.hp.com/t5/HP-LoadRunner-and-Performance/Who-s-up-for-some-mobile-performance-testing/ba-p/5974699#.UmVK1lARCSp