HP Business Process monitor leverage HP LoadRunner
technology for monitoring. HP LoadRunner
is the industry-standard software for performance testing and HP BPM this
technology.
Here is a great post how to use VuGen to record scripts for
mobile.
How are mobile devices changing the way customers work?
A mobile device can be a smartphone, tablet, mini-tablet,
phablet, or other small handheld computer.
Gartner says that by 2013 (yes, now!), there will be more mobile devices
in use than traditional computers. This rise in the number of mobile devices is
changing the way people use applications, and has the following implications:
•
Organizations must make their applications accessible to mobile users
• Users expect
a different experience when using their mobile device than with a PC:
• The UI needs
to be customized to fit the mobile device’s screen
• The user
expects fewer clicks to achieve their goal
• Apps should
respond quickly. If they don’t, users
are likely to abandon them.
• For the best
experience, organizations develop native apps to be downloaded onto the mobile
device. This is in addition to the existing browser application, because
infrequent users won’t want to download the dedicated app for occasional use.
They’ll use their mobile device’s browser, but the application will need to
respond differently when accessed by a mobile browser, to account for the
different screen size etc.
What challenges do mobile devices present to app providers?
Here are some of the challenges that application providers
need to address:
• Both
dedicated apps and browser-based applications need to support the plethora of
mobile device types, models, platforms and platform versions
• Mobile
applications are typically updated pretty frequently - continuous delivery,
anyone?
• In order to
provide mobile browser users with a mobile look and feel, applications are
often redirected to a different server which serves up pages designed for
mobile. Many organizations need to provide and support both native apps and
browser-based applications.
• Mobile
devices use different networks (eg. 3G, 4G, LTE, etc.) with different
bandwidths, and the application is expected to perform well on each of them.
There are also network conditions that are common to mobile applications, such
as latency, packet loss and dropped connections.
Yes, but how does that affect performance testing?
Let’s look at the ways performance testing for mobile
applications differs from traditional performance testing
• Mobile
devices hold connections longer than regular devices. The implication is that
there is a rise in the number of concurrent open connections. Your application,
together with its backend servers and infrastructure, needs to cope with this
rise to ensure that the connections aren’t exhausted and that other users
aren’t denied service.
• Data
consumption from mobile devices is typically slower than regular services. Applications need to ensure that this will
not cause a drain on system resources.
• Mobile devices communicate with cells, and
as the device changes location, the cell it communicates with changes. This can
result in a bandwidth change, and even complete drop-outs. This can affect the
way the application behaves.
• To record a
script for performance testing, you need to be able to record the activity of
users working on mobile devices. But:
• You can’t
simulate it using a regular browser, because the network traffic sent from a
mobile browser is different, both in the content of the UI that is served, and
the HTTP headers.
• Nor can you
use a regular browser to simulate a native application hosted on the mobile
device.
You can also check out this blog post from John Jeremiah
which discusses how usage and deployment of mobile applications differ from
that of traditional ones. He discusses how mobile applications raise new
challenges, particularly around mobile performance testing.
I see. So how do I
record scripts for load testing then?
Performance testing software will take care of most of the
hard work for you. HP LoadRunner is the industry-standard software for
performance testing. It can test an application under the load of tens,
hundreds or even thousands of potential users. HP LoadRunner load tests your application
by emulating the environment where multiple users work concurrently. While the
application is under load, LoadRunner accurately measures, monitors and
analyzes a system’s performance and functionality.
How does LoadRunner work?
HP LoadRunner consists of three main components – Virtual
User Generator (VuGen), Controller and Analysis.
The VuGen component is an Interactive Development
Environment (IDE) that records and tunes real business processes that a user
would perform in production. These can then be played back to emulate the real
users. Using minimal hardware resources, the Controller component then emulates
hundreds or thousands of concurrent virtual users to apply production workloads
to almost any application platform or environment. HP LoadRunner stresses an
application from end-to-end—applying consistent, measurable and repeatable
loads—then uses the data to identify scalability issues that can affect real
users in production.
As it drives load against the system, HP LoadRunner captures
end-user response times for business processes and transactions. It determines whether the application can
meet the required service level agreements. Non-intrusive, real-time
performance monitors obtain and display performance data from every tier,
server and system component. It can also collect application-tier and
code-level data from HP Diagnostics.
After the load test completes, the Analysis component
provides a single view of end-user, system-level and code-level performance
data. It includes a patented AutoCorrelation engine to scan end-user, system,
and diagnostics data to provide the most likely causes of system slowdown. The
data is correlated to quickly pinpoint problem areas and identify the root
cause of performance bottlenecks. This helps your engineers quickly determine
whether they have met their performance goals.
If they have not met these goals it helps them identify the reason and
who owns the problem.
What support does LoadRunner offer for mobile applications?
HP LoadRunner 11.50 has two new protocols for helping to
record mobile applications(*):
• Mobile
Application – HTTP/HTML, for recording scripts at the transport level for both
browser-based mobile applications and native mobile applications, that
communicate with their servers over HTTP
• Mobile
TruClient, for recording scripts for browser-based mobile applications through
the browser-based user interface. This
protocol is based on Ajax TruClient, using a browser modified to emulate the
mobile browser.
These protocols are independent of mobile operating systems,
so will work on different versions of iOS, Android, Windows Mobile, WebOs
(Palm), Blackberry, etc.
This table will help you choose the right protocol:
Application Type
|
Mobile TruClient
|
Mobile Application – HTTP/HTML
|
Browser-based app
|
Yes
|
Yes
|
Native app
|
Yes
|
If your mobile app is
browser-based, and supports Firefox(**), it’s a no-brainer. With the Mobile
TruClient protocol, you select the type of mobile device you want the Firefox
browser to emulate, and then perform the actions inside the browser just as if
you were performing it on the mobile device.
There are some other considerations to take into account
when using the Mobile Application-- HTTP/HTML protocol though. Here’s another table which sums them up:
Scenario
|
If…
|
Then…
|
Proxy Recording
|
- The mobile device can be connected to the same network as the VuGen machine, and;
- The app allows proxy configuration
|
You can configure your mobile app to use VuGen proxy(***) to record the traffic
|
Server-Side Recording
|
- You don’t want (or can’t) record from the actual device, and;
- The device, server, and VuGen machine are in the same network, and;
- You only need to record on one server
|
Install VuGen’s Mobile Sniffer Agent on the server. You can then use the protocol’s ‘Record and Analyze Traffic’ feature, which will use the agent to record the traffic
|
Script from Network Capture
|
You already have a network capture file, such as WireShark’s ‘pcap’ file
|
Use the protocol’s ‘Analyze Traffic’ feature to turn it into a script
|
Device Emulator Recording
|
- You don’t want (or can’t) record from the actual device, and;
- The mobile OS is Android, and;
You have a device emulator available
|
Record from the emulator by using the protocol’s ‘Emulator Recording’ feature
|
On-Device Recording
|
- Your Android device is rooted (to allow access to the device’s system)
|
You can install the LoadRunner Mobile Recorder on the device (***), and use it to record the traffic as a standard ‘pcap’ file. You can then open the file on your VuGen machine to create a script.
|
Where do the network capture files come from, in the ‘Script
from Network Capture’ scenario?
Recording application traffic to a capture file is effective
when you are unable to record an application using VuGen, as is the case with
mobile applications. A capture file is a trace file containing a log of all TCP
traffic over the network (although you should note that only HTTP traffic will
be included in the script generated from the ‘Analyze Traffic’ option). Using a
sniffer application, such as WireShark, you can obtain a dump of all of the
network traffic. The sniffer captures all of the events on the network and
saves them to a capture file. To generate a smaller, more manageable script,
you should try to capture the network traffic only for the time that you
perform actions in your application. To
further reduce the script’s complexity, you can configure the ‘Analyze Traffic’
option to filter the data by the client’s or server’s IP.
Depending on your operating system and device, you have many
options on where and how to record a capture file. See the ‘Recording Traffic
into a Capture (Sniffer) File’ section of the LoadRunner documentation for more
details.
Right. You mentioned
network speeds?
Different cellular networks have different speed settings,
which can have an effect on the application. You should make sure the app can
cope with a variety of networks. LoadRunner lets you simulate different network
bandwidths for a large number of standard networks. If that’s not enough, you
can configure your own values if you’re expecting unusual speeds from network
providers in specific geographies.
You should think about how different groups of users will be
affected by their networks, and configure your scenarios accordingly in
LoadRunner’s Controller.
And what about things that can go wrong with the network?
You mean packet loss, latency, and things like that? Yes, there are lots of network issues. You
should know that delays in a mobile application will cause users to abandon the
application. The longer the delay, the
more users you’re going to lose—and some of these users are never going to use
your application again. Some of these delays might be a result of the
application itself, but delays are also a ‘feature’ of the communications
infrastructure that the mobile device itself is using. Here are some examples:
• Limited
bandwidth – mobile networks typically have a smaller bandwidth than wireless
networks, which means it takes data longer to get through
• Network
latency – the mobile device needs to communicate with a cellular tower, which
in turn needs to send the data off to the server, and the response comes back
through the tower to the mobile device.
All this takes time.
• Packet loss
– data packets being transmitted between any two points on the network can get
lost, or corrupted. Although the network generally corrects these problems by
itself, it will cause an additional delay
In order to accurately simulate your application’s behavior
under these circumstances, LoadRunner is integrated withShunra’s Network
Virtualization (NV), which provides the following capabilities:
• Native
integration with HP LoadRunner and HP Performance Center—because of the tight
integration, no additional scripting requirements are needed
• Robust
network virtualization—each load generator can emulate a different location
with a unique set of network conditions. This enables the simulation of
multiple user populations, and more accurately reproduces actual conditions
that can affect the end-user experience
•
Location-aware analytics—results are integrated with the load test
results, which gives location-specific performance information, identification
of poorly performing transactions, and root causes of performance issues
• Automatic
optimization recommendations—this is Shunra’s proprietary performance
scorecard. It grades application performance and offers suggestions to improve
performance.
For more details on Shunra NV, check out the datasheets and
white paper from http://www.shunra.com/products/shunra-nv-hp-software.
Is there anything I should know about functional testing of
mobile apps?
That’s a great question. Although we’ve been talking about
performance testing, we should really throw in a few words about functional
testing.
Your performance testing should be based around real
scenarios that users are likely to perform. You need to ensure that your
application behaves correctly in when it’s being used by an end-user. This is
what we mean by functional testing.
Mobile applications are expected to meet the same quality
standards as desktop applications and desktop browser-based
applications—whether they are native applications or mobile browser-based
applications. This is why functional testing of the mobile application should be
integrated into your application lifecycle management infrastructure.
You’ve probably heard of HP Unified Functional Testing
(UFT), HP’s flagship functional testing product. HP UFT Mobile extends UFT to
enable full end-to-end testing of mobile applications on real devices. UFT
Mobile lets you develop and run automated tests on multiple devices. It
provides a dedicated cloud-based environment of mobile devices to support your
testing needs, and makes them available to you wherever you are.
So together with HP LoadRunner and HP Performance Center, HP
UFT and HP UFT Mobile can introduce real device application side testing into
your mobile load and performance tests. This full, end-to-end testing allows
you to create very realistic test scenarios that will closely match and predict
what the end user will actually experience. Check out HP’s mobile testing suite
here.
Summary
We’ve seen how to record and prepare scripts for
representing real users on the mobile devices that your application will run
on. Once you’ve decided which network
conditions that you want to emulate, build your load testing scenario in the
same way as you’d build it for traditional load testing, and you continue to
have the full power of LoadRunner at your fingertips.
When you run the load test, you can monitor and measure the
metrics you’d normally measure in a load test. For example these metrics
include: server load and utilization, database and backend server performance,
etc. And you can see exactly how your
emulated network behaves by looking at the relevant network emulation monitors.
When the load test has finished running, you have access to all of the data
collected during the execution of the load test. You can correlate data,
analyze how specific virtual users, or scripts, affected system performance,
and how network conditions impact the behavior of the system, and ultimately,
the end-user’s experience.
This should help you answer some of the questions that
mobile applications raise:
• With so many
users using so many different devices and versions of operating systems, how
can I validate the performance of my application?
• How can I be
sure my customers can reach the application’s servers?
• What’s the
end-to-end user experience like?
• How does the addition of a mobile interface
affect my existing users?
You’ll find a fully BSM - BPM functional trial version
at http://goo.gl/0UBBtP
Here is link to the LR blog http://h30499.www3.hp.com/t5/HP-LoadRunner-and-Performance/Who-s-up-for-some-mobile-performance-testing/ba-p/5974699#.UmVK1lARCSp
