We talked
about emulators and simulators and how can they help us to see how our
websites will be rendered on real devices. These tools are very useful and
provide a simple, fast, and fairly accurate testing solution. If it
doesn’t work in the emulator, it probably will not work on the real
device, and if it works in the emulator, it probably will work on the real
device (probably, again probably!).There are some problems with this testing approach, though. For one
thing, there are hundreds of differences between real devices, and
hundreds of bugs. Furthermore, there are several platforms without
emulation. That is why real device testing is mandatory.
But how can we get access to multiple real devices? Here are a few
suggestions:
Acquire as many friends as you can (with different devices, if
possible).
Buy or rent devices. Some vendors offer promotions for buying or
renting devices for developers and their partners.
Use a testing house company. This is an expensive solution and
not recommended for mobile web developers; we need to be as close as
possible to the devices.
Note:
Mob4Hire (http://www.mob4hire.com)
is a mobile social network aimed at joining testers with mobile devices
around the world and developers who want to test applications or
websites using a payment service. You can search for testers by country,
operator, and device to access the devices you want.
The last item of the previous list sounds good: a remote lab. What
is this?
1. Remote Labs
“Any sufficiently advanced technology is indistinguishable
from magic,” said sci-fi writer Arthur C. Clarke in 1961. When I
demonstrate some of these remote labs in my classes, I see a lot of
astonished faces.
A remote lab is a web service that allows us to use a real device
remotely without being physically in the same place. It is a simple but
very powerful solution that gives us access to thousands of real
devices, connected to real networks all over the world, with a single
click. You can think of it as a remote desktop for mobile phones.
There are three kinds of remote lab solutions for mobile
devices:
Software-based solutions, using a resident application on the
device that captures the screen, sends it to the server, and
emulates keyboard input or touches on the screen.
Hardware-based solutions, using some technology (magic, I
believe) to connect the server to the hardware components of the
device (screen, touchscreen, keypad, lights, audio, etc.).
Mixed solutions, having some hardware connection, some
software additions, and maybe a video camera for screen
recording.
Note:
As these are real devices, only one user can make use of them at
any given time. As such, the devices are a limited resource.
Let’s take a look at some of the remote lab solutions currently on
the market.
1.1. Remote Device Access
Forum Nokia offers a free (yes, free!) remote lab
solution for Symbian and Maemo devices called Remote Device Access
(RDA), shown in Figures Figure 1 and Figure 2. To use the service, you can access
http://www.mobilexweb.com/go/rda (you’ll need to
have already created a Forum Nokia account). You will need Java
Runtime 5.0 or newer, because RDA is a WebStart Java
application.
At present, usage is limited to eight hours per day. The main
features are:
Complete usage of the device
3G and WiFi connection support
Application installation
Device rebooting
Changing screen orientation
Browser and widget WRT support
Reservation of devices for future usage
Usage of devices with SIM cards connected in Europe
Saving screenshot images
Incoming calls and SMS available
At the time of this writing, there are more than 50 devices
available. There is no audio or accelerometer support, and depending
on your network bandwidth you can select the video quality you
want.
Warning:
There is a bug in the display recording mechanism in RDA that
doesn’t show the cursor arrow over non-touch devices. This makes web
browsing difficult. You can include the meta tag to apply focus navigation to solve this
problem.
1.2. Samsung Lab.Dev
Samsung also offers a free remote lab web service, using
the same solution provider as Nokia’s RDA, called Lab.Dev. It includes
some Windows Mobile devices. Depending on your membership with Samsung
(free or partner) you will have access to more duplicated devices for
testing purposes.
Note:
Using some commercial products (http://www.projectaphone.com) and even using a simple
webcam and following some instructions (http://www.gotomobile.com/archives/diy-gotomobiles-mobile-cam),
you can create a small testing lab with real devices that can take
screenshots and save videos of a real mobile screen. There are also
some software-based solutions.
Lab.Dev has the same features as RDA, so you can test web
applications and widgets using this solution. The devices don’t have
SIM cards, though, so you can only test WiFi connections (not
3G).
You can access this remote lab from http://www.mobilexweb.com/go/labdev.
1.3. DeviceAnywhere
DeviceAnywhere is the leader and pioneer in remote lab
solutions for mobile testing. It offers a hardware solution that
allows any device (low-end, mid-end, or smartphone, from any vendor)
to plug into the architecture.
Note:
DeviceAnywhere is the selected provider for this book’s
testing suites, and for some of the screenshots taken.
This is a commercial solution, with different price models
depending on the package. DeviceAnywhere Test Center offers more than
2000 devices (iPhone, Android, Nokia, Motorola, Sony Ericsson,
Samsung, BlackBerry, LG, Sanyo, Sharp, HTC, and more) connected to 50
live networks all over the world.
You can apply for a free trial at http://www.deviceanywhere.com. The IDE (Device Anywhere
Studio) is a Java application, so it should work on any OS.
Easy-to-install packages are available for Windows and Mac OS X, as
shown in Figure 3.
The company offers a special plan prepared for mobile web
testing.
The solution includes:
Access to the registered packages and devices from DA
Studio
Access to all hardware features (lock/unlock, close and
open, change orientation, power off and on)
Ability to place calls, send and receive SMS messages,
access the carrier’s portal, and buy premium content (as the
devices are on live networks)
Pixel-based perfect image rendering, so you can save
screenshots and videos of your testing for offline review (audio
is also supported as an optional feature)
Ability to manage multiple devices at the same time
Virtual onscreen keyboard, and shortcuts to use your own
desktop keyboard for testing
Team management for testing a device and sharing the screen
with other users
DOM Inspector and HTTP headers viewer using an included
proxy
Many manufacturers and carriers have selected DeviceAnywhere as
the official testing solution for their Virtual Developer Labs (VDLs).
Some of the Virtual Lab solutions include:
Forum Nokia VDL (Series 40 and S60)
Sony Ericsson VDL
Palm VDL (Palm OS, Windows Mobile, and webOS)
Motorola VDL (Motorola OS, Windows Mobile, and
Android)
BlackBerry VDL
Symbian VDL (Nokia, Motorola, Samsung, Sanyo, Sony
Ericsson)
If you want to access VDL websites, you can see a list at http://www.mobilexweb.com/go/vdl.
To use DeviceAnywhere, you’ll need to subscribe to one or more
packages. On top of the monthly subscription fee (averaging $100), you
will pay on a per-hour-basis or subscribe to a prepaid plan. On a
per-hour-basis, the maximum price is $16/hr. There are also other
promotions available on the website, and different manufacturers’ VDLs
can have different pricing models.
Note:
If you apply for a free trial, you will get 3 or 5 hours of
free usage and you will have to enter valid credit card details. It
is safe to add this information, and it is a requirement because
this is a live network where you can buy premium content.
The time spent on the system is calculated beginning from when
you open a device and finishing when you release it, in
6-minute-minimum time slots.
Package options include:
- Carrier package
You select a carrier and a country (for example, Verizon
in the U.S. or Orange in France), and you will have access to
all the devices available.
- Manufacturer package
You select a vendor, and you will have access to several
devices from that manufacturer.
- Official VDL
You select a vendor or a carrier’s official
package.
- Web Developer package
This is a special package for web developer testing. You
can select U.S. or Europe-based devices (from a range of
manufacturers).
Note:
For independent developers, there is a special package called
the Independent Developer Plan that provides access to iPhone and
Android devices. The fee is $30 per month, and $20 per hour.
1.3.1. Usage for mobile web testing
As DeviceAnywhere uses real devices from different
manufacturers, you will need to learn to use every operating system
interface to access the web browsers. You will generally find an
icon in the home or applications menu labeled “Browser,” “Internet,”
or even the name of the carrier’s online service (for example,
“MediaNET,” the AT&T Wireless service).
When in the mobile browser you will need to type your URL
using the phone’s features: a numeric keyboard, a QWERTY keyboard,
or an onscreen touch keyboard. DeviceAnywhere also offers a feature
where you can type or paste any URL and then press a button to
automatically generate all the keypresses required on the hardware
to type the URL.
In numeric keypad devices the URL typing process can be slow,
so it’s better if you first minimize the URL using a shortener
service, like http://www.mobiletinyurl.com.
For mobile web debugging purposes, DeviceAnywhere includes an
excellent proxy-based browsing solution that brings into the IDE an
HTTP sniffer and a DOM Inspector so you can see what markup is
actually rendered on the device.
Note:
Remember that these are real devices on real networks. If
you want to test an application or installable widget you will
need first to upload it to a web server (DA offers a solution) and
then access the URL from the browser, typing it or sending it by
SMS to the device.
1.3.2. Testing automation
DeviceAnywhere offers many advanced features. One of them is
testing automation, a premium service that allows you to create
testing scripts and schedule them to be tested on several devices
automatically. You can then access the results via a web
report.
1.4. Perfecto Mobile
Perfecto Mobile (http://www.perfectomobile.com) is a new company
offering a software/hardware hybrid solution for mobile testing, shown
in Figure 4.
Perfecto Mobile uses a video camera for screen recording. A good point
for Perfecto Mobile is that the whole environment is built on top of
the Adobe Flash Player, so you don’t need to install anything, and it
works from any desktop browser. You can try the system by registering
for a free trial; it will be activated in minutes.
With this service, you have access to the whole list of devices
and carriers from the same pricing policy. The devices are on real
networks in Canada, Israel, the U.S., the UK, and France. The company
also has an agreement with the French company PACA Mobile Center
(http://www.pacamobilecenter.com), where you can
access 1100 devices connected to French operators with a 10-hour trial
promotion.
Note:
If you are using non-touch devices, for website scrolling it
is better to have a key pressed down for a long time. You can
emulate this using the Control key on your desktop keyboard.
1.4.1. Pricing structure
Perfecto Mobile has a simple pricing model. There is no
monthly fee or other package costs, and you can access the full
cloud of devices from $16/hr, or with a prepaid plan starting at
$12/hr. There is also an Android-only option (the Droid Cloud) that
you can access at a rate of $9/hr, or $99 for 20 hours (without the
hours expiring). The charging is done in 1-minute time slots.
Warning:
In both DeviceAnywhere and Perfecto Mobile you can use two
or more devices at the same time. Your per-minute charges will be
counted separately, so you will be spending two or more minutes at
a time.
1.4.2. Main features
The main features of Perfecto Mobile for mobile web testing
are:
When you take screenshots it uses the real screen image,
not the camera one.
You can record videos and share or embed them
easily.
You can send an SMS or invoke a call to the device from
the UI.
You can transfer files to the device (if file transfer is
supported).
There is an OTA mechanism where you can upload your app or
widget and the device will receive an SMS link to download it
within a 15-minute timeslot.
You can easily share a URL, so customers and coworkers can
see what you are doing with the device via live streaming. The
only requirement for the other parties is that they use a
browser with Adobe Flash Player support.
You can request Automation, a macro-like recording feature
that supports advanced actions and wait conditions using screen
recognition and OCR (for example, “go to this URL, wait for the
word “Hello” to appear on the screen, then take a
snapshot”).
In Automation, there is a ScriptOnce technology that
includes multiplatform templates for common actions.
You can test how your mobile website is rendering on
multiple devices at the same time without your intervention.
This feature, called Website Validation, is shown in Figure 5.
