A lot of work goes into configuring and
assembling an SAP solution that is supported by its Solution Stack
vendors and effective in servicing its end users. When such a supported
configuration is in place, the goal of the SAP Technical Support
Organization should be to minimize technical changes to the stack
unless absolutely required. The reasons cited most often for making
technology stack changes are
Performance
reasons, especially with regard to whatever the current bottleneck
tends to be (most often the disk subsystem or a factor related to CPU
utilization, like growth in system usage or volume of transactions)
Stability reasons, like published issues with specific software drivers, database releases, firmware revisions, and so on
New
functionality reasons, like the need for additional application servers
or more horsepower in the existing servers when incremental SAP
components, modules, and users are added to the landscape.
With these three reasons in mind, I’ll next touch upon a few sample scenarios and how each impacts the solution stack.
The Hardware, OS, and Database Layers
Hardware
and software vendors typically embrace a rigorous testing process of
their own prior to releasing new products. On top of this process,
vendors with SAP Competency Centers take the testing to a new
SAP-specific level, and add what I term an SAP Filter
to the testing process. The SAP Filter serves to filter out variables
in the SAP Solution Stack. Thus, instead of trying to support or
certify every single hardware platform, controller, software update,
firmware release, OS patch, and so on for SAP, the filtering process
seeks to test and support a specific combination of the stack’s
components, and “filter out” others.
The
SAP Filter is therefore good for the SAP technology vendor, good for
SAP AG (which finds itself supporting fewer combinations of technology
platform variables) and good for the SAP customer. The only drawback
might seem to be a final selection of products too limited for a
particular customer. In reality, though, I have never seen this to be
the case—vendors are naturally motivated to test/support their latest
products or most compelling solution sets. Consider the following list
of tested products and releases embraced by one of the author’s
favorite SAP competency centers, and judge for yourself:
New releases of hardware platforms and major components, including servers, disk subsystems, and disk controllers
Note
As
of August 2001, disk host bus adapters, or HBAs, are no longer
specifically required by SAP AG to be tested; many SAP hardware vendors
still seem to perform this level of customer-assurance testing anyway.
Updated firmware releases that address severe stability, performance, or security issues
New Operating System releases specific to previously tested and approved hardware platforms
Operating
System service packs and patches that will not likely be replaced in
the next six months, or that address severe stability, performance, or
security issues
Hardware-specific OS drivers and updates, though typically less often than service packs/patches
New
database releases (typically just major releases only, or as
specifically requested and funded by database partners or customers)
Database
service packs/patches that will probably not be replaced in the next
six months, or that address severe stability, performance, or security
issues
New major releases of mySAP.com
components, historically centered on new SAP Basis releases of R/3, BW,
APO, EBP/SRM, WP, CRM, and other components as they are released
Support
Packages for each major SAP Basis release (that is 4.5B, 4.6C, 6.20,
and so on. Other less common releases (like 4.6B and 6.10) may also be
tested, but usually not without internal or otherwise-sponsored funding)
With
such a comprehensive list as this one, it’s easy to see why not every
SAP Solution Stack option needs to be tested—indeed, much of the stack is tested, along with the most compelling product sets within each layer of the stack.
To
perform this comprehensive testing, Solution Stack partners and vendors
should embrace a new-product testing process similar to that described
in the following list. These are sometimes referred to as standard test plans
for SAP, and are often employed by large SAP customers as well as
solution stack vendors. A valid test plan should be consistent with
SAP’s general change-management recommendations (and their own internal
practices), consisting of a process something like the following:
1. | Perform
a “clean” install of the product to be tested, or install a clean
solution stack foundation if an upgrade to a particular product is to
be tested (the latter is a much more common testing scenario). For this
example, we’re focused on a new release of R/3 with a new release of
SQL Server 2000.
|
2. | Perform the SQL Server and R/3 installation, noting any issues and resolutions. |
3. | Log in to R/3 via the SAPGUI that ships with the particular release being tested.
|
4. | Perform an R/3 data dictionary check (DDIC).
|
5. | Perform an R/3 Client Copy.
|
6. | Perform an R/3 license check.
|
7. | If
applicable, import data into or out of the system (more applicable to
SAP BW, APO and other components that rely on a core transactional
system).
|
8. | Customize
existing load scripts (using AutoIT, CATT, or a more robust tool like
AutoTester ONE or eCATT), or run a “mini” standard SAP benchmark
against the system, to simply generate a load. Allow this to run for
12–24 hours (whatever is consistent with your previous testing),
serving as a “burn-in” for any other new components in the solution
stack. Record the transaction load, average response times observed,
and issues.
|
9. | If funded, run a complete SAP Standard Benchmark, as appropriate.
|
10. | If
on the roadmap or otherwise appropriate, use this system to perform
delta testing, including database upgrades, hardware upgrades, and so
on.
|
11. | Publish the testing results internally or as requested.
|
Such
comprehensive testing by a hardware/software vendor assures the
vendor’s customer base that the platform in question is truly ready for
prime time, and it also allows the partner to say without a doubt that
a particular solution stack has been fully tested and is therefore
supported.
The preceding example reflected what I call general new-product testing,
where a change impacts all or nearly all layers in the solution stack.
If we view the SAP solution like a set of concentric circles, general
testing in support of change management is typically performed from the
database server out to the other components in the solution, like the
Central Instance and Application Servers, and then out to the ITS Agate
and Wgate servers (or Web Application Server, as applicable), and
finally out to the SAPGUI or WebGUI front-end client. As shown in Figure 1,
then, general testing tends to be the most comprehensive form of
testing performed by hardware and software partners. It represents
testing of an end-to-end solution, “inside-out.”
Other
testing tends to be more tactical in nature. This is referred to as
tactical, characterization, or delta testing, and addresses a smaller
piece of the circle. It also involves less testing in general, often a
subset of the standard test plan we looked at earlier. The goal of
delta testing is to prove more that a new solution component works, rather than exactly how well
it works. And in the process, we generally uncover and learn from the
problems with installation, integration issues, and other issues as
they crop up.
After
each test is completed, the system log files/event log,
upgrade/installation logs, and other pertinent logs are reviewed for
errors, and the errors are analyzed. Further testing may then be
warranted, sometimes uniquely crafted to isolate or highlight a
particular layer or component in the solution stack.
After all testing and error-analysis, the new product is either approved or failed
(pass or fail, go or no-go). Most testing results are shared with the
appropriate internal and external organizations, including product
engineering, various software support groups, the OS vendor or
organization, the database vendor, SAP AG, and so on through either
formal certification processes or informal competency center
communications.
SAP Application Layer—Transport Strategies and More
Transport
strategies represent a critical piece of the overall change management
strategy. For it is by this process that business processes are created
and changed, support packages are applied, and other changes to the
core SAP internals are performed. Historically, SAP provided something
called the Correction and Transport System (CTS) to perform the work of transports. More recently, they renamed it to Change and Transport System, and introduced the much-improved Transport Management System
to work with Transport Organizer, Workbench Organizer, and Customizing
Organizer. Together with the tp and r3trans transport tools, the new
system forms a comprehensive albeit application-focused change
management utility.
The transport strategy or process implemented by an SAP customer should address or facilitate
Transport request forms, which should be consistent and complete
A process for reviewing the forms before and after the fact, to ensure that the change actually accomplishes its goal
A process for approving changes (that is, change management meetings and review boards)
A method for ensuring that the transported code indeed passed through the approval process
Attention to expedited emergency transports, or those changes that must find their way into production as soon as possible
Upgrades—Realizing the Benefits of Change Control
With
an SAP upgrade comes the opportunity to really benefit from good change
control practices. The system should already be completely documented
and all business cases thoroughly tested. Therefore, as the first pilot
upgrades are tested in various sandbox or crash-and-burn environments,
the go/no-go decision becomes easier—the test cases either work, in
which case the upgrade was successful, or fail, in which case any
number of issues might need to be addressed. Even if the test case
fails (which is likely), good documentation will make the point of
failure more clear than otherwise, and tend to shorten the
time-to-resolution significantly. Often, the failure is actually the
result of a change in how SAPGUI display screens are laid out, the
names or location of specific data entry fields, and so on, rather than
a failure in the business process itself. In other words, many
perceived failures in the initial tests of an upgrade are likely to be
errors in the test mechanism itself and not actually the underlying
upgrade. By quickly resolving the “what you expected was not what you
got” GUI-related display issues, the validity of the business process
can be quickly confirmed. At that point, you can then check off another
task in your upgrade checklist as complete, and continue moving forward
with the upgrade.
Finally, because the
upgrade process forces you to update nearly all of your test cases to
some extent, you can take this opportunity to learn from test-case
shortcomings or no-longer-applicable requirements, and look at some of
the following possibilities as you reconstruct your test cases:
Evaluate
a new testing tool or approach, one that might be less expensive, or
easier to use, or offer more benefits like automated testing and
improved regression-testing capabilities, and so on.
With
your new test cases, you might better integrate or address the needs of
the documentation and training teams, as well as the teams tasked with
stress testing or load testing.
Take the
opportunity to educate new or less experienced team members in the
process of building, testing, and maintaining test cases.
Improve
upon the test cases themselves. For example, one customer of mine
rewrote most of their test scripts to reflect end-to-end business
processes, such that all data needed in one script was created from a
previous one. In this way, they no longer needed to worry about
“stocking” a test client prior to executing test cases. And in another
example, my customer took the opportunity to create more variables
(instead of relying on many hard-coded input parameters) so that
alternative test case scenarios could be easily created and tested.
With
managing changes to the SAP Solution Stack behind us, a discussion of
structuring the organization actually tasked with managing change is in
order.
