WEBSITE

IIS 7.0 : Troubleshooting - Using Tools and Utilities

2/4/2011 5:09:02 PM

Using Tools and Utilities

This section briefly explains a core set of troubleshooting tools and utilities. Use them to systematically examine the many components that can be at fault when a Web site or application is not responding as it should.

Become well-versed in using these utilities. When you are faced with a challenging issue, your experience will not only help you efficiently work through the problem but will also guide you in your understanding of the full range of possible causes to be explored.

Note

The IIS 6 Diagnostics Toolkit (which includes AuthDiag, DebugDiag, and SSL Diag) does not work with IIS 7.0, but refer to http://www.iis.net for possible updates to this valuable troubleshooting tool.


Table 1 lists the tools and utilities covered in detail later in this article. The table briefly describes when and how you might apply these tools.

Table 1. When to Use Troubleshooting Tools
Tool or UtilityApplicabilitySecurity or Performance Considerations for Production System
net startCheck whether or not a service is started and start it if it is stoppedNone
sc queryCheck whether or not a service is startedNone
pingCheck for basic network connectivityNone
PortCheckCheck for connectivity through a specific port on the serverNone
tasklistFind the PID for a service; use the PID to check status with netstatNone
netstatList all listening ports with their protocol, local and foreign address, state, and PID (look for the PID you found with tasklist)None
WFetch 1.4Send and receive actual headers and requests without interference from a user-friendly browser; view the raw request and responsePoses a security risk because of stored passwords and test certificates
AppcmdList and start sites and application pools; check for correct network bindingsNone
Process MonitorMonitor file system, registry, process, thread, and dynamic-link library (DLL) activity; monitor worker processesTurn off when not in use
IIS ManagerExamine currently running worker processes and requestsNone
Event ViewerCheck for error events in Application and System logsNone
Failed Request Tracing (FRT)Generate trace logs, especially for errors that are intermittent or hard to reproduceMinimize possible performance impact by narrowly limiting the scope of monitored requests
Reliability and Performance MonitorCheck overall system performance; identify suspect processes when server appears to hangTurn off when not in use
Network MonitorCapture network packets and diagnose Transmission Control Protocol (TCP) session dataTurn off when not in use

Using net start and sc query

Note

Explore the sections in this article on net start, ping, and the PortCheck tool when confronted with a 503 error code or a “Page cannot be displayed” error.


You can quickly check which services are running on a machine by using the following command at a command prompt.

net start

All active services will be listed. In particular, to accept requests, the server must be running the Windows Process Activation Service (WAS) and the World Wide Web Publishing Service (W3SVC). You can quickly check (and start if necessary) the W3SVC service, for example, as follows.

net start W3SVC

If the service was not started, it will start. Otherwise, a message will indicate that the service has already been started. W3SVC depends on WAS. If you try to stop WAS when W3SVC is running, you will be asked to confirm that you want to stop both services. Likewise, if you try to start W3SVC when WAS is stopped, the command will automatically (although silently) start WAS.

If you want to check the status of a service and not start it if it is stopped, you can alternatively use the following command, where serviceName is the name of the service you want to check.

sc query serviceName

For example, the output from running sc query W3SVC looks like this.

SERVICE_NAME: w3svc
TYPE : 20 WIN32_SHARE_PROCESS
STATE : 4 RUNNING
(STOPPABLE, NOT_PAUSABLE, ACCEPTS_SHUTDOWN)
WIN32_EXIT_CODE : 0 (0x0)
SERVICE_EXIT_CODE : 0 (0x0)
CHECKPOINT : 0x0
WAIT_HINT : 0x0


Using ping

Note

Refer to the information in this section when you are confronted with a 503 error code or a “Page cannot be displayed” error.


You can rule out basic network issues by pinging the server from the client. First try pinging the URL host name to make sure it is being correctly resolved to an IP address. Use the ping command line tool from a command prompt, as in the following example.

ping www.contoso.com

The output should show an appropriate IP address. If it does not, make sure the client’s Domain Name System (DNS) server is working properly. Also check to see if the client has a hosts file that might be interfering with proper IP address resolution. Assuming IP address resolution is not a problem, if you can’t ping the server’s IP address itself, there could be a problem with the network connection, either at the client or the server end. But do check to make sure firewall settings on the client aren’t preventing the connection. 

Using PortCheck

If you’re unable to ping a server, it could be because the server’s firewall is rejecting the connection, or the service isn’t listening on the required port. The PortCheck tool can check Transmision Control Protocol (TCP) connectivity for a specific port at the IP address in question. The tool provides a detailed breakdown of the different connectivity issues you might be facing.

Note

The PortCheck tool is included on the companion media, or you can download the tool from the following Web site:

http://mvolo.com/blogs/serverside/pages/Check-network-service-connectivity-with-PortChecker-tool.aspx.


Use the PortCheck tool from the command line to check for connectivity through a specific port on a URL as follows.

portcheck www.contoso.com 80

Or you can specify the IP address, like this.

portcheck 192.168.1.101 80

Using tasklist and netstat

If you suspect problems with a service or process listening on a port, you can list all services and their process IDs (PIDs) by entering the following command at the command prompt.

tasklist /svc

The output, showing PID 564 for the W3SVC and WAS services, for example, looks like this.

Image Name                     PID Services
========================= ======== ============
System Idle Process 0 N/A
System 4 N/A
smss.exe 360 N/A
...
svchost.exe 564 W3SVC, WAS

Use the output from tasklist to find the PID for the service you are troubleshooting. Next, to list all listening ports with their protocol, local and foreign address, state, and PID, enter the following command at the command prompt.

netstat -ano

Look for the PID that you located with tasklist in the netstat output and check the address and state, or look for processes that are listening on a port you are interested in. The output, showing PID 4 listening on port 80, for example, looks like this.

Active Connections

Proto Local Address Foreign Address State PID
TCP 0.0.0.0:80 0.0.0.0:0 LISTENING 4
TCP 0.0.0.0:88 0.0.0.0:0 LISTENING 560
TCP 0.0.0.0:135 0.0.0.0:0 LISTENING 844
TCP 0.0.0.0:389 0.0.0.0:0 LISTENING 560
...

Using WFetch 1.4

Note

WFetch 1.4 is included in the IIS 6 Diagnostics Toolkit. You can download WFetch 1.4 separately at the following location: http://www.iis.net/go/1307.


Sometimes the error reporting that the browser provides can partially obscure the symptoms of a problem. You can bypass the typical browser’s user-friendly packaging by using WFetch, which is essentially a browser with no enhancements. WFetch lets you see the raw data sent between browser and server, so it’s ideal for testing authentication issues or looking at custom headers in both the request and response packets.

Caution

Because WFetch enables you to store passwords and manipulate test security certificates, WFetch has the potential to expose your site to security risks. Therefore, Microsoft recommends that this tool be used only in testing, not in a production environment.


To run WFetch 1.4 (on an x86 system for this example), execute WFetch 1.4 from the IIS Diagnostics (32bit)\WFetch 1.4 program group. Figure 1 shows the WFetch 1.4 user interface with log output after requesting the default Web page on localhost. You can use the interface to specify an HTTP verb (GET, HEAD, PUT, DELETE, TRACE, POST, OPTIONS), the host, the port, the HTTP version, and the path, as well as authentication and connection types. For complicated scenarios, you can manually add portions of a request in the Advanced Request section or have WFetch read part of the request from a file.

Figure 1. The WFetch 1.4 user interface.

Note

The file format for help files included with the current release of the IIS 6 Diagnostics Toolkit is not supported on Microsoft Windows Vista or Windows Server 2008. Use online help instead or install WinHlp32.exe to use the help files. See the following Knowledge Base article for more information and a link to the WinHlp32.exe download location: http://support.microsoft.com/kb/917607.


Using Appcmd

If a site isn’t running, or if the network bindings for the site are misconfigured because of an error in your IIS configuration or in the site definition, the IIS server won’t be able to receive requests for that site. You can use Appcmd to quickly check that a site is running and that its network bindings are correctly configured. Enter the following command at the command prompt.

AppCmd list sites

If the IIS configuration is OK, the site definition’s bindings are listed, along with the current state (Started, Stopped, or Unknown). Check that the bindings make sense. If the site is not started, a configuration error might be preventing IIS from starting the site or the application pool. You can start a site by using Appcmd as follows.

AppCmd start site sitename

Similarly, you can check whether or not an application pool is started by using this command.

AppCmd list apppools

If necessary, start the application pool by using this command.

AppCmd start apppool poolname

Using Process Monitor

Process Monitor lets you monitor the file system, registry, processes, threads, and DLLs. It replaces and extends two tools used in earlier versions of IIS: Filemon and Regmon. Process Monitor is a sophisticated tool that provides detailed system information along with searching, filters, and simultaneous file logging to help you sort through the wealth of data. You can examine the process data in real time, or you can capture and review it later.

To start Process Monitor, run Procmon.exe from its installation location. Figure 2 shows the Process Monitor window displaying a small portion of the operations that take place when a user browses a site. You can toggle the display of operations related to registry activity, to file system activity, and to process/thread activity. You can filter the operations according to an extensive list of categories and values. Process Monitor also lets you search the list and jump directly to an object involved in the operation, for example, a temporary file or registry key.

Figure 2. The Process Monitor window after the client has browsed a site with Internet Explorer.

Double-clicking an operation displays properties for the operation, from which you can view information about the event, the process (as shown in Figure 3), or the stack.

Figure 3. The Process tab of an operation’s event properties.


You can view summary reports as well as a tree view of processes, as shown in Figure 4.

Figure 4. Process Monitor’s Process Tree tool.

You can use Process Monitor to monitor your worker processes (filter for “Process Name is w3wp.exe”). To check which application pool the worker processes serve, use IIS Manager as explained in the next section to look at the Worker Processes list and the list of active requests. Doing so lets you peer into the worker process to see running requests and in what stage those requests are, as well as what appDomains are loaded into that process.

You can also use Process Monitor to examine system-wide details for a series of events when your troubleshooting efforts take you outside the scope of information captured by IIS-specific tools such as FRT.

Note

Process Monitor is not part of the default install for Windows Vista or Windows Server 2008—you must select it as part of a custom installation. If necessary, you can also download Process Monitor v1.26 from the following location: http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx.

Using IIS Manager

IIS Manager uses the Runtime State and Control API (RSCA) to provide a viewport onto currently running worker processes and any requests running under those processes. In IIS Manager, select the server in the tree control and then scroll down in the central pane and double-click Worker Processes in the IIS group. IIS Manager displays currently running worker processes, as shown in Figure 5.

Figure 5. Using IIS Manager to view currently running worker processes.

Select the worker process you want to investigate and click View Current Requests in the Actions pane to display the list of current requests for that worker process, as shown in Figure 6.

Figure 6. Using IIS Manager to view currently running requests.

Knowing what worker processes and requests are currently executing (the “requests in flight”) can help you resolve slow request responses when the culprit is an unrelated request that’s excessively consuming system resources. You might also find that an application interferes with other applications in the same application pool and decide to separate the application into its own application pool. Or you could create an FRT rule for the currently executing requests and see where they’re all blocking and what happened to those requests prior to the block.

Note

ou can programmatically access RSCA through the WMI provider or through managed code and the Microsoft.Web.Administration namespace, which exposes the following information for a request through the WorkerProcess class. (GUID is the unique request ID that is also emitted when tracing is turned on for the request.)

GUIDSiteClientIPAddressCurrentModule
ConnectionIDLocalIPAddressTimeElapsedTimeInModule
VerbLocalPortPipelineState 
URLHostNameTimeInState 

For examples of accessing RSCA via code, go to http://www.iis.net and search for “How to Access IIS7 RSCA Data,” then click on the article with the same name.


Using Event Viewer

You can use Event Viewer to check why an application pool has been stopped or cannot be started. To start Event Viewer, launch Event Viewer from the Administrative Tools program group. Alternatively, from the Run text box, type eventvwr. In the tree, expand Windows Logs and then select Application. You can filter the list of events by event source.

Important

Refer to the following online documentation, known as the IIS 7.0 “health model” (in other words, diagnostic and failure conditions experienced by software), for a list of event IDs related to IIS 7.0. Expand the documentation nodes in the left tree control or follow the links in the documentation to drill down to a list of events relating to specific managed entities or aspects of managed entities: http://technet2.microsoft.com/windowsserver2008/en/library/b19873a2-9f72-40c8-b73d-89f39cda62781033.mspx?mfr=true.


Errors the IIS-W3SVC-WP source generated, as shown in Figure 7, indicate that the worker process started but failed during initialization. This might happen, for example, if the configuration section is invalid, if a module failed to load, or if a module failed to initialize. You can double-click an event to see details about it.

Figure 7. Errors in the Application log in Event Viewer indicate a problem occurred while initializing a worker process.

Also select the System node under Windows Logs. Errors generated by the IIS-W3SVC-WP or WAS source in the System log indicate that WAS could not start the worker process. This might be caused, for example, by a problem in the configuration, by a problem with the application pool identity credentials, by exceeding the number of worker processes allowed, or by running out of some other resource.

An error in the System log might also be caused by a bug in custom application code. WAS will disable an application pool if an application in the pool triggers Rapid Fail Protection (RFP). RFP is a mechanism that IIS uses to prevent repeated failures in a faulty application from bringing down the system. When IIS identifies repeated worker process crashes or hangs in a given application, RFP will shut down the entire application pool in which the application is running.

Using FRT

Use FRT to generate trace logs of IIS responses that violate conditions you set in the FRT component of IIS Manager. To use FRT most efficiently in your troubleshooting process, narrowly restrict the FRT rules you add so that you can examine the minimal subset of URLs that are experiencing problems.


Using Reliability and Performance Monitor

Use Reliability and Performance Monitor to examine the overall condition of your server. You can start Reliability and Performance from the Administrative Tools program group. Select the Performance Monitor node under the Monitoring Tools node in the tree. You can add and configure counters for all aspects of the server’s performance, as shown in Figure 8.

Figure 8. Viewing performance counters in Performance Monitor.

You will almost always want to monitor Process and Thread counters such as Processor Time and % Processor Time. When troubleshooting an ASP or ASP.NET application, also look for the Active Server Pages counters, the various .NET CLR counters, and the ASP.NET and ASP.NET Applications counters. In addition, IIS 7.0 adds many new performance counters under the groupings W3SVC_W3WP and WAS_W3WP.

Reliability and Performance Monitor is especially useful when you are investigating a server that appears to hang. You can save a performance log by creating a Data Collector Set. To do so, expand the Data Collector Sets node in the tree and then right-click the User Defined node and select New\Data Collector Set from the context menu. To isolate the problem, analyze the performance log in combination with a memory dump from the process that is running on the server. You must time the memory dump to coincide both with the hanging behavior and with a performance log that is actively collecting data.

Using Network Monitor

If you need to troubleshoot a problem that involves connectivity issues, you can use Network Monitor to capture network packets and diagnose TCP session data. You can start Network Monitor 3.1 from the Microsoft Network Monitor 3.1 program group. After capturing a sequence of network packets, you can double-click a frame in the sequence to see complete details (including the hexadecimal contents) of the frame, as shown in Figure 9.

Figure 9. Viewing network packets in Network Monitor.

Note

Network Monitor 3.1 is not part of the default install for Windows Vista or Windows Server 2008. To download Network Monitor 3.1, go to http://www.microsoft.com/downloads and search for “Microsoft Network Monitor 3.1.”

Other  
 
Most View
Arcam FMJ A19 Integrated Amplifier - Jumping Jack (Part 2)
Windows Small Business Server 2011 : Customizing a SharePoint Site - Introducing SharePoint Foundation 2010
Windows Server 2003 : Domain Name System - Zones Versus Domains, Resource Records
Wireless Connections: What You Need To Know (Part 3)
AOC E2343FI – It Isn’t Just Incredibly Slim
How To Set Up Virtu On A Desktop PC
Corsair Dominator Platinum Dual-Channel DDR3 Memory Kits (Part 1)
Desktops Disguise - All-In-One Computers (Part 4) - Asus Transformer AiO
Life Logging - Is It Worth The Effort? (Part 3)
Evga GTX 690 - Hail To The King
Top 10
Windows Phone 8 : Microsoft Office Mobile - Excel (part 10) - Entering Formulas,AutoSum
Windows Phone 8 : Microsoft Office Mobile - Excel (part 9) - Sharing a Spreadsheet, Fitting and Hiding Text
Windows Phone 8 : Microsoft Office Mobile - Excel (part 8) - Accessing Charts and Sheets, Saving a Spreadsheet
Windows Phone 8 : Microsoft Office Mobile - Excel (part 7) - Undo and Redo, Charting
Windows Phone 8 : Microsoft Office Mobile - Excel (part 6) - Sorting a Column, Filtering
Windows Phone 8 : Microsoft Office Mobile - Excel (part 5) - Freezing a Pane, Applying Formatting Options
Windows Phone 8 : Microsoft Office Mobile - Excel (part 4) - Selecting Multiple Cells, Viewing Cell Text
Windows Phone 8 : Microsoft Office Mobile - Excel (part 3) - Viewing and Navigating Comments,Searching a Spreadsheet
Windows Phone 8 : Microsoft Office Mobile - Excel (part 2) - Adding a Comment to a Cell, Changing the Comment Author
Windows Phone 8 : Microsoft Office Mobile - Excel (part 1) - Creating a New Spreadsheet, Working with the Excel Interface