Windows Server 2003 : Specifics of the Windows Implementation

The Windows IPSec implementation is supported by an IPSec Policy Agent service (the IPSec Service) and an IPSec Driver. The IPSec Service is enabled by default and provides the IPSec driver with the IPSec filter list. During normal network operation, the IPSec driver uses the list to filter all inbound and outbound traffic. If an inbound or outbound packet matches the list, the driver applies the filter action (block, permit, or secure). IPSec policies can be written and assigned using a MMC snap-in or with scripts. If Windows XP Professional, Windows 2000, or Windows Server 2003 computers are joined in a Windows 2000 or Windows Server 2003 domain, IPSec policies can be implemented as part of Group Policy. When Group Policy is used, different policies can be quickly implemented and updated per Organizational Unit (OU).

1. The IPSec Policy Agent Service

The IPSec Policy Agent is implemented as a service. The service is responsible for obtaining the policy from the registry or from Group Policy, polling for changes in policy and passing the policy on to the IPSec Driver. The Policy Agent retrieves the policy from the registry (if the computer is a workgroup member), and from Group Policy (if the computer is a member of a domain and IPSec has been implemented via Group Policy for that member computer). The Group Policy-based IPSec policy (one enabled on the host computer) is cached in its registry and can be used if the computer is not able to connect to a domain controller (DC) or otherwise has problems receiving or implementing Group Policy. When the computer is next able to connect to a DC, any updated IPSec policy will be downloaded.

Retrieval occurs at startup, during the time specified in the policy and at the normal Group Policy polling. An administrator can manually update Group Policy by using the gpupdate command. During computer boot and operating system startup, there will be a short period of time after which the networking driver has started, but the IPSec Policy Agent service has not. Therefore, the policy is not working to protect communications. While the time is small, automated attacks (if they occur during this time) might compromise the system. It is possible, with Windows Server 2003 IPSec to create a persistent policy and other protections during this timeframe. 

2. The IPSec Driver

If the traffic matches a filter in the IPSec policy, the action that the IPSec Driver takes depends on the filter action recorded in the policy for the filter.

If the filter action is "negotiate security," the IPSec main mode negotiation and quick mode negotiation are attempted. If they succeed, SAs are created. (If they fail, packets are discarded.) The driver stores the current quick mode SAs in a local database.

If an inbound packet is secured by IPSec, the Driver uses the SPI to match packets to the correct SA in its database. This matching allows the Driver to identify the encryption key required for encryption or decryption.

If an unsecured inbound packet is received, the Driver attempts to match it to its filter list. If a match is not made, the packet is delivered. If a match is made and the block or permit filter action is applied, the packet is blocked or permitted accordingly. If a match is made and the filter action is secure, the unsecured packet will be dropped.

3. New in Windows Server 2003

IPSec was fully implemented in Microsoft Windows 2000. While it was praised for its ease of use, administrators had three major issues with the first version:

• The existence of default exemptions (protocols that were not blocked by default when a "block all" policy was enabled)

• The inability to use IPSec across a NAT server

• The need for more comprehensive scripting and troubleshooting facilities

To answer these complaints and further enhance IPSec for Windows Server 2003, several changes were made, including the following:

Resultant Set of Policy (RSOP) support

RSOP can be used to help troubleshot Group Policy issues.

2048-bit Diffie-Hellman group 3

The largesize of the numbers used in the Diffie-Hellman algorithm means increased protection.

NAT traversal support

The ability to use IPSec across a NAT server.

Command-line support using netsh

Ipsecpol and ipseccmd tools are replaced by netsh ipsec commands.

Default exemptions are reduced

Only Internet Security Association and Key Management Protocol (ISAKMP) communications (the messages used in IPSec policy negotiation) are exempted from IPSec.

Persistent IPSec policies

These can be used to protect systems during startup.

Stateful filtering during boot

This provides additional protection during boot.

IPSec Security Monitor

This is a comprehensive monitoring and troubleshooting tool for IPSec.

3.1. Disable default exemptions

A "block-all" policy, by definition, blocks all traffic, or does it? Windows 2000 exempted the following communications—even if a block-all policy was configured:

• Kerberos (might be used for authentication)

• ISAKMP (used in IPSec negotiations)

• Broadcast traffic

• Multicast traffic

• RSVP Quality of Service traffic

Windows Server 2003 exempts only IKE (ISAKMP). While it might be argued that a "block all" policy should also block IKE, a negotiated policy requires IKE. Hence, if IKE were blocked, negotiations to determine whether communications should occur could not take place. Rather than create a "catch-22," Windows Server 2003 IPSec exempts IKE.

To modify the Windows 2000 exemptions, you must modify the registry. Begin this procedure by going to Start → Run, and enter regedit (to start the regedit.exe program). Navigate to the following registry key:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\IPSec

Add a new value by right-clicking IPSec and selecting New → "DWORD value." Name the entry NoDefaultExempt. Assign the new value one of the following:

0

Kerberos, multicast, broadcast, RSVP, and ISAKMP traffic are exempt (the default for Windows 2000 and Windows XP).

1

Multicast, broadcast, and ISAKMP traffic are exempt.

2

Kerberos, ISAKMP, and RSVP traffic are exempt.

3

ISAKMP is exempt (the default for Windows Server 2003).

Close the registry editor.

Exemptions can be modified in Windows Server 2003 using the netsh command. To change the exemption on Windows Server 2003 to be like Windows 2000, open a command prompt and run the following command:

    netsh ipsec dynamic set config ipsecexempt value=0
					

3.2. IPSec NAT traversal

IPSec hashes and signatures may include source IP address and port information. This information may also be encrypted. If this information is modified or must be modified during transport, then IPSec security mechanisms will reject the packets as being tampered with, or won't be able to process the packets. IPSec cannot differentiate between a malicious modification and one that might occur when Network Address Translation (NAT) is used. This means that, as originally implemented in Windows 2000, IPSec could not be used to protect communications that occur across a NAT box.

Windows Server 2003 supports IPSec NAT traversal (NAT-T), as defined in two RFC documents (RFC 3948, Proposed Standard, UDP Encapsulation of IPSec Packets, version 2; and RFC 3947, Proposed Standard, Negotiation of NAT-Traversal in the IKE, version 2). This means that a NAT-T compliant sender notifies of its NAT-T capability during main mode and sends a hash of the source and destination packets address and port number so that the receiving peer can tell if NAT changed the IP address or port. If the receiving peer discovers that the address or port has been changed, it assumes that NAT is being used and defines a new IKE header that uses UDP port 4500. Thus, the IPSec communications are encapsulated by UDP, IKE negotiations can complete, and IPSec traffic can traverse the NAT box.

4. Default IPSec Policies

Windows IPSec has three default policies that are available for you to examine and test. However, you should not use these polices, but rather create your own. Feel free to examine them to see how they are configured, but you should test them before attempting to implement a policy in your production environment. Figure 1 displays the IPSec policy node of Group Policy and the three default policies.

Figure 1. Windows IPSec default policies

The default policies include:

Client

The recipient of an IPSec ISAKMP negotiation request will respond using its configured specifications.

Server (Request)

The server will initiate an IPSec ISAKMP negotiation request. However, if the recipient is not configured to use IPSec, the server will drop back to unsecured communications.

Server (Security Required)

The server initiates an IPSec ISAKMP negotiation request. However, if the recipient is not configured to use IPSec, the connection will be dropped.

You should now have a good understanding of how the IPSec protocol works and what new features have been introduced in Windows Server 2003. This is a good time to start examining how to configure Windows IPSec policy .