Long story short, IPsec isn't just for VPNs anymore. While IPsec is popular when used in conjunction with virtual private networks, the technology has reached a level of maturity that now allows it to be used for basic packet filtering and other isolative security practices.
Windows Server 2008 takes some steps forward when it comes to broadening the reach of IPsec. Let's look at the ways Windows Server 2008 takes steps to broaden the reach of IPsec.
Correcting the befuddling deployment process
Traditionally, configuring IPsec has been, shall we say, less than easy. It involved a bizarre deployment and configuration process and a non-intuitive console interface. Microsoft answered the cries of administrators everywhere by releasing the Simple Policy Update for IPsec. This update, for Windows XP and Server 2003, was not well-received when it came out in 2006, but it certainly is a step forward. You can find this update in Windows Server 2003 Service Pack 2.
With the release of Vista and now Windows Server 2008, the configuration console for IPsec has melded with Windows Firewall, making it infinitely easier to correctly deploy IPsec policies in tandem with other technologies. For instance, the addition of the "New Connection Security Rule Wizard," is really useful for getting any type of IPsec configuration correct, be it an isolation policy, a tunnel or server-to-server filtering.
Server and domain isolation
IPsec's somewhat hidden strength, however, is its built-in capability for shielding legitimate machines on your network from communications with machines that (a) are not managed and (b) are not authenticated. IPsec can require authentication, based on Kerberos, certificates or pre-shared keys, and enforce the presence of those factors before it allows actual communications between two machines. This is enormously powerful in the context of server isolation -- it's a sort of pre-Network Access Protection (NAP) way of ensuring that your most precious machines on the network aren't being threatened by zombie PCs that come on the wire.
You can create this type of isolation rule with the aforementioned wizard. On the first screen of the New Connection Security Rule Wizard, select the Isolation option and proceed through the creation of your own IPsec isolation policy rule.
NAP and cross-platform compatibility
Perhaps the biggest cheerleader for wider IPsec implementation is its place in conjunction with NAP. NAP is a network-wide, software-hardware solution for restricting unfettered network access from unmanaged hosts whose health hasn't been verified. IPsec with NAP takes server and domain isolation even further and isolates the whole network from clients that don't meet certain administrator-set health guidelines.
The idea of IPsec on Windows might raise some admins' eyebrows -- those with multi-platform deployments -- but the good news here is that Microsoft has been working with more than 100 NAP partners to extend the technology to any device, any platform. Several NAP clients for Macintosh and Linux flavours are already working and available, if in beta form.
Bottom line: IPsec has made tremendous progress over the last few years. Take a look at this new way of deploying IPsec in Windows Server 2008, with better configuration features and its integration with NAP. The time may be right for you to consider IPsec.
About the author: Jonathan Hassell's books include RADIUS and Learning Windows Server 2003 for O'Reilly Media and Hardening Windows for Apress.