Testing Times for CISO's

Performance and effectiveness claims from vendors of network security products can never be taken at face value. In a process crucial to making the right buying decisions, how do the CISO, CIO and other security professionals ensure that new in-line security products are tested thoroughly in an environment that replicates as closely as possible that found in his or her own network?

Selecting security products is a complex process that carries significant risks if not executed correctly; poorly chosen products can fail to protect against serious threats, cause serious performance problems for enterprise networks and waste scarce financial resources. CISO’s, CIO’s and other security professionals need to develop and execute an enterprise-specific in-house testing plan before evaluating and purchasing security products.

Failing to test security products before buying them means organizations run the risk of performance limitations, security failures and overspending. Weaknesses in security coverage can often remain undiscovered for long periods of time, leaving those organizations at risk of losing corporate assets or compliance status. Installing in-line security devices such as Firewalls, Intrusion Prevention Systems (IPS), and Secure Web Gateways can lead to a false sense of security unless vendor claims are verified. Critical servers often remain unpatched in the belief they are protected by an IPS, when claimed coverage is actually less effective than promised. In addition, fear of false positives can lead enterprises to run IPS devices in a less secure IDS mode, thereby forfeiting protective properties and increasing operating costs and risk. Selecting the wrong network security device can thus expose a company to serious threats from both inside and outside the network perimeter.

Poor performance from an in-line device once placed in a live network can also have serious consequences as latency increases to unacceptable levels. High latency or frequent “fail closed” events can result in active devices being redeployed in a passive state or having blocking disabled, significantly reducing their effectiveness.

Cost is an issue too. Without performing relevant tests in-house, organizations could be persuaded to overspend significantly, purchasing devices with performance and coverage levels that are not required.

In-house testing can help alleviate many of these problems, and it is important for organizations to use testing procedures designed for their own threat environment to determine the best in-line network security products for their specific needs.

NSS Labs has recently published an Analysis Brief covering key points CISO’s need to know about testing security products, entitled The CISO’s Guide to the Importance of Testing Security Devices (subscription required). Follow me on Twitter (@bwalder) to keep informed as new research is released.