Cisco CSS & ACE
Certificate Spoofing and Header Manipulation

Release Date 2010-07-02
Product Cisco Content Services Switch (CSS) and ACE Products
Versions Cisco CSS 11500 - 08.20.1.01
Cisco ACE 4710 - Version A3(2.5) [build 3.0(0)A3(2.5)
(Other versions may be affected)
Severity High (in specific configurations)
Discovered by George D. Gal
Advisory by George D. Gal <ggal (at) vsecurity.com>
Vendor Status Cisco CSS vulnerability remains unpatched, workarounds available
Cisco ACE workarounds available
CVE Candidate(s) CVE-2010-1575 - Certificate Spoofing Flaw
CVE-2010-1576 - HTTP Request Parsing Flaw
Reference Original Advisory

Product Description

From [1]:

"The Cisco CSS 11500 Series Content Services Switch is a high-performance, high-availability modular architecture for Web infrastructures. As the premiere switch for the Cisco Web Network Services Software, the Cisco CSS 11500 Series helps businesses to build global Web networks optimized for content delivery and e-commerce. By activating HTTP headers, the CSS 11500 Series helps to ensure availability, optimize utilization, reduce latency, increase scalability, and enhance security for Websites, server farms, cache clusters, and firewall systems."

From [2]:

"Cisco(R) ACE Application Control Engine application switches represent the state of the art in next-generation application switches for increasing the availability, performance, and security of data center applications. The Cisco ACE family of application switches includes the Cisco ACE Service Module for the Cisco Catalyst(R) 6500 Series Switches and Cisco 7600 Series Routers, as well as the Cisco ACE 4710 Appliance in a standalone form factor for discrete data center deployments."

Vulnerability Overview

On June 4th 2009, VSR identified multiple weaknesses in the Cisco CSS 11500's handling of HTTP header interpretation and client-side SSL certificates. Individually, these issues may be considered minor, but combined they could allow for the compromise of an application that relies on a vulnerable CSS to assist in authenticating clients. If successfully exploited, an attacker could spoof another application user's identity without possession of the victim's client certificate.

Additionally, due to the fact that the Cisco CSS product has been effectively superceded by the Cisco ACE, VSR has also identified similar issues to those described below in the ACE in particular configurations.

These issues may affect any CSS installation, but would have the greatest impact on deployments that have the following feature enabled in the configuration:

 ssl-server <context> http-header client-cert

Similarly, on the Cisco ACE, these issues may manifest themselves when using a policy map with a class-default class, as shown below:

 policy-map type loadbalance first-match SLB-VIP-REDIRECT
   class class-default
     serverfarm TEST-FARM
     action DO-SOMETHING-WITH-HEADERS
     insert-http X-SRC-IP header-value "%is"

Issue 1: Weak Enforcement of Authority in HTTP Certificate Headers

Cisco Bug Id - CSCSZ04690
Affects - Cisco CSS

The first weakness affecting the Cisco CSS is that, in a typical client certificate configuration, HTTP clients may confuse web applications by injecting their own certificate headers. When utilizing the CSS to terminate SSL communications, SSL client certificates are first authenticated by the CSS. From there, the CSS will normally pass the client's identity to the back-end web server in the form of several HTTP headers as shown below:

ClientCert-Subject: XXX
ClientCert-Subject-CN: XXX
ClientCert-Fingerprint: XXX
ClientCert-Subject-CN: XXX
ClientCert-Issuer-CN: XXX
ClientCert-Certificate-Version: XXX
ClientCert-Serial-Number: XXX
ClientCert-Data-Signature-Algorithm: XXX
ClientCert-Subject: XXX
ClientCert-Issuer: XXX
ClientCert-Not-Before: XXX
ClientCert-Not-After: XXX
ClientCert-Public-Key-Algorithm: XXX
ClientCert-RSA-Modulus-Size: XXX
ClientCert-RSA-Modulus: XXX
ClientCert-RSA-Exponent: XXX
ClientCert-X509v3-Subject-Key-Identifier: XXX
ClientCert-X509v3-Authority-Key-Identifier: XXX
ClientCert-Signature-Algorithm: XXX
ClientCert-Signature: XXX

However, there is no attempt by the CSS to prevent clients from supplying their own ClientCert-* headers. Depending on how application developers handle multiple copies of these headers, an attacker may be able to impersonate other users.

For example, assuming that a back-end web application simply trusts the user identity supplied by the CSS in the ClientCert-Subject-CN header and userX wants to impersonate userY, he may simply insert the following HTTP header(s) in the HTTP request issued to the server:

 ClientCert-Subject-CN: CN=userY
or
 ClientCert-Subject: C=US, ST=MA, L=Boston, O=xxx, OU=xxx, CN=userY
Upon injecting the attacker-supplied HTTP headers the application would receive an HTTP request similar to that shown below:
POST /targetapp HTTP/1.1
Content-Type: text/xml; charset=utf-8
ClientCert-Subject: C=US, ST=MA, L=Boston, O=xxx, OU=xxx, CN=userY
ClientCert-Subject-CN: CN=userY
Host: test.vsecurity.com
Content-Length: 1024
ClientCert-Subject: C=US, ST=MA, L=Boston, O=xxx, OU=xxx, CN=userX
ClientCert-Subject-CN: CN=userX
ClientCert-Fingerprint: XXX
ClientCert-Subject-CN: XXX
ClientCert-Issuer-CN: XXX
ClientCert-Certificate-Version: XXX
ClientCert-Serial-Number: XXX
ClientCert-Data-Signature-Algorithm: XXX
ClientCert-Subject: XXX
ClientCert-Issuer: XXX
ClientCert-Not-Before: XXX
ClientCert-Not-After: XXX
ClientCert-Public-Key-Algorithm: XXX
ClientCert-RSA-Modulus-Size: XXX
ClientCert-RSA-Modulus: XXX
ClientCert-RSA-Exponent: XXX
ClientCert-X509v3-Subject-Key-Identifier: XXX
ClientCert-X509v3-Authority-Key-Identifier: XXX
ClientCert-Signature-Algorithm: XXX
ClientCert-Signature: XXX

Since existing ClientCert-* headers are left intact, application developers are expected to trust only the last instance of a given certificate header. This approach is clearly prone to error if application developers do not carefully test this attack scenario.

An alternative approach to securing these headers can be achieved through an optional configuration where the CSS places an additional prefix string on the inserted certificate headers [4]. For instance, a server administrator could select a random header prefix through a command such as:

 ssl-server <context> http-header prefix "<random_prefix>"
This would cause the new certificate headers to be included with the form: <random_prefix>-ClientCert-*

So long as an attacker could not discover this random prefix, then there would be no way to spoof these headers. However, this solution is far from ideal, since there may be several ways for an attacker to obtain a copy of these headers. Examples include TRACE/TRACK requests being honored by the back-end web servers or debugging components in web applications which echo client headers.

Issue 2: Lack of HTTP Request Validation

Cisco Bug Id - CSCTA04885
Affects - Cisco CSS & ACE

A second weakness that manifests itself on the CSS and ACE through different interpretation of HTTP newline sequences between the content switch and commonly used web servers. RFC 2616 [3] defines the US ASCII carriage return/line feed (CRLF) sequence as the end-of-line marker for protocol elements (excluding the entity-body). Indeed, the CSS and ACE appear to adhere relatively closely to this requirement.

Popular web servers, however, permit various permutations of the CRLF sequence as end-of-line markers, including: LF, CR, and LFCR. This difference in interpretation could lead to serious consequences if the device relies on any of these headers and end-of-line delimiters or makes changes to any of them. This has been well documented in the past in relation to HTTP request smuggling and related attacks [5].

Discussion

It is difficult to consider either of these two issues, taken alone, as extremely critical flaws. In the case of certificate header handling, there are clearly work-arounds for header spoofing attacks, even if they are error-prone or imperfect. For HTTP newline interpretation, it is difficult to fault Cisco for adhering strictly to the RFC. However, in combining these issues in typical deployment scenarios the end result could be catastrophic for an application using a CSS and relying on client certificates for user identification and authorization.

During testing, VSR found that use of invalid newline sequences caused the CSS to fail to insert it's own ClientCert-* headers, though the back-end Apache web server accepted these newline sequences. This clearly defeats the approach that some application developers might take in trying to rely only on the last set of certificate headers.

The following simple HTTP request demonstrates how a client could cause the CSS to omit its insertion of the ClientCert-* headers:

 GET /protected_resource HTTP/1.0\x0a
 ClientCert-Subject: C=US, ST=MA, L=Boston, O=xxx, OU=xxx, CN=userX\x0a
 ClientCert-Subject-CN: CN=userX\x0a\x0a
However, various combinations of end-of-line delimiters should be possible. In fact, Cisco describes their logic for the fix as follows:

"The HTTP request detection will attempt to determine the end-of-line marker following the HTTP/1.x line. If it consists of a single LF, then the logic will attempt to identify a LFLF."

Unfortunately, this logic may be problematic when attempting to parse HTTP requests consisting of various combinations of end-of-line markers within the same HTTP request. For instance, many web servers accept requests with mixed newline sequences such as:

 GET /protected_resource HTTP/1.0\x0d\x0a
 ClientCert-Subject: C=US, ST=MA, L=Boston, O=xxx, OU=xxx, CN=userX\x0a
 ClientCert-Subject-CN: CN=userX\x0a\x0a

While testing a Cisco ACE appliance, VSR was able to cause rules that normally add or remove headers in HTTP requests to be omitted when the class-default class was used in a policy map as opposed to a layer 7 class for http communications. It is possible that if a class-default class were used for client certificate processing on the Cisco ACE Module, that client certificate headers could also possibly be omitted, allowing an attacker to insert their own headers used to set the authenticated user context.

Cisco's recommendation to utilize a layer 7 policy class map as shown below appears to mitigate this issue, preventing malformed or unexpected end-of-line delimiters:

 class-map type http loadbalance match-any http-cm
   2 match http url .*

 policy-map type loadbalance http first-match http-pm
   class http-cm
     serverfarm TEST-FARM
     action DO-SOMETHING-WITH-HEADERS
     insert-http X-SRC-IP header-value "%is"

Recommendations for Cisco and Competing Vendors

VSR recommends that Cisco (and any other vendors who provide similar products) implement more stringent request validation and/or corrections when receiving requests which do not utilize HTTP-compliant newlines. While failing to validate requests forwarded to back-end servers is a reasonable approach when requests are not interpreted or modified, VSR believes that some responsibility for correctness should be assigned when HTTP requests are modified in transit.

Three primary approaches are possible for Cisco CSS/ACE devices and similar load balancers upon receiving invalid newlines (CR which lack a trailing LF and LFs which lack a preceeding CR):

Each of these approaches carries its own set of security, speed and compatibility risks which should be weighed carefully. However, given that load balancers are border devices which commonly terminate SSL, some level of validation should be implemented to help protect internal systems.

In addition to addressing newline issues, Cisco CSS devices should provide better tools and documentation on securing certificate headers. VSR recommends one or both of the following approaches be adopted:

2013-06-19
IBM WebSphere Commerce: Encrypted URL Parameter Vulnerable to POA

2012-10-23
Timothy D. Morgan presents No Crack Required: Cryptanalysis in Real-World Applications at OWASP AppSecUSA 2012.

2012-07-29
Michael Coppola presents Owning the Network: Adventures in Router Rootkits at DEF CON 20 [slides].

2012-04-20
HTC IQRD Android Permission Leakage

more...

Contact us by phone,
fax or e-mail:

Phone: 617.933.8919
Fax: 617.933.8920
Email: inquiry@vsecurity.com