Case Studies

Testbed: two identical machines, one as client and one as server. The characteristics of machines and the network can be found in the Methodology page.

Apache web server

• Version: 2.4.18
• Configuration: default (MPM event model: hybrid multi-process multi-threaded)
• Dependencies¹:
  • OpenSSL 1.0.1f (susceptible to the Heartbleed bug)
  • apr-1.5.2
  • apr-util-1.5.4
  • PCRE 8.38
• Workload:
  • ab benchmark
  • with keepalive connections
  • fetching of a 2.3K static web-page
  • using HTTP get

Performance

Additional statistics²:

Performance Summary: GCC-MPX, ICC-MPX, and AddressSanitizer all show minimal overheads, achieving 95.3%, 95.7%, and 97.5% of native throughput. Overhead in latency did not exceed 5%. Such good performance is explained by the fact that our experiment was limited by the network and not CPU or memory.

Memory Summary: AddressSanitizer exhibits an expected 3.5X overhead. In contrast, MPX variants have dramatic 12.8X increase in memory consumption. This is explained by the fact that Apache allocates an additional 1MB of pointer-heavy data per each client, which in turn leads to the allocation of many Bounds Tables.

Security

• Bug: Heartbleed, official web-site and detailed explanation
• Exploit script: by Jared Stafford and Travis Lee

Results:

Summary: AddressSanitizer and GCC-MPX detect Heartbleed without any problems.

Usability

Issue 1: While testing against Heartbleed, we discovered that ICC-MPX suffers from a run-time Intel compiler bug 5 in the x509_cb OpenSSL function, leading to a crash of Apache. This bug triggered only on HTTPS connections, thus allowing us to still run performance experiments on ICC-MPX. See bug here.

Up to table of contents

Nginx web server

• Version: 1.4.0 (susceptible to this bug)
• Configuration: worker_processes = auto (1 process per core)
• Dependencies¹: OpenSSL 1.0.1f
• Workload:
  • ab benchmark
  • with keepalive connections
  • fetching of a 2.3K static web-page
  • using HTTP get

Performance

Additional statistics²:

Performance Summary: AddressSanitizer reaches 95% of native throughput, while GCC-MPX and ICC-MPX lag behind with 86% and 89.5% respectively. Similar to Apache, this experiment was network-bound, with CPU usage of 225% for native, 265% for MPX, and 300% for AddressSanitizer. (CPU usage numbers prove that HW-assisted approaches impose less CPU overheads.)

Memory Summary: MPX variants have a reasonable 4.2X memory overhead, but AddressSanitizer eats up 88X more memory (it also has 625X more page faults and 13% more LLC cache misses).

Security

• Bug: Stack buffer overflow, CVE-2013-2028 and detailed explanation
• Exploit script: This Ruby script

Results:

Summary: AddressSanitizer detects this bug, but both versions of MPX do not. The root cause is the run-time wrapper library: AddressSanitizer wraps all C library functions including recv, and the wrapper – not the Nginx instrumented code – detects the stack buffer overflow. In case of both GCC-MPX and ICC-MPX, only the most widely used functions are wrapped and bounds-checked. That is why when recv is called, the overflow happens in the unprotected C library function and goes undetected by MPX.

Usability

Issue 1: To successfully run Nginx under GCC-MPX with narrowing of bounds, we had to manually fix a variable-length array name[1] in the ngx_hash_elt_t struct to name[0].

Issue 2: ICC-MPX first crashed with a false positive in ngx_http_merge_locations function. The reason for this bug was a cast from a smaller type, which rendered the bounds too narrow for the new, larger type. Note that GCC-MPX did not experience the same problem because it enforces the first struct’s field to inherit the bounds of the whole object by default – in contrast to ICC-MPX which takes a more rigorous stance. For our evaluation, we used the version of ICC-MPX with narrowing of bounds disabled.

Up to table of contents

Memcached caching system

• Version: 1.4.15 (susceptible to this bug)
• Configuration: 8 threads (to keep all CPU cores busy)
• Dependencies¹: libevent 2.0.22-stable
• Workload:
  • memaslap benchmark from libmemcached 1.0.16
  • with 10% ~400B sets and 90% ~1,700B gets

Performance

Additional statistics²:

Performance Summary: AddressSanitizer performs on par with the native version. Both GCC-MPX and ICC-MPX achieved only 48−50% of maximum native throughput. In case of native and AddressSanitizer, performance of Memcached was limited by network. But it was not the case for MPX: Memcached exercised only 70% of the network bandwidth.

Memory Summary: While AddressSanitizer imposed only 30% memory overhead, both MPX variants used 350MB of memory (4.8X more than native). This huge memory overhead broke cache locality and resulted in 5.4X more page faults and 10−15% LLC misses, making MPX versions essentially memory-bound. (Indeed, the CPU utilization never exceeded 320%.)

Security

• Bug: Buffer overflow due to signedness error, CVE-2011-4971 and detailed explanation
• Exploit script: Specially crafted TCP/IP packet

Results:

Summary: All approaches detected buffer overflow in the affected function’s arguments and stopped the execution.

Usability

We experienced no usability problems.

Up to table of contents