Virtually all compilers — programs that transform human-readable source code into computer-executable machine code — are vulnerable to an insidious attack in which an adversary can introduce targeted vulnerabilities into any software without being detected, new research released today warns. The vulnerability disclosure was coordinated with multiple organizations, some of whom are now releasing updates to address the security weakness.

Researchers with the University of Cambridge discovered a bug that affects most computer code compilers and many software development environments. At issue is a component of the digital text encoding standard Unicode, which allows computers to exchange information regardless of the language used. Unicode currently defines more than 143,000 characters across 154 different language scripts (in addition to many non-script character sets, such as emojis).

Specifically, the weakness involves Unicode’s bi-directional or “Bidi” algorithm, which handles displaying text that includes mixed scripts with different display orders, such as Arabic — which is read right to left — and English (left to right).

But computer systems need to have a deterministic way of resolving conflicting directionality in text. Enter the “Bidi override,” which can be used to make left-to-right text read right-to-left, and vice versa.

“In some scenarios, the default ordering set by the Bidi Algorithm may not be sufficient,” the Cambridge researchers wrote. “For these cases, Bidi override control characters enable switching the display ordering of groups of characters.”

Bidi overrides enable even single-script characters to be displayed in an order different from their logical encoding. As the researchers point out, this fact has previously been exploited to disguise the file extensions of malware disseminated via email.

Here’s the problem: Most programming languages let you put these Bidi overrides in comments and strings. This is bad because most programming languages allow comments within which all text — including control characters — is ignored by compilers and interpreters. Also, it’s bad because most programming languages allow string literals that may contain arbitrary characters, including control characters.

“So you can use them in source code that appears innocuous to a human reviewer [that] can actually do something nasty,” said Ross Anderson, a professor of computer security at Cambridge and co-author of the research. “That’s bad news for projects like Linux and Webkit that accept contributions from random people, subject them to manual review, then incorporate them into critical code. This vulnerability is, as far as I know, the first one to affect almost everything.”

The research paper, which dubbed the vulnerability “Trojan Source,” notes that while both comments and strings will have syntax-specific semantics indicating their start and end, these bounds are not respected by Bidi overrides. From the paper:

“Therefore, by placing Bidi override characters exclusively within comments and strings, we can smuggle them into source code in a manner that most compilers will accept. Our key insight is that we can reorder source code characters in such a way that the resulting display order also represents syntactically valid source code.”

“Bringing all this together, we arrive at a novel supply-chain attack on source code. By injecting Unicode Bidi override characters into comments and strings, an adversary can produce syntactically-valid source code in most modern languages for which the display order of characters presents logic that diverges from the real logic. In effect, we anagram program A into program B.”

Anderson said such an attack could be challenging for a human code reviewer to detect, as the rendered source code looks perfectly acceptable.

“If the change in logic is subtle enough to go undetected in subsequent testing, an adversary could introduce targeted vulnerabilities without being detected,” he said.

Equally concerning is that Bidi override characters persist through the copy-and-paste functions on most modern browsers, editors, and operating systems.

“Any developer who copies code from an untrusted source into a protected code base may inadvertently introduce an invisible vulnerability,” Anderson told KrebsOnSecurity. “Such code copying is a significant source of real-world security exploits.”

Image: XKCD.com/2347/

Matthew Green, an associate professor at the Johns Hopkins Information Security Institute, said the Cambridge research clearly shows that most compilers can be tricked with Unicode into processing code in a different way than a reader would expect it to be processed.

“Before reading this paper, the idea that Unicode could be exploited in some way wouldn’t have surprised me,” Green told KrebsOnSecurity. “What does surprise me is how many compilers will happily parse Unicode without any defenses, and how effective their right-to-left encoding technique is at sneaking code into codebases. That’s a really clever trick I didn’t even know was possible. Yikes.”

Green said the good news is that the researchers conducted a widespread vulnerability scan, but were unable to find evidence that anyone was exploiting this. Yet.

“The bad news is that there were no defenses to it, and now that people know about it they might start exploiting it,” Green said. “Hopefully compiler and code editor developers will patch this quickly! But since some people don’t update their development tools regularly there will be some risk for a while at least.” Continue reading →