lardbucket: plaidctf 2015: radhos writeup


PlaidCTF 2015: radhos Writeup

Filed under: Ego, Hacks, Programming — Andy @ 6:44 pm

Along with several coworkers, I participated a bit in the PlaidCTF 2015 competition this past weekend. The competition is geared mainly toward binary exploitation, with reverse engineering a secondary priority. There are also a number of challenges which focus on cryptography, forensics, and other security-related issues. Unfortunately, my skills lie more toward the latter category than the former, so I wasn’t as broadly useful as I might have otherwise liked to be. Nonetheless, I was able to contribute solutions to three problems: “RE GEX” (a large regular expression, where it was necessary to find a string that didn’t match the regular expression), “PNG Uncorrupt” (a PNG file with several missing “\x0d” bytes before “\x0a” bytes), and “radhos” (a Python hash collision challenge). I’ve seen good writeups online for RE GEX and PNG Uncorrupt, but haven’t yet seen any good ones for radhos, so I’m contributing my solution here.

You can download the radhos challenge here (or here if the first link has disappeared). If running it locally, replace line 24 with “ return ''” to keep the server from having errors.

radhos is a simple HTTP server that emulates a small hashtable-based key-value store. Each value is stored using the hash of its key, where the hash is the lower 32 bits of Python’s hash() function. (This appears to be intended to give consistent behavior between 32-bit and 64-bit versions of Python.) The level’s flag is stored in the hashtable’s location for the key “you_want_it_LOLOLOL?”, but the server will reject requests that ask for that key by name. Therefore, the remaining solution is to find something that will generate a collision in hash(key). To encourage players to solve the challenge without simply guessing random keys, the server restarts (and gets a random hash() seed) every seven minutes.

Hash Collisions

Using HTTP requests, you can get the value for a key you select, or alternatively, you can set the value for a key. Setting the value for a key will also return the old value in that location in the hashtable, if any, as well as the value of hash(key). It’s this information leak that allows us to break Python’s randomized hash function, generate a new key that has the same hash value as “you_want_it_LOLOLOL?”, and retrieve the flag.

I initially thought that this would be an instance of the bug described at, but that doesn’t appear to be the case. I spent far too long looking at that issue, and trying to replicate similar situations that might allow a collision in this case. However, the issue does have an attachment,, that gave Python code for the hash algorithm, which was somewhat useful for analysis. The (pre-3.4) Python hash() function used two secret values to obscure the hash value: a “prefix” that affects the whole hash, and a “suffix” that is simply XOR’d in afterward to obscure the result and prevent attackers simply guessing the prefix. If an attacker is able to determine the “prefix”, they can generate hash collisions on their own. (Having the suffix is unnecessary if collisions are the only goal, as it is simply XOR’d on at the end.)

Here’s a bit of code for generating the hash:

HASHMASK = 0xffffffff

def hashbytes32(data):
    if len(data) == 0:
        return 0
    x = prefix
    x ^= ord(data[0]) << 7
    for byte in data:
        x = (1000003 * x) ^ ord(byte)
        x &= HASHMASK
    x ^= len(data)
    if x == -1:
        x = -2
    return x

Finding the prefix

Luckily, there was a talk on hash collisions (Hash-flooding DoS reloaded: attacks and defenses) that referenced a tool that could be used to determine the hash secret in Python. I was able to find the tool at, and verified that it worked.

Unfortunately, it worked based on hashes for “\x00” and “\x00\x00” (one and two null bytes), which we couldn’t submit to the radhos server (as far as I know). A small amount of math allows us to change the target used in the algorithm, however. Let’s say that we have h1 = hash("A") and h2 = hash("B"). Then we know:

h1 = ((prefix ^ (65 << 7)) * 1000003) ^ 65 ^ 1 ^ suffix
h2 = ((prefix ^ (66 << 7)) * 1000003) ^ 66 ^ 1 ^ suffix

Let’s XOR the two, and note that the 1 and suffix terms cancel out, as they are on both sides. Then we have:

h1 ^ h2 = ((prefix ^ (65 << 7)) * 1000003) ^ 65 ^ ((prefix ^ (66 << 7)) * 1000003) ^ 66

The tool I mentioned above uses this fact, and guesses the value of prefix one bit at a time, narrowing down possibilities as it goes. It then performs a verification step, testing those possibilities with a known hash of a third value. I was able to verify that I received the correct prefix value by testing locally, so I was satisfied with that part of the code.

As an aside, if the bit-by-bit guessing didn’t work to determine the prefix, it would be possible to use the h1 ^ h2 math above and several additional hashes to brute-force the prefix, but that would likely take a while. Distributing the problem among multiple cores (or computers) might get a result before the server restarted, however, so it might have been worthwhile if the proof of concept code were not available. (Note that brute-forcing the suffix is unnecessary in this case, as XOR’ing the hashes removes it from consideration.)

Finding an actual collision

Simply having the prefix value is insufficient, as we need to actually generate something that will collide. The obvious solution is a brute-force attack. This is easy to implement, and can be parallelized to get a result before the server restarts. Unfortunately, it’s also fairly slow in general, when another relatively simple solution is available.

Because the hash code is relatively simple, we can perform what is known as a “meet in the middle” attack. In this case, we start with the result we want (we can already calculate the hash for “you_want_it_LOLOLOL?”, because we have prefix and suffix
mask = 0xffffffff
def hashback(instr, target):
val = target
for byte in instr[::-1]:
val = ((val ^ ord(byte)) * 2021759595) & mask
return val

This works because 1000003 * 2021759595 = 1 mod 232, so multiplying by 2021759595 is equivalent to dividing by 1000003. (Note that in this function, for simplicity, we have already removed the suffix and the size of the string to be hashed.)

So we’ve worked backward and determined what the hash state should be for a given suffix. How does this help? Well, we can do the same thing for many suffixes, and have many possible targets for our brute-forcing. The most efficient way to do this is to try 216 suffixes. This turns the brute-forcing from an expected 232 operations with 1 unit of storage into 2*216 operations with 216 storage. RAM is relatively plentiful for this task, so it’s a good trade-off. I’ll let the code below speak for itself, but feel free to leave a comment if you have a question.

Putting it all together

The code I ended up using is below, after some minor cleaning up:

Based on "recovery of the secret seed of hash() in Python 2.7.3 and 3.2.3"

original authors: 
Jean-Philippe Aumasson, Daniel J. Bernstein

def hand(k, mask=0xffffffff):
  return hash(k) & mask

candidates = []
solutions = []
mask = 0xffffffff

def bytes_hash( p, prefix, suffix ):
  if len(p) == 0: return 0
  x = prefix ^ (ord( p[0] )<<7)
  for i in range( len(p) ):
    x = ( ( x * 1000003 ) ^ ord(p[i]) ) & mask
  x ^= len(p) ^ suffix
  if x == -1: x = -2
  return x

def solvebit( h1, h2, prefix, bits ):
  f1 = 1000003
  target = h1^h2
  if bits == 32:
    if ((((prefix ^ (65 << 7)) * f1) ^ 65 ^ 1)^(((prefix ^ (66 << 7)) * f1) ^ 66 ^ 1)^target) & mask: return
    suffix = h1^(((prefix ^ (65 << 7)) * f1) ^ 65 ^ 1)
    suffix&= mask
    candidates.append( (prefix,suffix) )
    if ((((prefix ^ (65 << 7)) * f1) ^ 65 ^ 1)^(((prefix ^ (66 << 7)) * f1) ^ 66 ^ 1)^target) & ((1<<bits)-1):
    solvebit(h1,h2,prefix,bits + 1)
    solvebit(h1,h2,prefix + (1 << bits),bits + 1)

h1 = 0x99dd9d1e #hand("A")      & mask
h2 = 0xf0e1309d #hand("B")      & mask
h3 = 0xf2a1cda6 #hand("python") & mask

solvebit( h1, h2, 0, 0 )

print("%d candidate solutions" % (len(candidates)))

def hashback(instr, target):
  val = target
  for byte in instr[::-1]:
    val = ((val ^ ord(byte)) * 2021759595) & mask
  return val

for s in candidates:
  if bytes_hash("python",s[0],s[1]) == h3 & mask: 
    if ok: 
      print("Prefix: 0x%08x Suffix: 0x%08x" % (s[0],s[1]))
      suffixes = {}
      prelen = bytes_hash("you_want_it_LOLOLOL?", s[0], 0) ^ 10
      for sint in range(0, 1 << 16):
        stry = "%04x" % sint
        suffixes[hashback(stry, prelen)] = stry
      for sint in range(0, 1 << 24):
        stry = "%06x" % sint
        h = bytes_hash(stry, s[0], 0) ^ 6
        if h in suffixes:
          print "  Try: " + stry + suffixes[h]

The values of h1, h2, and h3 need to be filled in by making requests to radhos for “A”, “B”, and “python”, respectively. An integer will be returned for the key, and the last four bytes should be filled in to the script. (This can be done by using “(value + 2**64) & 0xffffffff” in the Python REPL. Adding the 2**64 corrects any issues with the server’s response including a signed integer.)

After that, it’s a simple cURL command to win: “curl -d'{"key":"010b381a50","value":"NOTAFLAG"}' --header "Content-Type: application/json"” would get you the flag as a response.

Finally, note that this issue was (probably?) resolved in Python 3.4 by using SipHash. Certainly, any attack on SipHash would be more complicated, and it’s not clear that one is even possible.

If you have any questions, feel free to ask below.

No Comments »

No comments yet.

RSS feed for comments on this post.

Leave a comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

My Stuff
Blog Stuff