Mask freelist entries in tcmalloc

Mask freelist entries in tcmalloc

The goal here is to prevent freelist spraying in exploits, and as a bonus to make UAF fail a bit more early and obviously.

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=157857

[jar (doing other things), 2012-09-20 17:13:57]

http://codereview.chromium.org/10944024/diff/1003/third_party/tcmalloc/chromi...
File third_party/tcmalloc/chromium/src/free_list.cc (right):

http://codereview.chromium.org/10944024/diff/1003/third_party/tcmalloc/chromi...
third_party/tcmalloc/chromium/src/free_list.cc:87: const uintptr_t mask =
static_cast<intptr_t>(!p) - 1;
Why play these fancy math games? Why not:

if (!p)
  return p;

[jschuh, 2012-09-20 17:26:13]

http://codereview.chromium.org/10944024/diff/1003/third_party/tcmalloc/chromi...
File third_party/tcmalloc/chromium/src/free_list.cc (right):

http://codereview.chromium.org/10944024/diff/1003/third_party/tcmalloc/chromi...
third_party/tcmalloc/chromium/src/free_list.cc:87: const uintptr_t mask =
static_cast<intptr_t>(!p) - 1;
On 2012/09/20 17:13:57, jar wrote:
> Why play these fancy math games? Why not:
> 
> if (!p)
>   return p;

Because I thought you'd be displeased with me for adding a branch. But I'm happy
to assume the compiler will do the cleverness instead.

[jar (doing other things), 2012-09-20 17:33:30]

lgtm

http://codereview.chromium.org/10944024/diff/1003/third_party/tcmalloc/chromi...
File third_party/tcmalloc/chromium/src/free_list.cc (right):

http://codereview.chromium.org/10944024/diff/1003/third_party/tcmalloc/chromi...
third_party/tcmalloc/chromium/src/free_list.cc:87: const uintptr_t mask =
static_cast<intptr_t>(!p) - 1;
On 2012/09/20 17:26:14, Justin Schuh wrote:
> On 2012/09/20 17:13:57, jar wrote:
> > Why play these fancy math games? Why not:
> > 
> > if (!p)
> >   return p;
> 
> Because I thought you'd be displeased with me for adding a branch. But I'm
happy
> to assume the compiler will do the cleverness instead.

I would never be displeased with you ;-).

IF you want to play compiler optimizer, you could hope that we don't hit the end
of the lists that often, and optimized for the "fast" case of XOR:

if (p)
  return ... ^ p;
return p;

I think the compiler optimizes for the if-true case... but I suspect this will
not break us (or be any where near measurable, compared with the forward-back
pointer stuff we've already landed).

[jschuh, 2012-09-20 18:15:24]

On 2012/09/20 17:33:30, jar wrote:
> lgtm
> 
>
http://codereview.chromium.org/10944024/diff/1003/third_party/tcmalloc/chromi...
> File third_party/tcmalloc/chromium/src/free_list.cc (right):
> 
>
http://codereview.chromium.org/10944024/diff/1003/third_party/tcmalloc/chromi...
> third_party/tcmalloc/chromium/src/free_list.cc:87: const uintptr_t mask =
> static_cast<intptr_t>(!p) - 1;
> On 2012/09/20 17:26:14, Justin Schuh wrote:
> > On 2012/09/20 17:13:57, jar wrote:
> > > Why play these fancy math games? Why not:
> > > 
> > > if (!p)
> > >   return p;
> > 
> > Because I thought you'd be displeased with me for adding a branch. But I'm
> happy
> > to assume the compiler will do the cleverness instead.
> 
> I would never be displeased with you ;-).
> 
> IF you want to play compiler optimizer, you could hope that we don't hit the
end
> of the lists that often, and optimized for the "fast" case of XOR:
> 
> if (p)
>   return ... ^ p;
> return p;
> 
> I think the compiler optimizes for the if-true case... but I suspect this will
> not break us (or be any where near measurable, compared with the forward-back
> pointer stuff we've already landed).

Okay, I'll let the compiler play compiler then, and upload a minor tweak. :)

[commit-bot: I haz the power, 2012-09-20 19:05:22]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/jschuh@chromium.org/10944024/15002

[commit-bot: I haz the power, 2012-09-20 22:34:24]

Change committed as 157857