|
|
Created:
8 years, 8 months ago by Sergey Ulanov Modified:
8 years, 8 months ago CC:
chromium-reviews, joi+watch-content_chromium.org, darin-cc_chromium.org, Wez Base URL:
svn://svn.chromium.org/chrome/trunk/src Visibility:
Public. |
DescriptionRedirect fopen("/dev/urandom") so that NSS can properly seed its RNG.
BUG=122169
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=131808
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=132106
Patch Set 1 : #
Total comments: 6
Patch Set 2 : #Patch Set 3 : #Patch Set 4 : #Patch Set 5 : #
Total comments: 12
Patch Set 6 : #
Total comments: 4
Patch Set 7 : #Patch Set 8 : override stat() #
Total comments: 2
Patch Set 9 : #Patch Set 10 : #Patch Set 11 : #Patch Set 12 : #Patch Set 13 : #Patch Set 14 : #
Total comments: 3
Messages
Total messages: 32 (0 generated)
On 2012/04/10 01:54:30, sergeyu wrote: I have a similar fix but use ld -wrap to generate the wrapper. Would that be a simpler approach so that we don't need to do any dlysm?
On 2012/04/10 02:00:52, Ben Chan wrote: > On 2012/04/10 01:54:30, sergeyu wrote: > > I have a similar fix but use ld -wrap to generate the wrapper. Would that be a > simpler approach so that we don't need to do any dlysm? Please ignore my comment. Your approach is better as ld wrap seems to only affect compile-time binding (and we don't want to recompile NSS)
http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main... File content/browser/zygote_main_linux.cc (right): http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main... content/browser/zygote_main_linux.cc:710: if (g_am_zygote_or_renderer && strcmp(path, "/dev/urandom") == 0) { static const char kDevUrandomFile[] = "/dev/urandom"; strncmp(path, kDevUrandomFile, sizeof(kDevUrandomFile)) == 0 http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main... content/browser/zygote_main_linux.cc:732: return fdopen(fd, mode); indentation
On 2012/04/10 02:17:15, Ben Chan wrote: > http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main... > File content/browser/zygote_main_linux.cc (right): > > http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main... > content/browser/zygote_main_linux.cc:710: if (g_am_zygote_or_renderer && > strcmp(path, "/dev/urandom") == 0) { > static const char kDevUrandomFile[] = "/dev/urandom"; > > strncmp(path, kDevUrandomFile, sizeof(kDevUrandomFile)) == 0 > > http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main... > content/browser/zygote_main_linux.cc:732: return fdopen(fd, mode); > indentation I couldn't get your patch compiled. It complains about "asm declaration ignored due to conflict with previous rename"
http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main... File content/browser/zygote_main_linux.cc (right): http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main... content/browser/zygote_main_linux.cc:710: if (g_am_zygote_or_renderer && strcmp(path, "/dev/urandom") == 0) { On 2012/04/10 02:17:15, Ben Chan wrote: > static const char kDevUrandomFile[] = "/dev/urandom"; > > strncmp(path, kDevUrandomFile, sizeof(kDevUrandomFile)) == 0 How would that be better than strcmp()? It would also match /dev/urandom0, etc. Is that what we want? http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main... content/browser/zygote_main_linux.cc:732: return fdopen(fd, mode); On 2012/04/10 02:17:15, Ben Chan wrote: > indentation Done.
http://chromiumcodereview.appspot.com/10031027/diff/2001/content/browser/zygo... File content/browser/zygote_main_linux.cc (right): http://chromiumcodereview.appspot.com/10031027/diff/2001/content/browser/zygo... content/browser/zygote_main_linux.cc:710: if (g_am_zygote_or_renderer && strcmp(path, "/dev/urandom") == 0) { On 2012/04/10 03:02:53, sergeyu wrote: > On 2012/04/10 02:17:15, Ben Chan wrote: > > static const char kDevUrandomFile[] = "/dev/urandom"; > > > > strncmp(path, kDevUrandomFile, sizeof(kDevUrandomFile)) == 0 > > How would that be better than strcmp()? It would also match /dev/urandom0, etc. > Is that what we want? strcmp should be fine. But I think it's still good to define kDevUrandomFile[], probably exposed from URandomFD.
On Mon, Apr 9, 2012 at 7:30 PM, <benchan@chromium.org> wrote: > On 2012/04/10 02:17:15, Ben Chan wrote: > > http://codereview.chromium.**org/10031027/diff/2001/** > content/browser/zygote_main_**linux.cc<http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main_linux.cc> > >> File content/browser/zygote_main_**linux.cc (right): >> > > > http://codereview.chromium.**org/10031027/diff/2001/** > content/browser/zygote_main_**linux.cc#newcode710<http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main_linux.cc#newcode710> > >> content/browser/zygote_main_**linux.cc:710: if (g_am_zygote_or_renderer >> && >> strcmp(path, "/dev/urandom") == 0) { >> static const char kDevUrandomFile[] = "/dev/urandom"; >> > > strncmp(path, kDevUrandomFile, sizeof(kDevUrandomFile)) == 0 >> > > > http://codereview.chromium.**org/10031027/diff/2001/** > content/browser/zygote_main_**linux.cc#newcode732<http://codereview.chromium.org/10031027/diff/2001/content/browser/zygote_main_linux.cc#newcode732> > >> content/browser/zygote_main_**linux.cc:732: return fdopen(fd, mode); >> indentation >> > > I couldn't get your patch compiled. It complains about "asm declaration > ignored > due to conflict with previous rename" > I tried compiling it with clang only. It could be that it doesn't compile with GCC. There is some magic used in stdio.h to redirect fopen() calls to fopen64(), and we have to workaround it somehow. I'll check tomorrow if there is some other way to do it. > > http://codereview.chromium.**org/10031027/<http://codereview.chromium.org/100... >
http://chromiumcodereview.appspot.com/10031027/diff/2001/content/browser/zygo... File content/browser/zygote_main_linux.cc (right): http://chromiumcodereview.appspot.com/10031027/diff/2001/content/browser/zygo... content/browser/zygote_main_linux.cc:710: if (g_am_zygote_or_renderer && strcmp(path, "/dev/urandom") == 0) { On 2012/04/10 03:24:41, Ben Chan wrote: > On 2012/04/10 03:02:53, sergeyu wrote: > > On 2012/04/10 02:17:15, Ben Chan wrote: > > > static const char kDevUrandomFile[] = "/dev/urandom"; > > > > > > strncmp(path, kDevUrandomFile, sizeof(kDevUrandomFile)) == 0 > > > > How would that be better than strcmp()? It would also match /dev/urandom0, > etc. > > Is that what we want? > > strcmp should be fine. But I think it's still good to define kDevUrandomFile[], > probably exposed from URandomFD. Done.
lgtm http://chromiumcodereview.appspot.com/10031027/diff/6007/content/browser/zygo... File content/browser/zygote_main_linux.cc (right): http://chromiumcodereview.appspot.com/10031027/diff/6007/content/browser/zygo... content/browser/zygote_main_linux.cc:69: static char kUrandomDevPath[] = "/dev/urandom"; const http://chromiumcodereview.appspot.com/10031027/diff/6007/content/browser/zygo... content/browser/zygote_main_linux.cc:700: dlsym(RTLD_NEXT, "fopen")); nit: indentation here should be four spaces from the start of the previous line I think. Same a couple of lines down. http://chromiumcodereview.appspot.com/10031027/diff/6007/content/browser/zygo... content/browser/zygote_main_linux.cc:720: FILE* fopen_override(const char* path, const char* mode) __asm__ ("fopen"); nit: blank line between these. http://chromiumcodereview.appspot.com/10031027/diff/6007/content/browser/zygo... content/browser/zygote_main_linux.cc:724: int fd = dup(GetUrandomFD()); needs to be wrapped with HANDLE_EINTR http://chromiumcodereview.appspot.com/10031027/diff/6007/content/browser/zygo... content/browser/zygote_main_linux.cc:726: LOG(ERROR) << "dup() failed."; PLOG http://chromiumcodereview.appspot.com/10031027/diff/6007/content/browser/zygo... content/browser/zygote_main_linux.cc:740: int fd = dup(GetUrandomFD()); needs to be wrapped with HANDLE_EINTR
On 2012/04/10 06:03:37, sergeyu wrote: > http://chromiumcodereview.appspot.com/10031027/diff/2001/content/browser/zygo... > File content/browser/zygote_main_linux.cc (right): > > http://chromiumcodereview.appspot.com/10031027/diff/2001/content/browser/zygo... > content/browser/zygote_main_linux.cc:710: if (g_am_zygote_or_renderer && > strcmp(path, "/dev/urandom") == 0) { > On 2012/04/10 03:24:41, Ben Chan wrote: > > On 2012/04/10 03:02:53, sergeyu wrote: > > > On 2012/04/10 02:17:15, Ben Chan wrote: > > > > static const char kDevUrandomFile[] = "/dev/urandom"; > > > > > > > > strncmp(path, kDevUrandomFile, sizeof(kDevUrandomFile)) == 0 > > > > > > How would that be better than strcmp()? It would also match /dev/urandom0, > > etc. > > > Is that what we want? > > > > strcmp should be fine. But I think it's still good to define > kDevUrandomFile[], > > probably exposed from URandomFD. > > Done. Thanks. Could you also try your patch through cros_amd64, linux_chromeos trybots?
http://codereview.chromium.org/10031027/diff/6007/content/browser/zygote_main... File content/browser/zygote_main_linux.cc (right): http://codereview.chromium.org/10031027/diff/6007/content/browser/zygote_main... content/browser/zygote_main_linux.cc:69: static char kUrandomDevPath[] = "/dev/urandom"; On 2012/04/10 15:18:13, agl wrote: > const Done. http://codereview.chromium.org/10031027/diff/6007/content/browser/zygote_main... content/browser/zygote_main_linux.cc:700: dlsym(RTLD_NEXT, "fopen")); On 2012/04/10 15:18:13, agl wrote: > nit: indentation here should be four spaces from the start of the previous line > I think. Same a couple of lines down. Done. http://codereview.chromium.org/10031027/diff/6007/content/browser/zygote_main... content/browser/zygote_main_linux.cc:720: FILE* fopen_override(const char* path, const char* mode) __asm__ ("fopen"); On 2012/04/10 15:18:13, agl wrote: > nit: blank line between these. Done. http://codereview.chromium.org/10031027/diff/6007/content/browser/zygote_main... content/browser/zygote_main_linux.cc:724: int fd = dup(GetUrandomFD()); On 2012/04/10 15:18:13, agl wrote: > needs to be wrapped with HANDLE_EINTR Done. http://codereview.chromium.org/10031027/diff/6007/content/browser/zygote_main... content/browser/zygote_main_linux.cc:726: LOG(ERROR) << "dup() failed."; On 2012/04/10 15:18:13, agl wrote: > PLOG Done. http://codereview.chromium.org/10031027/diff/6007/content/browser/zygote_main... content/browser/zygote_main_linux.cc:740: int fd = dup(GetUrandomFD()); On 2012/04/10 15:18:13, agl wrote: > needs to be wrapped with HANDLE_EINTR Done.
+brettw for OWNERS rubber stamp
http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... File content/browser/zygote_main_linux.cc (right): http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... content/browser/zygote_main_linux.cc:705: LOG(ERROR) << "Failed to get fopen() from glibc."; nit: Is fopen64() defined in all the libc versions we support? http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... content/browser/zygote_main_linux.cc:730: return fdopen(fd, mode); Not that this is the right place to check it, but doesn't /dev/urandom let you write data to add entropy? Is there any way that could be used by content in the sandbox to impact the random quality for other applications?
http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... File content/browser/zygote_main_linux.cc (right): http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... content/browser/zygote_main_linux.cc:730: return fdopen(fd, mode); On 2012/04/10 17:34:13, Wez wrote: > Not that this is the right place to check it, but doesn't /dev/urandom let you > write data to add entropy? We open the file O_RDONLY.
http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... File content/browser/zygote_main_linux.cc (right): http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... content/browser/zygote_main_linux.cc:705: LOG(ERROR) << "Failed to get fopen() from glibc."; On 2012/04/10 17:34:13, Wez wrote: > nit: Is fopen64() defined in all the libc versions we support? I believe all modern Linux distros should have it as it is part of glibc. BSD systems may not have it. This code should still work properly on such systems because fopen64() should never be called there. Anyway, I've changed this code so that it just uses fopen() when it fails to find fopen64().
On 2012/04/10 19:08:44, sergeyu wrote: > http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... > File content/browser/zygote_main_linux.cc (right): > > http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... > content/browser/zygote_main_linux.cc:705: LOG(ERROR) << "Failed to get fopen() > from glibc."; > On 2012/04/10 17:34:13, Wez wrote: > > nit: Is fopen64() defined in all the libc versions we support? > > I believe all modern Linux distros should have it as it is part of glibc. BSD > systems may not have it. This code should still work properly on such systems > because fopen64() should never be called there. Anyway, I've changed this code > so that it just uses fopen() when it fails to find fopen64(). I tried this patch on trybot and observed that NSS still failed to access /dev/urandom (errno = EACCES). I'm looking into whether fopen/fopen64 is actually redirected in the render process.
On 2012/04/10 22:26:40, Ben Chan wrote: > On 2012/04/10 19:08:44, sergeyu wrote: > > > http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... > > File content/browser/zygote_main_linux.cc (right): > > > > > http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... > > content/browser/zygote_main_linux.cc:705: LOG(ERROR) << "Failed to get fopen() > > from glibc."; > > On 2012/04/10 17:34:13, Wez wrote: > > > nit: Is fopen64() defined in all the libc versions we support? > > > > I believe all modern Linux distros should have it as it is part of glibc. BSD > > systems may not have it. This code should still work properly on such systems > > because fopen64() should never be called there. Anyway, I've changed this code > > so that it just uses fopen() when it fails to find fopen64(). > > I tried this patch on trybot and observed that NSS still failed to access > /dev/urandom (errno = EACCES). I'm looking into whether fopen/fopen64 is > actually redirected in the render process. I did some more debugging, NSS actually failed at stat() before fopen(). So we need to interpose stat() as well.
On Tue, Apr 10, 2012 at 4:42 PM, <benchan@chromium.org> wrote: > On 2012/04/10 22:26:40, Ben Chan wrote: > >> On 2012/04/10 19:08:44, sergeyu wrote: >> > >> > > http://codereview.chromium.**org/10031027/diff/5008/** > content/browser/zygote_main_**linux.cc<http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main_linux.cc> > >> > File content/browser/zygote_main_**linux.cc (right): >> > >> > >> > > http://codereview.chromium.**org/10031027/diff/5008/** > content/browser/zygote_main_**linux.cc#newcode705<http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main_linux.cc#newcode705> > >> > content/browser/zygote_main_**linux.cc:705: LOG(ERROR) << "Failed to >> get >> > fopen() > >> > from glibc."; >> > On 2012/04/10 17:34:13, Wez wrote: >> > > nit: Is fopen64() defined in all the libc versions we support? >> > >> > I believe all modern Linux distros should have it as it is part of >> glibc. >> > BSD > >> > systems may not have it. This code should still work properly on such >> > systems > >> > because fopen64() should never be called there. Anyway, I've changed >> this >> > code > >> > so that it just uses fopen() when it fails to find fopen64(). >> > > I tried this patch on trybot and observed that NSS still failed to access >> /dev/urandom (errno = EACCES). I'm looking into whether fopen/fopen64 is >> actually redirected in the render process. >> > > I did some more debugging, NSS actually failed at stat() before fopen(). > So we > need to interpose stat() as well. > I tested it by verifying that fopen() gets called for urandom during NSS initialization. It looks like NSS uses urandom in two places and in one case it calls stat() before opening urandom, and in the other one it doesn't. Great you caught this, I'll add stat() wrapper too. > > http://chromiumcodereview.**appspot.com/10031027/<http://chromiumcodereview.a... >
Ok, Please try the latest version now. It overrides stat() too. On 2012/04/10 23:42:10, Ben Chan wrote: > On 2012/04/10 22:26:40, Ben Chan wrote: > > On 2012/04/10 19:08:44, sergeyu wrote: > > > > > > http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... > > > File content/browser/zygote_main_linux.cc (right): > > > > > > > > > http://codereview.chromium.org/10031027/diff/5008/content/browser/zygote_main... > > > content/browser/zygote_main_linux.cc:705: LOG(ERROR) << "Failed to get > fopen() > > > from glibc."; > > > On 2012/04/10 17:34:13, Wez wrote: > > > > nit: Is fopen64() defined in all the libc versions we support? > > > > > > I believe all modern Linux distros should have it as it is part of glibc. > BSD > > > systems may not have it. This code should still work properly on such > systems > > > because fopen64() should never be called there. Anyway, I've changed this > code > > > so that it just uses fopen() when it fails to find fopen64(). > > > > I tried this patch on trybot and observed that NSS still failed to access > > /dev/urandom (errno = EACCES). I'm looking into whether fopen/fopen64 is > > actually redirected in the render process. > > I did some more debugging, NSS actually failed at stat() before fopen(). So we > need to interpose stat() as well.
I verified the latest patch on x86-generic and amd64-generic trybots. NSS was able to call stat() and fopen() on /dev/urandom. Thanks for the fix! http://chromiumcodereview.appspot.com/10031027/diff/14001/content/browser/zyg... File content/browser/zygote_main_linux.cc (right): http://chromiumcodereview.appspot.com/10031027/diff/14001/content/browser/zyg... content/browser/zygote_main_linux.cc:781: struct stat *buf) __asm__ ("__xstat"); These symbol names look a bit scary. Do they depend on the version of libc or the compiler used to compile libc? I wonder if we have the test coverage to detect when these symbols are changed.
http://chromiumcodereview.appspot.com/10031027/diff/14001/content/browser/zyg... File content/browser/zygote_main_linux.cc (right): http://chromiumcodereview.appspot.com/10031027/diff/14001/content/browser/zyg... content/browser/zygote_main_linux.cc:781: struct stat *buf) __asm__ ("__xstat"); On 2012/04/11 05:13:47, Ben Chan wrote: > These symbol names look a bit scary. Do they depend on the version of libc or > the compiler used to compile libc? > > I wonder if we have the test coverage to detect when these symbols are changed. There is LOG(FATAL) that would crash the process if we fail to resolve __xstat. Most Linux distros should work fine with this code. Some weird ones, that don't use glibc, may not work. This code will also fail on BSDs. I'm not confident I can fix it correctly for all systems without having such system to test on, and they are not officially supported anyway, so I just added TODO to fix it later.
On 2012/04/11 06:36:54, sergeyu wrote: > http://chromiumcodereview.appspot.com/10031027/diff/14001/content/browser/zyg... > File content/browser/zygote_main_linux.cc (right): > > http://chromiumcodereview.appspot.com/10031027/diff/14001/content/browser/zyg... > content/browser/zygote_main_linux.cc:781: struct stat *buf) __asm__ > ("__xstat"); > On 2012/04/11 05:13:47, Ben Chan wrote: > > These symbol names look a bit scary. Do they depend on the version of libc or > > the compiler used to compile libc? > > > > I wonder if we have the test coverage to detect when these symbols are > changed. > > There is LOG(FATAL) that would crash the process if we fail to resolve __xstat. > > Most Linux distros should work fine with this code. Some weird ones, that don't > use glibc, may not work. This code will also fail on BSDs. I'm not confident I > can fix it correctly for all systems without having such system to test on, and > they are not officially supported anyway, so I just added TODO to fix it later. sergey, are we still missing the owner lgtm for landing this CL?
yes On Wed, Apr 11, 2012 at 9:56 AM, <benchan@chromium.org> wrote: > On 2012/04/11 06:36:54, sergeyu wrote: > > http://chromiumcodereview.**appspot.com/10031027/diff/** > 14001/content/browser/zygote_**main_linux.cc<http://chromiumcodereview.appspot.com/10031027/diff/14001/content/browser/zygote_main_linux.cc> > >> File content/browser/zygote_main_**linux.cc (right): >> > > > http://chromiumcodereview.**appspot.com/10031027/diff/** > 14001/content/browser/zygote_**main_linux.cc#newcode781<http://chromiumcodereview.appspot.com/10031027/diff/14001/content/browser/zygote_main_linux.cc#newcode781> > >> content/browser/zygote_main_**linux.cc:781: struct stat *buf) __asm__ >> ("__xstat"); >> On 2012/04/11 05:13:47, Ben Chan wrote: >> > These symbol names look a bit scary. Do they depend on the version of >> libc >> > or > >> > the compiler used to compile libc? >> > >> > I wonder if we have the test coverage to detect when these symbols are >> changed. >> > > There is LOG(FATAL) that would crash the process if we fail to resolve >> > __xstat. > > Most Linux distros should work fine with this code. Some weird ones, that >> > don't > >> use glibc, may not work. This code will also fail on BSDs. I'm not >> confident I >> can fix it correctly for all systems without having such system to test >> on, >> > and > >> they are not officially supported anyway, so I just added TODO to fix it >> > later. > > sergey, are we still missing the owner lgtm for landing this CL? > > http://chromiumcodereview.**appspot.com/10031027/<http://chromiumcodereview.a... >
Owners rubberstamp LGTM
LGTM. I would recommend patching NSS so that it aborts if it fails to open its random source. That will stop this silently regressing in the future.
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/sergeyu@chromium.org/10031027/20002
On 2012/04/11 17:32:26, agl wrote: > LGTM. I would recommend patching NSS so that it aborts if it fails to open its > random source. That will stop this silently regressing in the future. Yes, we are planning to do that on the Chrome OS side. We are also discussing about switching to seccomp/bpf to sandbox Chrome on Chrome OS and interpose functions at the system call level, which has fewer variants.
Reverted because it was causing content_unittests fail on ASAN bots: http://build.chromium.org/p/chromium.memory/builders/ASAN%20Tests%20%281%29/b...
Uploaded new version of the patch that fixes problem with ASAN builds.
http://chromiumcodereview.appspot.com/10031027/diff/24005/content/browser/zyg... File content/browser/zygote_main_linux.cc (right): http://chromiumcodereview.appspot.com/10031027/diff/24005/content/browser/zyg... content/browser/zygote_main_linux.cc:740: // remoting in the sendbox. Typo: sendbox => sandbox http://chromiumcodereview.appspot.com/10031027/diff/24005/content/browser/zyg... content/browser/zygote_main_linux.cc:745: // the code below defines fopen_override() function with asm name Typo: remove "the" I suggest you just move "the" to the front of "fopen_override() function". http://chromiumcodereview.appspot.com/10031027/diff/24005/content/browser/zyg... content/browser/zygote_main_linux.cc:784: // the same trick to override it. You should explain why you need to intercept stat(). Which functions we need to intercept depends on what functions NSS calls on "/dev/urandom". So this CL may be broken by a future change to the NSS code in its unix_rand.c file. I agree with agl that we should also patch NSS to fail (or even call abort()) if it cannot use /dev/urandom on platforms that are known to have /dev/urandom: http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/freebl/u... http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/security/nss/lib/freebl/u... |