Intial experiment for building dynamic images without glibc under pnacl

Intial experiment for building dynamic imaged without glibc under pnacl
Committed: https://src.chromium.org/viewvc/native_client?view=rev&revision=8477

[Robert Muth (chromium), 2012-05-01 20:10:33]

PTAL

[jvoung - send to chromium..., 2012-05-01 23:30:49]

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
File tests/pnacl_ld_experiments/Makefile (right):

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:7: # which is then loaded by a regular
(non-sandboxed) images
Is it really "loaded by a regular (non-sandboxed) images" ?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:23: RUNNER=$(BOOTER) $(LOADER) -S -E
LD_DEBUG=all --r_debug=0xXXXXXXXXXXXXXXXX -a -- $(LDSO) \
80 col

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:30: $(PNACL_CC)  -c -I../../..
$(DEBUG_FLAGS) -o $@ $^
why that -I path? to get to "native_client/blah/blah2.h"?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:30: $(PNACL_CC)  -c -I../../..
$(DEBUG_FLAGS) -o $@ $^
I don't think DEBUG_FLAGS is set anywhere.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:42: #	$(PNACL_CC) $(LD_FLAGS) -nostdlib
-shared -fPIC  -arch x86-32 -o $@ $^
Commented out stuff, but I don't think LD_FLAGS is set anywhere.  Perhaps also
set a soname for this library, and start with a .pso then go to a .so with
pnacl-translate.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:48: # The one above from $(PATH_CRT_SRC)
So, why are both crtbegins linked in?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:49: INIT = crtbegin.bc $(PATH_CRT)/crt1.bc
$(PATH_CRT)/crti.bc  $(PATH_CRT)/crtdummy.bc  $(PATH_CRT)/crtbegin.bc
80 col

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:51: FINI = crtend.bc
why is this crtend used instead?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:56: #
--add-extra-dt-needed=ld-nacl-x86-32.so.1 which does not trigger the
What happens if you build a .pexe first instead?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:75: rm -f *.bc *.o *.ll *.so main *.pso
*.raw
clean out *.nexe too.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
File tests/pnacl_ld_experiments/fortytwo.c (right):

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/fortytwo.c:1: /* Copyright 2011 The Native Client
Authors.  All rights reserved.
2012

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/fortytwo.c:6: #include "simple.h"
Missing header n the CL?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
File tests/pnacl_ld_experiments/hello1.c (right):

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:1: #include <unistd.h>
Copyright notice.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:4: #define NACL_INSTR_BLOCK_SHIFT         5
Note about this experiment being tailored to x86?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:16: #include <bits/nacl_syscalls.h>
Does this have to be included down here, or can it be included with the rest of
the includes?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:18: /*
====================================================================== */
Comment about the purpose of this particular sub-experiment?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:50: ssize_t  write(int l, const void* buf,
size_t n)  {
Is that parameter a "l" or a "1"? Could you name it something else, like "fd"?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:72: myprint(buffer);
perhaps print "tdata1 (expecting 1): " first, same with "tdata2 (expecting 3): "

[Robert Muth (chromium), 2012-05-02 16:18:32]

PTAL

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
File tests/pnacl_ld_experiments/Makefile (right):

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:7: # which is then loaded by a regular
(non-sandboxed) images
On 2012/05/01 23:30:49, jvoung wrote:
> Is it really "loaded by a regular (non-sandboxed) images" ?

Done.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:23: RUNNER=$(BOOTER) $(LOADER) -S -E
LD_DEBUG=all --r_debug=0xXXXXXXXXXXXXXXXX -a -- $(LDSO) \
On 2012/05/01 23:30:49, jvoung wrote:
> 80 col

Done.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:30: $(PNACL_CC)  -c -I../../..
$(DEBUG_FLAGS) -o $@ $^
On 2012/05/01 23:30:49, jvoung wrote:
> I don't think DEBUG_FLAGS is set anywhere.

Done.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:42: #	$(PNACL_CC) $(LD_FLAGS) -nostdlib
-shared -fPIC  -arch x86-32 -o $@ $^
I'd like to explore this in the next CL

On 2012/05/01 23:30:49, jvoung wrote:
> Commented out stuff, but I don't think LD_FLAGS is set anywhere.  Perhaps also
> set a soname for this library, and start with a .pso then go to a .so with
> pnacl-translate.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:48: # The one above from $(PATH_CRT_SRC)
Good question. I think this is what we are currently doing
in our drivers.
Definitely something to revisit. 

On 2012/05/01 23:30:49, jvoung wrote:
> So, why are both crtbegins linked in?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:51: FINI = crtend.bc
On 2012/05/01 23:30:49, jvoung wrote:
> why is this crtend used instead?

Just mimicking the drivers for now

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:56: #
--add-extra-dt-needed=ld-nacl-x86-32.so.1 which does not trigger the
IIRC this did not help - I think the problem is that there
there is not other shared lib on the command line to the linker which would
"force" a dynamic image.

On 2012/05/01 23:30:49, jvoung wrote:
> What happens if you build a .pexe first instead?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:75: rm -f *.bc *.o *.ll *.so main *.pso
*.raw
On 2012/05/01 23:30:49, jvoung wrote:
> clean out *.nexe too.

Done.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
File tests/pnacl_ld_experiments/fortytwo.c (right):

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/fortytwo.c:1: /* Copyright 2011 The Native Client
Authors.  All rights reserved.
On 2012/05/01 23:30:49, jvoung wrote:
> 2012

Done.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/fortytwo.c:6: #include "simple.h"
On 2012/05/01 23:30:49, jvoung wrote:
> Missing header n the CL?

Done.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
File tests/pnacl_ld_experiments/hello1.c (right):

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:4: #define NACL_INSTR_BLOCK_SHIFT         5
On 2012/05/01 23:30:49, jvoung wrote:
> Note about this experiment being tailored to x86?

IIRC this works for all archs

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:16: #include <bits/nacl_syscalls.h>
On 2012/05/01 23:30:49, jvoung wrote:
> Does this have to be included down here, or can it be included with the rest
of
> the includes?

Done.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:18: /*
====================================================================== */
On 2012/05/01 23:30:49, jvoung wrote:
> Comment about the purpose of this particular sub-experiment?

Done.

[jvoung - send to chromium..., 2012-05-02 17:20:42]

comments below, but LGTM

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
File tests/pnacl_ld_experiments/Makefile (right):

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:48: # The one above from $(PATH_CRT_SRC)
On 2012/05/02 16:18:32, Robert Muth (chromium) wrote:
> Good question. I think this is what we are currently doing
> in our drivers.
> Definitely something to revisit. 
> 
> On 2012/05/01 23:30:49, jvoung wrote:
> > So, why are both crtbegins linked in?
> 

Hmm, I'm pretty sure the current driver doesn't do that.  For the bitcode link:

/usr/local/google/home/nacl2/native_client/toolchain/pnacl_linux_x86_64/pkg/binutils/bin/arm-pc-nacl-ld.gold
\
    --native-client \
    --oformat \
    elf32-nacl \
//... SKIP
    toolchain/pnacl_linux_x86_64/glibc/lib/crt1.bc \
    toolchain/pnacl_linux_x86_64/glibc/lib/crti.bc \
    toolchain/pnacl_linux_x86_64/glibc/lib/crtdummy.bc \
    toolchain/pnacl_linux_x86_64/glibc/lib/crtbegin.bc \
   
scons-out/nacl-x86-32-pnacl-glibc-clang/obj/tests/browser_startup_time/hello_world.bc
\
// ... SKIP

(and there is no crtend.bc???)


However, for the native link step:


/usr/local/google/home/nacl2/native_client/toolchain/pnacl_linux_x86_64/pkg/binutils/bin/arm-pc-nacl-ld.alt
\
    --rosegment \
    --bsssegment \
    --native-client \
// SKIP ...
    toolchain/pnacl_linux_x86_64/lib-x86-32/crtbegin.o \
// ...
    toolchain/pnacl_linux_x86_64/lib-x86-32/crtend.o \
// ...


Where the .o are compiled from your PATH_CRT_SRC, but the .bc are from
PATH_CRT_SRC/bitcode.  I guess clarify that one of the crtbegins is for
exception handling (to be paired with libgcc_s), while the other is for
ctor/dtors?

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:56: #
--add-extra-dt-needed=ld-nacl-x86-32.so.1 which does not trigger the
Would it be posssible at the native link step to set --dynamic-linker=...?  I
guess if you do that, you still need to call PNACL_LD manually to pass flags to
the linker.  With a .pso linked, it will certainly be dynamic though.

On 2012/05/02 16:18:32, Robert Muth (chromium) wrote:
> IIRC this did not help - I think the problem is that there
> there is not other shared lib on the command line to the linker which would
> "force" a dynamic image.
> 
> On 2012/05/01 23:30:49, jvoung wrote:
> > What happens if you build a .pexe first instead?
>

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
File tests/pnacl_ld_experiments/hello1.c (right):

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:50: ssize_t  write(int l, const void* buf,
size_t n)  {
On 2012/05/01 23:30:49, jvoung wrote:
> Is that parameter a "l" or a "1"? Could you name it something else, like "fd"?

Well, I do think it will be clearer, since you do (write)(1, s, mystrlen(s))
above, which is a 1 instead of l.

https://chromiumcodereview.appspot.com/10268028/diff/7001/tests/pnacl_ld_expe...
File tests/pnacl_ld_experiments/simple.h (right):

https://chromiumcodereview.appspot.com/10268028/diff/7001/tests/pnacl_ld_expe...
tests/pnacl_ld_experiments/simple.h:2: * Copyright (c) 2011 The Native Client
Authors. All rights reserved.
2012

[jvoung - send to chromium..., 2012-05-02 17:27:27]

https://chromiumcodereview.appspot.com/10268028/diff/7001/tests/pnacl_ld_expe...
File tests/pnacl_ld_experiments/Makefile (right):

https://chromiumcodereview.appspot.com/10268028/diff/7001/tests/pnacl_ld_expe...
tests/pnacl_ld_experiments/Makefile:66: $(PNACL_LD) $(COMMON_FLAGS) -arch x86-32
  $(LIBS) hello1.x86-32.nexe.dummy---linked.o -o $@
Actually, so the crtbegin.o and crtend.o mentioned above should really be down
here, right?

[robertm, 2012-05-02 17:33:12]

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
File tests/pnacl_ld_experiments/Makefile (right):

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:49: INIT = crtbegin.bc $(PATH_CRT)/crt1.bc
$(PATH_CRT)/crti.bc  $(PATH_CRT)/crtdummy.bc  $(PATH_CRT)/crtbegin.bc
On 2012/05/01 23:30:49, jvoung wrote:
> 80 col

Done.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/Makefile:56: #
--add-extra-dt-needed=ld-nacl-x86-32.so.1 which does not trigger the
the explore this together once the CL is in.

On 2012/05/02 17:20:42, jvoung wrote:
> Would it be posssible at the native link step to set --dynamic-linker=...?  I
> guess if you do that, you still need to call PNACL_LD manually to pass flags
to
> the linker.  With a .pso linked, it will certainly be dynamic though.
> 
> On 2012/05/02 16:18:32, Robert Muth (chromium) wrote:
> > IIRC this did not help - I think the problem is that there
> > there is not other shared lib on the command line to the linker which would
> > "force" a dynamic image.
> > 
> > On 2012/05/01 23:30:49, jvoung wrote:
> > > What happens if you build a .pexe first instead?
> > 
>

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
File tests/pnacl_ld_experiments/hello1.c (right):

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:1: #include <unistd.h>
On 2012/05/01 23:30:49, jvoung wrote:
> Copyright notice.

Done.

https://chromiumcodereview.appspot.com/10268028/diff/1/tests/pnacl_ld_experim...
tests/pnacl_ld_experiments/hello1.c:72: myprint(buffer);
On 2012/05/01 23:30:49, jvoung wrote:
> perhaps print "tdata1 (expecting 1): " first, same with "tdata2 (expecting 3):
"

Done.

https://chromiumcodereview.appspot.com/10268028/diff/7001/tests/pnacl_ld_expe...
File tests/pnacl_ld_experiments/Makefile (right):

https://chromiumcodereview.appspot.com/10268028/diff/7001/tests/pnacl_ld_expe...
tests/pnacl_ld_experiments/Makefile:66: $(PNACL_LD) $(COMMON_FLAGS) -arch x86-32
  $(LIBS) hello1.x86-32.nexe.dummy---linked.o -o $@
yes, not sure whether hello1.x86-32.nexe will still build without it, will
explore this in the next CL

On 2012/05/02 17:27:27, jvoung wrote:
> Actually, so the crtbegin.o and crtend.o mentioned above should really be down
> here, right?