Support generating the dart vm snapshot binary on Android

Support generating the dart vm snapshot binary on Android

This change splits the generate_snapshot_file build target into two targets:

generate_snapshot_bin - executes gen_snapshot to create snapshot_bin_file
generate_snapshot_file - creates snapshot_cc_file

It's not strictly necessary to split the generation into two steps, but
it is convenient to do so, because only the first step needs to know about
Android. The second step can be shared between all builds.

For Android we copy the gen_snapshot executable and all its input files
to Android, run it there, and copy the output files back to the host.

Committed: https://code.google.com/p/dart/source/detail?r=10676

[jackpal, 2012-08-09 18:23:20]

Patch 4 of 6 for building Dart VM for Android

This patch lets us create the Dart VM snapshot bin file on
Android.

[Emily Fortuna, 2012-08-09 20:15:57]

again, I'm going to defer to VM people on this one

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
File runtime/tools/create_snapshot_bin.py (right):

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:7: # Script to create snapshot bin file.
use """ ... """

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:58: def makeString(input_file):
inconsistent capitalization here compared to above:
MakeString 
(other cases below)

[ahe, 2012-08-09 20:57:16]

The Python code doesn't follow Google style guides. You'll have to find someone
else to sign off on that :-)

The GYP changes do not look right to me, but I'm aware of a problem that causes
certain sections to require one less .. because the GYP and GYPI files aren't in
the same directory. If it works on Linux, I think you should be fine.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/bin/bin.gypi
File runtime/bin/bin.gypi (right):

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/bin/bin.gypi#n...
runtime/bin/bin.gypi:370: 'tools/create_snapshot_bin.py',
This doesn't match line 362. Are you sure the dependencies are right?

I know there is a problem because this gypi file is included by a gyp file one
directory up.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/bin/bin.gypi#n...
runtime/bin/bin.gypi:371: '--executable', '<(PRODUCT_DIR)/gen_snapshot',
This should match line 363.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/bin/bin.gypi#n...
runtime/bin/bin.gypi:399: 'tools/create_snapshot_file.py',
This doesn't match line 390.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
File runtime/tools/create_snapshot_bin.py (right):

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:1: #!/usr/bin/env python
Since this is a new file, you should use the external Python style guide.

See http://google-styleguide.googlecode.com/svn/trunk/pyguide.html

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:3: # Copyright (c) 2011, the Dart project
authors.  Please see the AUTHORS file
This comment is so last year.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:12: from os.path import abspath, basename,
dirname, join
This violates "Use imports for packages and modules only."

[Ivan Posva, 2012-08-09 21:48:14]

Almost LGTM...

-Ivan

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/bin/bin.gypi
File runtime/bin/bin.gypi (right):

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/bin/bin.gypi#n...
runtime/bin/bin.gypi:399: 'tools/create_snapshot_file.py',
On 2012/08/09 20:57:16, ahe wrote:
> This doesn't match line 390.

I agree with Peter that the paths in input and action sections should match.
Please fix.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
File runtime/tools/create_snapshot_bin.py (right):

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:1: #!/usr/bin/env python
On 2012/08/09 20:57:16, ahe wrote:
> Since this is a new file, you should use the external Python style guide.
> 
> See http://google-styleguide.googlecode.com/svn/trunk/pyguide.html

Peter, I disagree. This is essentially a copy of the original
create_snapshot_file.py. I am not sure whether the fact that we do not see this
is due to gcl or git, but I don't this is the time to have Jack cleanup our past
sins.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:7: # Script to create snapshot bin file.
On 2012/08/09 20:15:57, Emily Fortuna wrote:
> use """ ... """

ditto from above: Emily, what is the benefit of this requested change?

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:58: def makeString(input_file):
On 2012/08/09 20:15:57, Emily Fortuna wrote:
> inconsistent capitalization here compared to above:
> MakeString 
> (other cases below)

makeString use has been moved out to the other forked file, please remove it
from here.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:109: android_url_arg =
join(android_workspace, basename(url_arg))
I don't think the logic here works because you are not getting any of the
sources that are loaded into the library. There might be other issues too with
URL remappings, but I have not looked at too much detail for that. How about
just disabling url-mapping for snapshot creation on Android for now and failing
instead? I don't think we use it in the regular Dart build anywhere.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:116: android_script_name =
join(android_workspace, basename(options.script))
ditto.

[ahe, 2012-08-09 22:29:10]

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
File runtime/tools/create_snapshot_bin.py (right):

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:1: #!/usr/bin/env python
On 2012/08/09 21:48:15, Ivan Posva wrote:
> On 2012/08/09 20:57:16, ahe wrote:
> > Since this is a new file, you should use the external Python style guide.
> > 
> > See http://google-styleguide.googlecode.com/svn/trunk/pyguide.html
> 
> Peter, I disagree. This is essentially a copy of the original
> create_snapshot_file.py. I am not sure whether the fact that we do not see
this
> is due to gcl or git, but I don't this is the time to have Jack cleanup our
past
> sins.

Ivan, I defer to your judgement regarding the style guide.

[Søren Gjesse, 2012-08-10 07:18:24]

lgtm

[jackpal, 2012-08-13 20:19:45]

I've incorporated everyone's feedback and am publishing a new version of the
change shortly.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/bin/bin.gypi
File runtime/bin/bin.gypi (right):

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/bin/bin.gypi#n...
runtime/bin/bin.gypi:370: 'tools/create_snapshot_bin.py',
On 2012/08/09 20:57:16, ahe wrote:
> This doesn't match line 362. Are you sure the dependencies are right?
> 
> I know there is a problem because this gypi file is included by a gyp file one
> directory up.

See the discussion below. This line is correct as written,
due to this gypi file being included from one directory up.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/bin/bin.gypi#n...
runtime/bin/bin.gypi:371: '--executable', '<(PRODUCT_DIR)/gen_snapshot',
On 2012/08/09 20:57:16, ahe wrote:
> This should match line 363.

Done.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/bin/bin.gypi#n...
runtime/bin/bin.gypi:399: 'tools/create_snapshot_file.py',
On 2012/08/09 21:48:15, Ivan Posva wrote:
> On 2012/08/09 20:57:16, ahe wrote:
> > This doesn't match line 390.
> 
> I agree with Peter that the paths in input and action sections should match.
> Please fix.

Ivan and I discussed this and came to the conclusion that the current code is
correct.

action scripts are executed relative to the directory where the top-level .gyp
file is run, while dependencies and essentially all other file references while
running gyp are resolved relative to gyp[i] file containing them.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
File runtime/tools/create_snapshot_bin.py (right):

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:3: # Copyright (c) 2011, the Dart project
authors.  Please see the AUTHORS file
On 2012/08/09 20:57:16, ahe wrote:
> This comment is so last year.

Done.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:58: def makeString(input_file):
On 2012/08/09 21:48:15, Ivan Posva wrote:
> On 2012/08/09 20:15:57, Emily Fortuna wrote:
> > inconsistent capitalization here compared to above:
> > MakeString 
> > (other cases below)
> 
> makeString use has been moved out to the other forked file, please remove it
> from here.

Done.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:58: def makeString(input_file):
On 2012/08/09 20:15:57, Emily Fortuna wrote:
> inconsistent capitalization here compared to above:
> MakeString 
> (other cases below)

Done.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:109: android_url_arg =
join(android_workspace, basename(url_arg))
On 2012/08/09 21:48:15, Ivan Posva wrote:
> I don't think the logic here works because you are not getting any of the
> sources that are loaded into the library. There might be other issues too with
> URL remappings, but I have not looked at too much detail for that. How about
> just disabling url-mapping for snapshot creation on Android for now and
failing
> instead? I don't think we use it in the regular Dart build anywhere.

Done.

https://chromiumcodereview.appspot.com/10827250/diff/1/runtime/tools/create_s...
runtime/tools/create_snapshot_bin.py:116: android_script_name =
join(android_workspace, basename(options.script))
On 2012/08/09 21:48:15, Ivan Posva wrote:
> ditto.

Done.

[Emily Fortuna, 2012-08-13 22:35:24]

lgtm