Generate API docs during build.

Generate API docs during build.

Committed: https://code.google.com/p/dart/source/detail?r=6065

[Bob Nystrom, 2012-03-16 01:30:31]

This is my first attempt at hooking up generating our API docs to the regular
build. Let me know what I didn't do right. :)

The only stuff you need to look at here are the .gyp changes. The other changes
here are handled by this review:
https://chromiumcodereview.appspot.com/9705078/. SVN is down right now, so I
can't submit that one. Once that's fixed, I'll rebase and remove those changes
from this patch.

Thanks!

[ahe, 2012-03-16 08:07:40]

I know, GYP is complicated. Let me know if you want me to make the GYP changes.

Cheers,
Peter

https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp
File dart.gyp (right):

https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp#newcode110
dart.gyp:110: 'target_name': 'api_docs',
I think this target belongs in its own GYP file, for example,
utils/apidoc/apidoc.gyp. See the 'frog' target above.

https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp#newcode119
dart.gyp:119: 'utils/apidoc',
No directory dependencies, please. Consider:

* If you change a file in a subdirectory, Make, Xcode, and Visual Studio
correctly see that the directory has changed (I'm pretty sure that this doesn't
work based on quick experiments with directory time stamps on my Mac).

* Xcode and Visual Studio correctly list the files that belong to this target.

* if you change nothing, but run "svn update" and .svn is touched, this target
won't be rebuilt.

Also, you need to list the library dependencies. Fortunately, the VM team is
already maintaining those. Try running: "find corelib runtime -name \*.gypi"

https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp#newcode126
dart.gyp:126: 'utils/apidoc/apidoc',
This needs to be installed in PRODUCT_DIR as a real program. Otherwise, you
cannot reliably find the right version of the dart VM.

I once sent you an example of how to do this in pure dart:
https://chromiumcodereview.appspot.com/9454026/

An alternative is to use something similar to the generate_frog action in
frog/dart-frog.gyp.

[Bob Nystrom, 2012-03-16 23:48:17]

On 2012/03/16 08:07:40, ahe wrote:
> I know, GYP is complicated. Let me know if you want me to make the GYP
changes.

Thanks. I want to understand more about how this works, so I'd like to go
through the process myself.

I'll respond to your other notes, but first I'm going to get this patch in:
https://chromiumcodereview.appspot.com/9722012/

That gets rid of the bash scripts completely, which should make this change
simpler.

> https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp
> File dart.gyp (right):
> 
> https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp#newcode110
> dart.gyp:110: 'target_name': 'api_docs',
> I think this target belongs in its own GYP file, for example,
> utils/apidoc/apidoc.gyp. See the 'frog' target above.
> 
> https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp#newcode119
> dart.gyp:119: 'utils/apidoc',
> No directory dependencies, please. Consider:
> 
> * If you change a file in a subdirectory, Make, Xcode, and Visual Studio
> correctly see that the directory has changed (I'm pretty sure that this
doesn't
> work based on quick experiments with directory time stamps on my Mac).
> 
> * Xcode and Visual Studio correctly list the files that belong to this target.
> 
> * if you change nothing, but run "svn update" and .svn is touched, this target
> won't be rebuilt.
> 
> Also, you need to list the library dependencies. Fortunately, the VM team is
> already maintaining those. Try running: "find corelib runtime -name \*.gypi"
> 
> https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp#newcode126
> dart.gyp:126: 'utils/apidoc/apidoc',
> This needs to be installed in PRODUCT_DIR as a real program. Otherwise, you
> cannot reliably find the right version of the dart VM.
> 
> I once sent you an example of how to do this in pure dart:
> https://chromiumcodereview.appspot.com/9454026/
> 
> An alternative is to use something similar to the generate_frog action in
> frog/dart-frog.gyp.

[ahe, 2012-03-17 20:14:05]

On 2012/03/16 23:48:17, Bob Nystrom wrote:
> Thanks. I want to understand more about how this works, so I'd like to go
> through the process myself.

Sounds good. This may be of help:

https://chromiumcodereview.appspot.com/9704098/
https://chromiumcodereview.appspot.com/9719019/

Cheers,
Peter

[Bob Nystrom, 2012-03-28 20:19:36]

How does it look now?

https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp
File dart.gyp (right):

https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp#newcode110
dart.gyp:110: 'target_name': 'api_docs',
On 2012/03/16 08:07:40, ahe wrote:
> I think this target belongs in its own GYP file, for example,
> utils/apidoc/apidoc.gyp. See the 'frog' target above.

Done.

https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp#newcode119
dart.gyp:119: 'utils/apidoc',
On 2012/03/16 08:07:40, ahe wrote:
> No directory dependencies, please. Consider:
> 
> * If you change a file in a subdirectory, Make, Xcode, and Visual Studio
> correctly see that the directory has changed (I'm pretty sure that this
doesn't
> work based on quick experiments with directory time stamps on my Mac).
> 
> * Xcode and Visual Studio correctly list the files that belong to this target.
> 
> * if you change nothing, but run "svn update" and .svn is touched, this target
> won't be rebuilt.
> 
> Also, you need to list the library dependencies. Fortunately, the VM team is
> already maintaining those. Try running: "find corelib runtime -name \*.gypi"

Done, but I'm having some problems. I wrote a "list files" script similar to
what frog uses. It should work in theory, but it doesn't seem to. When I change
an input file, GYP doesn't seem to notice that the API docs are out of date and
rebuild them. I have to manually delete the output directory instead.

Any idea what's going on here?

https://chromiumcodereview.appspot.com/9705088/diff/1/dart.gyp#newcode126
dart.gyp:126: 'utils/apidoc/apidoc',
On 2012/03/16 08:07:40, ahe wrote:
> This needs to be installed in PRODUCT_DIR as a real program. Otherwise, you
> cannot reliably find the right version of the dart VM.
> 
> I once sent you an example of how to do this in pure dart:
> https://chromiumcodereview.appspot.com/9454026/
> 
> An alternative is to use something similar to the generate_frog action in
> frog/dart-frog.gyp.

Done. Just invoking a dart script now.

[Bob Nystrom, 2012-03-28 21:37:29]

With Siggy's help, I figured out what was going on. You can't exactly invoke
dart during project generation time since you may not have already built it!

I believe this is all working now. The change in frog's reader.dart is because
leg has moved in the repo. Again apidoc was spontaneously broken without someone
realizing.

PTAL and let me know if there's anything else I need to do so I can get this in.

[dgrove, 2012-03-30 17:46:31]

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/apidoc...
File utils/apidoc/apidoc.dart (right):

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/apidoc...
utils/apidoc/apidoc.dart:50: final compilerPath = joinPaths(frogPath,
'minfrog');
minfrog is going away. why not just run frog out of the SDK? That will get you
on the right path for running dart2js later.

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/script...
File utils/apidoc/scripts/list_files.py (right):

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/script...
utils/apidoc/scripts/list_files.py:8: # need to be regenerated (see
`apidoc.gyp`).
Can this be written in dart?

[Bob Nystrom, 2012-03-30 18:01:40]

Thanks!

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/apidoc...
File utils/apidoc/apidoc.dart (right):

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/apidoc...
utils/apidoc/apidoc.dart:50: final compilerPath = joinPaths(frogPath,
'minfrog');
On 2012/03/30 17:46:31, dgrove wrote:
> minfrog is going away. why not just run frog out of the SDK?

There will be enormous changes to dartdoc and apidoc when frog goes away.
Running frog from the SDK would mean (I'm guessing) figuring out where you build
directory is, which is kind of hairy. Can this wait until a later patch? I want
to get this in as soon as possible.

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/script...
File utils/apidoc/scripts/list_files.py (right):

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/script...
utils/apidoc/scripts/list_files.py:8: # need to be regenerated (see
`apidoc.gyp`).
On 2012/03/30 17:46:31, dgrove wrote:
> Can this be written in dart?

It was initially. It turns out, no. This script is run during project generation
time (i.e. gclient runhooks) before you've actually built anything, including
the standalone VM.

[dgrove, 2012-03-30 18:16:24]

lgtm

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/apidoc...
File utils/apidoc/apidoc.dart (right):

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/apidoc...
utils/apidoc/apidoc.dart:50: final compilerPath = joinPaths(frogPath,
'minfrog');
yes. this can wait, given the reality that there will be big changes soon.

On 2012/03/30 18:01:40, Bob Nystrom wrote:
> On 2012/03/30 17:46:31, dgrove wrote:
> > minfrog is going away. why not just run frog out of the SDK?
> 
> There will be enormous changes to dartdoc and apidoc when frog goes away.
> Running frog from the SDK would mean (I'm guessing) figuring out where you
build
> directory is, which is kind of hairy. Can this wait until a later patch? I
want
> to get this in as soon as possible.

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/script...
File utils/apidoc/scripts/list_files.py (right):

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/script...
utils/apidoc/scripts/list_files.py:8: # need to be regenerated (see
`apidoc.gyp`).
As we discussed, this could use the prebuilt VM that we're using for tests. That
can come later - please add a todo.
On 2012/03/30 18:01:40, Bob Nystrom wrote:
> On 2012/03/30 17:46:31, dgrove wrote:
> > Can this be written in dart?
> 
> It was initially. It turns out, no. This script is run during project
generation
> time (i.e. gclient runhooks) before you've actually built anything, including
> the standalone VM.

[Bob Nystrom, 2012-03-30 19:00:58]

Thanks!

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/script...
File utils/apidoc/scripts/list_files.py (right):

https://chromiumcodereview.appspot.com/9705088/diff/11001/utils/apidoc/script...
utils/apidoc/scripts/list_files.py:8: # need to be regenerated (see
`apidoc.gyp`).
On 2012/03/30 18:16:25, dgrove wrote:
> As we discussed, this could use the prebuilt VM that we're using for tests.
That
> can come later - please add a todo.
> On 2012/03/30 18:01:40, Bob Nystrom wrote:
> > On 2012/03/30 17:46:31, dgrove wrote:
> > > Can this be written in dart?
> > 
> > It was initially. It turns out, no. This script is run during project
> generation
> > time (i.e. gclient runhooks) before you've actually built anything,
including
> > the standalone VM.
> 

Done.

[Ivan Posva, 2012-03-30 21:45:01]

LGTM -ip

https://chromiumcodereview.appspot.com/9705088/diff/19019/utils/apidoc/script...
File utils/apidoc/scripts/list_files.py (right):

https://chromiumcodereview.appspot.com/9705088/diff/19019/utils/apidoc/script...
utils/apidoc/scripts/list_files.py:10: # TODO(rnystrom): Port this to dart. It
can be run before dart has been built
We don't want to be dependent on Dart to build Dart. Since you want to generate
API docs as part of the Dart build this is a non-starter. From my point of view
you can remove this TODO.

[ahe, 2012-04-19 09:25:26]

https://chromiumcodereview.appspot.com/9705088/diff/19019/utils/apidoc/apidoc...
File utils/apidoc/apidoc.gyp (right):

https://chromiumcodereview.appspot.com/9705088/diff/19019/utils/apidoc/apidoc...
utils/apidoc/apidoc.gyp:18: '<(PRODUCT_DIR)/dart',
As I'm learning more about Windows, I think this should be:

'<(PRODUCT_DIR)/<(EXECUTABLE_PREFIX)dart<(EXECUTABLE_SUFFIX)',

https://chromiumcodereview.appspot.com/9705088/diff/19019/utils/apidoc/apidoc...
utils/apidoc/apidoc.gyp:22: '<(PRODUCT_DIR)/api_docs',
I think this output is a directory. That is still a problem for the same reasons
I mentioned in my previous comments.

https://chromiumcodereview.appspot.com/9705088/diff/19019/utils/apidoc/apidoc...
utils/apidoc/apidoc.gyp:25: '<(PRODUCT_DIR)/dart',
See comment above.