Update unit_tests.isolate on linux.

Update unit_tests.isolate on linux.

Should improve unit_tests_swarm success rate on linux, in theory it could
become green.

R=thakis@chromium.org
NOTRY=true
BUG=98636
TEST=


Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=142437

[Nico, 2012-06-15 16:27:10]

lgtm stamp


Keeping these up-to-date manually seems very busy-workish and error-prone. Can
we keep these files up-to-date automatically somehow?

[M-A Ruel, 2012-06-15 18:34:20]

On 2012/06/15 16:27:10, Nico wrote:
> lgtm stamp
> 
> 
> Keeping these up-to-date manually seems very busy-workish and error-prone. Can
> we keep these files up-to-date automatically somehow?

The file was generated semi-automatically with the following steps:
1. cd tools/isolate
2. ./trace_test_cases.py out/Release/unit_tests -V PRODUCT_DIR out/Release -c
chrome --root-dir ../..
# I have to pipe in a separate file otherwise the source file gets deleted too
early.
3. ./merge_isolate.py ../../out/Release/unit_tests.test_cases.isolate
../../chrome/unit_tests.isolate > ../../chrome/unit_tests2.isolate
4. mv ../../chrome/unit_tests2.isolate ../../chrome/unit_tests2.isolate
5. The manual edit were:
5a. Added back copyright.
5b. Removed fr.pak, since I'm a freak with French UI.
5c. Removes testserver.log, since my new blacklist missed it.

So basically the only manual edit that will remain is due to me using a French
locale and the unit test not enforcing an English locale when testing.

TODOs are:
A) Merge trace_test_cases.py functionality (parallel tracing) into isolate.py,
so the command line 2. will be much shorter.
B) Add a --output to merge_isolate.py so step 4. is unnecessary.
C) Have merge_isolate.py keep copyright so 5a is not necessary anymore.
D) Fix isolate_common.py to properly blacklist testserver.log to remove step 5c.
It'd be nicer if it was generated at the right place but that'll be kept for
later.

I duno about fixing step 5b beside switching UI or fixing the tests.

[commit-bot: I haz the power, 2012-06-15 19:02:54]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/maruel@chromium.org/10559009/1

[commit-bot: I haz the power, 2012-06-15 19:03:06]

Change committed as 142437

[M-A Ruel, 2012-06-15 19:43:28]

On 2012/06/15 18:34:20, Marc-Antoine Ruel wrote:
> TODOs are:
> A) Merge trace_test_cases.py functionality (parallel tracing) into isolate.py,
> so the command line 2. will be much shorter.
> B) Add a --output to merge_isolate.py so step 4. is unnecessary.
> C) Have merge_isolate.py keep copyright so 5a is not necessary anymore.
> D) Fix isolate_common.py to properly blacklist testserver.log to remove step
5c.

B) and C) are addressed with https://chromiumcodereview.appspot.com/10565008/.

[M-A Ruel, 2012-06-15 19:46:51]

On 2012/06/15 19:43:28, Marc-Antoine Ruel wrote:
> On 2012/06/15 18:34:20, Marc-Antoine Ruel wrote:
> > TODOs are:
> > A) Merge trace_test_cases.py functionality (parallel tracing) into
isolate.py,
> > so the command line 2. will be much shorter.
> > B) Add a --output to merge_isolate.py so step 4. is unnecessary.
> > C) Have merge_isolate.py keep copyright so 5a is not necessary anymore.
> > D) Fix isolate_common.py to properly blacklist testserver.log to remove step
> 5c.
> 
> B) and C) are addressed with https://chromiumcodereview.appspot.com/10565008/.

https://chromiumcodereview.appspot.com/10540172 now addresses C). So my next
.isolate CL shouldn't involve any manual editing anymore except removing fr.pak.

[jam, 2012-06-15 19:49:07]

It might be helpful to get a conversation started on chromium-dev started
about this work to get more feedback about this process (even if it's not
ready yet, i.e. to explain the future plans, what would have to be
maintained etc)?

On Fri, Jun 15, 2012 at 11:34 AM, <maruel@chromium.org> wrote:

> On 2012/06/15 16:27:10, Nico wrote:
>
>> lgtm stamp
>>
>
>
>  Keeping these up-to-date manually seems very busy-workish and
>> error-prone. Can
>> we keep these files up-to-date automatically somehow?
>>
>
> The file was generated semi-automatically with the following steps:
> 1. cd tools/isolate
> 2. ./trace_test_cases.py out/Release/unit_tests -V PRODUCT_DIR out/Release
> -c
> chrome --root-dir ../..
> # I have to pipe in a separate file otherwise the source file gets deleted
> too
> early.
> 3. ./merge_isolate.py ../../out/Release/unit_tests.**test_cases.isolate
> ../../chrome/unit_tests.**isolate > ../../chrome/unit_tests2.**isolate
> 4. mv ../../chrome/unit_tests2.**isolate ../../chrome/unit_tests2.**
> isolate
> 5. The manual edit were:
> 5a. Added back copyright.
> 5b. Removed fr.pak, since I'm a freak with French UI.
> 5c. Removes testserver.log, since my new blacklist missed it.
>
> So basically the only manual edit that will remain is due to me using a
> French
> locale and the unit test not enforcing an English locale when testing.
>
> TODOs are:
> A) Merge trace_test_cases.py functionality (parallel tracing) into
> isolate.py,
> so the command line 2. will be much shorter.
> B) Add a --output to merge_isolate.py so step 4. is unnecessary.
> C) Have merge_isolate.py keep copyright so 5a is not necessary anymore.
> D) Fix isolate_common.py to properly blacklist testserver.log to remove
> step 5c.
> It'd be nicer if it was generated at the right place but that'll be kept
> for
> later.
>
> I duno about fixing step 5b beside switching UI or fixing the tests.
>
>
https://chromiumcodereview.**appspot.com/10559009/<https://chromiumcodereview...
>

[M-A Ruel, 2012-06-15 20:01:45]

On 2012/06/15 19:49:07, John Abd-El-Malek wrote:
> It might be helpful to get a conversation started on chromium-dev started
> about this work to get more feedback about this process (even if it's not
> ready yet, i.e. to explain the future plans, what would have to be
> maintained etc)?

I want to get into "race" mode before asking for more feedback. Right now, I'm
trying to make sure my check-ins don't block anyone and nobody has to care about
it yet.

There are a few issues. For example, tracing base_unittests currently takes
around 40 minutes on Windows, 36 being solely python string parsing (!) versus
~3 minutes on linux, so:
1. There's a fair amount of optimization work to do in the parser. I'm not even
sure I'll stay with python. Independent of the optimization I'll be doing,
parsing a 8gb CSV text file is going to be slow in python. :) That's the rough
size of a browser_tests.exe trace.
2. The command CLI interface is still in flux, for example I am playing with the
idea of ./isolate.py --mode=<command> versus ./isolate.py <command>. I don't
want to have to support any legacy CLI interface. So I don't want any other test
to have a .isolate configuration yet.
3. I still hit pretty severe but fixable bugs. For example, the log parser
chockes on process id reuse, which happens all the time on Windows. Totally
fixable, I just need more time. Same for strace log corruptions, etc.
4. I want http://build.chromium.org/p/chromium.swarm/waterfall to be in a
slightly better shape first. My first focus is on getting linux green including
browser test, catching all the major bugs there and then moving to other
platforms, so I'm not blocking Chris&Mahdi and they can start performance checks
on the whole infrastructure ASAP.

So overall, I have enough P0 bugs on my plate that I don't have the need to
request feedback yet and I wouldn't be able to do anything with it. I'm just
trying to automate things as much as possible at the moment.

[jam, 2012-06-15 23:17:55]

I see. Just to be clear, I didn't mean the implementation of the tracing
tool. I meant how we want to represent dependencies of tests etc (i.e. what
the average developer will have to do).

On Fri, Jun 15, 2012 at 1:01 PM, <maruel@chromium.org> wrote:

> On 2012/06/15 19:49:07, John Abd-El-Malek wrote:
>
>> It might be helpful to get a conversation started on chromium-dev started
>> about this work to get more feedback about this process (even if it's not
>> ready yet, i.e. to explain the future plans, what would have to be
>> maintained etc)?
>>
>
> I want to get into "race" mode before asking for more feedback. Right now,
> I'm
> trying to make sure my check-ins don't block anyone and nobody has to care
> about
> it yet.
>
> There are a few issues. For example, tracing base_unittests currently takes
> around 40 minutes on Windows, 36 being solely python string parsing (!)
> versus
> ~3 minutes on linux, so:
> 1. There's a fair amount of optimization work to do in the parser. I'm not
> even
> sure I'll stay with python. Independent of the optimization I'll be doing,
> parsing a 8gb CSV text file is going to be slow in python. :) That's the
> rough
> size of a browser_tests.exe trace.
> 2. The command CLI interface is still in flux, for example I am playing
> with the
> idea of ./isolate.py --mode=<command> versus ./isolate.py <command>. I
> don't
> want to have to support any legacy CLI interface. So I don't want any
> other test
> to have a .isolate configuration yet.
> 3. I still hit pretty severe but fixable bugs. For example, the log parser
> chockes on process id reuse, which happens all the time on Windows. Totally
> fixable, I just need more time. Same for strace log corruptions, etc.
> 4. I want
http://build.chromium.org/p/**chromium.swarm/waterfall<http://build.chromium....
be in a
> slightly better shape first. My first focus is on getting linux green
> including
> browser test, catching all the major bugs there and then moving to other
> platforms, so I'm not blocking Chris&Mahdi and they can start performance
> checks
> on the whole infrastructure ASAP.
>
> So overall, I have enough P0 bugs on my plate that I don't have the need to
> request feedback yet and I wouldn't be able to do anything with it. I'm
> just
> trying to automate things as much as possible at the moment.
>
>
https://chromiumcodereview.**appspot.com/10559009/<https://chromiumcodereview...
>

[M-A Ruel, 2012-07-05 14:55:20]

On 2012/06/15 23:17:55, John Abd-El-Malek wrote:
> I see. Just to be clear, I didn't mean the implementation of the tracing
> tool. I meant how we want to represent dependencies of tests etc (i.e. what
> the average developer will have to do).

I started updating the web pages to describe the current state of the tools.
Since I do not currently foresee modification to the .isolate format, I've
described it with an annotated example at this page:
https://sites.google.com/a/chromium.org/dev/developers/testing/isolated-testi...

This page does not describe how to generate them automatically, I'll do that
soonish in
https://sites.google.com/a/chromium.org/dev/developers/testing/isolated-testing/.