When running iOS tests on devices, write stdout to a file, then dump to NSLog.

When running iOS tests on devices, write stdout to a file, then dump to NSLog.

gtest only writes test output to stdout, but on iOS 6 devices, stdout is not
available to be retrieved from the device. This CL redirects stdout to a file
while the tests are running, then dumps the contents of that file to NSLog
(which is available to be retrieved from the device).

BUG=None


Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=190025

[lliabraa, 2013-02-25 13:49:46]

This is a second attempt to write gtest output to NSLog when running iOS tests
on devices. The first attempt is at:
https://chromiumcodereview.appspot.com/12047058/

The gtest maintainers were not interested in a generic solution for writing test
results to arbitrary destinations, but this workaround avoids the negative
points brought up in the review of the first attempt (namely, duplicating a
bunch of gtest formatting).

[Paweł Hajdan Jr., 2013-02-25 19:51:05]

On 2013/02/25 13:49:46, Lane LiaBraaten wrote:
> The gtest maintainers were not interested in a generic solution for writing
test
> results to arbitrary destinations,

Please post the link to the discussion.

> but this workaround avoids the negative
> points brought up in the review of the first attempt (namely, duplicating a
> bunch of gtest formatting).

This is indeed better, but what when the test crashes before you get the chance
to read the file?

[lliabraa, 2013-02-27 14:48:28]

On 2013/02/25 19:51:05, Paweł Hajdan Jr. wrote:
> On 2013/02/25 13:49:46, Lane LiaBraaten wrote:
> > The gtest maintainers were not interested in a generic solution for writing
> test
> > results to arbitrary destinations,
> 
> Please post the link to the discussion.
> 
> > but this workaround avoids the negative
> > points brought up in the review of the first attempt (namely, duplicating a
> > bunch of gtest formatting).
> 
> This is indeed better, but what when the test crashes before you get the
chance
> to read the file?

I suppose I could have another thread reading from the file and writing to nslog
while the tests are running. Are there any chromium classes to help do the file
i/o?

[Paweł Hajdan Jr., 2013-03-04 18:12:46]

On 2013/02/27 14:48:28, Lane LiaBraaten wrote:
> I suppose I could have another thread reading from the file and writing to
nslog
> while the tests are running. Are there any chromium classes to help do the
file
> i/o?

A thread in the same process would still be prone to the same problem. How about
a very simple launcher process (less crash-prone), that would launch the target
binary, read its stdout and write to nslog? Then if the child crashes we should
still have the portion of its output that made it into buffers - the best we
would have in any other case anyway.

[lliabraa, 2013-03-19 14:50:47]

On 2013/03/04 18:12:46, Paweł Hajdan Jr. wrote:
> On 2013/02/27 14:48:28, Lane LiaBraaten wrote:
> > I suppose I could have another thread reading from the file and writing to
> nslog
> > while the tests are running. Are there any chromium classes to help do the
> file
> > i/o?
> 
> A thread in the same process would still be prone to the same problem. How
about
> a very simple launcher process (less crash-prone), that would launch the
target
> binary, read its stdout and write to nslog? Then if the child crashes we
should
> still have the portion of its output that made it into buffers - the best we
> would have in any other case anyway.

On iOS we only get one process, so a background thread is the best we could do.
However, copying test results from the file to NSLog is only a stop-gap, so I'd
like to get this CL in to remove forking with downstream. I am working to have
the test harness pull the log file from the device (similar to your launcher
process approach), so any improvements we make in this file for piping from the
file to NSLog would just be removed at that point.

[Paweł Hajdan Jr., 2013-03-19 17:05:28]

On 2013/03/19 14:50:47, Lane LiaBraaten wrote:
> On iOS we only get one process, so a background thread is the best we could
do.
> However, copying test results from the file to NSLog is only a stop-gap, so
I'd
> like to get this CL in to remove forking with downstream. I am working to have
> the test harness pull the log file from the device (similar to your launcher
> process approach), so any improvements we make in this file for piping from
the
> file to NSLog would just be removed at that point.

1. Why not use solution from last Zhanyong's post at
http://goto.google.com/googletest-nslog (Googlers only)? You could buffer up to
full lines if needed, and also print immediately to stdout if possible (not sure
how much that idea makes sense, I'm not that familiar with ios).

2. If for some good reason the above is absolutely not possible, I'd rather go
with a thread than a file approach.

[lliabraa, 2013-03-19 17:19:13]

On 2013/03/19 17:05:28, Paweł Hajdan Jr. wrote:
> On 2013/03/19 14:50:47, Lane LiaBraaten wrote:
> > On iOS we only get one process, so a background thread is the best we could
> do.
> > However, copying test results from the file to NSLog is only a stop-gap, so
> I'd
> > like to get this CL in to remove forking with downstream. I am working to
have
> > the test harness pull the log file from the device (similar to your launcher
> > process approach), so any improvements we make in this file for piping from
> the
> > file to NSLog would just be removed at that point.
> 
> 1. Why not use solution from last Zhanyong's post at
> http://goto.google.com/googletest-nslog (Googlers only)? You could buffer up
to
> full lines if needed, and also print immediately to stdout if possible (not
sure
> how much that idea makes sense, I'm not that familiar with ios).

As I explained in that thread, NSLog is not reliable. That is why I am moving
toward writing to a file, then pulling the file off the device.

> 
> 2. If for some good reason the above is absolutely not possible, I'd rather go
> with a thread than a file approach.

[Paweł Hajdan Jr., 2013-03-19 17:50:50]

On 2013/03/19 17:19:13, Lane LiaBraaten wrote:
> As I explained in that thread, NSLog is not reliable. That is why I am moving
> toward writing to a file, then pulling the file off the device.

Could you explain more why do you think NSLog is dropping messages?

> > 2. If for some good reason the above is absolutely not possible, I'd rather
go
> > with a thread than a file approach.

After more thinking it seems that writing to file is actually more reliable than
another thread. In that case we should also redirect stderr to a file (and watch
out for mix-ups of the two).

I'd still like to learn what exactly is wrong with NSLog, to make sure we're
making the right decision here.

[lliabraa, 2013-03-20 19:04:01]

On 2013/03/19 17:50:50, Paweł Hajdan Jr. wrote:
> On 2013/03/19 17:19:13, Lane LiaBraaten wrote:
> > As I explained in that thread, NSLog is not reliable. That is why I am
moving
> > toward writing to a file, then pulling the file off the device.
> 
> Could you explain more why do you think NSLog is dropping messages?
> 
> > > 2. If for some good reason the above is absolutely not possible, I'd
rather
> go
> > > with a thread than a file approach.
> 
> After more thinking it seems that writing to file is actually more reliable
than
> another thread. In that case we should also redirect stderr to a file (and
watch
> out for mix-ups of the two).
> 
> I'd still like to learn what exactly is wrong with NSLog, to make sure we're
> making the right decision here.

I've added support for redirection stderr. I am just dumping one file after the
other to NSLog. I am not going to do any special parsing to dump messages
chronologically since writing this info to NSLog is not a long-term solution.

[Paweł Hajdan Jr., 2013-03-20 19:55:01]

LGTM

[commit-bot: I haz the power, 2013-03-21 11:36:13]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/lliabraa@chromium.org/12328083/15001

[commit-bot: I haz the power, 2013-03-21 16:21:46]

Retried try job too often on win7_aura for step(s) browser_tests
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=win7_aura&...

[commit-bot: I haz the power, 2013-03-21 16:39:30]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/lliabraa@chromium.org/12328083/15001

[commit-bot: I haz the power, 2013-03-22 10:50:07]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/lliabraa@chromium.org/12328083/15001

[commit-bot: I haz the power, 2013-03-22 18:11:24]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/lliabraa@chromium.org/12328083/15001

[commit-bot: I haz the power, 2013-03-23 14:43:17]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/lliabraa@chromium.org/12328083/15001

[commit-bot: I haz the power, 2013-03-23 16:28:11]

Change committed as 190025