Added the ability to filter which tests are run by name, using a --filter

Added the ability to filter which tests are run by name, using a --filter argument passed to test.dart.
Committed: https://code.google.com/p/dart/source/detail?r=9006

[gram, 2012-06-21 20:31:55]

PTAL. Tested with:

compiler=none runtime=vm
compiler=none runtime=drt
compiler=dart2js runtime=drt

[eub, 2012-06-21 20:58:45]

Fair bit of plumbing to add a flag, eh.  Thanks, Graham.

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/config...
File lib/unittest/config.dart (right):

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/config...
lib/unittest/config.dart:64: // bug in your unit tests, unless you are
filtering.
Is there a known use case where people want to filter down to zero tests and
have that silently succeed?  I can imagine it might exist but I don't see an
example.

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/unitte...
File lib/unittest/unittest.dart (right):

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/unitte...
lib/unittest/unittest.dart:216: /** If set, then only tests whose names match
this pattern will be run. */
s/pattern/regexp/ to be clear.

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/unitte...
lib/unittest/unittest.dart:216: /** If set, then only tests whose names match
this pattern will be run. */
Also, s/name/description/ globally unless that's already an established alias.

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/unitte...
lib/unittest/unittest.dart:661: if (options[i].startsWith('--filter=')) {
Another one for the Great Flag Parsing Unification of the Future?

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/unitte...
lib/unittest/unittest.dart:662: var filter = options[i].substring(10);
This chops off the first character, or did I miscount?

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/unitte...
lib/unittest/unittest.dart:663: filter = filter.substring(0, filter.length-1);
And this chops off the last character, so we're consistent... is this expecting
it to come in with quote bracketing or something?

[gram, 2012-06-21 21:18:03]

PTAL

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/config...
File lib/unittest/config.dart (right):

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/config...
lib/unittest/config.dart:64: // bug in your unit tests, unless you are
filtering.
On 2012/06/21 20:58:45, eub wrote:
> Is there a known use case where people want to filter down to zero tests and
> have that silently succeed?  I can imagine it might exist but I don't see an
> example.

Here's the thing - we are constraining individual tests, not test files. So if
you have three test files in a directory A, B, C, and B contains test 'foo', and
you filter for 'foo', then files A and C will otherwise show up as 'test
failures'. Hence this change.

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/unitte...
File lib/unittest/unittest.dart (right):

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/unitte...
lib/unittest/unittest.dart:216: /** If set, then only tests whose names match
this pattern will be run. */
On 2012/06/21 20:58:45, eub wrote:
> s/pattern/regexp/ to be clear.

Done.

https://chromiumcodereview.appspot.com/10627008/diff/1006/lib/unittest/unitte...
lib/unittest/unittest.dart:662: var filter = options[i].substring(10);
On 2012/06/21 20:58:45, eub wrote:
> This chops off the first character, or did I miscount?

Correct, I was assuming this is quoted. But you're right, that may not be true
and I should make sure, so I've tweaked this.

[eub, 2012-06-21 22:02:21]

LGTM.  Thanks for the feature!

[ahe, 2012-06-22 09:12:55]

I'm going to revert this.

Next time you make changes to shared test infrastructure, please make sure it
works across the board. Also, make sure you ask for review from the original
authors. None of the people who reviewed this change has been involved in
maintaining or designing our test infrastructure. I think it should be obvious
that you need to get review from people who are familiar with the code you're
changing.

$ tools/test.py -mrelease -cdart2js -rd8 --filter=contains lib/unittest
Test configuration: dart2js_d8_release_ia32
[00:00 |  --% | +    0 | -    0]
FAILED: dart2js-d8 release_ia32 lib/unittest/matchers_test
Expected: pass 
Actual: fail

stdout:
Usage: dart2js [options] dartfile

Compiles Dart to JavaScript.

Common options:
  -o<file> Generate the output into <file>.
  -c       Insert runtime type checks and enable assertions (checked mode).
  -h       Display this message (add -v for information about all options).

Error: Unknown option "--filter="contains"".

stderr:
test.dart: Compilation failed (step 1 of 2), exit code 1

...