Update test.dart for detection output of machine formatted errors

Update test.dart for detection output of machine formatted errors

This change causes the 'dartc' tests to be more sensitive to the difference
between a static type error and a compilation error.   Instead of
relying on the return value from the test being 0 or non zero, the
code now turns on '--error_format machine' and checks type
error level 'ERROR' or 'WARNING' and the type of error for
'STATIC_TYPE' if there is a static type error.  A new annotation
format adapted from multitests is introduced for standard test cases: 

  int foo = "hello";   /// static type error

The above annotation means that a static type error is expected.  
Currently enforcement is very lenient.  There must be at least as 
many STATIC_TYPE error messages as there are '/// static type error' 
annotations (and no ERROR level messages.)

Committed: https://code.google.com/p/dart/source/detail?r=4895

[zundel, 2012-03-01 20:00:49]

(see earlier version at https://chromiumcodereview.appspot.com/9479034/)

I merged other changes since then and didn't want them to get confused.

I went ahead and annotated a few of the changes to show what the tests would
look like.  Since there are so many ways the test can fail, I added special
messages that can get printed out when the test doesn't match expectations. (the
diagnostics field in TestOutput).

[Bill Hesse, 2012-03-02 08:55:18]

LGTM.  Are the boolean flag arguments needed on doTest because of the
non-multitest case, and they can't be merged with the multitest case's use of
output?

https://chromiumcodereview.appspot.com/9559007/diff/5016/tests/language/src/A...
File tests/language/src/Arithmetic2Test.dart (right):

https://chromiumcodereview.appspot.com/9559007/diff/5016/tests/language/src/A...
tests/language/src/Arithmetic2Test.dart:23: f();
Extra space.

https://chromiumcodereview.appspot.com/9559007/diff/5016/tests/language/src/B...
File tests/language/src/BoolTest.dart (right):

https://chromiumcodereview.appspot.com/9559007/diff/5016/tests/language/src/B...
tests/language/src/BoolTest.dart:10: Expect.equals(true, true);
Extra space.

https://chromiumcodereview.appspot.com/9559007/diff/5016/tools/testing/dart/m...
File tools/testing/dart/multitest.dart (right):

https://chromiumcodereview.appspot.com/9559007/diff/5016/tools/testing/dart/m...
tools/testing/dart/multitest.dart:161: enableFatalTypeErrors,
Can't you remove all the boolean flag arguments from doTest if you pass the
outcome, and do the tests in doTest?

https://chromiumcodereview.appspot.com/9559007/diff/5016/tools/testing/dart/t...
File tools/testing/dart/test_suite.dart (right):

https://chromiumcodereview.appspot.com/9559007/diff/5016/tools/testing/dart/t...
tools/testing/dart/test_suite.dart:344: // format of the rror line
Typo - rror

https://chromiumcodereview.appspot.com/9559007/diff/5016/tools/testing/dart/t...
tools/testing/dart/test_suite.dart:824: matches =
staticCleanRegExp.allMatches(contents);
Is this overkill?  Why not just a single hasMatch?

https://chromiumcodereview.appspot.com/9559007/diff/5016/tools/testing/dart/t...
tools/testing/dart/test_suite.dart:843: 
Whitespace.

https://chromiumcodereview.appspot.com/9559007/diff/5016/tools/testing/dart/t...
tools/testing/dart/test_suite.dart:852: for (var i in
staticTypeRegExp.allMatches(contents)) {
Var dummy?
There is really no length field?  It is an iterator?

[zundel, 2012-03-02 23:51:00]

http://codereview.chromium.org/9559007/diff/5016/tests/language/src/Arithmeti...
File tests/language/src/Arithmetic2Test.dart (right):

http://codereview.chromium.org/9559007/diff/5016/tests/language/src/Arithmeti...
tests/language/src/Arithmetic2Test.dart:23: f();
On 2012/03/02 08:55:18, Bill Hesse wrote:
> Extra space.

Done.

http://codereview.chromium.org/9559007/diff/5016/tests/language/src/BoolTest....
File tests/language/src/BoolTest.dart (right):

http://codereview.chromium.org/9559007/diff/5016/tests/language/src/BoolTest....
tests/language/src/BoolTest.dart:10: Expect.equals(true, true);
On 2012/03/02 08:55:18, Bill Hesse wrote:
> Extra space.

Done.

http://codereview.chromium.org/9559007/diff/5016/tests/language/src/Prefix22N...
File tests/language/src/Prefix22NegativeTest.dart (right):

http://codereview.chromium.org/9559007/diff/5016/tests/language/src/Prefix22N...
tests/language/src/Prefix22NegativeTest.dart:7: // VMOptions=
--enable_type_errors
reverted

http://codereview.chromium.org/9559007/diff/5016/tools/testing/dart/multitest...
File tools/testing/dart/multitest.dart (right):

http://codereview.chromium.org/9559007/diff/5016/tools/testing/dart/multitest...
tools/testing/dart/multitest.dart:161: enableFatalTypeErrors,
On 2012/03/02 08:55:18, Bill Hesse wrote:
> Can't you remove all the boolean flag arguments from doTest if you pass the
> outcome, and do the tests in doTest?
I think you are right, I added TODO.

http://codereview.chromium.org/9559007/diff/5016/tools/testing/dart/test_suit...
File tools/testing/dart/test_suite.dart (right):

http://codereview.chromium.org/9559007/diff/5016/tools/testing/dart/test_suit...
tools/testing/dart/test_suite.dart:824: matches =
staticCleanRegExp.allMatches(contents);
On 2012/03/02 08:55:18, Bill Hesse wrote:
> Is this overkill?  Why not just a single hasMatch?
I did that because I decided that multiple matches s/b fatal

http://codereview.chromium.org/9559007/diff/5016/tools/testing/dart/test_suit...
tools/testing/dart/test_suite.dart:852: for (var i in
staticTypeRegExp.allMatches(contents)) {
On 2012/03/02 08:55:18, Bill Hesse wrote:
> Var dummy?
> There is really no length field?  It is an iterator?

yes, its an Iterator.
http://api.dartlang.org/dart_core/RegExp.html