Parse unary plus corretly (i.e., by ignoring it when it's valid).

Parse unary plus corretly (i.e., by ignoring it when it's valid).

Committed: https://code.google.com/p/dart/source/detail?r=3898

[ngeoffray, 2012-02-03 11:28:35]

As discussed, I'm fine with this approach too. If we end up with this CL, you
should probably revert my changes in: http://codereview.chromium.org/9310066/.

[ahe, 2012-02-03 11:28:51]

LGTM provided you don't throw tokens away in the parser.

Obviously, I'd prefer to send the unary plus operator all the way through the
compiler pipeline.

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
File frog/leg/scanner/parser.dart (right):

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
frog/leg/scanner/parser.dart:947: // decimal literal. We scan it as a separate
token and then throws
throws -> throw

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
frog/leg/scanner/parser.dart:948: // it away here when we know that we have an
unary plus, not a
"an unary" -> "a unary"

http://owl.english.purdue.edu/owl/resource/591/1/

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
frog/leg/scanner/parser.dart:952: if (next.kind === INT_TOKEN) return
parseLiteralInt(next);
We shouldn't throw tokens away. Long term, it becomes a problem for IDEs.

Short term it would be better to throw the token away in the listener.

[Lasse Reichstein Nielsen, 2012-02-03 13:10:21]

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
File frog/leg/scanner/parser.dart (right):

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
frog/leg/scanner/parser.dart:952: if (next.kind === INT_TOKEN) return
parseLiteralInt(next);
I think we should throw it away in the listener in all cases. The language
doesn't contain a unary plus operator, so it shouldn't be in the AST either. We
can still link the number AST node to the appropriate Tokens.

We can also complain a little about having a lexical grammar that is ambiguous
(in itself, since it depends in the syntactical grammar): the characters "+0"
should be tokenized differently depending on the preceding token, e.g. ";" ("+0"
is one token) or "0" (two tokens). That's almost JavsScript bad :(

[ahe, 2012-02-03 13:19:11]

LGTM!

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
File frog/leg/scanner/parser.dart (right):

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
frog/leg/scanner/parser.dart:952: if (next.kind === INT_TOKEN) return
parseLiteralInt(next);
On 2012/02/03 13:10:22, Lasse Reichstein Nielsen wrote:
> I think we should throw it away in the listener in all cases. The language
> doesn't contain a unary plus operator, so it shouldn't be in the AST either.
We
> can still link the number AST node to the appropriate Tokens.

Unfortunately, that way of thinking is counter-productive for implementing IDE
features.

The more we accept and can represent, the better.

> We can also complain a little about having a lexical grammar that is ambiguous
> (in itself, since it depends in the syntactical grammar): the characters "+0"
> should be tokenized differently depending on the preceding token, e.g. ";"
("+0"
> is one token) or "0" (two tokens). That's almost JavsScript bad :(

The Dart language grammar is not designed to be sane, easy to implement, or
unambiguous.

Purists may weep. I find it easier to accept that the world doesn't want
Smalltalk or Scheme and move on. C syntax is horrible.

https://chromiumcodereview.appspot.com/9320064/diff/4001/frog/leg/scanner/par...
File frog/leg/scanner/parser.dart (right):

https://chromiumcodereview.appspot.com/9320064/diff/4001/frog/leg/scanner/par...
frog/leg/scanner/parser.dart:948: // it away here when we know that we have a
unary plus, not a
We are not throwing it away anymore :-)

[Lasse Reichstein Nielsen, 2012-02-03 13:30:17]

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
File frog/leg/scanner/parser.dart (right):

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
frog/leg/scanner/parser.dart:949: // binary one.
Comment updated.

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
frog/leg/scanner/parser.dart:952: if (next.kind === INT_TOKEN) return
parseLiteralInt(next);
I'm not advocating throwing the character away, but I do think we should treat
it as a single number literal (comprised of two tokens). We don't parse "0x" of
a hex literal as a separate "prefix hex operator", so we shouldn't need parse
"+" of "+0" as a separate "prefix plus operator".

[ahe, 2012-02-03 13:41:10]

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
File frog/leg/scanner/parser.dart (right):

https://chromiumcodereview.appspot.com/9320064/diff/1/frog/leg/scanner/parser...
frog/leg/scanner/parser.dart:952: if (next.kind === INT_TOKEN) return
parseLiteralInt(next);
On 2012/02/03 13:30:17, Lasse Reichstein Nielsen wrote:
> I'm not advocating throwing the character away, but I do think we should treat
> it as a single number literal (comprised of two tokens). We don't parse "0x"
of
> a hex literal as a separate "prefix hex operator", so we shouldn't need parse
> "+" of "+0" as a separate "prefix plus operator".

We could do that. I guess it would be limited to LiteralInt and LiteralDouble.