Experimental lexer generator: transfer actions across epsilon transitions correctly when constructi…

Experimental lexer generator: transfer actions across epsilon transitions correctly when constructing the nfa.

This produces a reasonable nfa (with actions) with the following grammar:

<default>
"break"       { BREAK } <<break>>
/[a-z]+/ { ID } <<break>>


BUG=
R=dcarney@chromium.org

Committed: https://code.google.com/p/v8/source/detail?r=17542

[marja, 2013-11-06 16:39:41]

ptal

[dcarney, 2013-11-07 08:08:54]

https://chromiumcodereview.appspot.com/62223002/diff/60001/tools/lexer_genera...
File tools/lexer_generator/nfa.py (right):

https://chromiumcodereview.appspot.com/62223002/diff/60001/tools/lexer_genera...
tools/lexer_generator/nfa.py:340: actions.append((action[2], action[0],
action[1]))
action should just always be (precedence, id, code), then no need for all the
reordering

https://chromiumcodereview.appspot.com/62223002/diff/60001/tools/lexer_genera...
tools/lexer_generator/nfa.py:343: # match state.
what's with all these spaces and comments - you want this code to be readable?

https://chromiumcodereview.appspot.com/62223002/diff/60001/tools/lexer_genera...
File tools/lexer_generator/rule_parser.py (right):

https://chromiumcodereview.appspot.com/62223002/diff/60001/tools/lexer_genera...
tools/lexer_generator/rule_parser.py:53: self.__rule_precedence_counter = 0
should either be class static or go in RuleParserState

[dcarney, 2013-11-07 08:09:23]

lgtm

[marja, 2013-11-07 08:25:40]

https://codereview.chromium.org/62223002/diff/60001/tools/lexer_generator/nfa.py
File tools/lexer_generator/nfa.py (right):

https://codereview.chromium.org/62223002/diff/60001/tools/lexer_generator/nfa...
tools/lexer_generator/nfa.py:340: actions.append((action[2], action[0],
action[1]))
On 2013/11/07 08:08:54, dcarney wrote:
> action should just always be (precedence, id, code), then no need for all the
> reordering

Done.

https://codereview.chromium.org/62223002/diff/60001/tools/lexer_generator/rul...
File tools/lexer_generator/rule_parser.py (right):

https://codereview.chromium.org/62223002/diff/60001/tools/lexer_generator/rul...
tools/lexer_generator/rule_parser.py:53: self.__rule_precedence_counter = 0
On 2013/11/07 08:08:54, dcarney wrote:
> should either be class static or go in RuleParserState

Done.

[marja, 2013-11-07 08:26:06]

Committed patchset #4 manually as r17542 (presubmit successful).