Separately format lines when a multisplit is split.

Separately format lines when a multisplit is split.

The LineWriter's job is to take a stream of chunks and deliver batches of them
to LineSplitter. Each batch must be isolated such that splitting one won't
affect the results of later ones.

If the LineWriter is too pessimistic about this, it can end up giving the
LineSplitter a long list of chunks that it then dutifully splits into a few
separate lines. The output is correct, but the performance isn't optimal
because the splitter goes (much) faster on shorter chunk lists.

One cases where the LineWriter wasn't fine-grained enough was multisplits.
When a hard split occurs inside a multisplit, LineWriter correctly hardens all
of the splits associated with that multisplit. For example, if you have:

  {key:"k", [[[// comment
  ]]}

As soon as the line comment is hit, the formatter knows all of the surrounding
collection literals must be split:

  {
    key:"k",
    [
      [
        [
          // comment
        ]
      ]
    ]
  }

What it doesn't do is go back and say, "Ah, now that I know some of those
earlier chunks have hard splits, can I process them as separate lines?" This
meant that, for example:

  method() {
    veryLongExpression(...)
  }

would be split as a single blob, instead of breaking a separate line at the
"{". That, in turn makes things like a deep statement right at the beginning
of a method body gets even worse since the LineSplitter rolls it into the
previous line too.

This fixes that. Makes the benchmark 3x faster.

R=pquitslund@google.com

Committed: ff182727c4b49f55a829645468b63dc23464741b

[Bob Nystrom, 2015-03-10 01:15:16]

rnystrom@google.com changed reviewers:
+ pquitslund@google.com

[Bob Nystrom, 2015-03-10 01:15:17]

Still not done optimizing, but this helps a bit and makes some other more hacky
optimizations I'm working on less intrusive.

[pquitslund, 2015-03-10 16:20:41]

LGTM.

Thanks for the big push!

[Bob Nystrom, 2015-03-10 19:14:21]

Committed patchset #2 (id:20001) manually as
ff182727c4b49f55a829645468b63dc23464741b (presubmit successful).