restructure string decoding to support iterable use and include benchmarks for UTF-8 decoding.

restructure string decoding to support iterable use and include benchmarks for UTF-8 decoding.
R=jat@google.com,ahe@google.com,sgjesse@google.com
BUG=
TEST=

Committed: https://code.google.com/p/dart/source/detail?r=4366

[jat, 2012-02-15 17:26:51]

lgtm

I'm not sure about the utility of the iterator methods (the cases where it would
be useful, I think having a CharInputStream or something on top of an
InputStream/etc would be more useful, and you couldn't really adapt the iterator
version for that use anyway), but if other people want it I don't guess it
hurts.

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
File utils/string_encoding/unicode_core.dart (right):

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
utils/string_encoding/unicode_core.dart:222: class ListRangeIteratorImpl<T>
implements ListRangeIterator<T> {
Why not _ListRange if it is internal?

[dcarlson, 2012-02-15 17:31:16]

Thanks for the feedback.

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
File utils/string_encoding/unicode_core.dart (right):

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
utils/string_encoding/unicode_core.dart:222: class ListRangeIteratorImpl<T>
implements ListRangeIterator<T> {
On 2012/02/15 17:26:51, jat wrote:
> Why not _ListRange if it is internal?

I want to use this in libraries (like utf16.dart). This entire library is meant
as implementation, but I don't want to source it because it gets used in
multiple libraries. Alternatively, I could see some version of this become
generally useful.

[Søren Gjesse, 2012-02-16 15:08:18]

lgtm

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
File utils/string_encoding/utf16.dart (right):

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
utils/string_encoding/utf16.dart:25: * Decodes the UTF-16BE bytes as an
iterable. Thus, the consumer can only convert
Long line.

This is repeated in 3 other places.

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
utils/string_encoding/utf16.dart:28: * Set the replacementCharacter to null to
throw an IllegalArgumentException
replacementCharacter in []s. Maybe also explain the default value for
replacementCharacter.

This is repeated in other places.

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
utils/string_encoding/utf16.dart:241: int _length = length != null ? length :
utf16EncodedBytes.length - offset;
Can't you just use length instead of introducing _length?

[dcarlson, 2012-02-17 17:32:33]

stoped using introduced variable _length. Improved docs.

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
File utils/string_encoding/utf16.dart (right):

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
utils/string_encoding/utf16.dart:25: * Decodes the UTF-16BE bytes as an
iterable. Thus, the consumer can only convert
On 2012/02/16 15:08:18, Søren Gjesse wrote:
> Long line.
> 
> This is repeated in 3 other places.

Done.

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
utils/string_encoding/utf16.dart:28: * Set the replacementCharacter to null to
throw an IllegalArgumentException
On 2012/02/16 15:08:18, Søren Gjesse wrote:
> replacementCharacter in []s. Maybe also explain the default value for
> replacementCharacter.
> 
> This is repeated in other places.

Done.

https://chromiumcodereview.appspot.com/9410001/diff/1/utils/string_encoding/u...
utils/string_encoding/utf16.dart:241: int _length = length != null ? length :
utf16EncodedBytes.length - offset;
On 2012/02/16 15:08:18, Søren Gjesse wrote:
> Can't you just use length instead of introducing _length?

Done.