Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(859)

Side by Side Diff: src/scanner.cc

Issue 9600009: Fix input and output to handle UTF16 surrogate pairs. (Closed) Base URL: http://v8.googlecode.com/svn/branches/bleeding_edge/
Patch Set: Created 8 years, 9 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
OLDNEW
1 // Copyright 2011 the V8 project authors. All rights reserved. 1 // Copyright 2011 the V8 project authors. All rights reserved.
2 // Redistribution and use in source and binary forms, with or without 2 // Redistribution and use in source and binary forms, with or without
3 // modification, are permitted provided that the following conditions are 3 // modification, are permitted provided that the following conditions are
4 // met: 4 // met:
5 // 5 //
6 // * Redistributions of source code must retain the above copyright 6 // * Redistributions of source code must retain the above copyright
7 // notice, this list of conditions and the following disclaimer. 7 // notice, this list of conditions and the following disclaimer.
8 // * Redistributions in binary form must reproduce the above 8 // * Redistributions in binary form must reproduce the above
9 // copyright notice, this list of conditions and the following 9 // copyright notice, this list of conditions and the following
10 // disclaimer in the documentation and/or other materials provided 10 // disclaimer in the documentation and/or other materials provided
(...skipping 27 matching lines...) Expand all
38 // ---------------------------------------------------------------------------- 38 // ----------------------------------------------------------------------------
39 // Scanner 39 // Scanner
40 40
41 Scanner::Scanner(UnicodeCache* unicode_cache) 41 Scanner::Scanner(UnicodeCache* unicode_cache)
42 : unicode_cache_(unicode_cache), 42 : unicode_cache_(unicode_cache),
43 octal_pos_(Location::invalid()), 43 octal_pos_(Location::invalid()),
44 harmony_scoping_(false), 44 harmony_scoping_(false),
45 harmony_modules_(false) { } 45 harmony_modules_(false) { }
46 46
47 47
48 void Scanner::Initialize(UC16CharacterStream* source) { 48 void Scanner::Initialize(Utf16CharacterStream* source) {
49 source_ = source; 49 source_ = source;
50 // Need to capture identifiers in order to recognize "get" and "set" 50 // Need to capture identifiers in order to recognize "get" and "set"
51 // in object literals. 51 // in object literals.
52 Init(); 52 Init();
53 // Skip initial whitespace allowing HTML comment ends just like 53 // Skip initial whitespace allowing HTML comment ends just like
54 // after a newline and scan first token. 54 // after a newline and scan first token.
55 has_line_terminator_before_next_ = true; 55 has_line_terminator_before_next_ = true;
56 SkipWhiteSpace(); 56 SkipWhiteSpace();
57 Scan(); 57 Scan();
58 } 58 }
(...skipping 1020 matching lines...) Expand 10 before | Expand all | Expand 10 after
1079 } 1079 }
1080 } 1080 }
1081 } 1081 }
1082 literal.Complete(); 1082 literal.Complete();
1083 1083
1084 next_.location.end_pos = source_pos() - 1; 1084 next_.location.end_pos = source_pos() - 1;
1085 return true; 1085 return true;
1086 } 1086 }
1087 1087
1088 } } // namespace v8::internal 1088 } } // namespace v8::internal
OLDNEW

Powered by Google App Engine
This is Rietveld 408576698