Introduce transformers for:

lib/src/transform/common.dart

+// Copyright (c) 2013, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+/** Common methods used by transfomers. */
+library polymer.src.transform.common;
+
+import 'dart:async';
+
+import 'package:path/path.dart' as path;
+import 'package:barback/barback.dart';
+import 'package:html5lib/dom.dart' show Document;
+import 'package:html5lib/parser.dart' show HtmlParser;
+import 'package:source_maps/span.dart' show Span;
+
+
+Future<String> getPrimaryContent(Transform transform) =>
+  transform.primaryInput.then((asset) => asset.readAsString());
+
+Future<String> getContent(Transform transform, AssetId id) =>
+  transform.getInput(id).then((asset) => asset.readAsString());
+
+/**
+ * Parses an HTML file [contents] and returns a DOM-like tree. Adds emitted
+ * error/warning to [logger].
+ */
+Document parseHtml(String contents, String sourcePath, TransformLogger logger) {
+  var parser = new HtmlParser(contents, generateSpans: true,

[Jennifer Messerly, 2013/08/19 22:57:26]

One recurring issue in DWC was the inability to supply the "transport level
encoding".


Ideally, in their "build file" or whatever the barback equivalent is, the user
can configure that their server is using "utf-8" and therefore html5lib does not
need to detect encoding.

Is that possible?

An alternative would be to make the "HtmlInputStream.defaultEncoding" a
parameter of html5lib, so if no other encoding is found (BOM, meta charset) it
will optimistically assume utf-8 instead of pessimistically assuming
windows-1252.

[Jennifer Messerly, 2013/08/20 17:50:56]

it sounds like Bob is saying barback has no config options right now.

I added a "defaultEncoding" to html5lib but in trying to write the doc comment,
I couldn't really justify when to use it over "encoding".

Here's my suggestion, let's just add "TODO(jmesserly): make HTTP encoding
configurable" and for now assume the server is sending charset=utf-8. Based on
this blog post from 2012:
http://googleblog.blogspot.com/2012/02/unicode-over-60-percent-of-web.html,
Unicode has already taken over, and utf-8 is the most popular unicode encoding,
so I think it's a safe bet.

[Siggi Cherem (dart-lang), 2013/08/21 20:35:42]

Done, added the utf8 argument as a default encoding. Is that what you meant?

+      sourceUrl: sourcePath);
+  var document = parser.parse();
+
+  // Note: errors aren't fatal in HTML (unless strict mode is on).
+  // So just print them as warnings.
+  for (var e in parser.errors) {

[Jennifer Messerly, 2013/08/19 22:57:26]

suggestion: filter out the following error:

if (isHtmlImport && e.errorCode == 'expected-doctype-but-got-start-tag')
continue;

[Siggi Cherem (dart-lang), 2013/08/21 20:35:42]

Done.

+    logger.warning(e.message, e.span);
+  }
+  return document;
+}
+
+/** Create an [AssetId] for a [url] seen in the [source] asset. */
+AssetId resolve(AssetId source, String url, TransformLogger logger, Span span) {
+  if (url == null || url == '') return null;

[Jennifer Messerly, 2013/08/19 22:57:26]

generally, it's nice if only one of these is allowed as the sentinel value

[Siggi Cherem (dart-lang), 2013/08/21 20:35:42]

Not sure I follow - in this case this is more checking for invalid input. We
could check for this on the call site, but I think resolve should be robust
enough to handle this correctly.

+  var uri = Uri.parse(url);
+  if (uri.host != '' || uri.scheme != '' || path.isAbsolute(url)) {
+    logger.error('absolute paths not allowed here: "$url"', span);

[Jennifer Messerly, 2013/08/19 22:57:26]

what is "here" :)

[Siggi Cherem (dart-lang), 2013/08/21 20:35:42]

"here" is now gone :)

+    return null;
+  }
+
+  var package;
+  var targetPath;
+  var segments = path.split(url);
+  if (segments[0] == 'packages') {
+    if (segments.length < 3) {
+      logger.error("incomplete packages/ path. It should have at least 3 "
+          "segments packages/name/path-from-name's-lib-dir", span);
+      return null;
+    }
+    package = segments[1];
+    targetPath = path.join('lib', path.joinAll(segments.sublist(2)));
+  } else if (segments[0] == 'assets') {
+    if (segments.length < 3) {
+      logger.error("incomplete assets/ path. It should have at least 3 "
+          "segments assets/name/path-from-name's-asset-dir", span);
+    }
+    package = segments[1];
+    targetPath = path.join('asset', path.joinAll(segments.sublist(2)));
+  } else {
+    package = source.package;
+    targetPath = path.normalize(path.join(path.dirname(source.path), url));
+  }

[Bob Nystrom, 2013/08/19 22:28:59]

Pub serve has identical parsing code for this. We should probably move it into
barback so you can just call it directly. File a bug and add a TODO?

[Siggi Cherem (dart-lang), 2013/08/21 20:35:42]

Done.

+  return new AssetId(package, targetPath);
+}