[Devtools] Product registry to support prefix & import of data

[Devtools] Product registry to support prefix & import of data

Updated product registry to support prefix domains and hash domain
names. We are using top 8 bytes of an sha1 hash for our domain lookup
key.

Truth table for domain lookup:
*** All prefixes assume base domain of "foo.bar" ***
(Top row prefixes, left row domain patterns)
             |  a  |  *  | [empty string]
foo.bar      |  0  |  1  | 1
ax.foo.bar   |  1  |  1  | 0
ax.b.foo.bar |  0  |  1  | 0
b.ax.foo.bar |  1  |  1  | 0

* Also added SHA1 from https://codereview.chromium.org/2766823002/.

R=caseq,alph
BUG=703762
Review-Url: https://codereview.chromium.org/2772493002
Cr-Commit-Position: refs/heads/master@{#459776}
Committed: https://chromium.googlesource.com/chromium/src/+/9626beccc3f01f82a06a05e563c40b7494ccd1ac

[allada, 2017-03-22 19:08:31]

Patchset #1 (id:1) has been deleted

[allada, 2017-03-22 19:08:48]

allada@chromium.org changed reviewers:
+ pfeldman@chromium.org

[allada, 2017-03-22 19:08:48]

PTL

[allada, 2017-03-22 19:26:54]

Patchset #1 (id:20001) has been deleted

[pfeldman, 2017-03-23 20:32:25]

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
File
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js
(right):

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js:14:
for (var domainParts = domain.split('.'); domainParts.length > 1;
domainParts.shift()) {
for (var index = domain.indexOf('.'); index >= 0; index = domain.indexOf('.',
index + 1)) {
  var subDomain = domain.substring(index + 1);
  ...
}

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js:22:
for (var prefix of Object.keys(prefixes)) {
old way? for (var prefix in entry.prefixes) {}

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js:23:
var domainPrefix = subDomain.substr(0, prefix.length);
subDomain will never include the prefix, no?

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js:31:
if (domain.startsWith('www.')) {
This should have been handled above?

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js:55:
for (var prefix of Object.keys(entry.prefixes)) {
old way? for (var prefix in entry.prefixes) {}

[allada, 2017-03-24 18:17:04]

PTaL - The loop is a little different.

The rules of the loop are:

* Do not process when the subdomain it's checking does not have any "." (ie: no
more subdomains). This will make it not check "com", "org", exc...
* On first iteration of the loop only look at '' and '*' for a match in that
order.
* Only look at the previous subdomain part, instead of the entire subdomain (ie:
"ab.foo.bar" will only look at "ab" when substringing, instead of "ab.foo.bar"
[this protects against getting "." in the subdomain pattern.])

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
File
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js
(right):

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js:14:
for (var domainParts = domain.split('.'); domainParts.length > 1;
domainParts.shift()) {
On 2017/03/23 20:32:24, pfeldman wrote:
> for (var index = domain.indexOf('.'); index >= 0; index = domain.indexOf('.',
> index + 1)) {
>   var subDomain = domain.substring(index + 1);
>   ...
> }

Done.

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js:22:
for (var prefix of Object.keys(prefixes)) {
On 2017/03/23 20:32:24, pfeldman wrote:
> old way? for (var prefix in entry.prefixes) {}

Done.

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js:23:
var domainPrefix = subDomain.substr(0, prefix.length);
On 2017/03/23 20:32:24, pfeldman wrote:
> subDomain will never include the prefix, no?

Done.

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js:31:
if (domain.startsWith('www.')) {
On 2017/03/23 20:32:24, pfeldman wrote:
> This should have been handled above?

I spoke to my contact at 3pas team and their data is structured in an unfriendly
way to hashing and dealing with "www.", We came to the agreement that we can
strip out the "www." when it is the only subdomain and do the same on the hash
table lookup side and it should be sufficient.

https://codereview.chromium.org/2772493002/diff/40001/third_party/WebKit/Sour...
third_party/WebKit/Source/devtools/front_end/product_registry/ProductNameForURL.js:55:
for (var prefix of Object.keys(entry.prefixes)) {
On 2017/03/23 20:32:24, pfeldman wrote:
> old way? for (var prefix in entry.prefixes) {}

Done.

[pfeldman, 2017-03-24 22:01:34]

lgtm

[allada, 2017-03-25 00:03:35]

The CQ bit was checked by allada@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-03-25 00:04:07]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-25 01:29:48]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-25 01:29:49]

Dry run: Try jobs failed on following builders:
  linux_chromium_chromeos_rel_ng on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)

[allada, 2017-03-25 01:31:07]

The CQ bit was checked by allada@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-03-25 01:31:28]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-25 02:46:21]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-25 02:46:22]

Dry run: This issue passed the CQ dry run.

[allada, 2017-03-27 00:01:55]

The CQ bit was checked by allada@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-03-27 00:02:06]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-27 01:37:55]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-27 01:37:56]

Dry run: This issue passed the CQ dry run.

[allada, 2017-03-27 14:22:32]

The CQ bit was checked by allada@chromium.org

[allada, 2017-03-27 14:22:32]

The patchset sent to the CQ was uploaded after l-g-t-m from
pfeldman@chromium.org
Link to the patchset: https://codereview.chromium.org/2772493002/#ps80001
(title: "Merge branch 'ADD_SHA1' into NEW_PRODUCT_REGISTRY_STRUCTURE")

[commit-bot: I haz the power, 2017-03-27 14:22:51]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-27 14:26:33]

CQ is committing da patch.

Bot data: {"patchset_id": 80001, "attempt_start_ts": 1490624552206060,
"parent_rev": "97660d9bcf2f675cae26595987837c4fe67e8d97", "commit_rev":
"9626beccc3f01f82a06a05e563c40b7494ccd1ac"}

[commit-bot: I haz the power, 2017-03-27 14:27:24]

Description was changed from

==========
[Devtools] Product registry to support prefix & import of data

Updated product registry to support prefix domains and hash domain
names. We are using top 8 bytes of an sha1 hash for our domain lookup
key.

Truth table for domain lookup:
*** All prefixes assume base domain of "foo.bar" ***
(Top row prefixes, left row domain patterns)
             |  a  |  *  | [empty string]
foo.bar      |  0  |  1  | 1
ax.foo.bar   |  1  |  1  | 0
ax.b.foo.bar |  0  |  1  | 0
b.ax.foo.bar |  1  |  1  | 0

* Also added SHA1 from https://codereview.chromium.org/2766823002/.

R=caseq,alph
BUG=703762
==========

to

==========
[Devtools] Product registry to support prefix & import of data

Updated product registry to support prefix domains and hash domain
names. We are using top 8 bytes of an sha1 hash for our domain lookup
key.

Truth table for domain lookup:
*** All prefixes assume base domain of "foo.bar" ***
(Top row prefixes, left row domain patterns)
             |  a  |  *  | [empty string]
foo.bar      |  0  |  1  | 1
ax.foo.bar   |  1  |  1  | 0
ax.b.foo.bar |  0  |  1  | 0
b.ax.foo.bar |  1  |  1  | 0

* Also added SHA1 from https://codereview.chromium.org/2766823002/.

R=caseq,alph
BUG=703762

Review-Url: https://codereview.chromium.org/2772493002
Cr-Commit-Position: refs/heads/master@{#459776}
Committed:
https://chromium.googlesource.com/chromium/src/+/9626beccc3f01f82a06a05e563c4...
==========

[commit-bot: I haz the power, 2017-03-27 14:27:25]

Committed patchset #3 (id:80001) as
https://chromium.googlesource.com/chromium/src/+/9626beccc3f01f82a06a05e563c4...