Chromium Code Reviews
Help | Chromium Project | Sign in
(932)

Issue 10701151: DuplicateContentResourceHandler to monitor resources and track how many times th… (Closed)

Can't Edit
Can't Publish+Mail
Start Review
Created:
1 year, 9 months ago by frankwang
Modified:
1 year, 8 months ago
Reviewers:
darin, gavinp
CC:
chromium-reviews_chromium.org, joi+watch-content_chromium.org, darin-cc_chromium.org, jam
Visibility:
Public.

Description

DuplicateContentResourceHandler looks at resources that come through and track, in histograms, which resources are duplicated, how big those resources are, and what type of resources they are.

Patch Set 1 #

Total comments: 27

Patch Set 2 : #

Total comments: 4

Patch Set 3 : #

Patch Set 4 : #

Total comments: 4

Patch Set 5 : #

Total comments: 1

Patch Set 6 : #

Total comments: 17

Patch Set 7 : #

Patch Set 8 : #

Total comments: 50

Patch Set 9 : #

Total comments: 2

Patch Set 10 : #

Total comments: 30

Patch Set 11 : Changes for second full pass-through #

Total comments: 6

Patch Set 12 : #

Total comments: 4

Patch Set 13 : Removed one histogram and changed uint32 types for windows compliance #

Total comments: 2

Patch Set 14 : Eliminated implicit type force. #

Total comments: 6

Patch Set 15 : Created helper function and moved comments. #

Patch Set 16 : #

Total comments: 32

Patch Set 17 : #

Patch Set 18 : #

Total comments: 4

Patch Set 19 : fixed some nits #

Total comments: 11

Patch Set 20 : more nits #

Patch Set 21 : fixed parameter spacing #

Patch Set 22 : final change to namespace #

Patch Set 23 : changed static instance to LazyInstance #

Unified diffs Side-by-side diffs Delta from patch set Stats (+224 lines, -2 lines) Lint Patch
M DEPS View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +1 line, -1 line 0 comments 0 errors Download
M content/browser/DEPS View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +1 line, -0 lines 0 comments 0 errors Download
A content/browser/renderer_host/duplicate_content_resource_handler.h View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 1 chunk +65 lines, -0 lines 0 comments 0 errors Download
A content/browser/renderer_host/duplicate_content_resource_handler.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 1 chunk +134 lines, -0 lines 0 comments 1 errors Download
M content/browser/renderer_host/resource_dispatcher_host_impl.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 2 chunks +11 lines, -0 lines 0 comments 0 errors Download
M content/content_browser.gypi View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 2 chunks +3 lines, -0 lines 0 comments 0 errors Download
M third_party/smhasher/README.chromium View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +1 line, -1 line 0 comments 0 errors Download
M third_party/smhasher/smhasher.gyp View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +8 lines, -0 lines 0 comments 0 errors Download
Commit:

Messages

Total messages: 52
gavinp
https://chromiumcodereview.appspot.com/10701151/diff/1/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/1/content/browser/renderer_host/duplicate_resource_handler.cc#newcode7 content/browser/renderer_host/duplicate_resource_handler.cc:7: #include "content/browser/renderer_host/duplicate_resource_handler.h" Move this above #include <string> https://chromiumcodereview.appspot.com/10701151/diff/1/content/browser/renderer_host/duplicate_resource_handler.cc#newcode20 content/browser/renderer_host/duplicate_resource_handler.cc:20: ...
1 year, 9 months ago #1
gavinp
https://chromiumcodereview.appspot.com/10701151/diff/1/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/1/content/browser/renderer_host/duplicate_resource_handler.cc#newcode72 content/browser/renderer_host/duplicate_resource_handler.cc:72: I'm not sure if it's kosher to keep read_buffer_ ...
1 year, 9 months ago #2
frankwang
I made a few changes as specified. Using Murmur on the read_buffer_ is currently not ...
1 year, 9 months ago #3
frankwang
I made the segfault go away. However, when I launch the browser, it does not ...
1 year, 9 months ago #4
frankwang
The browser now renders pages, but the histogram is not showing.
1 year, 9 months ago #5
gavinp
I just tested this, and I can't believe it renders. I have to go get ...
1 year, 9 months ago #6
frankwang
The problem was that I was not passing through correctly on the OnWillRead. The histogram ...
1 year, 9 months ago #7
frankwang
https://chromiumcodereview.appspot.com/10701151/diff/13001/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/13001/content/browser/renderer_host/duplicate_resource_handler.cc#newcode104 content/browser/renderer_host/duplicate_resource_handler.cc:104: UMA_HISTOGRAM_COUNTS("Duplicate.SizeKB.Miss.CurrentCache", log10(bytes_read_)); I changed this locally to "Duplicate.Size.Miss.CurrentCache" since ...
1 year, 9 months ago #8
gavinp
The IOBuffer and hash are still being used wrong. However, the histograms are looking better. ...
1 year, 9 months ago #9
frankwang
I fixed the issue with the IOBuffer but not hash. I thought it would be ...
1 year, 9 months ago #10
gavinp
On 2012/07/16 01:44:05, frankwang wrote: > I fixed the issue with the IOBuffer but not ...
1 year, 9 months ago #11
frankwang
Hash function fixed. Let me know if there are any other issues that need to ...
1 year, 9 months ago #12
gavinp
This is getting a lot better. I've taken a pass through, and commented on what ...
1 year, 9 months ago #13
gavinp
Some more histogram ideas, not sure which are most interesting: - hit rate binned by ...
1 year, 9 months ago #14
gavinp
How often do we believe collisions will occur? If we trust our hash is random, ...
1 year, 9 months ago #15
frankwang
I made some other notes, but the main note is in duplicate_resource_handler.cc. I directed you ...
1 year, 9 months ago #16
gavinp
heh. http://codereview.chromium.org/10701151/diff/25002/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): http://codereview.chromium.org/10701151/diff/25002/content/browser/renderer_host/duplicate_resource_handler.cc#newcode95 content/browser/renderer_host/duplicate_resource_handler.cc:95: const std::string url_spec = request_->url().spec(); It seems you ...
1 year, 9 months ago #17
gavinp
Another pass through. I think we're getting pretty close! http://codereview.chromium.org/10701151/diff/26002/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): http://codereview.chromium.org/10701151/diff/26002/content/browser/renderer_host/duplicate_resource_handler.cc#newcode22 content/browser/renderer_host/duplicate_resource_handler.cc:22: ...
1 year, 9 months ago #18
gavinp
You should figure out why check_deps is failing in the tries. It's blocking this change ...
1 year, 9 months ago #19
frankwang
I made all the noted changes or commented on them in this new patch. I ...
1 year, 9 months ago #20
gavinp
This is getting very good. So the tools warning comes from tools/checkdeps/checkdeps.py (found this by ...
1 year, 9 months ago #21
gavinp
Aha, reitveld won't let me upload a patch to your issue. So, you need to ...
1 year, 9 months ago #22
frankwang
I fixed the issues, and I decided to just keep the browser session instead of ...
1 year, 9 months ago #23
gavinp
http://codereview.chromium.org/10701151/diff/40003/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): http://codereview.chromium.org/10701151/diff/40003/content/browser/renderer_host/duplicate_resource_handler.cc#newcode53 content/browser/renderer_host/duplicate_resource_handler.cc:53: int64 browser_start_time_; Why isn't this of type base::TimeTicks ? ...
1 year, 9 months ago #24
frankwang
I decided against that histogram in question because I don't really see any use for ...
1 year, 9 months ago #25
gavinp
I just sent this in to the try bots. I think it's ready to change ...
1 year, 9 months ago #26
frankwang
There is still an error in win_rel that I need to fix. If you have ...
1 year, 9 months ago #27
gavinp
I tried the fix I suggested, and fired off a win_rel job. If it works, ...
1 year, 9 months ago #28
frankwang
Hopefully, this eliminates the compile error for win_rel. https://chromiumcodereview.appspot.com/10701151/diff/29006/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/29006/content/browser/renderer_host/duplicate_resource_handler.cc#newcode110 content/browser/renderer_host/duplicate_resource_handler.cc:110: const ...
1 year, 9 months ago #29
gavinp
On 2012/07/22 20:44:29, frankwang wrote: > Hopefully, this eliminates the compile error for win_rel. I ...
1 year, 9 months ago #30
frankwang
Hi Darin, I've heard you've done a lot of resource handler work lately, WDYT? Frank
1 year, 9 months ago #31
darin
Just some initial comments... https://chromiumcodereview.appspot.com/10701151/diff/37005/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/37005/content/browser/renderer_host/duplicate_resource_handler.cc#newcode89 content/browser/renderer_host/duplicate_resource_handler.cc:89: MH_UINT32 contents_hash = PMurHash32_Result(pmurhash_ph1_, nit: ...
1 year, 9 months ago #32
frankwang
Some changes, comments, and questions. https://chromiumcodereview.appspot.com/10701151/diff/37005/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/37005/content/browser/renderer_host/duplicate_resource_handler.cc#newcode89 content/browser/renderer_host/duplicate_resource_handler.cc:89: MH_UINT32 contents_hash = PMurHash32_Result(pmurhash_ph1_, ...
1 year, 9 months ago #33
frankwang
I changed the class from DuplicateResourceHandler to something more descriptive DuplicateContentResourceHandler. I have also added ...
1 year, 9 months ago #34
darin
https://chromiumcodereview.appspot.com/10701151/diff/48001/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/48001/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode17 content/browser/renderer_host/duplicate_content_resource_handler.cc:17: nit: no new line here https://chromiumcodereview.appspot.com/10701151/diff/48001/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode21 content/browser/renderer_host/duplicate_content_resource_handler.cc:21: public: nit: ...
1 year, 9 months ago #35
frankwang
I fixed some nits and some small parts of the code. I also added some ...
1 year, 9 months ago #36
darin
just a few nits... then you are good to go https://chromiumcodereview.appspot.com/10701151/diff/56001/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/56001/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode60 ...
1 year, 8 months ago #37
frankwang
nits fixed.
1 year, 8 months ago #38
gavinp
LGTM with nits fixed. (but I'm not an OWNER). https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode16 content/browser/renderer_host/duplicate_content_resource_handler.cc:16: ...
1 year, 8 months ago #39
frankwang
more nits fixed. darin, WDYT? https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode16 content/browser/renderer_host/duplicate_content_resource_handler.cc:16: namespace { On 2012/07/31 ...
1 year, 8 months ago #40
gavinp
https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode71 content/browser/renderer_host/duplicate_content_resource_handler.cc:71: bool DuplicateContentResourceHandler::OnReadCompleted( On 2012/07/31 20:26:18, frankwang wrote: > On ...
1 year, 8 months ago #41
darin
https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode16 content/browser/renderer_host/duplicate_content_resource_handler.cc:16: namespace { On 2012/07/31 19:26:18, gavinp wrote: > Why ...
1 year, 8 months ago #42
darin
1 year, 8 months ago #43
darin
LGTM
1 year, 8 months ago #44
frankwang
final nits fixed. I put the anon namespace back into the content namespace. I fixed ...
1 year, 8 months ago #45
I haz the power (commit-bot)
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/frankwang@google.com/10701151/51005
1 year, 8 months ago #46
I haz the power (commit-bot)
Try job failure for 10701151-51005 (retry) on mac_rel for step "compile" (clobber build). It's a ...
1 year, 8 months ago #47
gavinp
On 2012/07/31 21:10:57, I haz the power (commit-bot) wrote: > Try job failure for 10701151-51005 ...
1 year, 8 months ago #48
frankwang
Changed to static variable to static LazyInstance::Leaky. Hope it goes through.
1 year, 8 months ago #49
I haz the power (commit-bot)
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/frankwang@google.com/10701151/44014
1 year, 8 months ago #50
I haz the power (commit-bot)
Change committed as 149324
1 year, 8 months ago #51
scottmg
1 year, 8 months ago #52
Hi,

In investigating http://crbug.com/114570, once the WebKit patches listed there
are applied, the two |set|s in this CL are causing a slow leak.

Can you think of a way to prune GlobalDuplicateRecords::content_matches_ and
::content_and_url_matches_?

Alternatively, we should exclude or handle blob urls. They are of the form
"blob:http://google.com/<guid>/<raw_binary_data>". Because of the GUID, the
metrics are probably not reporting what you want in this case anyway, as they'll
always be unique.

data: urls are worth considering too. There's no guid, but a page could be
generating many unique data: urls which could eventually fill this set also.
Sign in to reply to this message.

Powered by Google App Engine
RSS Feeds Recent Issues | This issue
This is Rietveld 1280:2d3e6564b7b6