Chromium Code Reviews
Help | Chromium Project | Sign in
(8)

Issue 10701151: DuplicateContentResourceHandler to monitor resources and track how many times th… (Closed)

Can't Edit
Can't Publish+Mail
Start Review
Created:
2 years, 9 months ago by frankwang
Modified:
2 years, 8 months ago
CC:
chromium-reviews, joi+watch-content_chromium.org, darin-cc_chromium.org, jam
Visibility:
Public.

Description

DuplicateContentResourceHandler looks at resources that come through and track, in histograms, which resources are duplicated, how big those resources are, and what type of resources they are.

Patch Set 1 #

Total comments: 27

Patch Set 2 : #

Total comments: 4

Patch Set 3 : #

Patch Set 4 : #

Total comments: 4

Patch Set 5 : #

Total comments: 1

Patch Set 6 : #

Total comments: 17

Patch Set 7 : #

Patch Set 8 : #

Total comments: 50

Patch Set 9 : #

Total comments: 2

Patch Set 10 : #

Total comments: 30

Patch Set 11 : Changes for second full pass-through #

Total comments: 6

Patch Set 12 : #

Total comments: 4

Patch Set 13 : Removed one histogram and changed uint32 types for windows compliance #

Total comments: 2

Patch Set 14 : Eliminated implicit type force. #

Total comments: 6

Patch Set 15 : Created helper function and moved comments. #

Patch Set 16 : #

Total comments: 32

Patch Set 17 : #

Patch Set 18 : #

Total comments: 4

Patch Set 19 : fixed some nits #

Total comments: 11

Patch Set 20 : more nits #

Patch Set 21 : fixed parameter spacing #

Patch Set 22 : final change to namespace #

Patch Set 23 : changed static instance to LazyInstance #

Unified diffs Side-by-side diffs Delta from patch set Stats (+224 lines, -2 lines) Patch
M DEPS View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +1 line, -1 line 0 comments Download
M content/browser/DEPS View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +1 line, -0 lines 0 comments Download
A content/browser/renderer_host/duplicate_content_resource_handler.h View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 1 chunk +65 lines, -0 lines 0 comments Download
A content/browser/renderer_host/duplicate_content_resource_handler.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 1 chunk +134 lines, -0 lines 0 comments Download
M content/browser/renderer_host/resource_dispatcher_host_impl.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 2 chunks +11 lines, -0 lines 0 comments Download
M content/content_browser.gypi View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 2 chunks +3 lines, -0 lines 0 comments Download
M third_party/smhasher/README.chromium View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +1 line, -1 line 0 comments Download
M third_party/smhasher/smhasher.gyp View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +8 lines, -0 lines 0 comments Download
Commit: CQ not working?

Messages

Total messages: 52 (0 generated)
gavinp
https://chromiumcodereview.appspot.com/10701151/diff/1/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/1/content/browser/renderer_host/duplicate_resource_handler.cc#newcode7 content/browser/renderer_host/duplicate_resource_handler.cc:7: #include "content/browser/renderer_host/duplicate_resource_handler.h" Move this above #include <string> https://chromiumcodereview.appspot.com/10701151/diff/1/content/browser/renderer_host/duplicate_resource_handler.cc#newcode20 content/browser/renderer_host/duplicate_resource_handler.cc:20: ...
2 years, 9 months ago (2012-07-11 23:39:46 UTC) #1
gavinp
https://chromiumcodereview.appspot.com/10701151/diff/1/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/1/content/browser/renderer_host/duplicate_resource_handler.cc#newcode72 content/browser/renderer_host/duplicate_resource_handler.cc:72: I'm not sure if it's kosher to keep read_buffer_ ...
2 years, 9 months ago (2012-07-11 23:41:30 UTC) #2
frankwang
I made a few changes as specified. Using Murmur on the read_buffer_ is currently not ...
2 years, 9 months ago (2012-07-12 01:50:36 UTC) #3
frankwang
I made the segfault go away. However, when I launch the browser, it does not ...
2 years, 9 months ago (2012-07-12 21:50:36 UTC) #4
frankwang
The browser now renders pages, but the histogram is not showing.
2 years, 9 months ago (2012-07-13 04:02:34 UTC) #5
gavinp
I just tested this, and I can't believe it renders. I have to go get ...
2 years, 9 months ago (2012-07-13 21:16:04 UTC) #6
frankwang
The problem was that I was not passing through correctly on the OnWillRead. The histogram ...
2 years, 9 months ago (2012-07-14 02:06:40 UTC) #7
frankwang
https://chromiumcodereview.appspot.com/10701151/diff/13001/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/13001/content/browser/renderer_host/duplicate_resource_handler.cc#newcode104 content/browser/renderer_host/duplicate_resource_handler.cc:104: UMA_HISTOGRAM_COUNTS("Duplicate.SizeKB.Miss.CurrentCache", log10(bytes_read_)); I changed this locally to "Duplicate.Size.Miss.CurrentCache" since ...
2 years, 9 months ago (2012-07-14 02:13:59 UTC) #8
gavinp
The IOBuffer and hash are still being used wrong. However, the histograms are looking better. ...
2 years, 9 months ago (2012-07-14 19:05:06 UTC) #9
frankwang
I fixed the issue with the IOBuffer but not hash. I thought it would be ...
2 years, 9 months ago (2012-07-16 01:44:05 UTC) #10
gavinp
On 2012/07/16 01:44:05, frankwang wrote: > I fixed the issue with the IOBuffer but not ...
2 years, 9 months ago (2012-07-16 16:42:32 UTC) #11
frankwang
Hash function fixed. Let me know if there are any other issues that need to ...
2 years, 9 months ago (2012-07-16 19:54:04 UTC) #12
gavinp
This is getting a lot better. I've taken a pass through, and commented on what ...
2 years, 9 months ago (2012-07-18 12:13:32 UTC) #13
gavinp
Some more histogram ideas, not sure which are most interesting: - hit rate binned by ...
2 years, 9 months ago (2012-07-18 13:01:57 UTC) #14
gavinp
How often do we believe collisions will occur? If we trust our hash is random, ...
2 years, 9 months ago (2012-07-19 11:11:27 UTC) #15
frankwang
I made some other notes, but the main note is in duplicate_resource_handler.cc. I directed you ...
2 years, 9 months ago (2012-07-19 16:10:26 UTC) #16
gavinp
heh. http://codereview.chromium.org/10701151/diff/25002/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): http://codereview.chromium.org/10701151/diff/25002/content/browser/renderer_host/duplicate_resource_handler.cc#newcode95 content/browser/renderer_host/duplicate_resource_handler.cc:95: const std::string url_spec = request_->url().spec(); It seems you ...
2 years, 9 months ago (2012-07-19 16:14:13 UTC) #17
gavinp
Another pass through. I think we're getting pretty close! http://codereview.chromium.org/10701151/diff/26002/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): http://codereview.chromium.org/10701151/diff/26002/content/browser/renderer_host/duplicate_resource_handler.cc#newcode22 content/browser/renderer_host/duplicate_resource_handler.cc:22: ...
2 years, 9 months ago (2012-07-20 01:37:59 UTC) #18
gavinp
You should figure out why check_deps is failing in the tries. It's blocking this change ...
2 years, 9 months ago (2012-07-20 01:38:58 UTC) #19
frankwang
I made all the noted changes or commented on them in this new patch. I ...
2 years, 9 months ago (2012-07-20 04:50:00 UTC) #20
gavinp
This is getting very good. So the tools warning comes from tools/checkdeps/checkdeps.py (found this by ...
2 years, 9 months ago (2012-07-20 11:38:46 UTC) #21
gavinp
Aha, reitveld won't let me upload a patch to your issue. So, you need to ...
2 years, 9 months ago (2012-07-20 11:44:15 UTC) #22
frankwang
I fixed the issues, and I decided to just keep the browser session instead of ...
2 years, 9 months ago (2012-07-20 17:51:37 UTC) #23
gavinp
http://codereview.chromium.org/10701151/diff/40003/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): http://codereview.chromium.org/10701151/diff/40003/content/browser/renderer_host/duplicate_resource_handler.cc#newcode53 content/browser/renderer_host/duplicate_resource_handler.cc:53: int64 browser_start_time_; Why isn't this of type base::TimeTicks ? ...
2 years, 9 months ago (2012-07-20 17:58:17 UTC) #24
frankwang
I decided against that histogram in question because I don't really see any use for ...
2 years, 9 months ago (2012-07-20 19:59:13 UTC) #25
gavinp
I just sent this in to the try bots. I think it's ready to change ...
2 years, 9 months ago (2012-07-20 23:30:51 UTC) #26
frankwang
There is still an error in win_rel that I need to fix. If you have ...
2 years, 9 months ago (2012-07-21 01:36:24 UTC) #27
gavinp
I tried the fix I suggested, and fired off a win_rel job. If it works, ...
2 years, 8 months ago (2012-07-22 17:20:18 UTC) #28
frankwang
Hopefully, this eliminates the compile error for win_rel. https://chromiumcodereview.appspot.com/10701151/diff/29006/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/29006/content/browser/renderer_host/duplicate_resource_handler.cc#newcode110 content/browser/renderer_host/duplicate_resource_handler.cc:110: const ...
2 years, 8 months ago (2012-07-22 20:44:29 UTC) #29
gavinp
On 2012/07/22 20:44:29, frankwang wrote: > Hopefully, this eliminates the compile error for win_rel. I ...
2 years, 8 months ago (2012-07-22 22:34:18 UTC) #30
frankwang
Hi Darin, I've heard you've done a lot of resource handler work lately, WDYT? Frank
2 years, 8 months ago (2012-07-23 00:59:02 UTC) #31
darin (slow to review)
Just some initial comments... https://chromiumcodereview.appspot.com/10701151/diff/37005/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/37005/content/browser/renderer_host/duplicate_resource_handler.cc#newcode89 content/browser/renderer_host/duplicate_resource_handler.cc:89: MH_UINT32 contents_hash = PMurHash32_Result(pmurhash_ph1_, nit: ...
2 years, 8 months ago (2012-07-23 23:30:14 UTC) #32
frankwang
Some changes, comments, and questions. https://chromiumcodereview.appspot.com/10701151/diff/37005/content/browser/renderer_host/duplicate_resource_handler.cc File content/browser/renderer_host/duplicate_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/37005/content/browser/renderer_host/duplicate_resource_handler.cc#newcode89 content/browser/renderer_host/duplicate_resource_handler.cc:89: MH_UINT32 contents_hash = PMurHash32_Result(pmurhash_ph1_, ...
2 years, 8 months ago (2012-07-24 02:59:57 UTC) #33
frankwang
I changed the class from DuplicateResourceHandler to something more descriptive DuplicateContentResourceHandler. I have also added ...
2 years, 8 months ago (2012-07-25 21:32:14 UTC) #34
darin (slow to review)
https://chromiumcodereview.appspot.com/10701151/diff/48001/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/48001/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode17 content/browser/renderer_host/duplicate_content_resource_handler.cc:17: nit: no new line here https://chromiumcodereview.appspot.com/10701151/diff/48001/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode21 content/browser/renderer_host/duplicate_content_resource_handler.cc:21: public: nit: ...
2 years, 8 months ago (2012-07-25 22:02:05 UTC) #35
frankwang
I fixed some nits and some small parts of the code. I also added some ...
2 years, 8 months ago (2012-07-26 21:46:19 UTC) #36
darin (slow to review)
just a few nits... then you are good to go https://chromiumcodereview.appspot.com/10701151/diff/56001/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/56001/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode60 ...
2 years, 8 months ago (2012-07-31 18:21:02 UTC) #37
frankwang
nits fixed.
2 years, 8 months ago (2012-07-31 18:38:31 UTC) #38
gavinp
LGTM with nits fixed. (but I'm not an OWNER). https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode16 content/browser/renderer_host/duplicate_content_resource_handler.cc:16: ...
2 years, 8 months ago (2012-07-31 19:26:18 UTC) #39
frankwang
more nits fixed. darin, WDYT? https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode16 content/browser/renderer_host/duplicate_content_resource_handler.cc:16: namespace { On 2012/07/31 ...
2 years, 8 months ago (2012-07-31 20:26:18 UTC) #40
gavinp
https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode71 content/browser/renderer_host/duplicate_content_resource_handler.cc:71: bool DuplicateContentResourceHandler::OnReadCompleted( On 2012/07/31 20:26:18, frankwang wrote: > On ...
2 years, 8 months ago (2012-07-31 20:29:16 UTC) #41
darin (slow to review)
https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc File content/browser/renderer_host/duplicate_content_resource_handler.cc (right): https://chromiumcodereview.appspot.com/10701151/diff/54004/content/browser/renderer_host/duplicate_content_resource_handler.cc#newcode16 content/browser/renderer_host/duplicate_content_resource_handler.cc:16: namespace { On 2012/07/31 19:26:18, gavinp wrote: > Why ...
2 years, 8 months ago (2012-07-31 20:35:22 UTC) #42
darin (slow to review)
2 years, 8 months ago (2012-07-31 20:35:31 UTC) #43
darin (slow to review)
LGTM
2 years, 8 months ago (2012-07-31 20:36:05 UTC) #44
frankwang
final nits fixed. I put the anon namespace back into the content namespace. I fixed ...
2 years, 8 months ago (2012-07-31 20:40:53 UTC) #45
I haz the power (commit-bot)
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/frankwang@google.com/10701151/51005
2 years, 8 months ago (2012-07-31 20:50:11 UTC) #46
I haz the power (commit-bot)
Try job failure for 10701151-51005 (retry) on mac_rel for step "compile" (clobber build). It's a ...
2 years, 8 months ago (2012-07-31 21:10:57 UTC) #47
gavinp
On 2012/07/31 21:10:57, I haz the power (commit-bot) wrote: > Try job failure for 10701151-51005 ...
2 years, 8 months ago (2012-07-31 21:22:56 UTC) #48
frankwang
Changed to static variable to static LazyInstance::Leaky. Hope it goes through.
2 years, 8 months ago (2012-07-31 22:54:30 UTC) #49
I haz the power (commit-bot)
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/frankwang@google.com/10701151/44014
2 years, 8 months ago (2012-07-31 22:56:05 UTC) #50
I haz the power (commit-bot)
Change committed as 149324
2 years, 8 months ago (2012-08-01 00:08:11 UTC) #51
scottmg
2 years, 8 months ago (2012-08-09 15:45:27 UTC) #52
Hi,

In investigating http://crbug.com/114570, once the WebKit patches listed there
are applied, the two |set|s in this CL are causing a slow leak.

Can you think of a way to prune GlobalDuplicateRecords::content_matches_ and
::content_and_url_matches_?

Alternatively, we should exclude or handle blob urls. They are of the form
"blob:http://google.com/<guid>/<raw_binary_data>". Because of the GUID, the
metrics are probably not reporting what you want in this case anyway, as they'll
always be unique.

data: urls are worth considering too. There's no guid, but a page could be
generating many unique data: urls which could eventually fill this set also.
Sign in to reply to this message.

Powered by Google App Engine
RSS Feeds Recent Issues | This issue
This is Rietveld 700cc9d