Fix memory issues with the Data Reduction proxy LevelDB

Fix memory issues with the Data Reduction proxy LevelDB

Delete the whole database on DeleteHistoricalDataUsage so that the log
file isn't bloated with deletes to entries that don't exist. Also, don't 
reuse the log on open.

Store if there's historical data in a pref and only clear the data usage
stats if it's been populated before.

BUG=710909
Review-Url: https://codereview.chromium.org/2810393002
Cr-Commit-Position: refs/heads/master@{#464600}
Committed: https://chromium.googlesource.com/chromium/src/+/945a5edc332c030923556edde964cce5a4e24795

[megjablon, 2017-04-13 04:16:18]

Description was changed from

==========
Only delete the Data Reduction data usage stats if they've been populated

Store if there's historical data in a pref and only clear the data usage
stats if ti's been populated before.

BUG=710909
==========

to

==========
Only delete the Data Reduction data usage stats if they've been populated

Store if there's historical data in a pref and only clear the data usage
stats if it's been populated before.

BUG=710909
==========

[megjablon, 2017-04-13 19:52:18]

Patchset #1 (id:1) has been deleted

[megjablon, 2017-04-13 19:52:39]

The CQ bit was checked by megjablon@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-04-13 19:53:08]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[megjablon, 2017-04-13 19:54:17]

Description was changed from

==========
Only delete the Data Reduction data usage stats if they've been populated

Store if there's historical data in a pref and only clear the data usage
stats if it's been populated before.

BUG=710909
==========

to

==========
Fix memory issues with the Data Reduction proxy LevelDB

Delete the whole data base on DeleteHistoricalDataUsage so that the log
file isn't bloated with deletes to entries that don't exist. Also, don't 
reuse the log on open.

Store if there's historical data in a pref and only clear the data usage
stats if it's been populated before.

BUG=710909
==========

[megjablon, 2017-04-13 19:57:19]

megjablon@chromium.org changed reviewers:
+ ryansturm@chromium.org

[megjablon, 2017-04-13 19:57:20]

Might still have to fix tests, but PTAL at this fix. I should get it landed
ASAP.

[megjablon, 2017-04-13 20:00:24]

Description was changed from

==========
Fix memory issues with the Data Reduction proxy LevelDB

Delete the whole data base on DeleteHistoricalDataUsage so that the log
file isn't bloated with deletes to entries that don't exist. Also, don't 
reuse the log on open.

Store if there's historical data in a pref and only clear the data usage
stats if it's been populated before.

BUG=710909
==========

to

==========
Fix memory issues with the Data Reduction proxy LevelDB

Delete the whole database on DeleteHistoricalDataUsage so that the log
file isn't bloated with deletes to entries that don't exist. Also, don't 
reuse the log on open.

Store if there's historical data in a pref and only clear the data usage
stats if it's been populated before.

BUG=710909
==========

[commit-bot: I haz the power, 2017-04-13 20:19:26]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-04-13 20:19:27]

Dry run: Try jobs failed on following builders:
  linux_chromium_rel_ng on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  mac_chromium_rel_ng on master.tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_...)

[megjablon, 2017-04-13 20:45:19]

The CQ bit was checked by megjablon@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-04-13 20:45:55]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[RyanSturm, 2017-04-13 21:17:59]

also talked about prefs offline.

https://codereview.chromium.org/2810393002/diff/60001/components/data_reducti...
File components/data_reduction_proxy/core/browser/data_store_impl.cc (right):

https://codereview.chromium.org/2810393002/diff/60001/components/data_reducti...
components/data_reduction_proxy/core/browser/data_store_impl.cc:152:
LOG(WARNING) << "Deleting Data Reduction Proxy LevelDB";
This LOG probably isn't great to have since it might be noisy now that it isn't
just for corrupt LevelDB stuff.

https://codereview.chromium.org/2810393002/diff/60001/components/data_reducti...
File components/data_reduction_proxy/core/browser/data_store_impl.h (right):

https://codereview.chromium.org/2810393002/diff/60001/components/data_reducti...
components/data_reduction_proxy/core/browser/data_store_impl.h:40: // returns a
|CORRUPTED| status.
nit s/status/status or the database is cleared/

[commit-bot: I haz the power, 2017-04-13 21:32:04]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-04-13 21:32:05]

Dry run: This issue passed the CQ dry run.

[megjablon, 2017-04-13 21:45:21]

The CQ bit was checked by megjablon@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-04-13 21:46:34]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[megjablon, 2017-04-13 21:47:02]

The CQ bit was checked by megjablon@chromium.org to run a CQ dry run

[megjablon, 2017-04-13 21:47:07]

Patchset #4 (id:80001) has been deleted

[megjablon, 2017-04-13 21:47:19]

https://codereview.chromium.org/2810393002/diff/60001/components/data_reducti...
File components/data_reduction_proxy/core/browser/data_store_impl.cc (right):

https://codereview.chromium.org/2810393002/diff/60001/components/data_reducti...
components/data_reduction_proxy/core/browser/data_store_impl.cc:152:
LOG(WARNING) << "Deleting Data Reduction Proxy LevelDB";
On 2017/04/13 21:17:58, Ryan Sturm wrote:
> This LOG probably isn't great to have since it might be noisy now that it
isn't
> just for corrupt LevelDB stuff.

Done.

https://codereview.chromium.org/2810393002/diff/60001/components/data_reducti...
File components/data_reduction_proxy/core/browser/data_store_impl.h (right):

https://codereview.chromium.org/2810393002/diff/60001/components/data_reducti...
components/data_reduction_proxy/core/browser/data_store_impl.h:40: // returns a
|CORRUPTED| status.
On 2017/04/13 21:17:58, Ryan Sturm wrote:
> nit s/status/status or the database is cleared/

Done.

[commit-bot: I haz the power, 2017-04-13 21:48:01]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[RyanSturm, 2017-04-13 21:53:37]

lgtm % tests if those are necessary.

[megjablon, 2017-04-13 22:04:52]

On 2017/04/13 21:53:37, Ryan Sturm wrote:
> lgtm % tests if those are necessary.

The tests should already be covering this, hence why they failed and I had to
update data_reduction_proxy_test_utils.

[commit-bot: I haz the power, 2017-04-13 22:40:36]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-04-13 22:40:37]

Dry run: This issue passed the CQ dry run.

[megjablon, 2017-04-13 23:07:34]

The CQ bit was checked by megjablon@chromium.org

[commit-bot: I haz the power, 2017-04-13 23:08:26]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-04-13 23:17:26]

CQ is committing da patch.

Bot data: {"patchset_id": 100001, "attempt_start_ts": 1492124854086030,
"parent_rev": "ec2f0c7a07f26bb15dbf42c94153c370eccdbfe3", "commit_rev":
"945a5edc332c030923556edde964cce5a4e24795"}

[commit-bot: I haz the power, 2017-04-13 23:18:12]

Description was changed from

==========
Fix memory issues with the Data Reduction proxy LevelDB

Delete the whole database on DeleteHistoricalDataUsage so that the log
file isn't bloated with deletes to entries that don't exist. Also, don't 
reuse the log on open.

Store if there's historical data in a pref and only clear the data usage
stats if it's been populated before.

BUG=710909
==========

to

==========
Fix memory issues with the Data Reduction proxy LevelDB

Delete the whole database on DeleteHistoricalDataUsage so that the log
file isn't bloated with deletes to entries that don't exist. Also, don't 
reuse the log on open.

Store if there's historical data in a pref and only clear the data usage
stats if it's been populated before.

BUG=710909

Review-Url: https://codereview.chromium.org/2810393002
Cr-Commit-Position: refs/heads/master@{#464600}
Committed:
https://chromium.googlesource.com/chromium/src/+/945a5edc332c030923556edde964...
==========

[commit-bot: I haz the power, 2017-04-13 23:18:14]

Committed patchset #4 (id:100001) as
https://chromium.googlesource.com/chromium/src/+/945a5edc332c030923556edde964...