Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(3)

Unified Diff: third_party/gsutil/boto/boto/emr/step.py

Issue 10199002: Upgrade gsutil to 3.4 (Closed) Base URL: https://dart.googlecode.com/svn/branches/bleeding_edge/dart
Patch Set: Addressed comments Created 8 years, 8 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « third_party/gsutil/boto/boto/emr/instance_group.py ('k') | third_party/gsutil/boto/boto/exception.py » ('j') | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: third_party/gsutil/boto/boto/emr/step.py
diff --git a/third_party/gsutil/20110627/boto/boto/emr/step.py b/third_party/gsutil/boto/boto/emr/step.py
similarity index 95%
rename from third_party/gsutil/20110627/boto/boto/emr/step.py
rename to third_party/gsutil/boto/boto/emr/step.py
index 74bef3a13a5bc9d9b6f629f386e7790ea2be9023..15dfe8897f8dd1591568b16f7c175ba722a29fb4 100644
--- a/third_party/gsutil/20110627/boto/boto/emr/step.py
+++ b/third_party/gsutil/boto/boto/emr/step.py
@@ -95,7 +95,7 @@ class StreamingStep(Step):
"""
Hadoop streaming step
"""
- def __init__(self, name, mapper, reducer=None,
+ def __init__(self, name, mapper, reducer=None, combiner=None,
action_on_failure='TERMINATE_JOB_FLOW',
cache_files=None, cache_archives=None,
step_args=None, input=None, output=None,
@@ -109,6 +109,8 @@ class StreamingStep(Step):
:param mapper: The mapper URI
:type reducer: str
:param reducer: The reducer URI
+ :type combiner: str
+ :param combiner: The combiner URI. Only works for Hadoop 0.20 and later!
:type action_on_failure: str
:param action_on_failure: An action, defined in the EMR docs to take on failure.
:type cache_files: list(str)
@@ -127,6 +129,7 @@ class StreamingStep(Step):
self.name = name
self.mapper = mapper
self.reducer = reducer
+ self.combiner = combiner
self.action_on_failure = action_on_failure
self.cache_files = cache_files
self.cache_archives = cache_archives
@@ -155,6 +158,9 @@ class StreamingStep(Step):
args.extend(['-mapper', self.mapper])
+ if self.combiner:
+ args.extend(['-combiner', self.combiner])
+
if self.reducer:
args.extend(['-reducer', self.reducer])
else:
« no previous file with comments | « third_party/gsutil/boto/boto/emr/instance_group.py ('k') | third_party/gsutil/boto/boto/exception.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698