Buildbot enhancements: Builder/Tester split

build/android/buildbot_functions.sh

Index: build/android/buildbot_functions.sh
diff --git a/build/android/buildbot_functions.sh b/build/android/buildbot_functions.sh
index 4c26610d897bff73aa67db603a59263dbb1db5b6..7795fd8ad120c351afeb48a4d25e22735d1b6383 100755
--- a/build/android/buildbot_functions.sh
+++ b/build/android/buildbot_functions.sh
@@ -13,6 +13,28 @@ JOBS="${JOBS:-4}"
 # Clobber build?  Overridden by bots with BUILDBOT_CLOBBER.
 NEED_CLOBBER="${NEED_CLOBBER:-0}"
 
+
+# Parse named arguments passed into the annotator script
+# and assign them global variable names

[cmp, 2012/06/19 20:46:42]

nit: please append periods to the end of all sentences in comments (here, line
68, line 72, 270, 276, 285).

+function bb_parse_args {
+  while [[ $1 ]]; do
+    case "$1" in
+      --factory-properties=*)
+        FACTORY_PROPERTIES="$(echo "$1" | sed 's/^[^=]*=//')"

[cmp, 2012/06/19 20:46:42]

The content of FACTORY_PROPERTIES and BUILD_PROPERTIES has been designed by us
to be easily marshallable into a programming language that supports JSON
imports.  Python has loads() for example.  JS could eval it.  etc.  Shell
doesn't have these things, though.  Ideally, this script using the data would be
Python to harmonize with other parts of the system, but I understand a
conversion probably won't happen soon here.

So when it comes to actually using the contents of these *PROPERTIES values, it
may be worth adding a Python helper script that takes one of them on the
command-line as one arg and a key as a second arg for which value to extract. 
Then you can get back the string value of that key's entry in the dictionary. 
The caller here could look like:

  build_url=$(helper.py "$FACTORY_PROPERTIES" "build_url")

This would give you maximum flexibility: you won't need to do sh acrobatics to
dig out the right key value, you can reuse our methods for de-serializing, and
you can open the door to switching more parts of this over to Python as-needed.

[Isaac (away), 2012/06/20 00:34:53]

Right now zip_build and extract_build already parse factory_properties and
build_properties properly.  While this seems like a good idea if I need to use
infomation from this variables in bash, I don't see the need to extract
information before passed the blobs to extract build.

Can you clarify the value in this?

On 2012/06/19 20:46:42, cmp wrote:

[cmp, 2012/06/20 17:30:06]

Right, I didn't expect you to need to extract anything from it for this change,
but maybe in the future.
It is just a note considering how to extract data from here in the future if
needed.  Feel free to ignore for now.

+        ;;
+      --build-properties=*)
+        BUILD_PROPERTIES="$(echo "$1" | sed 's/^[^=]*=//')"
+        ;;
+      *)
+        echo "@@@STEP_WARNINGS@@@"
+        echo "Warning, unparsed input argument: '$1'"
+        ;;
+    esac
+    shift
+  done
+}
+
+
 # Setup environment for Android build.
 # Called from bb_baseline_setup.
 # Moved to top of file so it is easier to find.
@@ -30,6 +52,7 @@ function bb_install_build_deps {
   if [[ -f "$script" ]]; then
     "$script"
   else
+    echo "@@@STEP_WARNINGS@@@"
     echo "Cannot find $script; why?"
   fi
 }
@@ -42,9 +65,15 @@ function bb_force_bot_green_and_exit {
 
 # Basic setup for all bots to run after a source tree checkout.
 # $1: source root.
+# Beyond $1: key value pairs to be parsed by bb_parse_args
 function bb_baseline_setup {
   echo "@@@BUILD_STEP cd into source root@@@"
   SRC_ROOT="$1"
+  # Remove SRC_ROOT param
+  shift
+
+  bb_parse_args "$@"
+
   if [ ! -d "${SRC_ROOT}" ] ; then
     echo "Please specify a valid source root directory as an arg"
     echo '@@@STEP_FAILURE@@@'
@@ -237,3 +266,25 @@ function bb_run_apk_tests {
   echo "@@@BUILD_STEP Run APK Tests on actual hardware@@@"
   build/android/run_tests.py --xvfb --verbose --apk=True
 }
+
+# Zip and archive a build
+function bb_zip_build {
+  echo "@@@BUILD_STEP Zip build@@@"
+  python ../../../../scripts/slave/zip_build.py
+}
+
+# Download and extract a build
+function bb_extract_build {
+  echo "@@@BUILD_STEP Download and extract build@@@"
+  if [[ -z $FACTORY_PROPERTIES ]]; then
+    echo "@@@STEP_FAILURES@@@"
+    echo "FACTORY_PROPERTIES is null"
+    return 1
+  fi
+
+  # FACTORY_PROPERTIES contains spaces
+  python ../../../../scripts/slave/extract_build.py --no-append-dir \
+    --build-dir out \
+    --target Release \
+    --factory-properties "$FACTORY_PROPERTIES"
+}