Buildbot enhancements: Builder/Tester split

build/android/buildbot_functions.sh

 #!/bin/bash
 # Copyright (c) 2012 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 #
 # Bash functions used by buildbot annotator scripts for the android
 # build of chromium.  Executing this script should not perform actions
 # other than setting variables and defining of functions.
 
 # Number of jobs on the compile line; e.g.  make -j"${JOBS}"
 JOBS="${JOBS:-4}"
 
 # Clobber build?  Overridden by bots with BUILDBOT_CLOBBER.
 NEED_CLOBBER="${NEED_CLOBBER:-0}"
 
+
+# Parse named arguments passed into the annotator script
+# and assign them global variable names

[cmp, 2012/06/19 20:46:42]

nit: please append periods to the end of all sentences in comments (here, line
68, line 72, 270, 276, 285).

+function bb_parse_args {
+  while [[ $1 ]]; do
+    case "$1" in
+      --factory-properties=*)
+        FACTORY_PROPERTIES="$(echo "$1" | sed 's/^[^=]*=//')"

[cmp, 2012/06/19 20:46:42]

The content of FACTORY_PROPERTIES and BUILD_PROPERTIES has been designed by us
to be easily marshallable into a programming language that supports JSON
imports.  Python has loads() for example.  JS could eval it.  etc.  Shell
doesn't have these things, though.  Ideally, this script using the data would be
Python to harmonize with other parts of the system, but I understand a
conversion probably won't happen soon here.

So when it comes to actually using the contents of these *PROPERTIES values, it
may be worth adding a Python helper script that takes one of them on the
command-line as one arg and a key as a second arg for which value to extract. 
Then you can get back the string value of that key's entry in the dictionary. 
The caller here could look like:

  build_url=$(helper.py "$FACTORY_PROPERTIES" "build_url")

This would give you maximum flexibility: you won't need to do sh acrobatics to
dig out the right key value, you can reuse our methods for de-serializing, and
you can open the door to switching more parts of this over to Python as-needed.

[Isaac (away), 2012/06/20 00:34:53]

Right now zip_build and extract_build already parse factory_properties and
build_properties properly.  While this seems like a good idea if I need to use
infomation from this variables in bash, I don't see the need to extract
information before passed the blobs to extract build.

Can you clarify the value in this?

On 2012/06/19 20:46:42, cmp wrote:

[cmp, 2012/06/20 17:30:06]

Right, I didn't expect you to need to extract anything from it for this change,
but maybe in the future.
It is just a note considering how to extract data from here in the future if
needed.  Feel free to ignore for now.

+        ;;
+      --build-properties=*)
+        BUILD_PROPERTIES="$(echo "$1" | sed 's/^[^=]*=//')"
+        ;;
+      *)
+        echo "@@@STEP_WARNINGS@@@"
+        echo "Warning, unparsed input argument: '$1'"
+        ;;
+    esac
+    shift
+  done
+}
+
+
 # Setup environment for Android build.
 # Called from bb_baseline_setup.
 # Moved to top of file so it is easier to find.
 function bb_setup_environment {
   export ANDROID_SDK_ROOT=/usr/local/google/android-sdk-linux
   export ANDROID_NDK_ROOT=/usr/local/google/android-ndk-r7
 }
 
 # Install the build deps by running
 # build/install-build-deps-android-sdk.sh.  This may update local tools.
 # $1: source root.
 function bb_install_build_deps {
   echo "@@@BUILD_STEP install build deps android@@@"
   local script="$1/build/install-build-deps-android-sdk.sh"
   if [[ -f "$script" ]]; then
     "$script"
   else
+    echo "@@@STEP_WARNINGS@@@"
     echo "Cannot find $script; why?"
   fi
 }
 
 # Function to force-green a bot.
 function bb_force_bot_green_and_exit {
   echo "@@@BUILD_STEP Bot forced green.@@@"
   exit 0
 }
 
 # Basic setup for all bots to run after a source tree checkout.
 # $1: source root.
+# Beyond $1: key value pairs to be parsed by bb_parse_args
 function bb_baseline_setup {
   echo "@@@BUILD_STEP cd into source root@@@"
   SRC_ROOT="$1"
+  # Remove SRC_ROOT param
+  shift
+
+  bb_parse_args "$@"
+
   if [ ! -d "${SRC_ROOT}" ] ; then
     echo "Please specify a valid source root directory as an arg"
     echo '@@@STEP_FAILURE@@@'
     return 1
   fi
   cd $SRC_ROOT
 
   if [ ! -f build/android/envsetup.sh ] ; then
     echo "No envsetup.sh"
     echo "@@@STEP_FAILURE@@@"
@@ skipping 172 matching lines @@
 function bb_run_tests {
   echo "@@@BUILD_STEP Run Tests on actual hardware@@@"
   build/android/run_tests.py --xvfb --verbose
 }
 
 # Run APK tests on an actual device.
 function bb_run_apk_tests {
   echo "@@@BUILD_STEP Run APK Tests on actual hardware@@@"
   build/android/run_tests.py --xvfb --verbose --apk=True
 }
+
+# Zip and archive a build
+function bb_zip_build {
+  echo "@@@BUILD_STEP Zip build@@@"
+  python ../../../../scripts/slave/zip_build.py
+}
+
+# Download and extract a build
+function bb_extract_build {
+  echo "@@@BUILD_STEP Download and extract build@@@"
+  if [[ -z $FACTORY_PROPERTIES ]]; then
+    echo "@@@STEP_FAILURES@@@"
+    echo "FACTORY_PROPERTIES is null"
+    return 1
+  fi
+
+  # FACTORY_PROPERTIES contains spaces
+  python ../../../../scripts/slave/extract_build.py --no-append-dir \
+    --build-dir out \
+    --target Release \
+    --factory-properties "$FACTORY_PROPERTIES"
+}