Backed out changeset ce13a8b349ac (re-enabling mozharness 0.4)

.gitignore

@@ -0,0 +1 @@
+.hgignore

.hgignore

@@ -1,5 +1,7 @@
 syntax: glob
 
+.hgignore
+.gitignore
 *.log
 *.pyc
 *.swp

CHANGES

@@ -0,0 +1,318 @@
+=== 0.4 ===
+2011-07-24  Aki Sasaki  <aki@mozilla.com>
+* scripts/mozmill_updates.py (MozmillUpdate.run_mozmill):
+    Added more substrs to MozmillErrorList.
+
+2011-07-21  Aki Sasaki  <aki@mozilla.com>
+* scripts/mozmill_updates.py (MozmillUpdate.run_mozmill):
+    Created report_json in work_dir, then copied to upload_dir at the end.
+
+* scripts/mozmill_updates.py (MozmillUpdate._clobber):
+    No longer mkdir_p work_dir / upload_dir.
+
+* mozharness/base/python.py (create_virtualenv):
+    Now using abspath of virtualenv_path.
+
+* mozharness/base/script.py (BaseScript.run):
+    Copied log files to upload_dir at end of run().
+
+* mozharness/base/script.py (BaseScript.copy_to_upload_dir):
+    Added max_backups, short_desc, long_desc.
+
+* test/test_base_script.py (TestHelperFunctions._create_temp_file):
+    Allowed for custom contents
+
+2011-07-20  Aki Sasaki  <aki@mozilla.com>
+* mozharness/base/script.py (OSMixin.rmtree):
+    return 0 on success, -1 on unable to remove path
+
+* mozharness/base/script.py (BaseScript.copy_to_upload_dir):
+    Quick'n'dirty.  Need to test, write tests. This has been on my todo list for a long time; glad to have it.
+
+2011-07-19  Aki Sasaki  <aki@mozilla.com>
+* scripts/mozmill_updates.py (MozmillUpdate):
+    Use VirtualenvMixin and virtualenv_config_options from new mozharness/base/python.py file.
+    Specify virtualenv_modules in __init__; if this diverges between versions we can move to config files.
+
+* mozharness/base/python.py: Created file.
+    Tear out the virtualenv creation/usage generic functions from scripts/mozmill_updates.py and into this generic library.
+
+* TODO: Removed file
+
+* mozharness/base/script.py (query_abs_dirs):
+    put upload_dir in work_dir/
+    keep logs under base_work_dir/
+    TODO: copy_to_upload_dir with file rotation, copy logs there
+    This will let me clobber work_dir without blowing away logs
+
+2011-07-18  Aki Sasaki  <aki@mozilla.com>
+* mozharness/scripts/mozmill_updates.py:
+    Added --create-virtualenv action. Not sure if this is useful or needed, but I was asked why I didn't, so here it is.
+    Added error checking + report.json parsing.
+
+* mozharness/base/script.py (run):
+    Exit self.return_code. Initialize in __init__.
+
+* mozharness/base/script.py (new_log_obj):
+    Change default log_name from 'test' to 'log'
+
+2011-07-15  Aki Sasaki  <aki@mozilla.com>
+* mozharness/scripts/mozmill_updates.py:
+    Added file.
+
+* mozharness/base/config.py (_create_config_parser):
+    Set default work_dir to 'build' instead of 'work_dir'
+
+* mozharness/base/script.py (query_abs_dirs):
+    Set default upload_dir to 'upload' instead of 'upload_dir'
+
+2011-07-12  Aki Sasaki  <aki@mozilla.com>
+* mozharness/base/config.py (parse_config_file):
+    Explicitly close fh when using .json config files
+
+* mozharness/base/vcs/mercurial.py (MercurialMixin): removed
+
+* mozharness/base/vcs/mercurial.py (MercurialScript): removed
+
+* mozharness/base/vcs/vcsbase.py (VCSMixin):
+    Created to use MercurialVCS (or any other VCS object).
+
+* mozharness/base/vcs/vcsbase.py (VCSScript):
+    Quick'n'dirty wrapper around VCSScript + BaseScript.
+
+* mozharness/base/vcs/vcsbase.py (MercurialScript):
+    Quick'n'dirty wrapper around VCSScript, with self.default_vcs = 'hg'
+
+* mozharness/l10n/multi_locale_build.py (MultiLocaleBuild):
+    s,scm_checkout_repos,vcs_checkout_repos.
+    Pointed to the new vcsbase.MercurialScript
+
+* scripts/signdebs.py (MaemoDebSigner.create_repos):
+    s,scm_checkout_repos,vcs_checkout_repos.
+    Pointed to the new vcsbase.MercurialScript
+
+* configs/multi_locale/*:
+    s,dir_name,dest,g
+
+* configs/deb_repos/*:
+    s,dir_name,dest,g
+
+* test/networked/test_base_vcs_vcsbase.py renamed from test_base_vcs_mercurial.py
+    Run a vcs_checkout.  Clobber no longer applies, though I may want to re-add.
+
+2011-05-03  Aki Sasaki  <aki@mozilla.com>
+
+* mozharness/base/vcs/mercurial.py (MercurialVCS.push):
+    If hg_ver < 1.6.0, don't use --new-branch. (for windows, since mozilla-build appears to have hg 1.5.4. we need to decide whether we want some sort of exception or fatal here? i'll let script writers decide that.)
+
+* mozharness/base/script.py (ShellMixin.get_output_from_command):
+    Explicitly closed tmp_stdout, tmp_stderr, fh filehandles for windows.
+
+* test/test_base_vcs_mercurial.py (*):
+    Altered to pass some tests on windows.
+    TestMakeAbsolute.* disabled on windows.
+    init_hgrepo.sh called via os.path.join and explicitly calling bash due to windows.
+
+* test/test_base_script.py (*):
+    Altered to pass tests on windows.
+    test_chmod no longer running on windows.
+    test_chdir uses os.path.join() now.
+
+* unit.sh:
+    Added pylint as a dependency in the header.
+    Ignoring perma-warnings from pylint.
+    Platform detection to try to work on windows, mac, linux
+
+2011-05-02  Aki Sasaki  <aki@mozilla.com>
+
+* mozharness/base/vcs/mercurial.py (MercurialVCS.*):
+    Added self.info() calls at the beginnings of methods to announce what we're planning to do.
+
+* mozharness/base/vcs/script.py (ShellMixin.get_output_from_command):
+    Allow for throw_exception=True.
+    Clean up *before* self.fatal() or raise
+
+* mozharness/base/vcs/script.py (ShellMixin.run_command):
+    Allow for throw_exception=True
+
+* mozharness/base/vcs/mercurial.py (MercurialVCS._ensure_shared_repo_and_revision):
+    self.info() if nuking dest due to lack of DEST/.hg/sharedpath
+    Don't nuke shared_path if not needed.
+
+* mozharness/base/vcs/mercurial.py (MercurialVCS.ensure_repo_and_revision):
+    Try to pull dest if it exists, rather than nuke/clone.  Not sure if this is the desired behavior, but it's the behavior specified in the unit tests.
+
+* mozharness/base/vcs/mercurial.py (MercurialVCS.pull):
+    self.error() if dest doesn't exist
+
+* mozharness/base/vcs/mercurial.py (MercurialVCS.clone):
+    self.info() that we're about to rmtree(dest)
+
+* mozharness/base/log.py (LogMixin.dump_exception):
+    Allow for message=None
+
+* mozharness/base/log.py (LogMixin.log):
+    Changed to use _log_level_at_least and _print, so we can hide wanted errors during unittest runs.
+
+* mozharness/base/log.py (LogMixin._log_level_at_least): Added method.
+
+* mozharness/base/log.py (LogMixin._print): Added method.
+
+* mozharness/base/log.py (LogMixin.debug):
+    No longer checking to see if log_level is debug, as that check happens in log() now.
+
+* test/test_base_vcs_mercurial:
+    Ported the rest of the tests from build/tools.
+
+2011-04-28  Aki Sasaki  <aki@mozilla.com>
+
+* test/helper_files/init_hgrepo.sh: Added file (copied from build/tools)
+
+* mozharness/base/script.py (ShellMixin.query_env):
+    Fixed partial_env logic.
+
+* test/test_base_script.py:
+    Added test_chmod method.
+    Renamed TestSummary to TestScriptLogging.
+    Added query_env testing.
+
+* mozharness/base/script.py (OSMixin.chown): Removed method.
+    I can't see using it as a non-root user; as a root user we'll need to sudo.
+
+* mozharness/base/script.py (OSMixin.chmod):
+    Fixed logging.
+
+* scripts/configtest.py:
+    Ignore test_malformed* config files.
+
+* configs/test/test_malformed.json: Added file.
+
+* configs/test/test_malformed.py: Added file.
+
+* mozharness/base/config.py (parse_config_file):
+    Fixed python config files.
+    Raise assertion if config file does not end in .json or .py.
+    Raise assertion if config file is not found.
+
+* test/test_base_config.py:
+    TestJsonConfig -> TestParseConfigFile.
+    Added python config file tests, illegal/malformed config file checking.
+
+* configs/test/test.py: Added file.
+
+* mozharness/l10n/locales.py (LocalesMixin.query_locales):
+    Fixed some borked logic.
+
+* mozharness/l10n/locales.py (LocalesMixin.parse_locales_file):
+    Removed duplicate ignore_locales logic that should be handled in query_locales.
+
+* test/helper_files/locales.json: Added file.
+
+* test/helper_files/locales.txt: Added file.
+
+* test/test_l10n_locales.py: Added file.
+
+* test/*: Renamed to match directory structure.
+
+2011-04-27  Aki Sasaki  <aki@mozilla.com>
+
+* test/networked/test_script.py: Added file.
+
+* test/test_script.py (TestHelperFunctions.test_download_file): Removed method.
+
+* unit.sh:
+    Added a separate networked/non-networked call to nosetests.  In the future, we should add an option to skip the network tests.
+
+* mozharness/base/vcs/hgtool.py: Created softlink -> sourcetool.py.
+
+* mozharness/base/vcs/sourcetool.py:
+    Was able to test a successful |sourcetool.py repo|, as well as a |sourcetool.py -s shared_dir repo|.
+    Created ability to change the default VCS by looking at the filename (softlink to hgtool.py and it'll use hg by default; softlink to gittool.py and it'll use git when we enable it. I also see potential svntool.py and cvstool.py at some point.)
+
+* mozharness/base/vcs/sourcetool.py (SourceTool._pre_config_lock):
+    Verified that self.config['vcs'] is set; else self.fatal()
+
+* mozharness/base/vcs/sourcetool.py (SourceTool.source):
+    Cleaned up old TODO comments
+
+* mozharness/base/vcs/mercurial.py:
+    Worked through the various scenarios in sourcetool.py; fixing bugs.
+
+2011-04-26  Aki Sasaki  <aki@mozilla.com>
+
+* scripts/sourcetool.py: Moved from scripts/hgtool.py.
+
+* test/test_log.py (TestLog.test_console_log): Removed method.
+    Coverage didn't change at all (console logging is tested in test_script.py) but we removed a spurious line of output from unit.sh.
+
+* mozharness/base/log.py (LogMixin.warn): Removed method.
+    Also removed references to it.
+
+* mozharness/base/script.py (ShellMixin.get_output_from_command):
+    fixed the NamedTemporaryFile issue between 2.5 and 2.6, by allowing you to name your own tmpfile if you want to munge the contents later.
+
+* mozharness/base/script.py (BaseScript):
+    Pulled all os and shell methods out to mixins.
+
+* mozharness/base/vcs/mercurial.py (MercurialVCS): Added class.
+    This will hopefully be a standalone object that takes the baseScript log_obj and perhaps config, but will allow for multiple VCS objects per BaseScript.
+    First pass: changed standalone functions to a MercurialVCS object.
+
+2011-04-25  Aki Sasaki  <aki@mozilla.com>
+
+* mozharness/base/log.py (BaseLogger.log_message): Renamed from log() to be compatible with LogMixin.
+
+* mozharness/base/log.py (LogMixin): New class.
+    Pull the generic logging methods from BaseScript to this new class.
+    This is specifically to allow for non-BaseScript objects to use the log_obj from BaseScript (set a LogMixin's self.log_obj to the logging object)
+
+* mozharness/base/script.py (BaseScript):
+    Pull the generic logging methods from BaseScript to a new LogMixin class.
+    Change BaseScript to inherit LogMixin by default.
+
+* mozharness/base/config.py (BaseConfig.parse_args):
+    Fixed the args list sent to parse_args (sys.argv[1:])
+    Set self.options and self.args after parse_args for later access in BaseScript._pre_config_lock
+
+* mozharness/base/config.py (ExtendedOptionParser):
+    Removed self.variables hack and add_option override.
+
+* mozharness/base/script.py (__init__):
+    Added a _pre_config_lock() call before _config_lock() to allow for things like hgtool reading the buildprops json into self.config.
+    Moved --simple-log and --multi-log options to mozharness.base.config._create_config_parser.
+
+* mozharness/base/script.py (_config_lock):
+    Renamed to match _pre_config_lock (was _lock_config).
+
+* mozharness/base/config.py (_create_config_parser):
+    Allow for --ACTION to do the same thing as --only-ACTION.
+    Added action and logging OptionGroups.
+    Moved --simple-log and --multi-log options here.
+    Disable noop option argument.
+    Removed dependency on config_parser.variables (config_parser.defaults.keys() gives the same info).
+
+    The noop option disable forced a change of s,self.config['noop'],self.config.get('noop'), throughout mozharness.
+
+* scripts/hgtool.py: New file.
+
+* CHANGES: New file.
+
+* TODO: New file.
+
+* mozharness/__init__.py: Bumped version to 0.4.
+
+2011-04-21  Aki Sasaki  <aki@mozilla.com>
+
+* mozharness/base/script.py (MercurialMixin, MercurialScript):
+    Moved to mozharness/base/vcs/mercurial.py.
+    Adjusted other files accordingly.
+
+* mozharness/base/script.py (BaseScript.dump_exception): New function.
+
+* mozharness/base/script.py (BaseScript.copyfile):
+    Adjusted to use dump_exception and a tighter try/except.
+
+* mozharness/base/config.py, script.py, scripts/configtest.py, test/*:
+    Switched simplejson/json import order, as simplejson is faster.
+
+=== 0.3 and before are non-changelogged. ===

mozharness/__init__.py

@@ -1 +1 @@
-version = "0.3"
+version = "0.4"