builder: fx-team-linux-pgo slave: bld-linux64-spot-220 starttime: 1470702605.21 results: exception (4) buildid: 20160808173001 builduid: 4d7a8406f94d4fd1b56b1fb59e7f712a revision: 5ff96be046af749644327f75b21ce9437d766886 ========= Started set props: master (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.213741) ========= master: http://buildbot-master94.bb.releng.use1.mozilla.com:8001/ ========= Finished set props: master (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.214160) ========= ========= Started downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.214449) ========= ========= Finished downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.237943) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.238300) ========= rm -rf scripts properties in dir /builds/slave/fx-team-lx-pgo-000000000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'scripts', 'properties'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-220.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/fx-team-lx-pgo-000000000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.003277 ========= master_lag: 0.05 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.288845) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 3 secs) (at 2016-08-08 17:30:05.289134) ========= bash -c 'python /tools/checkouts/build-tools/buildfarm/utils/archiver_client.py mozharness --repo integration/fx-team --rev 5ff96be046af749644327f75b21ce9437d766886 --destination scripts --debug' in dir /builds/slave/fx-team-lx-pgo-000000000000000/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'python /tools/checkouts/build-tools/buildfarm/utils/archiver_client.py mozharness --repo integration/fx-team --rev 5ff96be046af749644327f75b21ce9437d766886 --destination scripts --debug'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-220.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/fx-team-lx-pgo-000000000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False 2016-08-08 17:30:05,283 truncating revision to first 12 chars 2016-08-08 17:30:05,283 Setting DEBUG logging. 2016-08-08 17:30:05,283 attempt 1/10 2016-08-08 17:30:05,283 Getting archive location from https://api.pub.build.mozilla.org/archiver/hgmo/integration/fx-team/5ff96be046af?&preferred_region=us-west-2&suffix=tar.gz&subdir=testing/mozharness 2016-08-08 17:30:07,906 unpacking tar archive at: fx-team-5ff96be046af/testing/mozharness/ program finished with exit code 0 elapsedTime=3.186950 ========= master_lag: 0.01 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 3 secs) (at 2016-08-08 17:30:08.486541) ========= ========= Started set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:08.486849) ========= script_repo_revision: 5ff96be046af749644327f75b21ce9437d766886 ========= Finished set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:08.487202) ========= ========= Started downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:08.487480) ========= ========= Finished downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:08.499373) ========= ========= Started remove old nonce (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:08.499693) ========= rm -f nonce in dir /builds/slave/fx-team-lx-pgo-000000000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-f', 'nonce'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-220.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/fx-team-lx-pgo-000000000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.002852 ========= master_lag: 0.05 ========= ========= Finished remove old nonce (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:08.549191) ========= ========= Started download_token exception (results: 4, elapsed: 45 mins, 13 secs) (at 2016-08-08 17:30:08.549509) ========= Slave: bld-linux64-spot-220 IP: 10.134.55.229 Duration: 25200 URI: https://mac-v2-signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing2.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing2.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing2.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing2.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing2.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing3.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing3.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing3.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing3.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing3.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing5.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing5.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing5.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing5.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing5.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing1.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing1.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing1.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing1.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing1.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing7.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing7.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing7.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing7.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing7.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

<type 'exceptions.Exception'>: No more signing servers to try.

<type 'exceptions.Exception'>: No more signing servers to try.

Traceback (most recent call last): Failure: exceptions.Exception: No more signing servers to try. ========= Finished download_token exception (results: 4, elapsed: 45 mins, 13 secs) (at 2016-08-08 18:15:22.114119) ========= ========= Skipped (results: not started, elapsed: not started) ========= ========= Skipped (results: not started, elapsed: not started) ========= ========= Started no change (results: 0, elapsed: 0 secs) (at 2016-08-08 18:15:22.114666) ========= bash -c 'for file in `ls -1`; do cat $file; done' in dir /builds/slave/fx-team-lx-pgo-000000000000000/properties (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'for file in `ls -1`; do cat $file; done'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-220.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/fx-team-lx-pgo-000000000000000/properties SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.004891 ========= master_lag: 0.06 ========= ========= Finished no change (results: 0, elapsed: 0 secs) (at 2016-08-08 18:15:22.180907) ========= ========= Started 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2016-08-08 18:15:22.181218) ========= rm -f oauth.txt in dir /builds/slave/fx-team-lx-pgo-000000000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-f', 'oauth.txt'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-220.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/fx-team-lx-pgo-000000000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.002478 ========= master_lag: 0.06 ========= ========= Finished 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2016-08-08 18:15:22.245230) ========= ========= Started reboot skipped (results: 3, elapsed: 0 secs) (at 2016-08-08 18:15:22.245620) ========= ========= Finished reboot skipped (results: 3, elapsed: 0 secs) (at 2016-08-08 18:15:22.245966) ========= ========= Total master_lag: 0.23 =========