qaclean:/u/sonmi/bin/qaclean
michael@0: michael@0:
The QA wrapper tests the nightly builds of NSS. The actual
michael@0: tests are being run are called from the QA script all.sh. I will add documentation
michael@0: for the actual QA soon. The main purpose of the wrapper is: find out which
michael@0: build (NSS version, date, Build Platform) to test on which machine (OS,
michael@0: OS version) and construct a summary report, which is then mailed to the
michael@0: nss developers (aka mailing list nss-qa-report@netscape.com). Please see
michael@0: also the feature section.
michael@0:
nssqa - the script that calls the actual
michael@0: qa script all.sh
michael@0:
qa_stat - sends out status reports
michael@0:
qaclean - if everything else fails
michael@0:
Sample global result, michael@0: individual result and log michael@0: files michael@0:
The QA wrapper consistst mainly of scripts, most located in security/nss/tests michael@0: and subdirectories, but run from /u/sonmi/bin michael@0:
nssqa and qa_stat, the main scripts both include a common header (header)
michael@0: and a common environment (set_environment).
michael@0:
Also used is mksymlinks and path_uniq
michael@0: and qaclean.
michael@0:
The scripts that are used on a daily basis are located in /u/sonmi/bin michael@0: and checked into security/nss/tests michael@0:
Parameters and Options are the same for most scripts. michael@0:
Parameters
michael@0:
nssversion (supported: 30b, 31, tip, default tip)
michael@0:
builddate (default - today, format mmdd)
michael@0:
Options
michael@0:
-y answer all questions with y - use at your own
michael@0: risk... ignores warnings
michael@0:
-s silent (only usefull with -y)
michael@0:
-h, -? -help you guessed right - displays the usage
michael@0:
-d debug
michael@0:
-f <filename> - write the (error)output to filename
michael@0:
-fcron writes resultfile in the same location as
michael@0: would the -cron
michael@0:
-m <mailinglist> - send filename to mailinglist
michael@0: (csl) only useful
michael@0:
with -f on nssqa
michael@0:
-l <mozroot> run on a local build - does not
michael@0: work at this time
michael@0:
-cron equivalient to -y -s -d -f $RESULTDIR/$HOST.<scriptname>
michael@0:
michael@0:
Please be aware that some iPlanet specific environments and features michael@0: are being used. michael@0:
-d Debug option might be removed from cron in a few weeks - or maybe
michael@0: not
michael@0:
-l QA on local build is not fully implemented yet - will not be implemented,
michael@0: all.sh can be called directly instead
michael@0:
Please do not use on Windows 95 and 98, ME platforms yet. michael@0:
use -d if script behaves strange or exits unexpectedly michael@0:
How to use the QA-wrapper
michael@0:
To test a build, first run nssqa on the required QA platforms (some
michael@0: buildplatforms require QA to be run on additional platforms - for example
michael@0: Solaris 2.6 has to be tested on 2.8 32 and 64bit) If QA has been run on
michael@0: multiple or all required platforms it makes sense to run qa_stat on the
michael@0: output of nssqa as well.
michael@0:
Before used on a new system (even if the same platform has been tested
michael@0: before) please use completely interactive, to see what the variables are
michael@0: being initialized to, and read the warnings. Same is true if being run
michael@0: from a different user account than svbld.
michael@0:
In any case, if you are using it, please let me know the results. michael@0:
the script that calls the actual qa script all.sh michael@0:
nssqa parameters and options michael@0:
view the script michael@0:
Pseudocode Description of nssqa
michael@0:
not quite up to date
michael@0:
header:init (global)
michael@0:
set flags and variables
michael@0: to default values
michael@0:
signal trap (for interupts
michael@0: and kills)
michael@0:
set HOST and DOMSUF variables
michael@0: if running from cron
michael@0:
parse parameters and options
michael@0:
determine os and set up
michael@0: the environment (espec. PATH)
michael@0:
set the directories to run
michael@0: in (influenced by parameters and -l option)
michael@0: set the directories for backward
michael@0: compatibility testing
michael@0:
set and initialize the tmp
michael@0: / debugging / output files
michael@0:
nssqa:init (local)
michael@0:
locking: if nssqa is already
michael@0: running on this systems (yes-exit,
michael@0:
michael@0: no-lockfile)
michael@0:
set HOST and DOMSUF variables
michael@0: if running interavtively
michael@0:
set flag to kill remaining
michael@0: selfserv processes during cleanup
michael@0:
if QA platform different
michael@0: from build platform create neccessary
michael@0:
michael@0: symbolic links
michael@0:
wait for the build to finish
michael@0: (max of 5h)
michael@0:
main:
michael@0:
repeated per test (optimized,
michael@0: debug, 32, 64 bit)
michael@0:
michael@0: set flags for this run of all.sh (optimized, debug, 32, 64 bit)
michael@0:
michael@0: set the DIST directory (where the binaries reside)
michael@0:
michael@0: kill running selfservers (sorry - just don't use the svbld
michael@0:
michael@0: account if you need to do your own testing... I will fix
michael@0:
michael@0: selfserv as soon as I can - but it hangs too often and
michael@0:
michael@0: disturbs all following QA)
michael@0:
michael@0: run all.sh
michael@0:
header:exit (global)
michael@0:
remove temporary files
michael@0:
kill remaining selfservers
michael@0:
send email to the list
michael@0:
michael@0:
errorhandling
michael@0:
Option / Parameter errors:
michael@0: Exit with usage information
michael@0:
Severe errors: Exit wit errormessage
michael@0:
michael@0: example: directory in which all.sh resides does not exist
michael@0:
michael@0: can't create files or directories
michael@0:
michael@0: build not done after 5 hours
michael@0:
michael@0: is already running
michael@0:
Other errors: User is prompted
michael@0: with the "errormessage - continue (y/n)?"
michael@0:
michael@0: example: local DIST dir does not exist (continues with next all.sh)
michael@0:
michael@0: outputdirectory does not exist (user can specify other)
michael@0:
Signals 2, 3, 15 are treated
michael@0: as severe errors
michael@0:
michael@0:
michael@0:
michael@0:
qaclean:/u/sonmi/bin/qaclean
michael@0:
michael@0:
Use qaclean as user "svbld" to get the propper permissions. It is supposed michael@0: to clean up after a "hanging" QA and will also brutally kill, interupt michael@0: and disturb any other nss related test or performance meassurement on the michael@0: named machine. NT and 2000 might require an additional reboot, since the michael@0: ps is not so good about telling us the actual programmname - so we can't michael@0: kill them... Please note that this is a brute force script, it should not michael@0: be used on a regular basis, file a bug whenever you have to use it, since michael@0: hanging QA is nothing that should occur frequently michael@0:
view the script michael@0:
What it does: michael@0:
qa_stat is the script that is being started from the svbld cron on kentuckyderby
michael@0: every morning at 10:00 and runs some (very primitive) analysis on the qa
michael@0: results.
michael@0:
I'd like to rewrite the whole thing in perl, and in a few weeks I might
michael@0: just do this...
michael@0:
view the script michael@0:
qa_stat parameters and options michael@0:
Why we need the QA wrapper michael@0:
We need the new QA wrapper, because we have to test on so many platforms, michael@0: that running the tests and evaluating the results for the nightly builds michael@0: took about an average workday. michael@0:
New Features: michael@0:
Cygnus Advantages: michael@0: