Index: /main/waeup.sirp/branches/ulif-stress-multimech/README.txt
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/README.txt	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/README.txt	(revision 7478)
@@ -0,0 +1,119 @@
+waeup.stresstest
+****************
+
+This package provides stresstests for *existing* WAeUP SIRP sites.
+
+It deploys the Python multi-mechanize programme/library to fire tons
+of requests against a running site. So, you don't want to use it with
+sites that are under load already. The requests try to simulate users
+using the portal simultanously.
+
+multi-mechanize registers the numbers and latencies of single requests
+and generates graphics/tables with the results. This way you can see
+what actions/requests need how much time. You can also see how the
+server reacts when many concurrent users use the portal.
+
+`waeup.stresstest` is code-wise independent from `waeup.sirp`. You can
+run it on any machine that can do HTTP requests to the target machine
+(which means the machine that should be stress-tested).
+
+Installation is a bit difficult (numpy and matplotlib, two complex
+Python libs for statistical computations) are required and not easily
+installable in the usual buildout-way (well, numpy is but matplotlib
+is a constant source of trouble in that respect). Please refer to the
+``Installation`` section below.
+
+Installation
+============
+
+The following steps are tested on Ubuntu Linux only.
+
+Requirements
+------------
+
+* Python 2.6 or Python 2.7.
+
+* Python packages numpy, mechanize and matplotlib
+
+With recent Ubuntu installs it should be possible to install the
+required libs systemwide by doing::
+
+  $ sudo apt-get install python-numpy
+  $ sudo apt-get install python-matplotlib
+  $ sudo apt-get install python-mechanize
+
+With older Ubuntu releases (for instance ones, that do not ship with
+Python 2.6 or newer), you have to install Python 2.6 or 2.7 locally
+and then install both packages using ``easy_install`` or ``pip``.
+
+The latter is also possible in ``virtualenv`` environments, i.e. if
+you don't want to install the additional packages system-wide.
+
+Install of ``multi-mechanize`` then is especially, ehrm, improvable,
+because this package (yet) lacks support for any package management
+(no setup.py or similar; just plain code packed in a ZIP file for
+download).
+
+We therefore packed the sources of version "1.011" (?) here and added
+some own project which can be run. See usage instructions below.
+
+Usage
+=====
+
+Quick guide:
+
+1) Make sure configuration is up-to-date in
+   `multi-mechanize/projects/sirp1/config.cfg`. This 'project' will
+   run against a SIRP portal installed locally as ``app-stress``
+
+2) Create a local waeup.sirp portal running on localhost:8080 and
+   named ``app-stress``
+
+3) Start the portal.
+
+4) On the shell, run::
+
+     $ cd multi-mechanize
+     $ python multi-mechanize sirp1
+
+The latter tells multi mechanize to run the stress tests in the
+``projects/sirp1`` directory. Make sure that the Python interpreter
+you use here is the one that has support for matplotlib and numpy.
+
+Results will be as HTML page in ``projects/sirp1/results/``.
+
+
+Tweaking and Playing Around
+===========================
+
+The current stress tests only include simulation of a bunch of users
+simultaneously trying to access the portal home page and the portal
+applicants page (as anonymous users).
+
+You can tweak and change ``projects/sirp1/test_scripts/`` to do
+different things. See http://code.google.com/p/multi-mechanize/ for
+first steps/hints.
+
+
+ToDo
+====
+
+multi-mechanize currently makes no use of Python packaging,
+entry-points and the like. Improving it in this way and making it a
+first-class-citizen of the Python 'packaging' community would be a
+first step to make it more useful for others.
+
+Notes
+=====
+
+Installing matplotlib with buildout (notes)
+-------------------------------------------
+
+To install `waeup.stresstest` you need matplotlib which in turn
+depends on `numpy`. While `numpy` can be installed during
+buildout-runs, `matplotlib` is a bit difficult. It does a manual check
+for already installed packages and does not detect already fetched
+numpy installs. As a result matplotlib cannot be installed easily with
+buildout, although there are a few workarounds:
+
+http://comments.gmane.org/gmane.comp.python.distutils.devel/13590
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/README
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/README	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/README	(revision 7478)
@@ -0,0 +1,9 @@
+    Multi-Mechanize
+    ---------------
+    
+    copyright: (c) 2010-2011 Corey Goldberg (corey@goldb.org)
+    
+    license: GNU Lesser General Public License version 3 (LGPLv3)
+    
+    visit: http://multimechanize.com for info and instructions
+    
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/__init__.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/__init__.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/__init__.py	(revision 7478)
@@ -0,0 +1,6 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/graph.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/graph.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/graph.py	(revision 7478)
@@ -0,0 +1,99 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+import sys
+
+try:
+    import matplotlib
+    matplotlib.use('Agg')  # use a non-GUI backend
+    from pylab import *
+except ImportError:
+    print 'ERROR: can not import Matplotlib. install Matplotlib to generate graphs'
+    
+
+
+# response time graph for raw data
+def resp_graph_raw(nested_resp_list, image_name, dir='./'):
+    fig = figure(figsize=(8, 3.3))  # image dimensions  
+    ax = fig.add_subplot(111)
+    ax.set_xlabel('Elapsed Time In Test (secs)', size='x-small')
+    ax.set_ylabel('Response Time (secs)' , size='x-small')
+    ax.grid(True, color='#666666')
+    xticks(size='x-small')
+    yticks(size='x-small')
+    x_seq = [item[0] for item in nested_resp_list] 
+    y_seq = [item[1] for item in nested_resp_list] 
+    ax.plot(x_seq, y_seq, 
+        color='blue', linestyle='-', linewidth=0.0, marker='o', 
+        markeredgecolor='blue', markerfacecolor='blue', markersize=2.0)
+    ax.plot([0.0,], [0.0,], linewidth=0.0, markersize=0.0)
+    savefig(dir + image_name) 
+    
+    
+
+# response time graph for bucketed data
+def resp_graph(avg_resptime_points_dict, percentile_80_resptime_points_dict, percentile_90_resptime_points_dict, image_name, dir='./'):
+    fig = figure(figsize=(8, 3.3))  # image dimensions  
+    ax = fig.add_subplot(111)
+    ax.set_xlabel('Elapsed Time In Test (secs)', size='x-small')
+    ax.set_ylabel('Response Time (secs)' , size='x-small')
+    ax.grid(True, color='#666666')
+    xticks(size='x-small')
+    yticks(size='x-small')
+    
+    x_seq = sorted(avg_resptime_points_dict.keys())
+    y_seq = [avg_resptime_points_dict[x] for x in x_seq]
+    ax.plot(x_seq, y_seq, 
+        color='green', linestyle='-', linewidth=0.75, marker='o', 
+        markeredgecolor='green', markerfacecolor='yellow', markersize=2.0)
+    
+    x_seq = sorted(percentile_80_resptime_points_dict.keys())
+    y_seq = [percentile_80_resptime_points_dict[x] for x in x_seq]
+    ax.plot(x_seq, y_seq, 
+        color='orange', linestyle='-', linewidth=0.75, marker='o', 
+        markeredgecolor='orange', markerfacecolor='yellow', markersize=2.0)
+    
+    x_seq = sorted(percentile_90_resptime_points_dict.keys())
+    y_seq = [percentile_90_resptime_points_dict[x] for x in x_seq]
+    ax.plot(x_seq, y_seq, 
+        color='purple', linestyle='-', linewidth=0.75, marker='o', 
+        markeredgecolor='purple', markerfacecolor='yellow', markersize=2.0)
+        
+    ax.plot([0.0,], [0.0,], linewidth=0.0, markersize=0.0)
+    
+    legend_lines = reversed(ax.get_lines()[:3])
+    ax.legend(
+            legend_lines, 
+            ('90pct', '80pct', 'Avg'), 
+            loc='best',
+            handlelength=1,
+            borderpad=1,                
+            prop=matplotlib.font_manager.FontProperties(size='xx-small')
+            )
+            
+    savefig(dir + image_name) 
+    
+    
+    
+# throughput graph
+def tp_graph(throughputs_dict, image_name, dir='./'):
+    fig = figure(figsize=(8, 3.3))  # image dimensions  
+    ax = fig.add_subplot(111)
+    ax.set_xlabel('Elapsed Time In Test (secs)', size='x-small')
+    ax.set_ylabel('Transactions Per Second (count)' , size='x-small')
+    ax.grid(True, color='#666666')
+    xticks(size='x-small')
+    yticks(size='x-small')
+    x_seq = sorted(throughputs_dict.keys())
+    y_seq = [throughputs_dict[x] for x in x_seq]
+    ax.plot(x_seq, y_seq, 
+        color='red', linestyle='-', linewidth=0.75, marker='o', 
+        markeredgecolor='red', markerfacecolor='yellow', markersize=2.0)
+    ax.plot([0.0,], [0.0,], linewidth=0.0, markersize=0.0)
+    savefig(dir + image_name) 
+    
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/progressbar.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/progressbar.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/progressbar.py	(revision 7478)
@@ -0,0 +1,35 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+
+class ProgressBar(object):
+    def __init__(self, duration):
+        self.duration = duration
+        self.prog_bar = '[]'
+        self.fill_char = '='
+        self.width = 40
+        self.__update_amount(0)
+    
+    def __update_amount(self, new_amount):
+        percent_done = int(round((new_amount / 100.0) * 100.0))
+        if percent_done > 100:
+            percent_done = 100
+        all_full = self.width - 2
+        num_hashes = int(round((percent_done / 100.0) * all_full))
+        self.prog_bar = '[' + self.fill_char * num_hashes + ' ' * (all_full - num_hashes) + ']'
+        pct_place = (len(self.prog_bar) / 2) - len(str(percent_done))
+        pct_string = '%i%%' % percent_done
+        self.prog_bar = self.prog_bar[0:pct_place] + \
+            (pct_string + self.prog_bar[pct_place + len(pct_string):])
+        
+    def update_time(self, elapsed_secs):
+        self.__update_amount((elapsed_secs / float(self.duration)) * 100.0)
+        self.prog_bar += '  %ds/%ss' % (elapsed_secs, self.duration)
+        
+    def __str__(self):
+        return str(self.prog_bar)
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/reportwriter.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/reportwriter.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/reportwriter.py	(revision 7478)
@@ -0,0 +1,106 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+#
+
+
+class Report(object):
+    def __init__(self, results_dir):
+        self.results_dir = results_dir
+        self.fn = results_dir + 'results.html'
+        self.write_head_html()
+
+    
+    def write_line(self, line):
+        with open(self.fn, 'a') as f:
+            f.write('%s\n' % line)
+
+
+    def write_head_html(self):
+        with open(self.fn, 'w') as f:
+            f.write("""\
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head>
+    <title>Multi-Mechanize - Results</title>
+    <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
+    <meta http-equiv="Content-Language" content="en" />
+    <style type="text/css">
+        body {
+            background-color: #FFFFFF;
+            color: #000000;
+            font-family: Verdana, sans-serif;
+            font-size: 11px;
+            padding: 5px;
+        }
+        h1 {
+            font-size: 16px;
+            background: #FF9933;
+            margin-bottom: 0;
+            padding-left: 5px;
+            padding-top: 2px;
+        }
+        h2 {
+            font-size: 13px;
+            background: #C0C0C0;
+            padding-left: 5px;
+            margin-top: 2em;
+            margin-bottom: .75em;
+        }
+        h3 {
+            font-size: 12px;
+            background: #EEEEEE;
+            padding-left: 5px;
+            margin-bottom: 0.5em;
+        }
+        h4 {
+            font-size: 11px;
+            padding-left: 20px;
+            margin-bottom: 0;
+        }
+        p {
+            margin: 0;
+            padding: 0;
+        }
+        table {
+            margin-left: 10px;
+        }
+        td {
+            text-align: right;
+            color: #000000;
+            background: #FFFFFF;
+            padding-left: 10px;
+            padding-right: 10px;
+            padding-bottom: 0;
+        }
+        th {
+            text-align: center;
+            padding-right: 10px;
+            padding-left: 10px;
+            color: #000000;
+            background: #FFFFFF;
+        }
+        div.summary {
+            padding-left: 20px;
+        }
+    </style>
+</head>
+<body>
+""")
+  
+
+    def write_closing_html(self):
+        with open(self.fn, 'a') as f:
+            f.write("""\
+</body>
+</html>
+""")
+
+
+
+
+
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/reportwriterxml.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/reportwriterxml.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/reportwriterxml.py	(revision 7478)
@@ -0,0 +1,59 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010-2011 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+from xml.etree import ElementTree as ET
+
+
+
+def write_jmeter_output(mm_data, output_path):
+    """
+    Take the list of ResponseStats objects and write a JMeter 2.1
+    formatted XML file to output_path.
+
+    JMeter JTL file documentation:
+    http://jakarta.apache.org/jmeter/usermanual/listeners.html
+    """
+    root = ET.Element('testResults')
+    root.set('version', "1.2")
+
+    for test_transaction in mm_data:
+        # each transaction might have multiple timers
+        transaction_root = ET.SubElement(root, 'sample')
+        # JMeter uses ms for time
+        ms_trans_time = test_transaction.trans_time * 1000
+        transaction_root.set('t', '%d' % ms_trans_time)
+        ms_timestamp = test_transaction.epoch_secs * 1000
+        transaction_root.set('ts', '%d' % ms_timestamp)
+        transaction_root.set('lb', test_transaction.user_group_name)  # label
+        transaction_root.set('sc', '1')  # sample count
+
+        if test_transaction.error:
+            transaction_root.set('ec', '1') # was an error
+            transaction_root.set('s', 'false') # was an error
+            # errors don't have custom_timers
+            continue
+        else:
+            transaction_root.set('ec', '0')
+            transaction_root.set('s', 'true')
+
+        # parse the custom_timers and add each as a JMeter sub-sample
+        for timer_name, timer_duration in test_transaction.custom_timers.items():
+            timer_duration = float(timer_duration)
+            timer_element = ET.SubElement(transaction_root, 'sample')
+            ms_trans_time = timer_duration * 1000
+            timer_element.set('t', '%d' % ms_trans_time)
+            # subtimers don't have timestamps, so use the Transaction ts
+            timer_element.set('ts', '%d' % ms_timestamp)
+            timer_element.set('lb', timer_name)
+            timer_element.set('sc', '1')
+            timer_element.set('ec', '0')
+            timer_element.set('s', 'true')
+
+    tree = ET.ElementTree(root)
+    tree.write(output_path + '/results.jtl')
+    tree.write('last_results.jtl')
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/results.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/results.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/results.py	(revision 7478)
@@ -0,0 +1,352 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010-2011 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+import time
+from collections import defaultdict
+import graph
+import reportwriter
+import reportwriterxml
+
+
+
+def output_results(results_dir, results_file, run_time, rampup, ts_interval, user_group_configs=None, xml_reports=False):
+    results = Results(results_dir + results_file, run_time)
+    
+    report = reportwriter.Report(results_dir)
+    
+    print 'transactions: %i' % results.total_transactions
+    print 'errors: %i' % results.total_errors
+    print ''
+    print 'test start: %s' % results.start_datetime
+    print 'test finish: %s' % results.finish_datetime
+    print ''
+    
+    # write the results in XML
+    if xml_reports:
+        reportwriterxml.write_jmeter_output(results.resp_stats_list, results_dir)
+
+    report.write_line('<h1>Performance Results Report</h1>')
+    
+    report.write_line('<h2>Summary</h2>')
+    
+    report.write_line('<div class="summary">')
+    report.write_line('<b>transactions:</b> %d<br />' % results.total_transactions)
+    report.write_line('<b>errors:</b> %d<br />' % results.total_errors)
+    report.write_line('<b>run time:</b> %d secs<br />' % run_time)
+    report.write_line('<b>rampup:</b> %d secs<br /><br />' % rampup)
+    report.write_line('<b>test start:</b> %s<br />' % results.start_datetime)
+    report.write_line('<b>test finish:</b> %s<br /><br />' % results.finish_datetime)
+    report.write_line('<b>time-series interval:</b> %s secs<br /><br /><br />' % ts_interval)
+    if user_group_configs:
+        report.write_line('<b>workload configuration:</b><br /><br />')
+        report.write_line('<table>')
+        report.write_line('<tr><th>group name</th><th>threads</th><th>script name</th></tr>')
+        for user_group_config in user_group_configs:
+            report.write_line('<tr><td>%s</td><td>%d</td><td>%s</td></tr>' % 
+                (user_group_config.name, user_group_config.num_threads, user_group_config.script_file))
+        report.write_line('</table>')
+    report.write_line('</div>')
+    
+    report.write_line('<h2>All Transactions</h2>')
+    
+    # all transactions - response times
+    trans_timer_points = []  # [elapsed, timervalue]
+    trans_timer_vals = []
+    for resp_stats in results.resp_stats_list:
+        t = (resp_stats.elapsed_time, resp_stats.trans_time)
+        trans_timer_points.append(t)
+        trans_timer_vals.append(resp_stats.trans_time)
+    graph.resp_graph_raw(trans_timer_points, 'All_Transactions_response_times.png', results_dir)
+    
+    report.write_line('<h3>Transaction Response Summary (secs)</h3>')
+    report.write_line('<table>')
+    report.write_line('<tr><th>count</th><th>min</th><th>avg</th><th>80pct</th><th>90pct</th><th>95pct</th><th>max</th><th>stdev</th></tr>') 
+    report.write_line('<tr><td>%i</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td></tr>'  % (
+        results.total_transactions, 
+        min(trans_timer_vals), 
+        average(trans_timer_vals), 
+        percentile(trans_timer_vals, 80), 
+        percentile(trans_timer_vals, 90), 
+        percentile(trans_timer_vals, 95), 
+        max(trans_timer_vals),
+        standard_dev(trans_timer_vals), 
+    ))
+    report.write_line('</table>')
+    
+    
+    # all transactions - interval details
+    avg_resptime_points = {}  # {intervalnumber: avg_resptime}
+    percentile_80_resptime_points = {}  # {intervalnumber: 80pct_resptime}
+    percentile_90_resptime_points = {}  # {intervalnumber: 90pct_resptime}
+    interval_secs = ts_interval
+    splat_series = split_series(trans_timer_points, interval_secs)
+    report.write_line('<h3>Interval Details (secs)</h3>')
+    report.write_line('<table>')
+    report.write_line('<tr><th>interval</th><th>count</th><th>rate</th><th>min</th><th>avg</th><th>80pct</th><th>90pct</th><th>95pct</th><th>max</th><th>stdev</th></tr>') 
+    for i, bucket in enumerate(splat_series):
+        interval_start = int((i + 1) * interval_secs)
+        cnt = len(bucket) 
+        
+        if cnt == 0:
+            report.write_line('<tr><td>%i</td><td>0</td><td>0</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td></tr>' % (i + 1))  
+        else:
+            rate = cnt / float(interval_secs)
+            mn = min(bucket)
+            avg = average(bucket)
+            pct_80 = percentile(bucket, 80)
+            pct_90 = percentile(bucket, 90)
+            pct_95 = percentile(bucket, 95)
+            mx = max(bucket)
+            stdev = standard_dev(bucket)
+            report.write_line('<tr><td>%i</td><td>%i</td><td>%.2f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td></tr>' % (i + 1, cnt, rate, mn, avg, pct_80, pct_90, pct_95, mx, stdev))
+
+            avg_resptime_points[interval_start] = avg
+            percentile_80_resptime_points[interval_start] = pct_80
+            percentile_90_resptime_points[interval_start] = pct_90
+
+    report.write_line('</table>') 
+    graph.resp_graph(avg_resptime_points, percentile_80_resptime_points, percentile_90_resptime_points, 'All_Transactions_response_times_intervals.png', results_dir)
+    
+    
+    report.write_line('<h3>Graphs</h3>')
+    report.write_line('<h4>Response Time: %s sec time-series</h4>' % ts_interval)
+    report.write_line('<img src="All_Transactions_response_times_intervals.png"></img>')     
+    report.write_line('<h4>Response Time: raw data (all points)</h4>')
+    report.write_line('<img src="All_Transactions_response_times.png"></img>') 
+    report.write_line('<h4>Throughput: 5 sec time-series</h4>')
+    report.write_line('<img src="All_Transactions_throughput.png"></img>')  
+    
+
+
+    # all transactions - throughput
+    throughput_points = {}  # {intervalnumber: numberofrequests}
+    interval_secs = ts_interval
+    splat_series = split_series(trans_timer_points, interval_secs)
+    for i, bucket in enumerate(splat_series):
+        throughput_points[int((i + 1) * interval_secs)] = (len(bucket) / interval_secs)
+    graph.tp_graph(throughput_points, 'All_Transactions_throughput.png', results_dir)
+    
+        
+        
+    # custom timers
+    for timer_name in sorted(results.uniq_timer_names):
+        custom_timer_vals = []
+        custom_timer_points = []
+        for resp_stats in results.resp_stats_list:
+            try:
+                val = resp_stats.custom_timers[timer_name]
+                custom_timer_points.append((resp_stats.elapsed_time, val)) 
+                custom_timer_vals.append(val)
+            except KeyError:
+                pass
+        graph.resp_graph_raw(custom_timer_points, timer_name + '_response_times.png', results_dir)
+        
+        throughput_points = {}  # {intervalnumber: numberofrequests}
+        interval_secs = ts_interval
+        splat_series = split_series(custom_timer_points, interval_secs)
+        for i, bucket in enumerate(splat_series):
+            throughput_points[int((i + 1) * interval_secs)] = (len(bucket) / interval_secs)
+        graph.tp_graph(throughput_points, timer_name + '_throughput.png', results_dir)
+        
+        report.write_line('<hr />')
+        report.write_line('<h2>Custom Timer: %s</h2>' % timer_name)
+        
+        report.write_line('<h3>Timer Summary (secs)</h3>')
+        
+        report.write_line('<table>')
+        report.write_line('<tr><th>count</th><th>min</th><th>avg</th><th>80pct</th><th>90pct</th><th>95pct</th><th>max</th><th>stdev</th></tr>') 
+        report.write_line('<tr><td>%i</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td></tr>'  % (
+            len(custom_timer_vals), 
+            min(custom_timer_vals), 
+            average(custom_timer_vals), 
+            percentile(custom_timer_vals, 80), 
+            percentile(custom_timer_vals, 90), 
+            percentile(custom_timer_vals, 95), 
+            max(custom_timer_vals),
+            standard_dev(custom_timer_vals)
+        ))
+        report.write_line('</table>')
+    
+        
+        # custom timers - interval details
+        avg_resptime_points = {}  # {intervalnumber: avg_resptime}
+        percentile_80_resptime_points = {}  # {intervalnumber: 80pct_resptime}
+        percentile_90_resptime_points = {}  # {intervalnumber: 90pct_resptime}
+        interval_secs = ts_interval
+        splat_series = split_series(custom_timer_points, interval_secs)
+        report.write_line('<h3>Interval Details (secs)</h3>')
+        report.write_line('<table>')
+        report.write_line('<tr><th>interval</th><th>count</th><th>rate</th><th>min</th><th>avg</th><th>80pct</th><th>90pct</th><th>95pct</th><th>max</th><th>stdev</th></tr>') 
+        for i, bucket in enumerate(splat_series):
+            interval_start = int((i + 1) * interval_secs)
+            cnt = len(bucket) 
+            
+            if cnt == 0:
+                report.write_line('<tr><td>%i</td><td>0</td><td>0</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td><td>N/A</td></tr>' % (i + 1))  
+            else:
+                rate = cnt / float(interval_secs)
+                mn = min(bucket)
+                avg = average(bucket)
+                pct_80 = percentile(bucket, 80)
+                pct_90 = percentile(bucket, 90)
+                pct_95 = percentile(bucket, 95)
+                mx = max(bucket)
+                stdev = standard_dev(bucket)
+
+                report.write_line('<tr><td>%i</td><td>%i</td><td>%.2f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td><td>%.3f</td></tr>' % (i + 1, cnt, rate, mn, avg, pct_80, pct_90, pct_95, mx, stdev))
+                 
+                avg_resptime_points[interval_start] = avg
+                percentile_80_resptime_points[interval_start] = pct_80
+                percentile_90_resptime_points[interval_start] = pct_90
+        report.write_line('</table>') 
+        graph.resp_graph(avg_resptime_points, percentile_80_resptime_points, percentile_90_resptime_points, timer_name + '_response_times_intervals.png', results_dir)
+    
+        
+        report.write_line('<h3>Graphs</h3>')
+        report.write_line('<h4>Response Time: %s sec time-series</h4>' % ts_interval)
+        report.write_line('<img src="%s_response_times_intervals.png"></img>' % timer_name)
+        report.write_line('<h4>Response Time: raw data (all points)</h4>')        
+        report.write_line('<img src="%s_response_times.png"></img>' % timer_name)
+        report.write_line('<h4>Throughput: %s sec time-series</h4>' % ts_interval)
+        report.write_line('<img src="%s_throughput.png"></img>' % timer_name) 
+        
+    
+        
+    ## user group times
+    #for user_group_name in sorted(results.uniq_user_group_names):
+    #    ug_timer_vals = []
+    #    for resp_stats in results.resp_stats_list:
+    #        if resp_stats.user_group_name == user_group_name: 
+    #            ug_timer_vals.append(resp_stats.trans_time)
+    #    print user_group_name
+    #    print 'min: %.3f' % min(ug_timer_vals)
+    #    print 'avg: %.3f' % average(ug_timer_vals)
+    #    print '80pct: %.3f' % percentile(ug_timer_vals, 80)
+    #    print '90pct: %.3f' % percentile(ug_timer_vals, 90)
+    #    print '95pct: %.3f' % percentile(ug_timer_vals, 95)
+    #    print 'max: %.3f' % max(ug_timer_vals)
+    #    print ''        
+    
+    report.write_line('<hr />')
+    report.write_closing_html()
+
+
+    
+    
+class Results(object):
+    def __init__(self, results_file_name, run_time):
+        self.results_file_name = results_file_name
+        self.run_time = run_time
+        self.total_transactions = 0
+        self.total_errors = 0
+        self.uniq_timer_names = set()
+        self.uniq_user_group_names = set()
+        
+        self.resp_stats_list = self.__parse_file()
+        
+        self.epoch_start = self.resp_stats_list[0].epoch_secs
+        self.epoch_finish = self.resp_stats_list[-1].epoch_secs
+        self.start_datetime = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(self.epoch_start))
+        self.finish_datetime = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(self.epoch_finish))
+        
+        
+        
+    def __parse_file(self):
+        f = open(self.results_file_name, 'rb')
+        resp_stats_list = []
+        for line in f:
+            fields = line.strip().split(',')
+            
+            request_num = int(fields[0])
+            elapsed_time = float(fields[1])
+            epoch_secs = int(fields[2])
+            user_group_name = fields[3]
+            trans_time = float(fields[4])
+            error = fields[5]
+            
+            self.uniq_user_group_names.add(user_group_name)
+            
+            custom_timers = {}
+            timers_string = ''.join(fields[6:]).replace('{', '').replace('}', '')
+            splat = timers_string.split("'")[1:]
+            timers = []
+            vals = []
+            for x in splat:
+                if ':' in x:
+                    x = float(x.replace(': ', ''))
+                    vals.append(x)
+                else:
+                    timers.append(x)
+                    self.uniq_timer_names.add(x)
+            for timer, val in zip(timers, vals):
+                custom_timers[timer] = val
+            
+            r = ResponseStats(request_num, elapsed_time, epoch_secs, user_group_name, trans_time, error, custom_timers)
+            
+            if elapsed_time < self.run_time:  # drop all times that appear after the last request was sent (incomplete interval)
+                resp_stats_list.append(r)
+            
+            if error != '':
+                self.total_errors += 1
+                
+            self.total_transactions += 1
+            
+        return resp_stats_list    
+   
+
+
+class ResponseStats(object):
+    def __init__(self, request_num, elapsed_time, epoch_secs, user_group_name, trans_time, error, custom_timers):
+        self.request_num = request_num
+        self.elapsed_time = elapsed_time
+        self.epoch_secs = epoch_secs
+        self.user_group_name = user_group_name
+        self.trans_time = trans_time
+        self.error = error
+        self.custom_timers = custom_timers
+        
+
+
+def split_series(points, interval):
+    offset = points[0][0]
+    maxval = int((points[-1][0] - offset) // interval)
+    vals = defaultdict(list)
+    for key, value in points:
+        vals[(key - offset) // interval].append(value)
+    series = [vals[i] for i in xrange(maxval + 1)]
+    return series
+
+
+
+def average(seq):
+    avg = (float(sum(seq)) / len(seq)) 
+    return avg
+
+
+
+def standard_dev(seq):
+    avg = average(seq)
+    sdsq = sum([(i - avg) ** 2 for i in seq])
+    try:
+        stdev = (sdsq / (len(seq) - 1)) ** .5
+    except ZeroDivisionError:
+        stdev = 0 
+    return stdev
+
+    
+    
+def percentile(seq, percentile):
+    i = int(len(seq) * (percentile / 100.0))
+    seq.sort()
+    return seq[i]
+
+
+
+
+if __name__ == '__main__':
+    output_results('./', 'results.csv', 60, 30, 10)
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/resultsloader.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/resultsloader.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/resultsloader.py	(revision 7478)
@@ -0,0 +1,179 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Brian Knox (taotetek@gmail.com)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+#
+"""a collection of functions and classes for multi-mechanize results files"""
+
+import re
+import fileinput
+from datetime import datetime
+
+try:
+    from sqlalchemy.ext.declarative import declarative_base
+    from sqlalchemy.orm import sessionmaker, relation
+    from sqlalchemy import create_engine
+    from sqlalchemy import Column, Integer, String, Float, DateTime
+    from sqlalchemy import ForeignKey, UniqueConstraint
+except ImportError:
+    print "(optional: please install sqlalchemy to enable db logging)"
+
+
+Base = declarative_base()
+
+class GlobalConfig(Base):
+    """class representing a muli-mechanize global config"""
+    __tablename__ = 'mechanize_global_configs'
+
+    id = Column(Integer, nullable=False, primary_key=True)
+    run_time = Column(Integer, nullable=False)
+    rampup = Column(Integer, nullable=False)
+    results_ts_interval = Column(Integer, nullable=False)
+    user_group_configs = relation("UserGroupConfig",
+        primaryjoin="UserGroupConfig.mechanize_global_configs_id==GlobalConfig.id")
+    results = relation("ResultRow",
+        primaryjoin="GlobalConfig.id==ResultRow.mechanize_global_configs_id")
+
+    def __init__(self, run_time=None, rampup=None, results_ts_interval=None):
+        self.run_time = str(run_time)
+        self.rampup = int(rampup)
+        """rampup time for the rest run"""
+        self.results_ts_interval = int(results_ts_interval)
+
+    def __repr__(self):
+        return "<GlobalConfig('%i', '%i', '%i')>" % (
+                self.run_time, self.rampup, self.results_ts_interval)
+
+class UserGroupConfig(Base):
+    """class representing a multi-mechanize user group config"""
+    __tablename__ = 'mechanize_user_group_configs'
+
+    id = Column (Integer, nullable=False, primary_key=True)
+    mechanize_global_configs_id = Column(Integer, ForeignKey('mechanize_global_configs.id'), nullable=False)
+    user_group = Column(String(50), nullable=False)
+    threads = Column(Integer, nullable=False)
+    script = Column(String(50), nullable=False)
+
+    def __init__(self, user_group=None, threads=None, script=None):
+        self.user_group = str(user_group)
+        self.threads = int(threads)
+        self.script = str(script)
+
+    def __repr__(self):
+        return "<UserGroupConfig('%s','%s','%s')>" % (
+                self.user_group, self.threads, self.script)
+
+class ResultRow(Base):
+    """class representing a multi-mechanize results.csv row"""
+    __tablename__ = 'mechanize_results'
+    __table_args__ = (
+        UniqueConstraint('run_id','trans_count', name='uix_1')
+        )
+
+    id = Column(Integer, nullable=False, primary_key=True)
+    mechanize_global_configs_id = Column(Integer, 
+        ForeignKey('mechanize_global_configs.id'), nullable=False)
+    project_name = Column(String(50), nullable=False, index=True)
+    run_id = Column(DateTime, nullable=False, index=True)
+    trans_count = Column(Integer, nullable=False, index=True)
+    elapsed = Column(Float, nullable=False, index=True)
+    epoch = Column(Float, nullable=False, index=True)
+    user_group_name = Column(String(50), nullable=False)
+    scriptrun_time = Column(Float, nullable=False)
+    error = Column(String(255))
+    custom_timers = Column(String(50))
+
+    global_config = relation("GlobalConfig",
+            primaryjoin="ResultRow.mechanize_global_configs_id==GlobalConfig.id")
+
+    timers = relation("TimerRow",
+        primaryjoin="ResultRow.id==TimerRow.mechanize_results_id")
+
+    def __init__(self, project_name=None, run_id=None, trans_count=None, 
+            elapsed=None, epoch=None, user_group_name=None,
+            scriptrun_time=None, error=None, custom_timers=None):
+        self.project_name = str(project_name)
+        self.run_id = run_id
+        self.trans_count = int(trans_count)
+        self.elapsed = float(elapsed)
+        self.epoch = int(epoch)
+        self.user_group_name = str(user_group_name)
+        self.scriptrun_time = float(scriptrun_time)
+        self.error = str(error)
+        self.custom_timers = str(custom_timers)
+
+    def __repr__(self):
+        return "<ResultRow('%s','%s','%i','%.3f','%i','%s','%.3f','%s','%s')>" % (
+                self.project_name, self.run_id, self.trans_count, self.elapsed, 
+                self.epoch, self.user_group_name, self.scriptrun_time, 
+                self.error, self.custom_timers)
+ 
+class TimerRow(Base):
+    """class representing a multi-mechanize custom timer result"""
+    __tablename__ = 'mechanize_custom_timers'
+    id = Column(Integer, nullable=False, primary_key=True)
+    mechanize_results_id = Column(Integer, 
+         ForeignKey('mechanize_results.id'), nullable=False)
+    timer_name = Column(String(50), nullable=False, index=True)
+    elapsed = Column(Float, nullable=False, index=True)
+
+    def __init__(self, timer_name=None, elapsed=None):
+        self.timer_name = str(timer_name)
+        self.elapsed = int(elapsed)
+
+    def __repr__(self):
+        return "<TimerRow('%s', '%s')>" % (self.timer_name, self.elapsed)
+
+    result_rows = relation("ResultRow",
+        primaryjoin="TimerRow.mechanize_results_id==ResultRow.id")
+
+def load_results_database(project_name, run_localtime, results_dir, 
+        results_database, run_time, rampup, results_ts_interval,
+        user_group_configs):
+    """parse and load a multi-mechanize results csv file into a database"""
+
+    logline_re = re.compile('(.+),(.+),(.+),(.+),(.+),(.?),(\{.+\})')
+    
+    engine = create_engine(results_database, echo=False)
+    ResultRow.metadata.create_all(engine)
+    TimerRow.metadata.create_all(engine)
+    GlobalConfig.metadata.create_all(engine)
+    UserGroupConfig.metadata.create_all(engine)
+
+    sa_session = sessionmaker(bind=engine)
+    sa_current_session = sa_session()
+    
+    run_id = datetime(run_localtime.tm_year, run_localtime.tm_mon,
+        run_localtime.tm_mday, run_localtime.tm_hour, run_localtime.tm_min,
+        run_localtime.tm_sec)
+
+    results_file = results_dir + 'results.csv'
+   
+    global_config = GlobalConfig(run_time, rampup, results_ts_interval)
+    sa_current_session.add(global_config)
+    
+    for i, ug_config in enumerate(user_group_configs):
+        user_group_config = UserGroupConfig(ug_config.name, 
+                ug_config.num_threads, ug_config.script_file)
+        global_config.user_group_configs.append(user_group_config)
+
+    for line in fileinput.input([results_file]):
+        line = line.rstrip()
+        match = logline_re.match(line)
+        if match:
+            result_row = ResultRow(project_name, run_id, match.group(1), 
+                    match.group(2), match.group(3), match.group(4),
+                    match.group(5), match.group(6), match.group(7))
+    
+            global_config.results.append(result_row)
+            timer_data = eval(match.group(7))
+            for index in timer_data:
+                timer_row = TimerRow(index, timer_data[index])
+                result_row.timers.append(timer_row)
+
+            sa_current_session.add(result_row)
+    
+    sa_current_session.commit()
+    sa_current_session.close()
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/rpcserver.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/rpcserver.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/rpcserver.py	(revision 7478)
@@ -0,0 +1,63 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+import SimpleXMLRPCServer
+import socket
+import thread
+    
+    
+    
+def launch_rpc_server(port, project_name, run_callback):  
+    host = socket.gethostbyaddr(socket.gethostname())[0]
+    server = SimpleXMLRPCServer.SimpleXMLRPCServer((host, port), logRequests=False)
+    server.register_instance(RemoteControl(project_name, run_callback))
+    server.register_introspection_functions()
+    print '\nMulti-Mechanize: %s listening on port %i' % (host, port)
+    print 'waiting for xml-rpc commands...\n'
+    try:
+        server.serve_forever()
+    except KeyboardInterrupt:
+        pass
+        
+
+
+class RemoteControl(object):
+    def __init__(self, project_name, run_callback):
+        self.project_name = project_name
+        self.run_callback = run_callback
+        self.test_running = False
+        self.output_dir = None
+    
+    def run_test(self):
+        if self.test_running:
+            return 'Test Already Running'
+        else:
+            thread.start_new_thread(self.run_callback, (self,))
+            return 'Test Started'    
+    
+    def check_test_running(self):
+        return self.test_running
+    
+    def update_config(self, config):
+        with open('projects/%s/config.cfg' % self.project_name, 'w') as f:
+            f.write(config)
+            return True
+ 
+    def get_config(self):
+        with open('projects/%s/config.cfg' % self.project_name, 'r') as f:
+            return f.read()
+            
+    def get_project_name(self):
+        return self.project_name
+
+    def get_results(self):
+        if self.output_dir is None:
+            return 'Results Not Available'
+        else:
+            with open(self.output_dir + 'results.csv', 'r') as f:
+                return f.read()
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/tools/dependency_checker.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/tools/dependency_checker.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/tools/dependency_checker.py	(revision 7478)
@@ -0,0 +1,42 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+""" script to verify all multi-mechanize dependencies are satisfied """
+
+
+import sys
+
+
+if sys.version_info >= (3,):
+    print 'sorry, no py3k support yet' 
+elif sys.version_info < (2, 6, 3):
+    print 'incompatible python version detected: %s.  Minimum version supported is 2.6' % repr(sys.version_info)
+else:
+    print 'compatible python version detected: %s' % repr(sys.version_info)
+
+
+try:
+    import mechanize
+    print 'imported Mechanize succesfully'
+except ImportError:
+    print 'can not import Mechanize'
+
+
+try:
+    import pylab
+    print 'imported Matplotlib succesfully'
+except ImportError:
+    print 'can not import Matplotlib'
+
+
+try:
+    import sqlalchemy
+    print 'imported SQLAlchemy succesfully'
+except ImportError:
+    print 'can not import SQLAlchemy'
+    
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/tools/grid_gui.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/tools/grid_gui.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/lib/tools/grid_gui.py	(revision 7478)
@@ -0,0 +1,136 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+""" 
+Multi-Mechanize Grid Controller
+sample gui application for controlling multi-mechanize instances via the remote management api
+"""
+
+
+import socket
+import ScrolledText
+import Tkinter
+import tkFileDialog
+import xmlrpclib
+
+
+
+# list of hosts:ports where multi-mechanize is listening
+NODES = [
+    '192.168.1.2:9001',
+    '192.168.1.3:9001',
+]
+
+
+
+class Application:
+    def __init__(self, root, hosts):
+        self.hosts = hosts
+        self.root = root
+        self.root.geometry('%dx%d%+d%+d' % (600, 400, 100, 100))
+        self.root.title('Multi-Mechanize Grid Controller')
+        
+        Tkinter.Button(self.root, text='List Nodes', command=self.list_nodes, width=15,).place(x=5, y=5)
+        Tkinter.Button(self.root, text='Check Tests', command=self.check_servers, width=15,).place(x=5, y=35)
+        Tkinter.Button(self.root, text='Get Project Names', command=self.get_project_names, width=15).place(x=5, y=65)
+        Tkinter.Button(self.root, text='Get Configs', command=self.get_configs, width=15).place(x=5, y=95)
+        Tkinter.Button(self.root, text='Update Configs', command=self.update_configs, width=15).place(x=5, y=125)
+        Tkinter.Button(self.root, text='Get Results', command=self.get_results, width=15).place(x=5, y=155)
+        Tkinter.Button(self.root, text='Run Tests', command=self.run_tests, width=15).place(x=5, y=185)
+        
+        self.text_box = ScrolledText.ScrolledText(self.root, width=59, height=24, font=('Helvetica', 9))
+        self.text_box.place(x=162, y=5)
+        
+        
+    def clear_window(self):
+        self.text_box.delete(1.0, Tkinter.END)
+            
+            
+    def list_nodes(self):
+        self.clear_window()
+        for host, port in self.hosts:
+            self.text_box.insert(Tkinter.END, '%s:%s\n' % (host, port))
+                
+                
+    def run_tests(self):
+        self.clear_window()
+        for host, port in self.hosts:
+            server = xmlrpclib.ServerProxy('http://%s:%s' % (host, port))
+            try:
+                status = server.run_test()
+                self.text_box.insert(Tkinter.END, '%s:%s:\n%s\n\n\n' % (host, port, status))
+            except socket.error:
+                self.text_box.insert(Tkinter.END, 'can not make connection to: %s:%s\n' % (host, port))
+            
+    
+    def get_configs(self):
+        self.clear_window()
+        for host, port in self.hosts:
+            server = xmlrpclib.ServerProxy('http://%s:%s' % (host, port))
+            try:
+                config = server.get_config()
+                self.text_box.insert(Tkinter.END, '%s:%s config:\n%s\n\n\n' % (host, port, config))
+            except socket.error:
+                self.text_box.insert(Tkinter.END, 'can not make connection to: %s:%s\n' % (host, port))
+
+    
+    def update_configs(self):
+        self.clear_window()
+        f = tkFileDialog.askopenfile(parent=self.root, initialdir='./', title='Select a Config File')
+        for host, port in self.hosts:
+            server = xmlrpclib.ServerProxy('http://%s:%s' % (host, port))
+            try:
+                status = server.update_config(f.read())
+                self.text_box.insert(Tkinter.END, '%s:%s config updated:\n%s\n\n' % (host, port, status))
+            except socket.error:
+                self.text_box.insert(Tkinter.END, 'can not make connection to: %s:%s\n' % (host, port))
+                
+                
+    def get_results(self):
+        self.clear_window()
+        for host, port in self.hosts:
+            server = xmlrpclib.ServerProxy('http://%s:%s' % (host, port))
+            try:
+                results = server.get_results()
+                self.text_box.insert(Tkinter.END, '%s:%s results:\n%s\n\n\n' % (host, port, results))
+            except socket.error:
+                self.text_box.insert(Tkinter.END, 'can not make connection to: %s:%s\n' % (host, port))
+            
+            
+    def get_project_names(self):
+        self.clear_window()
+        for host, port in self.hosts:
+            server = xmlrpclib.ServerProxy('http://%s:%s' % (host, port))
+            try:
+                name = server.get_project_name()
+                self.text_box.insert(Tkinter.END, '%s:%s project name:\n%s\n\n' % (host, port, name))
+            except socket.error:
+                self.text_box.insert(Tkinter.END, 'can not make connection to: %s:%s\n' % (host, port))
+            
+            
+    def check_servers(self):
+        self.clear_window()
+        for host, port in self.hosts:
+            server = xmlrpclib.ServerProxy('http://%s:%s' % (host, port))
+            try:
+                is_running = server.check_test_running()
+                self.text_box.insert(Tkinter.END, '%s:%s test running:\n%s\n\n' % (host, port, is_running))
+            except socket.error:
+                self.text_box.insert(Tkinter.END, 'can not make connection to: %s:%s\n' % (host, port))
+        
+        
+        
+def main():
+    hosts = [(host_port.split(':')[0], host_port.split(':')[1]) for host_port in NODES]
+    root = Tkinter.Tk()
+    app = Application(root, hosts)
+    root.mainloop()
+
+
+if __name__ == '__main__':
+    main()
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/multi-mechanize.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/multi-mechanize.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/multi-mechanize.py	(revision 7478)
@@ -0,0 +1,347 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010-2011 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3 - distributed under the terms of the GNU Lesser General Public License version 3
+#  
+#  This file is part of Multi-Mechanize:
+#       Multi-Process, Multi-Threaded, Web Load Generator, with python-mechanize agents
+#
+#  requires Python 2.6+
+
+
+
+import ConfigParser
+import glob
+import multiprocessing
+import optparse
+import os
+import Queue
+import shutil
+import subprocess
+import sys
+import threading
+import time
+import lib.results as results
+import lib.progressbar as progressbar        
+
+usage = 'Usage: %prog <project name> [options]'
+parser = optparse.OptionParser(usage=usage)
+parser.add_option('-p', '--port', dest='port', type='int', help='rpc listener port')
+parser.add_option('-r', '--results', dest='results_dir', help='results directory to reprocess')
+cmd_opts, args = parser.parse_args()
+
+try:
+    project_name = args[0]
+except IndexError:
+    sys.stderr.write('\nERROR: no project specified\n\n')
+    sys.stderr.write('usage: python multi-mechanize.py <project_name>\n')
+    sys.stderr.write('example: python multi-mechanize.py default_project\n\n')
+    sys.exit(1)  
+
+scripts_path = 'projects/%s/test_scripts' % project_name
+if not os.path.exists(scripts_path):
+    sys.stderr.write('\nERROR: can not find project: %s\n\n' % project_name)
+    sys.exit(1) 
+sys.path.append(scripts_path)          
+for f in glob.glob( '%s/*.py' % scripts_path):  # import all test scripts as modules
+    f = f.replace(scripts_path, '').replace(os.sep, '').replace('.py', '')
+    exec('import %s' % f)
+
+
+
+def main():
+    if cmd_opts.results_dir:  # don't run a test, just reprocess results
+        rerun_results(cmd_opts.results_dir)
+    elif cmd_opts.port:
+        import lib.rpcserver
+        lib.rpcserver.launch_rpc_server(cmd_opts.port, project_name, run_test)
+    else:  
+        run_test()
+    return
+        
+        
+    
+def run_test(remote_starter=None):
+    if remote_starter is not None:
+        remote_starter.test_running = True
+        remote_starter.output_dir = None
+        
+    run_time, rampup, results_ts_interval, console_logging, progress_bar, results_database, post_run_script, xml_report, user_group_configs = configure(project_name)
+    
+    run_localtime = time.localtime() 
+    output_dir = time.strftime('projects/' + project_name + '/results/results_%Y.%m.%d_%H.%M.%S/', run_localtime) 
+        
+    # this queue is shared between all processes/threads
+    queue = multiprocessing.Queue()
+    rw = ResultsWriter(queue, output_dir, console_logging)
+    rw.daemon = True
+    rw.start()
+    
+    user_groups = [] 
+    for i, ug_config in enumerate(user_group_configs):
+        ug = UserGroup(queue, i, ug_config.name, ug_config.num_threads, ug_config.script_file, run_time, rampup)
+        user_groups.append(ug)    
+    for user_group in user_groups:
+        user_group.start()
+        
+    start_time = time.time() 
+    
+    if console_logging:
+        for user_group in user_groups:
+            user_group.join()
+    else:
+        print '\n  user_groups:  %i' % len(user_groups)
+        print '  threads: %i\n' % (ug_config.num_threads * len(user_groups))
+        
+        if progress_bar:
+            p = progressbar.ProgressBar(run_time)
+            elapsed = 0
+            while elapsed < (run_time + 1):
+                p.update_time(elapsed)
+                if sys.platform.startswith('win'):
+                    print '%s   transactions: %i  timers: %i  errors: %i\r' % (p, rw.trans_count, rw.timer_count, rw.error_count),
+                else:
+                    print '%s   transactions: %i  timers: %i  errors: %i' % (p, rw.trans_count, rw.timer_count, rw.error_count)
+                    sys.stdout.write(chr(27) + '[A' )
+                time.sleep(1)
+                elapsed = time.time() - start_time
+            
+            print p
+            
+        while [user_group for user_group in user_groups if user_group.is_alive()] != []:
+            if progress_bar:
+                if sys.platform.startswith('win'):
+                    print 'waiting for all requests to finish...\r',
+                else:
+                    print 'waiting for all requests to finish...\r'
+                    sys.stdout.write(chr(27) + '[A' )
+            time.sleep(.5)
+            
+        if not sys.platform.startswith('win'):
+            print
+
+    # all agents are done running at this point
+    time.sleep(.2) # make sure the writer queue is flushed
+    print '\n\nanalyzing results...\n'
+    results.output_results(output_dir, 'results.csv', run_time, rampup, results_ts_interval, user_group_configs, xml_report)
+    print 'created: %sresults.html\n' % output_dir
+    if xml_report:
+        print 'created: %sresults.jtl' % output_dir
+        print 'created: last_results.jtl\n'
+    
+    # copy config file to results directory
+    project_config = os.sep.join(['projects', project_name, 'config.cfg'])
+    saved_config = os.sep.join([output_dir, 'config.cfg'])
+    shutil.copy(project_config, saved_config)
+    
+    if results_database is not None:
+        print 'loading results into database: %s\n' % results_database
+        import lib.resultsloader
+        lib.resultsloader.load_results_database(project_name, run_localtime, output_dir, results_database, 
+                run_time, rampup, results_ts_interval, user_group_configs)
+    
+    if post_run_script is not None:
+        print 'running post_run_script: %s\n' % post_run_script
+        subprocess.call(post_run_script)
+        
+    print 'done.\n'
+    
+    if remote_starter is not None:
+        remote_starter.test_running = False
+        remote_starter.output_dir = output_dir
+    
+    return
+    
+    
+    
+def rerun_results(results_dir):
+    output_dir = 'projects/%s/results/%s/' % (project_name, results_dir)
+    saved_config = '%s/config.cfg' % output_dir
+    run_time, rampup, results_ts_interval, console_logging, progress_bar, results_database, post_run_script, xml_report, user_group_configs = configure(project_name, config_file=saved_config)
+    print '\n\nanalyzing results...\n'
+    results.output_results(output_dir, 'results.csv', run_time, rampup, results_ts_interval, user_group_configs, xml_report)
+    print 'created: %sresults.html\n' % output_dir
+    if xml_report:
+        print 'created: %sresults.jtl' % output_dir
+        print 'created: last_results.jtl\n'
+    
+
+
+def configure(project_name, config_file=None):
+    user_group_configs = []
+    config = ConfigParser.ConfigParser()
+    if config_file is None:
+        config_file = 'projects/%s/config.cfg' % project_name
+    config.read(config_file)
+    for section in config.sections():
+        if section == 'global':
+            run_time = config.getint(section, 'run_time')
+            rampup = config.getint(section, 'rampup')
+            results_ts_interval = config.getint(section, 'results_ts_interval')
+            try: 
+                console_logging = config.getboolean(section, 'console_logging')
+            except ConfigParser.NoOptionError:
+                console_logging = False
+            try: 
+                progress_bar = config.getboolean(section, 'progress_bar')
+            except ConfigParser.NoOptionError:
+                progress_bar = True
+            try:
+                results_database = config.get(section, 'results_database')
+                if results_database == 'None': results_database = None
+            except ConfigParser.NoOptionError:
+                results_database = None
+            try:
+                post_run_script = config.get(section, 'post_run_script')
+                if post_run_script == 'None': post_run_script = None
+            except ConfigParser.NoOptionError:
+                post_run_script = None
+            try:
+                xml_report = config.getboolean(section, 'xml_report')
+            except ConfigParser.NoOptionError:
+                xml_report = False
+        else:
+            threads = config.getint(section, 'threads')
+            script = config.get(section, 'script')
+            user_group_name = section
+            ug_config = UserGroupConfig(threads, user_group_name, script)
+            user_group_configs.append(ug_config)
+
+    return (run_time, rampup, results_ts_interval, console_logging, progress_bar, results_database, post_run_script, xml_report, user_group_configs)
+    
+
+
+class UserGroupConfig(object):
+    def __init__(self, num_threads, name, script_file):
+        self.num_threads = num_threads
+        self.name = name
+        self.script_file = script_file
+    
+    
+    
+class UserGroup(multiprocessing.Process):
+    def __init__(self, queue, process_num, user_group_name, num_threads, script_file, run_time, rampup):
+        multiprocessing.Process.__init__(self)
+        self.queue = queue
+        self.process_num = process_num
+        self.user_group_name = user_group_name
+        self.num_threads = num_threads
+        self.script_file = script_file
+        self.run_time = run_time
+        self.rampup = rampup
+        self.start_time = time.time()
+        
+    def run(self):
+        threads = []
+        for i in range(self.num_threads):
+            spacing = float(self.rampup) / float(self.num_threads)
+            if i > 0:
+                time.sleep(spacing)
+            agent_thread = Agent(self.queue, self.process_num, i, self.start_time, self.run_time, self.user_group_name, self.script_file)
+            agent_thread.daemon = True
+            threads.append(agent_thread)
+            agent_thread.start()            
+        for agent_thread in threads:
+            agent_thread.join()
+        
+
+
+class Agent(threading.Thread):
+    def __init__(self, queue, process_num, thread_num, start_time, run_time, user_group_name, script_file):
+        threading.Thread.__init__(self)
+        self.queue = queue
+        self.process_num = process_num
+        self.thread_num = thread_num
+        self.start_time = start_time
+        self.run_time = run_time
+        self.user_group_name = user_group_name
+        self.script_file = script_file
+        
+        # choose most accurate timer to use (time.clock has finer granularity than time.time on windows, but shouldn't be used on other systems)
+        if sys.platform.startswith('win'):
+            self.default_timer = time.clock
+        else:
+            self.default_timer = time.time
+    
+    
+    def run(self):
+        elapsed = 0
+        
+        if self.script_file.lower().endswith('.py'):
+            module_name = self.script_file.replace('.py', '')
+        else:
+            sys.stderr.write('ERROR: scripts must have .py extension. can not run test script: %s.  aborting user group: %s\n' % (self.script_file, self.user_group_name))
+            return
+        try:
+            trans = eval(module_name + '.Transaction()')
+        except NameError, e:
+            sys.stderr.write('ERROR: can not find test script: %s.  aborting user group: %s\n' % (self.script_file, self.user_group_name))
+            return
+        except Exception, e:
+            sys.stderr.write('ERROR: failed initializing Transaction: %s.  aborting user group: %s\n' % (self.script_file, self.user_group_name))
+            return
+        
+        trans.custom_timers = {}
+        
+        # scripts have access to these vars, which can be useful for loading unique data
+        trans.thread_num = self.thread_num
+        trans.process_num = self.process_num
+            
+        while elapsed < self.run_time:
+            error = ''
+            start = self.default_timer()  
+            
+            try:
+                trans.run()
+            except Exception, e:  # test runner catches all script exceptions here
+                error = str(e).replace(',', '')
+
+            finish = self.default_timer()
+            
+            scriptrun_time = finish - start
+            elapsed = time.time() - self.start_time 
+
+            epoch = time.mktime(time.localtime())
+            
+            fields = (elapsed, epoch, self.user_group_name, scriptrun_time, error, trans.custom_timers)
+            self.queue.put(fields)
+            
+
+
+class ResultsWriter(threading.Thread):
+    def __init__(self, queue, output_dir, console_logging):
+        threading.Thread.__init__(self)
+        self.queue = queue
+        self.console_logging = console_logging
+        self.output_dir = output_dir
+        self.trans_count = 0
+        self.timer_count = 0
+        self.error_count = 0
+        
+        try:
+            os.makedirs(self.output_dir, 0755)
+        except OSError:
+            sys.stderr.write('ERROR: Can not create output directory\n')
+            sys.exit(1)    
+    
+    def run(self):
+        with open(self.output_dir + 'results.csv', 'w') as f:     
+            while True:
+                try:
+                    elapsed, epoch, self.user_group_name, scriptrun_time, error, custom_timers = self.queue.get(False)
+                    self.trans_count += 1
+                    self.timer_count += len(custom_timers)
+                    if error != '':
+                        self.error_count += 1
+                    f.write('%i,%.3f,%i,%s,%f,%s,%s\n' % (self.trans_count, elapsed, epoch, self.user_group_name, scriptrun_time, error, repr(custom_timers)))
+                    f.flush()
+                    if self.console_logging:
+                        print '%i, %.3f, %i, %s, %.3f, %s, %s' % (self.trans_count, elapsed, epoch, self.user_group_name, scriptrun_time, error, repr(custom_timers))
+                except Queue.Empty:
+                    time.sleep(.05)
+
+
+
+if __name__ == '__main__':
+    main()
+
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/config.cfg
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/config.cfg	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/config.cfg	(revision 7478)
@@ -0,0 +1,17 @@
+[global]
+run_time: 30
+rampup: 0
+results_ts_interval: 10
+
+progress_bar: on
+console_logging: off
+xml_report: off
+
+
+[user_group-1]
+threads: 3
+script: example_mock.py
+
+[user_group-2]
+threads: 3
+script: example_mock.py
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_httplib.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_httplib.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_httplib.py	(revision 7478)
@@ -0,0 +1,35 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+import httplib
+import time
+
+
+
+class Transaction(object):
+    def __init__(self):
+        self.custom_timers = {}
+    
+    def run(self):
+        start_timer = time.time()
+        conn = httplib.HTTPConnection('www.example.com')
+        conn.request('GET', '/')
+        resp = conn.getresponse()
+        content = resp.read()
+        latency = time.time() - start_timer
+        
+        self.custom_timers['Example_Homepage'] = latency
+        
+        assert (resp.status == 200), 'Bad HTTP Response'
+        assert ('Example Web Page' in content), 'Failed Content Verification'
+
+
+if __name__ == '__main__':
+    trans = Transaction()
+    trans.run()
+    print trans.custom_timers
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_mechanize_simple.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_mechanize_simple.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_mechanize_simple.py	(revision 7478)
@@ -0,0 +1,36 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+import mechanize
+import time
+
+
+
+class Transaction(object):
+    def __init__(self):
+        self.custom_timers = {}
+    
+    def run(self):
+        br = mechanize.Browser()
+        br.set_handle_robots(False)
+        
+        start_timer = time.time()
+        resp = br.open('http://www.example.com/')
+        resp.read()
+        latency = time.time() - start_timer
+        
+        self.custom_timers['Example_Homepage'] = latency
+        
+        assert (resp.code == 200), 'Bad HTTP Response'
+        assert ('Example Web Page' in resp.get_data()), 'Failed Content Verification'
+
+
+if __name__ == '__main__':
+    trans = Transaction()
+    trans.run()
+    print trans.custom_timers
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_mechanize_wikipedia.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_mechanize_wikipedia.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_mechanize_wikipedia.py	(revision 7478)
@@ -0,0 +1,72 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+import mechanize
+import time
+
+
+
+class Transaction(object):
+    def __init__(self):
+        self.custom_timers = {}
+    
+    def run(self):
+        # create a Browser instance
+        br = mechanize.Browser()
+        # don't bother with robots.txt
+        br.set_handle_robots(False)
+        # add a custom header so wikipedia allows our requests
+        br.addheaders = [('User-agent', 'Mozilla/5.0 Compatible')]
+        
+        # start the timer
+        start_timer = time.time()
+        # submit the request
+        resp = br.open('http://www.wikipedia.org/')
+        resp.read()
+        # stop the timer
+        latency = time.time() - start_timer
+        
+        # store the custom timer
+        self.custom_timers['Load_Front_Page'] = latency   
+        
+        # verify responses are valid
+        assert (resp.code == 200), 'Bad HTTP Response'
+        assert ('Wikipedia, the free encyclopedia' in resp.get_data()), 'Text Assertion Failed'
+        
+        # think-time
+        time.sleep(2)  
+        
+        # select first (zero-based) form on page
+        br.select_form(nr=0)
+        # set form field        
+        br.form['search'] = 'foo'  
+        
+        # start the timer
+        start_timer = time.time()
+        # submit the form
+        resp = br.submit()  
+        resp.read()
+        # stop the timer
+        latency = time.time() - start_timer
+        
+        # store the custom timer
+        self.custom_timers['Search'] = latency  
+        
+        # verify responses are valid
+        assert (resp.code == 200), 'Bad HTTP Response'
+        assert ('foobar' in resp.get_data()), 'Text Assertion Failed'
+        
+        # think-time
+        time.sleep(2)  
+
+
+
+if __name__ == '__main__':
+    trans = Transaction()
+    trans.run()
+    print trans.custom_timers
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_mock.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_mock.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_mock.py	(revision 7478)
@@ -0,0 +1,31 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+#
+#
+#  this is a mock plugin.  
+#  it does nothing but return random custom_timer data.
+
+
+import random
+import time
+
+
+
+class Transaction(object):
+    def __init__(self):
+        self.custom_timers = {}
+    
+    def run(self):
+        r = random.uniform(1, 2)
+        time.sleep(r)
+        self.custom_timers['Example_Timer'] = r
+
+ 
+if __name__ == '__main__':
+    trans = Transaction()
+    trans.run()
+    print trans.custom_timers
Index: /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_urllib2.py
===================================================================
--- /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_urllib2.py	(revision 7478)
+++ /main/waeup.sirp/branches/ulif-stress-multimech/multi-mechanize/projects/default_project/test_scripts/example_urllib2.py	(revision 7478)
@@ -0,0 +1,33 @@
+#!/usr/bin/env python
+#
+#  Copyright (c) 2010 Corey Goldberg (corey@goldb.org)
+#  License: GNU LGPLv3
+#  
+#  This file is part of Multi-Mechanize
+
+
+import urllib2
+import time
+
+
+
+class Transaction(object):
+    def __init__(self):
+        self.custom_timers = {}
+    
+    def run(self):
+        start_timer = time.time()
+        resp = urllib2.urlopen('http://www.example.com/')
+        content = resp.read()
+        latency = time.time() - start_timer
+        
+        self.custom_timers['Example_Homepage'] = latency
+        
+        assert (resp.code == 200), 'Bad HTTP Response'
+        assert ('Example Web Page' in content), 'Failed Content Verification'
+
+
+if __name__ == '__main__':
+    trans = Transaction()
+    trans.run()
+    print trans.custom_timers
