Monitoring Web Applications/Services With MRTG and WebInject

More tools by Corey Goldberg: www.goldb.org
Subscribe To Corey's Blog

MRTG

MRTG (Multi Router Traffic Grapher) is an open source tool developed by Tobias Oetiker. It was originally designed to monitor traffic load on network links, but has the ability to use External Monitoring Scripts to monitor almost anything. In this case, it functions as a data collector, storage repository, and graph generator for use with WebInject. MRTG generates HTML pages containing PNG images which are regularly updated to provide a visual representation of the stats (response times) we are monitoring. This provides graphical trending of time-series data showing Daily, Weekly, Monthly, and Yearly statistics.

For information on MRTG, visit http://www.mrtg.org

Using WebInject as an MRTG External Monitoring Script (Plugin)

WebInject has its own data collection and graphing capabilities. However, they are designed for ad hoc monitoring and are not very useful for graphical trending of web service-levels over a long period of time.

Alternately, WebInject can run in a mode that allows it to be used as an MRTG External Monitoring Script (plugin). This enables us to leverage MRTG's data collection, storage repository, and graph generating capabilities for use with WebInject as a solution to monitor and store web service-level history and trends.

WebInject Configuration

First you must install WebInject and the necessary Perl modules. See the WebInject Build page for help getting it setup.

Then you have to modify the config.xml and add this line in it to enable the MRTG plugin mode:

<reporttype>mrtg</reporttype>

Note: you may also create an alternate config file that you can then specify on the command line used to call webinject.pl

This will modify the output of WebInject to be compatible with MRTG.

Here is an example config file (we will refer to it as myconfig.xml in later examples) which is useable with MRTG:

<testcasefile>MyApplication.xml</testcasefile>
<reporttype>mrtg</reporttype>

(see the WebInject manual for a full list of config settings that are available)

Now you would setup a test case file for WebInject to run. You may use complex test case files consisting of multiple tests and verifications chained together, or you may use a simple test case to act as a probe. Here is a minimal MyApplication.xml test case file for a sample web application:

<testcases>

<case
    id="1"
    description1="sample login probe for WebInject Test Site"
    url="http://www.webinject.org/testsite/processlogin.cgi"
    method="post"
    postbody="username=foo&passwd=welcome&SUBMIT=Login"
    verifypositive="successful"
    verifynegative="fail"
/>

</testcases>

MRTG Configuration

These instructions assume you already have MRTG setup correctly. I use it on a GNU/Linux system, and it was trivial to install.

For a listing of all the configuration options available, see the MRTG Configuration Reference.

Each monitoring target must be identified by a unique name. This name must be appended to each parameter belonging to the same target. The name will also be used for naming the generated webpages, logfiles, and images for this target. In our example, our target is named 'myapplication'.

For setting up MRTG with WebInject, the MRTG configuration settings I use are:

WorkDir: Specifies where the webpages, logfiles, and images should be created. I use a directory that is available to my webserver so the output can be viewed directly from a browser. MRTG uses constant size logfiles. MRTG's logfiles do not grow thanks to the use of a unique data consolidation algorithm. The log is automatically consolidated so that it does not grow over time, but still contains all the relevant data for all the stats seen over the last two years.
RunAsDaemon: Enables daemon mode operation. Alternately, you remove this option and schedule MRTG from a cron job. The purpose of daemon mode is that MRTG is launched once and not repeatedly (as it is with cron). This behavior saves computing resources as loading and parsing of configuration files happens only once. Using daemon mode, MRTG itself is responsible for timing the measurement intervals. Therefore, it is important to set the Interval keyword to an appropriate value.
Interval: The interval (in minutes) that MRTG will poll for stats. The minimum value you may use is 5.
Target: Command to launch External Monitoring Script. This is what you would enter on the command lin to run your script. It is enclosed on backticks. In our example below, we assume webinject.pl is located in /usr/local/bin and it uses a config file named myconfig.xml.
MaxBytes: This option is required by MRTG even though it does not directly apply to the collecting of response times. I set this to a value that corresponds to the maximum value I will allow any stat to be. Values returned higher than this will be ignored by MRTG.
Options: Switches used by MTRG. See the MRTG Configuration Reference for more information.
XSize: Horizontal size of output graphs.
YSize: Vertical size of output graphs.
PNGTitle: Title displayed in the generated graphs.
LegendI: Text used in legend.
Ylegend: Text used in legend.
ShortLegend: Text used in legend.
Title: Title of output webpage.
PageTop: Text added to the top of the web page before the graphs are displayed.
AddHead: Text added to HTML header of output webpage. I use this to add CSS to style the page.

Sample MRTG Configuration File (mrtg.cfg)

Here is the sample mrtg.cfg file I use with WebInject:

WorkDir: /var/www/mrtg
RunAsDaemon: yes
Interval: 5

Target[myapplication]: `/usr/local/bin/webinject.pl -c myconfig.xml`
MaxBytes[myapplication]: 180
Options[myapplication]: gauge,growright,nopercent,noo,pngdate,nolegend,nobanner
XSize[myapplication]: 600
YSize[myapplication]: 175
PNGTitle[myapplication]: WebInject Response Times
LegendI[myapplication]: resp:
Ylegend[myapplication]: seconds
ShortLegend[myapplication]: s &nbsp;&nbsp;
Title[myapplication]: WebInject Response Time Analysis 
PageTop[myapplication]: <h1>Web Service-Level Monitor - MyApplication</h1>
                        <h2>WebInject Response Time Analysis</h2>

We can also add some CSS to the generated HTML header to style the output page a little better. Here is the final mrtg.cfg file with this added:

WorkDir: /var/www/mrtg
RunAsDaemon: yes

Target[myapplication]: `/usr/local/bin/webinject.pl -c myconfig.xml`
MaxBytes[myapplication]: 180
Options[myapplication]: gauge,growright,nopercent,noo,pngdate,nolegend,nobanner
XSize[myapplication]: 600
YSize[myapplication]: 175
LegendI[myapplication]: resp:
Ylegend[myapplication]: seconds
ShortLegend[myapplication]: s &nbsp;&nbsp;
Title[myapplication]: WebInject Response Time Analysis 
PageTop[myapplication]: <h1>Web Service-Level Monitor - MyApplication</h1>
                        <h2>WebInject Response Time Analysis</h2> 
AddHead[myapplication]: <style type="text/css"> 
                            body {
                                background-color: #EFEFEF;
                                color: #000000; 
                                font-family: sans-serif;
                                font-size: 12px; 
                            }
                            h1 {
                                background-color: #CCCCCC;
                                font-size: 18px;
                                font-weight: bold;
                                
                            }
                            h2 {
                                font-size: 16px;
                                font-weight: bold;
                            }
                        </style>

Sample MRTG Output

Below is a screenshot of the sample output produced by our MRTG/WebInject setup. It shows trends in response times of our [very slow] web service with several days of data collected.

MRTG Sample output:

(click image to enlarge)

You may also view the complete sample output (web page with embedded images).