{section}
{column}
h1. Introduction
Nagios is a complete infrastructure monitoring solution.
[Get more details about Nagios.|http://www.nagios.org/]
The integration of Petals ESB and Nagios is based on getting information about the JVM. This is done using SNMP probes.
{info}Following examples are based on a Petals ESB topology composed of two nodes on the same host.{info}
h2. Nagios screenshots extract samples
{gallery:align=center|include=petals-esb-overview.jpg,petals-esb-services-overview.jpg,petals-esb-services-grid.jpg}
{column}
{column:width=350px}
{panel:title=Table of contents}{toc:outline=true}{panel}
{panel:title=Contributors}{contributors:order=name|mode=list|showAnonymous=true|showCount=true|showLastTime=true}{panel}
{column}
{section}
h1. Installation
h2. Nagios installation
The Nagios integration was experienced on a Linux distribution "Ubuntu Quantal Quetzal (12.10)", using the Sun/Oracle JVM v1.6.0.43
Nagios 3 is available through the default Ubuntu repository. So, just install the package "nagios3" using the standard way.
h2. Enabling the SNMP agents of the Petals ESB JVMs
{include:Enabling the SNMP agent of a Petals ESB JVM}
h1. Nagios configuration
h2. Core configuration
All the following configurations are grouped in the attached file to put in your directory [{{/etc/nagios3/conf.d/petals-esb.cfg}}|https://anonymous:anonymous@svn.petalslink.org/svnroot/trunk/product/dev/prod/petals/client-tools/petals-nagios/src/cfg/petals-esb.cfg].
h3. Petals ESB administrators
Define your Petals ESB administrator as following. For each administrator, don't forget to add it to the contact group:
{code}
define contact {
contact_name cde
alias Christophe DENEUX
host_notification_period 24x7
service_notification_period 24x7
host_notification_options d,u,r
service_notification_options w,u,c,r
host_notification_commands notify-host-by-email
service_notification_commands notify-service-by-email
email christophe.deneux@linagora.com
}
define contactgroup {
contactgroup_name petals-esb
alias Petals ESB Administrators
members cde
}
{code}
h3. JVM host template
A best practice to monitor Java application is to create a template 'JVM host':
{code}
define host{
use generic-host
name jvm-host ; The name of this host template
notifications_enabled 1 ; Host notifications are enabled
event_handler_enabled 1 ; Host event handler is enabled
flap_detection_enabled 1 ; Flap detection is enabled
failure_prediction_enabled 1 ; Failure prediction is enabled
process_perf_data 1 ; Process performance data
retain_status_information 1 ; Retain status information across program restarts
retain_nonstatus_information 1 ; Retain non-status information across program restarts
check_command check-host-alive
max_check_attempts 10
notification_interval 0
notification_period 24x7
notification_options d,u,r
contact_groups admins
register 0 ; DONT REGISTER THIS DEFINITION - ITS NOT A REAL HOST, JUST A TEMPLATE!
# Specific attributes
_snmpport 161 ; Listening port of the JVM SNMP agent
}
{code}
h3. Nagios commands definition to interact with a SNMP agent
We recommand to define specific Nagios command to interact with the Java virtual machine MIB.
Interresting commands will be:
* jvm_heapused: to get the real heap memory used by the Petals ESB node,
* jvm_heapmaxused: to get the current allocated (commited) heap size. This value should be check against the heap max size at the JVM level to generate alarms.
{code}
# 'jvm_heapused' command definition
define command{
command_name jvm_heapused
command_line /usr/lib/nagios/plugins/check_snmp -H '$HOSTADDRESS$' -p '$_HOSTSNMPPORT$' -C '$ARG1$' -P '$ARG2$' -o 1.3.6.1.4.1.42.2.145.3.163.1.1.2.11.0 -w :'$ARG3$' -c :'$ARG4$' -l load
}
# 'jvm_heapmaxused' command definition
define command{
command_name jvm_heapmaxused
command_line /usr/lib/nagios/plugins/check_snmp -H '$HOSTADDRESS$' -p '$_HOSTSNMPPORT$' -C '$ARG1$' -P '$ARG2$' -o 1.3.6.1.4.1.42.2.145.3.163.1.1.2.12.0 -w :'$ARG3$' -c :'$ARG4$' -l load
}
{code}
h3. Petals ESB host template
A best practice to monitor Petals ESB nodes is to create a template 'Petals ESB host' that inherits from the 'JVM host':
{code}
define host{
use jvm-host
name petals-esb-host ; The name of this host template
register 0 ; DONT REGISTER THIS DEFINITION - ITS NOT A REAL HOST, JUST A TEMPLATE!
contact_groups petals-esb
# Specific attributes
_snmpport 16161 ; Listening port of the JVM SNMP agent. Redefined because Petals is not started by 'root'
_jmxport 7700 ; Listening port of the Petals JMX agent
_jmxuser petals ; JMX credentials: username
_jmxpassword petals ; JMX credentials: password
}
{code}
h3. Defining your Petals ESB hosts
For each Petals ESB node of your Petals ESB topology, create an instance of the template 'petals-esb-host':
{code}
define host{
use petals-esb-host ; Name of host template to use
host_name petals-esb-node-1
alias Petals ESB Node 1
address 127.0.0.1
# Specific attributes: Uncomment to specify a value different from the default one
# _snmpport 16161 ; Listening port of the JVM SNMP agent
# _jmxport 7700 ; Listening port of the Petals JMX agent
# _jmxuser petals ; JMX credentials: username
# _jmxpassword petals ; JMX credentials: password
}
define host{
use petals-esb-host ; Name of host template to use
host_name petals-esb-node-2
alias Petals ESB Node 2
address 192.168.1.25
# Specific attributes: Uncomment to specify a value different from the default one
# _snmpport 16161 ; Listening port of the JVM SNMP agent
# _jmxport 7700 ; Listening port of the Petals JMX agent
# _jmxuser petals ; JMX credentials: username
# _jmxpassword petals ; JMX credentials: password
}
{code}
h3. Defining your Petals ESB topology as a Petals ESB host group
The Petals ESB topology can be considered as a Nagios host group composed of your Petals ESB nodes:
{code}
define hostgroup {
hostgroup_name petals-esb
alias The ESB
members petals-esb-node-1, petals-esb-node-2
}
{code}
h3. Petals ESB host services
We group all Petals ESB host services:
{code}
define servicegroup{
servicegroup_name petals-services
alias Petals ESB Services
}
{code}
A best practice to monitor Petals ESB nodes is to create a template of services to associate to each Petals ESB nodes:
{code}
# Define a service to check the disk space of the partitions
# Warning if < 20% free, critical if < 10% free space on partition.
# Adapt the last parameter of "check_disk" to check only the
# partitions where dynamic data (as log files, repository) are
# written by petals (/var/lib/petals-esb, /var/log/petals-esb).
# Probably two services will be required with different thresholds.
define service{
use generic-service
hostgroup_name petals-esb
service_description Disk Space
check_command check_disk!20%!10%!/var
servicegroups petals-services
contact_groups petals-esb
}
# Define a service to check the load on the local machine.
define service{
use generic-service
hostgroup_name petals-esb
service_description Current Load
check_command check_load!5.0!4.0!3.0!10.0!6.0!4.0
servicegroups petals-services
contact_groups petals-esb
}
# Define a service to check the heap memory used by Petals ESB node.
# Warning if > 85% configured max heap size, critical if 95% configured
# max heap size.
# Note: The value returned by the SNMP Get is a byte value. So, for
# a max heap size of 1Go:
# 85% => 912680550 bytes,
# and 95% => 1020054733 bytes
define service{
use generic-service
hostgroup_name petals-esb
service_description Current heap memory
check_command jvm_heapused!public!2c!912680550!1020054732
servicegroups petals-services
contact_groups petals-esb
}
# Define a service to check the max (commited) heap memory used by Petals ESB node.
# Warning if > Warning if > 85% configured max heap size, critical if 95% configured
# max heap size.
# Note: The value returned by the SNMP Get is a byte value. So, for
# a max heap size of 1Go:
# 85% => 912680550 bytes,
# and 95% => 1020054733 bytes
define service{
use generic-service
hostgroup_name petals-esb
service_description Current commited heap memory size
check_command jvm_heapmaxused!public!2c!912680550!1020054732
servicegroups petals-services
contact_groups petals-esb
}
{code}
h2. Petals BC SOAP configuration
h3. Active checks
To add services associated to the BC SOAP, you can add a service based on the [here|https://anonymous:anonymous@svn.petalslink.org/svnroot/trunk/product/dev/prod/petals/client-tools/petals-nagios/src/cfg/petals-esb-bcsoap.cfg] that check the current number of allocated threads of the HTTP thread pool:
{code}
########################################################
#### Dedicated commands to monitor a Petals BC SOAP ####
########################################################
define command{
command_name petals_bcsoap_httpthreadpool_currentallocatedthreads
command_line /usr/lib/nagios/plugins/check_petals_bcsoap -h '$HOSTADDRESS$' -n '$_HOSTJMXPORT$' -u '$_HOSTJMXUSER$' -p '$_HOSTJMXPASSWORD$' --http-thread-pool-current-allocated-threads -i '$ARG1$' -w '$ARG2$' -c '$ARG3$'
}
# Define a service to check the HTTP Thread pool of the Petals BC SOAP against the current
# allocated thread number.
# According to default value: Warining if > 40, critical if > 45.
define service{
use generic-service
hostgroup_name petals-esb
service_description petals-esb-bcsoap-httpthreadpool-current-allocated
display_name Petals BC SOAP - HTTP Thread Pool - Current allocated thread
check_command petals_bcsoap_httpthreadpool_currentallocatedthreads!petals-bc-soap!40!45
servicegroups petals-services
contact_groups petals-esb
}
{code}
where {{check_petals_bcsoap}} is a nagios plugin wrapping Petals CLI. You can get a such plugin [here|https://anonymous:anonymous@svn.petalslink.org/svnroot/trunk/product/dev/prod/petals/client-tools/petals-nagios/src/scripts/check_petals_bcsoap].
h3. Passive check
The probes available in the BC SOAP are available to send JMX notifications when they detect a defect. These notifications can be sent to Nagios with a dedicated new monitoring command of Petals CLI (this command does not exist yet) that receive JMX notifications and write them in the channel of Nagios passive checks.
For example, when no more threads is available in the HTTP thread pool of the BC SOAP:
{code}
echo "[`date +%s`] PROCESS_SERVICE_CHECK_RESULT;petals-esb-node-1;petals-esb-bcsoap-httpthreadpool-current-allocated;2;No more thread available" > /var/lib/nagios3/rw/nagios.cmd
{code}
h2. Restart Nagios
Restart your Nagios service to take into account the configuration:
{code}
sudo service nagios3 restart
{code}
{column}
{section}
{column}
h1. Introduction
Nagios is a complete infrastructure monitoring solution.
[Get more details about Nagios.|http://www.nagios.org/]
The integration of Petals ESB and Nagios is based on getting information about the JVM. This is done using SNMP probes.
{info}Following examples are based on a Petals ESB topology composed of two nodes on the same host.{info}
h2. Nagios screenshots extract samples
{gallery:align=center|include=petals-esb-overview.jpg,petals-esb-services-overview.jpg,petals-esb-services-grid.jpg}
{column}
{column:width=350px}
{panel:title=Table of contents}{toc:outline=true}{panel}
{panel:title=Contributors}{contributors:order=name|mode=list|showAnonymous=true|showCount=true|showLastTime=true}{panel}
{column}
{section}
h1. Installation
h2. Nagios installation
The Nagios integration was experienced on a Linux distribution "Ubuntu Quantal Quetzal (12.10)", using the Sun/Oracle JVM v1.6.0.43
Nagios 3 is available through the default Ubuntu repository. So, just install the package "nagios3" using the standard way.
h2. Enabling the SNMP agents of the Petals ESB JVMs
{include:Enabling the SNMP agent of a Petals ESB JVM}
h1. Nagios configuration
h2. Core configuration
All the following configurations are grouped in the attached file to put in your directory [{{/etc/nagios3/conf.d/petals-esb.cfg}}|https://anonymous:anonymous@svn.petalslink.org/svnroot/trunk/product/dev/prod/petals/client-tools/petals-nagios/src/cfg/petals-esb.cfg].
h3. Petals ESB administrators
Define your Petals ESB administrator as following. For each administrator, don't forget to add it to the contact group:
{code}
define contact {
contact_name cde
alias Christophe DENEUX
host_notification_period 24x7
service_notification_period 24x7
host_notification_options d,u,r
service_notification_options w,u,c,r
host_notification_commands notify-host-by-email
service_notification_commands notify-service-by-email
email christophe.deneux@linagora.com
}
define contactgroup {
contactgroup_name petals-esb
alias Petals ESB Administrators
members cde
}
{code}
h3. JVM host template
A best practice to monitor Java application is to create a template 'JVM host':
{code}
define host{
use generic-host
name jvm-host ; The name of this host template
notifications_enabled 1 ; Host notifications are enabled
event_handler_enabled 1 ; Host event handler is enabled
flap_detection_enabled 1 ; Flap detection is enabled
failure_prediction_enabled 1 ; Failure prediction is enabled
process_perf_data 1 ; Process performance data
retain_status_information 1 ; Retain status information across program restarts
retain_nonstatus_information 1 ; Retain non-status information across program restarts
check_command check-host-alive
max_check_attempts 10
notification_interval 0
notification_period 24x7
notification_options d,u,r
contact_groups admins
register 0 ; DONT REGISTER THIS DEFINITION - ITS NOT A REAL HOST, JUST A TEMPLATE!
# Specific attributes
_snmpport 161 ; Listening port of the JVM SNMP agent
}
{code}
h3. Nagios commands definition to interact with a SNMP agent
We recommand to define specific Nagios command to interact with the Java virtual machine MIB.
Interresting commands will be:
* jvm_heapused: to get the real heap memory used by the Petals ESB node,
* jvm_heapmaxused: to get the current allocated (commited) heap size. This value should be check against the heap max size at the JVM level to generate alarms.
{code}
# 'jvm_heapused' command definition
define command{
command_name jvm_heapused
command_line /usr/lib/nagios/plugins/check_snmp -H '$HOSTADDRESS$' -p '$_HOSTSNMPPORT$' -C '$ARG1$' -P '$ARG2$' -o 1.3.6.1.4.1.42.2.145.3.163.1.1.2.11.0 -w :'$ARG3$' -c :'$ARG4$' -l load
}
# 'jvm_heapmaxused' command definition
define command{
command_name jvm_heapmaxused
command_line /usr/lib/nagios/plugins/check_snmp -H '$HOSTADDRESS$' -p '$_HOSTSNMPPORT$' -C '$ARG1$' -P '$ARG2$' -o 1.3.6.1.4.1.42.2.145.3.163.1.1.2.12.0 -w :'$ARG3$' -c :'$ARG4$' -l load
}
{code}
h3. Petals ESB host template
A best practice to monitor Petals ESB nodes is to create a template 'Petals ESB host' that inherits from the 'JVM host':
{code}
define host{
use jvm-host
name petals-esb-host ; The name of this host template
register 0 ; DONT REGISTER THIS DEFINITION - ITS NOT A REAL HOST, JUST A TEMPLATE!
contact_groups petals-esb
# Specific attributes
_snmpport 16161 ; Listening port of the JVM SNMP agent. Redefined because Petals is not started by 'root'
_jmxport 7700 ; Listening port of the Petals JMX agent
_jmxuser petals ; JMX credentials: username
_jmxpassword petals ; JMX credentials: password
}
{code}
h3. Defining your Petals ESB hosts
For each Petals ESB node of your Petals ESB topology, create an instance of the template 'petals-esb-host':
{code}
define host{
use petals-esb-host ; Name of host template to use
host_name petals-esb-node-1
alias Petals ESB Node 1
address 127.0.0.1
# Specific attributes: Uncomment to specify a value different from the default one
# _snmpport 16161 ; Listening port of the JVM SNMP agent
# _jmxport 7700 ; Listening port of the Petals JMX agent
# _jmxuser petals ; JMX credentials: username
# _jmxpassword petals ; JMX credentials: password
}
define host{
use petals-esb-host ; Name of host template to use
host_name petals-esb-node-2
alias Petals ESB Node 2
address 192.168.1.25
# Specific attributes: Uncomment to specify a value different from the default one
# _snmpport 16161 ; Listening port of the JVM SNMP agent
# _jmxport 7700 ; Listening port of the Petals JMX agent
# _jmxuser petals ; JMX credentials: username
# _jmxpassword petals ; JMX credentials: password
}
{code}
h3. Defining your Petals ESB topology as a Petals ESB host group
The Petals ESB topology can be considered as a Nagios host group composed of your Petals ESB nodes:
{code}
define hostgroup {
hostgroup_name petals-esb
alias The ESB
members petals-esb-node-1, petals-esb-node-2
}
{code}
h3. Petals ESB host services
We group all Petals ESB host services:
{code}
define servicegroup{
servicegroup_name petals-services
alias Petals ESB Services
}
{code}
A best practice to monitor Petals ESB nodes is to create a template of services to associate to each Petals ESB nodes:
{code}
# Define a service to check the disk space of the partitions
# Warning if < 20% free, critical if < 10% free space on partition.
# Adapt the last parameter of "check_disk" to check only the
# partitions where dynamic data (as log files, repository) are
# written by petals (/var/lib/petals-esb, /var/log/petals-esb).
# Probably two services will be required with different thresholds.
define service{
use generic-service
hostgroup_name petals-esb
service_description Disk Space
check_command check_disk!20%!10%!/var
servicegroups petals-services
contact_groups petals-esb
}
# Define a service to check the load on the local machine.
define service{
use generic-service
hostgroup_name petals-esb
service_description Current Load
check_command check_load!5.0!4.0!3.0!10.0!6.0!4.0
servicegroups petals-services
contact_groups petals-esb
}
# Define a service to check the heap memory used by Petals ESB node.
# Warning if > 85% configured max heap size, critical if 95% configured
# max heap size.
# Note: The value returned by the SNMP Get is a byte value. So, for
# a max heap size of 1Go:
# 85% => 912680550 bytes,
# and 95% => 1020054733 bytes
define service{
use generic-service
hostgroup_name petals-esb
service_description Current heap memory
check_command jvm_heapused!public!2c!912680550!1020054732
servicegroups petals-services
contact_groups petals-esb
}
# Define a service to check the max (commited) heap memory used by Petals ESB node.
# Warning if > Warning if > 85% configured max heap size, critical if 95% configured
# max heap size.
# Note: The value returned by the SNMP Get is a byte value. So, for
# a max heap size of 1Go:
# 85% => 912680550 bytes,
# and 95% => 1020054733 bytes
define service{
use generic-service
hostgroup_name petals-esb
service_description Current commited heap memory size
check_command jvm_heapmaxused!public!2c!912680550!1020054732
servicegroups petals-services
contact_groups petals-esb
}
{code}
h2. Petals BC SOAP configuration
h3. Active checks
To add services associated to the BC SOAP, you can add a service based on the [here|https://anonymous:anonymous@svn.petalslink.org/svnroot/trunk/product/dev/prod/petals/client-tools/petals-nagios/src/cfg/petals-esb-bcsoap.cfg] that check the current number of allocated threads of the HTTP thread pool:
{code}
########################################################
#### Dedicated commands to monitor a Petals BC SOAP ####
########################################################
define command{
command_name petals_bcsoap_httpthreadpool_currentallocatedthreads
command_line /usr/lib/nagios/plugins/check_petals_bcsoap -h '$HOSTADDRESS$' -n '$_HOSTJMXPORT$' -u '$_HOSTJMXUSER$' -p '$_HOSTJMXPASSWORD$' --http-thread-pool-current-allocated-threads -i '$ARG1$' -w '$ARG2$' -c '$ARG3$'
}
# Define a service to check the HTTP Thread pool of the Petals BC SOAP against the current
# allocated thread number.
# According to default value: Warining if > 40, critical if > 45.
define service{
use generic-service
hostgroup_name petals-esb
service_description petals-esb-bcsoap-httpthreadpool-current-allocated
display_name Petals BC SOAP - HTTP Thread Pool - Current allocated thread
check_command petals_bcsoap_httpthreadpool_currentallocatedthreads!petals-bc-soap!40!45
servicegroups petals-services
contact_groups petals-esb
}
{code}
where {{check_petals_bcsoap}} is a nagios plugin wrapping Petals CLI. You can get a such plugin [here|https://anonymous:anonymous@svn.petalslink.org/svnroot/trunk/product/dev/prod/petals/client-tools/petals-nagios/src/scripts/check_petals_bcsoap].
h3. Passive check
The probes available in the BC SOAP are available to send JMX notifications when they detect a defect. These notifications can be sent to Nagios with a dedicated new monitoring command of Petals CLI (this command does not exist yet) that receive JMX notifications and write them in the channel of Nagios passive checks.
For example, when no more threads is available in the HTTP thread pool of the BC SOAP:
{code}
echo "[`date +%s`] PROCESS_SERVICE_CHECK_RESULT;petals-esb-node-1;petals-esb-bcsoap-httpthreadpool-current-allocated;2;No more thread available" > /var/lib/nagios3/rw/nagios.cmd
{code}
h2. Restart Nagios
Restart your Nagios service to take into account the configuration:
{code}
sudo service nagios3 restart
{code}
{column}
{section}