RT Nagios integration
Nagios and Request Tracker Integration
Creating Tickets
Introduction
Goals
Learn how to configure Nagios to raise tickets in RT
Notes
Commands preceded with “$” imply that you should execute the command as a general user - not as root.
Commands preceded with “#” imply that you should be working as root.
Commands with more specific command lines (e.g. “bdr1.campusY>” or “mysql>”) imply that you are executing commands on remote equipment, or within another program.
Exercises
To configure RT and Nagios so that alerts from Nagios automatically create tickets requires a few steps:

Create a proper contact entry for Nagios in /etc/nagios3/conf.d/contacts_nagios2.cfg on your host.

Update either services_nagios2.cfg or an individual host entry to use the new contact group.

These next two items should already be done in RT if you have finished the RT exercises.

Install the rt-mailgate software and configure it properly in your /etc/aliases file for your MTA in use.

Configure the appropriate queues in RT to receive emails passed to it from Nagios via the rt-mailgate software.

Exercises
Log in to your virtual machine as the sysadm user.

Configure a Contact in Nagios
Become root on your server:

$ sudo -s
Edit the file /etc/nagios3/conf.d/contacts_nagios2.cfg

# editor /etc/nagios3/conf.d/contacts_nagios2.cfg
In this file we will first add a new contact name under the default root contact entry.

WARNING: DO NOT remove the root contact entry.

REALLY - READ THIS! DO NOT remove the root contact entry.

ADD the entry below just AFTER the ROOT contact entry

The new contact should look like this (UNDERNEATH the root contact entry):

define contact{
        contact_name                    net
        alias                           RT Alert Queue
        service_notification_period     24x7
        host_notification_period        24x7
        service_notification_options    c 
        host_notification_options       d 
        service_notification_commands   notify-service-by-email
        host_notification_commands      notify-host-by-email
        email                           net@srv1.campusY.ws.nsrc.org
        }
Be sure that you replace “campusY” with the number of your group. So, if you are in group 1, then you would use “net@srv1.campus1.ws.nsrc.org”.

At the end of the file add the following entry: (Note: do not delete or update other entries.)

DO THIS AT THE END OF THE FILE

define contactgroup{
        contactgroup_name       tickets
        alias                   email to ticket system for RT
        members                 net,root
        }
Save and exit from the file.

Notes
the service_notification_option of “c” means only notify once a service is considered “critical” by Nagios (i.e. down). The host_notification_option of “d” means down. By specifying only “c” and “d” this means that notifications will not be sent for other states.

Note the email address in use “net@srv1.campusY.ws.nsrc.org” - this is important as this was previously defined in the Request Tracker (RT) exercises.

You could leave off “root” as a member, but we’ve left this on to have another user that receives email to help us troubleshoot if there are issues.

Choose a Service to Monitor that Creates Tickets in RT
To send email to generate tickets in RT if SSH goes down on a box you would edit the SSH service check:

# editor /etc/nagios3/conf.d/services_nagios2.cfg
Find the service with the hostgroup_name of “ssh-servers” and add the “contact_groups” entry at the end of the definition. When you are done your definition should look like this:

define service {
  hostgroup_name                  ssh-servers
  service_description             SSH
  check_command                   check_ssh
  use                             generic-service
  notification_interval           0 ; set > 0 if you want to be renotified
  contact_groups                  tickets,admins
}
Save and exit from the file.

Notes
Note the additional item that we now have, “contact_groups.” You can do this for other entries as well if you wish.

We, also, included the default contact group of admins. You could leave this off if you wish.

Include noc as members of ssh-server
# editor /etc/nagios3/conf.d/hostgroups_nagios2.cfg
Find the group with the hostgroup_name of “ssh-servers” and add noc to the “members” entry at the end of the definition. When you are done your definition will have a “noc” item added to it.

The entry below is for a host on campus1, so please don’t copy and paste the entire entry, just add “noc” to what you have.

# A list of your ssh-accessible servers
define hostgroup {
        hostgroup_name  ssh-servers
                alias           SSH servers
                members         srv1.campus1, host1.campus1, host2.campus1, host3.campus1, \
                                host4.campus1, host5.campus1, host6.campus1, transit1.nren, \
                                bdr1.campus1, core1.campus1, dist1-b1.campus1, dist1-b2.campus1, \
                                noc
        }
Now save and exit from the file.

To create a "NOC entry add this to the file pcs.cfg at the bottom of the file:

define host {
    use         generic-host
    host_name   noc
    alias       Main NOC server for NMM Workshop
    address     noc.ws.nsrc.org
}
Save and exit from the file.

Now verify that your Nagios configuration looks good:

# nagios3 -v /etc/nagios3/nagios.cfg
If there are no errors, then restart your nagios instance:

# systemctl restart nagios3
If SSH goes down on any of the devices you are monitoring Nagios should generate a new ticket in Request Tracker. We will stop the SSH service on the classroom NOC server. If you are not monitoring this machine, then you will need to add an entry for NOC in your Nagios configuration and add it to the ssh hostgroup defined in the file hostgroups_nagios2.cfg.

Save, exit from the file and restart Nagios:

# systemctl restart nagios3
See Nagios Tickets in RT
It will take a bit (up to 5 minutes) for Nagios to report that SSH is “critical”. Then you must wait for a total of 4 checks before the SSH service is deemed to be down “hard”. At that point Nagios will send an email to net@srv1.campusY.ws.nsrc.org and a ticket will be created in RT.

Remember to see this go to http://oob.srv1.campusY.ws.nsrc.org/rt/ and log in as Username “sysadm-hostX” with the password you used for your host. The new ticket should appear in the “10 newest unowned tickets” box in the main net queue on the main login page in RT.