Executing commands from third party soft


#1

Hi there,

I’m trying to execute a command in nagios monitoring host but triggered by another host.
I.E. i want to execute a command in a pre backup (on a backup host) that tells nagios host to execute a check, is it possible?
I thought abt using passive checks but as far as i know i can’t execute commands that way, am i wrong?
If there’s any idea please let me know :slight_smile:

Thanks in advance!


#2

possibly if you tell us what you are trying to do we can find a way to get it done :slight_smile:

if you want to check for something like “are enough tapes available” you can do it having a check made every 30-60 minutes for a partiucular OID via check_snmp or you could think of some sort of snmp trap triggering an alert… that’s what comes to mind first…

Luca


#3

Ok, i’ll be more explicit:

I’m checking a productive SAP, and sometime in the month it goes down for backup. Obviously the check will fail, and the notification will be sent. The matter is that the people that recives it does not know if the server was really down because a failure.
I was asked to make TSM (the backup system) notify nagios when the back up is inminent, to make nagios send a mail saying “Hey, SAP is goining down for backup”. And the other notifications (SAP down and recovery) would be proof that the back up went ok.

About the mail I can design a check/misc command that sends a mail saying what i need.

So the question is what can i make TSM execute that check.

I know it would be easier to make TSM send the mail, but their idea is to make nagios send all kind of notifications.

I hope to be clearer this time.

cheers


#4

if the backup is at scheduled times i think it would be best to define a timeperiod where this particular machine is not checked.

Luca


#5

Well, al this stuff is because I can’t know when TSM does the backup, it is not synchronic. But instead TSM can execute pre and post backup commands.

Also i’m interested to know when the back up takes down and up the host, so i prefer not to touch the timeperiod.

I just need to send a that mail.

What i thought is to define a dummy service check and sumbit a passive check that makes it fail and then notify with the command, that i defined before. Tought it’s not a simplistic method, it may work.

Thanks for the reply!


#6

possibly not easy,… but worth a thought…

have a check on some value on the server. let the prebackup script set some flag, the check finds this flag and executes a command (nagios can do this) setting the service in scheduled downtime (this one could be a bit nasty) when the post backup script runs it sets the previous flag back to 0 and the check finding a recovery resets the sdcheduled downtime on the nagios service…

Luca

PS: If you don’t have too many contacts monitoring that particular service you could think of a simple mail comand BACKUP STARTING at pre backup… and obviously BACKUP COMPLETE at post backup…
Edited Wed Dec 28 2005, 10:12AM ]


#7

Yes, that would be much easier, but this people wants nagios to send all the mails… IMHO it’s not a good policy but that’s what they pay for.

Thanks again.


#8

This guy did it using NSCA and an event handler:

nagiosexchange.org/Wiki_Home … drwiki_pi1[keyword]=RemoteHostScheduledDowntimeAtReboot

But if it were me, I’d probably write a script on the remote host that rsync’s or scp’s some token filename with a few bytes in it over to the nagios host, and have a cron job run a script on the nagios host every minute checking for the existance of that file that isn’t zero-length, and if it exists, create a filename.running file (so that the script doesn’t trip over itself the next minute if still running), and then write a “DISABLE_SVC_NOTIFICATIONS” command to the nagios external cmd pipe and leave the filename.running in place. When the remote host is done, it can rsync or scp over a zero-length file of the same filename, and the next minute the script executes, it sees the zero-length file and deletes the both the filename and filename.running, and sends a “ENABLE_SVC_NOTIFICATIONS” command to the nagios external cmd pipe.

For a full list of external commands:
nagios.org/developerinfo/ext … ndlist.php

I just whipped up the script below and tested it (sorry, the forums screw up the indents), I created /var/spool/tsmbackup with the word MEAT in it and ran the script, which disabled active checks on a particular service, and then later I did a “> /var/spool/tsmbackup” (zero-lengthed the file) and ran the script again and it enabled the service checks and cleaned up after itself.

#!/bin/bash

now=date +%s

commandfile=’/var/spool/nagios/nagios.cmd’

tokenfile=’/var/spool/tsmbackup’

if -f $tokenfile ] ; then

if  ! -s $tokenfile ] ; then
  # backup has completed

  echo -e "${now}] ENABLE_SVC_NOTIFICATIONS;sn-db2;CDR Database Push\\n" > $commandfile

  if  -f ${tokenfile}.running ] ; then
    rm ${tokenfile}.running
  fi

  rm $tokenfile

else
  # backup is occuring

  if  ! -f ${tokenfile}.running ] ; then
    touch ${tokenfile}.running
    echo -e "${now}] DISABLE_SVC_NOTIFICATIONS;sn-db2;CDR Database Push\\n" > $commandfile
  fi

fi

fi

You could add some mail commands into the script if you want to warn people that the backup is happening and the service isn’t being monitored or is done and is being monitored again. You could also add a check to see how long the .running file has been hanging around, and re-enable the service checks and clean up if it’s been over a certain duration, so that if the database is still down (backup program barfed?) after the expected backup period, nagios will be the one to send out the alert. Also, instead of using DISABLE_SVC_NOTIFICATIONS, you could let the service-down alert happen, and instead play with ACKNOWLEDGE_SVC_PROBLEM, which gets you your requirement above to only use nagios for all emails.

-mike

Edited Thu Dec 29 2005, 12:03AM ]


#9

and even if nagios is not sending the notifications… you could make them look like nagios notifications… ok ok i know this is not good… but oif people WANT… well they can get it :smiley:

Luca


#10

On your remote host, setup a small nagios setup that will use send_nsca to the central nagios server. On the remote host, your check commands will be active checks, sent to nagios server as passive checks. To satisfy your need of “when backups are being done…” make one of your service checks check_procs. Setup service dependancies on the remote host, so that "If service check check_procs “backup running” is running, then the other service checks will be dependant on that service check.
Nagios central server will be getting this info via nsca and will send out notifications of any state changes like “service check “backup running” found 1 process with args “tapebackup””. This is a state change and will send out an email. The other service checks are dependant on that service, so you will NOT get emails about all your other service checks.

The above would satisfy your criteria of not knowing when the backups occur. Nagios would handle it since you have setup “service dependancies”
Edited Fri Dec 30 2005, 05:48PM ]