Please help with delayed notification

Nagios Nagios Configuration support
1

I’m trying to set up certain notifications that aren’t really important enough to wake up in the middle of the night but I’d like to get to first thing in the morning. If a service flips to ‘warning’ state and it is at night, I’d like a notification first thing in the morning. Furthermore, I’d like to automatically reset the state of the service to ‘OK’ immediately after the notification.

In other words, I’d like nagios to watch for a condition over night and send a single notification in the morning if it has occurred and not have to worry about resetting the state back to ok.

I was thinking I would implement this using a passive check… But I have not been able to get it to work properly. Does anyone have any tips? Thanks.

Edit: I just got very, very close by making the service a volatle one… The status of it gets reset how I want. However, when the first check happens, the service goes from warning to OK and I don’t seem to be able to get a notification.

Please help!

Second edit–
I think I’ve got a pretty good solution for this using normal unix ‘at’. I set up my service with the following:

active_checks_enabled 0
passive_checks_enabled 1
is_volatile 1
notification_period 24x7

I think that’s all that is important. Then I copied the service and host email commands and at the end of the command I put:
| at $NEXTVALIDTIME:wakehours$

…my theory is that notifications will go out from Nagios as normal; even a five minute ‘flip’ that resets before daylight. However, the notifications will sit in the at queue if it is night, and go out right away if it is day.

I’ll tell you if it works with another edit.

2

I was able to get this working in a rather nice way… Although it took a bit more work then I thought. I’m alittle surprised that this is the easiest way to accomplish a queue of notifications and someone please tell me if I’m missing something… Basically what I ended up doing was creating a perl script that takes parameters message, subject, time, and email and queues up ‘mail’ with the ‘at’ command at the time provided. Here is the script:

#!/usr/bin/perl
use Getopt::Std;

%options=();
getopts("t:e:s:m:",\%options);

if (! defined $options{t} ||
    ! defined $options{e} ||
    ! defined $options{s} ||
    ! defined $options{m}
    ) {
    print "You are missing an arguement.\n";
    exit;
}

# Need to add a minute to prevent calling at with a past time
$epoch = int($options{t}) + 60;

my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($epoch);
$time = sprintf("%02s:%02s",$hour,$min);

`echo '/usr/bin/printf "%b" "$options{m}" | /usr/bin/mail -s "$options{s}" $options{e}' | at $time`;

Also I needed to rearrange my notification send command a little bit:

# 'notify-service-by-sms-delayed' command definition
define command{
        command_name    notify-service-by-sms-delayed
        command_line    /usr/local/nagios/libexec/at_mail.pl -t $NEXTVALIDTIME:wakehours$ -e $CONTACTPAGER$ -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" -m "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$"
        }

The end effect of this is having notifications queue up if they are not within the time range, and they will wait until the time range begins then page out. This occurs no matter how long the service in question is in a pagable state.

Also, I don’t need this now AND THIS IS UNTESTED but I had a quick idea how a person could modify this script to queue up a notification digest to be sent at increments throughout the day. Basically the idea is to have a holding file for notifications and create an at job to mail it at some point later if it doesn’t exist yet. Then you simply append notifications to it as they come.

#!/usr/bin/perl
use Getopt::Std;

$digestdir = "/tmp/digest";
`mkdir -p $digestdir`;

%options=();
getopts("t:e:s:m:",\%options);

if (! defined $options{t} ||
    ! defined $options{e} ||
    ! defined $options{s} ||
    ! defined $options{m}
    ) {
    print "You are missing an arguement.\n";
    exit;
}


$epoch = int($option(t))+60;
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($epoch);

$time = sprintf("%02s:%02s",$hour,$min);
$digestfile = "$digestdir/digest.$year$mon$day.$time";

$makejob=(! -f "$digestfile");

`echo $options{s} >> $digestfile`;
`echo '/usr/bin/printf "%b" "$options{m}" >> $digestfile`;

`echo 'cat $digestfile | /usr/bin/mail -s "$year/$month/$day Alert Digest" $options{e}' | at $time` if ($makejob);
3

thanks for your own feedback; i may be handy for some people ^^