Data is stale error every one hour


#1

Hi ,
I’ve been getting “data is stale” error every one hour . Here is the error:

[Fri Oct 22 00:55:08 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 00:55:13 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 00:55:13 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 01:56:08 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 01:56:13 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 01:56:13 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 02:57:08 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 02:57:13 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 02:57:13 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 03:58:09 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 03:58:13 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 03:58:13 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 05:00:14 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 05:00:18 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 05:00:18 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 06:00:08 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 06:00:13 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 06:00:13 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 07:01:43 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 07:01:48 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 07:01:48 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 08:04:44 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 08:04:48 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 08:04:48 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 09:03:08 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 09:03:13 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 09:03:13 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 10:04:44 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 10:04:48 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 10:04:48 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 11:05:24 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 11:05:28 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 11:05:28 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 12:06:43 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 12:06:48 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 12:06:48 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 13:09:43 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 13:09:48 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 13:09:48 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale
[Fri Oct 22 14:08:07 2010] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 14:08:11 2010] PASSIVE SERVICE CHECK: blxc7;NodeInfo;1;[blxc37] WARNING: data is stale
[Fri Oct 22 14:08:11 2010] SERVICE ALERT: blxc7;NodeInfo;WARNING;SOFT;1;[blxc37] WARNING: data is stale

./check_metrics -d 0 btime

SUPERMON OK - Retrieved metrics from 10 hosts.

Could you tell me how to stop these error from appearing?


#2

A bit more info wouldn’t be bad… what check is it? how is it configured?

It looks like you mihgt have a passive check which needs to be run a bit more often… but it’s a while since i had to use passive checks so i might be wrong :slight_smile:


#3

Hi ,
Thanks a lot for your prompt reply…:slight_smile:

Nagios throws error every 1 hour on these three services (Nodeinfo, Load Average, System Free Space).
Yes,the passive checks are enabled.

This is the service definition for the above three services:

Service definition

define service{
use nrpe ; Name of service template to use

    host_name                       %HOST%
    service_description             NodeInfo
    is_volatile                     0
    check_period                    24x7
    max_check_attempts              3
    normal_check_interval           60
    contact_groups                  admins

    active_checks_enabled           0
    passive_checks_enabled          1
    check_command                   check_dummy!1!"data is stale"
    register                        1
    #SERVICEGROUP                   PassiveChecks
    }

Service definition

define service{
use nrpe ; Name of service template to use

    host_name                       %HOST%
    service_description             Load Average
    is_volatile                     0
    check_period                    24x7
    max_check_attempts              3
    normal_check_interval           60
    contact_groups                  admins
    active_checks_enabled           0
    passive_checks_enabled          1

check_load_average is not called here since we are passive

Called from check_metrics

    check_command                   check_dummy!1!"load average data is stale"
    register                        1
    #SERVICEGROUP                   PassiveChecks
    }

Service definition

define service{
use nrpe ; Name of service template to use

    host_name                       %HOST%
    service_description             System Free Space
    is_volatile                     0
    check_period                    24x7
    max_check_attempts              3
    normal_check_interval           60
    contact_groups                  admins
    check_command                   check_dummy!1!"data is stale"
    active_checks_enabled           0
    passive_checks_enabled          1
    register                        1
    #SERVICEGROUP                   PassiveChecks
    }

Could you let me know what paramaters must be added so that the stale errors are not recorded?


#4

It looks like you are activley generating the stale errors every 60 minutes… :slight_smile:

Have a read through this part of the docs.
nagios.sourceforge.net/docs/3_0/freshness.html


#5

Hi,
I’m still facing the issue that for a few services in the /opt/hptc/nagios/var/nagios.log, i keep getting stale messages.
I get these kind of messages:
[Wed Feb 2 04:00:51 2011] Warning: The results of service ‘NodeInfo’ on host ‘node7’ are stale by 0d 0h 2m 45s (threshold=0d 1h 0m 15s). I’m forcing an immediate check of the service.

I have tried adding check_freshness parameter in nrpe_template.cfg and set it to 0 for each service, but it doesn’t stop these stale messages from coming.

Is there any way to stop these stale warning from coming up for all the services?
If yes what paramaters need to be added in which files?
FYI: I have two management servers for nagios, and one of them is the management_hub so the modifications should be done on both the servers? or just the hub?

Thanks in advance!