We have a distributed/redundant setup, using one master and one slave in each of two different sites. Each slave updates both masters. Master A is the only one with notifications turned on. Master B watches master A. If master A goes down, master B resumes notifications. Both masters have active service checks turned off, but have freshness checks enabled.
The service check on master B to check on master A is right out of the documented redundancy model, using the host_state and proc_state event handlers. Right now, since active service checks are disabled on master B, the only time master B checks on master A is when the freshness check detects a stale update and forces an active service check. This works, but it’s slower and more inconsistent than what I’d like. I’m aware of the external command SCHEDULE_FORCED_SVC_CHECK, but I’m under the impression the best way to use that is through a cron job, and I’d rather keep all my scheduling in one place (within Nagios) for ease of administration. I’m also aware that I can enable service checks for a single service by turning global active service checks on, then turning active service checks for each individual service off except the one I want to use. That seems a little drastic, but it’s the best solution I can come up with so far.
Is there a way to enable active service checks for a single service, while still keeping the global active service checks turned off? Or maybe another way to use the SCHEDULE_FORCED_SVC_CHECK command from within Nagios? I’m open to any suggestions.