Triggering active checks on remote hosts

Well I finally have my setup figured out, and I’m about ready to deploy my distributed systems, but I have a questions (well more of a mental exercise).

I would like to be able to trigger service/host checks on one of the remote nodes from the master server.

I would like the service on the main host to show up as a warning if the service goes stale, then contact a forced check on remote server. Could I do this with nrpe?

What processing logic could I use here to get this process:

  1. Service is stale
  2. Set service to ‘warning’
  3. Trigger the forced check plugin
  4. Plugin writes to remote servers command file to force the check
  5. If the plugin successfully writes to the file, go to hard warning state to wait for passive check to come in
  6. If the plugin fails, retry on set retry_interval
  7. If plugin fails after max_attempts, go to hard critical and force an active host check on the remote server (which can be directly pinged from the central server)