5 minute monitor alert until email received


Right now my critical alert is set to send an email once.
I would like to change that to send out every 5 minutes which is no problem, BUT I want it to stop sending the alert if I was to reply back to that email. Is there something already written that will do this?
I think it would be best to only shut off the alert when it receives a reply from a specific distribution list like mon @ nagios.com