We have 2 VPN Tunnels connecting our Annapolis and Pittsburgh offices. We found a bug in the Juniper Netscreen routers we are using that causes routes to be dropped on the Pittsburgh Netscreen pointing back to subnets in the Annapolis Office. Turns out the routes are all routes sitting behind our Cisco Core Switch/Router.
We initially wrote a CCS Script to tell us when the routes were down by running a Ping from our Core Cisco router to a router in PIttsburgh. This was great and notified us when the bug reared its nasty head. However we did not know how to resolve other than rebooting the NetScreen firewall.
Eventually we found a better way to get the routes back by forcing the Cisco 3750 to restart OSPF by issuing the command “clear ip ospf process \cmy”
So here is the script that we wrote to run a ping from our Cisco 3750 to a Netscreen across the VPN which checks if the Netscreen has a route back the Cisco 3750. If the ping fails we then raise an issue and run the “clear ip ospf process \cmy” command to restart OSPF and get the routes back
Script:
Ping A Host
Script-Description:
Ping a host and fire an issue if it's not reachable
Script-Filter:
$IPAddress eq 10.30.3.1
Script-Devices:
10.30.3.1
Script-Schedule:
*/3 * * * *
#################################################################
Action:
Perform Ping
Action-Description:
Run the ping command against our host
Action-Variables:
SET: $hostDead = "no"
Action-Commands:
ping 172.23.1.1
Output-Triggers:
Parse Output
#################################################################
Trigger:
Parse Output
Trigger-Description:
Find out if we can ping our host, update a variable.
Trigger-Variables:
$pingable string
Trigger-Template:
Success rate is [[$pingable]] percent .+
Trigger-Commands:{ $pingable eq "0" }
SET: $hostDead = "yes"
Output-Triggers:
Run Command
##################################################################
Trigger:
Run Command
Trigger-Description:
Clear IP OSPF Proccess on 3750
Trigger-Filter:
$hostDead eq "yes"
Trigger-Commands:
clear ip ospf process \cmy
copy run start \cm
q
Output-Triggers:
Host Down
#################################################################
Issue:
Host Down
Issue-ID:
pittsFWdown
Issue-Severity:
error
Issue-Description:
Cannot ping the remote host
Issue-Filter:
$hostDead eq "yes"
Issue-Details:
Host $IPAddress
#################################################################
Here is the issue we created for this:

Here is what the Script Script Schedule look slike for this: