ISSUE SUMMARY

On Tuesday, 12 November 2019, the Frankfurt site was not serving DNS traffic for 1 hour and 10 minutes. It started at 11:41 PM PST and by 12:50 AM PST, 13 November 2019, the traffic to Frankfurt Site was re-routed to the nearest other Sites.

Customers may have experienced issues with DNS resolution and latency.We apologize to our customers whose services or businesses were impacted during this incident, and we are taking immediate steps to improve the platform’s performance and availability.

ROOT CAUSE AND REMEDIATION

Infoblox Cloud platform was affected due to network congestion issues reported by Amazon Web Services in their Frankfurt Availability Zone.In an event of traffic impact, the services are routed to other Sites, in this particular case AWS services were down hence the traffic was not rerouted immediately.

We are planning to put more automation and health checks in place so that there is no impact in similar situation again.