Attack On The OrderBox IP Subnet

Root Cause Analysis for the OrderBox Downtime on 29th November 2017

Timeline

Attack Start Time: 16:15 HRS GMT(21:45 HRS IST)

Attack End Time: 20:00 HRS GMT(Thursday 30th November 01:00 HRS IST)

Duration: 03 Hours and 45 Minutes

What happened?

1. At around 16:15 HRS GMT(21:45 HRS IST) we observed an attack vector of around 800 Mbps on our OrderBox IP 162.210.71.136
2. Neustar mitigation was immediately enabled, however there was no noticeable difference
3. Neustar then confirmed that attack traffic was hitting other IPs of the Subnet 162.210.71.0/24 as well
4. As a precautionary measure traffic was allowed through only ports 80 and 443 for the Subnet 162.210.71.0/24
5. Post this most services came back online, however our Domain Check Availability Service was still not functional
6. Upon further investigation our team found that the Domain Check Availability service was working on ports other than 80 and 443
7. Based on this this analysis the Network Team quickly decided to NULL Route the IP 162.210.71.136 on all our routers and allowed access for 162.210.71.0/24 on all ports
8. Post this all services resumed normal operations by 20:00 HRS GMT(Thursday 30th November 01:00 HRS IST)

Action items to avoid this in the future

1. Enforce OrderBox branding via CNAME only
2. Reconfigure inbound rules for public Subnets and allow only required ports
3. Reconfigure and fortify our Neustar mitigation template
4. Use a new Subnet for critical OrderBox services

The issue is resolved.

RCA will be put up shortly.

We are facing an attack on our OrderBox IP Subnet, due to which, you might face intermittent issues accessing your control panel or placing any actions via SuperSite or API.