Update - DDOS Update:

Last night about 7:06PM CST and 7:21PM CST 2 DDOS attacks were mitigated in under 2 minutes.

We have received an update that our Datacenter has implement an additional layer of Defense with a scrubbing service from one of the upstream providers. While this does not stop the attack entirely it does make disruption of the network limited as seen with the 2 events last night.

Some important information:

What happened:
This DDOS is targeted at a colocation client located in the Chicago datacenter unrelated to Adeptcloud or our customers. The attack saturated bandwidth on blended routes causing the network disruptions you all have seen over the last few days.

Will it happen again?
While the possibility of more network disruptions today does exist, the scrubbing service employed should make the impact minimal.

When will it end?
DDOS is not something that has a time frame unfortunately, the bad actors at some point will give up the battle once the attacks are proven to not cause impact. With that said we wont have an update saying the attack is over as another one could be started at any moment.

What is being done about future threats?
We do have more calls scheduled today with DC NetOps and Engineering and will provide updates as data comes in. Adeptcloud does have a plan in action to mitigate future impact and distributing workloads, we are working with carriers and datacenters to prevent future impacts and to build defenses against any further disruptions. We are continuously monitoring the network and We will provide updates as they become available.
Jun 18, 10:33 CDT

Update - We have received official confirmation that Telia has been removed from our blended bandwidth and none of our traffic is currently utilizing Telia as a carrier at this time. All circuits have been increased to 100GB (UDP rate limits) by the provider and the datacenter is in the process of implementing this change on specific ports to help mitigate future attacks. These circuit upgrades are being implemented with providers such as Cogent, Comcast, Telia, CenturyLink and others. Any malicious traffic is still being routed to the Telia carrier at this time. Given that Telia is no longer being utilized by us for any inbound or outbound traffic, we're not expecting to see any issues with latency at this time. The last attack/partial disruption that occurred today was due to the attackers changing their plan of attack and attacking random IP subnets (in a wide-spread manner) in an attempt to disrupt network connectivity. They were partially successful with this attack as multiple customers, including Adeptcloud, experienced temporary loss of network connectivity until the attack was mitigated by the provider.

The DDOS attack was occurring via UDP and the estimated size (bandwidth) of the attack varied between 100Gbps to 150Gbps, however, we're still working to determine whether the 100-150Gbps numbers were calculated from malicious traffic as a whole or from individual attacks on random IP's. We're sure more details will be forthcoming once the situation has calmed down and NOC staff at the datacenter has a chance to put together a more concise report.

Network connectivity and traffic appears to have stabilized at this time and we are continuing to monitor inbound and outbound connectivity for any future issues.
Jun 17, 11:58 CDT

Update - Additional attacks have been identified and mitigated this late morning by the datacenter provider. We're still working with the provider to ensure none of our inbound and outbound traffic utilizes Telia at this time while these attacks continue to come in. As mentioned in our earlier update, the malicious traffic is being routed to Telia by the provider so separating our network away from this carrier will help tremendously in ensuring our network is not affected by these incoming attacks. We're continuing to monitor this situation and are working on gathering more technical details to put out regarding the scope of this attack, the average bandwidth associated with it and the type of attack. At the time of this update, the network appears to be stable, however, we cannot guarantee that this issue has been completely resolved at this time. We apologize for any inconvenience caused thus far and are working diligently on resolving this issue.
Jun 17, 11:00 CDT

Update - Current status of DDoS:

Our Datacenter has reported that a client of theirs was and continues to be a target of a DDOS attack. At this time, traffic destined to this customer has been isolated to one carrier (Telia) in order to separate other customer networks and to handle the scrubbing of malicious traffic. This change effectively takes Telia out of our blended bandwidth in order to reduce the impact of this attack on our networks.

We are continuing to monitor this situation and are in constant communication with our NetOps and datacenter teams to determine how this can be prevented more effectively in the future.

We will provide an update as soon as more information is known.
Jun 17, 09:05 CDT

Update - We have received notice that another attack was just mitigated by the datacenter. We saw a brief (three minute) period in which latency spiked and some amount of traffic was not getting through until the attack was mitigated. At the time of writing this update, ping times are back to normal and connections seem stable. We're still awaiting confirmations of this issue being completely resolved and are working with the datacenter provider on a more technical update in regards to these issues.
Jun 16, 12:48 CDT

Update - Our datacenter provider has informed us that they are currently mitigating a large DDoS attack that is impacting multiple customers in the Chicago datacenter. They have mitigated some of these attacks but new ones continue to come in causing some networking related stability issues. The network engineers are still working on mitigating these new attacks that are coming in. Given that this is impacting several customers, cabinets and switches, the attacks must be occurring on some of the core routing/switching infrastructure and are not targeting us or any of our customers specifically. We'll continue to post updates as this situation develops.
Jun 15, 16:22 CDT

Monitoring - Networking appears to have stabilized at this point and we have not seen networking related issues occur in the past 20 minutes. No network disruptions have been reported. We are still investigating the situation with our Datacenter. In the past hour, networking was stable for 10 minutes and would go down for 2-3 minutes. Currently we have no confirmation that network issues are fully resolved. We will provide an update as soon as possible based on the information that we have.
Jun 15, 15:26 CDT

Identified - We can confirm current network stability is impacted datacenter networking is looking into the issue.
Jun 15, 14:25 CDT

Investigating - We are receiving a few reports of network connectivity issues happening in CHI3 Datacenter. We are currently investigating the issue.
Jun 15, 13:59 CDT