A small Nigerian Internet service provider (ISP) hijacked traffic meant for Google data centers on Monday, re-routing local traffic through China and Russia and making some hosted services temporarily unavailable for users.
The culprit of the activity that took down some of Google’s hosted services Monday is called MainOne Cable Company, which erroneously advertised that it was hosting Google IP addresses normally assigned to Google’s data-center network, leaking 212 Google prefixes to China Telecom. The event caused traffic to be redirected and or dropped, resulting in service outages.
BGPmon, a service that monitors Internet traffic routes through small ISPs, discovered the abnormal network activity Monday afternoon Pacific time and tweeted about it, publishing a list of the affected prefixes and describing the whole situation in a Twitter play-by-play.
“Appears that Nigerian ISP AS37282 ‘MainOne Cable Company’ leaked many @google prefixes to China Telecom, who then advertised it to AS20485 TRANSTELECOM (Russia),” the company tweeted Monday. “From there on others appear to have picked this up.”
See also: Russia blocks 800K Amazon sites in Telegram crackdown, disables legitimate ones in the process
The misidentification of the IP addresses caused a chain reaction of re-routing traffic intended for Google’s network to MainOne instead, resulting in Google customer disruptions and outages for about 74 minutes, BGPmon reported.
Denial of service for customers
ThousandEyes, an Internet network intelligence company, was one of the victims of the traffic hijacking, with staff experiencing denial of service for G Suite–Google’s cloud-based productivity and collaboration tools–and other Google services for more than an hour, Ameet Naik, a technical marketing management, said in a blog post.
“The outage not only affected G Suite, but also Google Search as well as Google Analytics,” he wrote. “What caught our attention was that traffic to Google was getting dropped at China Telecom. Why would traffic from a San Francisco office traversing to Google go all the way to China? We also noticed a Russian ISP in the traffic path, which definitely sparked some concerns.”
See also: China Using Big Brother-Like System to Track, Monitor Minorities
With China and Russia well-known for conducting state-sponsored cybercrime, ThousandEyes researchers decided to probe further to get to the root of the problem, Naik said.
“Our analysis indicates that the origin of this leak was the BGP peering relationship between MainOne, the Nigerian provider, and China Telecom,” he explained in the post. “MainOne has a peering relationship with Google via IXPN in Lagos and has direct routes to Google, which leaked into China Telecom.”
Naik also noted that the leak impacted mainly business-grade transit and not consumer ISP networks.
MainOne attributed the issue to “an error during a planned network upgrade due to a misconfiguration on our BGP filters,” the company tweeted. “The error was corrected within 74mins and processes put in place to avoid reoccurrence.”
BGP stands for “Border Gateway Protocol,” an inherently insecure protocol created in the early 1980s that allows anyone on the network to announce a bad BGP route and receive traffic that was not intended for them.
Security experts consider the protocol a high risk and–if hijacked–there are serious consequences, because an unauthorized network can intercept traffic not intended for it that can be analyzed and later decrypted for malicious use.
Accident–or accidentally on purpose?
Google acknowledged the situation as it was in progress on Monday, posting on its website that it received a report of an issue with Google Cloud Networking, with IP addresses hosted by the service “being erroneously advertised by Internet service providers other than Google.”
The company resolved the issue–about which it first posted at 2:26 p.m.–about a half-hour later, it said. However, it’s troubling that it even occurred, and there is precedent to wonder if the incident was indeed just an accident, Naik said.
“While we don’t know if this was a misconfiguration or a malicious act, these leaked routes propagated from China Telecom, via TransTelecom to NTT and other transit ISPs,” he said.
Indeed, China Telecom has been found in the past to engage in significant traffic hijacking by exploiting the BGP, something that was the topic of a recently published, controversial paper.
In the paper, researchers from the U.S. Naval War College and Tel Aviv University claimed that in 2016 and 2017, China Telecom deliberately steered numerous Internet traffic routes through China, including routes from Canada to Korean government sites, traffic from the United States to a bank headquarters in Milan, Italy, and traffic from Sweden and Norway to the Japanese network of a large American news organization.
Naik also pointed to a notable rise in BGP hijacking incidents, and warned companies to keep a close eye on the situation to “detect such incidents quickly in order to mitigate any service impacts to their business.”
