Spammers have been working methodically to hijack large chunks of Internet real estate by exploiting a technical and bureaucratic loophole in the way that various regions of the globe keep track of the world’s Internet address ranges.
Last week, KrebsOnSecurity featured an in-depth piece about a well-known junk email artist who acknowledged sending from two Bulgarian hosting providers. These two providers had commandeered tens of thousands of Internet addresses from ISPs around the globe, including Brazil, China, India, Japan, Mexico, South Africa, Taiwan and Vietnam.
For example, a closer look at the Internet addresses hijacked by one of the Bulgarian providers — aptly named “Mega-Spred” with an email contact of “abuse@grimhosting” — shows that this provider has been slowly gobbling up far-flung IP address ranges since late August 2014.
This table, with data from the RIPE NCC — of the regional Internet Registries, shows IP address hijacking activity by Bulgarian host Mega-Spred.
According to several security and anti-spam experts who’ve been following this activity, Mega-Spred and the other hosting provider in question (known as Kandi EOOD) have been taking advantage of an administrative weakness in the way that some countries and regions of the world keep tabs on the IP address ranges assigned to various hosting providers and ISPs. Neither Kandi nor Mega-Spred responded to requests for comment.
IP address hijacking is hardly a new phenomenon. Spammers sometimes hijack Internet address ranges that go unused for periods of time. Dormant or “unannounced” address ranges are ripe for abuse partly because of the way the global routing system works: Miscreants can “announce” to the rest of the Internet that their hosting facilities are the authorized location for given Internet addresses. If nothing or nobody objects to the change, the Internet address ranges fall into the hands of the hijacker.
Experts say the hijackers also are exploiting a fundamental problem with record-keeping activities of RIPE NCC, the regional Internet registry (RIR) that oversees the allocation and registration of IP addresses for Europe, the Middle East and parts of Central Asia. RIPE is one of several RIRs, including ARIN (which handles mostly North American IP space) and APNIC (Asia Pacific), LACNIC (Latin America) and AFRINIC (Africa).
Ron Guilmette, an anti-spam crusader who is active in numerous Internet governance communities, said the problem is that a network owner in RIPE’s region can hijack Internet addresses that belong to network owners in regions managed by other RIRs, and if the hijackers then claim to RIPE that they’re the rightful owners of those hijacked IP ranges, RIPE will simply accept that claim without verifying or authenticating it.
Worse yet, Guilmette and others say, those bogus entries — once accepted by RIPE — get exported to other databases that are used to check the validity of global IP address routing tables, meaning that parties all over the Internet who are checking the validity of a route may be doing so against bogus information created by the hijacker himself.
“RIPE is now acutely aware of what is going on, and what has been going on, with the blatantly crooked activities of this rogue provider,” Guilmette said. “However, due to the exceptionally clever way that the proprietors of Mega-Spred have performed their hijacks, the people at RIPE still can’t even agree on how to even undo this mess, let alone how to prevent it from happening again in the future.”
And here is where the story perhaps unavoidably warps into Geek Factor 5. For its part, RIPE said in an emailed statement to KrebsOnSecurity that the RIPE NCC has no knowledge of the agreements made between network operators or with address space holders.
“It’s important to note the distinction between an Internet Number Registry (INR) and an Internet Routing Registry (IRR). The RIPE Database (and many of the other RIR databases) combine these separate functionalities. An INR records who holds which Internet number resources, and the sub-allocations and assignments they have made to End Users.
On the other hand, an IRRcontains route and other objects — which detail a network’s policies regarding who it will peer with, along with the Internet number resources reachable through a specific ASN/network. There are 34 separate IRRs globally — therefore, this isn’t something that happens at the RIR level, but rather at the Internet Routing Registry level.”
“It is not possible therefore for the RIRs to verify the routing information entered into Internet Routing Registries or monitor the accuracy of the route objects,” the organization concluded.
Guilmette said RIPE’s response seems crafted to draw attention away from RIPE’s central role in this mess.
“That it is somewhat disingenuous, I think for this RIPE representative to wave this whole mess off as a problem with the
IRRs when in this specific case, the IRR that first accepted and then promulgated these bogus routing validation records was RIPE,” he said.
RIPE notes that network owners can reduce the occurrence of IP address hijacking by taking advantage of Resource Certification (RPKI), a free service to RIPE members and non-members that allows network operators to request a digital certificate listing the Internet number resources they hold. This allows other network operators to verify that routing information contained in this system is published by the legitimate holder of the resources. In addition, the system enables the holder to receive notifications when a routing prefix is hijacked, RIPE said.
While RPKI (and other solutions to this project, such as DNSSEC) have been around for years, obviously not all network providers currently deploy these security methods. Erik Bais, a director at A2B Internet BV — a Dutch ISP — said while broader adoption of solutions like RPKI would certainly help in the long run, one short-term fix is for RIPE to block its Internet providers from claiming routes in address ranges managed by other RIRs.
“This is a quick fix, but it will break things in the future for legitimate usage,” Bais said.
According to RIPE, this very issue was discussed at length at the recent RIPE 69 Meeting in London last week.
“The RIPE NCC is now working with the RIPE community to investigate ways of making such improvements,” RIPE said in a statement.
This is a complex problem to be sure, but I think this story is a great reminder of two qualities about Internet security in general that are fairly static (for better or worse): First, much of the Internet works thanks to the efforts of a relatively small group of people who work very hard to balance openness and ease-of-use with security and stability concerns. Second, global Internet address routing issues are extraordinarily complex — not just in technical terms but also because they also require coordination and consensus between and among multiple stakeholders with sometimes radically different geographic and cultural perspectives. Unfortunately, complexity is the enemy of security, and spammers and other ne’er-do-wells understand and exploit this gap as often as possible.
Wow, pretty crazy and a little scary. I’m a network guy, but this one is a little over my head.
Is there ever a valid reason for someone to advertise networks in this cross registry fashion in terms of originating routes? If so doesn’t that break any ability to draw internet boundaries around national origin which seems bad for security particularly in war time?
Is there something the ISPs are failing to do in addition to the failing in the RIPE world? I like to think ISPs are the last line of defense in ensuring that their prefix lists allow customers only to originate and advertise networks for which they have ownership. Perhaps they could check the proper registry for the address space in addition to RIPE to add a layer of validation.
The problem is when bad guys set up fraud-based ISPs. Then the bad guys are in charge of vetting route additions.
In the US, at least for all the small ISP’s I’ve ever worked for, we always had to coordinate with our upstream ISPs and tell them any new netblock additions (by us, the ISP, or by our downstream customers).
Sound like this isn’t the case elsewhere.
(My connection is being eaten by a Grue.)
> Is there ever a valid reason for someone to advertise networks in this cross registry fashion in terms of originating routes?
probably
> If so doesn’t that break any ability to draw internet boundaries around national origin which seems bad for security …
possibly
> particularly in war time?
irrelevant
http://www.wired.com/2008/12/mediterranean-c/
http://techland.time.com/2011/04/07/how-an-elderly-woman-cut-off-the-internet-in-two-countries/
http://www.bbc.com/news/world-middle-east-21963100
In times of war…
http://www.history.com/this-day-in-history/griersons-raid-cuts-telegraph-wires-near-macon
Lines of communication get cut. That’s something that you just do. Forget this polite “lying announcements that everyone can here”.
Consider something simpler:
http://www.policeone.com/standoff/articles/1247470-Hostage-negotiations-Psychological-strategies-for-resolving-crises/
One of the things you see in every movie about robbing a bank and taking hostages is that the police cut off the phone lines to the bank, and leave only a line between the bank and the police negotiator.
At the end of the day, the Internet isn’t special here. There are lines, and in a War, or a Battle w/ a bank robber, the guy surrounding the little guy is going to cut off outside connections and control the flow of all information.
Now, you could say “hey, I can use Wireless”.
http://en.wikipedia.org/wiki/Radio_jamming#World_War_II
(My connection is being eaten by a Grue.)
> Is there ever a valid reason for someone to advertise networks in this cross registry fashion in terms of originating routes?
probably
> If so doesn’t that break any ability to draw internet boundaries around national origin which seems bad for security …
possibly
Say you’re HP, and you want to set up a datacenter in Africa/Asia/Europe/South America.
You want to set up 2000 publicly addressable computers w/ distinct host names running https to support wXP / IE6 (SNI isn’t available).
You go to the local ISP and ask for the addresses.
Your ISP apologizes, they just don’t have that many IP addresses to give away, they point you to:
http://en.wikipedia.org/wiki/IPv4_address_exhaustion#Notable_exhaustion_advisories
http://www.lacnic.net/en/web/lacnic/agotamiento-ipv4
> During this phase, only assignments from the equivalent of a /24 to a /22 will be made. Requesting organizations may request additional resources every 6 months. This process will be implemented every day until the /11 reserved for gradual exhaustion is finally exhausted.
That’s ~255 addresses every 6 months.
You’d need 4 years to get your addresses. And that’s assuming things don’t slow down as resources get scarce (which they will).
But, you’re HP, you ask around internally and discover:
15.0.0.0/8 Hewlett-Packard Company
16.0.0.0/8 Digital Equipment Corporation
You own 1/128th of the addressable IPv4 Internet, that’s ~33 million addresses.
Most of those aren’t in use (unless you’ve managed to assign one to each light bulb and ethernet jack in every location you occupy).
You could easily assign 16.200.0.0/21:
16.200.0.1 – 16.200.7.254
that gives you your 2000 ip addresses.
A single /21 is much better than a 8 /24s in terms of memory overhead for routers’ routing tables.
Thankfully, most routers that care about routing tables already crashed a few months ago:
https://blogs.cisco.com/sp/global-internet-routing-table-reaches-512k-milestone/
So 1 route, or 8, it isn’t a big deal, but if you offer a single one, you’ll make those routers slightly less annoyed than if they need to waste space for 8.
If you needed to grow your allotment from /21, to /17, it’s no big deal. You have plenty of space, as long as you don’t actively give out /28s alternating between continents.
Note: HP is fairly special, but Apple, MIT and a dozen others also have huge reserves that they could use if they wanted to in a similar fashion. And it would make sense.
Heck, it’d make sense for Amazon to buy the 16.0.0.0/8 from HP.
Thanks Timeless! Informative and fascinating. Sort of proves Brian’s point that this stuff is complicated enough to make good cover for clever, unscrupulous people. The fact that the internet works at all astounds me.
Never a dull convoluted moment in internet security eh! Brian.
“…Is there ever a valid reason for someone to advertise networks in this cross registry fashion in terms of originating routes?…”
Unfortunately the short answer is that companies buy/sell IPv4 space all the time. IP addressing isn’t contiguous along national or even regional boundaries – so a network being advertised out of Silicon valley may be numerically adjacent to one being advertised out of Dublin… To make matters worse, there is an acute shortage of IPv4 space, which means that as prices go up, and demand continues to increase, these networks get whittled into smaller and smaller pieces, and get bought/sold left and right, only further shuffling the deck.
I’ve got a vague solution: Allow another regional Internet registry, such as ARIN, or another “good” registry to take over the duties of RIPE since it can’t figure out how to maintain it’s area of responsibility.
“Allow another regional Internet registry, such as ARIN, or another “good” registry to take over the duties of RIPE since it can’t figure out how to maintain it’s area of responsibility.”
That’s ironic. RIPE is one of the few RIR based IRRs that follow the security model for IRR data, FOR THE ADDRESSES IN THEIR REGION. They don’t know anything about who is authorized for addresses or ASNs from other regions, so there’s nothing they can do. ARIN is one of the RIRs that do not follow the security model for IRR data, even for addresses they hand out. Ironic.
Read this article to understand why the Internet Routing Registry (IRR) operated by the RIPE NCC is operated is the way it is: https://labs.ripe.net/Members/denis/using-the-ripe-database-as-an-internet-routing-registry
There’s a related spam operation on the Kandi EOOD network: IP space 77.95.234.0/23 is being used for spam runs using registrars: Dynadot (currently registrant owner Eva Group).
The same spam operation is running (or has) currently (moving from network to networ) on the following networks:
NephoScale Inc-Evernet Hosting (AS46717 206.206.190.0/24 and 64.58.114.0/24)
Oso Grande Technologies, Inc (198.59.176.0/21 and 198.59.128.0/18 both on AS2901 and AS26640 198.59.178.0/24; with one path going thru NephoScale Inc-Evernet Hosting)
B2 Net Solutions Inc . (AS55286 network 138.128.0.0/20 U.S.)
The spam operation is also running registrant owner MARQUETTE & SMITH (registrar Enom)
The MARQUETTE & SMITH spam operation has been running moving from network to network, moving in the last few days from SC-HOSTINGBIT (AS57773 network 5.83.32.0/21 Romania) to B2 Net Solutions Inc.
I’ve seen this before with spam where a odd I.P. number in a specific net block will be associated with a Internet Provider in the United States. You know the Internet Provider in the United States doesn’t own the I.P. numbers in that net-block because the network WHOIS information is listed to a person or company in a third world country.
When we all get old & grey, and they finally ditch IPv4 for IPv6, will it have the same weakness?
http://www.ripe.net/ripe/docs/ripe-552
4.1. Address space not to be considered property
It is contrary to the goals of this document and is not in the interests of the Internet community as a whole for address space to be considered freehold property.
The policies in this document are based upon the understanding that globally unique IPv6 unicast address space is licensed for use rather than owned. Specifically, IP addresses will be allocated and assigned on a license basis, with licenses subject to renewal on a periodic basis. The granting of a license is subject to specific conditions applied at the start or renewal of the license.
RIRs will generally renew licenses automatically, provided requesting organisations are making a “good faith” effort at meeting the criteria under which they qualified for or were granted an allocation or assignment. However, in those cases where a requesting organisation is not using the address space as intended, or is showing bad faith in following through on the associated obligation, RIRs reserve the right to not renew the license. Note that when a license is renewed, the new license will be evaluated under and governed by the applicable IPv6 address policies in place at the time of renewal, which may differ from the policy in place at the time of the original allocation or assignment.
And yet, major companies (ahem, Microsoft) buy and sell IP space all the time — just like property.
http://www.networkworld.com/article/2228854/microsoft-subnet/microsoft-pays-nortel–7-5-million-for-ipv4-addresses.html
@timeless, the document that you referenced is an out-dated ipv6 allocation and assigment document .. In order to see what is currently active, the most recent version for IPv4 is : http://www.ripe.net/ripe/docs/ripe-623
To answer your question better, there are so-called pre-rir space or Legacy blocks. They have been provided to companies directly by IANA before the 1994 timeframe. Those are assets of those companies and can be transferred or sold.
IP space provided by RIPE is owned by RIPE. IP space is provided based on your membership, however if you cease to be a member, your resources will be deregistered. Current policy allows transfers of that right of use IP space ( IPv4 PA) space. The last flavour of IP space is the direct assignments or Provider independent (PI space) .. And that policy is already approved but this in progress of implementation. http://www.ripe.net/ripe/policies/proposals/2014-02
I see the new 6.4 section.
Question to the audience: At what time will the hijacked space in Mega-Spred and Kandi EOOD be taken?
Last I checked, the AS201640 was not announcing anything anymore ..
You can also see it at the RIPE stat website https://stat.ripe.net/AS201640#tabId=at-a-glance
” It was last seen on 2014-11-11 08:00:00 UTC”
@Brian, Chris asked if the problem would happen with IPv6. My reference was to a (draft) document that stipulated that IPv6 addresses (unlike IPv4 addresses) are not property. I see the stipulation as a response to the flaw in the IPv4 allocation design.
I’m not saying that companies don’t trade in IPv4 addresses. Just hoping that the future (IPv6) will be better.
There is a general problem with IRR data quality and it should be improved upon. However, the RIPE NCC runs by far the best and most capable database out there. Most of the other RIRs do not provide any protection for IRR data for their own in-region resources, let alone out-of-region resources. So much of the content and comment picking out RIPE for special treatment is ill-informed.
On the bright side, this article sparked a proposal to easily filter out authoritative data from non authoritative data:
http://www.ripe.net/ripe/mail/archives/db-wg/2014-November/004421.html
The only thing you need to hijack address blocks on the Internet is a complicit transit provider: nothing more, nothing less. In most situations, registering bogus objects in an IRR database won’t make much of a difference.
Most transit providers don’t use the IRR databases for bgp edge filtering, but instead maintain their own internal lists of address blocks for each customer. For those who use the public IRR databases, there are plenty of IRR DBs where routing information can be registered. The RIPE NCC operates only one out of around 35 worldwide and to its credit, is one of only a handful with any level of protection against spoofing for subsets of its data.
The real problem with hijacking lies with the transit providers who don’t do any edge filtering at all, who will accept any prefix that they receive from a their downstream customers and who don’t take action against customers who abuse this. If there’s righteous anger to be dished out, it should be directed at these organisations.