Subject: IP Aggregation HowTo
From: Alex Kemp
Date: Thursday, 4 February 2016 00:00:00 +0000
To: All

IP Aggregation Utility: /tools/aggregate.php
CIDR within ASN report: cidr-report.org

GPL Downloads: (the following files have been compressed using 7-Zip)
/files/aggregator-2.0.3b.7z (2.1 MB)
/files/aggregator-v2.0.3c.7z (2.1 MB)
/files/Conteg.include-0.13.13.7z (27.7 kB)
/files/Conteg.include-0.13.12.7z (27.7 kB)

IP Aggregation

Audience:
You are likely to be interested in this if you are a Webmaster/Webmistress, Network Engineer, etc.; essentially, any situation where you are engaged in network configuration / setup and using a tool that accepts CIDR notation. Typical examples of such tools are:

Toxic ASN:
In the same way that cities tend to have their Bad Neighbourhoods, the Internet tends to have toxic ASN. Thus, if you are a Network operator receiving a DOS/DDOS attack, it is possible that the source of the attacks may be concentrated within a few small CIDR. If a Web-Master, -Mistress, it is likely that most of the Forum spam or abuse comes from a handful of ASN. The trick is to:

  1. Identify the IPs/ASN
  2. Obtain an aggregated list of CIDR
  3. Put those CIDR into the Firewall

The ip aggregator can NOT help with the first (look here for that), but is one of the very few utilities that can do the second. In brief, the Aggregator:

  1. Accepts an ASN or free Text:
  2. Has 6 output options:
    1. CIDR (default): report as IPv4/Prefix (eg 192.168.37.192/27)
    2. Mask: report as IPv4/Mask (eg 192.168.37.215/255.255.255.224)
    3. Range: report as IPv4 - IPv4 (eg 192.168.37.192 - 192.168.37.223)
    4. Bad-Neighbours: report as iptables script for use in a Bad-Neighbours section
    5. .htaccess: report as script for use in a .htaccess config file
    6. ZB-Block: report as script for use in a ZB-Block config file

What is it?:
‘IP aggregation’ is a method of representing a heterogeneous mixture of IP Addresses within the smallest possible space.

A practical example now follows to try to make this point in the simplest & quickest way.

.htaccess Blacklisting example
The ip aggregator accepts free text containing any mixture of single ips, ranges, ip/mask and/or cidr. It currently has 6 output options, one of which is ‘HTA’. Pressing the Submit button with that output option selected could give the following example output: 0 Ranges, 0 Net/Mask + 4,294,967,296 IPs and/or CIDR supplied 1 aggregated .htaccess Deny lines from these IPs shown below: # Blacklisting # BE MOST CAREFUL! Can deny yourself, important IPs, or everyone # See http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html # # 2016-02-04 updated # Deny listing from The Aggregator # http://etmg.altervista.org/tools/aggregate.php Order allow,deny Allow from all Deny from 0.0.0.0/0 (and yes, if you suspect that that example is tainted then you are right; however, it does make the point that the 4 billion IPs of the IPv4 address space can be represented in a much more compact manner than by listing them one-by-one)

Classless Inter-Domain Routing

Read Wikipedia to gain a more thorough explanation of CIDR; what follows will be the briefest run-through, and is aligned to practical methods in using the aggregator. To understand CIDR we need to first consider the modern organisation of the Internet, which immediately leads to ASN, which itself leads to CIDR & route aggregation.

Internet & ASN:
The “Internet” is a network of Autonomous Systems, each of which is uniquely identified by it’s
ASN (“Autonomous System Number”). Essentially, an ASN is an ISP, although you will not be surprised to learn that the detail is far more complicated than that.

Strictly, it is AS=‘Autonomous System’ + ASN=‘Autonomous System Number’. This page will tend to use ‘ASN’ for either.

To try to help, here are some well-known ASN (and note that the aggregator allows you to enter an ASN directly if you wish to obtain all IPs currently administered by a specific ASN):

Network OperatorASN + AS-Name
BBC:AS2818 BBC
Google, Inc.:AS15169 GOOGLE
Wikipedia:AS43821 WIKIMEDIA-EU

The basic idea behind the entire setup is that each ASN is personally responsible for all network setup within their own network, whilst the Border Gateway Protocol (BGP) handles all communication between ASN (BGP is often also used by ASN within their network, though let’s just pretend for the moment that it is only used ASN to ASN).

CIDR was introduced together with version-4 of BGP in 1994; route aggregation became an issue at the same moment.

HowTo find ASN:
Before taking this further, let's show how to find an ASN.

Using ‘whois’ at a command-prompt is probably the simplest method; look for ‘AS’ or ‘OriginAS’ in the results. However, the ASN is not always included in the output from default whois, so you may need to use one of the specialist whois:

Try whois.cymru.com (see also team-cymru.org/IP-ASN-mapping.html) or asn.shadowserver.org: $ whois -h whois.cymru.com "8.8.8.8" AS | IP | AS Name 15169 | 8.8.8.8 | GOOGLE - Google Inc.,US $ whois -h asn.shadowserver.org "origin 8.8.8.8" 15169 | 8.8.8.0/24 | GOOGLE | US | google.com | Google Inc.

Recap + expand:
‘Internet’ == ‘network of ASN’
Each ASN has a unique AS number
(originally that number was 16-bit, and thus ranged from 1→65534 (first + last are reserved))
(now that number is 32-bit, and thus ranges from 1→4,294,967,295 (first + last are still reserved))
The Internet Assigned Numbers Authority (IANA) was allocated all ASN at inception
(IANA is a department of ICANN, itself established in 1998 by the USA government)
IANA assigns blocks of ASN to each Regional Internet Registry (RIR)
(current IANA assignments of ASN to the 5 RIR are shown here)
The RIR assign ASN to network operators

Thus, the chain of authority is: ICANN → IANA → RIR → network operator

BGP is the protocol that governs routing between ASN
The current BGP is v4, introduced 1994 (earlier versions now obsolete)
The major feature of BGPv4 is the use of CIDR within routing tables + aggregation to decrease table size

1999: >  5,000 ASN
2008: > 30,000 ASN
2010: > 35,000 ASN
2012: > 42,000 ASN
2014: > 47,000 ASN
(routing table growth is exponential with linear ASN number growth, hence the stress upon route aggregation)

ASN & IP Addresses:
Whilst the focus for ASN at the Internet-level is on BGP routing tables, the focus for ASN customers is more likely to be on individual IP Addresses.

RIR allocate the unique number which identifies the ASN that carries that number. The same RIR also allocates blocks of IP Addresses (yet more unique numbers) to the ASN. Each ASN use their IPs to setup their own Network Operation Centre (NoC), and ASN customers are delegated – directly or indirectly, temporarily or permanently – one or more of those IPs to give them access via the ASN to the Internet.

This process of ICANN → IANA → RIR → network operator → end user is one of continuous CIDR fragmentation, which can lead to extreme stress upon routing table size. The NoC operators do their best to try to keep the CIDR fragmentation under control.

---------
Alex Kemp