NAT and multi-homed systems: iptables, ipnatctl, ip
David Ford
david@kalifornia.com
Mon, 11 Oct 1999 01:01:37 -0100
Help :)
Alexey, Paul, anyone!
here's the scoop. i have a growing number of systems that are doing this and my
kludges aren't going to last forever. i have multi-homed linux routers and need to
get the right IPs out the right port. i describe my home system. yes these
addresses are correct. may the great penguin leave rotten apples in your closet if
you abuse this information.
James: (linux router at NOC running 2.3.19+)
eth0: goes to public network 207.213.0.47
eth1: goes to an empty hub 207.213.15.129/207.213.15.130
Booterz: (linux router at home running 2.3.20)
eth0: local lan 207.213.14.0/25
eth1: cable modem d.h.c.p (normally 24.30.182.n)
eth2: DSL 216.32.34.187
gre1 connects the cable modem and 207.213.15.129
gre2 connects the DSL modem and 207.213.15.130
booterz default route is gre1+gre2. and on james the 207.213.14.0/25 is gre1/gre2.
routing tables: (arranged for clarity)
james:
# ip r s
207.213.0.32/27 dev eth0 proto kernel scope link src 207.213.0.47
207.213.14.122/31 dev gre1 proto kernel scope link src 207.213.14.122
207.213.14.124/31 dev gre2 proto kernel scope link src 207.213.14.124
207.213.14.0/25
nexthop dev gre1 weight 103
nexthop dev gre2 weight 102
207.213.15.0/25 dev eth0 proto kernel scope link src 207.213.15.99
207.213.15.176/29 via 207.213.15.129 dev eth1 scope link
207.213.15.224/28 via 207.213.15.129 dev eth1 scope link
207.213.15.192/27 via 207.213.15.129 dev eth1 scope link
default via 207.213.0.33 dev eth0
note the 207.213.15.xxx routes are unused at present as the machine for them is
dead.
booterz:
# ip r s
207.213.15.129 via 24.30.182.1 dev eth1 src 24.30.182.16
207.213.15.130 via 216.32.34.1 dev eth2 src 216.32.34.187
207.213.14.122/31 dev gre2 proto kernel scope link src 207.213.14.123
207.213.14.124/31 dev gre1 proto kernel scope link src 207.213.14.125
207.213.14.0/25 dev eth0 proto kernel scope link src 207.213.14.11
24.30.182.0/24 dev eth1 proto kernel scope link src 24.30.182.16
216.32.34.0/24 dev eth2 proto kernel scope link src 216.32.34.187
default
nexthop dev gre1 weight 103
nexthop dev gre2 weight 102
desires:
- masquerade outbound ftp/http traffic to the d.h.c.p address
- aggregate all other traffice over the two tunnels so that i achieve maximum
bandwidth but only lose the bandwidth of link X if link X goes down.
problems:
- first i notice that my maximum bandwidth is the maximum bandwidth of the DSL
which is about 25K/s. if i do traffic specifically over the cable modem, i achieve
~250K/s.
- there is a lot of arping for addresses with the _wrong_ ip on booterz.
example:
18:02:09.995523 arp who-has 216.32.34.1 tell 0.0.0.0
18:02:10.995522 arp who-has 216.32.34.1 tell 0.0.0.0
18:02:11.995523 arp who-has 216.32.34.1 tell 0.0.0.0
18:02:12.995524 arp who-has 216.32.34.1 tell 0.0.0.0
18:02:13.995525 arp who-has 216.32.34.1 tell 0.0.0.0
18:02:14.995522 arp who-has 216.32.34.1 tell 0.0.0.0
18:02:16.699315 arp who-has 216.32.34.1 tell 216.32.34.187
18:02:16.732892 arp reply 216.32.34.1 is-at 0:10:67:0:39:5
this sequence repeats every 30 seconds roughly. sometimes a random pick of
addresses gets into the arp request. i.e. strangely 207.213.14.12x happens along
now and then.
this is james:
# ip a s lo
1: lo: <LOOPBACK,UP> mtu 3924 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
link/ether 00:40:05:23:33:da brd ff:ff:ff:ff:ff:ff
inet 207.213.0.47/27 brd 207.213.0.63 scope global eth0
inet 207.213.15.99/25 brd 207.213.15.127 scope global eth0:1
inet 207.213.15.143/32 brd 207.213.15.143 scope global eth0:9
inet 207.213.15.139/32 brd 207.213.15.139 scope global eth0:27
inet 207.213.15.242/32 brd 207.213.15.242 scope global eth0:28
inet 207.213.0.45/27 brd 207.213.0.63 scope global secondary eth0:gnome
inet 207.213.15.82/25 brd 207.213.15.127 scope global secondary eth0:2
[...]
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 100
link/ether 00:40:05:40:f4:2f brd ff:ff:ff:ff:ff:ff
inet 207.213.15.129/32 brd 207.213.15.129 scope global eth1
inet 207.213.15.130/32 brd 207.213.15.130 scope global eth1
inet 207.213.15.200/32 brd 207.213.15.200 scope global eth1
[...]
8: gre1@NONE: <POINTOPOINT,NOARP,UP> mtu 1476 qdisc noqueue
link/gre 207.213.15.129 peer 24.30.182.16
inet 207.213.14.122/31 scope global gre1
9: gre2@NONE: <POINTOPOINT,NOARP,UP> mtu 1476 qdisc noqueue
link/gre 207.213.15.130 peer 216.32.34.187
inet 207.213.14.124/31 scope global gre2
this is booterz:
# ip a s
1: lo: <LOOPBACK,UP> mtu 3924 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
link/ether 00:a0:cc:54:e1:ee brd ff:ff:ff:ff:ff:ff
inet 207.213.14.11/25 brd 207.213.14.127 scope host eth0
inet 207.213.14.14/25 brd 207.213.14.127 scope host secondary eth0
inet 207.213.14.6/25 brd 207.213.14.127 scope host secondary eth0
inet 207.213.14.3/25 brd 207.213.14.127 scope host secondary eth0
3: eth1: <BROADCAST,MULTICAST,NOTRAILERS,UP> mtu 1500 qdisc pfifo_fast qlen 100
link/ether 00:80:c8:1d:39:2b brd ff:ff:ff:ff:ff:ff
inet 24.30.182.16/24 brd 24.30.182.255 scope global eth1
4: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
link/ether 00:80:c8:1d:37:fc brd ff:ff:ff:ff:ff:ff
inet 216.32.34.187/24 brd 216.32.34.255 scope host eth2
[...]
15: gre1@NONE: <POINTOPOINT,NOARP,UP> mtu 1476 qdisc noqueue
link/gre 24.30.182.16 peer 207.213.15.129
inet 207.213.14.125/31 scope global gre1
16: gre2@NONE: <POINTOPOINT,NOARP,UP> mtu 1476 qdisc noqueue
link/gre 216.32.34.187 peer 207.213.15.130
inet 207.213.14.123/31 scope global gre2
now i've been very careful playing with ipnatctl because the masquerading likes to
blow up the box. iptables itself (yes i know both it and 2.3 are dev) seems to
make the box unstable. 2.3.20 is much more stable than previous :)
now, down to work.
first, oh routing and network wizards, please point out bugs in how i did it and
please explain why it is bad or why another method is better. i need to solve the
arp issue, i believe it's having a small affect on packets going out that link. i
haven't the foggiest idea why it happens only on the dsl line. the script i use to
manage the network is at http://stuph.org/rc.networking and i'll try to keep it up
to date.
second, this one is for rusty, how the heck do i get ipnatctl to do this :) i
want to nat outbound ftp/http to the ip of eth1 and make sure it goes out eth1 and
not any other devices.
this brings me to my second setup which is much more simplified. these addresses
are faked :)
netA 200.200.200.0/24 -> routerA
\_____ eth0
[linux router] eth1 (internal lan, mixed 1:1 NAT)
/
netB 100.100.100.0/24 -> routerB
this one gets a wee bit silly.
most of the internal lan is 1:1 NAT to netA, however about 15% of it is 1:1 NAT to
netB. like this: __________--------_____--______________________
the mappings do not overlap.
the problem i have here is determining which interface packets go out on.
i need some ip rules that work, all the sparse examples i've found haven't enabled
me to setup the rule tables and routes so that the following occurs:
request comes in netA for an http session, session takes place over router A
*always*
request comes in netB for etc, takes place over router B *always*
right now, everything heads out to routerA because they're not filtering packets
:) unfortunately netB has filters in place to prevent packet exit if it's not from
that network.
thankyou very very much for comments and critiques. anything that gets me further
than i am now will be immensely appreciated :)
-d
--
This is Linux Country. On a quiet night, you can hear Windows NT reboot!
Do you remember how to -think- ? Do you remember how to experiment? Linux
__ is an operating system that brings back the fun and adventure in computing.
\/ for linux-kernel: please read linux/Documentation/* before posting problems