Version 2 (modified by sayler, 10 years ago)

--

Notes on networking in Eucalyptus 1.3

When you create an instance, it has two interfaces available. These interfaces each have a unique and related MAC

machine 1:

eth0      Link encap:Ethernet  HWaddr AA:CC:00:00:00:67  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:21 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:12390 (12.0 KiB)  TX bytes:10620 (10.3 KiB)

eth1      Link encap:Ethernet  HWaddr AA:DD:00:00:00:67  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2299 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:254602 (248.6 KiB)  TX bytes:10620 (10.3 KiB)

machine 2:

eth0      Link encap:Ethernet  HWaddr AA:CC:00:00:00:68  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:30 errors:0 dropped:0 overruns:0 frame:0
          TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:17700 (17.2 KiB)  TX bytes:20060 (19.5 KiB)

eth1      Link encap:Ethernet  HWaddr AA:DD:00:00:00:68  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3563 errors:0 dropped:0 overruns:0 frame:0
          TX packets:30 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:349896 (341.6 KiB)  TX bytes:17700 (17.2 KiB)

Note that neither of these machines has received a DHCP lease yet.

Looking at the NC that hosts both of these machines, we see

eth0      Link encap:Ethernet  HWaddr 00:0d:60:d5:2b:74  
          inet addr:152.3.144.207  Bcast:152.3.145.255  Mask:255.255.254.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2358926 errors:0 dropped:0 overruns:0 frame:0
          TX packets:721155 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:821622203 (783.5 MiB)  TX bytes:193376648 (184.4 MiB)

eucabr3   Link encap:Ethernet  HWaddr 7a:67:18:df:d1:89  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:89 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:51264 (50.0 KiB)  TX bytes:0 (0.0 B)

eucadev3  Link encap:Ethernet  HWaddr 7a:67:18:df:d1:89  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:206 errors:0 dropped:0 overruns:0 frame:0
          TX packets:73 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:10918 (10.6 KiB)  TX bytes:43070 (42.0 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:19189 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19189 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1162580 (1.1 MiB)  TX bytes:1162580 (1.1 MiB)

peth0     Link encap:Ethernet  HWaddr 00:0d:60:d5:2b:74  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2499852 errors:0 dropped:0 overruns:0 frame:0
          TX packets:721230 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:874421674 (833.9 MiB)  TX bytes:196364681 (187.2 MiB)
          Interrupt:16 

vif3.0    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:36 errors:0 dropped:0 overruns:0 frame:0
          TX packets:37 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:20736 (20.2 KiB)  TX bytes:21830 (21.3 KiB)

vif3.1    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:36 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4364 errors:0 dropped:135 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:20736 (20.2 KiB)  TX bytes:436185 (425.9 KiB)

vif4.0    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:37 errors:0 dropped:0 overruns:0 frame:0
          TX packets:33 errors:0 dropped:1 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:21312 (20.8 KiB)  TX bytes:19470 (19.0 KiB)

vif4.1    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:33 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4140 errors:0 dropped:204 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:19008 (18.5 KiB)  TX bytes:395485 (386.2 KiB)

and looking at the bridges configured:

shirako037:~# brctl show
bridge name	bridge id		STP enabled	interfaces
eth0		8000.000d60d52b74	no		peth0
							vif3.1
							vif4.1
eucabr3		8000.7a6718dfd189	no		eucadev3
							vif3.0
							vif4.0

Further understanding of just how this is set up requires looking at how Eucalyptus sets up VDE (http://wiki.virtualsquare.org/index.php/VDE).

On each guest VM instance, eth0 is a "public" interface, and eth1 is a "private" interface.

The cloud controller runs DHCP for both interfaces in our setup. We answer requests on eth0 of shirako029 (connected to the cluster network) with 10.128.1.x "public" addresses. Theoretically, we answer requests on eucadev3 of shirako029 for the private interfaces' virtual bridged vlan. Packets don't seem to make it from the NCs to the CC.. so this doesn't work presently.