====== Network ======
The network is maintained by the [[mailto:rbg-noc@in.tum.de|RBG Network]] and [[mailto:rbg-system@in.tum.de|System Group]].\\
==== VLANs ====
We have five different VLANs/Networks.
^VLAN-Name:|il11|il11_2|il11_3|il11_4|il11_5|il11_6|il11_8|
^VLAN-ID:|vlan_83|vlan_132|vlan_133|vlan_235|vlan_240|vlan_241|vlan_253|
^Note:|chair|server mgmt (iDrac)|server intern|wlan (free)|server mwn|testbed back|testbed front|
^Netzwerk:|131.159.24.0/23|10.0.0.0/8|10.200.112.0/20|172.24.21.192/27|172.24.24.0/23|10.200.104.0/21|10.200.96.0/21|
^Gateway:|131.159.25.254|10.0.0.1|10.200.127.254|172.24.21.219(DHCP:..21.220)|172.24.25.254|10.200.111.254|10.200.103.254|
^HostMin:|131.159.24.1|10.0.0.1|10.200.112.1|172.24.21.193|172.24.24.1|10.200.104.1|10.200.96.1|
^HostMax:|131.159.25.253|10.255.255.254|10.200.127.249|172.24.21.222|172.24.25.253|10.200.111.249|10.200.103.254|
^Broadcast:|131.159.25.255|10.255.255.255|10.200.127.255|172.24.21.223|172.24.25.255|10.200.111.255|10.200.103.255|
^Netzmaske:|255.255.254.0 = 23|255.0.0.0 = 8|255.255.240.0 = 20|55.255.255.224 = 27|255.255.254.0 = 23| 255.255.248.0 = 21|255.255.248.0 = 21|
^Hosts/Netz:|510|4094|4094|30|510|2046|2046|
==== IP address ====
The addresses from this range can be directly assigned to the device. For some networks an internal RBG DHCP server can be used to automatically provide addresses. \\
** Chair IPs (il11)** \\
Chair addresses can be managed through the [[https://rbgwebapp.in.tum.de/struktur/login|RBG Webapp]]. The MAC address needs to be whitelisted for the DHCP server. In this way each IP address is coupled with a user (MAC address). \\
==== DHCP Configurations ====
=== il11_3 (server) ===
* 10.200.112.1 - 10.200.119.254 (0.200.112.0/21)
* -> no DHCP
* 10.200.120.1 - 10.200.127.254 (10.200.120.0/21)
* -> static DHCP
* -> create a new entry on StrukDB and IP address will be assigned to the Host by DHCP
=== il11_5 (mwn) ===
* 172.24.24.1 - 172.24.24.254 (172.24.24.0/24)
* -> static DHCP (StrukDB)
* 172.24.25.1 - 172.24.25.254 (172.24.25.0/24)
* -> no DHCP
=== il11_6 (testbed) ===
* 10.200.96.1 - 10.200.109.254
* -> no DHCP
* 10.200.110.1 - 10.200.111.254 (10.200.110.0/23)
* -> static DHCP
* -> assign IPs only with entry in StrukDB
==== VPN ====
The VPN access is granted by an entry in a LDAP group (vpnil11). Only employees should be added to the ldap group!! The Ldap group is managed through the RBGWebapp.\\
To give students access to chair services/vms/servers the firewall should be opened to allow connections from the LRZ VPN range to this particular service/vm.
==== Firewall ====
The chair firewall is managed by the network group. An E-Mail has to be sent with the rule that needs to be added. Each network has its own firewall (-> ac_il11, ac_il11_2, etc.). Usually the chair firewall is meant. The specific rules and setup can be found under [[:infrastructure:firewall]]\\
E.g. vmott4 (IP) <- Port 443 <- alle Netze, net01 (IP) <- Port 22 <- LRZ VPN, etc.\\
Before adding any new firewall rules read the instructions under [[:infrastructure:firewall]]!!
==== WLAN il11 ====
The network group does not like external access points. They interfere with the existing wifi infrastructure and the situation gets worse. We have one hidden chair network that is accessible with only a password from all access points that also broadcast the eduroam network in the FMI building. Two ports of the meeting room switch are also connected to this network. Visitors can plug in a Ethernet cable and should get immediate internet connection.
^VLAN:|il11_4|
''look at the table above for the rest of the information'' \\
This network has a gateway to control access to our chair network, this gateway is managed by us and can be accessed on vmott3. More information on how the gateway is set up can be found in the sections below.
===== Gateway il11_4 =====
This is the gateway controlling the access from the chair WIFI (il11_4) to the rest of the chair network. This gateway is vmott3.\\
**Setup**
* Uncomment ''net.ipv4.ip_forward=1'' in ''/etc/sysctl.conf''
* Enable network interface in ''/etc/network/interfaces'' auto ens192
auto ens192
iface ens192 inet dhcp
* iptable rules ''ens160'' = chair network, ''ens192'' = il11 network
#nat everything except traffic between chair and il11 (both ways)
iptables -t nat -A POSTROUTING ! -s 131.159.24.0/23 ! -d 131.159.24.0/23 -j SNAT --to-source 131.159.24.141
#allow traffic only from chair network to gateway (INPUT)
iptables -A INPUT -i ens160 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -i ens160 -s 131.159.24.0/23 -j ACCEPT
#allow special traffic to be forwarded (FORWARD)
iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
# allow influxDB and MonoDB from il11 to testbed02 (FORWARD)
iptables -A FORWARD -i ens192 -o ens160 -p tcp -s 172.24.21.192/27 -d 131.159.24.150 --sport 1024:65535 --dport 8086 -j ACCEPT
iptables -A FORWARD -i ens192 -o ens160 -p tcp -s 172.24.21.192/27 -d 131.159.24.150 --sport 1024:65535 --dport 27017 -j ACCEPT
# allow influx DB and Node.js from testbed02 to il11
iptables -A FORWARD -i ens160 -o ens192 -p tcp -s 131.159.24.150 -d 172.24.21.192/27 --dport 8086 -j ACCEPT
iptables -A FORWARD -i ens160 -o ens192 -p tcp -s 131.159.24.150 -d 172.24.21.192/27 --dport 8080 -j ACCEPT
# allow MYSQL from vmott1
iptables -A FORWARD -i ens160 -o ens192 -p tcp -s 131.159.24.38 -d 172.24.21.192/27 --dport 3306 -j ACCEPT
#set default policy to drop any traffic forwarded or addressed to the gateway machine
iptables -P INPUT DROP
iptables -P FORWARD DROP
* save iptables to a file for reboot iptables-save > /etc/firewall.conf add new file ''/etc/network/if-up.d/iptables'' with following content #!/bin/sh
iptables-restore /etc/firewall.conf make the file executable chmod +x /etc/network/if-up.d/iptables
* save changes on your firewall with ''iptables-save > /etc/firewall.conf''
====== Server Network ======
Servers are wired as instructed by the chair admin at installation time by the RBG-network group. There are usually multiple uplinks on one server:
* iDrac (mandatory): One Ethernet cable goes into the floor to a RBG router, the port is "switched" to the il11_2 network where all server iDracs are connected to. This network is only reachable from vmott2, a RBG VM with two interfaces where one is connected to il11_2 and the other one to the chair network.
* Chair (optional): One Ethernet cable goes into the floor to a RBG router, the port is "switched" to the il11 network which is our chair network. The Mac address of this port has to be declared in the RBG Webapp in order to get an IP address. After that the server is reachable from the chair network as any other machine.
* Brocade Router (optional): One Fiber cable goes to the Brocade Switch at the top of the right Rack. There are multiple options to use this connection. The RBG controls the Brocade Switch and can put any network/vlan on any port. Each port on the switch has one default/untagged vlan and can optionally have multiple additional tagged vlans.
UPDATE:
* Chair: One Ethernet cable goes into the floor to a RBG router, the port is "switched" to the il11_3 network, which is the server management network. The MAC address of this port (Interface 4 in iDrac) has to be declared in the RBG Webapp in order to get an IP address. After that the server is reachable from the chair network as any other machine.
===== Network hardware =====
The server room network consists of our own infrastructure and uplinks to the RBG network.
Most of the hardware is wired internally with fibreglass (10G/40G). Most of the servers have a direct link to the RBG switches (chair-network + internet) with a normal 1Gb Ethernet cable. Each server should have at least two interfaces. One interface for the iDrac network (cm-mgmt, Server Management) and one for the chair-network/server-network.
==== Transceiver, Cables, etc. ====
Most of our network cards and all the switches have SFP+ Ports (10G) and QSFP (40G) Ports.
The Port itself can be equipped with any kind of transceiver. The transceiver converts the electrical signal to the cable and on the other side back to the electrical signal. So the same port can be used for copper and also fiberglass. For each connection between server and switch two transceivers are needed and one cable in between.
=== SFP+ Transceiver ===
SFP+ Ports are for bandwidths up to 10G. If purchasing transceiver they should be chosen by compatibility between transceiver and server/switch. For the Intel X520 Network cards the E10GSFPSR Transceiver is compatible. With this number also OEM products can be chosen. They don't need to be directly from the original vendor (e.g. Intel) but Vendor certified (OEM) guarantees compatibility. We need SR transceiver for Short Lengths, there are also LR Transceiver which are used for lengths starting from a couple of kilometers. For the exact model and number you can alway ask the RBG-Network group [[mailto:rbg-noc@in.tum.de]].
===DAC - Direct Attach Cables===
There are cables that already provide two transceiver and one cable in the middle. For example the cables from the emu03-emu14 servers to the Dell Switch are all Dell Networking Direct Attach Copper Cables with 3m. They are basically transceiver and cable combined.