Separate Neutron Services onto Separate Nodes in OpenStack

OpenStack builds with good architecture for scalability. For example, if you want to separate neutron service from the controller node into a dedicated network node, this can be done on OpenStack.

Environment:

  • OpenStack 1 controller and 2 compute build with  packstack
  • Flat Network for external communication
  • eth0 for API and data communication
  • eth1 for external communication

With this following IP Configuration

NodeInterfaceIP Address
controllereth010.200.200.110
eth110.201.201.110
compute1eth010.200.200.111
eth110.201.201.110
compute2eth010.200.200.112
eth110.201.201.110
networketh010.200.200.113
eth110.201.201.110

On a new network node, Install neutron package

yum install -y openstack-neutron.noarch
yum install -y openstack-neutron-common.noarch
yum install -y openstack-neutron-openvswitch.noarch
yum install -y openstack-neutron-metering-agent.noarch
yum install -y openstack-selinux.noarch
yum install -y iptables-services

Add firewall rules in the controller node to allow a network node to access

iptables -I INPUT 1 -s 10.200.200.113/32 -p tcp -m multiport --dports 5671,5672 -m comment --comment "001 amqp incoming amqp_10.200.200.113" -j ACCEPT
iptables -I INPUT 1 -s 10.200.200.113/32 -p tcp -m multiport --dports 3306 -m comment --comment "001 mariadb incoming mariadb_10.200.200.113" -j ACCEPT
nano /etc/sysconfig/iptables
...
-A INPUT -s 10.200.200.113/32 -p tcp -m multiport --dports 5671,5672 -m comment --comment "001 amqp incoming amqp_10.200.200.113" -j ACCEPT
-A INPUT -s 10.200.200.113/32 -p tcp -m multiport --dports 3306 -m comment --comment "001 mariadb incoming mariadb_10.200.200.113" -j ACCEPT
...

Add firewall rules in the compute node to allow a network node to create the vxlan tunnel

iptables -I INPUT 1 -s 10.200.200.113/32 -p udp -m multiport --dports 4789 -m comment --comment "001 neutron tunnel port incoming neutron_tunnel_10.200.200.113" -j ACCEPT
nano /etc/sysconfig/iptables
...
-A INPUT -s 10.200.200.113/32 -p udp -m multiport --dports 4789 -m comment --comment "001 neutron tunnel port incoming neutron_tunnel_10.200.200.113" -j ACCEPT
...

Create firewall rules  in a network node

nano /etc/sysconfig/iptables
...
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p udp -m multiport --dports 67 -m comment --comment "001 neutron dhcp in incoming neutron_dhcp_in_10.200.200.113" -j ACCEPT
-A OUTPUT -p udp -m multiport --dports 68 -m comment --comment "001 neutron dhcp out outgoing neutron_dhcp_out_10.200.200.113" -j ACCEPT
-A INPUT -s 10.200.200.111/32 -p udp -m multiport --dports 4789 -m comment --comment "001 neutron tunnel port incoming neutron_tunnel_10.200.200.111_10.200.200.111" -j ACCEPT
-A INPUT -s 10.200.200.112/32 -p udp -m multiport --dports 4789 -m comment --comment "001 neutron tunnel port incoming neutron_tunnel_10.200.200.112_10.200.200.112" -j ACCEPT
-A INPUT -s 10.200.200.113/32 -p udp -m multiport --dports 4789 -m comment --comment "001 neutron tunnel port incoming neutron_tunnel_10.200.200.113_10.200.200.113" -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT
...
service iptables restart

Copy configuration from the controller to the network node

egrep -v ^'(#|$)' /etc/neutron/neutron.conf | ssh zu-os-neutron tee /etc/neutron/neutron.conf
egrep -v ^'(#|$)' /etc/neutron/plugins/ml2/openvswitch_agent.ini | ssh zu-os-neutron tee /etc/neutron/plugins/ml2/openvswitch_agent.ini
egrep -v ^'(#|$)' /etc/neutron/l3_agent.ini | ssh zu-os-neutron tee /etc/neutron/l3_agent.ini
egrep -v ^'(#|$)' /etc/neutron/dhcp_agent.ini | ssh zu-os-neutron tee /etc/neutron/dhcp_agent.ini
egrep -v ^'(#|$)' /etc/neutron/metadata_agent.ini | ssh zu-os-neutron tee /etc/neutron/metadata_agent.ini
egrep -v ^'(#|$)' /etc/neutron/metering_agent.ini | ssh zu-os-neutron tee /etc/neutron/metering_agent.ini

Edit openswitch_agent.ini in the network node and change the local_ip variable.

nano /etc/neutron/plugins/ml2/openvswitch_agent.ini
...
[ovs]
local_ip=10.200.200.113
...

Add br-ex interface in the network node

nano /etc/sysconfig/network-scripts/ifcfg-br-ex
...
PROXY_METHOD=none
BROWSER_ONLY=no
DEFROUTE=yes
UUID=ff4e4560-0112-47e8-893e-f6feeed80ed6
ONBOOT=yes
IPADDR=10.201.201.113
PREFIX=24
DEVICE=br-ex
NAME=br-ex
DEVICETYPE=ovs
OVSBOOTPROTO=none
TYPE=OVSBridge
OVS_EXTRA="set bridge br-ex fail_mode=standalone"
...

join eth1 into br-ex in the network node

nano /etc/sysconfig/network-scripts/ifcfg-eth1
...
DEVICE=eth1
NAME=eth1
DEVICETYPE=ovs
TYPE=OVSPort
OVS_BRIDGE=br-ex
ONBOOT=yes
BOOTPROTO=none
...
ifup br-ex
ifdown eth1
ifup eth1

Start network service in the network node

systemctl enable \
  neutron-openvswitch-agent.service neutron-dhcp-agent.service \
  neutron-metadata-agent.service neutron-metering-agent.service
systemctl start \
  neutron-openvswitch-agent.service neutron-dhcp-agent.service \
  neutron-metadata-agent.service neutron-metering-agent.service

systemctl enable neutron-l3-agent.service
systemctl start neutron-l3-agent.service

systemctl status \
  neutron-openvswitch-agent.service neutron-dhcp-agent.service \
  neutron-metadata-agent.service neutron-metering-agent.service neutron-l3-agent.service

Check network agent

openstack network agent list

In the step, you have 2 neutron service, 1 running on a controller and the other is running on the network node.

Migrate DHCP and Router

you can migrate the network, first check all the network, in my cases, I only have 1 external network and 1 internal network, and don’t forget to change all variables listed in this step.

openstack network list
NETWORK_INTERNAL=35630c3e-e642-4970-b334-36db776c831d
NETWORK_EXTERNAL=f93b0992-2e36-407c-a4ce-f4233926b918

check DHCP agent

openstack network agent list --agent-type dhcp
CONTROLLER_AGENT_ID=4b2b8d43-4621-438e-be3a-423952c15263
NETWORK_AGENT_ID=1d90da23-92a0-428e-bc34-975e30b51534

migrate DHCP agent from the controller into network node (you must do every network)

openstack network agent add network --dhcp $NETWORK_AGENT_ID $NETWORK_INTERNAL
openstack network agent add network --dhcp $NETWORK_AGENT_ID $NETWORK_EXTERNAL

openstack network agent remove network --dhcp $CONTROLLER_AGENT_ID $NETWORK_INTERNAL
openstack network agent remove network --dhcp $CONTROLLER_AGENT_ID $NETWORK_EXTERNAL

openstack network agent list --network $NETWORK_INTERNAL
openstack network agent list --network $NETWORK_EXTERNAL

check Router

openstack router list
ROUTER_ID=b89e771b-aa15-47f3-8ed7-f9409cdbf5e8

Check L3 agent

openstack network agent list --agent-type l3
R_CONTROLLER_AGENT_ID=27b445f4-bf4f-49fc-8728-2a8917461c9c
R_NETWORK_AGENT_ID=1db6bea6-eaa4-40c0-b350-86851589bb3e

Migrate Router

openstack network agent list --router $ROUTER_ID
openstack network agent remove router --l3 $R_CONTROLLER_AGENT_ID $ROUTER_ID
openstack network agent add router --l3 $R_NETWORK_AGENT_ID $ROUTER_ID

Check router and network, make sure they have already in network node

openstack network agent list --router $ROUTER_ID
openstack network agent list --network $NETWORK_INTERNAL
openstack network agent list --network $NETWORK_EXTERNAL

Your migration is done, if you want to stop neutron service in controller,

systemctl stop \
  neutron-openvswitch-agent.service neutron-dhcp-agent.service \
  neutron-metadata-agent.service neutron-metering-agent.service neutron-l3-agent.service
systemctl stop \
  neutron-openvswitch-agent.service neutron-dhcp-agent.service \
  neutron-metadata-agent.service neutron-metering-agent.service neutron-l3-agent.service
systemctl status neutron-server.service \
  neutron-openvswitch-agent.service neutron-dhcp-agent.service \
  neutron-metadata-agent.service neutron-metering-agent.service neutron-l3-agent.service

openstack network agent list

Comments are closed.