Setup Docker Hypervisor on Multi Node DVR Cluster RDO Mitaka

UPDATE 04/03/2016
  In meantime  use  repositories for RC1, rather then Delorean trunks
END UPDATE

DVR && Nova-Docker Driver (stable/mitaka) tested fine on RDO Mitaka build 20160329) with no issues described in previous notice for RDO Liberty
So, create DVR deployment with Contrpoller/Network + N(*)Compute Nodes. Switch to Docker Hypervisor on each Compute Node and make requiered updates to glance and filters file on Controller. You are all set. Nova-Dockers instances FIP(s) are available from outside via Neutron Distributed Router (DNAT) using "fg" interface ( fip-namespace ) residing on same host as Docker Hypervisor. South-North traffic is not related with VXLAN tunneling on DVR systems.

Why DVR come into concern ?

  Refreshing in memory similar problem with Nova-Docker Driver (Kilo)
with which I had same kind of problems (VXLAN connection Controller <==> Compute) on F22 (OVS 2.4.0) when the same driver worked fine on CentOS 7.1 (OVS 2.3.1). I just guess that Nova-Docker driver has a problem with OVS 2.4.0  no matter of stable/kilo, stable/liberty, stable/mitaka branches been checked out for driver build.

I have to notice that issue is related specifically with ML2&OVS&VXLAN setup, RDO Mitaka deployment ML2&OVS&VLAN  works with Nova-Docker (stable/mitaka) with no problems. However, VXLAN tunneling is pretty much common on RDO deployments.

Thus as quick and efficient workaround I suggest DVR deployment setup,
to kill two birds with one stone. It will result South-North traffic
to be forwarded right away from host running Docker Hypervisor to Internet
and vice/versa due to basic "fg" functionality (outgoing interface of
fip-namespace,residing on Compute node having L3 agent running in "dvr"
agent_mode).

**************************
Procedure in details
**************************
First install repositories for RDO Mitaka (the most recent build passed CI):-

# yum -y install yum-plugin-priorities
# cd /etc/yum.repos.d
# curl -O https://trunk.rdoproject.org/centos7-mitaka/delorean-deps.repo
# curl -O https://trunk.rdoproject.org/centos7-mitaka/current-passed-ci/delorean.repo
# yum -y install openstack-packstack (Controller only)

Now proceed as follows :-

1. Here is   Answer file to deploy pre DVR Cluster
2. Convert cluster to DVR as advised in  "RDO Liberty DVR Neutron workflow on CentOS 7.2"  :-
http://dbaxps.blogspot.com/2015/10/rdo-liberty-rc-dvr-deployment.html

Just one notice on RDO Mitaka on each compute node, first create br-ex and add port eth0

# ovs-vsctl add-br br-ex
# ovs-vsctl add-port br-ex eth0

Then configure

*********************************
Compute nodes X=(3,4)
*********************************
# cat ifcfg-br-ex
DEVICE="br-ex"
BOOTPROTO="static"
IPADDR="192.169.142.1(X)7"
NETMASK="255.255.255.0"
DNS1="83.221.202.254"
BROADCAST="192.169.142.255"
GATEWAY="192.169.142.1"
NM_CONTROLLED="no"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="yes"
IPV6INIT=no
ONBOOT="yes"
TYPE="OVSIntPort"
OVS_BRIDGE=br-ex
DEVICETYPE="ovs"

# cat ifcfg-eth0
DEVICE="eth0"
ONBOOT="yes"
TYPE="OVSPort"
DEVICETYPE="ovs"
OVS_BRIDGE=br-ex
NM_CONTROLLED=no
IPV6INIT=no


***************************
Then run script
***************************
#!/bin/bash -x
chkconfig network on
systemctl stop NetworkManager
systemctl disable NetworkManager
service network restart

Reboot node.

**********************************************
Nova-Docker Setup on each Compute
**********************************************
# curl -sSL https://get.docker.com/ | sh
# usermod -aG docker nova      ( seems not help to set 660 for docker.sock )
# systemctl start docker
# systemctl enable docker
# chmod 666  /var/run/docker.sock (add to /etc/rc.d/rc.local)
# easy_install pip
# git clone -b stable/mitaka   https://github.com/openstack/nova-docker

*******************
Driver build
*******************
# cd nova-docker
# pip install -r requirements.txt
# python setup.py install

********************************************
Switch nova-compute to DockerDriver
********************************************
vi /etc/nova/nova.conf
compute_driver=novadocker.virt.docker.DockerDriver


******************************************************************
Next on Controller/Network Node and each Compute Node
******************************************************************

mkdir /etc/nova/rootwrap.d
vi /etc/nova/rootwrap.d/docker.filters
[Filters]
# nova/virt/docker/driver.py: 'ln', '-sf', '/var/run/netns/.*'
ln: CommandFilter, /bin/ln, root

**********************************************************
Nova Compute Service restart on Compute Nodes
**********************************************************
# systemctl restart openstack-nova-compute

***********************************************
Glance API Service restart on Controller
**********************************************
vi /etc/glance/glance-api.conf
container_formats=ami,ari,aki,bare,ovf,ova,docker
# systemctl restart openstack-glance-api
  

****************************************
Status on Controller/Network
****************************************

[root@ip-192-169-142-127 ~(keystone_admin)]# nova-manage version
13.0.0-0.2.0rc3.el7

[root@ip-192-169-142-127 ~(keystone_admin)]# ip netns
snat-1c7ae3d0-3783-491b-b1da-a17d118cb145
qrouter-1c7ae3d0-3783-491b-b1da-a17d118cb145
qdhcp-11e9f7f5-d1a0-407e-9124-a3d26068f372

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron router-show RouterDSA
+-------------------------+---------------------------------------------------------------------+
| Field                   | Value                                                               |
+-------------------------+---------------------------------------------------------------------+
| admin_state_up          | True                                                                |
| availability_zone_hints |                                                                     |
| availability_zones      | nova                                                                |
| description             |                                                                     |
| distributed             | True                                                                |
| external_gateway_info   | {"network_id": "9f869fe1-e15d-40a4-b4fc-df89f3e39490",              |
|                         | "enable_snat": true, "external_fixed_ips": [{"subnet_id":           |
|                         | "1b04cc74-838f-4410-96c7-db21fef88ed1", "ip_address":               |
|                         | "192.169.142.150"}]}                                                |
| ha                      | False                                                               |
| id                      | 1c7ae3d0-3783-491b-b1da-a17d118cb145                                |
| name                    | RouterDSA                                                           |
| routes                  |                                                                     |
| status                  | ACTIVE                                                              |
| tenant_id               | edf899e41b4546689a9678db4c8e7567                                    |
+-------------------------+---------------------------------------------------------------------+

[root@ip-192-169-142-127 ~(keystone_admin)]# neutron l3-agent-list-hosting-router RouterDSA
+-----------------------------+-----------------------------+----------------+-------+----------+
| id                          | host                        | admin_state_up | alive | ha_state |
+-----------------------------+-----------------------------+----------------+-------+----------+
| 390b4dd0-e1b9-4446-877c-    | ip-192-169-142-127.ip.secur | True      | :-)   |
| e7ed0123fb73                        | eserver.net                          |                |      |      
+-----------------------------+-----------------------------+----------------+-------+----------+