Switch to Overcloud with Network isolation been setup via TripleO Master branch

UPDATE 09/23/2016

 Bug 1622683 gets status "Fix Released". Master's delorean trunk 09/23 works for me. So you shouldn't have to apply workaround - redeployment works
Already  tested.

END UPDATE

This post follows up TripleO deployment of 'master' branch via instack-virt-setup
Launchpad bug "Updating plans breaks deployment" https://bugs.launchpad.net/tripleo/+bug/1622683  still has status "In Progress" so to be able redeploy overcloud the workaround from https://bugs.launchpad.net/tripleo/+bug/1622720/comments/1  would be applied
**************************
Redeployment
**************************

[stack@instack ~]$ openstack stack delete overcloud
[stack@instack ~]$ . stackrc
[stack@instack ~]$ mistral environment-delete overcloud
   Request to delete environment overcloud has been accepted.
[stack@instack ~]$ swift delete --all

Add NAT Default VNIC to each of bare metal nodes (VMs)
To enable Internet connectivity from Controller after
overcloud deployment at the moment when "overcloud" stack got
gracefully deleted and status of bare metal nodes (VMs)  is down

****************************************
Make following updates on instack
****************************************

   $  sudo ovs-vsctl show
   $  sudo vi /etc/sysconfig/network-scripts/ifcfg-vlan10
       DEVICE=vlan10
       ONBOOT=yes
       DEVICETYPE=ovs
       TYPE=OVSIntPort
       BOOTPROTO=static
       IPADDR=10.0.0.1
       NETMASK=255.255.255.0
       OVS_BRIDGE=br-ctlplane
       OVS_OPTIONS="tag=10"

   $  sudo ifup vlan10

**********************************************************************
Make sure ovs-vsctl on undercloud has been updated
**********************************************************************
[stack@instack ~]$ sudo ovs-vsctl show
3dfb403a-c31d-4bb3-9851-08f2e7b7778f
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port int-br-ctlplane
            Interface int-br-ctlplane
                type: patch
                options: {peer=phy-br-ctlplane}
        Port "tapb104ab9a-36"
            tag: 1
            Interface "tapb104ab9a-36"
                type: internal
        Port br-int
            Interface br-int
                type: internal
    Bridge br-ctlplane
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "eth1"
            Interface "eth1"
        Port phy-br-ctlplane
            Interface phy-br-ctlplane
                type: patch
                options: {peer=int-br-ctlplane}
        Port "vlan10"
            tag: 10
            Interface "vlan10"
                type: internal
        Port br-ctlplane
            Interface br-ctlplane
                type: internal
    ovs_version: "2.5.0"
***************************************************
Create network_env.yaml under ~stack/
***************************************************

   [stack@instack ~]$ cat network_env.yaml
    {
    "parameter_defaults": {
        "ControlPlaneDefaultRoute": "192.0.2.1",
        "ControlPlaneSubnetCidr": "24",
        "DnsServers": [
            "192.168.122.5"
        ],
        "EC2MetadataIp": "192.0.2.1",
        "ExternalAllocationPools": [
            {
                "end": "10.0.0.250",
                "start": "10.0.0.4"
            }
        ],
        "ExternalNetCidr": "10.0.0.1/24",
        "NeutronExternalNetworkBridge": ""
    }
   }

Where 192.168.122.5 is instack VM Ip.

*************************
Deploy overcloud
*************************

#!/bin/bash -x
source /home/stack/stackrc
openstack overcloud deploy  \
--libvirt-type qemu  \
--ntp-server pool.ntp.org  \
--templates /home/stack/tripleo-heat-templates  \
-e /home/stack/tripleo-heat-templates/overcloud-resource-registry-puppet.yaml  \
-e  /home/stack/tripleo-heat-templates/environments/network-isolation.yaml  \
-e  /home/stack/tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml  \
-e  $HOME/network_env.yaml  \
--control-scale 1 --compute-scale 2

********************************************************************************
Up on completion proceed on undercloud (instack VM)  as follows
********************************************************************************


Add route to ctlplane network
 
[stack@instack ~]$ sudo route add -net 192.0.2.0/24 gw 192.0.2.1

[stack@instack ~]$ sudo ip route
default via 192.168.122.1 dev eth0
10.0.0.0/24 dev vlan10  proto kernel  scope link  src 10.0.0.1
192.0.2.0/24 via 192.0.2.1 dev br-ctlplane  scope link
192.0.2.0/24 dev br-ctlplane  proto kernel  scope link  src 192.0.2.1
192.168.122.0/24 dev eth0  proto kernel  scope link  src 192.168.122.5


[stack@instack ~]$ . stackrc
[stack@instack ~]$ nova list

+--------------------------------------+-------------------------+--------+------------+-------------+---------------------+
| ID                                   | Name                    | Status | Task State | Power State | Networks            |
+--------------------------------------+-------------------------+--------+------------+-------------+---------------------+
| 0212a5cc-c73e-43c3-bddb-51cac22f0060 | overcloud-controller-0  | ACTIVE | -          | Running     | ctlplane=192.0.2.9  |
| a421c80b-54a5-4cc8-9414-45d45a27845b | overcloud-novacompute-0 | ACTIVE | -          | Running     | ctlplane=192.0.2.18 |
| 3641a8da-c5fa-4975-9e43-c926522ecc2b | overcloud-novacompute-1 | ACTIVE | -          | Running     | ctlplane=192.0.2.13 |
+--------------------------------------+-------------------------+--------+------------+-------------+---------------------+

[stack@instack ~]$ neutron net-list

+--------------------------------------+--------------+----------------------------------------+
| id                                   | name         | subnets                                |
+--------------------------------------+--------------+----------------------------------------+
| 5309b1a3-f6c6-4bdd-a0bc-93f418853080 | external     | 56fe052f-ba26-437b-94ab-b03688e06ad9   |
|                                      |              | 10.0.0.0/24                            |
| 77440f54-0ce4-444c-8983-2ef2ae1408b4 | ctlplane     | 76055a99-45e4-4b5a-b1fc-846c91137427   |
|                                      |              | 192.0.2.0/24                           |
| 7b3e788a-ebdd-4e7c-b076-517ca62befb3 | tenant       | 0a028e34-8a0a-48ce-88d8-5523b19eac0f   |
|                                      |              | 172.16.0.0/24                          |
| 813d17c3-bd58-490f-94a4-aefeb2057d22 | storage_mgmt | e3cdcf74-64fa-4837-b480-304a1329d109   |
|                                      |              | 172.16.3.0/24                          |
| bcba764c-0b27-4785-b875-8b20bd28cd96 | internal_api | 1de0ff85-7525-4e1f-94ea-1bc6e060a096   |
|                                      |              | 172.16.2.0/24                          |
| d4c8e9d8-bffc-4803-8ee4-bbff63eef9e1 | storage      | f76d3eeb-c7d8-47e9-a2e3-95765975c292   |
|                                      |              | 172.16.1.0/24                          |
+--------------------------------------+--------------+----------------------------------------+

  

[root@overcloud-controller-0 ~]# nova service-list

+----+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host                                | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+
| 3  | nova-consoleauth | overcloud-controller-0.localdomain  | internal | enabled | up    | 2016-09-19T10:02:37.000000 | -               |
| 4  | nova-scheduler   | overcloud-controller-0.localdomain  | internal | enabled | up    | 2016-09-19T10:02:31.000000 | -               |
| 5  | nova-conductor   | overcloud-controller-0.localdomain  | internal | enabled | up    | 2016-09-19T10:02:30.000000 | -               |
| 6  | nova-compute     | overcloud-novacompute-1.localdomain | nova     | enabled | up    | 2016-09-19T10:02:29.000000 | -               |
| 7  | nova-compute     | overcloud-novacompute-0.localdomain | nova     | enabled | up    | 2016-09-19T10:02:35.000000 | -               |
+----+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+

FoxyProxy tuned for external network

  

   List of instances launched and running via Nova CLI

   

*****************************************************
Controller's ovs-vsctl show report
*****************************************************
[root@overcloud-controller-0 ~]# ovs-vsctl show
d818c01e-d0ce-425d-a9c8-07e0ff541ea9
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tap19ce4553-8f"
            tag: 2
            Interface "tap19ce4553-8f"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port "qr-4a00fb57-90"
            tag: 2
            Interface "qr-4a00fb57-90"
                type: internal
        Port "qg-5b1fb5eb-d5"
            tag: 4
            Interface "qg-5b1fb5eb-d5"
                type: internal
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
    Bridge br-ex
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
        Port "vlan40"
            tag: 40
            Interface "vlan40"
                type: internal
        Port "eth0"
            Interface "eth0"
        Port "vlan20"
            tag: 20
            Interface "vlan20"
                type: internal
        Port br-ex
            Interface br-ex
                type: internal
        Port "vlan30"
            tag: 30
            Interface "vlan30"
                type: internal
        Port "vlan50"
            tag: 50
            Interface "vlan50"
                type: internal
        Port "vlan10"
            tag: 10
            Interface "vlan10"
                type: internal
    Bridge br-tun
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port br-tun
            Interface br-tun
                type: internal
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port "vxlan-ac100009"
            Interface "vxlan-ac100009"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="172.16.0.12", out_key=flow, remote_ip="172.16.0.9"}
        Port "vxlan-ac10000d"
            Interface "vxlan-ac10000d"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="172.16.0.12", out_key=flow, remote_ip="172.16.0.13"}
    ovs_version: "2.5.0"

**********************************************************************
 Hypervisor status on Compute nodes (Newton RC1)
 Qemu-kvm-ev-2.31 gets installed by default
**********************************************************************
[root@overcloud-novacompute-0 ~]# virsh --connect qemu:///system
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # version
Compiled against library: libvirt 1.2.17
Using library: libvirt 1.2.17
Using API: QEMU 1.2.17
Running hypervisor: QEMU 2.3.0

virsh # list --all
 Id    Name                           State
----------------------------------------------------
 6     instance-00000004              running


[root@overcloud-novacompute-1 ~]# virsh --connect qemu:///system
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # version
Compiled against library: libvirt 1.2.17
Using library: libvirt 1.2.17
Using API: QEMU 1.2.17
Running hypervisor: QEMU 2.3.0

virsh # list --all
 Id    Name                           State
----------------------------------------------------
 5     instance-00000005              running

*************************************
VIRTHOST Configuration
*************************************
[root@ServerVIRT1608 ~]# brctl show
bridge name    bridge id        STP enabled    interfaces
brext        8000.525400b017dc    no        brext-nic
brovc        8000.525400948dc8    no        brovc-nic
virbr0        8000.525400f83b3b    yes        virbr0-nic
                            vnet0
                            vnet3
                            vnet5
                            vnet7

[root@ServerVIRT1608 ~]# ovs-vsctl show
96876d44-cca3-4e93-b89c-8238b4745c3c
    Bridge brbm
        Port "vnet6"
            Interface "vnet6"
        Port "vnet4"
            Interface "vnet4"
        Port "vnet1"
            Interface "vnet1"
        Port "vnet2"
            Interface "vnet2"
        Port brbm
            Interface brbm
                type: internal
    ovs_version: "2.5.0"