Tuesday, September 27, 2016

TripleO deployment of 'master' branch via instack-virt-setup on VIRTHOST (2)

UPDATE 09/30/2016

 $ sudo route add -net 192.0.2.0/24 gw 192.0.2.1 ( on instack VM )
 no longer needed , moreover affects ssh connect to overcloud nodes

***************************
Overcloud-deploy.sh
***************************
#!/bin/bash -x
source /home/stack/stackrc
openstack overcloud deploy \
  --libvirt-type qemu \
  --ntp-server pool.ntp.org \
  --templates /home/stack/tripleo-heat-templates \
  -e  /home/stack/tripleo-heat-templates/environments/network-isolation.yaml \
  -e  /home/stack/tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml \
  -e  $HOME/network_env.yaml \
  --control-scale 1 --compute-scale 2 
END UPDATE

Upstream gets close to Newton Release , bugs scheduled for RC2 went away.
Following bellow is  a clean and smoothly running procedure of Overcloud deployment TripleO Master branch via instack-virt-setup on 32 GB VIRTHOST  Network isolation in overcloud is pre-configured on instack (undercloud ) , step which is usually very hard to locate in official docs http://tripleo.org/index.html
and which is silently skipped in TripleO related blogs. Also mistral execution list gets verified up on overcloud deployment completion. Running `systemctl status keepalived -l` on overcloud controller we see sending gratuitous ARPs what confirms network isolation implementation.

Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]:
VRRP_Instance(51) Sending gratuitous ARPs on br-ex for 192.0.2.13
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]: 
VRRP_Instance(56) Entering MASTER STATE
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]:
VRRP_Instance(56) setting protocol VIPs.
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]:
VRRP_Instance(56) Sending gratuitous ARPs on vlan20 for 172.16.2.7
Sep 27 14:55:03 overcloud-controller-0 Keepalived_healthcheckers[18505]:
Netlink reflector reports IP 172.16.2.7 added
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]:
VRRP_Instance(52) Entering MASTER STATE
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]:
VRRP_Instance(52) setting protocol VIPs.
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]:
VRRP_Instance(52) Sending gratuitous ARPs on br-ex for 10.0.0.4
Sep 27 14:55:03 overcloud-controller-0 Keepalived_healthcheckers[18505]:
Netlink reflector reports IP 10.0.0.4 added

*****************************************
Tune stack environment on VIRTHOST
*****************************************
# useradd stack
# echo "stack:stack" | chpasswd
# echo "stack ALL=(root) NOPASSWD:ALL" | sudo tee -a /etc/sudoers.d/stack
#  chmod 0440 /etc/sudoers.d/stack
#  su - stack

***************************
Tune stack ENV
**************************
export NODE_DIST=centos7
export NODE_CPU=2
export NODE_MEM=7550
export NODE_COUNT=3
export UNDERCLOUD_NODE_CPU=4
export UNDERCLOUD_NODE_MEM=9000
export FS_TYPE=ext4

****************************************************************
Re-login to stack (highlight long line and copy if needed)
****************************************************************

 $ sudo yum -y install epel-release 
 $ sudo yum -y install yum-plugin-priorities 
 $ sudo curl -o /etc/yum.repos.d/delorean.repo \
   http://buildlogs.centos.org/centos/7/cloud/x86_64/rdo-trunk-master-tripleo/delorean.repo 
 $ sudo curl -o /etc/yum.repos.d/delorean-deps.repo  \
   http://trunk.rdoproject.org/centos7/delorean-deps.repo 
 $ sudo yum install -y instack-undercloud
 $ instack-virt-setup


*********************
On instack VM
*********************
Create swap file per http://www.anstack.com/blog/2016/07/04/manually-installing-tripleo-recipe.html  :-

 #Add a 4GB swap file to the Undercloud
  sudo dd if=/dev/zero of=/swapfile bs=1024 count=4194304
  sudo mkswap /swapfile
  #Turn ON the swap file
  sudo chmod 600 /swapfile
  sudo swapon /swapfile
  #Enable it on start
  sudo echo "/swapfile   swap   swap    defaults        0 0" >> /etc/fstab

***************************
Restart instack VM
***************************

Next :-
# su - stack

*************************************
Update .bashrc under ~stack/ 
*************************************

  export USE_DELOREAN_TRUNK=1
  export DELOREAN_TRUNK_REPO="http://buildlogs.centos.org/centos/7/cloud/x86_64/rdo-trunk-master-tripleo/"
  export DELOREAN_REPO_FILE="delorean.repo"
  export FS_TYPE=ext4

[stack@instack ~]$ git clone https://github.com/openstack/tripleo-heat-templates
[stack@instack ~]$ git clone https://github.com/openstack-infra/tripleo-ci.git

[stack@instack ~]$ ./tripleo-ci/scripts/tripleo.sh --repo-setup
[stack@instack ~]$ ./tripleo-ci/scripts/tripleo.sh --undercloud
[stack@instack ~]$ source stackrc
[stack@instack ~]$ ./tripleo-ci/scripts/tripleo.sh --overcloud-images
[stack@instack ~]$ ./tripleo-ci/scripts/tripleo.sh --register-nodes
[stack@instack ~]$ ./tripleo-ci/scripts/tripleo.sh --introspect-nodes

Image file overcloud-full.qcow2 created...
Successfully built all requested images
You must source a stackrc file for the Undercloud.
Attempting to source /home/stack/stackrc
Done
WARNING: openstackclient.common.utils is deprecated and will be removed after Jun 2017. Please use osc_lib.utils
Image "overcloud-full-vmlinuz" was uploaded.
+--------------------------------------+------------------------+-------------+---------+--------+
|                  ID                  |          Name          | Disk Format |   Size  | Status |
+--------------------------------------+------------------------+-------------+---------+--------+
| 37dba3bf-5683-4a33-b6d5-9ed90e1f189d | overcloud-full-vmlinuz |     aki     | 5157296 | active |
+--------------------------------------+------------------------+-------------+---------+--------+
Image "overcloud-full-initrd" was uploaded.
+--------------------------------------+-----------------------+-------------+----------+--------+
|                  ID                  |          Name         | Disk Format |   Size   | Status |
+--------------------------------------+-----------------------+-------------+----------+--------+
| 0bfd61f2-1c03-43ab-82e5-811c346dadd0 | overcloud-full-initrd |     ari     | 42124221 | active |
+--------------------------------------+-----------------------+-------------+----------+--------+
Image "overcloud-full" was uploaded.
+--------------------------------------+----------------+-------------+------------+--------+
|                  ID                  |      Name      | Disk Format |    Size    | Status |
+--------------------------------------+----------------+-------------+------------+--------+
| d2c41746-fb4c-4438-995b-22811df6f772 | overcloud-full |    qcow2    | 1178590720 | active |
+--------------------------------------+----------------+-------------+------------+--------+
Image "bm-deploy-kernel" was uploaded.
+--------------------------------------+------------------+-------------+---------+--------+
|                  ID                  |       Name       | Disk Format |   Size  | Status |
+--------------------------------------+------------------+-------------+---------+--------+
| f237b9a5-33a8-4f33-998a-571059f0522b | bm-deploy-kernel |     aki     | 5157296 | active |
+--------------------------------------+------------------+-------------+---------+--------+
Image "bm-deploy-ramdisk" was uploaded.
+--------------------------------------+-------------------+-------------+-----------+--------+
|                  ID                  |        Name       | Disk Format |    Size   | Status |
+--------------------------------------+-------------------+-------------+-----------+--------+
| 97c40ed7-296f-42b8-9d3c-3d40b36040eb | bm-deploy-ramdisk |     ari     | 318648193 | active |
+--------------------------------------+-------------------+-------------+-----------+--------+
~
#################
tripleo.sh -- Overcloud images - DONE.
#################
#################
[stack@instack ~]$ ./tripleo-ci/scripts/tripleo.sh --register-nodes
#################
tripleo.sh -- Register nodes
#################
You must source a stackrc file for the Undercloud.
Attempting to source /home/stack/stackrc
Done
WARNING: openstackclient.common.utils is deprecated and will be removed after Jun 2017. Please use osc_lib.utils
Started Mistral Workflow. Execution ID: 9a148b8b-fe55-43b1-b3e1-cb13fad49759
Successfully registered node UUID 1f031f3f-edb1-434c-8b6f-c60bffce9941
Successfully registered node UUID cca63d2d-6912-4878-9ea7-a90510fc09b2
Successfully registered node UUID 584bb979-b715-4c08-836f-2200c6d4d937
Started Mistral Workflow. Execution ID: 071e4b85-2b7c-420e-96bd-bbbe980f9db7
Successfully set all nodes to available.
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| UUID                                 | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
| 1f031f3f-edb1-434c-8b6f-c60bffce9941 | None | None          | power off   | available          | False       |
| cca63d2d-6912-4878-9ea7-a90510fc09b2 | None | None          | power off   | available          | False       |
| 584bb979-b715-4c08-836f-2200c6d4d937 | None | None          | power off   | available          | False       |
+--------------------------------------+------+---------------+-------------+--------------------+-------------+
#################
tripleo.sh -- Register nodes - DONE.
#################
[stack@instack ~]$  ./tripleo-ci/scripts/tripleo.sh --introspect-nodes
#################
tripleo.sh -- Introspect nodes
#################
You must source a stackrc file for the Undercloud.
Attempting to source /home/stack/stackrc
Done
WARNING: openstackclient.common.utils is deprecated and will be removed after Jun 2017. Please use osc_lib.utils
Setting nodes for introspection to manageable...
Starting introspection of manageable nodes
Started Mistral Workflow. Execution ID: e4e63d1a-3e6c-42d5-9575-e4166853cdd0
Waiting for introspection to finish...
Introspection for UUID 1f031f3f-edb1-434c-8b6f-c60bffce9941 finished successfully.
Introspection for UUID cca63d2d-6912-4878-9ea7-a90510fc09b2 finished successfully.
Introspection for UUID 584bb979-b715-4c08-836f-2200c6d4d937 finished successfully.
Introspection completed.
Setting manageable nodes to available...
Started Mistral Workflow. Execution ID: 1d119a65-a5a8-4b81-b5da-2fd3b15f26e1

#################
tripleo.sh -- Introspect nodes - DONE.
#################
 
Now create external interface vlan10. 
 
[stack@instack ~]$ sudo vi /etc/sysconfig/network-scripts/ifcfg-vlan10
DEVICE=vlan10
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSIntPort
BOOTPROTO=static
IPADDR=10.0.0.1
NETMASK=255.255.255.0
OVS_BRIDGE=br-ctlplane
OVS_OPTIONS="tag=10"

[stack@instack ~]$ sudo ifup vlan10
 
[stack@instack ~]$ sudo ovs-vsctl show
43ccb3e7-74ed-4192-a87d-80b5a71a7e80
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port int-br-ctlplane
            Interface int-br-ctlplane
                type: patch
                options: {peer=phy-br-ctlplane}
        Port br-int
            Interface br-int
                type: internal
        Port "tap0d0fb165-79"
            tag: 1
            Interface "tap0d0fb165-79"
                type: internal
    Bridge br-ctlplane
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "eth1"
            Interface "eth1"
        Port br-ctlplane
            Interface br-ctlplane
                type: internal
        Port "vlan10"
            tag: 10
            Interface "vlan10"
                type: internal
        Port phy-br-ctlplane
            Interface phy-br-ctlplane
                type: patch
                options: {peer=int-br-ctlplane}
    ovs_version: "2.5.0"
[stack@instack ~]$ ifconfig
br-ctlplane: flags=4163  mtu 1500
        inet 192.0.2.1  netmask 255.255.255.0  broadcast 192.0.2.255
        inet6 fe80::283:bbff:feda:c642  prefixlen 64  scopeid 0x20
        ether 00:83:bb:da:c6:42  txqueuelen 0  (Ethernet)
        RX packets 43022  bytes 2956223 (2.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 15525  bytes 972453334 (927.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163  mtu 1500
        inet 192.168.122.90  netmask 255.255.255.0  broadcast 192.168.122.255
        inet6 fe80::5054:ff:fe28:530d  prefixlen 64  scopeid 0x20
        ether 52:54:00:28:53:0d  txqueuelen 1000  (Ethernet)
        RX packets 881966  bytes 1281751784 (1.1 GiB)
        RX errors 0  dropped 3  overruns 0  frame 0
        TX packets 539560  bytes 43216702 (41.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163  mtu 1500
        inet6 fe80::283:bbff:feda:c642  prefixlen 64  scopeid 0x20
        ether 00:83:bb:da:c6:42  txqueuelen 1000  (Ethernet)
        RX packets 43015  bytes 2955825 (2.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 15538  bytes 972454368 (927.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10
        loop  txqueuelen 0  (Local Loopback)
        RX packets 867304  bytes 4826379602 (4.4 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 867304  bytes 4826379602 (4.4 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vlan10: flags=4163  mtu 1500
        inet 10.0.0.1  netmask 255.255.255.0  broadcast 10.0.0.255
        inet6 fe80::dc07:f5ff:fe72:2c9  prefixlen 64  scopeid 0x20
        ether de:07:f5:72:02:c9  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 12  bytes 816 (816.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
Instack IP is 192.168.122.90
 
[stack@instack ~]$ vi  network_env.yaml
{
    "parameter_defaults": {
        "ControlPlaneDefaultRoute": "192.0.2.1",
        "ControlPlaneSubnetCidr": "24",
        "DnsServers": [
            "192.168.122.90"
        ],
        "EC2MetadataIp": "192.0.2.1",
        "ExternalAllocationPools": [
            {
                "end": "10.0.0.250",
                "start": "10.0.0.4"
            }
        ],
        "ExternalNetCidr": "10.0.0.1/24",
        "NeutronExternalNetworkBridge": ""
    }
}  
[stack@instack ~]$ sudo iptables -A BOOTSTACK_MASQ -s 10.0.0.0/24 ! -d 10.0.0.0/24 -j MASQUERADE -t nat
 
[stack@instack ~]$  vi overcloud-deploy.sh
#!/bin/bash -x
source /home/stack/stackrc
openstack overcloud deploy \
  --libvirt-type qemu \
  --ntp-server pool.ntp.org \
  --templates /home/stack/tripleo-heat-templates \
  -e /home/stack/tripleo-heat-templates/overcloud-resource-registry-puppet.yaml \<== Skip as of 09/30
  -e  /home/stack/tripleo-heat-templates/environments/network-isolation.yaml \
  -e  /home/stack/tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml \
  -e  $HOME/network_env.yaml \
  --control-scale 1 --compute-scale 2 
 
[stack@instack ~]$ cat $HOME/network_env.yaml
[stack@instack ~]$ chmod a+x overcloud-deploy.sh
[stack@instack ~]$ touch -f  /home/stack/tripleo-heat-templates/puppet/post.yaml 
 [stack@instack ~]$ neutron subnet-list
+------------------------------------+------+--------------+------------------------------------+
| id                                 | name | cidr         | allocation_pools                   |
+------------------------------------+------+--------------+------------------------------------+
| bc762c84-558a-4091-aeca-           |      | 192.0.2.0/24 | {"start": "192.0.2.5", "end":      |
| b0b1a428e5f1                       |      |              | "192.0.2.24"}                      |
+------------------------------------+------+--------------+------------------------------------+
[stack@instack ~]$ neutron subnet-update bc762c84-558a-4091-aeca-b0b1a428e5f1 --dns-nameserver 83.221.202.254
Updated subnet: bc762c84-558a-4091-aeca-b0b1a428e5f1 
[stack@instack ~]$ chmod a+x overcloud-deploy.sh
[stack@instack ~]$ ./overcloud-deploy.sh
+ source /home/stack/stackrc
++ export NOVA_VERSION=1.1
++ NOVA_VERSION=1.1
+++ sudo hiera admin_password
++ export OS_PASSWORD=05440e148840c9bb860584d4e9c42fe573096409
++ OS_PASSWORD=05440e148840c9bb860584d4e9c42fe573096409
++ export OS_AUTH_URL=http://192.0.2.1:5000/v2.0
++ OS_AUTH_URL=http://192.0.2.1:5000/v2.0
++ export OS_USERNAME=admin
++ OS_USERNAME=admin
++ export OS_TENANT_NAME=admin
++ OS_TENANT_NAME=admin
++ export COMPUTE_API_VERSION=1.1
++ COMPUTE_API_VERSION=1.1
++ export OS_BAREMETAL_API_VERSION=1.15
++ OS_BAREMETAL_API_VERSION=1.15
++ export OS_NO_CACHE=True
++ OS_NO_CACHE=True
++ export OS_CLOUDNAME=undercloud
++ OS_CLOUDNAME=undercloud
++ export OS_IMAGE_API_VERSION=1
++ OS_IMAGE_API_VERSION=1
+ openstack overcloud deploy --libvirt-type qemu \
--ntp-server pool.ntp.org --templates \
/home/stack/tripleo-heat-templates \
-e /home/stack/tripleo-heat-templates/overcloud-resource-registry-puppet.yaml\
-e /home/stack/tripleo-heat-templates/environments/network-isolation.yaml \
-e /home/stack/tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml \
-e /home/stack/network_env.yaml \
--control-scale 1 --compute-scale 2


WARNING: openstackclient.common.utils is deprecated and will be removed after Jun 2017. Please use osc_lib.utils
Removing the current plan files
Uploading new plan files
Started Mistral Workflow. Execution ID: 08b899f5-0444-4eb8-8719-a6eba9a81fa0
Plan updated
Deploying templates in the directory /home/stack/tripleo-heat-templates
Started Mistral Workflow. Execution ID: 1c291c68-aec7-49e8-836f-658b06763c92
2016-09-27 14:09:37Z [overcloud]: CREATE_IN_PROGRESS  Stack CREATE started
2016-09-27 14:09:37Z [MysqlRootPassword]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:37Z [RabbitCookie]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:37Z [HorizonSecret]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:37Z [HeatAuthEncryptionKey]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:37Z [Networks]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:37Z [PcsdPassword]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [ServiceNetMap]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [overcloud-Networks-oqxpnzrtiaem]: CREATE_IN_PROGRESS  Stack CREATE started
2016-09-27 14:09:38Z [InternalNetwork]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [ManagementNetwork]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [StorageMgmtNetwork]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [overcloud-Networks-oqxpnzrtiaem-InternalNetwork-l4jjn3botrgj]: CREATE_IN_PROGRESS  Stack CREATE started
2016-09-27 14:09:38Z [InternalApiNetwork]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [RabbitCookie]: CREATE_COMPLETE  state changed
2016-09-27 14:09:38Z [StorageNetwork]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [HorizonSecret]: CREATE_COMPLETE  state changed
2016-09-27 14:09:38Z [PcsdPassword]: CREATE_COMPLETE  state changed
2016-09-27 14:09:38Z [overcloud-Networks-oqxpnzrtiaem-StorageMgmtNetwork-jnuiofatb5tu]: CREATE_IN_PROGRESS  Stack CREATE started
2016-09-27 14:09:38Z [HeatAuthEncryptionKey]: CREATE_COMPLETE  state changed
2016-09-27 14:09:38Z [MysqlRootPassword]: CREATE_COMPLETE  state changed
2016-09-27 14:09:38Z [StorageMgmtNetwork]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [TenantNetwork]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [DefaultPasswords]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [overcloud-Networks-oqxpnzrtiaem-StorageNetwork-x3pafjtlirmz]: CREATE_IN_PROGRESS  Stack CREATE started
2016-09-27 14:09:38Z [StorageNetwork]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [ExternalNetwork]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [overcloud-Networks-oqxpnzrtiaem-TenantNetwork-2nndp6au2sfp]: CREATE_IN_PROGRESS  Stack CREATE started
2016-09-27 14:09:38Z [TenantNetwork]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [NetworkExtraConfig]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:09:38Z [overcloud-Networks-oqxpnzrtiaem-ExternalNetwork-ht7dkpzmiskb]: CREATE_IN_PROGRESS  Stack CREATE started
2016-09-27 14:09:38Z [ManagementNetwork]: CREATE_COMPLETE  state changed
 
. . . . . . 

2016-09-27 14:38:22Z [0]: CREATE_COMPLETE  state changed
2016-09-27 14:38:22Z [overcloud-AllNodesDeploySteps-zs2vx53nvajt-ControllerDeployment_Step5-is6nnvfdpzxg]: CREATE_COMPLETE  Stack CREATE completed successfully
2016-09-27 14:38:22Z [ControllerDeployment_Step5]: CREATE_COMPLETE  state changed
2016-09-27 14:38:22Z [BlockStoragePostConfig]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:38:22Z [ObjectStoragePostConfig]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:38:22Z [ComputePostConfig]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:38:22Z [CephStoragePostConfig]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:38:22Z [ControllerPostConfig]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:38:23Z [BlockStoragePostConfig]: CREATE_COMPLETE  state changed
2016-09-27 14:38:23Z [ObjectStoragePostConfig]: CREATE_COMPLETE  state changed
2016-09-27 14:38:23Z [ComputePostConfig]: CREATE_COMPLETE  state changed
2016-09-27 14:38:24Z [CephStoragePostConfig]: CREATE_COMPLETE  state changed
2016-09-27 14:38:24Z [ControllerPostConfig]: CREATE_COMPLETE  state changed
2016-09-27 14:38:24Z [CephStorageExtraConfigPost]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:38:24Z [ComputeExtraConfigPost]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:38:24Z [BlockStorageExtraConfigPost]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:38:24Z [ControllerExtraConfigPost]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:38:24Z [ObjectStorageExtraConfigPost]: CREATE_IN_PROGRESS  state changed
2016-09-27 14:38:25Z [ComputeExtraConfigPost]: CREATE_COMPLETE  state changed
2016-09-27 14:38:25Z [CephStorageExtraConfigPost]: CREATE_COMPLETE  state changed
2016-09-27 14:38:25Z [ControllerExtraConfigPost]: CREATE_COMPLETE  state changed
2016-09-27 14:38:25Z [ObjectStorageExtraConfigPost]: CREATE_COMPLETE  state changed
2016-09-27 14:38:25Z [BlockStorageExtraConfigPost]: CREATE_COMPLETE  state changed
2016-09-27 14:38:25Z [overcloud-AllNodesDeploySteps-zs2vx53nvajt]: CREATE_COMPLETE  Stack CREATE completed successfully
2016-09-27 14:38:26Z [AllNodesDeploySteps]: CREATE_COMPLETE  state changed
2016-09-27 14:38:26Z [overcloud]: CREATE_COMPLETE  Stack CREATE completed successfully

 Stack overcloud CREATE_COMPLETE 

Overcloud Endpoint: http://10.0.0.4:5000/v2.0
Overcloud Deployed
 
***********************************************************
Checking for errors in mistral execution list
which is new in Newton release
***********************************************************

[stack@instack ~]$ mistral execution-list 
+----------+-------------+---------------+-------------+-------------------+---------+------------+------------+---------------+
| ID       | Workflow ID | Workflow name | Description | Task Execution ID | State   | State info | Created at | Updated at    |
+----------+-------------+---------------+-------------+-------------------+---------+------------+------------+---------------+
| 5bce3202 | bde1cc99-ef | tripleo.plan_ |             | <none>            | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -c2b3-47 | e5-40df-    | management.v1 |             |                   |         |            | 12:31:44   | 12:31:58      |
| 35-ad40- | bc5a-359141 | .create_defau |             |                   |         |            |            |               |
| 2b35e775 | e48a73      | lt_deployment |             |                   |         |            |            |               |
| 4647     |             | _plan         |             |                   |         |            |            |               |
| 9a148b8b | 75fb3808    | tripleo.barem |             | <none>            | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -fe55-43 | -142c-4d98- | etal.v1.regis |             |                   |         |            | 13:40:34   | 13:40:49      |
| b1-b3e1- | a509-edfcee | ter_or_update |             |                   |         |            |            |               |
| cb13fad4 | 056fe5      |               |             |                   |         |            |            |               |
| 9759     |             |               |             |                   |         |            |            |               |
| 0a450c25 | bcc2d68d-cd | tripleo.barem | sub-        | 191d237c-1322     | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -bf68    | da-4919-998 | etal.v1.set_n | workflow    | -4b3c-867e-       |         |            | 13:40:42   | 13:40:45      |
| -487a-af | 4-00e03bbdf | ode_state     | execution   | ddbd7121e01f      |         |            |            |               |
| 98-7125e | db8         |               |             |                   |         |            |            |               |
| 809a737  |             |               |             |                   |         |            |            |               |
| 3e8eff39 | bcc2d68d-cd | tripleo.barem | sub-        | 191d237c-1322     | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -f732-49 | da-4919-998 | etal.v1.set_n | workflow    | -4b3c-867e-       |         |            | 13:40:42   | 13:40:45      |
| 76-b541- | 4-00e03bbdf | ode_state     | execution   | ddbd7121e01f      |         |            |            |               |
| b3446c6c | db8         |               |             |                   |         |            |            |               |
| 6fde     |             |               |             |                   |         |            |            |               |
| 55385132 | bcc2d68d-cd | tripleo.barem | sub-        | 191d237c-1322     | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -6e79    | da-4919-998 | etal.v1.set_n | workflow    | -4b3c-867e-       |         |            | 13:40:42   | 13:40:45      |
| -400b-   | 4-00e03bbdf | ode_state     | execution   | ddbd7121e01f      |         |            |            |               |
| 965d-1a2 | db8         |               |             |                   |         |            |            |               |
| f6ccada2 |             |               |             |                   |         |            |            |               |
| 4        |             |               |             |                   |         |            |            |               |
| 071e4b85 | 22cd7376    | tripleo.barem |             | <none>            | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -2b7c-   | -d6cd-49db- | etal.v1.provi |             |                   |         |            | 13:40:49   | 13:40:56      |
| 420e-    | 801b-74ef4e | de            |             |                   |         |            |            |               |
| 96bd-bbb | 197f3f      |               |             |                   |         |            |            |               |
| e980f9db |             |               |             |                   |         |            |            |               |
| 7        |             |               |             |                   |         |            |            |               |
| 288e2ff2 | bcc2d68d-cd | tripleo.barem | sub-        | 52352bbc-8667-458 | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -2161    | da-4919-998 | etal.v1.set_n | workflow    | 5-8099-39f6738381 |         |            | 13:40:49   | 13:40:52      |
| -473f-84 | 4-00e03bbdf | ode_state     | execution   | 5a                |         |            |            |               |
| c9-382bd | db8         |               |             |                   |         |            |            |               |
| 7b6b0bd  |             |               |             |                   |         |            |            |               |
| 28e41b0a | bcc2d68d-cd | tripleo.barem | sub-        | 52352bbc-8667-458 | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -eb5f-4e | da-4919-998 | etal.v1.set_n | workflow    | 5-8099-39f6738381 |         |            | 13:40:49   | 13:40:52      |
| 70-b537- | 4-00e03bbdf | ode_state     | execution   | 5a                |         |            |            |               |
| e5da84e2 | db8         |               |             |                   |         |            |            |               |
| af7b     |             |               |             |                   |         |            |            |               |
| 2c30a482 | bcc2d68d-cd | tripleo.barem | sub-        | 52352bbc-8667-458 | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -a4fc-   | da-4919-998 | etal.v1.set_n | workflow    | 5-8099-39f6738381 |         |            | 13:40:49   | 13:40:52      |
| 42be-    | 4-00e03bbdf | ode_state     | execution   | 5a                |         |            |            |               |
| a66b-568 | db8         |               |             |                   |         |            |            |               |
| b5bb7fa5 |             |               |             |                   |         |            |            |               |
| 0        |             |               |             |                   |         |            |            |               |
| 2026ebb3 | a3b2b56e-   | tripleo.barem | sub-        | 68dfd5f3-ba31-432 | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -2cf9-43 | 0a12-4354   | etal.v1.intro | workflow    | 1-9b0e-           |         |            | 13:41:12   | 13:43:19      |
| 90-90fc- | -b7de-2cbaa | spect         | execution   | 1d35b5181c49      |         |            |            |               |
| 05688171 | c7b3406     |               |             |                   |         |            |            |               |
| 9836     |             |               |             |                   |         |            |            |               |
| e4e63d1a | 4130daad-38 | tripleo.barem |             | <none>            | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -3e6c-42 | 0d-4406-b02 | etal.v1.intro |             |                   |         |            | 13:41:12   | 13:43:22      |
| d5-9575- | 1-9e8b4bb3b | spect_managea |             |                   |         |            |            |               |
| e4166853 | 0e1         | ble_nodes     |             |                   |         |            |            |               |
| cdd0     |             |               |             |                   |         |            |            |               |
| 1d119a65 | 29c51fab-   | tripleo.barem |             | <none>            | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -a5a8-4b | f6b9-4cc1   | etal.v1.provi |             |                   |         |            | 13:43:21   | 13:43:33      |
| 81-b5da- | -94bc-14144 | de_manageable |             |                   |         |            |            |               |
| 2fd3b15f | bc8284f     | _nodes        |             |                   |         |            |            |               |
| 26e1     |             |               |             |                   |         |            |            |               |
| 9262a91e | 22cd7376    | tripleo.barem | sub-        | 8052fcbb-b511     | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -a7bf-   | -d6cd-49db- | etal.v1.provi | workflow    | -423a-bfdc-       |         |            | 13:43:22   | 13:43:30      |
| 439b-    | 801b-74ef4e | de            | execution   | 8555b8e169a8      |         |            |            |               |
| b9fc-e48 | 197f3f      |               |             |                   |         |            |            |               |
| ff49f5aa |             |               |             |                   |         |            |            |               |
| d        |             |               |             |                   |         |            |            |               |
| ab696f08 | bcc2d68d-cd | tripleo.barem | sub-        | da2c83ad-30bf-425 | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -76a8    | da-4919-998 | etal.v1.set_n | workflow    | 1-8859-20c8d9b895 |         |            | 13:43:22   | 13:43:25      |
| -478c-8c | 4-00e03bbdf | ode_state     | execution   | cb                |         |            |            |               |
| d2-7b5bb | db8         |               |             |                   |         |            |            |               |
| 3f1fba0  |             |               |             |                   |         |            |            |               |
| 77ede1a7 | bcc2d68d-cd | tripleo.barem | sub-        | da2c83ad-30bf-425 | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -4167-48 | da-4919-998 | etal.v1.set_n | workflow    | 1-8859-20c8d9b895 |         |            | 13:43:23   | 13:43:26      |
| 95-b786- | 4-00e03bbdf | ode_state     | execution   | cb                |         |            |            |               |
| 4ff65ce7 | db8         |               |             |                   |         |            |            |               |
| 1c6b     |             |               |             |                   |         |            |            |               |
| feec4ba0 | bcc2d68d-cd | tripleo.barem | sub-        | da2c83ad-30bf-425 | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -7331    | da-4919-998 | etal.v1.set_n | workflow    | 1-8859-20c8d9b895 |         |            | 13:43:23   | 13:43:26      |
| -4adb-8a | 4-00e03bbdf | ode_state     | execution   | cb                |         |            |            |               |
| 08-352c3 | db8         |               |             |                   |         |            |            |               |
| 151965c  |             |               |             |                   |         |            |            |               |
| 08b899f5 | 9210744f-   | tripleo.plan_ |             | <none>            | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -0444-4e | 42f1-45de-  | management.v1 |             |                   |         |            | 14:09:19   | 14:09:26      |
| b8-8719- | 902a-a1e0a1 | .update_deplo |             |                   |         |            |            |               |
| a6eba9a8 | 4f91bb      | yment_plan    |             |                   |         |            |            |               |
| 1fa0     |             |               |             |                   |         |            |            |               |
| 1c291c68 | feef43e7-28 | tripleo.deplo |             | <none>            | SUCCESS | None       | 2016-09-27 | 2016-09-27    |
| -aec7-49 | 65-4123-b0e | yment.v1.depl |             |                   |         |            | 14:09:26   | 14:09:40      |
| e8-836f- | 9-f4eaf6d5d | oy_plan       |             |                   |         |            |            |               |
| 658b0676 | 77d         |               |             |                   |         |            |            |               |
| 3c92     |             |               |             |                   |         |            |            |               |
+----------+-------------+---------------+-------------+-------------------+---------+------------+------------+---------------+

[stack@instack ~]$ sudo route add -net 192.0.2.0/24 gw 192.0.2.1
[stack@instack ~]$ sudo route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.122.1   0.0.0.0         UG    0      0        0 eth0
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 vlan10
192.0.2.0       192.0.2.1       255.255.255.0   UG    0      0        0 br-ctlplane
192.0.2.0       0.0.0.0         255.255.255.0   U     0      0        0 br-ctlplane
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0


[root@overcloud-controller-0 ~]# nova service-list
+----+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host                                | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+
| 3  | nova-consoleauth | overcloud-controller-0.localdomain  | internal | enabled | up    | 2016-09-27T14:54:34.000000 | -               |
| 4  | nova-scheduler   | overcloud-controller-0.localdomain  | internal | enabled | up    | 2016-09-27T14:54:35.000000 | -               |
| 5  | nova-conductor   | overcloud-controller-0.localdomain  | internal | enabled | up    | 2016-09-27T14:54:27.000000 | -               |
| 6  | nova-compute     | overcloud-novacompute-0.localdomain | nova     | enabled | up    | 2016-09-27T14:54:26.000000 | -               |
| 7  | nova-compute     | overcloud-novacompute-1.localdomain | nova     | enabled | up    | 2016-09-27T14:54:27.000000 | -               |
+----+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+

****************************************************************************************************
Verification status VIP 10.0.0.4 via keepalived status on overcloud-controller-0.localdomain
****************************************************************************************************
[root@overcloud-controller-0 ~]# systemctl status keepalived -l
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2016-09-27 14:55:01 UTC; 2s ago
  Process: 18503 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 850 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/keepalived.service
           ├─18504 /usr/sbin/keepalived -D
           ├─18505 /usr/sbin/keepalived -D
           └─18506 /usr/sbin/keepalived -D

Sep 27 14:55:03 overcloud-controller-0 Keepalived_healthcheckers[18505]: Netlink reflector reports IP 192.0.2.13 added
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]: VRRP_Instance(51) Sending gratuitous ARPs on br-ex for 192.0.2.13
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]: VRRP_Instance(56) Entering MASTER STATE
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]: VRRP_Instance(56) setting protocol VIPs.
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]: VRRP_Instance(56) Sending gratuitous ARPs on vlan20 for 172.16.2.7
Sep 27 14:55:03 overcloud-controller-0 Keepalived_healthcheckers[18505]: Netlink reflector reports IP 172.16.2.7 added
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]: VRRP_Instance(52) Entering MASTER STATE
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]: VRRP_Instance(52) setting protocol VIPs.
Sep 27 14:55:03 overcloud-controller-0 Keepalived_vrrp[18506]: VRRP_Instance(52) Sending gratuitous ARPs on br-ex for 10.0.0.4
Sep 27 14:55:03 overcloud-controller-0 Keepalived_healthcheckers[18505]: Netlink reflector reports IP 10.0.0.4 added
 
  

  

********************************
Compute Node Status
********************************

[heat-admin@overcloud-novacompute-0 ~]$ sudo su -
Last login: Tue Sep 27 15:30:34 UTC 2016 on pts/0
[root@overcloud-novacompute-0 ~]# virsh --connect qemu:///system
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # version
Compiled against library: libvirt 1.2.17
Using library: libvirt 1.2.17
Using API: QEMU 1.2.17
Running hypervisor: QEMU 2.3.0

virsh # list --all
 Id    Name                           State
----------------------------------------------------
 3     instance-00000001              running 
 
 
    

No comments:

Post a Comment