Tuesday, February 6, 2018

Attempt of evaluation TripleO QuickStart (Master) overcloud containerized HA deployment on 32 GB VIRTHOST

UPDATE 03/04/18
Currently "current-passed-ci" trunk been used for deployment is under "Queens"
branch. Deployment for any template crashes at stage 4.0 with same errors
reports.I guess around March 15-16 issue would be fixed for sure.
Now attempt to deploy as follows 
[boris@fedora27workstation ~]$ bash ./tripleo-quickstart/quickstart.sh  \
                        --clean \
                        --release queens \
                        --teardown all \
                        --tags all \
                        -e @$CONFIG \
                         $VIRTHOST 
Files /etc/yum.repos.d generated on undercloud VM.

[root@undercloud yum.repos.d]# cat delorean.repo
[delorean]
name=delorean
baseurl=https://trunk.rdoproject.org/centos7-queens/45/aa
/45aa672926767592a4e8a26370e04501be9c5bb4_109973be/
gpgcheck=0
enabled=1
priority=20

[root@undercloud yum.repos.d]# cat delorean-deps.repo
[delorean-queens-testing]
name=dlrn-queens-testing
baseurl=https://trunk.rdoproject.org/centos7-master/deps/latest/
enabled=1
gpgcheck=0
[rdo-qemu-ev]
name=RDO CentOS-$releasever - QEMU EV
baseurl=http://mirror.centos.org/centos/7/virt/$basearch/kvm-common/
gpgcheck=0
enabled=1
skip_if_unavailable=1
END UPDATE 

UPDATE 02/09/18
Another the most recent deployment protocol
Same deployment done on 02/09/18
Compare timings for "Prepare for the containerized deployment" phase
Second was done almost two times faster.
The complete deployment procedure ended up with 480MB in swap area.
END UPDATE 
Following bellow is a sample of tuning deploy-configHA.yaml which allows to obtain
fairly functional overcloud topology with PCS HA Controller's cluster, then create
Networks, HA Router and launch F27 VM. Install on VM links text browser and surf the Net
pretty fast and smoothly. Sizes of PCS Controller's were tuned rather then default size
of undercloud as well as number VCPUS running undercloud VM. 
Initial swap area after deployment is 800 MB
Setup Repos 
[boris@fedora27workstation ~]$git clone \
   https://github.com/openstack/tripleo-quickstart
[boris@fedora27workstation ~]$bash ./tripleo-quickstart/quickstart.sh \
  --install-deps 
Configure Environment
[boris@fedora27workstation ~]$ export CONFIG= ~boris/deploy-configHA.yaml 
[boris@fedora27workstation ~]$ cat  ~boris/deploy-configHA.yaml 
# 32 GB VIRTHOST 
controller_memory: 7500
compute memory: 6500 
 overcloud_nodes:
  - name: control_0
    flavor: control
    virtualbmc_port: 6230
  - name: control_1
    flavor: control
    virtualbmc_port: 6231
  - name: control_2
    flavor: control
    virtualbmc_port: 6232

  - name: compute_0
    flavor: compute
    virtualbmc_port: 6233
node_count: 4
containerized_overcloud: true
delete_docker_cache: true
enable_pacemaker: true
run_tempest: false
extra_args: >-
  --libvirt-type qemu
  --ntp-server pool.ntp.org
  --control-scale 3
  --compute-scale 1
  -e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml
  -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml
[boris@fedora27workstation ~]$ export CONFIG=~boris/deploy-configHA.yaml
[boris@fedora27workstation ~]$ bash ./tripleo-quickstart/quickstart.sh  \
                        --clean \
                        --release master \
                        --teardown all \
                        --tags all \
                        -e @$CONFIG \
                         $VIRTHOST
. . . . . . . 

 
PLAY RECAP **************************************************************************************
192.168.0.74               : ok=141  changed=68   unreachable=0    failed=0   
localhost                  : ok=21   changed=10   unreachable=0    failed=1   
undercloud                 : ok=104  changed=65   unreachable=0    failed=0   

Wednesday 07 February 2018  00:38:54 +0300 (0:00:00.998)       4:32:08.488 **** 
=============================================================================== 
overcloud-prep-containers : Prepare for the containerized deployment ------------------ 6971.98s
/home/boris/.quickstart/usr/local/share/ansible/roles/overcloud-prep-containers/tasks/overcloud-prep-containers.yml:28 
overcloud-deploy : Deploy the overcloud ----------------------------------------------- 4465.27s
/home/boris/.quickstart/usr/local/share/ansible/roles/overcloud-deploy/tasks/deploy-overcloud.yml:1 
undercloud-deploy : Install the undercloud -------------------------------------------- 1843.86s
/home/boris/.quickstart/usr/local/share/ansible/roles/undercloud-deploy/tasks/install-undercloud.yml:20 
modify-image : Run virt-customize on the provided image -------------------------------- 523.19s
/home/boris/.quickstart/usr/local/share/ansible/roles/modify-image/tasks/libguestfs.yml:46 -----
tripleo-validations : Run validations tests through Mistral ---------------------------- 392.37s
/home/boris/.quickstart/usr/local/share/ansible/roles/tripleo-validations/tasks/main.yml:21 ----
overcloud-prep-images : Prepare the overcloud images for deploy ------------------------ 275.26s
/home/boris/.quickstart/usr/local/share/ansible/roles/overcloud-prep-images/tasks/overcloud-prep-images.yml:1 
convert-image : convert image ---------------------------------------------------------- 211.66s
/home/boris/.quickstart/tripleo-quickstart/roles/convert-image/tasks/main.yml:25 ---------------
tripleo-validations : Run validations tests through Mistral ---------------------------- 211.07s
/home/boris/.quickstart/usr/local/share/ansible/roles/tripleo-validations/tasks/main.yml:21 ----
tripleo-validations : Run validations tests through Mistral ---------------------------- 154.89s
/home/boris/.quickstart/usr/local/share/ansible/roles/tripleo-validations/tasks/main.yml:21 ----
setup/undercloud : Upload undercloud volume to storage pool ---------------------------- 100.16s
/home/boris/.quickstart/tripleo-quickstart/roles/libvirt/setup/undercloud/tasks/main.yml:280 ---
setup/undercloud : Perform selinux relabel on undercloud image -------------------------- 88.45s
/home/boris/.quickstart/tripleo-quickstart/roles/libvirt/setup/undercloud/tasks/main.yml:162 ---
setup/undercloud : Inject additional images --------------------------------------------- 49.26s
/home/boris/.quickstart/tripleo-quickstart/roles/libvirt/setup/undercloud/tasks/main.yml:67 ----
overcloud-prep-flavors : Prepare the scripts for overcloud flavors ---------------------- 45.31s
/home/boris/.quickstart/usr/local/share/ansible/roles/overcloud-prep-flavors/tasks/overcloud-prep-flavors.yml:1 
tripleo-validations : Run negative tests for pre-introspection group -------------------- 36.80s
/home/boris/.quickstart/usr/local/share/ansible/roles/tripleo-validations/tasks/main.yml:55 ----
validate-tempest : Install openstack services tempest plugins --------------------------- 33.27s
/home/boris/.quickstart/usr/local/share/ansible/roles/validate-tempest/tasks/pre-tempest.yml:32 
undercloud-deploy : Start the Virtual BMCs ---------------------------------------------- 32.41s
/home/boris/.quickstart/usr/local/share/ansible/roles/undercloud-deploy/tasks/configure-vbmc.yml:107 
setup/undercloud : iptables ------------------------------------------------------------- 20.50s
/home/boris/.quickstart/tripleo-quickstart/roles/libvirt/setup/undercloud/tasks/main.yml:387 ---
fetch-images : Get tar images from cache ------------------------------------------------ 19.26s
/home/boris/.quickstart/tripleo-quickstart/roles/fetch-images/tasks/fetch.yml:186 --------------
convert-image : Resize the undercloud image using qemu-image resize --------------------- 18.34s
/home/boris/.quickstart/tripleo-quickstart/roles/convert-image/tasks/main.yml:19 ---------------
undercloud-deploy : Create the Virtual BMCs --------------------------------------------- 14.55s
/home/boris/.quickstart/usr/local/share/ansible/roles/undercloud-deploy/tasks/configure-vbmc.yml:75 
+ set +x
##################################
Virtual Environment Setup Complete
##################################

Access the undercloud by:

    ssh -F /home/boris/.quickstart/ssh.config.ansible undercloud

Follow the documentation in the link below to complete your deployment.

    http://ow.ly/c44w304begR
##################################
Virtual Environment Setup Complete
##################################
 


[root@overcloud-controller-0 ~]# pcs status
Cluster name: tripleo_cluster
Stack: corosync
Current DC: overcloud-controller-1 (version 1.1.16-12.el7_4.7-94ff4df) - partition with quorum
Last updated: Tue Feb  6 21:44:16 2018
Last change: Tue Feb  6 21:28:38 2018 by root via cibadmin on overcloud-controller-0

12 nodes configured
37 resources configured

Online: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]
GuestOnline: [ galera-bundle-0@overcloud-controller-0 galera-bundle-1@overcloud-controller-1 galera-bundle-2@overcloud-controller-2 rabbitmq-bundle-0@overcloud-controller-0 rabbitmq-bundle-1@overcloud-controller-1 rabbitmq-bundle-2@overcloud-controller-2 redis-bundle-0@overcloud-controller-0 redis-bundle-1@overcloud-controller-1 redis-bundle-2@overcloud-controller-2 ]

Full list of resources:

 Docker container set: rabbitmq-bundle [192.168.24.1:8787/master/centos-binary-rabbitmq:pcmklatest]
   rabbitmq-bundle-0 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-controller-0
   rabbitmq-bundle-1 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-controller-1
   rabbitmq-bundle-2 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-controller-2
 Docker container set: galera-bundle [192.168.24.1:8787/master/centos-binary-mariadb:pcmklatest]
   galera-bundle-0 (ocf::heartbeat:galera): Master overcloud-controller-0
   galera-bundle-1 (ocf::heartbeat:galera): Master overcloud-controller-1
   galera-bundle-2 (ocf::heartbeat:galera): Master overcloud-controller-2
 Docker container set: redis-bundle [192.168.24.1:8787/master/centos-binary-redis:pcmklatest]
   redis-bundle-0 (ocf::heartbeat:redis): Master overcloud-controller-0
   redis-bundle-1 (ocf::heartbeat:redis): Slave overcloud-controller-1
   redis-bundle-2 (ocf::heartbeat:redis): Slave overcloud-controller-2
 ip-192.168.24.10 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0
 ip-10.0.0.5 (ocf::heartbeat:IPaddr2): Started overcloud-controller-1
 ip-172.16.2.7 (ocf::heartbeat:IPaddr2): Started overcloud-controller-2
 ip-172.16.2.11 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0
 ip-172.16.1.10 (ocf::heartbeat:IPaddr2): Started overcloud-controller-1
 ip-172.16.3.7 (ocf::heartbeat:IPaddr2): Started overcloud-controller-2
 Docker container set: haproxy-bundle [192.168.24.1:8787/master/centos-binary-haproxy:pcmklatest]
   haproxy-bundle-docker-0 (ocf::heartbeat:docker): Started overcloud-controller-0
   haproxy-bundle-docker-1 (ocf::heartbeat:docker): Started overcloud-controller-1
   haproxy-bundle-docker-2 (ocf::heartbeat:docker): Started overcloud-controller-2
 Docker container: openstack-cinder-volume [192.168.24.1:8787/master/centos-binary-cinder-volume:pcmklatest]
   openstack-cinder-volume-docker-0 (ocf::heartbeat:docker): Started overcloud-controller-0

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled
[root@overcloud-controller-0 ~]# vi overcloudrc
[root@overcloud-controller-0 ~]# . overcloudrc
(overcloud) [root@overcloud-controller-0 ~]# nova service-list
+--------------------------------------+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+
| Id                                   | Binary           | Host                                | Zone     | Status  | State | Updated_at                 | Disabled Reason | Forced down |
+--------------------------------------+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+
| d6dfd3d4-2cd1-4e3b-8ae9-9858c09d0b72 | nova-scheduler   | overcloud-controller-2.localdomain  | internal | enabled | up    | 2018-02-06T21:46:04.000000 | -               | False       |
| 2f0dc430-d65d-4c4e-8742-ba304ed7327e | nova-consoleauth | overcloud-controller-2.localdomain  | internal | enabled | up    | 2018-02-06T21:46:05.000000 | -               | False       |
| 7a6ee283-bfb9-437c-8aba-4409a9493ab6 | nova-scheduler   | overcloud-controller-0.localdomain  | internal | enabled | up    | 2018-02-06T21:46:04.000000 | -               | False       |
| 3a3b8465-d289-4fa4-9662-e2120f8e97f0 | nova-scheduler   | overcloud-controller-1.localdomain  | internal | enabled | up    | 2018-02-06T21:46:04.000000 | -               | False       |
| 5a5633d2-f7a1-4128-8af0-7ef19fe9a2f4 | nova-consoleauth | overcloud-controller-1.localdomain  | internal | enabled | up    | 2018-02-06T21:46:07.000000 | -               | False       |
| a8d96303-c142-4330-93f2-eee3d9e66fe5 | nova-consoleauth | overcloud-controller-0.localdomain  | internal | enabled | up    | 2018-02-06T21:46:11.000000 | -               | False       |
| f72a2749-f74d-4d7b-9db9-56ad8c1594e3 | nova-conductor   | overcloud-controller-2.localdomain  | internal | enabled | up    | 2018-02-06T21:46:07.000000 | -               | False       |
| 6bcd148c-d7b3-446e-9d93-36c82c7d3999 | nova-compute     | overcloud-novacompute-0.localdomain | nova     | enabled | up    | 2018-02-06T21:46:02.000000 | -               | False       |
| e39fe2be-68b5-43a2-8833-19816df83c89 | nova-conductor   | overcloud-controller-1.localdomain  | internal | enabled | up    | 2018-02-06T21:46:05.000000 | -               | False       |
| eb7be01e-2eb5-45fd-a171-90419556f16f | nova-conductor   | overcloud-controller-0.localdomain  | internal | enabled | up    | 2018-02-06T21:46:05.000000 | -               | False       |
+--------------------------------------+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+
(overcloud) [root@overcloud-controller-0 ~]# neutron agent-list
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------------+--------------------+-------------------------------------+-------------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host                                | availability_zone | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+-------------------------------------+-------------------+-------+----------------+---------------------------+
| 01e84d5f-16e6-4ecd-aeea-441374159ebf | L3 agent           | overcloud-controller-1.localdomain  | nova              | :-)   | True           | neutron-l3-agent          |
| 31cf8798-eb4c-4e9e-b0fc-866076433f5a | L3 agent           | overcloud-controller-2.localdomain  | nova              | :-)   | True           | neutron-l3-agent          |
| 42876c4c-1653-48ac-bb19-94b0bfdce143 | Metadata agent     | overcloud-controller-0.localdomain  |                   | :-)   | True           | neutron-metadata-agent    |
| 467d67f1-2c5a-487f-a065-18f4c4596473 | DHCP agent         | overcloud-controller-1.localdomain  | nova              | :-)   | True           | neutron-dhcp-agent        |
| 4d02ec5b-c980-4c28-b209-e33bc32cd532 | Metadata agent     | overcloud-controller-1.localdomain  |                   | :-)   | True           | neutron-metadata-agent    |
| 55b28c32-cee0-4876-acd5-2bfa353fb100 | Open vSwitch agent | overcloud-novacompute-0.localdomain |                   | :-)   | True           | neutron-openvswitch-agent |
| a00a55d2-4ff4-439b-abf4-557859a3ab89 | DHCP agent         | overcloud-controller-0.localdomain  | nova              | :-)   | True           | neutron-dhcp-agent        |
| a0a2186e-d562-4321-8bee-250996dd0647 | Open vSwitch agent | overcloud-controller-2.localdomain  |                   | :-)   | True           | neutron-openvswitch-agent |
| a4fcca87-0bca-4aab-89ef-882ee0da3e47 | Open vSwitch agent | overcloud-controller-0.localdomain  |                   | :-)   | True           | neutron-openvswitch-agent |
| ac5a2e4d-a4b5-45e8-ba8b-7e8be62aeb67 | Open vSwitch agent | overcloud-controller-1.localdomain  |                   | :-)   | True           | neutron-openvswitch-agent |
| af17a025-9587-4b31-8005-8cc2eb8c266e | Metadata agent     | overcloud-controller-2.localdomain  |                   | :-)   | True           | neutron-metadata-agent    |
| bb84495c-fe6c-495d-b69a-69a15183142b | DHCP agent         | overcloud-controller-2.localdomain  | nova              | :-)   | True           | neutron-dhcp-agent        |
| c757082f-b26b-4c50-8f00-8bb222d75b4c | L3 agent           | overcloud-controller-0.localdomain  | nova              | :-)   | True           | neutron-l3-agent          |
+--------------------------------------+--------------------+-------------------------------------+-------------------+-------+----------------+---------------------------+
(overcloud) [root@overcloud-controller-0 ~]# yum install wget -y
Loaded plugins: fastestmirror, priorities
Loading mirror speeds from cached hostfile
 * base: mirror.corbina.net
 * extras: mirror.corbina.net
 * updates: mirror.corbina.net
2 packages excluded due to repository priority protections
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-15.el7_4.1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================
 Package                  Arch                       Version                                Repository                   Size
==============================================================================================================================
Installing:
 wget                     x86_64                     1.14-15.el7_4.1                        updates                     547 k

Transaction Summary
==============================================================================================================================
Install  1 Package

Total download size: 547 k
Installed size: 2.0 M
Downloading packages:
wget-1.14-15.el7_4.1.x86_64.rpm                                                                        | 547 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : wget-1.14-15.el7_4.1.x86_64                                                                                1/1 
  Verifying  : wget-1.14-15.el7_4.1.x86_64                                                                                1/1 

Installed:
  wget.x86_64 0:1.14-15.el7_4.1                                                                                               

Complete!
(overcloud) [root@overcloud-controller-0 ~]# wget https://download.fedoraproject.org/pub/fedora/linux/releases/27/CloudImages/x86_64/images/Fedora-Cloud-Base-27-1.6.x86_64.qcow2 
--2018-02-06 21:49:15--  https://download.fedoraproject.org/pub/fedora/linux/releases/27/CloudImages/x86_64/images/Fedora-Cloud-Base-27-1.6.x86_64.qcow2
Resolving download.fedoraproject.org (download.fedoraproject.org)... 209.132.181.16, 185.141.165.254, 85.236.55.6, ...
Connecting to download.fedoraproject.org (download.fedoraproject.org)|209.132.181.16|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://mirror.yandex.ru/fedora/linux/releases/27/CloudImages/x86_64/images/Fedora-Cloud-Base-27-1.6.x86_64.qcow2 [following]
--2018-02-06 21:49:16--  https://mirror.yandex.ru/fedora/linux/releases/27/CloudImages/x86_64/images/Fedora-Cloud-Base-27-1.6.x86_64.qcow2
Resolving mirror.yandex.ru (mirror.yandex.ru)... 213.180.204.183, 2a02:6b8::183
Connecting to mirror.yandex.ru (mirror.yandex.ru)|213.180.204.183|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 232325120 (222M) [application/octet-stream]
Saving to: ‘Fedora-Cloud-Base-27-1.6.x86_64.qcow2’

100%[====================================================================================>] 232,325,120 3.16MB/s   in 67s    

2018-02-06 21:50:23 (3.30 MB/s) - ‘Fedora-Cloud-Base-27-1.6.x86_64.qcow2’ saved [232325120/232325120]

(overcloud) [root@overcloud-controller-0 ~]# openstack image create --disk-format qcow2 --container-format bare \
>  --public --file ./Fedora-Cloud-Base-27-1.6.x86_64.qcow2  VF27Cloud-image
+------------------+------------------------------------------------------------------------------+
| Field            | Value                                                                        |
+------------------+------------------------------------------------------------------------------+
| checksum         | e0713a4f26427691da90701ff7a823da                                             |
| container_format | bare                                                                         |
| created_at       | 2018-02-06T21:51:01Z                                                         |
| disk_format      | qcow2                                                                        |
| file             | /v2/images/884b6e90-2033-457d-93b8-c0f5880a54de/file                         |
| id               | 884b6e90-2033-457d-93b8-c0f5880a54de                                         |
| min_disk         | 0                                                                            |
| min_ram          | 0                                                                            |
| name             | VF27Cloud-image                                                              |
| owner            | 42d03a8472da43b49d2f233ada15d8aa                                             |
| properties       | direct_url='swift+config://ref1/glance/884b6e90-2033-457d-93b8-c0f5880a54de' |
| protected        | False                                                                        |
| schema           | /v2/schemas/image                                                            |
| size             | 232325120                                                                    |
| status           | active                                                                       |
| tags             |                                                                              |
| updated_at       | 2018-02-06T21:51:23Z                                                         |
| virtual_size     | None                                                                         |
| visibility       | public                                                                       |
+------------------+------------------------------------------------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack image list
+--------------------------------------+-----------------+--------+
| ID                                   | Name            | Status |
+--------------------------------------+-----------------+--------+
| 884b6e90-2033-457d-93b8-c0f5880a54de | VF27Cloud-image | active |
+--------------------------------------+-----------------+--------+
(overcloud) [root@overcloud-controller-0 ~]# openstack flavor create --public m1.small --id 2 \
>   --ram 2000 --disk 20 --vcpus 1 --rxtx-factor 1
+----------------------------+----------+
| Field                      | Value    |
+----------------------------+----------+
| OS-FLV-DISABLED:disabled   | False    |
| OS-FLV-EXT-DATA:ephemeral  | 0        |
| disk                       | 20       |
| id                         | 2        |
| name                       | m1.small |
| os-flavor-access:is_public | True     |
| properties                 |          |
| ram                        | 2000     |
| rxtx_factor                | 1.0      |
| swap                       |          |
| vcpus                      | 1        |
+----------------------------+----------+
(overcloud) [root@overcloud-controller-0 ~]# openstack keypair create oskey070218 > oskey070218.pem
(overcloud) [root@overcloud-controller-0 ~]# openstack  network create ext-net --external \
> --provider-physical-network datacentre  --provider-network-type flat
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | UP                                   |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2018-02-06T21:55:34Z                 |
| description               |                                      |
| dns_domain                | None                                 |
| id                        | 8ea290af-30bb-4531-910f-520794f7f71b |
| ipv4_address_scope        | None                                 |
| ipv6_address_scope        | None                                 |
| is_default                | False                                |
| is_vlan_transparent       | None                                 |
| mtu                       | 1500                                 |
| name                      | ext-net                              |
| port_security_enabled     | True                                 |
| project_id                | 42d03a8472da43b49d2f233ada15d8aa     |
| provider:network_type     | flat                                 |
| provider:physical_network | datacentre                           |
| provider:segmentation_id  | None                                 |
| qos_policy_id             | None                                 |
| revision_number           | 4                                    |
| router:external           | External                             |
| segments                  | None                                 |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| updated_at                | 2018-02-06T21:55:34Z                 |
+---------------------------+--------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack subnet create ext-subnet --network ext-net \
>  --subnet-range 192.168.24.0/24  --allocation-pool start=192.168.24.100,end=192.168.24.120 \
>  --no-dhcp --gateway 192.168.24.1
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| allocation_pools  | 192.168.24.100-192.168.24.120        |
| cidr              | 192.168.24.0/24                      |
| created_at        | 2018-02-06T21:56:03Z                 |
| description       |                                      |
| dns_nameservers   |                                      |
| enable_dhcp       | False                                |
| gateway_ip        | 192.168.24.1                         |
| host_routes       |                                      |
| id                | d73fef86-8b9a-4839-b4c5-af37973e334e |
| ip_version        | 4                                    |
| ipv6_address_mode | None                                 |
| ipv6_ra_mode      | None                                 |
| name              | ext-subnet                           |
| network_id        | 8ea290af-30bb-4531-910f-520794f7f71b |
| project_id        | 42d03a8472da43b49d2f233ada15d8aa     |
| revision_number   | 0                                    |
| segment_id        | None                                 |
| service_types     |                                      |
| subnetpool_id     | None                                 |
| tags              |                                      |
| updated_at        | 2018-02-06T21:56:03Z                 |
+-------------------+--------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack router create RouterDSA
+-------------------------+--------------------------------------+
| Field                   | Value                                |
+-------------------------+--------------------------------------+
| admin_state_up          | UP                                   |
| availability_zone_hints |                                      |
| availability_zones      |                                      |
| created_at              | 2018-02-06T21:56:39Z                 |
| description             |                                      |
| distributed             | False                                |
| external_gateway_info   | None                                 |
| flavor_id               | None                                 |
| ha                      | True                                 |
| id                      | 3875a372-76f2-4366-9d89-27ff339808a9 |
| name                    | RouterDSA                            |
| project_id              | 42d03a8472da43b49d2f233ada15d8aa     |
| revision_number         | None                                 |
| routes                  |                                      |
| status                  | ACTIVE                               |
| tags                    |                                      |
| updated_at              | 2018-02-06T21:56:39Z                 |
+-------------------------+--------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack router set RouterDSA --external-gateway ext-net
(overcloud) [root@overcloud-controller-0 ~]# openstack network create internal01
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | UP                                   |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2018-02-06T21:57:49Z                 |
| description               |                                      |
| dns_domain                | None                                 |
| id                        | 12ecd2bc-5428-4b0f-8b61-d53253d19a11 |
| ipv4_address_scope        | None                                 |
| ipv6_address_scope        | None                                 |
| is_default                | False                                |
| is_vlan_transparent       | None                                 |
| mtu                       | 1450                                 |
| name                      | internal01                           |
| port_security_enabled     | True                                 |
| project_id                | 42d03a8472da43b49d2f233ada15d8aa     |
| provider:network_type     | vxlan                                |
| provider:physical_network | None                                 |
| provider:segmentation_id  | 96                                   |
| qos_policy_id             | None                                 |
| revision_number           | 3                                    |
| router:external           | Internal                             |
| segments                  | None                                 |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| updated_at                | 2018-02-06T21:57:49Z                 |
+---------------------------+--------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack subnet create sub-internal01 --network internal01 \
>  --subnet-range 50.0.0.0/24  --dns-nameserver  83.221.202.254
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| allocation_pools  | 50.0.0.2-50.0.0.254                  |
| cidr              | 50.0.0.0/24                          |
| created_at        | 2018-02-06T21:58:19Z                 |
| description       |                                      |
| dns_nameservers   | 83.221.202.254                       |
| enable_dhcp       | True                                 |
| gateway_ip        | 50.0.0.1                             |
| host_routes       |                                      |
| id                | 0ec616d7-f281-445f-b143-96e696d9fcfe |
| ip_version        | 4                                    |
| ipv6_address_mode | None                                 |
| ipv6_ra_mode      | None                                 |
| name              | sub-internal01                       |
| network_id        | 12ecd2bc-5428-4b0f-8b61-d53253d19a11 |
| project_id        | 42d03a8472da43b49d2f233ada15d8aa     |
| revision_number   | 0                                    |
| segment_id        | None                                 |
| service_types     |                                      |
| subnetpool_id     | None                                 |
| tags              |                                      |
| updated_at        | 2018-02-06T21:58:19Z                 |
+-------------------+--------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack router add subnet RouterDSA 0ec616d7-f281-445f-b143-96e696d9fcfe


(overcloud) [root@overcloud-controller-0 ~]# openstack project list
+----------------------------------+---------+
| ID                               | Name    |
+----------------------------------+---------+
| 42d03a8472da43b49d2f233ada15d8aa | admin   |
| 97c20ef2e67042319c971ff84ef0d3e1 | service |
+----------------------------------+---------+
(overcloud) [root@overcloud-controller-0 ~]# openstack security group list
+--------------------------------------+---------+------------------------+----------------------------------+
| ID                                   | Name    | Description            | Project                          |
+--------------------------------------+---------+------------------------+----------------------------------+
| 7aaa6fa7-8d63-4e7c-a5bc-2959356452ce | default | Default security group |                                  |
| ef8ecb55-6299-4543-8c75-a2040311454d | default | Default security group | 42d03a8472da43b49d2f233ada15d8aa |
+--------------------------------------+---------+------------------------+----------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack security group rule create ef8ecb55-6299-4543-8c75-a2040311454d \
>  --protocol tcp --dst-port 22:22 --remote-ip 0.0.0.0/0
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| created_at        | 2018-02-06T22:01:24Z                 |
| description       |                                      |
| direction         | ingress                              |
| ether_type        | IPv4                                 |
| id                | 274a9e4e-e5a6-4eb5-89fc-362611e852ef |
| name              | None                                 |
| port_range_max    | 22                                   |
| port_range_min    | 22                                   |
| project_id        | 42d03a8472da43b49d2f233ada15d8aa     |
| protocol          | tcp                                  |
| remote_group_id   | None                                 |
| remote_ip_prefix  | 0.0.0.0/0                            |
| revision_number   | 0                                    |
| security_group_id | ef8ecb55-6299-4543-8c75-a2040311454d |
| updated_at        | 2018-02-06T22:01:24Z                 |
+-------------------+--------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack image list
+--------------------------------------+-----------------+--------+
| ID                                   | Name            | Status |
+--------------------------------------+-----------------+--------+
| 884b6e90-2033-457d-93b8-c0f5880a54de | VF27Cloud-image | active |
+--------------------------------------+-----------------+--------+
(overcloud) [root@overcloud-controller-0 ~]# openstack network list
+--------------------------------------+----------------------------------------------------+--------------------------------------+
| ID                                   | Name                                               | Subnets                              |
+--------------------------------------+----------------------------------------------------+--------------------------------------+
| 12ecd2bc-5428-4b0f-8b61-d53253d19a11 | internal01                                         | 0ec616d7-f281-445f-b143-96e696d9fcfe |
| 60c6a1a6-617a-4131-ba98-bf58976637a8 | HA network tenant 42d03a8472da43b49d2f233ada15d8aa | bcb768ef-442a-4912-a482-23ba24252030 |
| 8ea290af-30bb-4531-910f-520794f7f71b | ext-net                                            | d73fef86-8b9a-4839-b4c5-af37973e334e |
+--------------------------------------+----------------------------------------------------+--------------------------------------+

(overcloud) [root@overcloud-controller-0 ~]# nova keypair-list
+-------------+------+-------------------------------------------------+
| Name        | Type | Fingerprint                                     |
+-------------+------+-------------------------------------------------+
| oskey070218 | ssh  | 25:54:f8:1d:dd:ff:c9:6c:73:d1:1d:be:0c:79:cf:2f |
+-------------+------+-------------------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack image list
+--------------------------------------+-----------------+--------+
| ID                                   | Name            | Status |
+--------------------------------------+-----------------+--------+
| 884b6e90-2033-457d-93b8-c0f5880a54de | VF27Cloud-image | active |
+--------------------------------------+-----------------+--------+
(overcloud) [root@overcloud-controller-0 ~]# openstack network list
+--------------------------------------+----------------------------------------------------+--------------------------------------+
| ID                                   | Name                                               | Subnets                              |
+--------------------------------------+----------------------------------------------------+--------------------------------------+
| 12ecd2bc-5428-4b0f-8b61-d53253d19a11 | internal01                                         | 0ec616d7-f281-445f-b143-96e696d9fcfe |
| 60c6a1a6-617a-4131-ba98-bf58976637a8 | HA network tenant 42d03a8472da43b49d2f233ada15d8aa | bcb768ef-442a-4912-a482-23ba24252030 |
| 8ea290af-30bb-4531-910f-520794f7f71b | ext-net                                            | d73fef86-8b9a-4839-b4c5-af37973e334e |
+--------------------------------------+----------------------------------------------------+--------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack server create --flavor 2 --key-name oskey070218  \
> --image 884b6e90-2033-457d-93b8-c0f5880a54de \
> --nic net-id=12ecd2bc-5428-4b0f-8b61-d53253d19a11 ServerFDR27
+-------------------------------------+--------------------------------------------------------+
| Field                               | Value                                                  |
+-------------------------------------+--------------------------------------------------------+
| OS-DCF:diskConfig                   | MANUAL                                                 |
| OS-EXT-AZ:availability_zone         |                                                        |
| OS-EXT-SRV-ATTR:host                | None                                                   |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                   |
| OS-EXT-SRV-ATTR:instance_name       |                                                        |
| OS-EXT-STS:power_state              | NOSTATE                                                |
| OS-EXT-STS:task_state               | scheduling                                             |
| OS-EXT-STS:vm_state                 | building                                               |
| OS-SRV-USG:launched_at              | None                                                   |
| OS-SRV-USG:terminated_at            | None                                                   |
| accessIPv4                          |                                                        |
| accessIPv6                          |                                                        |
| addresses                           |                                                        |
| adminPass                           | eD7Xxd3cRPs7                                           |
| config_drive                        |                                                        |
| created                             | 2018-02-06T22:10:51Z                                   |
| flavor                              | m1.small (2)                                           |
| hostId                              |                                                        |
| id                                  | f89e34bc-6fb1-4556-8490-bd2c1844c1a6                   |
| image                               | VF27Cloud-image (884b6e90-2033-457d-93b8-c0f5880a54de) |
| key_name                            | oskey070218                                            |
| name                                | ServerFDR27                                            |
| progress                            | 0                                                      |
| project_id                          | 42d03a8472da43b49d2f233ada15d8aa                       |
| properties                          |                                                        |
| security_groups                     | name='default'                                         |
| status                              | BUILD                                                  |
| updated                             | 2018-02-06T22:10:51Z                                   |
| user_id                             | eb9b8cd6e4de434b9206babca19577d1                       |
| volumes_attached                    |                                                        |
+-------------------------------------+--------------------------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# nova list
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
| ID                                   | Name        | Status | Task State | Power State | Networks             |
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
| f89e34bc-6fb1-4556-8490-bd2c1844c1a6 | ServerFDR27 | BUILD  | spawning   | NOSTATE     | internal01=50.0.0.14 |
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
(overcloud) [root@overcloud-controller-0 ~]# nova list
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
| ID                                   | Name        | Status | Task State | Power State | Networks             |
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
| f89e34bc-6fb1-4556-8490-bd2c1844c1a6 | ServerFDR27 | ACTIVE | -          | Running     | internal01=50.0.0.14 |
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
(overcloud) [root@overcloud-controller-0 ~]# ls -l
total 226904
-rw-------. 1 root root      6921 Jan  8 16:38 anaconda-ks.cfg
-rw-r--r--. 1 root root 232325120 Nov  5 07:29 Fedora-Cloud-Base-27-1.6.x86_64.qcow2
-rw-------. 1 root root      6577 Jan  8 16:38 original-ks.cfg
-rw-r--r--. 1 root root      1676 Feb  6 21:55 oskey070218.pem
-rw-r--r--. 1 root root       905 Feb  6 21:45 overcloudrc
(overcloud) [root@overcloud-controller-0 ~]# cat oskey070218.pem
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAxoVlRsf01KRUgKIIvv9wfX+cM72xH0tz0LEiyzM4cRYn+iO1
XFd1PS2H1LEY5Vm6ZQODI7KfEga3FBqZzhPoDf99hhW5oJfgMLMgtohvSle9RYsM
qFB+nW9lyoHd2cN3HW/UvkTubS1MyLLQsivqLo7eMD3aHo15xZc2u4w1chQwX0kf
WouS5MulK/n/eTJEVPdmIaStCTcHtI73QbaIAf62PiYJakuQkHZ7WlxfUN8OssfO
oRhcTsexOcR2+mg/uN0XHCH1E2fIUuK4Fu7Pq3sgkzrrkrPztMCqY2slucQTIFYH
MK1+7ZWs/ZKGVG7mAtvVk6c7YZay8P6yOFVgTQIDAQABAoIBABlxGRSkJaVm1gmS
Ooq30+PRieQ7qEp5cgNJ4EqoatBbZY4yK1kFQFoxzpmchLNH95Jgi9r0VlyKDcZ+
zj1R4ciBeMXJXPvO2wWCv2uFOrYNXo1eD6Y2grOMhkAYuqTg3xAft+UWknPdfVyv
Rf2/kQiceK84Dg8iVHvVMEeC5YqT//qB3+m5r2mNuFA+VD0fFFmBamPgO3NqEUlR
+oj+ukZfLC+LuIBpakRHuvpji9Buh0INwKNJt2m59pfF0/S+gxE3pIAYCV+hzTZB
+uRN4zxE7l280kdtEDrMv5vE1h4MeHSIjvjOKTuOf+rg4q/G+K2c69Bc+2/EJZO7
sEB2D6UCgYEA6P3+j1XkXOu+HF4mfPH8TOUkXqDMW0SnvrslBbMQMJiL3FS0ZFLX
CJ9JTibyTxNn7m4EDguq2YHPTdJYu6QPIvlt4HRMUxcE/LMk6Fx1Iir2yqqu49Nh
uaixrX9H2MkFYDYdTfNNjvhSewXSlMyP8htKHuNWnLy0sJw9VfAOWFMCgYEA2h/6
PEWWYbwPCC4nQPPM+s1fwgAcGLhptmpdLUxou8dnfuhhRpmjFWCzxH/AEkCf8q78
i6tkludigmEo3glPsyS6SzzNsZoFry88DeYJpeCVkVqoYEDRy5eoIvHXGr5f1JCE
4JP+jZu4jvsML8+xGOOmRkU+2yd4mXQkNP1B0N8CgYEAsoC2INaraM1sD4vKbPtw
UvFTVfRX3KzS33u9t4oiozicHqf+tMYK3R48KVYJWrABdlhvXtttf8rGKlc3iDr/
Jcp3dflj6FzAg2I0qpj7XLwR69+ssftm6bjmt4fGYb5REg8jkG8RQrHZb0tctIes
2M4zkUqXNuoUfSKWk1qhguUCgYA+Bbe2qabQFZREpDL/MhPIANvIoxVaJh3RA7+I
4OAq/NE98b+vPoJz0rJrHlQ0HHIepIklAdAxZIvLPOtM72LdMoVSXXCznnMdVBYE
cbOumgSNdzTHYccF2BK0FQYwEY/YNlvCK/LGu0+oQFQ/yBO4WwrtkkUENd4B7Fdb
ReQQCwKBgCDnhunm0ONoZgaMCYYukTGYvFngku0SQAFMQeU/ioJ3fsj9RLgP09vS
1RFV80B9+AAcuvYqcXHeHRgduSv7T/QZWatoomhSl//+di890nKvQpJKdjh0Csv/
IpTAuLbQrMe5gSINMgBNWkxrVhBOoRY1bt+t9fwTIOOoPO+tkxUl
-----END RSA PRIVATE KEY-----

(overcloud) [root@overcloud-controller-0 ~]# nova list
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
| ID                                   | Name        | Status | Task State | Power State | Networks             |
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
| f89e34bc-6fb1-4556-8490-bd2c1844c1a6 | ServerFDR27 | ACTIVE | -          | Running     | internal01=50.0.0.14 |
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack console log show ServerFDR27
[    0.000000] random: get_random_bytes called from start_kernel+0x42/0x4e6 with crng_init=0
[    0.000000] Linux version 4.13.9-300.fc27.x86_64 (mockbuild@bkernel01.phx2.fedoraproject.org) (gcc version 7.2.1 20170915 (Red Hat 7.2.1-2) (GCC)) #1 SMP Mon Oct 23 13:41:58 UTC 2017
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.13.9-300.fc27.x86_64 root=UUID=d020d197-657b-4030-bde5-0db8417bc39c ro no_timer_check console=tty1 console=ttyS0,115200n8 LANG=en_US.UTF-8
[    0.000000] x86/fpu: x87 FPU will use FXSAVE
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007cfdbfff] usable
[    0.000000] BIOS-e820: [mem 0x000000007cfdc000-0x000000007cffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] random: fast init done
[    0.000000] SMBIOS 2.8 present.
[    0.000000] DMI: RDO OpenStack Compute, BIOS 1.10.2-3.el7_4.1 04/01/2014
[    0.000000] tsc: Fast TSC calibration using PIT
[    0.000000] e820: last_pfn = 0x7cfdc max_arch_pfn = 0x400000000
[    0.000000] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WC  UC- WT  
[    0.000000] found SMP MP-table at [mem 0x000f72f0-0x000f72ff] mapped at [ffff9a5e800f72f0]
[    0.000000] RAMDISK: [mem 0x35a31000-0x36d10fff]
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x00000000000F70A0 000014 (v00 BOCHS )
[    0.000000] ACPI: RSDT 0x000000007CFE14C9 00002C (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
[    0.000000] ACPI: FACP 0x000000007CFE13DD 000074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
[    0.000000] ACPI: DSDT 0x000000007CFE0040 00139D (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
[    0.000000] ACPI: FACS 0x000000007CFE0000 000040
[    0.000000] ACPI: APIC 0x000000007CFE1451 000078 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at [mem 0x0000000000000000-0x000000007cfdbfff]
[    0.000000] NODE_DATA(0) allocated [mem 0x7cfb1000-0x7cfdbfff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.000000]   DMA32    [mem 0x0000000001000000-0x000000007cfdbfff]
[    0.000000]   Normal   empty
[    0.000000]   Device   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.000000]   node   0: [mem 0x0000000000100000-0x000000007cfdbfff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000007cfdbfff]
[    0.000000] ACPI: PM-Timer IO Port: 0x608
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[    0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
. . . . . . . . . . . . . . . 
         Starting Initial cloud-init job (metadata service crawler)...
[[0;32m  OK  [0m] Reached target Network.
[  211.118245] random: crng init done
[  226.377715] EXT4-fs (vda1): resizing filesystem from 1048320 to 5242624 blocks
[  227.090992] EXT4-fs (vda1): resized filesystem to 5242624
         Starting Hostname Service...
[[0;32m  OK  [0m] Started Hostname Service.
[  243.419973] cloud-init[688]: Cloud-init v. 0.7.9 running 'init' at Tue, 06 Feb 2018 22:14:49 +0000. Up 167.89 seconds.
[  243.440648] cloud-init[688]: ci-info: +++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++
[  243.447390] cloud-init[688]: ci-info: +--------+------+-----------+---------------+-------+-------------------+
[  243.461861] cloud-init[688]: ci-info: | Device |  Up  |  Address  |      Mask     | Scope |     Hw-Address    |
[  243.472326] cloud-init[688]: ci-info: +--------+------+-----------+---------------+-------+-------------------+
[  243.484967] cloud-init[688]: ci-info: | eth0:  | True | 50.0.0.14 | 255.255.255.0 |   .   | fa:16:3e:4d:42:7d |
[  243.499641] cloud-init[688]: ci-info: | eth0:  | True |     .     |       .       |   d   | fa:16:3e:4d:42:7d |
[  243.512412] cloud-init[688]: ci-info: |  lo:   | True | 127.0.0.1 |   255.0.0.0   |   .   |         .         |
[  243.522629] cloud-init[688]: ci-info: |  lo:   | True |     .     |       .       |   d   |         .         |
[  243.531991] cloud-init[688]: ci-info: +--------+------+-----------+---------------+-------+-------------------+
[  243.547778] cloud-init[688]: ci-info: ++++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++++
[  243.569460] cloud-init[688]: ci-info: +-------+-----------------+----------+-----------------+-----------+-------+
[  243.575745] cloud-init[688]: ci-info: | Route |   Destination   | Gateway  |     Genmask     | Interface | Flags |
[  243.588100] cloud-init[688]: ci-info: +-------+-----------------+----------+-----------------+-----------+-------+
[  243.591585] cloud-init[688]: ci-info: |   0   |     0.0.0.0     | 50.0.0.1 |     0.0.0.0     |    eth0   |   UG  |
[  243.605635] cloud-init[688]: ci-info: |   1   |     50.0.0.0    | 0.0.0.0  |  255.255.255.0  |    eth0   |   U   |
[  243.612943] cloud-init[688]: ci-info: |   2   | 169.254.169.254 | 50.0.0.1 | 255.255.255.255 |    eth0   |  UGH  |
[  243.618058] cloud-init[688]: ci-info: +-------+-----------------+----------+-----------------+-----------+-------+
[[0;32m  OK  [0m] Started Initial cloud-init job (metadata service crawler).
[[0;32m  OK  [0m] Reached target Cloud-config availability.
         Starting Permit User Sessions...
[[0;32m  OK  [0m] Reached target Network is Online.
         Starting Apply the settings specified in cloud-config...
         Starting OpenSSH server daemon...
[[0;32m  OK  [0m] Started Permit User Sessions.
[[0;32m  OK  [0m] Started Serial Getty on ttyS0.
[[0;32m  OK  [0m] Started Getty on tty1.
[[0;32m  OK  [0m] Reached target Login Prompts.
[[0;32m  OK  [0m] Started Command Scheduler.
[[0;32m  OK  [0m] Started OpenSSH server daemon.
[[0;32m  OK  [0m] Reached target Multi-User System.
         Starting Update UTMP about System Runlevel Changes...
[[0;32m  OK  [0m] Started Update UTMP about System Runlevel Changes.

Fedora 27 (Cloud Edition)
Kernel 4.13.9-300.fc27.x86_64 on an x86_64 (ttyS0)

serverfdr27 login: (overcloud) [root@overcloud-controller-0 ~]# nova list
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
| ID                                   | Name        | Status | Task State | Power State | Networks             |
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
| f89e34bc-6fb1-4556-8490-bd2c1844c1a6 | ServerFDR27 | ACTIVE | -          | Running     | internal01=50.0.0.14 |
+--------------------------------------+-------------+--------+------------+-------------+----------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack floating ip create ext-net
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| created_at          | 2018-02-06T22:17:28Z                 |
| description         |                                      |
| fixed_ip_address    | None                                 |
| floating_ip_address | 192.168.24.109                       |
| floating_network_id | 8ea290af-30bb-4531-910f-520794f7f71b |
| id                  | 1776a8c6-5da7-49df-afca-f531aa1e22c7 |
| name                | 192.168.24.109                       |
| port_id             | None                                 |
| project_id          | 42d03a8472da43b49d2f233ada15d8aa     |
| revision_number     | 0                                    |
| router_id           | None                                 |
| status              | DOWN                                 |
| updated_at          | 2018-02-06T22:17:28Z                 |
+---------------------+--------------------------------------+
(overcloud) [root@overcloud-controller-0 ~]# openstack server add floating ip ServerFDR27 192.168.24.109
(overcloud) [root@overcloud-controller-0 ~]# nova list
+--------------------------------------+-------------+--------+------------+-------------+--------------------------------------+
| ID                                   | Name        | Status | Task State | Power State | Networks                             |
+--------------------------------------+-------------+--------+------------+-------------+--------------------------------------+
| f89e34bc-6fb1-4556-8490-bd2c1844c1a6 | ServerFDR27 | ACTIVE | -          | Running     | internal01=50.0.0.14, 192.168.24.109 |
+--------------------------------------+-------------+--------+------------+-------------+--------------------------------------+

(overcloud) [root@overcloud-controller-0 ~]# date
Tue Feb  6 22:29:04 UTC 2018
(overcloud) [root@overcloud-controller-0 ~]# nova --version
9.1.0
(overcloud) [root@overcloud-controller-0 ~]# neutron l3-agent-list-hosting-router RouterDSA
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------------+------------------------------------+----------------+-------+----------+
| id                                   | host                               | admin_state_up | alive | ha_state |
+--------------------------------------+------------------------------------+----------------+-------+----------+
| 01e84d5f-16e6-4ecd-aeea-441374159ebf | overcloud-controller-1.localdomain | True           | :-)   | standby  |
| 31cf8798-eb4c-4e9e-b0fc-866076433f5a | overcloud-controller-2.localdomain | True           | :-)   | active   |
| c757082f-b26b-4c50-8f00-8bb222d75b4c | overcloud-controller-0.localdomain | True           | :-)   | standby  |
+--------------------------------------+------------------------------------+----------------+-------+----------+
*********************************************************************** 
Verify PCS Cluster status again having all job descibed above done.
*********************************************************************** 
(overcloud) [root@overcloud-controller-0 ~]# pcs status
Cluster name: tripleo_cluster
Stack: corosync
Current DC: overcloud-controller-1 (version 1.1.16-12.el7_4.7-94ff4df) - partition with quorum
Last updated: Tue Feb  6 22:30:05 2018
Last change: Tue Feb  6 21:28:38 2018 by root via cibadmin on overcloud-controller-0

12 nodes configured
37 resources configured

Online: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]
GuestOnline: [ galera-bundle-0@overcloud-controller-0 galera-bundle-1@overcloud-controller-1 galera-bundle-2@overcloud-controller-2 rabbitmq-bundle-0@overcloud-controller-0 rabbitmq-bundle-1@overcloud-controller-1 rabbitmq-bundle-2@overcloud-controller-2 redis-bundle-0@overcloud-controller-0 redis-bundle-1@overcloud-controller-1 redis-bundle-2@overcloud-controller-2 ]

Full list of resources:

 Docker container set: rabbitmq-bundle [192.168.24.1:8787/master/centos-binary-rabbitmq:pcmklatest]
   rabbitmq-bundle-0 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-controller-0
   rabbitmq-bundle-1 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-controller-1
   rabbitmq-bundle-2 (ocf::heartbeat:rabbitmq-cluster): Started overcloud-controller-2
 Docker container set: galera-bundle [192.168.24.1:8787/master/centos-binary-mariadb:pcmklatest]
   galera-bundle-0 (ocf::heartbeat:galera): Master overcloud-controller-0
   galera-bundle-1 (ocf::heartbeat:galera): Master overcloud-controller-1
   galera-bundle-2 (ocf::heartbeat:galera): Master overcloud-controller-2
 Docker container set: redis-bundle [192.168.24.1:8787/master/centos-binary-redis:pcmklatest]
   redis-bundle-0 (ocf::heartbeat:redis): Master overcloud-controller-0
   redis-bundle-1 (ocf::heartbeat:redis): Slave overcloud-controller-1
   redis-bundle-2 (ocf::heartbeat:redis): Slave overcloud-controller-2
 ip-192.168.24.10 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0
 ip-10.0.0.5 (ocf::heartbeat:IPaddr2): Started overcloud-controller-1
 ip-172.16.2.7 (ocf::heartbeat:IPaddr2): Started overcloud-controller-2
 ip-172.16.2.11 (ocf::heartbeat:IPaddr2): Started overcloud-controller-0
 ip-172.16.1.10 (ocf::heartbeat:IPaddr2): Started overcloud-controller-1
 ip-172.16.3.7 (ocf::heartbeat:IPaddr2): Started overcloud-controller-2
 Docker container set: haproxy-bundle [192.168.24.1:8787/master/centos-binary-haproxy:pcmklatest]
   haproxy-bundle-docker-0 (ocf::heartbeat:docker): Started overcloud-controller-0
   haproxy-bundle-docker-1 (ocf::heartbeat:docker): Started overcloud-controller-1
   haproxy-bundle-docker-2 (ocf::heartbeat:docker): Started overcloud-controller-2
 Docker container: openstack-cinder-volume [192.168.24.1:8787/master/centos-binary-cinder-volume:pcmklatest]
   openstack-cinder-volume-docker-0 (ocf::heartbeat:docker): Started overcloud-controller-0

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled

Initial Virthost Status

***************************************************** Links session runing on launched VM ServerFDR27
***************************************************** 
Run on F27 WKS sshuttle as follows:
$ sshuttle -e "ssh -F $HOME/.quickstart/ssh.config.ansible" \
-r undercloud -v 10.0.0.0/24  192.168.24.0/24

Copy and paste oskey060228.pem from Controller_0 to F27 WKS
and establish connection to VM running in Overcloud :-
$ ssh -i oskey070228.pem fedora@192.168.24.109
Inside VM install text Web Browser Links and verify
via surfing the Net. 
 


    Final VIRTHOST status.





   References

Monday, January 29, 2018

TripleO QuickStart (Master) overcloud containerized HA deployment ( 3 Nodes HA PCS Cluster + 1 Compute Node )

In general, we follow Carslos's Camacho New TripleO quickstart cheatsheet 
However, syntax of deploy-config.yaml template was changed to 
handle three node PCS Controllers HA cluster deployment. Also standard configuration  
connecting Fedora WorkStation 27 to VIRTHOST (32 GB) stack account has 
been used as usual. Containerized Deployment finishes up with 1.2 GB
Swap area. So it's just a POC done on 32 GB VIRTHOST. System is too slow for testing.
At least 48 (64) GB are needed to perform to networks&&Router creation and launching VMs
in overcloud. Undercloud requires 12 GB 6 VCPUS for deployment fast enough system,
which might be tested like in was done in post.  
 TripleO QuickStart (Master) overcloud containerized deployment with several compute nodes 

Each overcloud node should have at least 8 GB RAM and 2 VCPUS.
Decreasing Node RAM just to 7.8 GB on my system causes failure to set up PCS Cluster
[boris@fedora27workstation ~]$ cat  ~boris/deploy-configHA.yaml 
# 32 GB VIRTHOST 
undercloud_memory 10000 
 overcloud_nodes:
  - name: control_0
    flavor: control
    virtualbmc_port: 6230
  - name: control_1
    flavor: control
    virtualbmc_port: 6231
  - name: control_2
    flavor: control
    virtualbmc_port: 6232

  - name: compute_0
    flavor: compute
    virtualbmc_port: 6233
node_count: 4
containerized_overcloud: true
delete_docker_cache: true
enable_pacemaker: true
run_tempest: false
extra_args: >-
  --libvirt-type qemu
  --ntp-server pool.ntp.org
  --control-scale 3
  --compute-scale 1
  -e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml
  -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml
[boris@fedora27workstation ~]$ export CONFIG=~boris/deploy-configHA.yaml
[boris@fedora27workstation ~]$ bash ./tripleo-quickstart/quickstart.sh  \
                        --clean \
                        --release master \
                        --teardown all \
                        --tags all \
                        -e @$CONFIG \
                         $VIRTHOST


PLAY RECAP **************************************************************************************
192.168.0.74               : ok=141  changed=68   unreachable=0    failed=0   
localhost                  : ok=21   changed=10   unreachable=0    failed=1   
undercloud                 : ok=103  changed=64   unreachable=0    failed=0   

Tuesday 30 January 2018  01:45:42 +0300 (0:00:00.981)       3:31:54.695 ******* 
=============================================================================== 
overcloud-deploy : Deploy the overcloud ----------------------------------------------- 4677.93s
/home/boris/.quickstart/usr/local/share/ansible/roles/overcloud-deploy/tasks/deploy-overcloud.yml:1 
overcloud-prep-containers : Prepare for the containerized deployment ------------------ 3587.06s
/home/boris/.quickstart/usr/local/share/ansible/roles/overcloud-prep-containers/tasks/overcloud-prep-containers.yml:28 
undercloud-deploy : Install the undercloud -------------------------------------------- 1611.95s
/home/boris/.quickstart/usr/local/share/ansible/roles/undercloud-deploy/tasks/install-undercloud.yml:20 
tripleo-validations : Run validations tests through Mistral ---------------------------- 469.42s
/home/boris/.quickstart/usr/local/share/ansible/roles/tripleo-validations/tasks/main.yml:21 ----
overcloud-prep-images : Prepare the overcloud images for deploy ------------------------ 287.63s
/home/boris/.quickstart/usr/local/share/ansible/roles/overcloud-prep-images/tasks/overcloud-prep-images.yml:1 
modify-image : Run virt-customize on the provided image -------------------------------- 224.35s
/home/boris/.quickstart/usr/local/share/ansible/roles/modify-image/tasks/libguestfs.yml:46 -----
tripleo-validations : Run validations tests through Mistral ---------------------------- 210.62s
/home/boris/.quickstart/usr/local/share/ansible/roles/tripleo-validations/tasks/main.yml:21 ----
convert-image : convert image ---------------------------------------------------------- 209.64s
/home/boris/.quickstart/tripleo-quickstart/roles/convert-image/tasks/main.yml:25 ---------------
tripleo-validations : Run validations tests through Mistral ---------------------------- 154.94s
/home/boris/.quickstart/usr/local/share/ansible/roles/tripleo-validations/tasks/main.yml:21 ----
setup/undercloud : Perform selinux relabel on undercloud image ------------------------- 101.78s
/home/boris/.quickstart/tripleo-quickstart/roles/libvirt/setup/undercloud/tasks/main.yml:162 ---
setup/undercloud : Upload undercloud volume to storage pool ----------------------------- 92.34s
/home/boris/.quickstart/tripleo-quickstart/roles/libvirt/setup/undercloud/tasks/main.yml:280 ---
validate-tempest : Install openstack services tempest plugins --------------------------- 50.38s
/home/boris/.quickstart/usr/local/share/ansible/roles/validate-tempest/tasks/pre-tempest.yml:32 
overcloud-prep-flavors : Prepare the scripts for overcloud flavors ---------------------- 46.05s
/home/boris/.quickstart/usr/local/share/ansible/roles/overcloud-prep-flavors/tasks/overcloud-prep-flavors.yml:1 
setup/undercloud : Inject additional images --------------------------------------------- 44.02s
/home/boris/.quickstart/tripleo-quickstart/roles/libvirt/setup/undercloud/tasks/main.yml:67 ----
tripleo-validations : Run negative tests for pre-introspection group -------------------- 36.72s
/home/boris/.quickstart/usr/local/share/ansible/roles/tripleo-validations/tasks/main.yml:55 ----
undercloud-deploy : Start the Virtual BMCs ---------------------------------------------- 28.00s
/home/boris/.quickstart/usr/local/share/ansible/roles/undercloud-deploy/tasks/configure-vbmc.yml:107 
setup/undercloud : iptables ------------------------------------------------------------- 20.20s
/home/boris/.quickstart/tripleo-quickstart/roles/libvirt/setup/undercloud/tasks/main.yml:387 ---
fetch-images : Get tar images from cache ------------------------------------------------ 19.12s
/home/boris/.quickstart/tripleo-quickstart/roles/fetch-images/tasks/fetch.yml:186 --------------
convert-image : Resize the undercloud image using qemu-image resize --------------------- 16.95s
/home/boris/.quickstart/tripleo-quickstart/roles/convert-image/tasks/main.yml:19 ---------------
undercloud-deploy : Create the Virtual BMCs --------------------------------------------- 14.77s
/home/boris/.quickstart/usr/local/share/ansible/roles/undercloud-deploy/tasks/configure-vbmc.yml:75 
+ set +x
##################################
Virtual Environment Setup Complete
##################################

Access the undercloud by:

    ssh -F /home/boris/.quickstart/ssh.config.ansible undercloud

Follow the documentation in the link below to complete your deployment.

    http://ow.ly/c44w304begR

##################################
Virtual Environment Setup Complete
##################################

[boris@fedora27workstation ~]$ ssh -F /home/boris/.quickstart/ssh.config.ansible undercloud
Warning: Permanently added '192.168.0.74' (ECDSA) to the list of known hosts.
Warning: Permanently added 'undercloud' (ECDSA) to the list of known hosts.
Last login: Mon Jan 29 22:47:33 2018 from gateway
[stack@undercloud ~]$ . stackrc
(undercloud) [stack@undercloud ~]$ date
Mon Jan 29 23:16:05 UTC 2018
(undercloud) [stack@undercloud ~]$ nova --version
9.1.0
(undercloud) [stack@undercloud ~]$ nova list
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
| ID                                   | Name                    | Status | Task State | Power State | Networks               |
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
| 635c8dfe-0d6f-41fe-933d-7e6b847b6dd7 | overcloud-controller-0  | ACTIVE | -          | Running     | ctlplane=192.168.24.11 |
| 5ab9b3a1-5ff0-49bf-9d8e-6657953b163d | overcloud-controller-1  | ACTIVE | -          | Running     | ctlplane=192.168.24.7  |
| c15b80b5-bbd0-439d-a9ef-3c9715656341 | overcloud-controller-2  | ACTIVE | -          | Running     | ctlplane=192.168.24.9  |
| 64866562-4acf-4376-89a0-6bd8def60ef9 | overcloud-novacompute-0 | ACTIVE | -          | Running     | ctlplane=192.168.24.16 |
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
(undercloud) [stack@undercloud ~]$ ssh heat-admin@192.168.24.11
Last login: Mon Jan 29 22:54:18 2018 from 192.168.24.1
[heat-admin@overcloud-controller-0 ~]$ sudo su -
Last login: Mon Jan 29 22:54:20 UTC 2018 on pts/0
[root@overcloud-controller-0 ~]# . overcloudrc

(overcloud) [root@overcloud-controller-0 ~]# nova service-list
+--------------------------------------+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+
| Id                                   | Binary           | Host                                | Zone     | Status  | State | Updated_at                 | Disabled Reason | Forced down |
+--------------------------------------+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+
| b8f093c6-6d0e-4e81-8e9d-d87fe859b4f4 | nova-scheduler   | overcloud-controller-2.localdomain  | internal | enabled | up    | 2018-01-29T23:17:47.000000 | -               | False       |
| 084215f8-3f96-4143-a1ec-875870b159c2 | nova-scheduler   | overcloud-controller-0.localdomain  | internal | enabled | up    | 2018-01-29T23:17:42.000000 | -               | False       |
| ff3d4d25-f0cd-44a0-89de-4640d1987ce4 | nova-scheduler   | overcloud-controller-1.localdomain  | internal | enabled | up    | 2018-01-29T23:17:47.000000 | -               | False       |
| a819f985-61e0-47aa-82c7-16ab494447cc | nova-consoleauth | overcloud-controller-2.localdomain  | internal | enabled | up    | 2018-01-29T23:17:47.000000 | -               | False       |
| 10168768-0956-432c-bf17-23d83c37f6c1 | nova-consoleauth | overcloud-controller-0.localdomain  | internal | enabled | up    | 2018-01-29T23:17:46.000000 | -               | False       |
| e6c88b98-33c1-4355-8e9a-28fa5973a7e4 | nova-consoleauth | overcloud-controller-1.localdomain  | internal | enabled | up    | 2018-01-29T23:17:47.000000 | -               | False       |
| 315b28e0-505a-443e-928c-5615e02bd940 | nova-conductor   | overcloud-controller-0.localdomain  | internal | enabled | up    | 2018-01-29T23:17:48.000000 | -               | False       |
| e31914a2-240d-4804-aeb2-e6ee583a9bb0 | nova-conductor   | overcloud-controller-2.localdomain  | internal | enabled | up    | 2018-01-29T23:17:50.000000 | -               | False       |
| ea7db4a3-afce-433a-9e0a-fdb34aaa7533 | nova-compute     | overcloud-novacompute-0.localdomain | nova     | enabled | up    | 2018-01-29T23:17:48.000000 | -               | False       |
| db9d5146-9d12-44d3-9811-f32084b05f17 | nova-conductor   | overcloud-controller-1.localdomain  | internal | enabled | up    | 2018-01-29T23:17:46.000000 | -               | False       |
+--------------------------------------+------------------+-------------------------------------+----------+---------+-------+----------------------------+-----------------+-------------+
(overcloud) [root@overcloud-controller-0 ~]# neutron agent-list
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------------+--------------------+-------------------------------------+-------------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host                                | availability_zone | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+-------------------------------------+-------------------+-------+----------------+---------------------------+
| 0e64991a-2500-43ca-a670-004bfd9949ba | L3 agent           | overcloud-controller-1.localdomain  | nova              | :-)   | True           | neutron-l3-agent          |
| 19b26bbc-4321-4367-aa32-424bf044fbdc | Metadata agent     | overcloud-controller-0.localdomain  |                   | :-)   | True           | neutron-metadata-agent    |
| 2fe10568-9ab4-4ffb-ad6c-7ffa46c4e2f1 | Open vSwitch agent | overcloud-controller-2.localdomain  |                   | :-)   | True           | neutron-openvswitch-agent |
| 3d1bcfe3-6b09-40d8-a550-ba1edb184278 | DHCP agent         | overcloud-controller-2.localdomain  | nova              | :-)   | True           | neutron-dhcp-agent        |
| 4a3bdbd5-3c9f-4d2b-8bfa-7afca9b4db91 | Open vSwitch agent | overcloud-controller-0.localdomain  |                   | :-)   | True           | neutron-openvswitch-agent |
| 5768b697-a5c7-46be-8c1e-0fbd8c2e4492 | Metadata agent     | overcloud-controller-2.localdomain  |                   | :-)   | True           | neutron-metadata-agent    |
| 5b00559e-f99e-4cf9-8b36-74507b4d33e8 | DHCP agent         | overcloud-controller-1.localdomain  | nova              | :-)   | True           | neutron-dhcp-agent        |
| 757663ab-cd80-435b-b09b-d1a72e93f02c | Open vSwitch agent | overcloud-controller-1.localdomain  |                   | :-)   | True           | neutron-openvswitch-agent |
| 7a5918c3-4b69-49cd-8947-807a31899e4f | DHCP agent         | overcloud-controller-0.localdomain  | nova              | :-)   | True           | neutron-dhcp-agent        |
| 8ca3afd6-2174-4f2e-ad96-e0072da00992 | Metadata agent     | overcloud-controller-1.localdomain  |                   | :-)   | True           | neutron-metadata-agent    |
| a17927a6-025b-4a0f-9024-636982e0fc57 | L3 agent           | overcloud-controller-2.localdomain  | nova              | :-)   | True           | neutron-l3-agent          |
| c8b21501-d6d8-4f03-b020-f86dc11da19a | L3 agent           | overcloud-controller-0.localdomain  | nova              | :-)   | True           | neutron-l3-agent          |
| f2485026-efdf-456f-bbfe-8a4f4cba0c02 | Open vSwitch agent | overcloud-novacompute-0.localdomain |                   | :-)   | True           | neutron-openvswitch-agent |
+--------------------------------------+--------------------+-------------------------------------+-------------------+-------+----------------+---------------------------+

(overcloud) [root@overcloud-controller-0 ~]# pcs status
Cluster name: tripleo_cluster
Stack: corosync
Current DC: overcloud-controller-2 (version 1.1.16-12.el7_4.7-94ff4df) - partition with quorum
Last updated: Mon Jan 29 23:19:47 2018
Last change: Mon Jan 29 22:32:51 2018 by root via cibadmin on overcloud-controller-0

12 nodes configured
37 resources configured

Online: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]
GuestOnline: [ galera-bundle-0@overcloud-controller-0 galera-bundle-1@overcloud-controller-1 galera-bundle-2@overcloud-controller-2 rabbitmq-bundle-0@overcloud-controller-0 rabbitmq-bundle-1@overcloud-controller-1 rabbitmq-bundle-2@overcloud-controller-2 redis-bundle-0@overcloud-controller-0 redis-bundle-1@overcloud-controller-1 redis-bundle-2@overcloud-controller-2 ]

Full list of resources:

 Docker container set: rabbitmq-bundle [192.168.24.1:8787/master/centos-binary-rabbitmq:pcmklatest]
   rabbitmq-bundle-0    (ocf::heartbeat:rabbitmq-cluster):    Started overcloud-controller-0
   rabbitmq-bundle-1    (ocf::heartbeat:rabbitmq-cluster):    Started overcloud-controller-1
   rabbitmq-bundle-2    (ocf::heartbeat:rabbitmq-cluster):    Started overcloud-controller-2
 Docker container set: galera-bundle [192.168.24.1:8787/master/centos-binary-mariadb:pcmklatest]
   galera-bundle-0    (ocf::heartbeat:galera):    Master overcloud-controller-0
   galera-bundle-1    (ocf::heartbeat:galera):    Master overcloud-controller-1
   galera-bundle-2    (ocf::heartbeat:galera):    Master overcloud-controller-2
 Docker container set: redis-bundle [192.168.24.1:8787/master/centos-binary-redis:pcmklatest]
   redis-bundle-0    (ocf::heartbeat:redis):    Master overcloud-controller-0
   redis-bundle-1    (ocf::heartbeat:redis):    Slave overcloud-controller-1
   redis-bundle-2    (ocf::heartbeat:redis):    Slave overcloud-controller-2
 ip-192.168.24.10    (ocf::heartbeat:IPaddr2):    Started overcloud-controller-0
 ip-10.0.0.5    (ocf::heartbeat:IPaddr2):    Started overcloud-controller-1
 ip-172.16.2.9    (ocf::heartbeat:IPaddr2):    Started overcloud-controller-2
 ip-172.16.2.5    (ocf::heartbeat:IPaddr2):    Started overcloud-controller-0
 ip-172.16.1.8    (ocf::heartbeat:IPaddr2):    Started overcloud-controller-1
 ip-172.16.3.8    (ocf::heartbeat:IPaddr2):    Started overcloud-controller-2
 Docker container set: haproxy-bundle [192.168.24.1:8787/master/centos-binary-haproxy:pcmklatest]
   haproxy-bundle-docker-0    (ocf::heartbeat:docker):    Started overcloud-controller-0
   haproxy-bundle-docker-1    (ocf::heartbeat:docker):    Started overcloud-controller-1
   haproxy-bundle-docker-2    (ocf::heartbeat:docker):    Started overcloud-controller-2
 Docker container: openstack-cinder-volume [192.168.24.1:8787/master/centos-binary-cinder-volume:pcmklatest]
   openstack-cinder-volume-docker-0    (ocf::heartbeat:docker):    Started overcloud-controller-0

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled
 
VIRTHOST Status upon completion
 
  Undercloud
  
 Controller_0
 
 
  
 
(overcloud) [root@overcloud-controller-0 ~]# docker ps
CONTAINER ID        IMAGE                                                                                  COMMAND                  CREATED             STATUS                       PORTS               NAMES
2f1a1fd440b6        192.168.24.1:8787/master/centos-binary-gnocchi-statsd:current-tripleo-rdo              "kolla_start"            About an hour ago   Up About an hour                                 gnocchi_statsd
c70b1c40d38e        192.168.24.1:8787/master/centos-binary-gnocchi-api:current-tripleo-rdo                 "kolla_start"            About an hour ago   Up About an hour                                 gnocchi_api
b09a7f06b5c2        192.168.24.1:8787/master/centos-binary-cinder-volume:pcmklatest                        "/bin/bash /usr/local"   About an hour ago   Up About an hour                                 openstack-cinder-volume-docker-0
ad138a4a352e        192.168.24.1:8787/master/centos-binary-gnocchi-metricd:current-tripleo-rdo             "kolla_start"            About an hour ago   Up About an hour                                 gnocchi_metricd
6e7c31591da4        192.168.24.1:8787/master/centos-binary-neutron-openvswitch-agent:current-tripleo-rdo   "kolla_start"            About an hour ago   Up About an hour (healthy)                       neutron_ovs_agent
4a64ca891e72        192.168.24.1:8787/master/centos-binary-neutron-l3-agent:current-tripleo-rdo            "kolla_start"            About an hour ago   Up About an hour (healthy)                       neutron_l3_agent
d264f3062b97        192.168.24.1:8787/master/centos-binary-neutron-metadata-agent:current-tripleo-rdo      "kolla_start"            About an hour ago   Up About an hour (healthy)                       neutron_metadata_agent
d251e6b003a5        192.168.24.1:8787/master/centos-binary-neutron-dhcp-agent:current-tripleo-rdo          "kolla_start"            About an hour ago   Up About an hour (healthy)                       neutron_dhcp
9d354acc19cd        192.168.24.1:8787/master/centos-binary-glance-api:current-tripleo-rdo                  "kolla_start"            About an hour ago   Up About an hour (healthy)                       glance_api
5eb5e8e55b68        192.168.24.1:8787/master/centos-binary-panko-api:current-tripleo-rdo                   "kolla_start"            About an hour ago   Up About an hour                                 panko_api
379dff60fc54        192.168.24.1:8787/master/centos-binary-nova-api:current-tripleo-rdo                    "kolla_start"            About an hour ago   Up About an hour                                 nova_metadata
e06e06896f90        192.168.24.1:8787/master/centos-binary-nova-api:current-tripleo-rdo                    "kolla_start"            About an hour ago   Up About an hour (healthy)                       nova_api
b4ddbd34391d        192.168.24.1:8787/master/centos-binary-cron:current-tripleo-rdo                        "kolla_start"            About an hour ago   Up About an hour                                 logrotate_crond
70b634740320        192.168.24.1:8787/master/centos-binary-heat-api-cfn:current-tripleo-rdo                "kolla_start"            About an hour ago   Up About an hour (healthy)                       heat_api_cfn
ffee483621a5        192.168.24.1:8787/master/centos-binary-neutron-server:current-tripleo-rdo              "kolla_start"            About an hour ago   Up About an hour                                 neutron_api
f853a6267c63        192.168.24.1:8787/master/centos-binary-aodh-listener:current-tripleo-rdo               "kolla_start"            About an hour ago   Up About an hour (healthy)                       aodh_listener
d16dec06cdbf        192.168.24.1:8787/master/centos-binary-swift-container:current-tripleo-rdo             "kolla_start"            About an hour ago   Up About an hour                                 swift_container_auditor
f1960df5fdb5        192.168.24.1:8787/master/centos-binary-heat-api:current-tripleo-rdo                    "kolla_start"            About an hour ago   Up About an hour                                 heat_api_cron
d6aa14a4170d        192.168.24.1:8787/master/centos-binary-swift-proxy-server:current-tripleo-rdo          "kolla_start"            About an hour ago   Up About an hour                                 swift_object_expirer
9d5a826ae82e        192.168.24.1:8787/master/centos-binary-swift-object:current-tripleo-rdo                "kolla_start"            About an hour ago   Up About an hour                                 swift_object_updater
dfe0b64565c2        192.168.24.1:8787/master/centos-binary-swift-container:current-tripleo-rdo             "kolla_start"            About an hour ago   Up About an hour                                 swift_container_replicator
280bcfbb5722        192.168.24.1:8787/master/centos-binary-swift-account:current-tripleo-rdo               "kolla_start"            About an hour ago   Up About an hour                                 swift_account_auditor
1add6982079e        192.168.24.1:8787/master/centos-binary-cinder-api:current-tripleo-rdo                  "kolla_start"            About an hour ago   Up About an hour                                 cinder_api_cron
f988792f32c2        192.168.24.1:8787/master/centos-binary-swift-account:current-tripleo-rdo               "kolla_start"            About an hour ago   Up About an hour (healthy)                       swift_account_server
5e235481efa7        192.168.24.1:8787/master/centos-binary-nova-conductor:current-tripleo-rdo              "kolla_start"            About an hour ago   Up About an hour (healthy)                       nova_conductor
114240f254dd        192.168.24.1:8787/master/centos-binary-cinder-scheduler:current-tripleo-rdo            "kolla_start"            About an hour ago   Up About an hour (healthy)                       cinder_scheduler
9be553e80f40        192.168.24.1:8787/master/centos-binary-swift-object:current-tripleo-rdo                "kolla_start"            About an hour ago   Up About an hour                                 swift_object_replicator
231067fb993c        192.168.24.1:8787/master/centos-binary-swift-container:current-tripleo-rdo             "kolla_start"            About an hour ago   Up About an hour (healthy)                       swift_container_server
45cd16a94e82        192.168.24.1:8787/master/centos-binary-heat-engine:current-tripleo-rdo                 "kolla_start"            About an hour ago   Up About an hour (healthy)                       heat_engine
bd791ffe3997        192.168.24.1:8787/master/centos-binary-aodh-api:current-tripleo-rdo                    "kolla_start"            About an hour ago   Up About an hour                                 aodh_api
607f445cba39        192.168.24.1:8787/master/centos-binary-swift-object:current-tripleo-rdo                "kolla_start"            About an hour ago   Up About an hour                                 swift_rsync
d33025c1a725        192.168.24.1:8787/master/centos-binary-nova-novncproxy:current-tripleo-rdo             "kolla_start"            About an hour ago   Up About an hour (healthy)                       nova_vnc_proxy
3a470014fe9b        192.168.24.1:8787/master/centos-binary-ceilometer-notification:current-tripleo-rdo     "kolla_start"            About an hour ago   Up About an hour (healthy)                       ceilometer_agent_notification
78bcbb6e36e6        192.168.24.1:8787/master/centos-binary-swift-account:current-tripleo-rdo               "kolla_start"            About an hour ago   Up About an hour                                 swift_account_reaper
bd706a367e3b        192.168.24.1:8787/master/centos-binary-nova-consoleauth:current-tripleo-rdo            "kolla_start"            About an hour ago   Up About an hour (healthy)                       nova_consoleauth
104cd9577e69        192.168.24.1:8787/master/centos-binary-nova-api:current-tripleo-rdo                    "kolla_start"            About an hour ago   Up About an hour                                 nova_api_cron
47644098c212        192.168.24.1:8787/master/centos-binary-aodh-notifier:current-tripleo-rdo               "kolla_start"            About an hour ago   Up About an hour (healthy)                       aodh_notifier
10b4af64795a        192.168.24.1:8787/master/centos-binary-ceilometer-central:current-tripleo-rdo          "kolla_start"            About an hour ago   Up About an hour (healthy)                       ceilometer_agent_central
7daf6480c0bc        192.168.24.1:8787/master/centos-binary-swift-account:current-tripleo-rdo               "kolla_start"            About an hour ago   Up About an hour                                 swift_account_replicator
6bb8ccdf0a0f        192.168.24.1:8787/master/centos-binary-swift-object:current-tripleo-rdo                "kolla_start"            About an hour ago   Up About an hour                                 swift_object_auditor
7b9425f5d77d        192.168.24.1:8787/master/centos-binary-heat-api:current-tripleo-rdo                    "kolla_start"            About an hour ago   Up About an hour (healthy)                       heat_api
846ea8501d87        192.168.24.1:8787/master/centos-binary-swift-proxy-server:current-tripleo-rdo          "kolla_start"            About an hour ago   Up About an hour (healthy)                       swift_proxy
b2e60a0d22b1        192.168.24.1:8787/master/centos-binary-cinder-api:current-tripleo-rdo                  "kolla_start"            About an hour ago   Up About an hour                                 cinder_api
ba424cc9b311        192.168.24.1:8787/master/centos-binary-swift-object:current-tripleo-rdo                "kolla_start"            About an hour ago   Up About an hour (healthy)                       swift_object_server
e578387a0f75        192.168.24.1:8787/master/centos-binary-nova-scheduler:current-tripleo-rdo              "kolla_start"            About an hour ago   Up About an hour (healthy)                       nova_scheduler
890b9a81677f        192.168.24.1:8787/master/centos-binary-aodh-evaluator:current-tripleo-rdo              "kolla_start"            About an hour ago   Up About an hour (healthy)                       aodh_evaluator
a6afda832a5b        192.168.24.1:8787/master/centos-binary-swift-container:current-tripleo-rdo             "kolla_start"            About an hour ago   Up About an hour                                 swift_container_updater
ad219cbf1a54        192.168.24.1:8787/master/centos-binary-keystone:current-tripleo-rdo                    "/bin/bash -c '/usr/l"   About an hour ago   Up About an hour                                 keystone_cron
3eb572b5575f        192.168.24.1:8787/master/centos-binary-keystone:current-tripleo-rdo                    "kolla_start"            About an hour ago   Up About an hour (healthy)                       keystone
45044f1af048        192.168.24.1:8787/master/centos-binary-iscsid:current-tripleo-rdo                      "kolla_start"            About an hour ago   Up About an hour                                 iscsid
5a6aa0ac55f2        192.168.24.1:8787/master/centos-binary-nova-placement-api:current-tripleo-rdo          "kolla_start"            About an hour ago   Up About an hour                                 nova_placement
e44468dc7ef2        192.168.24.1:8787/master/centos-binary-horizon:current-tripleo-rdo                     "kolla_start"            About an hour ago   Up About an hour                                 horizon
c7fa2c9f253e        192.168.24.1:8787/master/centos-binary-haproxy:pcmklatest                              "/bin/bash /usr/local"   About an hour ago   Up About an hour                                 haproxy-bundle-docker-0
9e324dde9ddc        192.168.24.1:8787/master/centos-binary-redis:pcmklatest                                "/bin/bash /usr/local"   About an hour ago   Up About an hour                                 redis-bundle-docker-0
5fa0e50556a3        192.168.24.1:8787/master/centos-binary-mariadb:current-tripleo-rdo                     "kolla_start"            About an hour ago   Up About an hour                                 clustercheck
0408e0d11fb7        192.168.24.1:8787/master/centos-binary-mariadb:pcmklatest                              "/bin/bash /usr/local"   About an hour ago   Up About an hour                                 galera-bundle-docker-0
f8410b4e6d1f        192.168.24.1:8787/master/centos-binary-rabbitmq:pcmklatest                             "/bin/bash /usr/local"   About an hour ago   Up About an hour                                 rabbitmq-bundle-docker-0
64cec7586f17        192.168.24.1:8787/master/centos-binary-memcached:current-tripleo-rdo                   "/bin/bash -c 'source"   About an hour ago   Up About an hour                                 memcached
 
(undercloud) [stack@undercloud ~]$ nova list
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
| ID                                   | Name                    | Status | Task State | Power State | Networks               |
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
| 635c8dfe-0d6f-41fe-933d-7e6b847b6dd7 | overcloud-controller-0  | ACTIVE | -          | Running     | ctlplane=192.168.24.11 |
| 5ab9b3a1-5ff0-49bf-9d8e-6657953b163d | overcloud-controller-1  | ACTIVE | -          | Running     | ctlplane=192.168.24.7  |
| c15b80b5-bbd0-439d-a9ef-3c9715656341 | overcloud-controller-2  | ACTIVE | -          | Running     | ctlplane=192.168.24.9  |
| 64866562-4acf-4376-89a0-6bd8def60ef9 | overcloud-novacompute-0 | ACTIVE | -          | Running     | ctlplane=192.168.24.16 |
+--------------------------------------+-------------------------+--------+------------+-------------+------------------------+
(undercloud) [stack@undercloud ~]$ ssh heat-admin@192.168.24.16
The authenticity of host '192.168.24.16 (192.168.24.16)' can't be established.
ECDSA key fingerprint is SHA256:3K+uHRZiRfixw+k6T69oba+Pul6OGw3eQZ2EepVFIpU.
ECDSA key fingerprint is MD5:0a:b1:88:1b:db:ed:c8:90:f8:c9:4e:a6:3b:01:f6:b5.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.24.16' (ECDSA) to the list of known hosts.
Last login: Mon Jan 29 22:41:47 2018 from gateway
[heat-admin@overcloud-novacompute-0 ~]$ sudo su -
[root@overcloud-novacompute-0 ~]# docker ps
CONTAINER ID        IMAGE                                                                                  COMMAND             CREATED             STATUS                       PORTS               NAMES
ca3ae322a7ec        192.168.24.1:8787/master/centos-binary-neutron-openvswitch-agent:current-tripleo-rdo   "kolla_start"       About an hour ago   Up About an hour (healthy)                       neutron_ovs_agent
be1a8e606e3f        192.168.24.1:8787/master/centos-binary-cron:current-tripleo-rdo                        "kolla_start"       About an hour ago   Up About an hour                                 logrotate_crond
b46ba5f9a60d        192.168.24.1:8787/master/centos-binary-nova-compute:current-tripleo-rdo                "kolla_start"       About an hour ago   Up About an hour                                 nova_migration_target
1de8367bbbc0        192.168.24.1:8787/master/centos-binary-ceilometer-compute:current-tripleo-rdo          "kolla_start"       About an hour ago   Up About an hour                                 ceilometer_agent_compute
a1ebba3cad97        192.168.24.1:8787/master/centos-binary-nova-compute:current-tripleo-rdo                "kolla_start"       About an hour ago   Up About an hour (healthy)                       nova_compute
afe7ca5a3362        192.168.24.1:8787/master/centos-binary-iscsid:current-tripleo-rdo                      "kolla_start"       About an hour ago   Up About an hour                                 iscsid
5f6aca45393e        192.168.24.1:8787/master/centos-binary-nova-libvirt:current-tripleo-rdo                "kolla_start"       About an hour ago   Up About an hour                                 nova_libvirt
a60c8ce83dce        192.168.24.1:8787/master/centos-binary-nova-libvirt:current-tripleo-rdo                "kolla_start"       About an hour ago   Up About an hour                                 nova_virtlogd