Instack-virt-setup deployment via TripleO Quickstart pre-built images on CentOS 7.3 (7.2)

UPDATE 11/22/2026

 If date/time of delorean.repo file in RDO Newton "current-passed-ci" trunk
differs from date/time of pre-built images for 10-12 hr same date, you may
go straight ahead skipping QuickStart initial deployment step to get
delorean repos from undercloud VM. In other words set repos as follows on both
VIRTHOST and INSTACK ( undercloud node in bare metal case )

sudo yum -y install yum-plugin-priorities
sudo curl -o /etc/yum.repos.d/delorean-newton.repo  https://buildlogs.centos.org/centos/7/cloud/x86_64/rdo-trunk-newton-tested/delorean.repo
sudo curl -o /etc/yum.repos.d/delorean-deps-newton.repo https://trunk.rdoproject.org/centos7-newton/delorean-deps.repo

This post is, actually, targeting  bare metal TripleO set up. Compare with suggested in http://docs.openstack.org/developer/tripleo-docs/basic_deployment/basic_deployment_cli.html strategy of setting up  just current trunk delorean.repo for stable releases.

END UPDATE

 Initiate TripleO QuickStart deployment , log into undercloud and pick up
files /etc/yum.repos.d/delorean.repo, /etc/yum.repos.d/delorean-deps.repo,
which are associated with RDO Newton trunk with status "current-passed-ci"
at the moment. On VIRTHOST and INSTACK VM  set same delorean repos.
Mentioned delorean repos will provide warranty that undercloud and overcloud packages match each other, e.g. tripleo-heat-templates package
on the undercloud will correspond openstack-puppet-modules packages on the overcloud images in case when we intend download prebuilt QuickStart images
from their standard location. Notice that same approach will work on TripleO
bare metal deployments as well.

***************************
Run on VIRTHOST
***************************
$ sudo yum -y update
$ sudo yum install -y instack-undercloud
$ instack-virt-setup

***************************
INSTACK VM SETUP
***************************

Set same  /etc/yum.repos.d/delorean.repo, /etc/yum.repos.d/delorean-deps.repo

sudo yum -y install yum-plugin-priorities
sudo curl -o /etc/yum.repos.d/delorean-newton.repo  https://buildlogs.centos.org/centos/7/cloud/x86_64/rdo-trunk-newton-tested/delorean.repo
sudo curl -o /etc/yum.repos.d/delorean-deps-newton.repo https://trunk.rdoproject.org/centos7-newton/delorean-deps.repo 
sudo yum -y upgrade mariadb-libs ( case 7.3 )
sudo yum -y install --enablerepo=extras centos-release-ceph-jewel
sudo sed -i -e 's%gpgcheck=.*%gpgcheck=0%' /etc/yum.repos.d/CentOS-Ceph-Jewel.repo

[stack@instack ~]# su - stack
[stack@instack ~]$ cat .bashrc

# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
    . /etc/bashrc
fi

# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=
export NODE_DIST=centos7
export USE_DELOREAN_TRUNK=1
export DELOREAN_TRUNK_REPO=" http://buildlogs.centos.org/centos/7/cloud/x86_64/rdo-trunk-newton-tested/"
export DELOREAN_REPO_FILE="delorean.repo"
export DIB_YUM_REPO_CONF=/etc/yum.repos.d/delorean*

# User specific aliases and functions

RELOGIN as stack=>root=>stack

[stack@instack ~]$ export DIB_YUM_REPO_CONF="$DIB_YUM_REPO_CONF /etc/yum.repos.d/CentOS-Ceph-Jewel.repo"
[stack@instack ~]$ echo $DIB_YUM_REPO_CONF
/etc/yum.repos.d/delorean-deps-newton.repo /etc/yum.repos.d/delorean-newton.repo /etc/yum.repos.d/CentOS-Ceph-Jewel.repo

****************************************************
Start installation on INSTACK VM  as usual
****************************************************

$ sudo yum install -y python-tripleoclient
$ openstack undercloud install
$ source stackrc
 

*********************************************** 
Download QuickStart prebuilt  images
***********************************************

sudo yum install wget -y

[stack@instack ~]$ wget http://buildlogs.centos.org/centos/7/cloud/x86_64/tripleo_images/newton/delorean/overcloud-full.tar
[stack@instack ~]$ wget http://buildlogs.centos.org/centos/7/cloud/x86_64/tripleo_images/newton/delorean/ironic-python-agent.tar
[stack@instack ~]$ wget http://buildlogs.centos.org/centos/7/cloud/x86_64/tripleo_images/newton/delorean/undercloud.qcow2 

[stack@instack ~]$ tar -xvf overcloud-full.tar
overcloud-full.d/
overcloud-full.d/dib-manifests/
overcloud-full.d/dib-manifests/dib-manifest-git-overcloud-full
overcloud-full.d/dib-manifests/dib-manifests-pip/
overcloud-full.d/dib-manifests/dib-element-manifest
overcloud-full.d/dib-manifests/dib_arguments
overcloud-full.d/dib-manifests/dib_environment
overcloud-full.initrd
overcloud-full.qcow2
overcloud-full.qcow2.log
overcloud-full.vmlinuz

[stack@instack ~]$ ls -l
total 5591996
-rw-r--r--. 1 stack stack      11755 Nov 17 15:00 instackenv.json
drwxr-xr-x. 3 stack stack       4096 Nov 17 04:44 overcloud-full.d
-rw-r--r--. 1 stack stack   41987282 Nov 17 04:44 overcloud-full.initrd
-rw-r--r--. 1 stack stack 1296985088 Nov 17 04:46 overcloud-full.qcow2
-rw-r--r--. 1 stack stack    1594247 Nov 17 04:46 overcloud-full.qcow2.log
-rw-rw-r--. 1 stack stack 1345751040 Nov 17 10:21 overcloud-full.tar
-rwxr-xr-x. 1 stack stack    5158864 Nov 17 04:44 overcloud-full.vmlinuz
-rw-------. 1 stack stack        324 Nov 17 15:27 stackrc
-rw-r--r--. 1 stack stack      10620 Nov 17 15:00 undercloud.conf
-rw-rw-r--. 1 stack stack       1650 Nov 17 15:16 undercloud-passwords.conf
-rw-r--r--. 1 stack stack 3034668544 Nov 17 15:30 undercloud.qcow2

[stack@instack ~]$ tar -xvf ironic-python-agent.tar
ironic-python-agent.d/
ironic-python-agent.d/dib-manifests/
ironic-python-agent.d/dib-manifests/dib_environment
ironic-python-agent.d/dib-manifests/dib_arguments
ironic-python-agent.d/dib-manifests/dib-element-manifest
ironic-python-agent.initramfs
ironic-python-agent.kernel
ironic-python-agent.qcow2.log
ironic-python-agent.vmlinuz

[stack@instack ~]$ ls -l
total 6404772
-rw-r--r--. 1 stack stack      11755 Nov 17 15:00 instackenv.json
drwxr-xr-x. 3 stack stack       4096 Nov 17 04:36 ironic-python-agent.d
-rw-r--r--. 1 stack stack  407642183 Nov 17 04:36 ironic-python-agent.initramfs
-rwxr-xr-x. 2 stack stack    5158864 Nov 17 04:36 ironic-python-agent.kernel
-rw-r--r--. 1 stack stack     745277 Nov 17 04:37 ironic-python-agent.qcow2.log
-rw-rw-r--. 1 stack stack  413562880 Nov 17 10:21 ironic-python-agent.tar
-rwxr-xr-x. 2 stack stack    5158864 Nov 17 04:36 ironic-python-agent.vmlinuz
drwxr-xr-x. 3 stack stack       4096 Nov 17 04:44 overcloud-full.d
-rw-r--r--. 1 stack stack   41987282 Nov 17 04:44 overcloud-full.initrd
-rw-r--r--. 1 stack stack 1296985088 Nov 17 04:46 overcloud-full.qcow2
-rw-r--r--. 1 stack stack    1594247 Nov 17 04:46 overcloud-full.qcow2.log
-rw-rw-r--. 1 stack stack 1345751040 Nov 17 10:21 overcloud-full.tar
-rwxr-xr-x. 1 stack stack    5158864 Nov 17 04:44 overcloud-full.vmlinuz
-rw-------. 1 stack stack        324 Nov 17 15:27 stackrc
-rw-r--r--. 1 stack stack      10620 Nov 17 15:00 undercloud.conf
-rw-rw-r--. 1 stack stack       1650 Nov 17 15:16 undercloud-passwords.conf
-rw-r--r--. 1 stack stack 3034668544 Nov 17 15:30 undercloud.qcow2

[stack@instack ~]$ . stackrc

[stack@instack ~]$ openstack overcloud image upload
WARNING: openstackclient.common.utils is deprecated and will be removed after Jun 2017. Please use osc_lib.utils
Image "overcloud-full-vmlinuz" was uploaded.
+--------------------------------------+------------------------+-------------+---------+--------+
|                  ID                  |          Name          | Disk Format |   Size  | Status |
+--------------------------------------+------------------------+-------------+---------+--------+
| b5c27d78-526f-43f4-95ef-655cd13e8851 | overcloud-full-vmlinuz |     aki     | 5158864 | active |
+--------------------------------------+------------------------+-------------+---------+--------+
Image "overcloud-full-initrd" was uploaded.
+--------------------------------------+-----------------------+-------------+----------+--------+
|                  ID                  |          Name         | Disk Format |   Size   | Status |
+--------------------------------------+-----------------------+-------------+----------+--------+
| c9ab88de-48dd-4265-8f18-49414af26272 | overcloud-full-initrd |     ari     | 41987282 | active |
+--------------------------------------+-----------------------+-------------+----------+--------+
Image "overcloud-full" was uploaded.
+--------------------------------------+----------------+-------------+------------+--------+
|                  ID                  |      Name      | Disk Format |    Size    | Status |
+--------------------------------------+----------------+-------------+------------+--------+
| 2a4fb040-0490-4de8-8d21-3b8680976ac6 | overcloud-full |    qcow2    | 1296985088 | active |
+--------------------------------------+----------------+-------------+------------+--------+
Image "bm-deploy-kernel" was uploaded.
+--------------------------------------+------------------+-------------+---------+--------+
|                  ID                  |       Name       | Disk Format |   Size  | Status |
+--------------------------------------+------------------+-------------+---------+--------+
| b5cbaeeb-945a-4fdb-9b82-da1a7b2e1490 | bm-deploy-kernel |     aki     | 5158864 | active |
+--------------------------------------+------------------+-------------+---------+--------+
Image "bm-deploy-ramdisk" was uploaded.
+--------------------------------------+-------------------+-------------+-----------+--------+
|                  ID                  |        Name       | Disk Format |    Size   | Status |
+--------------------------------------+-------------------+-------------+-----------+--------+
| 5dc1842f-c6c0-4160-bfa5-856cf61da816 | bm-deploy-ramdisk |     ari     | 407642183 | active |
+--------------------------------------+-------------------+-------------+-----------+--------+

[stack@instack ~]$  openstack baremetal import instackenv.json
WARNING: openstackclient.common.utils is deprecated and will be removed after Jun 2017. Please use osc_lib.utils
Started Mistral Workflow. Execution ID: 65620c7f-d3df-4f26-88d9-aef2eb287a67
Successfully registered node UUID 7132930c-720e-4566-aee6-8520233dc57a
Successfully registered node UUID 85aa181d-ec6e-4bb0-a53f-8f4c1000a249
Successfully registered node UUID 3d9511f7-fcf6-4024-a417-ee7e3fdbc5f9
Successfully registered node UUID 6d47d386-1adc-4c43-8837-707cad48b59b
Successfully registered node UUID 93502cb5-8fe8-4f69-a48c-91d87cf37925
Started Mistral Workflow. Execution ID: c8e0b9d2-92d4-4d0a-866c-777e51e97b6e  

$ openstack baremetal configure boot
$ openstack baremetal introspection bulk start

 In case when this command hangs, then start from scratch
        follow http://tripleo.org/advanced_deployment/introspect_single_node.html
 It might take long time and resolve the issue.
 

 $ neutron subnet-list
 $ neutron subnet-update 1b7d82e5-0bf1-4ba5-8008-4aa402598065 --dns-nameserver  192.168.122.1 

********************************************************************************************
Next step :- 
$ sudo vi  /usr/share/openstack-tripleo-heat-templates/puppet/services/ceph-mon.yaml
Update line :-
ceph::profile::params::osd_pool_default_size: 1 
instead of default value "3". This step is acceptable only in Virtual Environment.
Setting  the osd_pool_default_size set to 1,
 you will only have
one copy of the object. As a general rule, you should run your cluster 
with more than one OSD and a pool size greater than 1 object replica. So
having 48GB RAM on VIRTHOST  the optimal setting is osd_pool_default_size = 3 (at least 2) 

********************************************************************************************

outputs:
  role_data:
    description: Role data for the Ceph Monitor service.
    value:
      service_name: ceph_mon
      monitoring_subscription: {get_param: MonitoringSubscriptionCephMon}
      config_settings:
        map_merge:
          - get_attr: [CephBase, role_data, config_settings]
          - ceph::profile::params::ms_bind_ipv6: {get_param: CephIPv6}
            ceph::profile::params::mon_key: {get_param: CephMonKey}
            ceph::profile::params::osd_pool_default_pg_num: 32
            ceph::profile::params::osd_pool_default_pgp_num: 32
            ceph::profile::params::osd_pool_default_size: 1  <== instead of "3"      
            # repeat returns items in a list, so we need to map_merge twice
            tripleo::profile::base::ceph::mon::ceph_pools:
              map_merge:
                - map_merge:
                    repeat:
                      for_each:
                        <%pool%>:
                          - {get_param: CinderRbdPoolName}
                          - {get_param: CinderBackupRbdPoolName}
                          - {get_param: NovaRbdPoolName}
                          - {get_param: GlanceRbdPoolName}
                          - {get_param: GnocchiRbdPoolName}


******************************
Set up Network isolation
******************************

 [stack@instack ~]$ sudo vi /etc/sysconfig/network-scripts/ifcfg-vlan10
 DEVICE=vlan10
 ONBOOT=yes
 DEVICETYPE=ovs
 TYPE=OVSIntPort
 BOOTPROTO=static
 IPADDR=10.0.0.1
 NETMASK=255.255.255.0
 OVS_BRIDGE=br-ctlplane
 OVS_OPTIONS="tag=10"

[stack@instack ~]$ sudo ifup vlan10

*********************************************
192.168.122.134 is IP of "instack VM"
*********************************************

[stack@instack ~]$ cat network_env.yaml
{
    "parameter_defaults": {
        "ControlPlaneDefaultRoute": "192.0.2.1",
        "ControlPlaneSubnetCidr": "24",
        "DnsServers": [
            "192.168.122.134"
        ],
        "EC2MetadataIp": "192.0.2.1",
        "ExternalAllocationPools": [
            {
                "end": "10.0.0.250",
                "start": "10.0.0.4"
            }
        ],
        "ExternalNetCidr": "10.0.0.1/24",
        "NeutronExternalNetworkBridge": ""
    }
} 


[stack@undercloud ~]$ sudo iptables -A BOOTSTACK_MASQ -s 10.0.0.0/24 ! -d 10.0.0.0/24 -j MASQUERADE -t nat
[stack@undercloud ~]$ sudo touch -f  /usr/share/openstack-tripleo-heat-templates/puppet/post.yaml

**********************************
Script overcloud-deploy.sh
**********************************

#!/bin/bash -x
source /home/stack/stackrc  
openstack overcloud deploy  \
 --control-scale 3 --compute-scale 1 --ceph-storage-scale 1 \
 --libvirt-type qemu \
 --ntp-server pool.ntp.org  \
 --templates  /usr/share/openstack-tripleo-heat-templates \
 -e  /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml \
 -e  /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml \
 -e  /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
 -e  /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml \
 -e  $HOME/network_env.yaml 

  

  

  

   [root@overcloud-controller-0 ~]# ceph health detail --format=json-pretty

{
    "health": {
        "health_services": [
            {
                "mons": [
                    {
                        "name": "overcloud-controller-2",
                        "kb_total": 48214840,
                        "kb_used": 8346496,
                        "kb_avail": 39868344,
                        "avail_percent": 82,
                        "last_updated": "2016-11-17 23:10:04.864730",
                        "store_stats": {
                            "bytes_total": 15286594,
                            "bytes_sst": 14762290,
                            "bytes_log": 458752,
                            "bytes_misc": 65552,
                            "last_updated": "0.000000"
                        },
                        "health": "HEALTH_OK"
                    },
                    {
                        "name": "overcloud-controller-0",
                        "kb_total": 48214840,
                        "kb_used": 8857768,
                        "kb_avail": 39357072,
                        "avail_percent": 81,
                        "last_updated": "2016-11-17 23:09:57.658188",
                        "store_stats": {
                            "bytes_total": 15286603,
                            "bytes_sst": 14762299,
                            "bytes_log": 458752,
                            "bytes_misc": 65552,
                            "last_updated": "0.000000"
                        },
                        "health": "HEALTH_OK"
                    },
                    {
                        "name": "overcloud-controller-1",
                        "kb_total": 48214840,
                        "kb_used": 8338904,
                        "kb_avail": 39875936,
                        "avail_percent": 82,
                        "last_updated": "2016-11-17 23:10:04.681707",
                        "store_stats": {
                            "bytes_total": 15810891,
                            "bytes_sst": 14762299,
                            "bytes_log": 983040,
                            "bytes_misc": 65552,
                            "last_updated": "0.000000"
                        },
                        "health": "HEALTH_OK"
                    }
                ]
            }
        ]
    },
    "timechecks": {
        "epoch": 4,
        "round": 154,
        "round_status": "finished",
        "mons": [
            {
                "name": "overcloud-controller-2",
                "skew": 0.000000,
                "latency": 0.000000,
                "health": "HEALTH_OK"
            },
            {
                "name": "overcloud-controller-0",
                "skew": -0.002390,
                "latency": 0.001325,
                "health": "HEALTH_OK"
            },
            {
                "name": "overcloud-controller-1",
                "skew": 0.013829,
                "latency": 0.001581,
                "health": "HEALTH_OK"
            }
        ]
    },
    "summary": [],
    "overall_status": "HEALTH_OK",
    "detail": []
}