Thursday, April 27, 2017

Adding Second VNIC to running Fedora 24 && CentOS 7.3 Cloud VMs on RDO Ocata

UPDATE 29/04/2017
Feedback accepted at ask.openstack.org
https://ask.openstack.org/en/question/105905/vm-supposed-to-have-two-nic-cant-bring-up-second-one/
END UPDATE
Posting bellow addresses one of currently pending questions at 
ask.openstack,org regarding adding second vnic to currently
running cloud guest which private IP and VNIC associated with neutron
port are located on different private subnet. The one which was 
selected booting this guest originally.
For RH's overcloud guests as F24 or CentOS7 adding instruction 
"GATEWAYDEV=eth0"
to /etc/sysconfig/network works on RDO Ocata as smoothly as it did 
in IceHouse times,preventing original gateway device from changing 
after attaching neutron port on new network.Attaching new neutron 
port happens during overcloud VM  runtime.
Creating ifcfg-eth1 allows to bring up eth1 via dhcp request been
sent to another dnsmasq daemon serving new private network.
*****************************************
Topology been utilized for testing :-
*****************************************
[stack@undercloud ~]$ date
Thu Apr 27 20:34:50 UTC 2017

[stack@undercloud ~]$ nova-manage --version
15.0.4
[stack@undercloud ~]$ openstack server list
+----------------------+----------------------+--------+-----------------------+----------------+
| ID                   | Name                 | Status | Networks              | Image Name     |
+----------------------+----------------------+--------+-----------------------+----------------+
| 134f6705-8156-4284   | overcloud-           | ACTIVE |ctlplane=192.168.24.6  | overcloud-full |
| -acce-0610ceec8c39   | controller-1         |        |                       |                |
| 53f45785-5a47-4008-9 | overcloud-           | ACTIVE |ctlplane=192.168.24.18 | overcloud-full |
| 147-12c32b7fbdbc     | controller-2         |        |                       |                |
| 5aab1903-7cdf-4d00   | overcloud-           | ACTIVE |ctlplane=192.168.24.13 | overcloud-full |
| -afce-060b2f5fac64   | novacompute-0        |        |                       |                |
| 9928671e-81ae-466c-  | overcloud-           | ACTIVE |ctlplane=192.168.24.11 | overcloud-full |
| b65b-d04b9fa0b3fd    | controller-0         |        |                       |                |
| 97167614-961e-46f3   | overcloud-           | ACTIVE | ctlplane=192.168.24.7 | overcloud-full |
| -841c-6f834d42162d   | novacompute-1        |        |                       |                |
+----------------------+----------------------+--------+-----------------------+----------------+

***************** 
Launch VM 
*****************
[root@overcloud-controller-0 ~]# openstack image list
+--------------------------------------+-----------------+--------+
| ID                                   | Name            | Status |
+--------------------------------------+-----------------+--------+
| b2123182-3350-4073-a405-4999383d2c9b | VF24Cloud-image | active |
+--------------------------------------+-----------------+--------+
[root@overcloud-controller-0 ~]# openstack network list
+-------------------------------+-------------------------------+-------------------------------+
| ID                            | Name                          | Subnets                       |
+-------------------------------+-------------------------------+-------------------------------+
| 02db6681-fec5-4d10-ae38-18021 | internal01                    | a7b8e363-0f2b-4eda-           |
| d6b0132                       |                               | 8ea7-23d396a95e70             |
| 2f954150-fe15-4e4f-           | ext-net                       | 8265b729-64dc-4c6d-bf8b-      |
| 9d26-60ecfd334186             |                               | 77ef40828fd1                  |
| 373f2bc3-c078-42dc-           | HA network tenant 0a3c66ba5fc | ac059474-3f58-48c9-b433-d3701 |
| ba97-c91d490f9364             | 74b72bb44d1620b36fb30         | be6d15c                       |
| fa984290-816e-                | internal02                    | 0b3115c1-88d3-4eb7-b120-9a099 |
| 4c17-bec1-0163aa68098a        |                               | 4456378                       |
+-------------------------------+-------------------------------+-------------------------------+

[root@overcloud-controller-0 ~]# openstack server create --flavor 2 \
--key-name oskey042717  --image b2123182-3350-4073-a405-4999383d2c9b \
--nic net-id=02db6681-fec5-4d10-ae38-18021d6b0132  vf24devs01
+-------------------------------------+--------------------------------------------------------+
| Field                               | Value                                                  |
+-------------------------------------+--------------------------------------------------------+
| OS-DCF:diskConfig                   | MANUAL                                                 |
| OS-EXT-AZ:availability_zone         |                                                        |
| OS-EXT-SRV-ATTR:host                | None                                                   |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                   |
| OS-EXT-SRV-ATTR:instance_name       |                                                        |
| OS-EXT-STS:power_state              | NOSTATE                                                |
| OS-EXT-STS:task_state               | scheduling                                             |
| OS-EXT-STS:vm_state                 | building                                               |
| OS-SRV-USG:launched_at              | None                                                   |
| OS-SRV-USG:terminated_at            | None                                                   |
| accessIPv4                          |                                                        |
| accessIPv6                          |                                                        |
| addresses                           |                                                        |
| adminPass                           | krWUrXZPzG3h                                           |
| config_drive                        |                                                        |
| created                             | 2017-04-27T18:46:30Z                                   |
| flavor                              | m1.small (2)                                           |
| hostId                              |                                                        |
| id                                  | b9ba03c7-8336-4ff9-b861-60ee292c8491                   |
| image                               | VF24Cloud-image (b2123182-3350-4073-a405-4999383d2c9b) |
| key_name                            | oskey042717                                            |
| name                                | vf24devs01                                             |
| progress                            | 0                                                      |
| project_id                          | 0a3c66ba5fc74b72bb44d1620b36fb30                       |
| properties                          |                                                        |
| security_groups                     | name='default'                                         |
| status                              | BUILD                                                  |
| updated                             | 2017-04-27T18:46:30Z                                   |
| user_id                             | 363f2f57000b4d378514aadaab142804                       |
| volumes_attached                    |                                                        |
+-------------------------------------+--------------------------------------------------------+

[root@overcloud-controller-0 ~]# nova list
+--------------------------------------+------------+--------+------------+-------------+---------------------+
| ID                                   | Name       | Status | Task State | Power State | Networks            |
+--------------------------------------+------------+--------+------------+-------------+---------------------+
| b9ba03c7-8336-4ff9-b861-60ee292c8491 | vf24devs01 | ACTIVE | -          | Running     | internal01=50.0.0.7 |
+--------------------------------------+------------+--------+------------+-------------+----------------------

Assign FIP (192.168.24.105) via dashboard 


[root@overcloud-controller-0 ~]# nova list
+--------------------------------------+------------+--------+------------+-------------+-----------------------------------------------------------+
| ID                                   | Name       | Status | Task State | Power State | Networks                                                  |
+--------------------------------------+------------+--------+------------+-------------+-----------------------------------------------------------+
| b9ba03c7-8336-4ff9-b861-60ee292c8491 | vf24devs01 | ACTIVE | -          | Running     | internal01=50.0.0.7, 192.168.24.105; internal02=70.0.0.12 |
+--------------------------------------+------------+--------+------------+-------------+-----------------------------------------------------------+


[root@overcloud-controller-0 ~]# openstack network list
+-------------------------------+-------------------------------+-------------------------------+
| ID                            | Name                          | Subnets                       |
+-------------------------------+-------------------------------+-------------------------------+
| 02db6681-fec5-4d10-ae38-18021 | internal01                    | a7b8e363-0f2b-4eda-           |
| d6b0132                       |                               | 8ea7-23d396a95e70             |
| 2f954150-fe15-4e4f-           | ext-net                       | 8265b729-64dc-4c6d-bf8b-      |
| 9d26-60ecfd334186             |                               | 77ef40828fd1                  |
| 373f2bc3-c078-42dc-           | HA network tenant 0a3c66ba5fc | ac059474-3f58-48c9-b433-d3701 |
| ba97-c91d490f9364             | 74b72bb44d1620b36fb30         | be6d15c                       |
| fa984290-816e-                | internal02                    | 0b3115c1-88d3-4eb7-b120-9a099 |
| 4c17-bec1-0163aa68098a        |                               | 4456378                       |
+-------------------------------+-------------------------------+-------------------------------+
 
Here we create new neutron port belongs newly created network :-
 [root@overcloud-controller-0 ~]# neutron port-create internal02
Created a new port:
+-----------------------+-----------------------------------------------------------------------+
| Field                 | Value                                                                 |
+-----------------------+-----------------------------------------------------------------------+
| admin_state_up        | True                                                                  |
| allowed_address_pairs |                                                                       |
| binding:host_id       |                                                                       |
| binding:profile       | {}                                                                    |
| binding:vif_details   | {}                                                                    |
| binding:vif_type      | unbound                                                               |
| binding:vnic_type     | normal                                                                |
| created_at            | 2017-04-27T18:54:11Z                                                  |
| description           |                                                                       |
| device_id             |                                                                       |
| device_owner          |                                                                       |
| extra_dhcp_opts       |                                                                       |
| fixed_ips             | {"subnet_id": "0b3115c1-88d3-4eb7-b120-9a0994456378", "ip_address":   |
|                       | "70.0.0.12"}                                                          |
| id                    | 35f5c0a8-41a2-47b6-8a16-0ac8f77d0f93                                  |
| mac_address           | fa:16:3e:fd:d1:ea                                                     |
| name                  |                                                                       |
| network_id            | fa984290-816e-4c17-bec1-0163aa68098a                                  |
| port_security_enabled | True                                                                  |
| project_id            | 0a3c66ba5fc74b72bb44d1620b36fb30                                      |
| qos_policy_id         |                                                                       |
| revision_number       | 5                                                                     |
| security_groups       | d799177b-f826-472b-9540-befd8b0da76f                                  |
| status                | DOWN                                                                  |
| tags                  |                                                                       |
| tenant_id             | 0a3c66ba5fc74b72bb44d1620b36fb30                                      |
| updated_at            | 2017-04-27T18:54:12Z                                                  |
+-----------------------+-----------------------------------------------------------------------+
 
 Here we attach neutron port belongs newly created network to cloud VM running on different 
 private network:- 
 [root@overcloud-controller-0 ~]# nova interface-attach --port-id \
 35f5c0a8-41a2-47b6-8a16-0ac8f77d0f93 vf24devs01


[boris@fedora24wks ~]$ ssh -i oskey042717.pem fedora@192.168.24.105
Last login: Thu Apr 27 19:05:16 2017 from 192.168.24.1

[fedora@vf24devs01 ~]$ sudo su -

[root@vf24devs01 ~]# cat /etc/sysconfig/network
NETWORKING=yes
NOZEROCONF=yes
DEVTIMEOUT=10
GATEWAYDEV=eth0  <====  eth0 is set manually 
                      as gateway device for running VMs
                      after attaching port belongs to internal02
Next commands 
# cd /etc/sysconfig/network-scripts
# cp ifcfg-eth0 ifcfg-eth1
  Edit ifcfg-eth1 to match "eth1" entry
# ifup eth1
******************************************************************************************* Skipping setting GATEWAYDEV will result to hang ssh connection via FIP and force to manage like bellow:- ********************************************************************************************
[root@overcloud-controller-0 ~]# ip netns
qdhcp-773be634-da70-4213-b653-7a510097f3c1
qdhcp-1bec9cb9-fc19-4686-bb30-d24c909a11b9
qrouter-f0da93c3-3e1f-4eb5-883e-c86692b07ff9


[root@overcloud-controller-0 ~]# ip netns exec \
   qdhcp-773be634-da70-4213-b653-7a510097f3c1 \
   ssh -i  oskey042817.pem fedora@70.0.0.12
The authenticity of host '70.0.0.12 (70.0.0.12)' can't be established.
ECDSA key fingerprint is 52:23:09:65:a3:78:90:85:e5:80:79:89:de:d3:48:a2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '70.0.0.12' (ECDSA) to the list of known hosts.
Last login: Fri Apr 28 15:06:46 2017 from 192.168.24.1
[fedora@vf24devs01 ~]$ sudo su -
[root@vf24devs01 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 50.0.0.13  netmask 255.255.255.0  broadcast 50.0.0.255
        inet6 fe80::f816:3eff:fec0:eb37  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:c0:eb:37  txqueuelen 1000  (Ethernet)
        RX packets 576  bytes 58794 (57.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 481  bytes 63460 (61.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 70.0.0.12  netmask 255.255.255.0  broadcast 70.0.0.255
        inet6 fe80::f816:3eff:fe06:86b0  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:06:86:b0  txqueuelen 1000  (Ethernet)
        RX packets 81  bytes 10785 (10.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 399  bytes 23427 (22.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 139  bytes 39092 (38.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 139  bytes 39092 (38.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@vf24devs01 ~]# ip route
default via 70.0.0.1 dev eth1 
50.0.0.0/24 dev eth0  proto kernel  scope link  src 50.0.0.13 
70.0.0.0/24 dev eth1  proto kernel  scope link  src 70.0.0.12 
169.254.169.254 via 50.0.0.1 dev eth0  proto static 
******************************************************************************************
Update /etc/sysconfig/network as requested followed by cloud VM restart 
That will switch back to default Gateway device eth0 and recover floating IP for VM
******************************************************************************************
[root@vf24devs01 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 50.0.0.7  netmask 255.255.255.0  broadcast 50.0.0.255
        inet6 fe80::f816:3eff:feab:739e  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:ab:73:9e  txqueuelen 1000  (Ethernet)
        RX packets 384  bytes 43397 (42.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 383  bytes 41874 (40.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 70.0.0.12  netmask 255.255.255.0  broadcast 70.0.0.255
        inet6 fe80::f816:3eff:fefd:d1ea  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:fd:d1:ea  txqueuelen 1000  (Ethernet)
        RX packets 6  bytes 760 (760.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 11  bytes 1048 (1.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


[root@overcloud-controller-0 ~]# openstack port list | grep 50.0.0.7
| 95a4666d-eac8-461c-87b8-3fcaf797c263 |                                                 | fa:16:3e:ab:73:9e | ip_address='50.0.0.7', subnet_id='a7b8e363-0f2b-4eda-8ea7-23d396a95e70'       | ACTIVE |

[root@overcloud-controller-0 ~]# openstack port list | grep 70.0.0.12
| 35f5c0a8-41a2-47b6-8a16-0ac8f77d0f93 |                                                 | fa:16:3e:fd:d1:ea | ip_address='70.0.0.12', subnet_id='0b3115c1-88d3-4eb7-b120-9a0994456378'      | ACTIVE |

[root@overcloud-controller-0 ~]# openstack subnet list
+------------------------+------------------------+--------------------------+------------------+
| ID                     | Name                   | Network                  | Subnet           |
+------------------------+------------------------+--------------------------+------------------+
| 0b3115c1-88d3-4eb7-b12 | sub-internal02         | fa984290-816e-           | 70.0.0.0/24      |
| 0-9a0994456378         |                        | 4c17-bec1-0163aa68098a   |                  |
| 8265b729-64dc-4c6d-    | ext-subnet             | 2f954150-fe15-4e4f-      | 192.168.24.0/24  |
| bf8b-77ef40828fd1      |                        | 9d26-60ecfd334186        |                  |
| a7b8e363-0f2b-4eda-    | sub-internal01         | 02db6681-fec5-4d10-ae38- | 50.0.0.0/24      |
| 8ea7-23d396a95e70      |                        | 18021d6b0132             |                  |
| ac059474-3f58-48c9-b43 | HA subnet tenant 0a3c6 | 373f2bc3-c078-42dc-      | 169.254.192.0/18 |
| 3-d3701be6d15c         | 6ba5fc74b72bb44d1620b3 | ba97-c91d490f9364        |                  |
|                        | 6fb30                  |                          |                  |
+------------------------+------------------------+--------------------------+------------------+
Instance restarted
   Login into VM via qdhcp namespace
 
Same procedure works fine on CentOS 7.3 Cloud Guest


  

References
1.https://thornelabs.net/2014/09/03/configure-multiple-network-interfaces-on-an-openstack-instance.html

5 comments: