Tuesday, June 10, 2014

RDO IceHouse Setup Two Node (Controller+Compute) Neutron ML2&OVS&GRE Cluster on Fedora 20

Two KVMs have been created , each one having 2 virtual NICs (eth0,eth1) for
Controller && Compute Nodes setup. Before running `packstack --answer-file=twoNode-answer.txt` SELINUX set to permissive on both nodes.
Both eth1's assigned IPs from GRE Libvirts subnet before installation and set
to promiscuous mode (192.168.122.127, 192.168.122.137 ). Packstack bind to
public IP - eth0  192.169.142.127 , Compute Node 192.169.142.137

Answer file been used by packstack here  http://textuploader.com/0ei8 

Not sure my answer file is 100% correct, however it successful completion doesn't create file ml2_plugin.ini,/etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini on Controller && Compute Nodes which are required by Neutron ML2 plugin && OVS agent. This files have been manually created afterwards under /etc/neutron and allowed to configure neutron OVS agent on both nodes.

 Created under /etc/neutron on Controller && Compute Nodes :- ml2_conf.ini, plugin.ini , updated nova.conf files after packstack completion are here  http://textuploader.com/0ebe

Files missing after packstack run were created following http://kashyapc.fedorapeople.org/virt/openstack/rdo/IceHouse-Nova-Neutron-ML2-GRE-OVS.txt

Two Libvirt's  subnet created on F20 KVM Sever to support installation

 

 Public subnet :  192.169.142.0/24  
 GRE Tunnel  Support subnet:      192.168.122.0/24 


1. Create a new libvirt network (other than your default 198.162.x.x) file:

$ cat openstackvms.xml
 
<network>
   <name>openstackvms</name>
   <uuid>d0e9964a-f91a-40c0-b769-a609aee41bf2</uuid>
   <forward mode='nat'>
     <nat>
       <port start='1024' end='65535'/>
     </nat>
   </forward>
   <bridge name='virbr1' stp='on' delay='0' />
   <mac address='52:54:00:60:f8:6e'/>
   <ip address='192.169.142.1' netmask='255.255.255.0'>
     <dhcp>
       <range start='192.169.142.2' end='192.169.142.254' />
     </dhcp>
   </ip>
 </network> 
 
 2. Define the above network:

  $ virsh net-define openstackvms.xml

3. Start the network and enable it for "autostart" 
 
 $ virsh net-start openstackvms
 $ virsh net-autostart openstackvms


4. List your libvirt networks to see if it reflects:

  $ virsh net-list
  Name                 State      Autostart     Persistent
  ----------------------------------------------------------
  default              active     yes           yes
  openstackvms         active     yes           yes


5. Optionally, list your bridge devices:

  $ brctl show
  bridge name     bridge id               STP enabled     interfaces
  virbr0          8000.5254003339b3       yes             virbr0-nic
  virbr1          8000.52540060f86e       yes             virbr1-nic
 
 
IceHouse Neutron ML2&OVS&GRE setup creates database "neutron"  vs "ovs_neutron" database
in case of OVS&GRE Setup. Queries bellow demonstrate ML2 plugin setup. 
 
MariaDB [neutron]> show databases ;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| cinder             |
| glance             |
| keystone           |
| mysql              |
| neutron            |
| nova               |
| performance_schema |
| test               |
+--------------------+
9 rows in set (0.00 sec)

MariaDB [neutron]> show tables ;
+------------------------------+
| Tables_in_neutron            |
+------------------------------+
| agents                       |
| alembic_version              |
| allowedaddresspairs          |
| arista_provisioned_nets      |
| arista_provisioned_tenants   |
| arista_provisioned_vms       |
| cisco_ml2_credentials        |
| cisco_ml2_nexusport_bindings |
| consistencyhashes            |
| dnsnameservers               |
| externalnetworks             |
| extradhcpopts                |
| floatingips                  |
| ipallocationpools            |
| ipallocations                |
| ipavailabilityranges         |
| ml2_brocadenetworks          |
| ml2_brocadeports             |
| ml2_flat_allocations         |
| ml2_gre_allocations          |
| ml2_gre_endpoints            |
| ml2_network_segments         |
| ml2_port_bindings            |
| ml2_vlan_allocations         |
| ml2_vxlan_allocations        |
| ml2_vxlan_endpoints          |
| networkdhcpagentbindings     |
| networks                     |
| ports                        |
| quotas                       |
| routerl3agentbindings        |
| routerroutes                 |
| routers                      |
| securitygroupportbindings    |
| securitygrouprules           |
| securitygroups               |
| servicedefinitions           |
| servicetypes                 |
| subnetroutes                 |
| subnets                      |
+------------------------------+
40 rows in set (0.00 sec)

MariaDB [neutron]> select * from ml2_gre_endpoints;
+-----------------+
| ip_address      |
+-----------------+
| 192.169.142.127 |
| 192.169.142.137 |
+-----------------+
2 rows in set (0.00 sec)

MariaDB [neutron]> select * from ml2_port_bindings;
+--------------------------------------+----------------------------------------+----------+-------------+--------------------------------------+-----------+------------------------------------------------+---------+
| port_id                              | host                                   | vif_type | driver      | segment                              | vnic_type | vif_details                                    | profile |
+--------------------------------------+----------------------------------------+----------+-------------+--------------------------------------+-----------+------------------------------------------------+---------+
| 2136960c-d332-4515-9621-48f3110818ab |                                        | unbound  | NULL        | NULL                                 | normal    |                                                | {}      |
| 2cbc1e35-bfbb-46a0-b864-a618058b9494 | ip-192-169-142-137.ip.secureserver.net | ovs      | openvswitch | a8ffd0db-22f4-47fe-98ca-4a4f36740bd4 | normal    | {"port_filter": true, "ovs_hybrid_plug": true} |         |
| 345337a8-212b-4876-904b-61504a066e9c | ip-192-169-142-127.ip.secureserver.net | ovs      | openvswitch | cf159df0-7868-48b7-a37f-8a64db74b73a | normal    | {"port_filter": true, "ovs_hybrid_plug": true} | {}      |
| 844e2a96-2a40-47a3-9195-1f68cdc452e3 | ip-192-169-142-127.ip.secureserver.net | ovs      | openvswitch | a8ffd0db-22f4-47fe-98ca-4a4f36740bd4 | normal    | {"port_filter": true, "ovs_hybrid_plug": true} | {}      |
| 898669b0-6f3c-43d2-9dad-3baf31bd7b4f | ip-192-169-142-137.ip.secureserver.net | ovs      | openvswitch | a8ffd0db-22f4-47fe-98ca-4a4f36740bd4 | normal    | {"port_filter": true, "ovs_hybrid_plug": true} |         |
| a9ab4fb6-d7a8-45cf-bd55-f820ba618da7 | ip-192-169-142-127.ip.secureserver.net | ovs      | openvswitch | a8ffd0db-22f4-47fe-98ca-4a4f36740bd4 | normal    | {"port_filter": true, "ovs_hybrid_plug": true} | {}      |
| ae30ad31-e4bf-4584-8e31-227b7cb1a772 |                                        | unbound  | NULL        | NULL                                 | normal    |                                                | {}      |
| f44f8b76-568a-4a87-8529-5fc21b21627f |                                        | unbound  | NULL        | NULL                                 | normal    |                                                | {}      |
+--------------------------------------+----------------------------------------+----------+-------------+--------------------------------------+-----------+------------------------------------------------+---------+
8 rows in set (0.02 sec)

 MariaDB [neutron]> select * from ml2_network_segments ;
+--------------------------------------+--------------------------------------+--------------+------------------+-----------------+
| id                                   | network_id                           | network_type | physical_network | segmentation_id |
+--------------------------------------+--------------------------------------+--------------+------------------+-----------------+
| a8ffd0db-22f4-47fe-98ca-4a4f36740bd4 | c8f0d335-c08a-4775-a736-bd2cfe8e785b | gre          | NULL             |               2 |
| cf159df0-7868-48b7-a37f-8a64db74b73a | dbaed190-536a-4cd7-93cd-48e8b7e7c860 | gre          | NULL             |               1 |
+--------------------------------------+--------------------------------------+--------------+------------------+-----------------+
2 rows in set (0.02 sec)

MariaDB [neutron]> select * from networks ;
+----------------------------------+--------------------------------------+---------+--------+----------------+--------+
| tenant_id                        | id                                   | name    | status | admin_state_up | shared |
+----------------------------------+--------------------------------------+---------+--------+----------------+--------+
| 5ce08d72738b4493aa5aec5a1ae02608 | c8f0d335-c08a-4775-a736-bd2cfe8e785b | private | ACTIVE |              1 |      0 |
| 5ce08d72738b4493aa5aec5a1ae02608 | dbaed190-536a-4cd7-93cd-48e8b7e7c860 | public  | ACTIVE |              1 |      1 |
+----------------------------------+--------------------------------------+---------+--------+----------------+--------+
2 rows in set (0.03 sec)
 
[root@ip-192-169-142-127 ~(keystone_admin)]# openstack-status
== Nova services ==
openstack-nova-api:                     active
openstack-nova-cert:                    active
openstack-nova-compute:                 inactive  (disabled on boot)
openstack-nova-network:                 inactive  (disabled on boot)
openstack-nova-scheduler:               active
openstack-nova-volume:                  inactive  (disabled on boot)
openstack-nova-conductor:               active
== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:              active
== Keystone service ==
openstack-keystone:                     active
== Horizon service ==
openstack-dashboard:                    active
== neutron services ==
neutron-server:                         active
neutron-dhcp-agent:                     active
neutron-l3-agent:                       active
neutron-metadata-agent:                 active
neutron-lbaas-agent:                    inactive  (disabled on boot)
neutron-openvswitch-agent:              active
neutron-linuxbridge-agent:              inactive  (disabled on boot)
neutron-ryu-agent:                      inactive  (disabled on boot)
neutron-nec-agent:                      inactive  (disabled on boot)
neutron-mlnx-agent:                     inactive  (disabled on boot)
== Cinder services ==
openstack-cinder-api:                   active
openstack-cinder-scheduler:             active
openstack-cinder-volume:                active
openstack-cinder-backup:                inactive  (disabled on boot)
== Ceilometer services ==
openstack-ceilometer-api:               failed
openstack-ceilometer-central:           active
openstack-ceilometer-compute:           inactive  (disabled on boot)
openstack-ceilometer-collector:         active
openstack-ceilometer-alarm-notifier:    active
openstack-ceilometer-alarm-evaluator:   active
== Support services ==
openvswitch:                            active
dbus:                                   active
tgtd:                                   active
rabbitmq-server:                        active
memcached:                              active
== Keystone users ==
+----------------------------------+------------+---------+----------------------+
|                id                |    name    | enabled |        email         |
+----------------------------------+------------+---------+----------------------+
| 45d7ea9e3e02453196840cbe05068b95 |   admin    |   True  |    test@test.com     |
| 7aaf6f3f15f748e58e5718510455012c | ceilometer |   True  | ceilometer@localhost |
| 4e5be8d5415b4c3d93179d5169fd3965 |   cinder   |   True  |   cinder@localhost   |
| 6612f7861e8c4c64a64f97f85b071df6 |   glance   |   True  |   glance@localhost   |
| 4f2cf1d670044e41a00089095890dc20 |  neutron   |   True  |  neutron@localhost   |
| 03225b70bd1a479090c0e6b2b3666494 |    nova    |   True  |    nova@localhost    |
+----------------------------------+------------+---------+----------------------+
== Glance images ==
+--------------------------------------+--------------+-------------+------------------+-----------+--------+
| ID                                   | Name         | Disk Format | Container Format | Size      | Status |
+--------------------------------------+--------------+-------------+------------------+-----------+--------+
| f386318b-1b09-4973-a433-3bc58e0403a4 | CirrOS31     | qcow2       | bare             | 13147648  | active |
| f056516c-9712-4693-bd54-c793fbdb958f | Ubuntu 14.04 | qcow2       | bare             | 253821440 | active |
+--------------------------------------+--------------+-------------+------------------+-----------+--------+
== Nova managed services ==
+------------------+----------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Binary           | Host                                   | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+------------------+----------------------------------------+----------+---------+-------+----------------------------+-----------------+
| nova-consoleauth | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-06-10T18:16:18.000000 | -               |
| nova-scheduler   | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-06-10T18:16:21.000000 | -               |
| nova-conductor   | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-06-10T18:16:14.000000 | -               |
| nova-cert        | ip-192-169-142-127.ip.secureserver.net | internal | enabled | up    | 2014-06-10T18:16:15.000000 | -               |
| nova-compute     | ip-192-169-142-137.ip.secureserver.net | nova     | enabled | up    | 2014-06-10T18:16:20.000000 | -               |
+------------------+----------------------------------------+----------+---------+-------+----------------------------+-----------------+
== Nova networks ==
+--------------------------------------+---------+------+
| ID                                   | Label   | Cidr |
+--------------------------------------+---------+------+
| dbaed190-536a-4cd7-93cd-48e8b7e7c860 | public  | -    |
| c8f0d335-c08a-4775-a736-bd2cfe8e785b | private | -    |
+--------------------------------------+---------+------+
== Nova instance flavors ==
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| 1  | m1.tiny   | 512       | 1    | 0         |      | 1     | 1.0         | True      |
| 2  | m1.small  | 2048      | 20   | 0         |      | 1     | 1.0         | True      |
| 3  | m1.medium | 4096      | 40   | 0         |      | 2     | 1.0         | True      |
| 4  | m1.large  | 8192      | 80   | 0         |      | 4     | 1.0         | True      |
| 5  | m1.xlarge | 16384     | 160  | 0         |      | 8     | 1.0         | True      |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
== Nova instances ==
+--------------------------------------+---------------+-----------+------------+-------------+-----------------------------------+
| ID                                   | Name          | Status    | Task State | Power State | Networks                          |
+--------------------------------------+---------------+-----------+------------+-------------+-----------------------------------+
| cac5dc6b-387d-4c8f-adb0-963d1e588de0 | UbuntuSRV0609 | SUSPENDED | -          | Shutdown    | private=40.0.0.18, 192.169.142.53 |
| cadf3a2b-1448-4fa3-9eff-436933feb616 | UbuntuTRS0610 | SUSPENDED | -          | Shutdown    | private=40.0.0.17, 192.169.142.52 |
+--------------------------------------+---------------+-----------+------------+-------------+-- 
 
On Controller:- 

 
[root@ip-192-169-142-127 ~]# ovs-vsctl show
920cc7ea-a9df-4c2e-92f1-581463419293
    Bridge br-tun
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "gre-c0a98e89"
            Interface "gre-c0a98e89"
                type: gre
                options: {in_key=flow, local_ip="192.169.142.127", out_key=flow, remote_ip="192.169.142.137"}
    Bridge br-int
        Port br-int
            Interface br-int
                type: internal
        Port "tap844e2a96-2a"
            tag: 1
            Interface "tap844e2a96-2a"
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "qr-a9ab4fb6-d7"
            tag: 1
            Interface "qr-a9ab4fb6-d7"
                type: internal
    Bridge br-ex
        Port "eth0"
            Interface "eth0"
        Port br-ex
            Interface br-ex
                type: internal
        Port "qg-345337a8-21"
            Interface "qg-345337a8-21"
                type: internal
    ovs_version: "2.1.2"
 
On Compute:- 
 
[root@ip-192-169-142-137 ~]# ovs-vsctl show
6df0a73f-745d-489b-b995-0f60d0dcafe9
    Bridge br-int
        Port "qvo2cbc1e35-bf"
            tag: 1
            Interface "qvo2cbc1e35-bf"
        Port "qvo898669b0-6f"
            tag: 1
            Interface "qvo898669b0-6f"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port br-int
            Interface br-int
                type: internal
    Bridge br-tun
        Port br-tun
            Interface br-tun
                type: internal
        Port "gre-c0a98e7f"
            Interface "gre-c0a98e7f"
                type: gre
                options: {in_key=flow, local_ip="192.169.142.137", out_key=flow, remote_ip="192.169.142.127"}
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
    ovs_version: "2.1.2" 
 
[root@ip-192-169-142-127 ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.122.1   0.0.0.0         UG    0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1003   0        0 eth1
169.254.0.0     0.0.0.0         255.255.0.0     U     1012   0        0 br-ex
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.169.142.0   0.0.0.0         255.255.255.0   U     0      0        0 br-ex
 
[root@ip-192-169-142-137 ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.122.1   0.0.0.0         UG    0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1003   0        0 eth1
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.169.142.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0