Blog

How we orchestrate OpenStack at Prozeta

Featured Post Image

Monika Zajickova

November 14, 2017

CATEGORIES

Blog

Orchestration of infrastructure at Prozeta relies on the “role and profile” principle, using an orchestration stack based on Puppet, Foreman and Hiera.

Puppet and Foreman

Our orchestrated infrastructure is based primarily on Foreman and its surrounding ecosystem – Puppet, PuppetDB, Hiera and r10k.

We incorporate all of these components into our home-grown project called Mother, referring to a higher level “orchestration of orchestrator”. This allows us to easily create isolated orchestrators for cloud environments just by creating a Hiera configuration and spawning a VM.

We utilize the benefits of the well-known Discovery plugin for Foreman, which manages our physical machines’ provisioning and helps us configure essential parts of the installed OS (networking, partitions, etc.)

Here comes the most powerful factor: when you orchestrate everything using Foreman tools, you get all your data in one place: everything to do with HW and system (facts), roles & profiles (hostgroup) and system configuration state (Puppet reports).

Roles & Profiles

We have one massive profile Puppet module named BlackStack, which orchestrates everything about OpenStack that we need. This module uses many public puppet modules (Puppet Forge) which are resolved as dependencies by r10k. The profile class is then applied to all machines of an isolated stack (which we call a “section”) held in Foreman’s hostgroup, which is later divided into sub-hostgroups by a machine`s (or ring of machines’) role, i.e.

  • controller node
  • monitoring node
  • graphing node
  • compute node
  • storage node, etc.

Grouping nodes by roles opens up the possibility of auto-grouping and auto-discovery of clusters, so horizontal scaling is just a matter of adding a provisioning machine to the right host group.

Then we fetch a list of machines with the same role from PuppetDB (with help from the puppetdbquery module) and re-configure clusters with the new array of hosts. When you know system facts about all the machines in a stack, you can then easily feed them anywhere needed – e.g. your monitoring can automatically start monitoring new IPMI addresses, create every-to-every ICMP checks, etc.

Each section has its Hiera manifest, paired by hostgroup parameters evaluated in Puppet’s ENC. Any exceptions can be handled in Hiera’s host config file, so this solution is well-defined, self documented, yet flexible.

Environments

For needs of development and staging, we use Foreman’s (and Puppet’s) environments feature. Using r10k software, we can create different Puppet environments (using git branches and a different Puppet file). All deployments are instigated by drone-ci and triggered by git hooks, so all pre-deploy errors like broken dependencies or syntax errors are caught before they land on servers.

Foreman API

The power and elegance of all the afore-mentioned tricks together can be seen when you use Foreman’s hammer-cli and its extensions. With these, all that’s required to deploy a stack is one Hiera config file and a hammer-cli powered script that will deploy right-sized machines into the appropriate hostgroups.

Author: Tomáš Hejátko

Contact Us
Close icon

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Support

Contact details for existing customers.

+420 228 882 121

support@prozeta.eu


Sales

Got a question about our products or services? Please contact us directly.

+420 222 747 923

sales@prozeta.eu

PRO-ZETA a.s.

Address

Tiskařská 257/10

Praha 10, 108 00

Czech Republic

IČO

15891585

DIČ

CZ15891585

VAT

CZ15891585

Bank connection - CZK

Beneficiary name

PRO-ZETA a.s.

Bank

Československá Obchodní banka a.s.

Radlická 333/150, Praha 5

Account name

183670014/0300

IBAN

CZ4103000000000183670014

SWIFT

CEKOCZPP

Bank connection - USD

Beneficiary name

PRO-ZETA a.s.

Bank

Československá Obchodní banka a.s.

Radlická 333/150, Praha 5

Account name

235784348/0300

IBAN

CZ6703000000000235784348

SWIFT

CEKOCZPP

Bank connection - EUR

Beneficiary name

PRO-ZETA a.s.

Bank

Československá Obchodní banka a.s.

Radlická 333/150, Praha 5

Account name

277623198/0300

IBAN

CZ2203000000000277623198

SWIFT

CEKOCZPP