The hiera-file backend

The hiera-file backend (https://github.com/adrienthebo/hiera-file) conceived by Adrien Thebo to manage a type of data that previously couldn't be stored in a sane way in Hiera, that is, plain files.

To install it, just clone the previous git repository in modulepath or use its gem as follows:

gem install hiera-file

We configure it by specifying the file backend, the hierarchy setting we want, and the datadir path where our files are placed:

---
:backends:
  - file
:hierarchy:
  - "fqdn/%{fqdn}"
  - "role/%{role}"
  - "common"
:file:
  :datadir: /etc/puppet/data

Here, the key used for Hiera lookups is actually the name of a file present in the .d subdirectories inside datadir according to our hierarchy.

For example, consider the following Puppet code:

file { '/etc/ssh/sshd_config':
  ensure  => present,
  content => hiera('sshd_config'),
}

Given the previous hierarchy (the first file found is returned), the previous code will create a sshd_config file at /etc/ssh/ with the content taken from files searched in these places.

/etc/puppet/data/fqdn/<fqdn>.d/sshd_config
/etc/puppet/data/role/<role>.d/sshd_config
/etc/puppet/data/common.d/sshd_config

In the previous example, <fqdn> and <role> have to be substituted with the actual FQDN and the role of our nodes.

If we want to provide an ERB template using hiera-file, we can use the following syntax:

file { '/etc/ssh/sshd_config':
  ensure  => present,
  content => inline_template(hiera('sshd_config.erb')),
}

This will look for and parse an erb template in the following places:

/etc/puppet/data/fqdn/<fqdn>.d/sshd_config.erb
/etc/puppet/data/role/<role>.d/sshd_config.erb
/etc/puppet/data/common.d/sshd_config.erb

The hiera-file backend is quite simple to use and very powerful because it allows us to move to Hiera what is generally placed in (site) modules, that is, plain files with which we manage and configure our applications.

The hiera-gpg backend

Here's another plugin that makes a lot of sense; a backend that allows us to encrypt sensitive data, hiera-gpg (https://github.com/crayfishx/hiera-gpg). It's written by Craig Dunn, who is the author of other very interesting plugins that we are going to review later.

Puppet code and data are generally versioned with an SCM and are distributed accordingly; it has always been an issue to decide how and where to store reserved data such as passwords, private keys, and credentials. They generally were the values assigned to variables either in clear text or as MD5/SHA hashes, but the possibility to expose them has always been a concern for Puppeteers, and various more or less imaginative solutions have been attempted (sometimes, the solution has been to simply ignore the problem and have no solution).

Backends such as hiera-gpg are a good solution for these cases. We can install it via its gem (we also need to have the gpg executable in our PATH, gcc, and the Ruby development libraries package (ruby-devel)):

gem install hiera-gpg

A sample hiera.yaml code is as follows:

---
:backends:
  - gpg
:hierarchy:
  - %{env}
  - common
:gpg:
  :datadir: /etc/puppet/gpgdata
#  :key_dir: /etc/puppet/gpg

The key_dir declaration is where our gpg keys are searched for; if we don't specify it, by default they are searched for in ~/.gnupg; so on our Puppet Master, this would be the .gnupg directory in the home of the puppet user.

First of all, we must create a GPG key with the following command:

gpg –-gen-key

We will be asked for the kind of key, its size and duration (the default settings are acceptable), a name, an e-mail, and a passphrase (even if gpg will complain, do not specify a passphrase because hiera-gpg doesn't support them).

Once the key is created, we can show it with:

gpg --list-key

The output is something like the following:

/root/.gnupg/pubring.gpg
------------------------
pub 2048R/C96EECCF 2013-12-08
uid Puppet Master (Puppet) <al@lab42.it>
sub 2048R/0AFB6B1F 2013-12-08

Now, we can encrypt files, move into our gpg datadir, and create normal YAML files that contain our secrets, for example:

---
mysql::root_password: 'V3rys3cr3T!'

Note that this is a temporary file that we will probably want to delete because we'll use its encrypted version, which has to be created with the following command:

gpg --encrypt -o common.gpg -r C96EECCF common.yaml

The -r argument expects our key ID (as seen via gpg –list-key), and -o expects the output file, which must have the same name/path of our datasource with a .gpg suffix.

Then, we can finally use Hiera to get the key's value from the encrypted files, shown as follows:

hiera mysql::root_password -d

The output text contains debug information and the decrypted value as follows:

DEBUG: <datetime>: Hiera YAML backend starting
DEBUG: <datetime>: Looking up mysql::root_password in YAML backend
DEBUG: <datetime>: Looking for data source common
DEBUG: <datetime>: [gpg_backend]: Loaded gpg_backend
DEBUG: <datetime>: [gpg_backend]: Lookup called, key mysql::root_password resolution type is priority
DEBUG: <datetime>: [gpg_backend]: GNUPGHOME is /root/.gnupg
DEBUG: <datetime>: [gpg_backend]: loaded cipher: /etc/puppet/gpgdata/common.gpg
DEBUG: <datetime>: [gpg_backend]: result is a String ctx #<GPGME::Ctx:0x7fb6aaa2f810> txt ---
mysql::root_password: 'V3rys3cr3T!'
DEBUG: <datetime>: [gpg_backend]: GPG decrypt returned valid data
DEBUG: <datetime>: [gpg_backend]: Data contains valid YAML
DEBUG: <datetime>: [gpg_backend]: Key mysql::root_password found in YAML document, Passing answer to hiera
DEBUG: <datetime>: [gpg_backend]: Assigning answer variable

Now, we can delete the cleartext common.yaml file and safely commit in our repository the encrypted GPG file and use our public key for further edits.

When we need to edit our file, we can decrypt it with the following command:

gpg -o common.yaml -d common.gpg

Note that we'll need the gpg private key to decrypt a file; this is required on the Puppet Master, and we need it on any system where we intend to edit these files.

The hiera-gpg backend is a neat solution that is used to manage sensitive data, but it has some drawbacks. The most relevant one is that we have to work with full files and we don't have a clear control over who makes changes to it unless we distribute the gpg private key to each member of our team.

Other projects have tried to address these limitations: hiera-eyaml by Tom Poulton (https://github.com/TomPoulton/hiera-eyaml), hiera_yamlgpg (https://github.com/compete/hiera_yamlgpg), and Raziel (https://github.com/jbraeuer/raziel). All of them allow us to work on plain YAML files and encrypt only single values. Therefore, they allow editing of single key entries by any user who has the gpg public key (no private key is required to be shared in order to decrypt and edit a whole file).

The hiera-eyaml backend

Let's see how hiera-eyaml works as it seems to be the most used and most maintained of the mentioned projects. We install its gem using the following command:

gem install hiera-eyaml

We edit the hiera.yaml file to configure it as follows:

---
:backends:
  - eyaml
:hierarchy:
  - "nodes/%{fqdn}"
  - "env/%{env}"
  - common
:eyaml:
  :datadir: /etc/puppet/hieradata
  :pkcs7_private_key: /etc/puppet/keys/private_key.pkcs7.pem
  :pkcs7_public_key:  /etc/puppet/keys/public_key.pkcs7.pem

Now, we have at our disposal the powerful eyaml command, which makes the whole experience pretty easy and straightforward. We can use it to create our keys, encrypt and decrypt files or single strings, and directly edit on the fly files with encrypted values.

First, let's create our keys using the following command:

eyaml createkeys

They are placed in the ./keys directory. Make sure that the user under which the Puppet Master runs (usually puppet) has read access to the private key.

Now, we can generate the encrypted value of any Hiera key with the following command:

eyaml encrypt -l 'mysql::root_password' -s 'V3ryS3cr3T!'

This will print on stdout both the plain encrypted string and a block of configuration that we can directly copy in our .eyaml files, as follows:

---
mysql::root_password: >
 ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMII […] 
 +oefgBBdAJ60kXMMh/RHpaXQYX3T]

Note that the value is in the format ENC[PKCS7,Encrypted_Value].

Since we have the password stored in plain text in our bash history, we should clean it using the following command:

history | grep encrypt
 572 eyaml encrypt -l 'mysql::root_password' -s 'V3ryS3cr3T!'
history -d 572

Luckily, we have to generate the keys in a similar fashion only once, since great things happen when we have to change our encrypted values in our eyaml files. We can directly edit them with the following command:

eyaml edit /etc/puppet/hieradata/common.eyaml

Our editor will open the file and decrypt the encrypted values on the fly so that we can edit our secrets in clear text and save the file again (of course, we can do this only on a machine where we have access to the private key). This particularly makes the management and maintenance of our secrets handy.

To view the decrypted content of an eyaml file, we can use the following command:

eyaml decrypt -f /etc/puppet/hieradata/common.eyaml

Since hiera-eyaml manages both clear text and encrypted values, we can use it as our only backend if we want to work only on YAML files.

The hiera-http and hiera-mysql backends

The hiera-http (https://github.com/crayfishx/hiera-http) and hiera-mysql (https://github.com/crayfishx/hiera-mysql) backends are other powerful Hiera backends written by Craig Dunn. They perfectly interpret Hiera's modular and extendable design and allow us to retrieve our data either via a REST interface or via MySQL queries on a database.

A quick view of how they could be configured might give you an idea of how they can fit in different cases. To configure hiera-http in hiera.yaml, we can use settings like the one shown in the following code:

:backends:
  - http
:http:
  :host: 127.0.0.1
  :port: 5984
  :output: json
  :failure: graceful
  :paths:
    - /configuration/%{fqdn}
    - /configuration/%{env}
    - /configuration/common

To configure hiera-mysql, we might have settings like the following:

---
:backends: 
  - mysql
:mysql:
  :host: localhost
  :user: root
  :pass: examplepassword
  :database: config
  :query: SELECT val FROM configdata WHERE var='%{key}' AND environment='%{env}'

We will not get into the details of these; you can refer to the official documentation to know the implementation and usage details.

However, note how easy and intuitive the syntax to configure them is, and what powerful possibilities they open to let users manage Puppet data from, for example, a web interface, without touching Puppet code or even logging to a server and working with a SCM such as git.