Pretty Perfect Puppet Standards

“Making the simple complicated is commonplace; making the complicated simple, awesomely simple, that’s creativity.”
— Charles Mingus

So that's what we are going to try to do with Puppet. It's easy to get complicated, real complicated. Before you know if you will have a spaghetti mess of unmaintainable Puppet code. This isn't a Puppet thing. ANY programming code that doesn’t have a box around it is going to get out of hand.

We are going to avoid that.

This is an architectural standards document, and as such it cuts to the point. In other words, it is a "just do it this way" document. Where it makes sense explanation will be added. We also try to add examples and references where it will help the reader. But, if you are looking for a place to "learn Puppet" this is not it. There are many excellent tutorials and books that will do a better job than this.

With that let's get started:

Puppet Coding Rules

###Follow ALL rules in the Puppet Style Guide

The puppet style guide is an excellent standards guide. It covers pretty much anything pertaining to style and more. Consider this canon.

The Puppet Language Style Guide

###Manifests in one module should never reference files or templates stored in another module.

Good:

# /etc/puppetlabs/puppet/modules/tomcat/manifests/init.pp:
file { "${::tomcat::catalina_home}/config/context.xml":
  owner   => 'tomcat',
  group   => 'tomcat',
  content => epp(
    'tomcat_contexts/context_app.epp',
    {
      'app_name' => $my_app_name
    }
  ),
}

Bad:

# /etc/puppetlabs/puppet/modules/tomcat/manifests/init.pp:
file { "${::tomcat::catalina_home}/config/context.xml":
  owner   => 'tomcat',
  group   => 'tomcat',
  content => epp(
    'tomcat/context_app.epp',
    {
      'app_name' => $my_app_name
    }
  ),
}

Puppet Tips

Every parameter must include a datatype:

Puppet Data Types

Enum[‘red’, ‘yellow’, ‘green’] traffic_light_color =
  $::my_module::params::traffic_light_color
String favorite_food                               = $::my_module::params::favorite_food

Wrap optional parameters in the Optional data type:

Oftentimes a parameter needs to accept an undefined value. Defining data types prevents undef from being accepted.

When your parameters needs to accept an undef value wrap it in an Optional.

Optional Data Type

Optional[String] secondary_address = $::my_module::params:: secondary_address

###Base your requires, befores, and other ordering-related dependencies on classes rather than resources

Class-based ordering allows you to shield the implementation details of each class from the other classes.

This is a fancy way of saying that implementation details should be done inside of a class. If you need to change your implementation than go ahead. Your dependency ordering won't change; nor will your API.

However, if you need to add, edit or delete a resource, and you need to order these resources with other resource, you have an unmaitanable mess on your hands.

Sometimes resource ordering cannot be avoided. In these cases, use the meta-parameters, NOT chaining arrows. Meta-parameters help ensure that ordering will still take place if the resources are moved around.

Good

# /etc/puppetlabs/puppet/modules/tomcat/manifests/init.pp:
class tomcat {
  # Other code cut
  class { '::tomcat::install': } ->
  class { '::tomcat::config': }  ~>
  class { '::tomcat::service': }
}

# /etc/puppetlabs/puppet/modules/tomcat/manifests/config.pp:
class tomcat::config {
  # The following two configuration files should trigger a restart
  # This is done without a notify in the resource
  file { '/opt/tomcat/config/server.xml':
    ensure  => file,
    content => 'puppet:///modules/tomcat/server.xml',
  }

  file { '/opt/tomcat/config/context.xml':
    ensure  => file,
    content => 'puppet:///modules/tomcat/server.xml',
  }
}

# /etc/puppetlabs/puppet/modules/tomcat/manifests/service.pp:
class tomcat::service {
  service { 'tomcat':
    ensure  => present,
  }
}

Good

# /etc/puppetlabs/puppet/modules/tomcat/manifests/config.pp:
define tomcat::config::resource(
  String $context_name,
  String $my_app_name,
) {
  # The following file resource triggers a restart.
  # This is OK since it is notifying a class
  file { "/opt/tomcat/config/${context_name}":
    ensure  => file,
    content => epp(
      'tomcat/context_app.epp',
      {
        'app_name' => $my_app_name
      }
    ),
    notify  => Class['tomcat::service'],
  }
}

# /etc/puppetlabs/puppet/modules/tomcat/manifests/service.pp:
class tomcat::service {
  service { 'tomcat':
    ensure  => present,
  }
}

Not Good

# /etc/puppetlabs/puppet/modules/tomcat/manifests/pre_install.pp:
define tomcat::pre_install {
  # Not good
  # This is done to set a more relaxed mode for a fetched file
  # The proper way to do this is separate these two resources into 2 separate classes, or
  # for wget and alter it to allow owner / group / mode
  ::wget::fetch { $source_url:
    destination => "/opt/staging/tomcat/${apache_file}",
    timeout     => 0,
    verbose     => false,
    require     => File['/opt/staging/tomcat'],
  } ->
  file { "/opt/staging/tomcat/${apache_file}" :
    ensure => file,
    mode   => '0775'
  }
}

Bad

# The following file resource triggers a restart.
# This is not OK since it is tying to a resource
file { "/opt/tomcat/config/${context_name}":
  ensure  => file,
  content => epp(
    'tomcat/context_app.epp',
    {
      'app_name' => $my_app_name
    }
  ),
  notify  => Service['tomcat'],
}

More on Ordering

###When using a fact, use the “facts” hash.

More readable and maintainable code, by making facts visibly distinct from other variables. Eliminates possible confusion if you use a local variable whose name happens to match that of a common fact.

Good

$rootgroup = $::facts['osfamily'] ? {
    'Solaris'          => 'wheel',
    /(Darwin|FreeBSD)/ => 'wheel',
    default            => 'root',
}

Bad

$rootgroup = $osfamily ? {
    'Solaris'          => 'wheel',
    /(Darwin|FreeBSD)/ => 'wheel',
    default            => 'root',
}

More on the $facts['fact_name'] hash

##Puppet Building Blocks

In order to keep things organized we will follow this hierarchy when creating our code.

Puppet Class Heirarchy

First we have roles. A role is a class that contains, profile. That’s it. It contains only profiles, nothing else. A little more information to come.

Next is profiles. Profiles are classes. They can contain, well, anything. Modules classes, module resources, regular resources and yes even other profiles classes. That being said, there are some unique things that should be done in a profile class that makes them unique. We will dive into that too.

Finally, we have modules. They are a collection of classes and resources. They generally are responsible for the finely detailed installation, configuration and startup of a single piece of technology.
This being said, if you haven’t already please read the following work by Gary Larizza (Actually read all of his work). He goes into the why much deeper that this paper.

Building a Functional Puppet Workflow Part 1: Module Structure
Building a Functional Puppet Workflow Part 2: Roles and Profiles

###Modules

Do not put any organization code into your module

Modules should not contain any configuration data, security data, or PII. They are meant to be very generic.

While you might not intend to share your module, coding it in this way also forces you to keep it maintainable and reusable.

If you can't give your module to your meanest competitor down the street without giving away secrets you didn't do a good job.

####Use puppet module generate humana-<module name> to begin new modules.

This will automatically generate the correct directory structure, and in init.pp file.

Writing Modules

####Design your modules to manage a single piece of software from installation through setup, configuration, and service management.

You should add the following classes to your manifest after generation:

  • install.pp
  • config.pp
  • service.pp
  • params.pp

If any of the above classes are not used, keep them but leave them empty.

You of course may end up adding more classes, but init.pp and its associated sub-classes are typically of almost every installation and must be included.

More on the Puppet Module Structure

####Do NOT do Hiera lookups in your component modules!
The reasons for NOT using Hiera in your module are:

  • By doing Hiera calls at a higher level, you have a greater visibility on exactly what parameters were set by Hiera and which were set explicitly or by default values.
  • By doing Hiera calls elsewhere, your module is backwards-compatible for those folks who are NOT using Hiera

Building a Functional Puppet Workflow Part 1: Module Structure

####The parameters you expose to your top-level class should be treated as an API to your module.

This means that the 'only' class the user should pass parameters to is the top-level class, or init.pp class. This only applies to 'Classes', NOT 'Resources'. The best way to handle this is to remove all parameters from any class but the init.pp class.

Basic Module Class Setup

Notice that none of the classes in the above diagram have their own parameters. Any time they need a value they pull it from the top-level class. In our example class my_module::install uses $::mymodule_package_name and class my_module::services uses $mymodule::service_name.

Use parameterized class declarations in init.pp and explicitly pass values

Classes should declared using parameterized class declaration. This means do not use include or contain to declare parameters in init.pp.

Good:

# /etc/puppetlabs/puppet/modules/tomcat/manifests/init.pp:
class tomcat {
  # Other code cut
  class { '::tomcat::install': } ->
  class { '::tomcat::config': }  ~>
  class { '::tomcat::service': }
}

Bad:

# /etc/puppetlabs/puppet/modules/tomcat/manifests/init.pp:
class tomcat {
  # Other code cut
  include ::tomcat::install
  include ::tomcat::config
  include ::tomcat::service

  Class['::tomcat::install'] ->
  Class['::tomcat::config']  ~>
  Class['::tomcat::service']
}

Wait! Didn't we just decide to NOT use parameters? If so why do we care how they are declared?

If we use the class { '::name': } declaration it means these classes cannot be declared anywhere else. That is important because, we don't want the classes to be declared anywhere else.
Defaults
All default values for parameters go in the params.pp class. If a parameter doesn’t have a default assign it to ‘undef’.

###Profiles:
A profile contains all of the data and business logic needed to make build a standard piece of technology. It contains the module, classes and resources needed to make make a 'technology' happen in your organization.

Profiles are only declared inside of Roles. If they need to be ordered this will also occur inside the Role class.

####Do all Hiera lookups in the profile

As previously stated you do NOT do hiera lookups in the modules. You will not do them in the roles either. They should only be done in the profile class.

####Do not use automatic parameter lookup

Automatic parameter lookup makes it hard to debug and maintain code. All Hiera lookups should be done explicitly with a hiera function inside a profile class.

####Do not use default values in Hiera

Default values make debugging hard to do. If you modify a profile, and then forget to put that data into Hiera, the catalog will still compile.

Do not pass parameters to the profile

You should never use 'automatic parameter lookup', so you cannot pass parameters that way. Roles should only 'include' profiles, so they cannot pass parameters. Since roles are the only classes that will declare profiles (outside off other profiles) you should never open up your profile to parameters.

If you need data for your profile retrieve it through Hiera.

###Roles:
Roles contain only one thing, a list of profiles. Roles do not contain other roles. They stand alone.

Each node is assigned to one, and only one, role.

Roles include one or more profiles. They also optionally can order the profiles through class ordering. If profiles are ordered, order them using meta Class ordering.

class role::tomcat {
  include ::profile::base_linux
  include ::profile::java
  include ::profile::tomcat

  Class['::profile::base_linux'] ->
  Class['::profile::java']       ->
  Class['::profile::tomcat']
}

##Versioning:
Humana versions all of its roles and profile using semantic versioning: http://semver.org

Each role and profile class with have a suffix in its name.

The standard is vMM_mm_pp where:

MM = Major Number
mm = Minor Number
pp = Patch Number

Whenever code is moved from development into the first release branch (integration) the version number must be incremented. Version numbers in release should never be overwritten.

##Life Cycle:

The Puppet software development lifecycle runs from development through production. The three standard tiers in Humana are dev, integration, qa and finally production.

Integration, QA and production are set environments the development team will run through. There will typically be more than one development environment. Once a development team is ready to promote their code they will merge it into integration.

Deploying code from integration to qa and finally to production is straightforward. It consists of simply merging the code upwards. There may be times other environments need to be created, but that is outside the scope of this document.

Moving code from development to integration takes special care. Follow the following steps when deploying from a development environment to integration.

1 All modules worked on as part of this development effort must be tagged.
2 Any roles that were created or modified must have their version incremented.
3 Any profiles that were created and modified must also have their versions incremented.
4 The Puppetfile that is pushed to integration must have each and every module tagged with a version.

Special care must be taken with roles and profiles. Modified roles and profiles must not overwrite previous classes with the same version number.