FOSS in, FOSS out: software, process and operations

Puppet File Recurse Pitfall

Puppet has become my go to system management tool in no small part because it is the tool that the operations group at $DAYJOB has standardized on for our production infrastructure management. It took quite a while for me to get the hang of how Puppet does what it does, but today I’d say I’m a fairly decent Puppet programmer. Every once in a while however I stumble on something new and surprising.

A couple of weeks ago I got an interesting bug report from a user about a collection of Puppet manifests I help manage. The bug was that his testing server was pegged at 99% CPU utilization for multiple minutes during each puppet agent run. The bug reporter did a great job of investigating and had also found that strace showed a repetitive stream of stat() calls while the process was hogging the CPU.

This also turned out to the be the great kind of bug that was reproducible. The first testing server I tried the steps from the bug report on showed the exact same symptoms. I grabbed some very verbose logs by turning on the --debug logging in puppet agent and logging all of the system calls with strace at the same time:

$ TZ=UTC strace /usr/bin/ruby /usr/bin/puppet agent --onetime --verbose \
   --no-daemonize --no-splay --debug 2>&1 |
   tee /tmp/loud-puppet-strace.log

How Do You Know When You’re Done?

In scrum a story is “Done” when it meets the team’s shared “Definition of Done”. The definition of done is roughly a list of requirements that all parts of the software increment must adhere to to be called complete. Like most things in scrum the implementation details are left to the team to decide. When I was first working with scrum I had a hard time finding examples of what a typical definition of done would include. Most scrum authors (and even many trainers) wave their hands and say that it’s too specific to the team and their environment to generalize.

FileVault2 Hacks

Mac OS X 10.7 introduced a whole disk encryption service called FileVault2. This allows you to use AES 128 encryption to protect your data. This is a great feature but it has a few small drawbacks. It uses the password of your primary user account to unlock the system. I’m a fan of strong passwords but for encryption I’d prefer to use a longer pass phrase for increased entropy. Second the EFI-boot screen that is used to get the password to decrypt the disk shows the display name of all usersthat can unlock the system rather than blank fields for both username and password. This leaks information that I would really rather not leak. Fortunately I’ve found a little hack to work around both of these issues.

Yaml 1.1.1 PECL Module Released

I’m glad to announce that I finally got around to releasing the bug fix version of the YAML PECL module that I announced on 2013-04-23. Version 1.1.1 fixes several long standing bugs:

  • #61770 Crash on nonunicode character
  • #61923 Detect_scalar_type() is not aware of base 60 representation
  • #63086 Compiling PHP with YAML as static extension fails
  • #64019 Segmentation fault if yaml anchor ends with a colon
  • #64694 Segfault when array used as mapping key