Strewth!

A post on puppet-users got me thinking about improved output. A simple method is to just JSON encode the log and parse the message objects on the other side. So here’s an easy way to do that.

This works pretty well with puppet master & puppet apply:

tmp donavanm$ puppet apply /tmp/test.pp --logdest json
{"level":"notice","time":"Sat Feb 19 17:01:12 -0800 2011","tags":["notice"],"source":"Puppet","message":"this is a message"}
{"line":1,"level":"notice","time":"Sat Feb 19 17:01:12 -0800 2011","tags":["notice","notify","class"],"file":"/tmp/test.pp","source":"/Stage[main]//Notify[this is a message]/message","message":"defined 'message' as 'this is a message'"}
{"level":"notice","time":"Sat Feb 19 17:01:12 -0800 2011","tags":["notice"],"source":"Puppet","message":"Finished catalog run in 0.06 seconds"}

It works less well with puppet agent, unfortunately. If --verbose or --debug is specified the :console destination is added, so you’ll get duplicate messages. Lame.

Currently there’s no way to load this as a plugin, see #6522. Appending this method to puppet/util/log/destinations.rb works. Meh.

Ideally I think we’d have different options for transaction reports. Currently the agent application always forces reports to the rest terminus. Being able to set up multiple destinations, like nagios or a local cache, would be great.

Update I’d previously mentioned #4248, which is similar but not what I really want. Updated with the correct issue. Also opened #6523 to clean up logdest handling in applications like agent.

This has been sitting in a work dir for a month now. Hopefully posting it motivates me polish it up and release it to the internets.

A while back I got new DSL service at my house in Seattle. In the course of moving I had to reconfigure a few nodes, setup  a gateway, etc. And in doing so I discovered that dynamic dns providers totally suck. It’s incredible. $20/year and you can’t even properly do delegations?

Coincidently I also noticed the new hotness from AWS at about the same time. DNS is part of my infrastructure, and puppet manages my infrastructure… So time to make puppet manage my DNS. After an evening hacking this up I present The AWS Route 53 type & provider:

Ensure a record:

tmp donavanm$ sudo puppet apply /tmp/r53.pp 
notice: /Stage[main]//Route53[foo.strewth.org.]/ensure: created

Get a list of my current records:

tmp donavanm$ sudo puppet resource route53
route53 { 'foo.strewth.org.':
    ensure => 'present',
    value => ['192.168.0.1'],
    rtype => 'A',
    zone => 'strewth.org.',
    ttl => '360'
}

Change ensure => 'absent' and remove that record:

tmp donavanm$ sudo puppet apply /tmp/r53.pp 
notice: /Stage[main]//Route53[foo.strewth.org.]/ensure: removed

And yup, it’s really gone:

tmp donavanm$ sudo puppet resource route53
 
tmp donavanm$

Being a fully functional type and provider it should Just Work in any of the puppet applications. I think the most powerful model would be using something like exported resources with puppet agent and master. The clients would export a resource, like I’ve shown. A trusted master periodically collects and updates all of the entries.

# dynamic clients export their settings
class r53::client::dynamic {
    @@route53 { 
        "${fqdn}.":
            value => $ipaddress,
            rtype => 'A',
            zone => "${domain}.",
            ttl => '360'
    }
}
# A puppet master collects and updates
class r53::server::dynamic {
    Route53 <<| tag == 'r53::client::dynamic' |>>
}

I could see this being a great tool for people with cloudy puppet deployments. Or when you just really want your laptops dns record to be current.

At a dollar a month its half the cost of those dynamic dns guys, totally automated, and a thousand times cooler.

A while back I was writing some custom server side functions to interrogate an inventory system for live information about other nodes. The returned data can be used to dynamically configure all sorts of settings that rely on other nodes: DNS resolvers, LDAP servers, Ganglia forwarders, etc.

Returning an array of nodes I could iterate a Define multiple times to create a set of resources. Here’s an example creating a set of Host resources for all of my DNS resolvers:

$mynameservers = find_nearest_nodes("puppetclass == dns::resolver")
# => ['ns1.domain.tld','ns2.domain.tld']
define hostentry() {
    $entry_ip = find_node_fact($name, "ipaddress")
    $entry_hostname = find_node_fact($name, "hostname")
    host{ $name: ip => $entry_ip, host_aliases => $entry_hostname }
}
hostentry { $mynameservers }

The two lookups in the define bothered me a bit. Assuming each lookup had some overhead latency that could add up. Why not simply return a hash with all the information i needed?

$myhashes = find_nearest_nodes("puppetclass == dns::resolver", ['ipaddress', 'hostname'])
# => [ {"ns2.domain.tld"=>{"ipaddress"=>"10.0.0.1", "hostname"=>"ns2"}}, {"ns1.domain.tld"=>{"ipaddress"=>"192.168.0.1", "hostname"=>"ns1"}} ]

But this array of hashes isn’t very useful. Each array entry is passed as $name to the target resource. Native resource types like File or Host will expect a string as the $name. Passing them the hash object won’t do what you want, I’m guessing it’d be collapsed in to a string with #to_s.

We can solve that though by wrapping the native type with a defined type . Inside the define you’ll be able to access the subkeys of $name like a normal hash. This allows you to pass your hash values as parameters to the actual resource:

$myhashes = find_nearest_nodes("puppetclass == dns::resolver", ['ipaddress', 'hostname'])
# =>  [ {"ns2.domain.tld"=>{"ipaddress"=>"10.0.0.1", "hostname"=>"ns2"}}, {"ns1.domain.tld"=>{"ipaddress"=>"192.168.0.1", "hostname"=>"ns1"}} ]
 
define hostentry() {
    # $name looks like {"ns2.domain.tld"=>{"ipaddress"=>"10.0.0.1", "hostname"=>"ns2"}}
    host{ $name: ip => $name[ipaddress], host_aliases => $name[hostname] }
}
 
hostentry{ $myhashes: }

EDIT: 4/7/11
Sorry #puppet, I’ve updated that last example. I’d written this post from memory and obviously didnt try the examples. $myhashes is an Array of Hashes, not HoH, and I missed the : when calling the hostentry define.
And if you think you want to do this, don’t. Drop in to a template or Ruby DSL. Usage is non obvious and relies on undefined behavior with namevar. I wouldn’t be surprised if it breaks horribly in a point release (though it works in 2.6.7).