Route53 Latency Based Routing
AWS Route53 launched a new feature today, Latency Based Routing. While playing around with this I created a few bits.
First I set up a t1.micro instance running in each region which will serve you a cat picture. Using a CloudFormation template I created the EC2 instance, downloaded the content from an S3 bucket, setup Apache, created an A record for the host, and another A record encoding the region. Each instance is now directly available via http://<aws region string>.strewth.org/. Here’s the direct link to west coast cat in us-west-1.
The next step was create an LBR A record of region.strewth.org
with appropriate RRs for each region. This was done via the AWS console. When entering the IP address the console automatically detected which region the EC2 instance is associated with. Slick!
And now this image comes directly from an AWS region near you:
LBR isn’t limited to just A records, any record type should work. You could could send Puppet clients to their fastest Puppet Master using LBR SRV records, for example.
To troubleshoot “where am I routed to” questions I created a handy TXT record, region.strewth.org. This record simply echos back the AWS region it is associated with. For example:
# dig +short region.strewth.org txt
"us-west-1"
LBR is an awesome feature for directing end users to the fastest available dynamic content. I can’t wait to see what people use this for.
Fort Shanty: Mission Accomplished!
Was in PDX for puppetconf. I stayed over the weekend at a friends house. Using the last bits summer  we spent a day in the park. Our mission was to build a most awesome box fort. I do believe I can call this one a raging success:
Resurrectio – Trek Backroads
A Trek 520 by any other name?
Spent the afternoon reviving an old Trek Backroads. I haven’t been able to find any mention of these bikes. Apparently it was an REI bike used for touring in the Bay Area in the mid-late ’90s.
It seems to be a Trek 520 frame set with Trek 720 parts. Interestingly the BB shell is actually stamped ‘520’ opposite the serial number. Looking at other serials I’d guess the frame is from ’98 or so.
It’s a decent CrMo frame. TrueTemper OX tube, 700C wheels, double eyelets in the rear, mid fork bosses, slack angles, cantilever brakes. Came equipped with a wide range 7 speed, triple front rings, and flat bars. From the parts spec I’d believe this was a working bike or an REI special hybrid.
Repurposed as a commuter with some new and old bits:
- Shimano Alfine 8spd w/ Velocity Dyad
- 20 tooth Alfine cog
- VOÂ Left Bank Handlebar
- Second hand Sugino cranks w/ 44tooth ring
- VO City Bike Brake levers
- Cheap Shimano Cantis
- Chunky WTB Pathway 700Cx38 tires
- Charge Masher 1/8″ half link chain
Took it out around the block and it’s all as expected. Very mellow steering with an upright position. Gearing is 31-97 inches, 5th sits at 60 gear inches. Hills shouldn’t be much of a chore, and plenty of zip for flats. Happy with the 38mm tires, but not much room for fenders. May be able to butcher some Planet Bikes in there. Otherwise Looks like it may drop down to ~35s this winter.
Free at last
Done with workin’ for the man, starting an indefinite holiday for the summer. If only I could find something to do…
Structured log output from puppet
A post on puppet-users got me thinking about improved output. A simple method is to just JSON encode the log and parse the message objects on the other side. So here’s an easy way to do that.
This works pretty well with puppet master
& puppet apply
:
tmp donavanm$ puppet apply /tmp/test.pp --logdest json {"level":"notice","time":"Sat Feb 19 17:01:12 -0800 2011","tags":["notice"],"source":"Puppet","message":"this is a message"} {"line":1,"level":"notice","time":"Sat Feb 19 17:01:12 -0800 2011","tags":["notice","notify","class"],"file":"/tmp/test.pp","source":"/Stage[main]//Notify[this is a message]/message","message":"defined 'message' as 'this is a message'"} {"level":"notice","time":"Sat Feb 19 17:01:12 -0800 2011","tags":["notice"],"source":"Puppet","message":"Finished catalog run in 0.06 seconds"} |
It works less well with puppet agent
, unfortunately. If --verbose
or --debug
is specified the :console
destination is added, so you’ll get duplicate messages. Lame.
Currently there’s no way to load this as a plugin, see #6522. Appending this method to puppet/util/log/destinations.rb works. Meh.
Ideally I think we’d have different options for transaction reports. Currently the agent application always forces reports to the rest
terminus. Being able to set up multiple destinations, like nagios or a local cache, would be great.
Update I’d previously mentioned #4248, which is similar but not what I really want. Updated with the correct issue. Also opened #6523 to clean up logdest handling in applications like agent.
A simple IP Address reflector
Or, Why the hell do I have to build every tool myself. I find it incredible that no one else has wanted this before.
In the course of another side project I needed to determine a nodes externally visible IP Address and FQDN. After a while I eventually found this forum post. And their “bot” url does work, though it’s a bit limited. So I hacked up some simple PHP to do it in a flavor that I like.
donavanm$ curl https://strewth.org/ip.php {"ipaddress":"98.207.236.52","fqdn":"c-98-207-236-52.hsd1.ca.comcast.net"}
Will also accept source
and (reasonable) timeout
parameters. A simple TCP connection is attempted back to your IP and specified port. Handy to see if NAT/iptables/etc is functioning.
donavanm$ curl https://strewth.org/ip.php?port=80 {"ipaddress":"98.207.236.52","fqdn":"c-98-207-236-52.hsd1.ca.comcast.net","port":{"80":false}}
Managing Amazon Route 53 with Puppet
This has been sitting in a work dir for a month now. Hopefully posting it motivates me polish it up and release it to the internets.
A while back I got new DSL service at my house in Seattle. In the course of moving I had to reconfigure a few nodes, setup a gateway, etc. And in doing so I discovered that dynamic dns providers totally suck. It’s incredible. $20/year and you can’t even properly do delegations?
Coincidently I also noticed the new hotness from AWS at about the same time. DNS is part of my infrastructure, and puppet manages my infrastructure… So time to make puppet manage my DNS. After an evening hacking this up I present The AWS Route 53 type & provider:
Ensure a record:
tmp donavanm$ sudo puppet apply /tmp/r53.pp notice: /Stage[main]//Route53[foo.strewth.org.]/ensure: created |
Get a list of my current records:
tmp donavanm$ sudo puppet resource route53 route53 { 'foo.strewth.org.': ensure => 'present', value => ['192.168.0.1'], rtype => 'A', zone => 'strewth.org.', ttl => '360' } |
Change ensure => 'absent'
and remove that record:
tmp donavanm$ sudo puppet apply /tmp/r53.pp notice: /Stage[main]//Route53[foo.strewth.org.]/ensure: removed |
And yup, it’s really gone:
tmp donavanm$ sudo puppet resource route53
tmp donavanm$ |
Being a fully functional type and provider it should Just Work in any of the puppet applications. I think the most powerful model would be using something like exported resources with puppet agent and master. The clients would export a resource, like I’ve shown. A trusted master periodically collects and updates all of the entries.
# dynamic clients export their settings class r53::client::dynamic { @@route53 { "${fqdn}.": value => $ipaddress, rtype => 'A', zone => "${domain}.", ttl => '360' } } # A puppet master collects and updates class r53::server::dynamic { Route53 <<| tag == 'r53::client::dynamic' |>> } |
I could see this being a great tool for people with cloudy puppet deployments. Or when you just really want your laptops dns record to be current.
At a dollar a month its half the cost of those dynamic dns guys, totally automated, and a thousand times cooler.
Array of Hashes in Puppet DSL
A while back I was writing some custom server side functions to interrogate an inventory system for live information about other nodes. The returned data can be used to dynamically configure all sorts of settings that rely on other nodes: DNS resolvers, LDAP servers, Ganglia forwarders, etc.
Returning an array of nodes I could iterate a Define multiple times to create a set of resources. Here’s an example creating a set of Host resources for all of my DNS resolvers:
$mynameservers = find_nearest_nodes("puppetclass == dns::resolver") # => ['ns1.domain.tld','ns2.domain.tld'] define hostentry() { $entry_ip = find_node_fact($name, "ipaddress") $entry_hostname = find_node_fact($name, "hostname") host{ $name: ip => $entry_ip, host_aliases => $entry_hostname } } hostentry { $mynameservers } |
The two lookups in the define bothered me a bit. Assuming each lookup had some overhead latency that could add up. Why not simply return a hash with all the information i needed?
$myhashes = find_nearest_nodes("puppetclass == dns::resolver", ['ipaddress', 'hostname']) # => [ {"ns2.domain.tld"=>{"ipaddress"=>"10.0.0.1", "hostname"=>"ns2"}}, {"ns1.domain.tld"=>{"ipaddress"=>"192.168.0.1", "hostname"=>"ns1"}} ] |
But this array of hashes isn’t very useful. Each array entry is passed as $name to the target resource. Native resource types like File or Host will expect a string as the $name. Passing them the hash object won’t do what you want, I’m guessing it’d be collapsed in to a string with #to_s.
We can solve that though by wrapping the native type with a defined type . Inside the define you’ll be able to access the subkeys of $name like a normal hash. This allows you to pass your hash values as parameters to the actual resource:
$myhashes = find_nearest_nodes("puppetclass == dns::resolver", ['ipaddress', 'hostname']) # => [ {"ns2.domain.tld"=>{"ipaddress"=>"10.0.0.1", "hostname"=>"ns2"}}, {"ns1.domain.tld"=>{"ipaddress"=>"192.168.0.1", "hostname"=>"ns1"}} ] define hostentry() { # $name looks like {"ns2.domain.tld"=>{"ipaddress"=>"10.0.0.1", "hostname"=>"ns2"}} host{ $name: ip => $name[ipaddress], host_aliases => $name[hostname] } } hostentry{ $myhashes: } |
EDIT: 4/7/11
Sorry #puppet, I’ve updated that last example. I’d written this post from memory and obviously didnt try the examples. $myhashes is an Array of Hashes, not HoH, and I missed the : when calling the hostentry define.
And if you think you want to do this, don’t. Drop in to a template or Ruby DSL. Usage is non obvious and relies on undefined behavior with namevar. I wouldn’t be surprised if it breaks horribly in a point release (though it works in 2.6.7).
First Post!
Mostly notes to my future self, I suspect. Think Momento with less murder and more hating of software.