How to prevent duplicate exported resources across a Puppet Infrastructure

There are cases where we have multiple exported resources that are identical. There are nodes that provide the same services for failover reasons. It is likely to have duplicate resources, when exporting nagios_check from these nodes. From the exporting node's point of view everything is fine. But when it comes to realizing the resources on the monitoring server, the puppetrun will fail due to the duplicate addresses. So how to mitigate this issue?

Nodename in the title

To disentangle the nagios_check resources it is easy to just stick the FQDN of the exporting node to the title of the check. This will work in some cases but, when there are enough services and nodes the amount of redundant checks will put significant load on the monitoring system and the nodes that provide the services. Even the puppetrun on the monitoring node takes ages to finish.

Mutual exclusive exporting

To have just one resource exported across the whole infrastructure, we have to check if the resource exists in the PuppetDB before we export it.

But there is an edge case lurking: What if we want to change parameters on our check? Since we only exporting if there is no instance present in the PuppetDB, no changes are applied to the exported resource until it gets garbage collected.

In addition to the rule of just one instance we have to check which node exported the instance. So the condition is: if there is no instance exported by a different node, then we are good to export.

Example

Copy$type       = 'Nagios_check'
$name       = 'vrrp_instance_1'
$parameters = {}

$_other_node_resources = puppetdb_query(
"resources {
  type         = '${type}'
  and title    = '${name}'
  and certname != '${::fqdn}'
  and exported = true
}"
)

if $_other_node_resources.empty {
    create_resources(
      "@@${type}",
      { $name => $parameters},
    )
}