Table of content
Sensu
Sensu is an alternative monitoring solution. Similar to Nagios or zabbix, though obviously with its own flavour how things are done. In the end though its quite similar to nagios and can even re-use scripts and plugins for nagios as is.
Sensuctl
$sensuctl is the main way to configure and interact with a sensu instance. In order to use sensuctl as root, you either need to interactively login or generate a token and use that for quicker access.
Delete a namespace
$ namespace=
$ sensuctl --namespace $namespace dump checks,handlers,filters,assets -f /tmp/sensu_dump_${namespace}.yaml
$ sensuctl delete -f /tmp/sensu_dump_${namespace}.yaml
$ senuctl namespace delete $namespace
$ rm /tmp/sensu_dump_${namespace}.yaml
Troubleshooting
Rejoin a faulty cluster member
Rarely it happens that a cluster member looses its etcd sync state and can’t rejoin. Following steps helped the last time on an Arch Linux system.
On the faulty node: Stop the service and empty the cache. Then set the cluster state to existing
$ systemctl stop sensu-backend
$ rm -r /opt/sensu/etcd
$ rm /var/cache/sensu/sensu-backend/assets.db
$ sed -iE 's/etcd-initial-cluster-state: "[^"]+"/etcd-initial-cluster-state: "existing"/' /etc/sensu/backend.yml
On a healthy node: Get the Peer URL, member ID and name from the faulty member.
$ sensuctl cluster member-list
$ name=
$ id=
$ PeerURL=
On a healthy node: Delete the member and add it as new member:
$ sensuctl cluster member-remove $id
$ sensuctl cluster member-add $name $PeerURL
On the faulty node: Start the service again and monitor the health
$ systemctl start sensu-backend
$ sensuctl cluster health