10 January 2017

Easy Infrastructure Testing with Goss

In the world of infrastructure; new servers, VMs, containers or applications are often manually validated by a human. Some form of build document or checkbox exercise takes place to confirm that the piece of infrastructure is ready for use. Even with configuration management, mistakes happen and humans make mistakes (damn humans).

Wouldn’t it be great if there were an easy way to automatically validate new servers before they are live and we find the problems in production? Well, there is! Try Goss.

What is Goss?

Goss is a tool that let’s you easily and quickly validate infrastructure. Like Serverspec but without all the code. Goss allows you to define what a piece of infrastructure should look like with YAML or JSON. This is made even easier for us with the ability to auto add resources to the Goss configuration on the command line.

Goss allows you to validate many different resource types such as files, users, groups, packages, services and http connectivity. You can read the full Goss documentation here.

Why Goss?

Well, a few things make goss an awesome tool for server validation:

Written in Go - This means it’s a self contained binary with no dependencies on other libraries or interpreters.
It’s super fast - Taking advantage of Go’s concurrency model: tests are executed and returned almost instantly.
It’s easy to get started with - Defining resources in YAML or JSON makes it easy for your entire team to get to grips with.

An Example

We build a web server running Apache.

Before going live someone checks that the Apache httpd package is the correct version of 2.4.25, the deployment user is in the www-data group and there is an application directory at /srv/www/app. They also check the httpd service is running, we can connect to the application at http://localhost/app and going to http://localhost/ gives a 404 error page.

To automate the above procedure with Goss, the YAML configuration would look like this:

---
package:
  httpd:
    installed: true
    versions:
    - 2.4.25
user:
  deployment:
    exists: true
    groups:
    - deployment
    - www-data
file:
  /srv/www/app:
    exists: true
    filetype: directory
service:
  httpd:
    enabled: true
    running: true
http:
  http://localhost/app:
    status: 200
    timeout: 1000
  http://localhost:
    status: 404
    timeout: 1000

You can see clearly what is being validated and this goss.yaml file can be used to consistently validate all servers of the same configuration.

Server Validation and Monitoring

All this validation of files, processes, services, ports and connectiovity sounds familiar. It’s something that we quite often try to achieve with our monitoring tools like Nagios, Zabbix or Sensu.

With Goss we can create a single monitoring check that tests many resources at once. Goss has several different outputs. The nagios_verbose output gives you out of the box testing compatible with Nagios or Sensu and gives Nagios long output explaining failures:

Server validation can now become part of your monitoring ecosystem ensuring that any problems are identified quickly with a single monitoring check.

Goss won’t replace all of your checks, it doesn’t check things like HDD space, RAM usage or errors in log files. But it makes a sweet addition to server / service monitoring.

HTTP Health Endpoint

Many applications expose "health" endpoints for applications and services.

Tools like Google’s Borgmon monitoring system and Prometheus use this HTTP(S) scrape or pull model to retrieve monitoring and metrics results from there services.

Goss has a serve command that exposes a http endpoint for scraping. You can then then use something like PHP Server Monitor to show the validation status of each piece of infrastructure.

If instrumeting your applications interests you, there’s plenty of libraries to assist you. Check out: Dropwizard Metrics and Django Health Check.

Final Thoughts

Hopefully you can see the value of validation testing your infrastructure and building validation into your monitoring systems.

Goss is a young project and currently only supports Linux but it’s very active and open to contributions. You can help out by opening issues, discussing enhancements and submitting pull requests for review.

I’ll cover some advanced Goss usage in future posts. Thanks for reading!