As part of my work on standards under GA4GH, I frequently deal with API specifications, most often written in OpenAPI 3 (OAS 3). The setup is always similar – there’s a public GitHub repository containing an OAS 3 YAML file and a few common auxiliary files (.gitignore, LICENSE, CONTRIBUTING.md, and README.md). Reference implementations of the specifications and other tools live in separate repositories. Contributions happen through pull requests from the main repository or its forks, typically under the Gitflow model.

The OAS 3 YAML file is the core of a standard. As such, it’s important for us to keep it correct at all times, and in particular make sure it remains valid with each pull request. Different standards have different levels of compliance testing with respect to their reference implementations, but one thing is always the same – we want to validate the specification for each pull request as part of our continuous integration. This post describes how we do it.

To validate the specification, we use the Swagger Validator Badge tool run in Travis CI. As such, the whole setup is specified in a .travis.yml file in the root of the repository.

The file starts with a pretty standard preamble:

We use Java, because we need to run a Maven build to get the Swagger Validator Badge up and running (see below). We construct the URL to our YAML file, to validate against our local validator instance, which we store in environment variables:

The snippet above assumes the specification file is named openapi.yaml (change the value in $FILE_TO_VALIDATE if you’re using a different name), and constructs a URL to the file in a way that works across pull requests and forks. For more information, see Obtaining a URL to a file in a GitHub repository in a Travis CI build.

We follow with before_install phase, where we prepare a local Swagger Validator Badge instance. We do this by cloning the repository of the tool and building the v2.0.0 tag. At the time of writing this post, Swagger Validator Badge doesn’t fully support OAS 3.0. v2.0.0 is the latest release with some support, and that’s why we need to do this step. The tool is under active development, and the 2.0 branch changes in backward-incompatible ways (lesson learned the hard way), so it’s important to use a tag here.

If your API is specified using OAS 2, you can skip the whole before_install step and just execute a request against the hosted version of Swagger Validator Badge. If you’re using OAS 3, you’ll be able to remove the step once the hosted version switches to the 2.0 release line. Until then, we build the tool ourselves, run it in the background, and wait for a bit until it starts:

The actual validation is performed in the script section. We execute a curl request against the validator with the URL to our YAML file. We use the /debug endpoint of the validator, which returns validation messages in JSON format. This output includes errors pointing out issues with our YAML file, as well as warnings for things like features not yet supported by the validator tool. We print the output of the validator to make analysis of failed builds easier, and use jq to count errors, ignoring warnings. We fail the build if the number of errors is not 0:

Putting everything together, we get the following .travis.yml file:

You can see this approach in action with several GA4GH standards, e.g. Beacon, Service Registry, Service Info, Consent Codes, ADA-M, and others.