Skip to main content

Provider Testing Guide

Principles​

  • Testing the provider API is your responsibility. PactFlow simply ensures the specification is compatible with any consumers
  • Garbage in, Garbage out - PactFlow trusts any provider contract provided. This is true, whether it has been tested or not.
  • When using the BYO functional API testing strategy, you must ensure your API is compatible with (and ideally, implements) any specification.
  • Code-based approaches are generally preferred because they are less likely to drift from implementation. For example, using tools that generate OAS definitions from code/types is more reliable
  • When supported, test-based approaches such as ReadyAPI Functional test suites/postman collections, may also be more reliable, as they have embedded testing information in them. Uploading only the tested parts of the provider contract to PactFlow improves the guarantees we can provide.

Writing Provider Contracts​

Provider Test

Step 1: Authoring or generating your OpenAPI Specification​

You must have, or be able to produce, one of the following OpenAPI Specification (OAS) formats:

  • OAS v2.0
  • OAS v3.0.x
  • OAS v3.1.x

If you don't have an OAS, you can convert from a format you already have or from your code (e.g. via types/annotations) before uploading to Pact.

For example, there are tools that convert Postman collections or RAML documents to OAS.

Step 2: Choose an API testing tool​

There are many tools available. You may want to choose a black-box style functional API testing tool like ReadyAPI/SoapUI/Dredd or Postman, or white-box style tools such as RestAssured or Supertest.

The key consideration is ensuring your API is compatible with an OAS.

Step 3: Verifying the Provider Contract (Testing your API)​

Configure your CI pipeline to run these tests on every change. We suggest running these tests against a locally running server so that you have control and therefore determinism in your tests.

Running against a dedicated testing environment will likely result in flaky tests.

Step 4: Publish your Provider Contract and verification results​

After your tests have completed (pass/fail), you should upload the specification and results to PactFlow.

See publishing your OpenAPI Provider Contract to PactFlow for details and examples.

Step 4: Run can-i-deploy​

can-i-deploy gives you immediate feedback if you are safe to release a version of an application to a specified environment (such as production).

We recommend using the pact-broker can-i-deploy command from CLI Tools for this step.

Our examples use the Docker version to simplify administration.

The command output will provide a link to the verification results in PactFlow. Interpreting these results is contract specific.

Here is our pipeline to date for the first run of a provider:

Provider Pipeline First Run

Step 5: Deploy your application​

If can-i-deploy returns a successful response, you can deploy your application.

Once your application is deployed, you can notify PactFlow of the release - we recommend setting the branch property when you publish provider contracts and use record-deployment or record-release when you deploy/release.

Our examples use the Docker version to simplify administration.

Golden rule of deployments:

The Pact Broker needs to know which versions of each application are in each environment. So, it can return the correct contracts for verification and determine whether a particular application version is safe to deploy.

record-deployment automatically marks the previously deployed version as undeployed and is used for APIs and consumer applications deployed to known instances.

record-release does NOT change the status of any previously released version and is used for mobile applications and libraries made publicly available via an application store or repository.

Integrating it into your CI/CD pipeline​

A simplified view of a CI/CD pipeline for Pact looks like this:

Provider Pipeline

The standard principles are still relevant. Our CI/CD workshop is a useful reference (NOTE: the CI/CD workshop uses the consumer-driven mode using Pact).

Other​

Other examples of how to do this form of testing​