Raygun: Automated Testing for Rego

Black-box testing for Policy as Code

Open Policy Agent has a built-in testing solution, but it has some limitations. I found those limitations frustrating enough that I decided to develop Raygun, which treats OPA and the policies as a “black box”. More specifically:

• You create YAML-based test suites that are independent of the policy code

• You specify the policy bundle to use for the suite

• For each test, you provide:

  • The path to the policy

  • The input JSON

  • The expected results

Raygun will launch OPA with the specified bundle. It will then act as a client, calling OPA on the policy path you’ve specified, providing it with the input you’ve specified, and will compare the response from OPA against your expectations.

Finally, it will produce a report of the results. If there are test failures, it will return with a failure code, which makes it easy to integrate into build chains.

Advantages of this approach

• You don’t have to know Rego to write the tests

• You’re testing against an OPA process (loaded with a policy bundle), which is what the real-world policy consumers are doing

• The tests provide excellent examples for developers who need to consume policy

• The tests allow for both inline and file-based input JSON, which makes maintenance and updates easier

• The tests can be maintained separately from the policy code

• Raygun includes various quality-of-life features that make it easier to troubleshoot your policy logic

Already In Use

Raygun is already being used for improving the testing of internal PACLabs projects, as well as for third-party customer policy testing.

Feedback

Please take a look if you’re so inclined. I would appreciate feedback, suggestions and opportunities to collaborate.