v0.25.2

This release extends the HTTP server authorizer (--authorization=basic) to supply the HTTP message body in the input document. See the Authentication and Authorization section in the security documentation for details.

v0.25.1

This release contains a fix for running OPA under Docker with a non-default working directory (#2974):

/opa: error while loading shared libraries: libwasmer.so: cannot open shared object file: No such file or directory

v0.25.0

This release contains a number of improvements and fixes. Importantly, this release includes a notable change to built-in function error handling. See the section below for details.

Built-in Function Error Handling

Previously, built-in function errors would cause policy evaluation to halt immediately. Going forward, by default, built-in function errors no longer halt evaluation. Instead, expressions are treated as false/undefined if any of the invoked built-in functions return errors.

This change resolves a common issue people face when passing unsanitized input values to built-in functions. For example, prior to this change the expression io.jwt.decode("GARBAGE") would halt evaluation of the entire policy because the string is not a valid encoding of a JSON Web Token (JWT). If the expression was io.jwt.decode(input.token) and the user passed an invalid string value for input.token the same error would occur. With this change, the same expression is simply undefined, i.e., there is no result. This means policies can use negation to test for invalid values. For example:

decision := {"allowed": allow, "denial_reason": reason}

default allow = false

allow {
  io.jwt.verify_hs256(input.token, "secret")
  [_, payload, _] := io.jwt.decode(input.token)
  payload.role == "admin"
}

reason["invalid JWT supplied as input"] {
  not io.jwt.decode(input.token)
}

If you require the old behaviour, enable "strict" built-in errors on the query:

Caller	Example
HTTP	POST /v1/data/example/allow?strict-builtin-errors
Go (Library)	rego.New(rego.Query("data.example.allow"), rego.StrictBuiltinErrors(true))
CLI	opa eval --strict-builtin-errors 'data.example.allow'

If you have implemented custom built-in functions and require policy evaluation to halt on error in those built-in functions, modify your built-in functions to return the topdown.Halt error type.

Built-in Functions

This release includes a few new built-in functions:

• base64url.encode_no_pad, hex.encode, and hex.decode for dealing with encoded data (#2849) authored by @johanneslarsson
• json.patch for applying JSON patches to values inside of policies (#2839) authored by @jaspervdj-luminal
• json.is_valid and yaml.is_valid for testing validity of encoded values (authored by @jaspervdj-luminal)

There were also a few fixes to existing built-in functions:

• Fix unicode handling in a few string-related functions (#2799) authored by @anderseknert
• Fix http.send to override no-cache HTTP header when force_cache specified (#2841) authored by @anderseknert
• Fix strings.replace_n to replace overlapping patterns deterministically (#2822)
• Fix panic in units.parse_bytes when passed a zero-length string (#2901)

Miscellaneous

This release adds new credential providers for management services:

• GCP metadata server (#2938) authored by @kelseyhightower
• AWS Web Identity credentials (#2462) authored by @RichiCoder1
• OAuth2 (#1205) authored by @anderseknert

In addition the following server features were added:

• Add shutdown wait period flag to opa run (--shutdown-wait-period) (#2764) authored by @bcarlsson
• Add bundle file size limit configuration option (bundles[_].size_limit_bytes) to override default 1GiB limit (#2781)
• Separate decision log and status message logs from access logs (which useful for running OPA at log level error while continuing to report decision and status log to console) (#2733) authored by @anderseknert

Fixes

• Fix panic caused by race condition in the decision logger (#2835) authored by @kubaj
• Fix decision logger to flush on graceful shutdown (#780) authored by @anderseknert
• Fix --verification-key handling to accept PEM files (#2796)
• Fix --capabilities flag in opa build command (#2848) authored by @srenatus
• Fix loading of signed persisted bundles (#2824)
• Fix API response mutation caused by decision log masking (#2752) authored by @gshively11
• Fix evaluator to prevent with statements from mutating original input document (#2813)
• Fix set iteration runtime to be O(n) instead of O(n^2) (#2966)
• Increased OPA version telemetry report timeout from 1 second to 5 seconds to deal with slow networks

Documentation

• Improve docs to mention built-in function support in WebAssembly compiled policies
• Improve docs around JWT HMAC encoding (#2870) authored by @anderseknert
• Improve HTTP authorization tutorial steps for zsh (#2917 authored by @ClaudenirFreitas)
• Improve docs to describe meaning of Prometheus metrics
• Remove mention of unsafe (and unsupported) "none" signature algorithm from JWT documentation

WebAssembly

This release also includes a number of improvements to the Wasm support in OPA. Importantly, OPA now integrates a Wasm runtime that can be used to execute Wasm compiled policies. The runtime is integrated into the existing "topdown" evaluator so that specific portions of the policy can be compiled to Wasm as a performance optimization. When the evaluator executes a policy using the Wasm runtime it emits a special Wasm trace event. The Wasm runtime support in OPA is currently considered experimental and will be iterated on in coming releases.

This release also extends the Wasm compiler in OPA to natively support the following built-in functions (in alphabetical order):

• base64.encode, base64.decode, base64url.encode, and base64url.decode
• glob.match
• json.marshal and json.unmarshal
• net.cidr_contains, net.cidr_intersects, and net.cidr_overlap
• regex.match, regex.is_valid, and regex.find_all_string_submatch_n
• to_number
• walk

Backwards Compatibility

• The --insecure-addr flag (which was deprecated in v0.10.0) has been removed completely (#763)

v0.24.0

This release contains a number of small enhancements and bug fixes.

Bundle Persistence

This release adds support for persisting bundles for recovery purposes. When persistence is enabled, OPA will save activated bundles to disk. On startup, OPA checks for persisted bundles and activates them immediately. This allows OPA to startup if the bundle server is unavailable (#2097). For more information see the Bundle documentation.

Built-in Functions

This release includes a few new built-in functions:

• base64.is_valid for testing if strings are valid base64 encodings (#2690) authored by @carlpett
• net.cidr_merge function for merging sets of IPs and CIDRs (#2692)
• urlquery.decode_object for parsing URL query parameters into objects (#2647) authored by @GBrawl

In addition, http.send has been enhanced to support caching overrides and in-band error handling (#2666 and #2187).

Fixes

• Fix opa build to support custom built-in functions (#2738) authored by @gshively11
• Fix for file watching volume mounted configmaps (#2588) authored by @drewwells
• Fix discovery plugin to set last request and last successful request timestamps in status updates (#2630)
• Fix planner crash on virtual document iteration (#2601)
• Fix decision logger to requeue failed chunks (#2724 authored by @anderseknert)
• Fix object/set implementation in WASM-C library to avoid resizing.
• Fix JSON parser in WASM-C library to copy memory for strings and numbers.
• Improve WASM-C library to recycle object and set element structures while growing.

In addition, this release contains several fixes for panics identified by fuzzing:

• ast: Fix compiler to expand exprs in rule args (#2649)
• ast: Fix output var analysis to accept refs with non-var heads (#2678)
• ast: Fix panic during local var rewriting (#2720)
• ast: Fix panic in local var rewriting caused by object corruption (#2661)
• ast: Fix panic in parser post-processing of expressions (#2714)
• ast: Fix parser to ignore rules with args and key in head (#2662)
• ast: Fix object corruption during safety reordering
• types: Fix panic on reference to object with composite key (#2648)

Backwards Compatibility

• Renamed timer_rego_builtin_http.send_ns to timer_rego_builtin_http_send_ns to avoid issues with periods in metric keys.
• Removed deprecated watch package (#2265)

Miscellaneous

• Add support for H2C on HTTP listener (#2739 thanks @srenatus!).
• Add Go version information to opa version output (thanks @srenatus!)
• The official OPA build has been updated to Go v1.14.9. Previously it was using v1.13.7 which is no longer supported (thanks @srenatus!)

v0.23.2

This release contains a fix for a regression in v0.23.1 around bundle downloading. The bug caused OPA to cancel bundle downloads prematurely. Users affected by this issue would see the following error message in the OPA logs:

[ERROR] Bundle download failed: bundle read failed: archive read failed: context canceled
  plugin = "bundle"
  name = <bundle name>

v0.23.1

Fixes

• plugins/discovery: Set the last request and last successful request in discovery status (#2630)

Miscellaneous

• plugins/rest: Add response header timeout for REST client

v0.23.0

http.send Caching

The http.send built-in function now supports caching across policy queries. The caching.inter_query_builtin_cache.max_size_bytes configuration setting places a limit on the amount of memory that will be used for built-in function caching. By default, not limit is set. For http.send, cache duration is controlled by HTTP response headers. For more details see the http.send documentation.

Capabilities

OPA now supports a capabilities check on policies. The check allows callers to restrict the built-in functions that policies may depend on. If the policies passed to OPA require built-ins not listed in the capabilities structure, an error is returned. The capabilities check is currently supported by the check and build sub-commands and can be accessed programmatically on the ast.Compiler structure. The repository also includes a set of capabilities files for previous versions of OPA under the capabilities/ directory.

For example, given the following policy:

package example

deny["missing semantic version"] {
  not valid_semantic_version_tag
}

valid_semantic_version_tag {
  semver.is_valid(input.version)
}

We can check whether it is compatible with different versions of OPA:

# OK!
$ opa build ./policies/example.rego --capabilities ./capabilities/v0.22.0.json

# ERROR!
$ opa build ./policies/example.rego --capabilities ./capabilities/v0.21.1.json

Built-in Functions

This release includes a new built-in function to test if a string is a valid regular expression: regex.is_valid.

WebAssembly

• Host environments no longer have to provide the opa_println function when instantiating compiled policy modules.
• SDKs no longer have to set the heap top address during initialization.

Fixes

• Add a new inter-query cache to cache responses across queries (#1753)
• Fix opa CLI flags to match documentation (#2586) authored by @OmegaVVeapon
• Fix rule indexing when multiple glob.match mappers are required (#2617)
• Fix AST to marshal non-string object keys (#516)
• Fix signature calculation to include port if necessary (#2568)
• Fix partial evaluation to check function output for false values (#2573)

Miscellaneous

• Add http.send latency to query metrics (#2034)
• Add support for opa build unknowns under data (#2581)
• Add support to wait for plugin readiness before starting server
• Add parameter to set wall clock time during evaluation for replay purposes
• Fix groundness bit on objects during update
• Fix x509 built-in functions to parse PEM or DER inputs
• Fix bundle signing and verification to use standard JWT key ID header
• Optimize AST collections to cache hash values
• Optimize object iteration to avoid hashing
• Optimize evaluator by removing unnecessary term copying

Deprecations

• The watch query parameter on the Data API has been deprecated. The query watch feature was unused and the lack of incremental evaluation would have introduced scalability issues for users. The feature will be removed in a future release.

• The partial query parameter on the Data API has been deprecated. Note, this only applies to the partial query parameter that the Data API supports, not Partial Evaluation itself. The partial parameter allowed users to lazily trigger Partial Evaluation (for optimization purposes) during a policy query. While this is useful for kicking the tires in a development environment, putting optimization into the policy query path is not recommended. If users want to kick the tires with Partial Evaluation, we recommend running the opa build command.

Backwards Compatibilty

• The storage.Indexing interface has been removed. Storage indexing has not been supported since 0.5.12. It was time to remove the interface. Custom store implementations that may have included no-op implementations of the interface can be updated.

• The ast.Array type has been redefined a struct. Previously ast.Array was a type alias for []*ast.Term. This change is backwards incompatible because slice operations can no longer be performed directly on values of type ast.Array. To accomodate, the ast.Array type now exports functions for the same operations. This change decouples callers from the underlying array implementation which opens up room for future optimizations.

v0.22.0

Bundle Signing

OPA now supports digital signatures for policy bundles. Specifically, a signed bundle is a normal OPA bundle that includes a file named ".signatures.json" that dictates which files should be included in the bundle, what their SHA hashes are, and of course is cryptographically secure. When OPA receives a new bundle, it checks that it has been properly signed using a key that OPA has been configured with out-of-band. Only if that verification succeeds does OPA activate the new bundle; otherwise, OPA continues using its existing bundle and reports an activation failure via the status API and error logging. For more information see https://openpolicyagent.org/docs/latest/management/#signing. Many thanks to @ashish246 who co-designed the feature and provided valuable input to the development process with his proof-of-concept #1757.

Optimization Levels

opa build now supports multiple optimization levels. The first level (--optimize=1) enables constant folding (based on partial evaluation) that only inlines values that can be computed entirely at build time. The second level (--optimize=2) enables the existing (more aggressive) version of partial evaluation that eagerly inlines as much of the policy as possible. For more information on the optimization levels see the Optimization Levels section in the documentation.

Built-in Functions

• numbers.range (#2479) was added to support policies that need to generate a range of integers (e.g., a network port range).
• semver.is_valid and semver.compare (#2538) was added to support policies that need to validate semantic version numbers (authored by @charlieegan3).

WebAssembly

• All String built-in functions (except sprintf) are now implemented natively inside of Wasm-compiled policies.

Fixes

• A few small issues in the Go integration and rego package examples have been resolved (#2294) and #2367) authored by @gaga5lala.
• The Kubernetes Admission Controller tutorial as been updated to work with recent versions of Kubernetes (#2467 authored by @gaga5lala).
• A few issues in partial evaluation around negation inlining and partial rules have been resolved (e.g., #2492, #2491).

Miscellaneous

• OPA now supports IMDSv2 for the AWS metadata service. This improves the security posture of OPA deployments in AWS (#2482) authored by @nhw76.
• Several improvements to the project documentation including a policy style discussion, an integration option comparison, and discussion of bootstrapping and fail-open versus fail-closed modes.
• The project's CI/CD infrastructure has been migrated to GitHub Actions. The new CI/CD infrastructure is designed and implemented to be portable and includes a number of quality-of-life improvements.
• End-to-end query latency with decision logging enabled has been improved by 10%-15% in real-world cases.

Backwards Compatibility

• The rego.Tracer and rego.EvalTracer API's have been deprecated in favor of
  the newer rego.QueryTracer and rego.EvalQueryTracer API.
• The tester.Runner#SetCoverageTracer API has been deprecated in favor of the
  newer test.Runner#SetCoverageQueryTracer API.

v0.21.1

This release fixes #2497 where the comprehension indexing optimization produced incorrect results for nested comprehensions that close over variables in the outer scope. This issue only affects policies containing nested comprehensions that are recognized by the indexer (which is a relatively small percentage).

This release also backports the GitHub Actions migration and a fix to the Wasm library build step.

v0.21.0

Features

• Decision log masks can now mutate decision log events. Previously, the masks could only erase data in the events. With this change, users can implement masks that obfuscate or add information to the decision log events before they are emitted. Thanks to @dkiser for implementing this feature #2379)!

• This release contains a new built-in function for parsing X.509 Certificate Signing Requests (crypto.x509.parse_certificate_request). Thanks to @vivekbagade for implementing this feature #2402!

• This release adds support for aggregation and bit arithmetic operations for WebAssembly compiled policies. These functions no longer have to be provided by the host environment.

Fixes

• cmd: Fix bug in --disable-inlining option parsing (#2196) authored by @Syn3rman
• docs: Improve terraform example to incorporate child_modules (#1772)
• server: Fix panic caused by compiler misuse with bundles (#2197)
• topdown: Fix incorrect memoization during partial evaluation (#2455)
• topdown: Fix loss of precision in arithmetic and aggregate builtins (#2469)

Miscellaneous

• Thanks to @Syn3rman for implementing an improvement to our release process to automatically tag external contributors (#2323)!

• The coverage and profiling tracers no longer require variable values from the evaluator. This change improves perfomance significantly when coverage or profiling is enabled and policies inspect large data sets. Benchmarks show anywhere from 0.5x to over 30x speedup depending on the policy.

Backwards Compatibility

• topdown.Tracer has been deprecated in favor of a newer interface
  topdown.QueryTracer.
• All tracers (regardless of interface implementation) will now only be checked
  for being enabled at the beginning of query evaluation rather than on a
  per-event basis.
• topdown.BuiltinContext#Tracers has been deprecated in favor of
  topdown.BuiltinContext#QueryTracers. The older Tracers field will be nil
  starting this release, and eventually removed.