Skip to main content
Version: 6.0.0

Updating Expectations

Updating expectations​

When a contribution changes the CDM model, the expected results that the CDM's regression tests check against usually need to be regenerated to match. This process is referred to as updating expectations, and it is a required step before most Pull Requests can be contributed to the CDM.

NOTE: The editor being used/supported in the examples in IntelliJ

What are expectations?​

All CDM model changes are covered by tests: a set of sample data, organised into a Test Pack, is run through the model's synonym mappings, validation and qualification logic, and the results produced are captured as expected results (or expectations). These expectations are stored as JSON files alongside the samples, mainly under rosetta-source/src/main/resources, and are compared against the model's actual output every time the relevant tests are run.

  • Test Pack samples include FpML documents and other industry data used to exercise the model's synonym-based mapping (translation) logic.
  • Function input and output samples exercise the CDM's standardised functions.
  • Each test compares the current, actual output for a given sample against its stored expectation, and fails if the two differ.

Why expectations need to be kept up to date​

Editing a data type, a synonym mapping, a validation rule, a qualification rule or a function changes the output the model produces for the existing samples. Once that happens, the expectations recorded against the previous version of the model no longer match, and the affected tests fail, even when the model change is correct and intended.

Updating expectations regenerates those expected results from the current state of the model and its samples, so the Test Pack reflects the intended behaviour rather than the previous one. This is why the development guidelines require that synonyms are updated and mapping, validation and qualification levels are maintained or improved before a change is contributed.

Note: Updating expectations rewrites files rather than checking them. A Contributor should always review the resulting diff (git status and git diff) to confirm the changes are the ones expected, and note in the contribution whether the mapping, validation or qualification success numbers have changed, as contributions should not break the current model.

Updating expectations with environment variables​

CDM 6.0.0 does not have a dedicated Maven profile for this step. Instead, expectations are regenerated by running the test suite with two environment variables set, from the root of the repository:

WRITE_EXPECTATIONS=true TEST_WRITE_BASE_PATH=$(pwd)/rosetta-source/src/main/resources mvn test
  • WRITE_EXPECTATIONS=true tells the relevant test classes, for example FunctionInputCreationTest and SecLendingFunctionInputCreationTest, to overwrite a sample's recorded expectation whenever the model's actual output no longer matches it.
  • TEST_WRITE_BASE_PATH tells those tests where to write the updated expectation files, normally the rosetta-source/src/main/resources directory at the root of the checkout.
  • Because the assertion in each test still runs against the previous value even as the new one is written to disk, the affected tests are expected to fail on this first run. Running the test suite once more, with the two environment variables unset, confirms that it now passes against the newly written expectations.

Updating expectations from an IDE​

The same two environment variables can be set from an IDE's run configuration instead of the command line, for example in IntelliJ IDEA:

  1. Open the Maven tool window (View > Tool Windows > Maven) or a Run/Debug configuration for the test phase.
  2. Add WRITE_EXPECTATIONS=true and TEST_WRITE_BASE_PATH=<absolute path to rosetta-source/src/main/resources> to its environment variables.
  3. Run the test lifecycle phase.
  4. Run it once more with the environment variables removed, to confirm the suite now passes against the newly written expectations.

Updating expectations with Rosetta​

Setting up a local Java and Maven development environment is not the only way to work with the CDM's expectations. Contributors editing the model in the Rosetta Translate application can test mapping changes live against the Test Pack, as described in the development guidelines. Rosetta, the modelling platform for the Rune DSL, is available free of charge at its community tier. Users can create a read-write workspace to edit files, contribute their changes, and Rosetta will create an update-expectations PR with updated expectations. They will also have access to the 'Validation' tab to test.JSON files for validation errors.

Note: As with any CDM application component, FINOS encourages the adoption of CDM by software providers but does not endorse any CDM application component. Please refer to the Rosetta Documentation for details of its current testing and validation capabilities.

Next steps​