Karate Demo

This is a sample Spring Boot web-application that exposes some functionality as web-service end-points. And includes a set of Karate examples that test these services as well as demonstrate various Karate features and best-practices.

Example	Demonstrates
`greeting.feature`	Simple GET requests and multiple scenarios in a test.
`headers.feature`	Multiple examples of header management including dynamic setting of headers for each request using a JS file (`classpath:headers.js`). Also demonstrates handling of cookies, and path / query parameters.
`sign-in.feature`	HTML form POST example. Typically you use the response to get an authentication token that can be used to build headers for subsequent requests.
`cats.feature`	Great example of embedded-expressions (or JSON / XML templating). Also shows how to set the `Accept` header for getting XML from the server.
`kittens.feature`	Reading a complex payload expected response from a file. You can do the same for request payloads as well. Observe how JSON templating makes creating dynamic JSON super-easy, look at line #24 for example.
`upload.feature`	Multi-part file-upload example, as well as comparing the binary content of a download. Also shows how to assert for expected response headers. Plus an example of how to call custom Java code from Karate, you can extend this approach to make JDBC calls or do pretty much anything Java can.
`cats-java.feature`	Another example of how to call Java code showing how you can pass JSON data around.
`schema.feature`	Karate’s simpler approach to schema-validation compared with an actual JSON-schema example taken from json-schema.org. If you really want to perform JSON-schema validation, this example shows you how you can easily do so using Java interop and the json-schema-validator, but should you really ?
`dynamic-params.feature`	Using a `Scenario Outline` and `Examples` for data-driven testing. Since the `params` keyword takes JSON (and keys with null values are ignored), you can easily script different permutations of query parameters. This example also uses a JavaScript function (to simplify a custom assertion), which is defined in a separate file.
`call-feature.feature`	How you can re-use a sequence of HTTP calls in a `*.feature` file from other test scripts. This is hugely useful for those common authentication or ‘set up’ flows that create users, etc. Refer to the main documentation on how you can pass parameters in and get data back from the ‘called’ script.
`call-json-array.feature`	This example loads JSON data from a file and uses it to call a `*.feature` file in a loop. This approach can enable very dynamic data-driven tests, since there are a variety of ways by which you can create the JSON data, for example by calling custom Java code.
`call-table.feature`	This is a great example of how Karate combines with Cucumber and JsonPath to give you an extremely readable data-driven test. Karate’s `table` keyword is a super-elegant and readable way to create JSON arrays, perfect for setting up all manner of data-driven tests.
`call-dynamic-json.feature`	Shows how to dynamically create a JSON array and then use it to call a `.feature` file in a loop. In this example, the JSON is created using a JavaScript function, but it can very well be the response from an HTTP call, the result of a JsonPath expression or even a List of HashMap-s acquired by calling Java. This test actually calls a second `.feature` file in a loop to validate a ‘get by id’. Using JsonPath and `match each` to validate all items within a JSON array is also demonstrated.
`call-once.feature`	Cucumber has a limitation where `Background` steps are re-run for every `Scenario` and even for every `Examples` row within a `Scenario Outline`. This is a problem when you have expensive and time-consuming HTTP calls in your ‘set-up’ routines. Fortunately you have an elegant work-around with Karate’s `callonce` keyword.
`polling.feature`	If you need to keep polling until the response changes to something you expect, you can achieve this by combining JavaScript functions with calling another `*feature` file.
`JavaApiTest.java`	If you need to call a Karate test from Java code you can do so using the Java API. This is useful in some situations, for example if you want to mix API-calls into a Selenium / WebDriver test.
`CatsUiRunner.java`	(Experimental) You can use the Karate UI to debug and step-through (even replay) each step of a test. Here is a video that shows the possibilities: link

Configuration and Best Practices

| File | Demonstrates ———-| ——– karate-config.js | Shows how the demoBaseUrl property is injected into all the test scripts on startup. Notice how JavaScript allows you to perform simple conditional logic and string manipulation, while still being a ‘devops-friendly’ plain-text file. It is good practice to set the connectTimeout and readTimeout so that your tests ‘fail fast’ if servers don’t respond. Refer to the main documentation for more on configuration. TestBase.java | This is specific to Spring Boot, but this code takes care of starting the embedded app-server and dynamically chooses a free port. The chosen port value is passed to the above config routine via a Java System.setProperty() call. DemoTest.java | This Java class is strategically placed at the root of the directory structure containing *.feature files. Note how the @CucumberOptions annotation allows you to skip any *.feature files if they have @ignore at the start. The plugin option specifies the reports and formats desired, which can be over-ridden on the command-line or by the maven config described below. DemoTestParallel.java | Karate has a utility to run tests in parallel and this does not depend on JUnit, TestNG or even Maven. A JUnit XML report file and Cucumber JSON report file would be generated for each feature executed. You can easily configure your CI with the location of these files so that you get proper test-reports after a build. This is now the recommended way of running Karate as part of an automated build or CI pipeline. Here, the (optional) third-party cucumber-reporting library is being used (see details below). pom.xml | Line 81 shows how the maven-surefire-plugin can be configured to point to what is basically your ‘test-suite’. You may not even need to do this if you follow the recommended naming conventions and folder structure, and then Maven defaults would work as you would expect.

Example Report

This is optional and if you use the parallel runner as described above, the JUnit XML emitted is sufficient for most CI tools (e.g. Jenkins) to generate test reports and determine whether the build passed or failed.

Since the maven-cucumber-reporting plugin has an issue where reports will not be generated if the build fails, we recommend that you directly use the cucumber-reporting library programmatically in combination with the Karate parallel runner. Here is how:

Maven Dependency

Add the net.masterthought:cucumber-reporting jar as a dependency in test scope

<dependency>
    <groupId>net.masterthought</groupId>
    <artifactId>cucumber-reporting</artifactId>
    <version>3.8.0</version>
    <scope>test</scope>
</dependency>

Log4j Config File

If you don’t already have log4j (v2) in the mix, place this minimal config on the classpath as log4j2.properties (alongside karate-config.js).

log4j.rootLogger = INFO, CONSOLE
log4j.appender.CONSOLE = org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout = org.apache.log4j.PatternLayout

Generate Report

Refer to the code in the demo: DemoTestParallel.java, specifically the generateReport() method.

And here is the output, which goes into target/cucumber-html-reports if you follow the above steps:

Karate and Maven Cucumber Reporting

Code Coverage using Jacoco

In the pom.xml, code coverage using Jacoco is also demonstrated. Since this is set-up as a Maven profile, instrumentation and code-coverage reporting would be performed only when you use the coverage profile. Note that code-coverage data (binary) would be saved to this file: target/jacoco.exec.

So to run tests and perform code-coverage:

mvn clean test -Pcoverage

And the HTML reports would be output to target/site/jacoco/index.html.

Jacoco Code Coverage Report

As this demo example shows - if you are able to start your app-server and run Karate tests in the same JVM process, code-coverage reports for even HTTP integration tests will be very easy to generate.