Executing tests
Running tests
Microcks offers mocks but can also be used for Contract testing of API or services being under development. You spend a lot of time describing request/response pairs and matching rules: it would be a shame not to use this sample as test cases once the development is on its way!
From the page displaying basic information on your API or Service mocks, you have the ability to launch new tests against different endpoints that may be representing different environment into your development process. Hitting the NEW TEST… button, leads you to the following form where you will be able to specify a target URL for the test, as well as a Runner—a testing strategy for your new launch:
While it is convenient to launch test
on demand
manually, it may be interesting to consider launching new tests automatically when a new deployment of the application occurs for example… Microcks allows you such automation by offering API for ease of integration. See here for more details).
Test parameters
Service under test
Service under test is simply the reference of the API/Service specification we’d like to test. This a couple of Service Name
and Service Version
.
Test Endpoint
The Test Endpoint is simply a URI where a deployed component is providing API specification endpoint. In the testing literature, this is usually defined as the URI of the System Under Test.
HTTP based APIs
For HTTP based APIs (REST or SOAP), this is a simple URL that should respect following pattern:
http[s]://{service.endpoint.url:port}/{service.path}
Event based APIs
For Event based API through Async API testing, pattern is depending on the protocole binding you’d like to test.
Kafka
Kafka Test Endpoint have the following form with optional parameters placed just after a ?
and separated using &
character:
kafka://{kafka.broker.url:port}/{kafka.topic.name}[?param1=value1¶m2=value2]
Optional Params | Description |
---|---|
registryUrl |
The URL of schema registry that is associated to the tested topic. This parameter is required when using and testing Avro encoded messages. |
registryUsername |
The username used if access to the registry is secured. |
registryAuthCredSource |
The source for authentication credentials if any. Valid values are just USER_INFO . |
As an example, you may have this kind of Test Endpoint value: kafka://mybroker.example.com:443/test-topic?registryUrl=https://schema-registry.example.com®istryUsername=fred:letmein®istryAuthCredSource=USER_INFO
MQTT
MQTT Test Endpoint have the following form with no optional parameters:
mqtt://{mqtt.broker.url:port}/{mqtt.topic.name}
AMQP
AMQP 0.9.1 Test Endpoint have the following form with optional parameters placed just after a ?
and separated using &
character:
amqp://{amqp.broker.url:port}/[{amqp.vhost}/]{amqp.destination.type}/{amqp.destination.name}[?param1=value1¶m2=value2]
amqp.destination.type
is used to specify if we shoulf connect to either a queue (use the q
value) or an exchange speciyfing its type: d
dor direct, f
for fanout, t
for topic, h
for headers. Then you have to specify either the queue or exchange name in amqp.detaintion.name
.
Depending on the type of destination, you will need additional optional parameters as specified below:
Optional Params | Description |
---|---|
routingKey |
Used to specify a routing key for direct or topic exchanges. If not specified the * wildcard is used. |
durable |
Flag telling if exchange to connect to is durable or not. Default is false . |
h.{header} |
A bunch of headers where name starts with h. in order to deal with headers exchange. The x-match property is set to any to ghather the most message as possible. |
As an example, you may have this kind of Test Endpoint values: amqp://rabbitmq.example.com:5672/h/my-exchange-headers?h.h1=h1&h.h2=h2
or amqp://rabbitmq.example.com:5672/my-vhost/t/my-exchange-topic?routingKey=foo
WebSocket
WebSocket Test Endpoint have the following form with no optional parameters:
ws://{ws.endpoint.url:port}/{channel.name}
Test Runner
As stated above, Microcks offers different strategies for running tests on endpoints where our microservice being developed are deployed. Such strategies are implemented as Test Runners. Here are the default Test Runners available within Microcks:
Test Runner | API/Service Types | Description |
---|---|---|
HTTP |
REST and SOAP | Simplest test runner that only checks that valid target endpoints are deployed and available: returns a 20x or 404 Http status code when appropriated. This can be called a simple “smock test”. |
SOAP |
SOAP | Extension of HTTP Runner that also checks that the response is syntactically valid regarding SOAP WebService contract. It realizes a validation of the response payload using XSD schemas associated to service. |
SOAP_UI |
REST and SOAP | When the API artifact is defined using SoapUI: ensures that assertions put into Test cases are checked valid. Report failures. |
POSTMAN |
REST | When the API artifact is defined using Postman: executes test scripts as specified within Postman. Report failures. |
OPEN_API_SCHEMA |
REST | When the API artifact is defined using Open API: it executes example requests and check that results have the expected Http status and that payload is compliant with JSON / OpenAPI schema specified into OpenAPI specification. |
ASYNC_API_SCHEMA |
EVENT | When the API artifact is defined using Async API: it connects to specified broker endpoints, consume messages and check that payload is compliant with JSON / Avro / AsyncAPI schema specified into AsyncAPI specification. |
GRPC_PROTOBUF |
GRPC | When the API artifact is defined using gRPC: it executes example requests and check that results payload is compliant with Protocol Buffer schema specified into gRPC proto file. |
GRAPHQL_SCHEMA |
GRAPHQL | When the API is of type GraphQL: it executes example requests and check that results payload is compliant with the GraphQL Schema of the API. |
Timeout
Depending on the type of Service or Tests you are running, the specification of a Timeout maybe mandatory. This is a numerical value expressed in milliseconds.
Secret
Depending on the Test Endpoint you are connecting to, you may need additional authentication information - like credentials or custom X509 Certificates. You may reuse External Secrets that has been made available in the Microcks installation by the administrator.
Getting tests history and details
Tests history for an API/Service is easily accessible from the API | Service summary page. Microcks keep history of all the launched tests on an API/Service version. Success and failures are kept in database with unique identifier and test number to allow you to compare cases of success and failures.
Specific test details can be visualized : Microcks also records the request and response pairs exchanged with the tested endpoint so that you’ll be able to access payload content as well as header. Failures are tracked and violated assertions messages displayed as shown in the screenshot below :