Introduction

This idea came when I had to deal with external API integration. One of the requirements states that the system needs to connect with 3rd party REST API. From the coding point of view, it is not a big deal. Multiple libraries provide easy ways to consume a REST API. For example, Java + Spring Frameworks accommodate with RestTemplate or FeignClient. However, in the testing part, it is a different story. There are many difficulties (at least I faced them) during test execution.

1. Data Requirement Complexity

External API means you do not have total control over the data. Sometimes, there are too many specifications and validations to receive a 2xx HTTP response. And to make things worse, the test API is not immutable. The data becomes obsolete after you invoke the API (commonly in the POST method). I agree if you are in the UAT or pre-production phase. You need to keep logic and data consistent from end to end. But in the development phase, with a small set of data provided, you do not have the luxury to have a test API that changes your test data, and you cannot rerun your test with the same data.

2. No API for Testing

Contrary to a previous point, your API partner does not have a testing endpoint. Or they have an API for testing, but they still charge you. It will cost you a fortune if you want to do some test automation times and times again.

3. Corporate Proxy

The main problem. In some companies, there is a regulation about internet connections. For instance, development servers cannot access the internet. What will happen if you need to access an external API but do not have internet access? Blocker! I know we can go to our infra and security guy to open access for specific IP. But it is not last long. They usually give you one or two months to access then it is expired. Therefore you need to ask again and again.

This issue has another implication as well. With the corporate proxy, we cannot access some ready-made mock REST servers on the internet, such as https://getsandbox.com/.

Hand Made Mock REST API to Rescue

With those problem statements, I had the initiative to create a simple and configurable REST Mock API. The idea was simple. I picked some programming language that supports an HTTP server by default. Next, I separated the REST specifications into a configuration file. So, I chose Golang with Gin as a web framework as my programming language and YAML format for a configuration file.

Code Structure

The code repository is on my GitHub (https://github.com/ru-rocker/mock-rest-api). Please checkout first before continue reading the next part.

Not many surprises here. I separate the code base into two files. The first one is for the YAML parser. It is located under package parser (parser/yaml_parser.go). The second one is the main one for running an HTTP server. The parser will read a configuration file from the MOCK_CONFIG_FILE environment variable. The default location is under config/mock.yaml whenever you do not set the environment variable. Worth noting that you can pass the configuration file from the filesystem or the HTTP location. Please see the README.md for more details.

Configuration File

I am going to be a little deep for configuration file.

	name: sample-mock
	# advertised listening address.
	hostname: 0.0.0.0
	# http port
	port: 3000
	# pre-flight options
	options:
	accessControlAllowOrigin: '*'
	accessControlAllowCredentials: 'true'
	accessControlAllowHeaders: Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With
	accessControlAllowMethods: POST,HEAD,PATCH, OPTIONS, GET, PUT
	# list of routes
	routes:
	- method: GET
	endpoint: /product/:productId
	responses:
	- statusCode: 400
	body: \|
	{
	"code": "400",
	"message": "Bad Request",
	"productId": :productId
	}
	# Condition for this request.
	# GET /product/:productId will return response code 400 if there is no X_REQ_ID in the request header
	condition:
	# type: request_header, request_param, query_param and request_body
	type: request_header
	# the key. if the type is request_body, then it is in the form of json path.
	key: X_REQ_ID
	# request header value
	value:
	# state for comparison: absent, present or equal.
	# For request param, absent state does not make sense. Do not put absent state if you type is request_param.
	state: absent
	# GET /product/:productId will return response code 200 if there is a X_REQ_ID in the request header with value ABC
	- statusCode: 200
	body: \|
	{
	"code": "200",
	"message": "OK",
	"productId": :productId
	}
	condition:
	type: request_header
	key: X_REQ_ID
	value: ABC
	state: equal
	# GET /product/:productId will return response code 202 if there is a X_REQ_ID in the request header with value other than ABC
	- statusCode: 202
	body: \|
	{
	"code": "202",
	"message": "Accepted",
	"productId": :productId
	}
	condition:
	type: request_header
	key: X_REQ_ID
	value: ABC
	state: present
	- method: POST
	endpoint: /echo
	responses:
	# POST /echo will return response code 200 if there is a name field in the JSON request body.
	# The request will be delayed between 100 - 1000ms
	- statusCode: 200
	body: \|
	{
	"code": "200",
	"message": "OK"
	}
	# delay the response in millisecond. Generate random milliseconds time in range. Put min == max for fix time.
	delay:
	min: 100
	max: 1000
	condition:
	type: request_body
	key: $.name
	value:
	state: present
	# POST /echo will return response code 400 if there is no name field in the JSON request body.
	# The request will be delayed is 1000ms
	- statusCode: 400
	body: \|
	{
	"code": "400",
	"message": "Bad Request"
	}
	delay:
	min: 1000
	max: 1000
	condition:
	type: request_body
	key: $.name
	value:
	state: absent

view raw mockresponse.yaml hosted with ❤ by GitHub

The first one is the name field. Not a very useful field, but it is still a good thing for everything to have a name. The next ones are hostname and port. The hostname field is for the listening address. Meanwhile, the port field is for the target port. Next is the options field. You will need this if you need to simulate the pre-flight request. The last one is the routes part. It contains the required HTTP request and response information, such as HTTP method, endpoint, response code, response header, condition and response payload/body.

Delay

To simulate a production-like situation, you can put a delay parameter. The delay will generate a random number between the min and max values. Leave it empty if you do not want to set the delay.

Condition

The beauty part. You can have a condition in your routes. The condition part will select which response will be selected based on specific rules. For instance, you want to return response code 409 if the request does not contain the X_REQ_ID header. Otherwise, the system will return 200.

There are four conditional types: request_header, request_param, query_param, and request_body. Each type can have three states: absent, present, and equal. I think you already guessed the purpose of each state. Absent means the rule will be called if there is no value present. And for the present type, it is the opposite of absent. The equal type means the value must be identical. One last thing, the system always takes the first matching rules. Therefore, the order determines the precedence. A smaller response index is always the winner.

Running without Golang Runtime

Well, I already shipped the REST mock API server as a container. And good news, I already push the image into the docker hub. So pull the image with the name rurocker/mock-rest-api with tag latest. Whenever you need an assistant to execute the image as a docker container, please visit the README.md and go to Dockerized part.

Is it support OpenAPI 3.0?

The simple answer is no. Let’s be honest. There are many outdated APIs out there. This is a common thing as of now. The API consumption is growing, but the existing API is rarely updated. I could understand why they were reluctant to upgrade it. As a wise person says: if it is not broken, don’t fix it. Especially if the APIs are already consumed publicly in large transactions. You cannot expect a fully-fledged API specification, with OpenAPI 3.0 compliant. The best things you can get are the spec of a request method, request path, request parameters, request headers, and payload. In case you need an OpenAPI 3 compliant mock server, please visit this repository (https://github.com/muonsoft/openapi-mock).

Bonus Stage: Run Under Kubernetes

Basically, the article is finished. But I am in the mood for writing. Therefore I give you the Kubernetes part. So let’s begin the extra time. First, take a look at our Kubernetes deployment YAML.

	apiVersion: apps/v1
	kind: Deployment
	metadata:
	name: mock-endpoint
	namespace: ns1
	spec:
	selector:
	matchLabels:
	name: mock-endpoint
	template:
	metadata:
	labels:
	name: mock-endpoint
	spec:
	volumes:
	- name: mock-data
	configMap:
	name: mock-data
	containers:
	- name: mock-endpoint
	image: rurocker/mock-rest-api:latest
	env:
	- name: MOCK_CONFIG_FILE
	value: /app/mock-response.yaml
	imagePullPolicy: IfNotPresent
	ports:
	- name: http
	containerPort: 3000
	protocol: TCP
	resources:
	limits:
	cpu: 100m
	memory: 64Mi
	volumeMounts:
	- name: mock-data
	mountPath: /app/mock-response.yaml
	subPath: mock-response.yaml
	---
	apiVersion: v1
	kind: Service
	metadata:
	name: mock-endpoint
	namespace: ns1
	spec:
	type: ClusterIP
	ports:
	- port: 3000
	targetPort: 3000
	protocol: TCP
	selector:
	name: mock-endpoint

view raw deployment.yaml hosted with ❤ by GitHub

If you are familiar with Kubernetes, I believe you already know I will deploy the REST mock API and expose it as a service with port 3000. We will use the mock configuration file above and store it as a config map. Then the deployment will mount the config map as a volume and pass the mount volume to a MOCK_CONFIG_FILE environment variable. Now it is time to apply our configuration.

	# create configmap
	kubectl create configmap mock-data --from-file=mock-response.yaml --namespace ns1

	# deployment
	kubectl apply -f deployment.yaml

view raw ns.sh hosted with ❤ by GitHub

And the result will be similar like this.

/product/:productId

/echo

Note: please compare the configuration file and the json response.

Conclusion

Well, I know there are many REST mock API servers out there. Many of them support OpenAPI 3 and/or swagger. But I believe there is a little room for the more freestyler ones. Therefore I came up with this idea and wrote it in an article. Hopefully, this one will help you to speed up your development. That’s all for now, stay healthy and happy.

PS: If you want to support the content creator, please take a look at the README.md. I post the crypto wallet there.

Author: ru rocker

I have been a professional software developer since 2004. Java, Python, NodeJS, and Go-lang are my favorite programming languages. I also have an interest in DevOps. I hold professional certifications: SCJP, SCWCD, PSM 1, AWS Solution Architect Associate, and AWS Solution Architect Professional. View all posts by ru rocker

Tailor-Made REST Mock API Server to Speed Up Development Time