Micro-services Using Go-kit: gRPC endpoint

This article will show how to create micro-services using Golang programming language and Go-kit as its framework. The service will be exposed via gRPC protocol.

Sharing is caring!

Overview

In the previous article, I talked about creating micro-services using go programming language. Also, I introduced go-kit as micro-services framework. As you can see, both of them work very nicely to construct a service.

Just like before, in this article I will make another endpoint as a service. But, this time is a little bit different. Rather than exposing service through REST, I will use gRPC protocol as a communication interface.

gRPC

Image taken from http://www.grpc.io/docs/guides

I will be brief about gRPC, since  grpc.io already explain nicely about it.

In general, gRPC is a framework from google to support remote procedure call. By using gRPC, a client can invoke methods directly on a server in a different machine.

Moreover, by default gRPC uses protocol buffers as a mechanism for serializing structure data. This serialization is smaller and faster to encode and decode compare to XML or JSON.

If you want how good is protocol buffers performance vs JSON, you can visit this link.

Use Case

The use case is still as same as the previous article. I will create a service to generate “lorem ipsum” text. But there will be a slight difference this time. Rather than creating three functions (Word, Sentence and Paragraph) in one service, I will create one function called Lorem. And from that function, it will dispatch request type whether it will generate word, sentence or paragraph.

Step by Step

I assume we already have required library from the previous article. But it is not enough, we need additional libraries and protocol buffer installation.

  1. Download protocol buffer compiler from https://github.com/google/protobuf/releases. Extract and export bin folder into $PATH.
  2. Execute on terminal:
  3. Remember to put your GOBIN under your PATH.

Then create new folder, give it a name: lorem-grpc. In my case, I did it under $GOPATH/github.com/ru-rocker/gokit-playground/lorem-grpc. I will call this path WORKDIR.

Step 1:  Proto

To define the data structure, we will use protocol buffer language (of course). We will use third version, or known as proto3. Firstly, we need to create directory, inside WORKDIR, called pb, and a proto file under it. In the proto file, we specify our syntax using proto3 version. Also, we will define our service(s) and request response pair as well.

For this article, the service name is Lorem, with LoremRequest as input parameter and return LoremResponse.


syntax = "proto3";
package pb;
service Lorem {
rpc Lorem(LoremRequest) returns (LoremResponse) {}
}
message LoremRequest {
string requestType = 1;
int32 min = 2;
int32 max = 3;
}
message LoremResponse {
string message = 1;
string err = 2;
}

view raw

lorem.proto

hosted with ❤ by GitHub

The field inside message has following format:

The type value describes data type property. It can be string, bool, double, float, int32, int64, etc. The index is integer value for data stream to indicate the field position.

With protocol buffer compiler and Go plugin for protobuf, we will generate Go files based on proto file. Under pb folder:

This will create file lorem.pb.go.

Step 2: Defining Service

Just like previous article, create file service.go under our WORKDIR. Then create Lorem function inside it.


package lorem_grpc
import (
gl "github.com/drhodes/golorem"
"strings"
"errors"
"context"
)
var (
ErrRequestTypeNotFound = errors.New("Request type only valid for word, sentence and paragraph")
)
// Define service interface
type Service interface {
// generate a word with at least min letters and at most max letters.
Lorem(ctx context.Context, requestType string, min, max int) (string, error)
}
// Implement service with empty struct
type LoremService struct {
}
// Implement service functions
func (LoremService) Lorem(_ context.Context, requestType string, min, max int) (string, error) {
var result string
var err error
if strings.EqualFold(requestType, "Word") {
result = gl.Word(min, max)
} else if strings.EqualFold(requestType, "Sentence") {
result = gl.Sentence(min, max)
} else if strings.EqualFold(requestType, "Paragraph") {
result = gl.Paragraph(min, max)
} else {
err = ErrRequestTypeNotFound
}
return result, err
}

view raw

service.go

hosted with ❤ by GitHub

Step 3: Creating Endpoints

The endpoints is not much different between the previous one. One thing you need to take care of is, I implement Service interface with Endpoints struct. This mechanism is needed when you creating gRPC client connection.


package lorem_grpc
import (
"github.com/go-kit/kit/endpoint"
"context"
"errors"
)
//request
type LoremRequest struct {
RequestType string
Min int32
Max int32
}
//response
type LoremResponse struct {
Message string `json:"message"`
Err string `json:"err,omitempty"`
}
// endpoints wrapper
type Endpoints struct {
LoremEndpoint endpoint.Endpoint
}
// creating Lorem Ipsum Endpoint
func MakeLoremEndpoint(svc Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(LoremRequest)
var (
min, max int
)
min = int(req.Min)
max = int(req.Max)
txt, err := svc.Lorem(ctx, req.RequestType, min, max)
if err != nil {
return nil, err
}
return LoremResponse{Message: txt}, nil
}
}
// Wrapping Endpoints as a Service implementation.
// Will be used in gRPC client
func (e Endpoints) Lorem(ctx context.Context, requestType string, min, max int) (string, error) {
req := LoremRequest{
RequestType: requestType,
Min: int32(min),
Max: int32(max),
}
resp, err := e.LoremEndpoint(ctx, req)
if err != nil {
return "", err
}
loremResp := resp.(LoremResponse)
if loremResp.Err != "" {
return "", errors.New(loremResp.Err)
}
return loremResp.Message, nil
}

view raw

endpoints.go

hosted with ❤ by GitHub

Step 4: Model Request and Response

As usual, we need to encode / decode the request and response. For this purpose, create model.go and define encode/decode functions.


package lorem_grpc
import (
"context"
"github.com/ru-rocker/gokit-playground/lorem-grpc/pb"
)
//Encode and Decode Lorem Request
func EncodeGRPCLoremRequest(_ context.Context, r interface{}) (interface{}, error) {
req := r.(LoremRequest)
return &pb.LoremRequest{
RequestType: req.RequestType,
Max: req.Max,
Min: req.Min,
} , nil
}
func DecodeGRPCLoremRequest(ctx context.Context, r interface{}) (interface{}, error) {
req := r.(*pb.LoremRequest)
return LoremRequest{
RequestType: req.RequestType,
Max: req.Max,
Min: req.Min,
}, nil
}
// Encode and Decode Lorem Response
func EncodeGRPCLoremResponse(_ context.Context, r interface{}) (interface{}, error) {
resp := r.(LoremResponse)
return &pb.LoremResponse{
Message: resp.Message,
Err: resp.Err,
}, nil
}
func DecodeGRPCLoremResponse(_ context.Context, r interface{}) (interface{}, error) {
resp := r.(*pb.LoremResponse)
return LoremResponse{
Message: resp.Message,
Err: resp.Err,
}, nil
}

view raw

model.go

hosted with ❤ by GitHub

Step 5: Transport

In this step, we need to implement LoremServer interface for grpcServer type. Then create function to return grpcServer.


package lorem_grpc
import (
"golang.org/x/net/context"
grpctransport "github.com/go-kit/kit/transport/grpc"
"github.com/ru-rocker/gokit-playground/lorem-grpc/pb"
)
type grpcServer struct {
lorem grpctransport.Handler
}
// implement LoremServer Interface in lorem.pb.go
func (s *grpcServer) Lorem(ctx context.Context, r *pb.LoremRequest) (*pb.LoremResponse, error) {
_, resp, err := s.lorem.ServeGRPC(ctx, r)
if err != nil {
return nil, err
}
return resp.(*pb.LoremResponse), nil
}
// create new grpc server
func NewGRPCServer(ctx context.Context, endpoint Endpoints) pb.LoremServer {
return &grpcServer{
lorem: grpctransport.NewServer(
ctx,
endpoint.LoremEndpoint,
DecodeGRPCLoremRequest,
EncodeGRPCLoremResponse,
),
}
}

view raw

transport.go

hosted with ❤ by GitHub

Step 6: Server

Creating gRPC server in Go lang is almost as easy as creating HTTP server. The different is, we are using tcp protocol rather than http. Under your WORKDIR, create folder server and make file server_grpc_main.go.


package main
import (
"net"
"flag"
"github.com/ru-rocker/gokit-playground/lorem-grpc"
context "golang.org/x/net/context"
"google.golang.org/grpc"
"github.com/ru-rocker/gokit-playground/lorem-grpc/pb"
"os"
"os/signal"
"syscall"
"fmt"
)
func main() {
var (
gRPCAddr = flag.String("grpc", ":8081",
"gRPC listen address")
)
flag.Parse()
ctx := context.Background()
// init lorem service
var svc lorem_grpc.Service
svc = lorem_grpc.LoremService{}
errChan := make(chan error)
// creating Endpoints struct
endpoints := lorem_grpc.Endpoints{
LoremEndpoint: lorem_grpc.MakeLoremEndpoint(svc),
}
//execute grpc server
go func() {
listener, err := net.Listen("tcp", *gRPCAddr)
if err != nil {
errChan <- err
return
}
handler := lorem_grpc.NewGRPCServer(ctx, endpoints)
gRPCServer := grpc.NewServer()
pb.RegisterLoremServer(gRPCServer, handler)
errChan <- gRPCServer.Serve(listener)
}()
go func() {
c := make(chan os.Signal, 1)
signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
errChan <- fmt.Errorf("%s", <-c)
}()
go func() {
c := make(chan os.Signal, 1)
signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
errChan <- fmt.Errorf("%s", <-c)
}()
fmt.Println(<- errChan)
}

view raw

server_grpc_main.go

hosted with ❤ by GitHub

Step 7: Client

It is time to make the client side. What we need is create new client function that return Service. Because the NewClient function will be converted into Endpoint.


package client
import (
"github.com/ru-rocker/gokit-playground/lorem-grpc"
"github.com/ru-rocker/gokit-playground/lorem-grpc/pb"
grpctransport "github.com/go-kit/kit/transport/grpc"
"google.golang.org/grpc"
)
// Return new lorem_grpc service
func New(conn *grpc.ClientConn) lorem_grpc.Service {
var loremEndpoint = grpctransport.NewClient(
conn, "Lorem", "Lorem",
lorem_grpc.EncodeGRPCLoremRequest,
lorem_grpc.DecodeGRPCLoremResponse,
pb.LoremResponse{},
).Endpoint()
return lorem_grpc.Endpoints{
LoremEndpoint: loremEndpoint,
}
}

view raw

client.go

hosted with ❤ by GitHub

Take note, I return Endpoint for this function (see Step 3 about implementing Service interface).

Then create executable client under cmd directory.


package main
import (
"flag"
"time"
"log"
grpcClient "github.com/ru-rocker/gokit-playground/lorem-grpc/client"
"google.golang.org/grpc"
"golang.org/x/net/context"
"github.com/ru-rocker/gokit-playground/lorem-grpc"
"fmt"
"strconv"
)
func main() {
var (
grpcAddr = flag.String("addr", ":8081",
"gRPC address")
)
flag.Parse()
ctx := context.Background()
conn, err := grpc.Dial(*grpcAddr, grpc.WithInsecure(),
grpc.WithTimeout(1*time.Second))
if err != nil {
log.Fatalln("gRPC dial:", err)
}
defer conn.Close()
loremService := grpcClient.New(conn)
args := flag.Args()
var cmd string
cmd, args = pop(args)
switch cmd {
case "lorem":
var requestType, minStr, maxStr string
requestType, args = pop(args)
minStr, args = pop(args)
maxStr, args = pop(args)
min, _ := strconv.Atoi(minStr)
max, _ := strconv.Atoi(maxStr)
lorem(ctx, loremService, requestType, min, max)
default:
log.Fatalln("unknown command", cmd)
}
}
// parse command line argument one by one
func pop(s []string) (string, []string) {
if len(s) == 0 {
return "", s
}
return s[0], s[1:]
}
// call lorem service
func lorem(ctx context.Context, service lorem_grpc.Service, requestType string, min int, max int) {
mesg, err := service.Lorem(ctx, requestType, min, max)
if err != nil {
log.Fatalln(err.Error())
}
fmt.Println(mesg)
}

view raw

client_cmd_main.go

hosted with ❤ by GitHub

Running

To test how it is working, at first we need to run the gRPC server.

Next, execute the client

Output:

Conclusion

Creating gRPC endpoint is quite simple and only has a little bit different with REST endpoint. However, gRPC runs faster rather than REST. My feeling is, gRPC will replace JSON in the future.

Meanwhile, if you are interested in this sample, you can check the source code on my github.

Have a blessed day.

Author: ru rocker

I have been a professional software developer since 2004. Java, Python, NodeJS, and Go-lang are my favorite programming languages. I also have an interest in DevOps. I hold professional certifications: SCJP, SCWCD, PSM 1, AWS Solution Architect Associate, and AWS Solution Architect Professional.

12 thoughts on “Micro-services Using Go-kit: gRPC endpoint”

  2. Great tutorial, But I get error: code = Unimplemented desc = unknown service Lorem, I even cloned your repo, and made some fixes on ctx but no luck. any guidance would be appreciated

    Reply

      1. in client.go

        var loremEndpoint = grpctransport.NewClient(
        conn, "Lorem", "Lorem", // <- you change to this         conn, "pb.Lorem", "Lorem"
        lorem_grpc.EncodeGRPCLoremRequest,
        lorem_grpc.DecodeGRPCLoremResponse,
        pb.LoremResponse{},
        ).Endpoint()

        Reply

    2. Hi. I ran into the same error rpc error: code = Unimplemented desc = unknown service Lorem.
      Did you manage to fix it ?

      Reply

      1. @KAMAL
        In client.go under func New.
        Where it defines NewClient,
        conn, “Lorem”, “Lorem”
        should be
        conn. “pb.Lorem”, “Lorem”

        Reply
  4. Pingback: Go Kit (2) – Go kit design is well-suited for modern software design: both microservices and so-called elegant monoliths. – HPC Blog

  5. this works

    // New Return new lorem_grpc service
    func New(conn *grpc.ClientConn) lorem_grpc.Service {

    var loremEndpoint = grpctransport.NewClient(
    conn, "pb.Lorem", "Lorem",
    lorem_grpc.EncodeGRPCLoremRequest,
    lorem_grpc.DecodeGRPCLoremResponse,
    pb.LoremResponse{},
    ).Endpoint()

    return lorem_grpc.Endpoints{
    LoremEndpoint: loremEndpoint,
    }
    }

    Reply

  6. If anyone is still having problems, it should be here:

    func New(conn *grpc.ClientConn) loremgrpc.Service {
    var loremEndpoint = grpctransport.NewClient(
    conn, “pb.Lorem”, “Lorem”,
    loremgrpc.EncodeGRPCLoremRequest,
    loremgrpc.DecodeGRPCLoremResponse,
    pb.LoremResponse{},
    ).Endpoint()

    return loremgrpc.Endpoints{
    LoremEndpoint: loremEndpoint,
    }
    }

    Reply

  8. How to map encode and decode if we have multiple endpoints is there any example I can refer to which follows the same pattern

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *