Microservices and cloud native development using Micro

Introduction

For the past few years, Microservices has been all the rage and it has helped a lot of organizations to adopt continuous delivery and evolve its technology stack. There are quite a few programming tool kits and platforms that have sprung up to enable building Microservices. One among them is Micro, which is a cloud native development platform that addresses the key requirements for building services in the cloud. In their latest version v3, Micro provides service discovery, pub/sub, transport protocols and many other features out of the box. It is pretty easy to define protocol buffers and grpc services using Micro v3. Here I will provide a walk through of how to setup a simple micro service using Micro v3.

Micro gateway server

For this walk through, we want to be able to make HTTP calls to the service and the service is expected to provide JSON data. We’ll need a micro server that acts as the gateway and all services will register themselves with this server. The micro gateway can be made to authenticate or ratelimit requests by defining plugins, but let’s save this for a future post. For now, all we want is run a micro server that acts as the gateway and a service that actually has the business logic.

Now in order to run our micro gateway, we need to build the micro binary and run it. All of this is done on my local computer, without any docker containers, to make it simple.

$ git clone https://github.com/micro/micro.git

$ cd micro

$ git checkout v3.0.2

# Ensure Go is installed on your computer.
$ go version
go version go1.14.9 darwin/amd64

# Let's build the micro server binary.
$ go build -o micro .

# Note the location of this binary. We'll need it later.
$ cp ./micro /usr/local/bin/micro

Now that the micro server binary is ready, let’s start the micro gateway server.

$ ./micro server
2020-11-30 22:54:20  file=user/user.go:45 level=info Loading config key from /Users/greycell/.micro/config_secret_key
2020-11-30 22:54:20  file=user/user.go:80 level=info Loading keys /Users/greycell/.micro/id_rsa and /Users/greycell/.micro/id_rsa.pub
2020-11-30 22:54:20  file=server/server.go:86 level=info Starting server
2020-11-30 22:54:20  file=server/server.go:111 level=info Registering network
2020-11-30 22:54:20  file=server/server.go:111 level=info Registering runtime
2020-11-30 22:54:20  file=server/server.go:111 level=info Registering registry
2020-11-30 22:54:20  file=server/server.go:111 level=info Registering config
2020-11-30 22:54:20  file=server/server.go:111 level=info Registering store
2020-11-30 22:54:20  file=server/server.go:111 level=info Registering broker
2020-11-30 22:54:20  file=server/server.go:111 level=info Registering events
2020-11-30 22:54:20  file=server/server.go:111 level=info Registering auth
2020-11-30 22:54:20  file=server/server.go:111 level=info Registering proxy
2020-11-30 22:54:20  file=server/server.go:111 level=info Registering api
2020-11-30 22:54:20  file=server/server.go:198 level=info Starting server runtime
2020-11-30 22:54:20  file=user/user.go:80 level=info Loading keys /Users/greycell/.micro/id_rsa and /Users/greycell/.micro/id_rsa.pub
2020-11-30 22:54:20  file=service/service.go:190 level=info Starting [service] server
2020-11-30 22:54:20  file=grpc/grpc.go:923 level=info Server [grpc] Listening on [::]:10001
2020-11-30 22:54:20  file=grpc/grpc.go:753 level=info Registry [mdns] Registering node: server-f0fd58e3-a6cc-4b3c-9fe3-4ae4506ce358

Now that our micro gateway has started successfully, let’s create a new micro service, may be, a comment service.

Comment microservice

The micro gateway communicates with the comment service using grpc. It will use protocol buffers as a medium to communicate with the service. So, let’s first define our protocol buffers for comment service.

syntax = "proto3";

package comment;

service Comment {
    rpc New(NewRequest) returns (NewResponse) {}
    rpc List(ListRequest) returns (ListResponse) {}
}

message Item {
    string id = 1;
    string post = 2;
    string author = 3;
    string message = 4;
    int64 created = 5;
}

message NewRequest {
    string post = 1;
    string author = 2;
    string message = 3;
}

message NewResponse {
    Item comment = 1;
}

message ListRequest {
    string post = 1;
}

message ListResponse {
    repeated Item comments = 1;
}

Our comment service supports only two endpoints, one for creating a new comment and another to list comments in a post.

We also create a protocol buffer message called Item which basically defines the structure of a single comment. The rpc handler New is for creating a new comment. It accepts NewRequest as the input. The NewRequest message contains the information needed to create a new comment. It returns NewResponse which basically contains the comment data defined in Item.

The rpc handler List is for listing all the comments in a post. Naturally it’ll require the post id which is provided by the message ListRequest. The response is an array of comment Item which is provided by ListResponse message. Let’s save this file under a directory called proto. Our directory structure for the comment service will look like this.

.
├── Makefile
├── README.md
├── go.mod
├── go.sum
├── handler
│   └── comment.go
├── main.go
└── proto
    ├── comment.pb.go
    ├── comment.pb.micro.go
    └── comment.proto

In order to generate the protocol buffers, we’ll need a couple of packages. Let’s compile the list of all the commands we need into a Makefile.

GOPATH:=$(shell go env GOPATH)
MODIFY=Mgithub.com/micro/micro/proto/api/api.proto=github.com/micro/micro/v3/proto/api

.PHONY: init
init:
	go get -u github.com/golang/protobuf/proto
	go get -u github.com/golang/protobuf/protoc-gen-go
	go get github.com/micro/micro/v3/cmd/protoc-gen-micro

.PHONY: proto
proto:    
	protoc --proto_path=. --micro_out=${MODIFY}:. --go_out=${MODIFY}:. proto/comment.proto

.PHONY: build
build: proto
	go build -o service *.go

.PHONY: test
test:
	go test -v ./... -cover

Let’s run the init target to install the packages we need and then run the proto target to generate the micro service interfaces.

$ make init proto

The proto target will create two files under proto directory called comment.pb.micro.go and comment.pb.go. These two files will contain the Go structs for the protocol buffer messages we defined above and also the service handlers required to create and list comments.

Now, let’s define the business logic for our service handlers, New and List, in the file comment.go under handler directory.

// file: ./handler/comment.go

package handler

import (
	"context"
	"time"

	pb "github.com/abvarun226/comment/proto"
	"github.com/google/uuid"
)

// Comment struct.
type Comment struct{}

// NewComment creates a new comment handler.
func NewComment() *Comment {
	return &Comment{}
}

// New handler to create a new comment.
func (c *Comment) New(ctx context.Context, req *pb.NewRequest, rsp *pb.NewResponse) error {
	rsp.Comment = &pb.Item{
		Id:      uuid.New().String(),
		Post:    req.Post,
		Author:  req.Author,
		Message: req.Message,
		Created: time.Now().UTC().Unix(),
	}
	return nil
}

// List handler to list comments in a post.
func (c *Comment) List(ctx context.Context, req *pb.ListRequest, rsp *pb.ListResponse) error {
	rsp.Comments = []*pb.Item{
		{
			Id:      uuid.New().String(),
			Post:    req.Post,
			Author:  "greycell",
			Message: "this is a comment",
			Created: 1604650806,
		},
	}
	return nil
}

Now that we have our handlers to handle the business logic, let’s create the service and register it with the micro gateway server so that the gateway knows how to route requests to comment service.

// file: ./main.go

package main

import (
    "github.com/abvarun226/comment/handler"
    "github.com/micro/micro/v3/service"
    "github.com/micro/micro/v3/service/logger"
)

const (
    // ServiceName is the name of this service.
    ServiceName = "comment"

    // ServerAddress is the registry address.
    ServerAddress = "127.0.0.1:8445"
)

func main() {
    // Create comment service.
    srv := service.New(
        service.Name(ServiceName),
        service.Address(ServerAddress),
    )

    // Register handlers.
    srv.Handle(handler.NewComment())

    // Run service.
    if err := srv.Run(); err != nil {
        logger.Fatal(err)
    }
}

And this is it, the comment service is ready. Next, we’ll need to start the comment service, but before that, using the same micro binary, log into the micro server. Then, start the comment service.

# username is admin, password is micro
$ /usr/local/bin/micro login
Enter username: admin
Enter password: micro

# run the comment service
$ /usr/local/bin/micro run .

# check if the service is running.
$ /usr/local/bin/micro status
NAME    VERSION SOURCE STATUS  BUILD UPDATED METADATA
comment latest  ...    running n/a   4s ago  owner=admin...

If the status shows as error, then check the service logs.

$ /usr/local/bin/micro logs comment

Now, let’s see if the service responds to user HTTP requests. Note that micro gateway is running on localhost port 8080.

# List all comments in post `post1`
$ http --json --body 'http://localhost:8080/comment/list' post='post1'
{
    "comments": [
        {
            "author": "greycell",
            "created": "1604650806",
            "id": "830e9eef-251e-472a-b324-41545529ad76",
            "message": "this is a comment",
            "post": "post1"
        }
    ]
}


# Create a new comment.
$ http --json --body POST 'http://localhost:8080/comment/new' post='post1' author='greycell' message='test comment'
{
    "comment": {
        "author": "greycell",
        "created": "1606763037",
        "id": "6c68e596-0e38-4ff2-80ad-48e6f45758a1",
        "message": "test comment",
        "post": "post1"
    }
}

Here, the path in the API starts with /comment/ to indicate to the micro gateway that the user is requesting a response from the comment service. The gateway automatically unmarshalled the json body in the request appropriately into the protobuf request message. The gateway also mapped the request to appropriate business handler in the comment service based on the endpoints /comment/new or /comment/list.

Final Thoughts

In this post, we briefly discussed about Micro and how we can easily setup micro gateway and a simple micro service. In the next post, we’ll look at how to setup custom plugins and profiles in micro gateway in order to provide custom features like authentication and rate limits for our HTTP endpoints.

If you’d like to learn more about Micro, checkout https://micro.mu

You can access the full code for this chapter here: https://github.com/abvarun226/microservice/tree/chapter01

I hope this was helpful and if you find any errors or issues in this post, please do let me know in the comments section below.

In case you would like to get notified about more articles like this, please subscribe to my substack.