Receive Messages from AWS SQS using Go

development golang aws sqs

In order to work with AWS SQS using Go, we will use github.com/aws/aws-sdk-go-v2. In this gist, we will use the aws-sdk-go-v2 package to receive messages from a queue in AWS SQS. The complete source code for the snippet is available here https://github.com/abvarun226/blog-source-code/tree/master/receive-messages-from-aws-sqs-using-go

In order to test the snippet, we will use localstack to run SQS emulator in a docker container on the laptop. The container exposes SQS endpoint on port 4566. The snippet will then receive a message from queue first-queue in SQS.

Send Messages to AWS SQS using Go

development golang aws sqs

In order to work with AWS SQS using Go, we will use github.com/aws/aws-sdk-go-v2. In this gist, we will use the aws-sdk-go-v2 package to send messages to a queue in AWS SQS. The complete source code for the snippet is available here https://github.com/abvarun226/blog-source-code/tree/master/send-messages-to-aws-sqs-using-go

In order to test the snippet, we will use localstack to run SQS emulator in a docker container on the laptop. The container exposes SQS endpoint on port 4566. The snippet will then send a message to queue first-queue in SQS.

Delete Queues in AWS SQS using Go

development golang aws sqs

In order to work with AWS SQS using Go, we will use github.com/aws/aws-sdk-go-v2. In this gist, we will use the aws-sdk-go-v2 package to delete queues in AWS SQS. The complete source code for the snippet is available here https://github.com/abvarun226/blog-source-code/tree/master/delete-queues-in-aws-sqs-using-go

In order to test the snippet, we will use localstack to run SQS emulator in a docker container on the laptop. The container exposes SQS endpoint on port 4566. The snippet will then delete a queue first-queue in SQS.

List Queues in AWS SQS using Go

development golang aws sqs

In order to work with AWS SQS using Go, we will use github.com/aws/aws-sdk-go-v2. In this gist, we will use the aws-sdk-go-v2 package to list queues in AWS SQS. The complete source code for the snippet is available here https://github.com/abvarun226/blog-source-code/tree/master/list-queues-in-aws-sqs-using-go

In order to test the snippet, we will use localstack to run SQS emulator in a docker container on the laptop. The container exposes SQS endpoint on port 4566. The snippet will then list all queues in the AWS SQS account.

Create Queues in AWS SQS using Go

development golang aws sqs

In order to work with AWS SQS using Go, we will use github.com/aws/aws-sdk-go-v2. In this gist, we will use the aws-sdk-go-v2 package to create queues in AWS SQS. The complete source code for the snippet is available here https://github.com/abvarun226/blog-source-code/tree/master/create-queues-in-aws-sqs-using-go

In order to test the snippet, we will use localstack to run SQS emulator in a docker container on the laptop. The container exposes SQS endpoint on port 4566. The snippet will then create a new queue first-queue in SQS.

Implement Go version manager

development golang

Introduction In this post, we will learn more about Go programming language by implementing a Go version manager CLI tool. The goal of this CLI tool is to manage different versions of Go installed on our development machine. As part of implementing the tool, we will learn about command-line flags, bash completion, HTTP client to download archives and interaction with files or directories. Let’s dive into the requirements and the features we will support with the tool and the implementation of the tool using Go.

Search operation in Go

development golang search generics

Searching for an item in a collection or container is a common operation in programming. In Go, there are a few different ways that this can be done. In this snippet, let us explore the ways in which an item, be it a number or string, can be searched in a slice.

Strings Concatention In Go

development golang

In today’s snippet, let’s look at how to use string builder and bytes buffer to concatenate a list of strings. You can always use the + operator to concatenate strings, but it is memory inefficient just like in so many other languages. Each time + is used to concatenate a string, it allocates a new string in memory. This is because, like in most languages, strings in Go is immutable. To avoid this inefficiency, we can use Go’s strings.Builder and bytes.Buffer type. Let’s look at how we can use these types to concatenate strings.

Ordered Printing With Goroutines In Go

development golang

In today’s snippet, let’s look at how we can use synchronization techniques in Go to print natural numbers in order from two different goroutines. In our example here, we have two goroutines; one goroutine will print even numbers while the other goroutine will print odd numbers, but the output has to be in increasing order. To implement this, we will use a channel to synchronize the printing in the two goroutines. We will also use sync.WaitGroup to wait for the goroutines to finish before exiting the code. Note that we will need an unbuffered channel so that one goroutine is blocked while the other is printing the number. The even goroutine will use the done variable to indicate to the odd goroutine that it is done with the printing and there is nothing listening to the syncChannel. Without this, you will see a deadlock at the end of the printing.

Context Cancellation in Go

development golang context

In Go, we can use context to send cancellation signals to goroutines that is doing some work. The Done method in context returns a channel that acts as a cancellation signal to goroutines using the given context. When the channel is closed, it indicates to the goroutines that they should stop the work they are doing. We can use WithCancel context function to obtain a new context along with the cancel function that can be used to close the channel.

Delete Files From AWS S3 using Go

development golang aws

In order to work with AWS S3 using Go, we will use github.com/aws/aws-sdk-go/aws. In this gist, we will use the aws-sdk-go package to delete all files under a given directory in a given bucket. The complete source code for the snippet is available here https://github.com/abvarun226/blog-source-code/tree/master/delete-files-in-aws-s3-using-go

In order to test the snippet, we will use localstack to run s3 emulator in a docker container on the laptop. The container exposes s3 endpoint on port 4566. The snippet will then create a new bucket work-with-s3 in s3. The snippet will then delete all files from blog directory under work-with-s3 bucket.

Listing Files in AWS S3 using Go

development golang aws

In order to work with AWS S3 using Go, we will use github.com/aws/aws-sdk-go/aws. In this gist, we will use the aws-sdk-go package to list files under a given directory in a given bucket. The complete source code for the snippet is available here https://github.com/abvarun226/blog-source-code/tree/master/listing-files-in-aws-s3-using-go

In order to test the snippet, we will use localstack to run s3 emulator in a docker container on the laptop. The container exposes s3 endpoint on port 4566. The snippet will then create a new bucket work-with-s3 in s3. The snippet will then list all files in blog directory under work-with-s3 bucket.

Uploading Files to AWS S3 using Go

development golang aws

In order to work with AWS S3 using Go, we will use github.com/aws/aws-sdk-go/aws. In this gist, we will use the aws-sdk-go package to upload files under a given directory. The complete source code for the snippet is available here https://github.com/abvarun226/blog-source-code/tree/master/uploading-files-to-aws-s3-using-go

In order to test the snippet, we will use localstack to run s3 emulator in a docker container on the laptop. The container exposes s3 endpoint on port 4566. The snippet will then create a new bucket work-with-s3 in s3. The snippet will then upload couple of files to blog directory under work-with-s3 bucket.

Implementing mTLS in Go

development golang microservices

Introduction Mutual authentication with Transport Layer Security (mTLS) is a method for mutual authentication. It is often used in securing network communication between two services and ensures that the parties at each end of this communication are who they claim to be by verifying that they both have the correct private key. It has been a standard security and authentication mechanism in a service mesh like Istio, Linkerd, AWS App Mesh and others.

Background Tasks in Go

development golang workers

When you design a software system or REST API, there are times when you would want to process large amounts of data or do time consuming computations that cannot be completed within a decent amount of time. You cannot expect your users to make an HTTP call that takes several seconds or minutes to respond while the backend server handles the request by doing some time consuming task. Often the load balancer or the HTTP server will enforce a timeout for a request after which the connection is terminated and results in a gateway timeout error.

Ways to Rate Limit Requests in Go

development golang

Introduction Layer 7 DDoS attacks are quite common these days. With a simple script, any one can send thousands of requests to services to exhaust server resources. They can easily take down services and cause disruptions to a business. It is wise to implement strategies to mitigate such attacks using techniques that limit the number of requests a person can send to the service. It is possible to rate limit the user requests by IP address or a user ID, if available, although the former is more popular.

Vanity URL for Go packages

development golang

Introduction If you have been working with Go programming language for a while, you would have noticed that a lot of open source packages that you import start with github.com/…. You would then use go get command to download the package and add it to your go.mod file. For instance: $ go get -u github.com/abvarun226/goiplookup What if you did not want this dependency on Github and rather wanted to host your own git server?

Microservices and cloud native development using Micro

development golang microservices

Introduction For the past few years, Microservices has been all the rage and it has helped a lot of organizations to adopt continuous delivery and evolve its technology stack. There are quite a few programming tool kits and platforms that have sprung up to enable building Microservices. One among them is Micro, which is a cloud native development platform that addresses the key requirements for building services in the cloud. In their latest version v3, Micro provides service discovery, pub/sub, transport protocols and many other features out of the box.

Using Ristretto to cache objects in Go

development golang cache ristretto

Introduction In my previous article on Greycell, I wrote about caching objects using tags and how it can make cache invalidation easier. I used Redis as a temporary store for our objects that needed to be cached. Although Redis is really good for this, I wanted an embedded key-value store in order to avoid additional dependency of maintaining a Redis server. When I explored further, I came across many embedded key-value store like groupcache, fastcache, burrow cache and Dgraph’s Ristretto.

Validate email address in Go

development golang email

Firstly, based on the RFC 3696 Errata 1690, a valid email address has a maximum of 254 characters. Here is the relevant snippet from the RFC: A valid email address has a maximum of 64 characters in the “local part” (before the “@") and a maximum of 255 characters in the domain part (after the “@") for a total length of 320 characters. However, there is a restriction in RFC 2821 on the length of an address in MAIL and RCPT commands of 254 characters.

Caching records by tags in Go

development golang cache

Cache Invalidation When frequently accessed data which is stored in a cache is updated, it is necessary for the application to evict the stale data from the cache. So when subsequent requests for the same data is received by the application, it will retrieve fresh data from the database and store it in the cache. This way, users will never receive stale data from the application. The process of cache invalidation can be made easier by tagging data with specific tags when storing in the cache.

Validate Tor IP Address in Go

development golang tor

Tor Network Tor is a free and open-source software that enables anonymous communication. It directs Internet traffic through a free, worldwide, volunteer overlay network to conceal a user’s location and usage from anyone conducting network surveillance or traffic analysis. It has helped protect the personal privacy of its users and enabled the ability to conduct confidential communication by keeping their Internet activities unmonitored. Tor protect’s a user’s privacy, but does not hide the fact that someone is using Tor.

Multipart Requests in Go

development golang

MIME and Multipart Multipurpose Internet Mail Extensions (MIME) is an internet standard that extends the format of emails to support non-ASCII character sets, messages with multipart parts and non-text attachments like images, video, binary files etc. Multipart originates from MIME and a multipart message is a list of parts. Each part contains headers and a body. The MIME standard defines various multipart messages subtypes like mixed, digest, related, form-data, byteranges, encrypted and a few others.

About

Bharghava Varun Ayada is a Staff Software Engineer at Walmart Labs, living in Bangalore, India. This blog is about software engineering, system design, distributed systems and DevOps.


Github / Twitter / LinkedIn / Dev.to / Stackoverflow


Recommended Books
Designing Data-Intensive Applications / Staff Engineer / Building Microservices / Site Reliability Engineering

Get new posts by email