Understanding Concurrency in Go
Time to read: 8 minutes
October 17, 2023
Written by
Reviewed by
As computers further embed themselves into our way of living, increasing their performance is crucial — regardless of whether you're building a safety critical system or a simple tool to make life easier for your customers.
In terms of hardware, modern computers have multi-core processors, making it possible to execute more instructions simultaneously, but this is of little benefit if your software executes instructions synchronously.
Writing concurrent applications has been a big challenge — but Go changes all that. With built-in support for concurrent programming, Go makes it easy to write programs that efficiently utilise multiple CPU cores and handle concurrent tasks.
It does this using goroutines, lightweight threads managed by the Go runtime. Goroutines enable the execution of functions concurrently, allowing different parts of your program to run independently.
In this article, I will show you how to take advantage of concurrency to reduce the execution time of your applications. To do this, I will show you how to build a Go application which generates a usage report for your Twilio account.
Prerequisites
To follow along, you will need the following:
- A basic understanding of and experience with Go
- Go (1.19 or 1.20) — but not 1.21 (the Excel writer package seems to have an issue with it)
- A Twilio account. If you don't have one, you can sign up for a free trial account.
What you will build
You will be building an application that generates a usage report for your Twilio account. This report will be a spreadsheet with four sheets containing the following information:
- List of accounts
- All-time usage records
- Message records
- Call records
Let's get started
Create a new folder for the application, where you store your Go projects, navigate into it, and add module support using the following commands.
Next, add the project's dependencies. For this application, you will require the following:
- Excelize: This will help with generating the Excel spreadsheet
- GoDotEnv: This will help with managing environment variables
- Twilio's Go Helper Library: This simplifies interacting with the Twilio API
Add them using the following command:
Set the required environment variables
Now, create a new file called .env in the project's top-level folder, and paste the following code into it.
After that, create a local version of the .env file using the following command.
After that, retrieve your Twilio Auth Token, and Account SID from the Twilio Console Dashboard and insert them in place of the respective placeholders in .env.local.
Build the application
Helper module
Create a new folder named pkg in the application's top-level folder. This folder will contain code to help with retrieving Twilio records, as well as writing the results to a spreadsheet.
Retrieve records from Twilio
In the pkg folder, create a new file named twilio.go and add the following code to it.
The GetAccountDetails(), GetUsageRecords(), GetMessageRecords(), and GetCallRecords() methods each retrieve the corresponding records from Twilio using the Twilio helper client.
For each function, the corresponding parameters are set and the appropriate client function is called. In addition to returning the requested records, the client function can also return an error — hence the checkError() function call, before returning the requested records.
Write results to a spreadsheet
The next thing you’ll implement is the functionality to write the retrieved records to a spreadsheet. In the pkg folder, create a new file named writer.go and add the following code to it.
The four basic actions for writing the report are as follows:
- Setting the size of a cell: To make sure that the cell has enough space to contain the value written to it (without the user having to manually increase the size of the cell), the
setCellSize()function is used to adjust the cell width - Writing to a cell: This is handled by the
writeToCell()function - Applying a thin border: To make the spreadsheet easier to read, a thin border is applied to each row. This is handled by the
writeThinBorder()function - Applying a thick border: The header row for each spreadsheet requires a thick border. This is handled by the
writeThickBorder()function
These four functions are used by the writeAccountDetails(), writeUsageRecords(), writeMessageRecords(), and writeCallRecords() functions to write the corresponding records on a separate sheet. Each function follows the following pattern:
- Create a new sheet and give it a name
- Write the headers for the sheet
- Write a thick border for the header row
- Iterate through the records, write the appropriate cell value for each record, and apply a thin border for the row
- Adjust the cell sizes to make sure that the content is displayed properly
The last function is WriteResults() which is exported for use in other parts of the application. This function takes all the records (account details, usage records, message records, and call records), creates a new file, and passes each record to the appropriate function to be written accordingly. Once this is done, the file is saved and, in the absence of an error, the file is closed.
Synchronous report generation
In the project's top-level directory, create a new file, named main.go. In that file, paste the following code.
The main() function is the application's entry point. It is executed when the go run main.go command is executed. It starts by calling the setup() function. This retrieves the Twilio credentials from the previously created .env.local file, and instantiates the Twilio client.
Next, it marks the current time and then calls the generateReportSynchronously() function. This function calls the GetAccountDetails(), GetUsageRecords(), GetMessageRecords(), and GetCallRecords() functions you declared earlier, and passes the responses to the WriteResults() function. Finally, the execution time of the generateReportSynchronously() function is printed.
You can run the application to see how long it takes, using the following command.
You should see the result printed out as shown below
You'll also see a new file in the project's top-level directory, named Usage Report.xlsx.
Concurrent report generation
In the generateReportSynchronously() function, the Twilio API calls were made in the main thread. This meant that the application had to wait for each API call to be completed before making another one.
But there’s no reason why the API calls have to be made one at a time. So in the concurrent version, separate goroutines are created (one for each API call), and then the results passed on for writing to the spreadsheet.
In the main.go file, add the following function.
Make sure your imports are updated to match the following, as well.
To start with, this function declares a new variable named results. This variable is a struct with four fields - one for each expected result.
Next, a WaitGroup is created. The WaitGroup is used to make sure that the main thread does not exit before the goroutines in the WaitGroup have finished running. Since four API calls are going to be made, the value 4 is passed as an argument to the wg.Add() function. This sets the counter of the WaitGroup to 4.
Next come the goroutines. A goroutine is simply a function call with the go keyword before it. For each goroutine, a reference to the WaitGroup is passed. In the goroutine, an API call is made using the helper functions declared earlier. The result is saved in the appropriate field of the results struct.
After that, the wg.Done() function is called. This decreases the WaitGroup counter by 1. When the value of the counter is 0, the wg.Wait() function returns, allowing the main thread to continue. At that point, all the records needed to generate the report are available, and the writeResults() function is called.
To compare the synchronous and concurrent execution, update the main function to match the following.
You can run the application to see how long both processes take, using the following command.
You should see the result printed out as shown below
While the results may vary due to your internet connection speed and processing power etc., one thing that stands out is that there is a significant difference in time between the synchronous and concurrent executions.
Benchmarking the results
So far, you’ve compared the results by measuring the time taken for a single execution. Depending on several factors, the measured time may vary wildly. A more reliable way of measuring the performance of both functions is by running them several times to measure the average performance. This is known as benchmarking. The testing package provided by Go contains built-in tools for writing benchmark tests.
Before running the tests, you need to call the setup() function which is used to set up the Twilio client. To do this, create a new file named setup_test.go in the application's top-level folder and add the following code to it.
The TestMain() function is called before all the other written tests. You can use this function to prepare your test environment — in this case instantiate the Twilio client. Then, you can run the tests using the m.Run() function. This function returns an exit code which is passed to os.Exit() in order to complete the testing process.
Next, create a new file named main_test.go in the application's top-level folder and add the following code to it.
Files containing tests must end with _test.go and each benchmark function should have a signature of func BenchmarkXxx(*testing.B) as a signature. In each test, the function is run b.N times.
Run the tests using the following command.
In addition to showing the OS, architecture, and CPU details, you should see something similar to the following.
The -4 suffix for each function execution denotes the number of CPUs used to run the benchmark, as specified by GOMAXPROCS. After the function name, the number of times the loop was executed is shown (in this case once). Finally, the average execution time for each operation (in nanoseconds) is displayed.
At the moment, each test is run once, and for each test only one iteration is executed. If you want, you can run each test multiple times using the count argument. You can also modify the number of iterations for each test using the benchtime argument.
For example, you can run the benchmark tests five times with each test running four iterations using the following command.
Now you have a basic understanding of concurrency in Go
Well done for coming this far! By taking advantage of Go’s provisions for concurrency, you have cut your applications running time by almost 50%. This will help you build efficient applications that take more advantage of the available CPU resources, and reduce the waiting time for users, which is always an improvement for any application.
However, special consideration has to be given to avoid some pitfalls. One common pitfall is deadlocks. In this case, your goroutines were writing to different fields of the struct hence there was no risk of a deadlock. However, if you encounter a situation where goroutines are reading and writing to the same field, you will need to do some more work.
Go provides channels for safe communication between goroutines. Also, the sync package provides more than the WaitGroup type you saw earlier. Based on the concept of mutual exclusion, the Mutex and RWMutex types make it possible for multiple goroutines to concurrently read and write to a shared resource.
The entire codebase is available on GitHub should you get stuck at any point. I’m excited to see what else you come up with. Until next time, make peace not war ✌🏾
Joseph Udonsak is a software engineer with a passion for solving challenges – be it building applications, or conquering new frontiers on Candy Crush. When he’s not staring at his screens, he enjoys a cold beer and laughs with his family and friends. Find him at LinkedIn, Medium, and Dev.to.
"dcii_6" by Windell Oskay is licensed under the CC BY 2.0 Deed.
Related Resources
Twilio Docs
From APIs to SDKs to sample apps
API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
Resource Center
The latest ebooks, industry reports, and webinars
Learn from customer engagement experts to improve your own communication.
Ahoy
Twilio's developer community hub
Best practices, code samples, and inspiration to build communications and digital engagement experiences.