How to Read and Write CSV Files Using Go

Time to read: 5 minutes

August 06, 2024
Written by
Temitope Taiwo Oyedele
Contributor
Opinions expressed by Twilio contributors are their own
Reviewed by
Matthew Setter
Twilion

How to read and write data in CSV file using golang

CSV files are a common format for storing and exchanging data. They are simple text files in which each line represents a data record, and the values of each record are separated by commas or other delimiters. This format is widely used for importing and exporting data between different applications.

In this article, we'll explore how to read and write CSV files in Go. We'll cover the basics of handling CSV data, including reading from and writing to files, working with headers, handling errors, and formatting data. By the end, you should have a solid understanding of how to incorporate CSV file processing into your Go applications.

Prerequisites

Before diving into the details, ensure you have the following:

  • Go installed on your system. You can download it from the official website
  • A text editor or IDE of your choice to write and edit your Go code
  • A CSV file to test

Read CSV files in Go

Go provides the encoding/csv package in its standard library to handle CSV operations.

Open your terminal and create a directory for your project. 

mkdir go_csv
cd go_csv

Open up the folder in your code editor. Add a CSV file named data.csv inside the project's top-level folder. You can use this CSV file for demonstration purposes. Then, create a file called main.go.

Now, let’s break down the steps on how to read a CSV file in Go

Import the necessary packages

Before you start working with CSV files, you need to import the necessary packages namely:

In your main.go add the following:

package main

import (
    "bytes"
    "encoding/csv"
    "fmt"
    "io"
    "os"
)

The bytes package provides functions for working with byte slices. In our case, it's used specifically with bytes.NewReader(data) to create an in-memory representation of the CSV data for the csv package to read from.

Open the CSV file

To read data from a CSV file, you first need to open the file. You can use the os.Open() function to open the file and io.ReadAll() to read its raw contents into memory: 

func readCSVFile(filename string) ([]byte, error) {
    f, err := os.Open(filename)
    if err!= nil {
        return nil, err
    }
    defer f.Close()
    data, err := io.ReadAll(f)
    if err!= nil {
        return nil, err
    }
    return data, nil
}

The code above defines a readCSVFile() function that takes the CSV file's filename as an argument and returns the content of the file as a byte slice.

Create a CSV reader

Once you have the content of the CSV file, you can create a CSV reader to parse the data. The csv.NewReader() function from the encoding/csv package creates a new CSV reader that reads from the given byte slice: 

func parseCSV(data []byte) (*csv.Reader, error) {
    reader := csv.NewReader(bytes.NewReader(data))
    return reader, nil
}

Here, we define a parseCSV() function that takes the byte slice containing the CSV data which returns a csv.Reader instance.

Iterate through the records

Now that you have a CSV reader, you can iterate through the records in the CSV file: 

func processCSV(reader *csv.Reader) {
    for {
        record, err := reader.Read()
        if err == io.EOF {
            break
        } else if err!= nil {
            fmt.Println("Error reading CSV data:", err)
            break
        }
        fmt.Println(record)
    }
}

The processCSV() function uses a loop to repeatedly call the reader.Read() method on the CSV reader. This method returns a slice of strings representing the values in each record, along with an error value. We check for the io.EOF error to determine when we've reached the end of the file, and handle any other errors that might occur during reading.

Integrate the functions in the main function

Finally, let's demonstrate how to use these functions together in the main() function. This part shows how to integrate everything and handle potential errors: 

func main() {
    data, err := readCSVFile("data.csv")
    if err!= nil {
        fmt.Println("Error reading file:", err)
        return
    }
    reader, err := parseCSV(data)
    if err!= nil {
        fmt.Println("Error creating CSV reader:", err)
        return
    }
    processCSV(reader)
}

Put it all together

Now, let's put all the pieces together into a complete program, by updating main.go to match the code below: 

package main

import (
   "bytes"
   "encoding/csv"
   "fmt"
   "io"
   "os"
)

func readCSVFile(filename string) ([]byte, error) {
    f, err := os.Open(filename)
    if err != nil {
        return nil, err
    }
    defer f.Close()
    data, err := io.ReadAll(f)
    if err != nil {
        return nil, err
    }
    return data, nil
}

func parseCSV(data []byte) (*csv.Reader, error) {
    reader := csv.NewReader(bytes.NewReader(data))
    return reader, nil
}

func processCSV(reader *csv.Reader) {
    for {
        record, err := reader.Read()
        if err == io.EOF {
            break
        } else if err != nil {
            fmt.Println("Error reading CSV data:", err)
            break
        }
         fmt.Println(record)
    }
}

func main() {
    data, err := readCSVFile("data.csv")
    if err != nil {
        fmt.Println("Error reading file:", err)
        return
    }
    reader, err := parseCSV(data)
    if err != nil {
        fmt.Println("Error creating CSV reader:", err)
        return
    }
    processCSV(reader)
}

Run code using this command: 

go run main.go

You should see output matching the following printed to your terminal. 

[name email phone]
[John Doe john@example.com 123-456-7890]
[Jane Smith jane@example.com 987-654-3210]
[Bob Johnson bob@example.com 555-123-4567]
[Alice Brown alice@example.com 555-987-6543]
[John Doe john@example.com 123-456-7890]
[Jane Smith jane@example.com 987-654-3210]
[Bob Johnson bob@example.com 555-123-4567]
[Alice Brown alice@example.com 555-987-6543]
[Michael Jones michael@example.com 555-555-5555]
[Emily Johnson emily@example.com 555-555-5556]
[William Brown william@example.com 555-555-5557]
[Jessica White jessica@example.com 555-555-5558]
[David Black david@example.com 555-555-5559]
[Robert Green robert@example.com 555-555-6660]
[John Doe john@example.com 123-456-7890]
[Jane Smith jane@example.com 987-654-3210]
[Bob Johnson bob@example.com 555-123-4567]
[Alice Brown alice@example.com 555-987-6543]
[Michael Jones michael@example.com 555-555-5555]
[Emily Johnson emily@example.com 555-555-5556]
[William Brown william@example.com 555-555-5557]
[Jessica White jessica@example.com 555-555-5558]
[David Black david@example.com 555-555-5559]
[Robert Green robert@example.com 555-555-6660]

Use the header for column mapping

If you want to use the header row to map column names to values, you can read the header separately and create a mapping. To utilize the header row for column mapping in your Go application, you can modify the processCSV() function to also read the header, and then use it to access specific columns by name.

To do this, first, update the processCSV() function to accept an additional parameter for the header: 

func processCSV(reader *csv.Reader, header []string) {
    fmt.Println(header)
    for {
        record, err := reader.Read()
        if err == io.EOF {
            break
        } else if err!= nil {
            fmt.Println("Error reading CSV data:", err)
            break
        }
        for i, field := range header {
            fmt.Printf("%s: %s\n", field, record[i])
        }
    }
}

Next, modify the main() function to pass the header to the processCSV() function. After creating the CSV reader, you can get the header by calling the reader.Read() method once before entering the loop: 

func main() {
    data, err := readCSVFile("data.csv")
    if err!= nil {
        fmt.Println("Error reading file:", err)
        return
    }
    reader, err := parseCSV(data)
    if err!= nil {
        fmt.Println("Error creating CSV reader:", err)
        return
    }
    header, err := reader.Read()
    if err!= nil {
        fmt.Println("Error reading header:", err)
        return
    }
    processCSV(reader, header)
}

With the changes made, run the go run command, and you’ll see that your program will now print the header row and then use it to access and print specific columns by name for each record in the CSV file. 

[name email phone]
name: John Doe
email: john@example.com
phone: 123-456-7890
name: Jane Smith
email: jane@example.com
phone: 987-654-3210
name: Bob Johnson
email: bob@example.com
phone: 555-123-4567
name: Alice Brown
email: alice@example.com
phone: 555-987-6543
name: John Doe
email: john@example.com
phone: 123-456-7890
name: Jane Smith
email: jane@example.com
phone: 987-654-3210
name: Bob Johnson
email: bob@example.com
phone: 555-123-4567
name: Alice Brown
email: alice@example.com
phone: 555-987-6543
name: Michael Jones
email: michael@example.com
phone: 555-555-5555
name: Emily Johnson
email: emily@example.com
phone: 555-555-5556
name: William Brown
email: william@example.com
phone: 555-555-5557
name: Jessica White
email: jessica@example.com
phone: 555-555-5558
name: David Black
email: david@example.com
phone: 555-555-5559
name: Robert Green
email: robert@example.com
phone: 555-555-6660
name: John Doe
email: john@example.com
phone: 123-456-7890
name: Jane Smith
email: jane@example.com
phone: 987-654-3210
name: Bob Johnson
email: bob@example.com
phone: 555-123-4567
name: Alice Brown
email: alice@example.com
phone: 555-987-6543
name: Michael Jones
email: michael@example.com
phone: 555-555-5555
name: Emily Johnson
email: emily@example.com
phone: 555-555-5556
name: William Brown
email: william@example.com
phone: 555-555-5557
name: Jessica White
email: jessica@example.com
phone: 555-555-5558
name: David Black
email: david@example.com
phone: 555-555-5559
name: Robert Green
email: robert@example.com
phone: 555-555-6660

Write CSV files in Go

Now that we've covered reading CSV files, let's examine how to write data in a CSV file using Go.

Create a CSV writer

You first need to create a CSV writer to write data to a CSV file. The csv.NewWriter() function from the encoding/csv package creates a new CSV writer that writes to the given writer: 

func createCSVWriter(filename string) (*csv.Writer, *os.File, error) {
    f, err := os.Create(filename)
    if err != nil {
        return nil, nil, err
    }
    writer := csv.NewWriter(f)
    return writer, f, nil
}

Write records to the CSV file

Once you have a CSV writer, you can use the writer.Write() method to write records to the CSV file: 

func writeCSVRecord(writer *csv.Writer, record []string) {
    err := writer.Write(record)
    if err != nil {
        fmt.Println("Error writing record to CSV:", err)
    }
}

Integrate the functions in the main function

This function demonstrates how to use the previously described functions to create a CSV writer, write a header and records to a CSV file, and finally flush the writer. 

func main() {
    filename := "output.csv"
    writer, file, err := createCSVWriter(filename)
    if err != nil {
        fmt.Println("Error creating CSV writer:", err)
        return
    }
    defer file.Close()
    header := []string{"Name", "Age", "City"}
    writeCSVRecord(writer, header)
    records := [][]string{
        {"John", "30", "New York"},
        {"Alice", "25", "Tokyo"},
        {"Bob", "35", "London"},
    }
    for _, record := range records {
        writeCSVRecord(writer, record)
    }
    // Flush the writer and check for any errors
    writer.Flush()
    if err := writer.Error(); err != nil {
        fmt.Println("Error flushing CSV writer:", err)
    }
}

Putting it all together

Now, let's put together a complete program that writes data to a CSV file, by updating main.go to match the code below: 

package main

import (
    "encoding/csv"
    "fmt"
    "os"
)

func main() {
    filename := "output.csv"
    writer, file, err := createCSVWriter(filename)
    if err != nil {
        fmt.Println("Error creating CSV writer:", err)
        return
    }
    defer file.Close()
    header := []string{"Name", "Age", "City"}
    writeCSVRecord(writer, header)
    records := [][]string{
        {"John", "30", "New York"},
        {"Alice", "25", "Tokyo"},
        {"Bob", "35", "London"},
    }
    for _, record := range records {
        writeCSVRecord(writer, record)
    }
    // Flush the writer and check for any errors
    writer.Flush()
    if err := writer.Error(); err != nil {
        fmt.Println("Error flushing CSV writer:", err)
    }
}

func createCSVWriter(filename string) (*csv.Writer, *os.File, error) {
    f, err := os.Create(filename)
    if err != nil {
        return nil, nil, err
    }
    writer := csv.NewWriter(f)
    return writer, f, nil
}

func writeCSVRecord(writer *csv.Writer, record []string) {
    err := writer.Write(record)
    if err != nil {
        fmt.Println("Error writing record to CSV:", err)
    }
}

When you run the code, it generates an output.csv file in the project's top-level directory containing the data: 

Name,Age,City
John,30,New York
Alice,25,Tokyo
Bob,35,London

Customizing delimiters and quoting behavior

When creating a CSV writer, you can specify custom delimiters and control whether fields should be quoted. This is useful for handling data that includes commas, quotes, or other characters that could interfere with the parsing of the CSV file. Let’s see how it is done. Replace the code in the main.go with the following code:

 

package main

import (
   "encoding/csv"
   "fmt"
   "os"
   "strings"
)

func main() {
   filename := "custom_delimiter_and_quoting.csv"
   f, err := os.Create(filename)
   if err != nil {
       fmt.Println("Error creating file:", err)
       return
   }

   defer f.Close()
   writer := csv.NewWriter(f)
   writer.Comma = ';' 
   trimSpaces := func(s string) string {
       return strings.TrimSpace(s)
   }
   header := []string{"Name", "Age", "City"}
   err = writer.Write(header)
   if err != nil {
       fmt.Println("Error writing header:", err)
       return
   }

   records := [][]string{
       {"John Doe", "30", "New York"},
       {"Alice Smith", "25", "Tokyo"},
       {"Bob Johnson", "35", "London"},
   }

   for _, record := range records {
       trimmedRecord := make([]string, len(record))
       for i, field := range record {
           trimmedRecord[i] = trimSpaces(field)
       }
       err = writer.Write(trimmedRecord)
       if err != nil {
           fmt.Println("Error writing record:", err)
           return
       }
   }
   writer.Flush()             
   if writer.Error() != nil {
       fmt.Println("Error flushing writer:", writer.Error())
   }
}

In the code above, we create a CSV file named custom_delimiter_and_quoting.csv, which uses a semicolon (;) as the delimiter between fields, specified by setting writer.Comma = ';'.

Before writing them to the file, the spaces are trimmed from the start and the end of each field value. Once trimmed, it writes a header row followed by several records to the CSV file, handling any errors that occur during the write process. Finally, it flushes the writer to ensure all buffered data is written to the file and checks for any errors that occurred during the flush operation.

When you run code, you should see that it creates a CSV file named custom_delimiter_and_quoting.csv in the project's top-level directory with the delimiter data written into it, which you can see below. 

Name;Age;City
John Doe;30;New York
Alice Smith;25;Tokyo
Bob Johnson;35;London

That's how to read and write data in CSV files using Go

In this article, we've covered the basics of working with CSV files in Go, including reading, writing, handling headers, managing errors, and formatting data. Go's standard library provides a powerful set of tools for processing CSV files efficiently.

Remember to handle errors gracefully and consider the specific requirements of your application when working with CSV data. Happy coding!

Temitope Taiwo Oyedele is a software engineer and technical writer. He likes to write about things he’s learned and experienced.

The icon in the tutorial's main image was created by Freepik on Flaticon.

Related Posts

Related Resources

A newspaper article

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
An Open book

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.
User group reactions

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.