Golang : Remove or trim extra comma from CSV
Jotting down this note for future reference. In case it might be useful to you. A friend has a CSV file with extra commas in some lines at the trail(suffix) and he wanted to clean up the CSV before parsing the data. At first, we attempted to parse the "bad" CSV data by configuring the encoding/csv/#Reader options, but unsuccessful.
The "bad" CSV data looks like this :
1,a,,
2,,b,
# comment here
3,,,c
and the extra comma is causing his program to parse wrongly.
Not sure how he ended up with the bad CSV file... anyway, the main point of this post is to show you how to clean up CSV data in Golang. The steps taken are similar to this StackOverflow post(for PHP), which uses PHP's rtrim()
function. What we did was to read the CSV file line by line and then clean up the extra commas with strings.TrimSuffix()
and strings.Replace()
functions.
package main
import (
"bufio"
"fmt"
"io"
"os"
"strings"
)
func main() {
badCSVfile, err := os.Open("bad.csv")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer badCSVfile.Close()
reader := bufio.NewReader(badCSVfile)
scanner := bufio.NewScanner(reader)
goodCSVfile, err := os.Create("good.csv")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Println("Writing to good.csv")
defer goodCSVfile.Close()
for scanner.Scan() {
// clean extra comma
cleaned := strings.TrimSuffix(scanner.Text(), ",")
cleaned = strings.TrimSuffix(cleaned, ",")
// replace double or triple comma
cleaned = strings.Replace(cleaned, ",,,", ",", -1)
cleaned = strings.Replace(cleaned, ",,", ",", -1)
fmt.Println(cleaned)
_, err := io.WriteString(goodCSVfile, cleaned + "\n")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
}
}
Output of the good.csv
:
1,a
2,b
# comment here
3,c
References :
https://www.socketloop.com/tutorials/golang-read-a-file-line-by-line
https://golang.org/pkg/strings/#TrimSuffix
https://www.socketloop.com/tutorials/golang-write-file-io-writestring
See also : Golang : How to write CSV data to file
By Adam Ng
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+7.1k Golang : Emulate NumPy way of creating matrix example
+16.4k Golang : Read data from config file and assign to variables
+3.7k JavaScript: Add marker function on Google Map
+6k Golang : Fibonacci number generator examples
+4.8k PHP : Get client IP address
+4.8k Golang : Fix opencv.LoadHaarClassifierCascade The node does not represent a user object error
+50.5k Golang : How to get time in milliseconds?
+28.9k Golang : Calculate percentage change of two values
+13.6k Golang : Search folders for file recursively with wildcard support
+19.8k Golang : Get password from console input without echo or masked
+8.2k Random number generation with crypto/rand in Go
+7.7k Golang : Gonum standard normal random numbers example