Go is currently the most common programming language in cloud development. Though I use it very much in my work, I am still repeating certain mistakes. This article is more a record of these errors, figuring out the causes and solutions so that people who read this article will save themselves time when coming across the same problems.
Let’s cut through to the tips.
Don’t rely on index var in the for loop
The most common mistake we make is that we often create goroutine inside a for loop and rely on the index variable. For example,
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
items := []int{1, 2, 3, 4}
for index, _ := range items {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Printf("item:%v\\n", items[index])
}()
}
wg.Wait()
}
//output:
items:4
items:4
items:4
items:4
The code looks just right at first sight, while the output is always 4(the last value in the array). Why?
Because goroutines won’t start before the for loop finishes. And assigned with a new value(0,1,2,3) in each loop, the index is always 3 in the end.
To avoid it, you just stop relying on the index variable. Instead, you can either pass it into the goroutine or copy it to a new variable inside the loop. And the first method is my personal preference.
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
items := []int{1, 2, 3, 4, 5}
for index, _ := range items {
wg.Add(1)
go func(i int) {
defer wg.Done()
fmt.Printf("item:%v\\n", items[i])
}(index)
}
wg.Wait()
}Convert the default JSON number data into int
The code below won’t work.
package main
import (
"encoding/json"
"fmt"
)
func main() {
data := []byte(`{"number": 1}`)
var result map[string]interface{}
if err := json.Unmarshal(data, &result); err != nil {
fmt.Println("parse json error:", err)
return
}
n := result["number"].(int) //error
fmt.Println("number is:", n)
}The output error is:
panic: interface conversion: interface {} is float64, not int
goroutine 1 [running]:
main.main()
/tmp/sandbox635569672/prog.go:17 +0x252Obviously, the default type of a number from Unmarshall is not int but float64. We can make it out by continue using float64 and converting it to int later.
n := result["number"].(float64)
in := int(n)
fmt.Println("number is:", in) // output: number is : 1And I want to introduce a more popular solution, which is wrapping the field into a struct and converting it with json tag.
package main
import (
"encoding/json"
"fmt"
)
type Data struct {
Number int `json:"number"`
}
func main() {
dataStr := []byte(`{"number": 1}`)
data := Data{}
if err := json.Unmarshal(dataStr, &data); err != nil {
fmt.Println("parse json error:", err)
return
}
fmt.Println("number is:", data.Number)
}Don’t always trust DeepEqual
In Go, we can compare some types of variables without initialization using ==because they have default values.
package main
import (
"fmt"
)
func main() {
var n1 int
var n2 int
fmt.Println("n1 == n2:", n1 == n2);
var ch1 <- chan string
var ch2 <- chan string
fmt.Println("ch1 == ch2:", ch1 == ch2);
var i1 interface{}
var i2 interface{}
fmt.Println("i1 == i2:", i1 == i2);
//another supported types: bool, string, float, pointer, array
}However, there are some we cannot do so even after assigning them with new values. For instance,
package main
import (
"fmt"
)
func main() {
var m1 = map[string]int{"a":1, "b": 2}
var m2 = map[string]int{"b":2, "a":1}
fmt.Println("m1 == m2:", m1 == m2);
var b1 = []string{"1"}
var b2 = []string{"1"}
fmt.Println("b1 == b2:", b1 == b2);
var f1 = func() int {return 1}
var f2 = func() int {return 2}
fmt.Println("f1 == f2:", f1 == f2);
}
// output:
./prog.go:10:30: invalid operation: m1 == m2 (map can only be compared to nil)
./prog.go:14:30: invalid operation: b1 == b2 (slice can only be compared to nil)
./prog.go:18:30: invalid operation: f1 == f2 (func can only be compared to nil)
This rule also stands when these different types are assembled into a struct. We will fail to compare the two struct objects directly once one of the struct fields is not comparable.
We overcome it by using the reflect.DeepEqual function. Let’s do the above example.
package main
import (
"fmt"
"reflect"
)
func main() {
var m1 = map[string]int{"a":1, "b": 2}
var m2 = map[string]int{"b":2, "a":1}
fmt.Println("m1 == m2:", reflect.DeepEqual(m1, m2));
var b1 = []string{"1"}
var b2 = []string{"1"}
fmt.Println("b1 == b2:", reflect.DeepEqual(b1, b2));
var f1 = func() int {return 1}
var f2 = func() int {return 2}
fmt.Println("f1 == f2:", reflect.DeepEqual(f1, f2));
}
// output:
m1 == m2: true
b1 == b2: true
f1 == f2: falseKeep in mind that DeepEqual may not work as expected sometimes, especially for array and slice types. See below,
package main
import (
"fmt"
"reflect"
"bytes"
)
func main() {
var b1 []byte = nil
b2 := []byte{}
fmt.Println("b1 == b2:", reflect.DeepEqual(b1, b2)) //prints: b1 == b2: false
fmt.Println("b1 == b2:", bytes.Equal(b1, b2)) //prints: b1 == b2: true
v1 := []string{"one", "two"}
v2 := []interface{}{"one", "two"}
fmt.Println("v1 == v2:", reflect.DeepEqual(v1, v2)) // prints: v1 == v2: false
}PS: When comparing two []byte arrays, always use bytes.Equal or bytes.Compare
For slice, it is harder. I cannot find a common library that fixes the error in the above example, thus converting it and comparing them one by one is my only approach.
Don’t modify value reference variables
Updating the pass-by-value reference variables will not change the variables’ original value. Those pass-by-value references include:
- Function parameters, such as func abc( a int, b string).
- Struct’s object reference, such as
type Data struct { num int } func (d Data) setNum() { d.num = 8 }​ - Value reference in slice for-range loop.
numbers := []int{1,2,3} for _,v := range numbers { v = v * 10 // will not change the original slice } fmt.Println(numbers) // prints: [1 2 3]​
Always use pointers instead of value references if you prefer in-place updates.
Watch out for slice contamination
As one of the most widely used Go data types, there are various interception and extension operations on an original slice, so unexpected results may occur due to the underlying implementation. Let me give you some examples for a better understanding.
package main
import (
"bytes"
"fmt"
)
func main() {
// -------------- No.1 example
s1 := []int{1, 2, 3}
fmt.Println(len(s1), cap(s1), s1) //prints 3 3 [1 2 3]
s2 := s1[1:] // s2 still points to the s1 address
fmt.Println(len(s2), cap(s2), s2) //prints 2 2 [2 3]
for i := range s2 {
s2[i] += 20
}
fmt.Println(s1) //prints [1 22 23]
fmt.Println(s2) //prints [22 23]
// -------------- No.2 example
name := []byte("name:abc")
index := bytes.IndexByte(name, ':')
key := name[:index]
value := name[index+1:]
fmt.Println("key:", string(key)) //prints: key: name
fmt.Println("value:", string(value)) //prints: value: abc
key = append(key, "_str"...) // key and value are still share part of the memory
name = bytes.Join([][]byte{key, value}, []byte{':'})
fmt.Println("key:", string(key)) //prints: key: name_str
fmt.Println("value:", string(value)) //prints: value: str (not ok)
}The solution is very simple and always remember to copy a new slice.
package main
import (
"bytes"
"fmt"
)
func main() {
// -------------- No.1 example
s1 := []int{1, 2, 3}
fmt.Println(len(s1), cap(s1), s1) //prints 3 3 [1 2 3]
s2 := make([]int, 2) // s2 is totally new
copy(s2, s1[1:])
fmt.Println(len(s2), cap(s2), s2) //prints 2 2 [2 3]
for i := range s2 {
s2[i] += 20
}
fmt.Println(s1) //prints [1 2 3]
fmt.Println(s2) //prints [22 23]
// -------------- No.2 example
name := []byte("name:abc")
index := bytes.IndexByte(name, ':')
key := name[:index:index] //Very efficient way to create a new slice
value := name[index+1:]
fmt.Println("key:", string(key)) //prints: key: name
fmt.Println("value:", string(value)) //prints: value: abc
key = append(key, "_str"...) // key and value are still share part of the memory
name = bytes.Join([][]byte{key, value}, []byte{':'})
fmt.Println("key:", string(key)) //prints: key: name_str
fmt.Println("value:", string(value)) //prints: value: abc
}This blog about Go slice is worth your reading.
Close waitGroup in defer
You can find WaitGroup in most of Go’s open-source libraries because it is super helpful.
But we mistreat it sometimes, which results in serious issues as below.
func main() {
items := []int{1, 2, 3, 4, 5}
var wg sync.WaitGroup
for index, _ := range items {
wg.Add(1)
go func(i int) {
_, err := Do(i) // some logic returns an error
if err != nil {
log.Infof("err message:%v\\n", err)
return
}
wg.Done()
}(i)
}
wg.Wait()
}Have you ever noticed where this code goes wrong?
If the Do function returns an error, the goroutine will end directly without calling wg.Done(), and eventually blocks the waitGroup.
To prevent such mistakes, always call defer wg.Done() at the beginning of your goroutine code if you are using waitGroup.
func main() {
items := []int{1, 2, 3, 4, 5}
var wg sync.WaitGroup
for index, _ := range items {
wg.Add(1)
go func(i int) {
defer wg.Done()
_, err := Do(i) // some logic returns an error
if err != nil {
log.Infof("err message:%v\\n", err)
return
}
}(i)
}
wg.Wait()
}Recover from goroutine
To save ourselves from missing the error info and details, we always use recover. Otherwise, you may have a tough time debugging your goroutine code when it goes wrong.
Nevertheless, it can also go wrong sometimes. Looking at the code below, you will find that this code fails since the recover is called outside of goroutine.
package main
import (
"fmt"
"time"
)
func main() {
defer func() {
if err := recover(); err != nil {
fmt.Printf("%v\n", err)
}
}()
go func() {
panic("error")
}()
time.Sleep(2 * time.Second)
}The output is
panic: error
goroutine 34 [running]:
main.main.func2()
/tmp/sandbox430672375/prog.go:15 +0x39
created by main.main
/tmp/sandbox430672375/prog.go:14 +0x57The correct usage is like this.
package main
import (
"fmt"
"time"
)
func main() {
go func() {
defer func() {
if err := recover(); err != nil {
fmt.Printf("%v\n", err)
}
}()
panic("error")
}()
time.Sleep(2 * time.Second)
}But we can not add recover to all the goroutine in this way, which seems “stupid.”😂 Then how does Kubernetes source handle the below situation?
var PanicHandlers = []func(interface{}){logPanic}
func HandleCrash(additionalHandlers ...func(interface{})) {deads2k, 4 years ago: • start the apimachinery repo
if r := recover(); r != nil {
for _, fn := range PanicHandlers {
fn(r)
}
for _, fn := range additionalHandlers {
fn(r)
}
if ReallyCrash {
// Actually proceed to panic.
panic(r)
}
}
}
// logPanic logs the caller tree when a panic occurs (except in the special case of http.ErrAbortHandler).
func logPanic(r interface{}) {
if r == http.ErrAbortHandler {
return
}
const size = 64 << 10
stacktrace := make([]byte, size)
stacktrace = stacktrace[:runtime.Stack(stacktrace, false)]
if _, ok := r.(string); ok {
klog.Errorf("Observed a panic: %s\n%s", r, stacktrace)
} else {
klog.Errorf("Observed a panic: %#v (%v)\n%s", r, r, stacktrace)
}
}
It even includes the stack trace messages.
Now, goroutines in Kubernetes source call defer runtime.HandleCrash() in one line.
Treat channel carefully
Channel is a very powerful lightweight message queue in Go, and it is also where bugs usually occur. There are several Don’ts to follow to stay away from bugs.
- Don’t send data to a closed channel
- Don’t close a nil channel
- Don’t close a channel twice
An example for the first point,
package main
import (
"context"
"fmt"
"time"
)
func main() {
ch := make(chan struct{})
ctx, _ := context.WithTimeout(context.Background(), time.Second)
go func() {
time.Sleep(1 * time.Second)
ch <- struct{}{}
fmt.Println("end goroutine")
}()
select {
case <-ch:
fmt.Println("receive msg")
case <-ctx.Done():
fmt.Println("timeout")
}
time.Sleep(3 * time.Second)
}Here the goroutine waits two seconds, so the context ends before the goroutine can send any data to the channel, then the goroutine is blocked. The output is,
./prog.go:11:8: the cancel function returned by context.WithTimeout should be called, not discarded, to avoid a context leak
timeoutTo fix it, just simply change the channel’s buffer to 1, ch := make(chan struct{}, 1) . Then the output would be:
timeout
end goroutineCannot locate the address of Map’s value
In the previous tip, I mentioned that value reference change would not affect the original variable value. Here I will talk about how manipulating its value leads to compilation errors.
For Map, let’s see some illustrations.
package main
import (
"fmt"
)
type Data struct {
Number int
}
func (d *Data) print() {
fmt.Println("n:", d.Number)
}
type Print interface {
print()
}
func main() {
d := Data{}
d.Number = 1
m := map[string]Data{"a": d}
m["a"].Number = 2 // error: cannot assign to struct field m["a"].Number in map
fmt.Println("m:", m)
m["a"].print() // error: cannot call pointer method on m["a"]
}
The root is easy to see in the error message as well.
cannot take the address of m[“a”]
Obviously, you cannot update the object without knowing its address.
There are usually two alternatives to solve it.
- Assign the value to a temp variable and reassign it back after the change
- Use pointer types
package main
import (
"fmt"
)
type Data struct {
Number int
}
func (d *Data) print() {
fmt.Println("n:", d.Number)
}
type Print interface {
print()
}
func main() {
d := Data{}
d.Number = 1
m := map[string]Data{"a": d}
tmp := m["a"] // Use a temp variable
tmp.Number = 2
m["a"] = tmp
fmt.Println("m:", m) // print: map[a:{2}]
n := map[string]*Data{"a": &d} // use pointer
n["a"].print() // print: n: 1
}
Know the difference between nil interface{} and interface{} is nil
It is common to use interface{}as return types in functions since the default value of an interface{} is nil. But it is very tricky when you assign another nil value object to it.
package main
import (
"fmt"
)
func assign(s *string) interface{} {
return s
}
func main() {
var s *string
var i interface{}
fmt.Println(s, s == nil) //prints: true
fmt.Println(i, i == nil) //prints: true
i = assign(s)
fmt.Println(i, i == nil) //prints: false
}We have to be careful when returning a interface{} type. The right thing is always to return explicitly nil.
package main
import (
"fmt"
)
func assign(s *string) interface{} {
if s == nil {
return nil // explicit
}
return s
}
func main() {
var s *string
var i interface{}
fmt.Println(s, s == nil) //prints: true
fmt.Println(i, i == nil) //prints: true
i = assign(s)
fmt.Println(i, i == nil) //prints: true
}Summary
Go is currently one of the most efficient and engineered languages that facilitate programming. However, we will be in a debugging mess if we make the above mistakes. Keeping a list of these traps and their correspondent solutions in mind and comprehending Go’s underlying implementation will help us more proficient in the Go world.
Thanks for reading!
Note: The post is authorized by original author to republish on our site. Original author is Stefanie Lai who is currently a Spotify engineer and lives in Stockholm, original post is published here.
