Understand unsafe in GoLang

Before going to understand unsafe package in GoLang, the first thing needs to talk about is the pointer in GoLang. If you have a background of C language, you must know what pointer means and its usage. With pointer, you are free to operate any data at memory level which means you have great power, but this means that you have great responsibility as well. That's why it might be considered unsafe in lots of cases.

Take a look at a simple example of doubling an integer.

package main

import "fmt"

func double(x int) {
    x += x
}

func main() {
    var a = 3
    double(a)
    fmt.Println(a) // 3
}

The above code will not achieve the goal of doubling variable a.  The reason is that GoLang function passes parameter by value, when a is passed to double(), only a copy of its value is passed, the address of a is not passed. Hence when doubling it, it doubles its copy instead of a itself. But the value can be doubled as expected if now a pointer is passed.

package main

import "fmt"

func double(x *int) {
    *x += *x
    x = nil
}

func main() {
    var a = 3
    double(&a)
    fmt.Println(a) // 6

    p := &a
    double(p)
    fmt.Println(a, p == nil) // 12 false
}

Compared to pointer in C, pointer in GoLang comes with more restrictions. They cannot be used freely as C pointer but can still provide necessary flexibilities for most developers to use. The major restrictions of pointer in GoLang are:

No mathematic operations can be performed on pointer

It means that a pointer cannot have operations like addition/subtraction as in C. 

a := 5
p := &a

p++
p = &a + 3

The above code cannot be compiled as it will throw invalid operation error on p++.

Cannot convert between different types of pointer

Two different types of pointer cannot be converted between each other. i.e, cannot convert an *int to a *float64 pointer.

func main() {
    a := int(100)
    var f *float64

    f = &a
}

Above code will throw compilation error:

cannot use &a (type *int) as type *float64 in assignment

Different types of pointer cannot be compared with == or !=

Two pointers can be compared only when the two pointers have the same type or can be converted to each other. Otherwise they cannot be compared with == or !=.

Cannot assign one type of pointer to another type of pointer

Similar to reason in above restriction.

Now we have talked about pointer a bit. Let's move on to GoLang unsafe. The pointer talked about above is considered as type safe pointer. There is also type unsafe pointer, it is unsafe.Pointer residing in unsafe package.

unsafe package is normally used during code compilation. As its name suggests, it is not safe, hence it's not recommended to use by GoLang creators. But it does provide some capability which can help improve code efficiency a lot though it brings more danger.  It can be used to operate on memory directly and at the same time it can bypass type system check which was designed to safe guard the type safety in GoLang but also bring inefficiency.

In unsafe package, there is a Pointer defined.

type ArbitraryType int
type Pointer *ArbitraryType

This is similar to void* in C. Also three additional functions are defined.

func Sizeof(x ArbitraryType) uintptr
func Offsetof(x ArbitraryType) uintptr
func Alignof(x ArbitraryType) uintptr

• Sizeof returned the number of bytes x takes, it doesn't count the actual number of bytes its content takes. 
• Offsetof returns the position where one member of a struct within the struct away from the beginning position of the struct
• Alignof returns m which means the number of bytes which can be divided when align the memory in the struct.

All return type of above functions are uintptr, it can be converted to unsafe.Pointer and vice versa. 

unsafe package provide two important features:

1. Any pointer can be converted to unsafe.Pointer and vice versa
2. uintptr can be converted to unsafe.Pointer and vice versa

No mathematic operation can be performed on pointer directly, however mathematic operation can be performed on uintptr. Hence if want to perform mathematic operation on pointer, can first convert it to uintptr and perform mathematic operation and convert it back to pointer. 

After knowing this, we would show some use cases of unsafe. 

Get or update value of unexported property in struct

With Offsetof(), the position of each member in a struct can be found out and their memory can be accessed and updated accordingly.

package main

import (
    "fmt"
    "unsafe"
)

type Programmer struct {
    name string
    language string
}

func main() {
    p := Programmer{"stefno", "go"}
    fmt.Println(p)

    name := (*string)(unsafe.Pointer(&p))
    *name = "qcrao"

    lang := (*string)(unsafe.Pointer(uintptr(unsafe.Pointer(&p)) + unsafe.Offsetof(p.language)))
    *lang = "Golang"

    fmt.Println(p)
}

The output will be:

{stefno go}
{qcrao Golang}

Now if the struct is referenced in another package, unsafe.Pointer can be used to access its unexported values using Sizeof() to get the member size.

For example, if the struct Programmer is defined in package a:

package a

type Programmer struct {
    name string
    age int
    language string
}

And all three members are unexported and in another package can access and update its members using unsafe.

func main() {
    p := a.Programmer{"stefno", 18, "go"}
    fmt.Println(p)

    lang := (*string)(unsafe.Pointer(uintptr(unsafe.Pointer(&p)) + unsafe.Sizeof(int(0)) + unsafe.Sizeof(string(""))))
    *lang = "Golang"

    fmt.Println(p)
}

Output

{stefno 18 go}
{stefno 18 Golang}

Convert string to slice

A typical example is to convert string to bytes slice, but the requirement is zero-copy which means there shouldn't be a new copy of original data created. To do this, let's look at the underlying data structure of string and slice.

type StringHeader struct {
    Data uintptr
    Len  int
}

type SliceHeader struct {
    Data uintptr
    Len  int
    Cap  int
}

Here the Data is both an uintptr, basically what needs to be done is to just let both data type share the same underlying []byte array.

func string2bytes(s string) []byte {
    stringHeader := (*reflect.StringHeader)(unsafe.Pointer(&s))

    bh := reflect.SliceHeader{
        Data: stringHeader.Data,
        Len:  stringHeader.Len,
        Cap:  stringHeader.Len,
    }

    return *(*[]byte)(unsafe.Pointer(&bh))
}

func bytes2string(b []byte) string{
    sliceHeader := (*reflect.SliceHeader)(unsafe.Pointer(&b))

    sh := reflect.StringHeader{
        Data: sliceHeader.Data,
        Len:  sliceHeader.Len,
    }

    return *(*string)(unsafe.Pointer(&sh))
}

Reference: https://mp.weixin.qq.com/s?__biz=MzAxMTA4Njc0OQ==&mid=2651436596&idx=1&sn=ad104595eac75569fd3494444312b1e3&chksm=80bb68c6b7cce1d07d9f7d0dcd173a6868cc01cb742b76a01a048e37446d9f4a6fedda9164ba&scene=21