Note: The post is authorized by original author to republish on our site. Original author is Stefanie Lai who is currently a Spotify engineer and lives in Stockholm, original post is published here.

Last week, Go1.16 was released, bringing relatively more changes than version 1.15, which was influenced by the epidemic.

The update is in many aspects, including compilation, deployment, standard library, etc. In the official Go document, all changes are classified based on Tools, Runtime, Complier, etc.

It is undoubtedly good to dig into all the changes, but sometimes we understand them this moment and forget them the next second, not to mention truly grasping all of them. For easier memorizing, I divide the changes, I think are important to ordinary developers, into two categories.

• Underlining Change, including all non-code-related changes.
• Standard library updates, which is closely related to coding.

Then let’s see some changes and discover their impacts with specific examples.

Underlining

• Better support for macOS arm64

By working closely with the Apple M1 chip’s arm64 architecture, there is a better stand by for Mac-based development. Furthermore, both cgo and the functions of compiling dynamic/static link libraries are supported. For more information, please refer to this official blog.

• Enabling Go Module by default

We are not required to declare GO111MODULE anymore when this environment variable is enabled by default. It’s 2021 now, and who hasn’t used Go Modules yet?

• Go install command

The go install command can install go programs without affecting mod file dependencies. For example, executing go install module1/test@0.0.1 plus the version number is enough.

And in the future, go get will no longer be accepted to compile and install, and you can close it with -d option, which means go install will be the only installation command.

• No import of relative path

The import cannot start with ., and non-ASCII characters are not allowed.

• Improvement on Race Debugger accuracy

When you turn on race detect, you may see more race reports.

• Switching default memory management policy from MADV_FREE back to MADV_MONTNEED

It is a comprehensive low-layer choice putting changes in the Linux kernel version and the reaction of various tools that match Go into consideration. Seen from practice from version 1.12 to 1.15, the shift to MADV_FREE seems to have done more harm than good. Therefore, the Go team changed the configuration back to MADV_MONTNEED, which was used before Golang 1.12, to meet the community requirements. For more discussion, please refer to #42330.

Standard Library Updates

Version 1.16 has made many modifications to the standard library.

• The io/ioutil package is deprecated, and its related APIs are migrated to the io/ospackage.
• A new file read-only abstract-related interface, the io/fs package, and its test package testing/fstest are introduced.
• It supports the embed package, embedding files through //go:embed command during the compilation process and then accessing. Thus, compiling the data file and the Go code together into a binary becomes more straightforward and the deployment more efficient.
• You must use Error method to throw an error in the goroutine of test method, while the Fatal method is excluded.
• Some minor changes are made in crypto package.
• Encoding package also has changes involving json, xml, and asn1.
• Flag adds a new Func function to make reading options easier.
• log.Default is a new method that provides a default package level Logger.
• StripPrefix function in the net/http package supports striping Path and RawPath.
• There are performance improvements. For instance, new WalkDir function in the path/filepath package is faster compared to the Walk function, and strconv.ParseFloat speeded by up to a factor of 2(though I rarely use it).
• New emojis are created! By supporting Unicode 13.0.0, we can get emojis like \U0001F972,\U0001F90C, from U+30000 to U+3FFFF.

Use Cases

Now we shall deepen our understanding of these new changes with some practices.

The first step is to install Go1.16.

$ go get golang.org/dl/go1.16
$ go1.16 download

Unpacking /Users/xxx/sdk/go1.16/go1.16.darwin-amd64.tar.gz ...

Success. You may now run 'go1.16'

The SDK will be put under the /Users/xxx/sdk/go1.16 directory.

Let’s try it!

Go Embed

Golang had a flaw that the binary executable file will not work if it was moved into other directories, the consequence of the executable binary’s incapability to find the configuration file used.

Sometimes we don’t feel this issue thanks to docker images that include all the configuration files and the executable binary. And now Go has worked out its own solution, the embed package, packaging the files together inside the binary files.

Use //go:embed in the code to include static files. Below is a simple example.

package main

import _ "embed"

//go:embed resources/embed_input.txt
var s string

func main() {
  print(s)
}

And embed supports types like string, byte, slice, fs, etc. The code is as below.

package main

import "embed"

//go:embed resources/embed_byte.txt
var b []byte

//go:embed resources/embed_fs.txt resources/embed_fs_1.txt
var f embed.FS

func main() {
  print(string(b))

  fi, _ := f.ReadFile("resources/embed_fs.txt")
  print(string(fi))

  fj, _ := f.ReadFile("resources/embed_fs_1.txt")
  print(string(fj))
}

To be noticed, you can embed the whole directory by declaring //go:embed resources.

In my opinion, embed feature has three benefits.

• It supports different embed grammar and is handy to use.
• It is secure because all embeds happen in the compiling time.
• It improves the portability of Go that binary files can run everywhere, just like Java jar files.

The new io/fs package

It is a fascinating feature!

io/fs that defines an interface for read-only file trees
— from io/fs design draft

The io/fs package offers an abstraction of different file system types, giving a more generic interface. The FS interface is simple, and we can find other functions in its implementation but Open is the only thing needed here.

To read files, Open is not enough. Therefore, you can also findFile and ReadDir interfaces in the io/fs package, which gives you a path to reading files and directories, respectively.

By learning the embed.FS code, I can better comprehend the io/fs-related logic. To access the file content, embed implements the fs.FS interface, fs.File interface, and fs.ReadDirFile interface, etc.

Here are some relevant codes.

Other implementations of fs.FS are in packages like http/template, text/template, net/http, archive/zip, archive/tar, etc., facilitating the zip and tar files processing and greatly simplifying the code when processing http files.

Deprecated io/ioutil package

The io/ioutil package’s existence could be confusing to developers because it is not a pure util collection. And exporting the types/functions to where they belong sounds reasonable. The proposals are # 40025 and # 42026.

Eight types/functions are affected in total, cited from here.

Discard => io.Discard

NopCloser => io.NopCloser

ReadAll => io.ReadAll

ReadDir => os.ReadDir

ReadFile => os.ReadFile

TempDir => os.MkdirTemp

TempFile => os.CreateTemp

WriteFile => os.WriteFile

Replace Walk function with WalkDir

I created a simple benchmark test to prove the performance improvement.

package main

import (
	"fmt"
	"io/fs"
	"os"
	"path/filepath"
	"testing"
)

func BenchmarkWalk(b *testing.B) {
	tmpDir, err := os.MkdirTemp("", "")
	if err != nil {
		fmt.Errorf("error creating temp directory: %v\n", err)
	}

	err = os.MkdirAll(filepath.Join(tmpDir, "dip/temp/txt/fd/dkf/kdfjk/sdf/dks"), 0755)
	if err != nil {
		os.RemoveAll(tmpDir)
		return
	}
	defer os.RemoveAll(tmpDir)
	os.Chdir(tmpDir)

	err = filepath.Walk(".", func(path string, info fs.FileInfo, err error) error {
		if err != nil {
			fmt.Printf("prevent panic by handling failure accessing a path %q: %v\n", path, err)
			return err
		}
		return nil
	})
	if err != nil {
		fmt.Printf("error walking the path %q: %v\n", tmpDir, err)
		return
	}
}

func BenchmarkWalkDir(b *testing.B) {
	tmpDir, err := os.MkdirTemp("", "")
	if err != nil {
		fmt.Errorf("error creating temp directory: %v\n", err)
	}
	err = os.MkdirAll(filepath.Join(tmpDir, "dip/temp/txt"), 0755)
	if err != nil {
		os.RemoveAll(tmpDir)
		return
	}
	defer os.RemoveAll(tmpDir)
	os.Chdir(tmpDir)

	err = filepath.WalkDir(".", func(path string, info fs.DirEntry, err error) error {
		if err != nil {
			fmt.Printf("prevent panic by handling failure accessing a path %q: %v\n", path, err)
			return err
		}
		return nil
	})
	if err != nil {
		fmt.Printf("error walking the path %q: %v\n", tmpDir, err)
		return
	}
}

The code is very simple, creating a multi-level directory and then traversing. The only difference is which function you choose, WalkDir or Walk. Then run to see the result.

Intuitively, WalkDir is almost twice as fast, of course, only in scenarios with more than four layers of nested directories(my local test env). If there is only a one-layer directory, the two functions make no difference.

I am not about to demonstrate some other trivial changes one by one, such as the cryto package, which I have rarely used.

At the end

Go 1.16 has just been released, and whether there are any issues, whether developers will resist it, accept it or prefer it, let time tell. For me, just the go embed function is worth my upgrade and attempt. Let’s keep an eye on the community update. Generics is on the way.

Thanks for reading!

Reference

• https://blog.golang.org/go1.16
• https://eddycjy.com/posts/go/go1.16-2/
• https://docs.studygolang.com/blog/ports
• https://blog.golang.org/go116-module-changes
• https://go.googlesource.com/proposal/+/master/design/draft-iofs.md
• https://docs.google.com/document/d/1D13QhciikbdLtaI67U6Ble5d_1nsI4befEd6_k1z91U/view