Luca Sepe

Go's Package Management and Module Systems

Go Modules Coding

go mod

Is the package management system for Go.

a module is …

  • a collection of related Go packages
  • identified by a unique uri
  • semantically versioned

Downloaded dependencies are cached locally and reused.

Versioning modules

Modules should follow semantic versioning.

  • no removals or behavior changes unless it's a major version upgrade
  • tags in the form of v0.0.0

Minimal version selection (MVS)

Init the module

go mod init module_uri

└─ $ ▶ go mod init github.com/lucasepe/crumbs/crumbs
go: creating new go.mod: module github.com/lucasepe/crumbs/crumbs

…and write some code…

Build it

go build

└─ $ ▶ go build
go: downloading github.com/emicklei/dot v0.14.0
go: downloading github.com/teris-io/shortid v0.0.0-20171029131806-771a37caa5cf

The go.mod file

└─ $ ▶ cat go.mod 
module github.com/lucasepe/crumbs/crumbs

go 1.14

require github.com/lucasepe/crumbs v0.2.0
  • module [uri] - the import path prefix of the module
  • require ( … ) - list of modules our app depends on and the version we use
    • by default the minimum version we can use

The go.sum (dependencies checksum) file

Cryptographic hashes are stored in the go.sum file.

└─ $ ▶ cat go.sum 
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/emicklei/dot v0.14.0 h1:DJbbkKThQ0nW361NB79CqrWcKpYR1JoqJB3FcTUgBEU=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
...

each line contains three fields:

  • module uri
    • (es. github.com/davecgh/go-spew)
  • the related version (es. v1.1.0)
  • hash based on file tree and download timestamp
    • (es. h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=)

Both files go.mod and go.sum should be committed

https://github.com/golang/go/wiki/Modules#should-i-commit-my-gosum-file-as-well-as-my-gomod-file

Listing all available versions

go list -m -versions

What versions have been released of this dependency?

└─ $ ▶ go list -m -versions github.com/gorilla/mux
github.com/gorilla/mux v1.2.0 v1.3.0 v1.4.0 v1.5.0 v1.6.0 v1.6.1 v1.6.2 v1.7.0 v1.7.1 v1.7.2 v1.7.3 v1.7.4 v1.8.0

Select a specific version

latest version (default for ‘go get’)

go get repo/uri@latest

└─ $ ▶ go get github.com/gorilla/mux@latest
go: downloading github.com/gorilla/mux v1.8.0
go: github.com/gorilla/mux latest => v1.8.0

exact version

go get repo/uri@vX.Y.Z

└─ $ ▶ go get github.com/gorilla/mux@v1.7.2
go: downloading github.com/gorilla/mux v1.7.2

specific commit

go get repo/uri@commit-hash

└─ $ ▶ go get github.com/gorilla/mux@948bec34b5168796bf3fc4dbb09215baa970351a
go: github.com/gorilla/mux 948bec34b5168796bf3fc4dbb09215baa970351a => v1.7.5-0.20200517040254-948bec34b516
go: downloading github.com/gorilla/mux v1.7.5-0.20200517040254-948bec34b516

a branch

go get repo/uri@branch-name

└─ $ ▶ go get fyne.io/fyne@develop
go: downloading fyne.io/fyne v1.3.4-0.20200910103200-2c3e872d5686
go: fyne.io/fyne develop => v1.3.4-0.20200910103200-2c3e872d5686

Dependencies cache

Downloaded dependencies are cached in GOPATH:

└─ $ ▶ go env | grep GOPATH
GOPATH="/home/lus/go"

Downloaded dependencies located in $GOPATH/pkg/mod/:

└─ $ ▶ tree /home/lus/go/pkg/mod/github.com
/home/lus/go/pkg/mod/github.com
├── davecgh
│   ├── go-spew@v1.1.0
│   │   ├── ...
│   └── go-spew@v1.1.1
│       ├── ...
├── emicklei
│   └── dot@v0.14.0
│       ├── ...
├── gorilla
│   ├── mux@v1.7.2
│   │   ├── ...
│   ├── mux@v1.7.5-0.20200517040254-948bec34b516
│   │   ├── ...
│   └── mux@v1.8.0
│       ├── ...

Modules for dependencies located in $GOPATH/pkg/mod/cache/download/:

└─ $ ▶ tree /home/lus/go/pkg/mod/cache/download/github.com
/home/lus/go/pkg/mod/cache/download/github.com
├── gorilla
│   └── mux
│       └── @v
│           ├── v1.7.2.mod
│           ├── v1.7.5-0.20200517040254-948bec34b516.mod
│           ├── v1.8.0.mod

Clean the downloaded dependencies cache

To remove the entire module download cache, including unpacked source code of versioned dependencies

└─ $ ▶ go clean --modcache

Replacing dependency version

For example, to point to the local version of a dependency rather than the one over the internet.

└─ $ ▶ go mod edit -replace github.com/lucasepe/crumbs=../

Let's view the go.mod file content:

└─ $ ▶ cat go.mod 
module github.com/lucasepe/crumbs/crumbs

go 1.14

require github.com/lucasepe/crumbs v0.2.0

replace github.com/lucasepe/crumbs => ../

Download all dependencies

go mod download

For example to be able to work totally offline.

└─ $ ▶ go mod download -x
# get https://proxy.golang.org/github.com/stretchr/testify/@v/v1.6.1.mod
# get https://proxy.golang.org/github.com/emicklei/dot/@v/v0.14.0.mod
# get https://proxy.golang.org/github.com/davecgh/go-spew/@v/v1.1.1.mod
...
  • the -x flag causes download to print the commands download executes

Is a module really needed?

go mod why

└─ $ ▶ go mod why fyne.io/fyne
# fyne.io/fyne
(main module does not need package fyne.io/fyne)
└─ $ ▶ go mod why github.com/teris-io/shortid
# github.com/teris-io/shortid
github.com/lucasepe/crumbs
github.com/teris-io/shortid

…that is…

github.com/lucasepe/crumbs
 └──github.com/teris-io/shortid

Removing unused dependencies

go mod tidy

To remove dependencies not used by your code anymore:

└─ $ ▶ go mod tidy -v
unused fyne.io/fyne
  • the -v flag causes tidy to print information about removed modules to standard error

Verifying dependencies

go mod verify

You want to be sure that your go.sum is valid

  • that nobody did “git push — force” over a tag you’re using into something hacked (malicious code)
  • that dependencies in cache have the expected content
└─ $ ▶ go mod verify
all modules verified

Vendoring dependencies

go mod vendor

Vendoring is the act of making a copy of the third-party packages your project depends on. The copy will be placed in a vendor directory within your project.

Some benefits of vendoring:

  • be able to use git diff to see the changes when you update a dependency
    • this history will be maintained in your git repo
  • if a module disappears from the internet, you are covered
└─ $ ▶ go mod vendor -v
# github.com/davecgh/go-spew v1.1.1
## explicit
github.com/davecgh/go-spew/spew
# github.com/emicklei/dot v0.14.0
## explicit
github.com/emicklei/dot
...
  • the -v flag causes vendor to print the names of vendored modules and packages to standard error