Godep: Dependency Management in Go

Development

Reading Time: 6 minutes

Go differs from many other languages in that a wide range of dependency management methods and tools exist for it. The approach endorsed by the Go team involves vendoring dependencies within the project folder and modifying the the import statements to support the new location.

Godep works differently than the endorsed approach… Instead of requiring changes to be made to the source code to support the vendored import path, the GOPATH is modified to use the vendored folder. This is beneficial since the same code will work both within the Godep environment and in a standalone environment using go get to set up the GOPATH. Other dependency management methods and tools exist, so be sure to read up on them and pick one suitable for your project or organization.

For basic usage instructions, check out the Godep documentation. Based on my experience with Godep, I’ll give you a quick overview on the pros and cons of using it, as well as a few guidelines.

How Godep Works

A godep save command will copy all imported packages in their entirety from your current GOPATH into a vendored workspace folder in ./Godeps/_workspace. A list of those packages will be stored with relevant version information in a master file, Godeps/Godeps.json. This is done not just for the packages your project directly imports but also for any imported by your dependencies — essentially anything required to build your project. You must add this entire folder SCM and it should never be manually altered in any way.

Using Godep is as simple as prepending your normal Go commands like go test or go build with the godep command. This uses a temporarily extended GOPATH which prioritizes the Godep vendor directory.

$ godep save ./…       # saves your GOPATH to the Godep folder
$ godep go build ./…   # builds using the Godep vendored dependencies
$ godep go test ./…    # tests using the Godep vendored dependencies

From here, should you apply a change to your GOPATH, your project will be isolated.

$ go get -u github.com/golang/protobuf/… # update a dependency
$ go build ./…                          # build using standard GOPATH
$ godep go build ./…                     # build using Godep vendored version

godep update versus godep save

There are two methods to apply changes to your Godep folder. Understanding the correct cases for using each will save you a huge amount of time and frustration. godep save and godep update work very differently and alter your Godep in different ways. The key thing to remember here is that if you’re ever in doubt, use godep save.

godep update takes a specific dependency package and updates the vendored instance of that package with the version in your GOPATH. This will update files with changes, add new files, remove old ones, and update the version SHA listed in the Godeps.json file.

$ go get -u github.com/golang/protobuf/…
$ godep update github.com/golang/protobuf/…

This will not add or remove sub-packages from dependency management, nor will it update any other dependencies recursively. Only previously imported packages are listed in the Godeps.json file and only those listed are updated.

Updating the entire package will update any references to sub-packages; however no new packages will be added, nor old ones removed. Similarly, if your dependency update is dependent upon another change elsewhere in your dependency stack, you may run into issues. godep update only touches the packages listed in Godeps.json, which match the provided package pattern.

In contrast, godep save applies the entire relevant GOPATH to the Godeps folder and will add/remove packages as needed. Because it’s based off of your GOPATH, godep save can also check for build errors and non-clean repositories before applying changes, enforcing dependency cohesion.

Given the dangers of using godep update (missing packages and dependencies), it’s much safer to use godep save. The only situation where it’s safe to use godep update are when both of these conditions are satisfied:

  • No dependencies of your target dependency need to be updated.
  • No imports were added to or removed from your target dependency.

If the dependency is external to your organization, it can be difficult to determine what changes are taking place, so it is safer to never use **godep update` on anything third-party.

Sign up for a free Codeship Account

Caveats of Working with Godep

Beyond knowing the basic commands to use with Godep, there are a number of gotchas.

Godep can diverge across projects

In situations where a dependency is versioned from multiple sources, it can easily be represented at different versions across the same Godep environment.

Consider a case where internal codebase ProjectA lists ProjectB and ThirdPartyA as dependencies. Both ProjectB and ThirdPartyA list ThirdPartyB as a dependency. If ThirdPartyA is ever updated to require a new version of ThirdPartyB while ProjectB continues to reference the older version of ThirdPartyB, which version does ProjectA list in Godeps?

ProjectA
 ├-- ProjectB
 | └-- ThirdPartyB
 ├-- ThirdPartyA
 | └-- ThirdPartyB

The answer is simple.

Godep has no internal conflict resolution mechanism and does not resolve Godep requirements of dependencies. The decision about which version to use is left up to the developer; in this case, ProjectA would use whichever version of ThirdPartyB was last set in Godep. Since the last write was from the update of ThirdPartyA, ProjectA would reference the newer version. Any code from ProjectB would reference the newer version when loaded as a dependency.

This is a subtle point, but it highlights the need for caution when updating dependencies, as well as the need for all dependency version changes to be synchronized across projects. Compile errors would be caught when ProjectA was built, but subtle behavioural changes might not be caught till later.

Therefore, a dependency update should be applied to all projects simultaneously. Consider updating dependencies via a looped script, applying the change to all of your codebases at once.

for project in projects
  cd workspace/project
  git checkout master && git pull
  godep restore
  go get -u ThirdPartyB/…
  godep save ./…
  git checkout -b “updating-godeps”
  git commit ./Godeps -m “Updating Godeps”
  git push origin updating-godeps
end

In cases where several internal projects require a synchronized version of a dependency, it’s much simpler to use a forked version of a dependency and push dependency control up to the fork that your organization maintains. This fork becomes the central control for versioning that dependency. However, it’s more likely that an update to that dependency will cause critical breaking changes across multiple projects.

For a list of various dependency management methods and tools for Go, check out the Golang docs on the subject.

Godep still uses your local GOPATH

Despite Godep providing a versioned GOPATH instance, it still uses your original GOPATH. The vendored workspace is checked for a package before your main GOPATH. Generally this doesn’t cause issues, but it can be troublesome in some cases.

Since your main GOPATH is referenced, missing dependencies may not be caught until later in the development cycle. Any package missing from Godep is most likely present in your GOPATH; this seamless fallback makes it difficult to determine which dependency version a command ends up using. Due to this composite GOPATH, build errors can appear when dependency versions are incompatible. The best way to avoid this issue, and catch missing dependencies early, is to build in a throwaway GOPATH or in a container.

When to Use Godep

Not every project may require the broad dependency control as provided by Godep.

Unlike import path-based vendoring, Godep vendors the entire set of dependencies regardless of a specific desire to version them. This does mean that no dependencies will ever be updated unless explicitly altered, first through GOPATH and then through Godep.

Should your organization have a large number of common dependencies across different projects, you may want to look into using a forked dependency model. Godep provides a locally controlled, customizable dependency management system. When used with care, this system can support highly versioned and reproducible builds, especially in change resistive environments with few shared dependencies.

Subscribe via Email

Over 60,000 people from companies like Netflix, Apple, Spotify and O'Reilly are reading our articles.
Subscribe to receive a weekly newsletter with articles around Continuous Integration, Docker, and software development best practices.



We promise that we won't spam you. You can unsubscribe any time.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.

  • You should take a look into http://getgb.io/

  • That looks pretty cool Todd, it has a lot less sugar coating than Godep, but one of the lessons I’ve learned is that the only way to be sure of your dependencies is to do everything manually.

    That being said, it would be cool if gb had a Godep style `save` function to vendorize the relevant parts of your GOPATH, but it sounds like there is a gb-vendor plugin for just that.

    I really like the cache control flags, I’ll have to play round with this.

  • Luca Bruno

    I’m lately spamming this as I’m the author of the go packaging infrastructure in Nix. I think the go community should take a look at it because it solves most problems raised by the community: http://lethalman.blogspot.it/2015/02/developing-in-golang-with-nix-package.html .

    You don’t have to think about GOPATH anymore, or go get, or whatever else, and you can pin dependencies at a specific commit. Works per-project and with multiple go versions.

  • Pingback: 1 – Godep: Dependency Management in Go | Offer Your()