Migrating From Jenkins To Codeship: Testing

Codeship NewsDevelopment

Reading Time: 3 minutes

This tutorial originally is published on our Codeship Documentation. You can find the article and other migration tutorials there.

In every project, there is a need to test and confirm that functionality is working as planned. This takes the form of numerous practices to determine: from unit, integration, or user interface testing to smoke testing, black box, or others. Some of these forms of testing are simple while some are complex in their configuration, management, and usage.

Testing with Jenkins Systems

In the Jenkins system, there are a number of ways to initiate tests to run. There are plugins and other respective hacks, scripts, and webhooks to get unit tests or related actions to trigger.

Jenkins does a pretty decent job performing unit tests or basic integration tests if you keep it simple. However, having an environment in which tests are run in a singular pipeline can lead to side effects that require timely troubleshooting and problem-solving to fix. Because of this static server state versus running containers and rebuilding the environment every time, certain things get set, configured, or otherwise altered that do not mimic production environments.

The complexity of managing this in a set server build with Jenkins and the respective tests needed to manage that becomes a difficult task unto itself.

Testing Your Applications with Codeship Pro as Compared to Jenkins

With Codeship Pro, the testing options open up massively, without any plugins or extra configuration.

Codeship Pro’s simple configuration files (codeship-services.yml and codeship-steps.yml) can be run on the container instances that are completing the actual build. By association, not only does this provide testing insight into the Codeship build that is run, but it also provides the ability to run the build locally using the Codeship Pro Jet CLI.

Simply set up the codeship-steps.yml file as shown.

- name: deluge
  service: deluge_build
  command: echo 'A first build step for whatever.'
- name: deluge_tests
  service: go_tests
  command: go test
- name: deluge_another_test
  service: ruby_tests
  command: ruby tests

Now whenever the build is run, whether locally or via commit to the Codeship build process, the tests will be run and results displayed. For a recent setup, here are some of the results from a Codeship Pro build, when it is run with Codeship. The picture shows Codeship Pro’s web interface for the build output.


As mentioned, with the Jet CLI tool, you can run your builds locally before you push to your source code management system or to help you debug. Here you can see the output in the console after using the jet steps command.


With these steps, Codeship provides a means of unit test coverage for the build. This doesn’t provide a local build option, but tests could of course simply be executed manually via the Jet CLI, IDE, or whatever method for the local option.

Sign up for a free Codeship Account

Testing Your Applications with Codeship Basic as Compared to Jenkins

The most common testing mechanism is executing the unit tests within a code base. Let’s take a quick look at the setup of tests in Codeship Basic.

First, set the command within your project’s tests section of the interface.


Here is a quick code sample test for a Node.js application that we have here.

var assert = require('assert');

describe('Where the important things happen', function () {
  describe('where the functional is good all', function () {
    it('should be good', function () {
      assert.equal(true, true);

GitHub Testing Integration

When migrating to Codeship, once you added test commands in the Project Settings as shown in the previous section, then any code pushed to your repository on any branch will trigger the source code management system’s interface to identify if the merge will build successfully.

For example, click the Compare & pull request button on GitHub.


Next you’ll see the pending check running.


Finally the build succeeds, and the green check box displays. The Merge pull request button turns green.



In this post, we’ve shown some of the power and options for testing on Codeship Basic and Codeship Pro. It opens the door to add unit, integration, smoke, or other systems testing. Even further testing could be implemented by pushing deployment to utilize remote UI testing or other means.

If you’re interested in learning more benefits of switching from Jenkins to Codeship, you can learn about them here.

Subscribe via Email

Over 60,000 people from companies like Netflix, Apple, Spotify and O'Reilly are reading our articles.
Subscribe to receive a weekly newsletter with articles around Continuous Integration, Docker, and software development best practices.

We promise that we won't spam you. You can unsubscribe any time.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.