Create a Jenkins Pipeline

In this tutorial, you will explore the Pipeline functionality.

At the end of this tutorial, you will be able to:

  1. Create a simple pipeline

  2. Use a ‘Jenkinsfile’ in your project

  3. Use manual input steps in a pipeline

Part 1 – Create a Simple Pipeline

We can create a pipeline job that includes the pipeline script in the job configuration, or the pipeline script can be put into a ‘Jenkinsfile’ that’s checked-in to version control.

To get a taste of the pipeline, we’ll start off with a very simple pipeline defined in the job configuration.

Prerequisite: We need to have a project in source control to check-out and build. For this example, we’ll be using the ‘SimpleGreeting’ project that  cloned to a ‘Git’ repository at ‘C:\Software\repos\SimpleGreeting.git’.

For the above setup, refer to Part 2 of the blog Creating a Jenkins Job.

1. To connect to Jenkins, open Firefox and enter the following URL:

http://localhost:8080/

2. Login with your user/password.

3. Click on the New Item link.

4. Enter ‘SimpleGreetingPipeline‘ as the new item name, and select ‘Pipeline‘ as the item type.

5. When the input looks as above, click on OK to create the new item.

6. Scroll down to the Pipeline section and enter the following in the Script text window.

node {
    stage 'Checkout'
    git url: 'C:\\Software\\repos\\SimpleGreeting.git'
    
    stage 'Maven build'
    bat 'mvn install'
    
    stage 'Archive Test Results'
    step([$class: 'JUnitResultArchiver',
      testResults: '**/target/surefire-reports/TEST-*.xml'])
}

This pipeline is divided into three stages. First, we checkout the project from our ‘git’ repository. Then we use the ‘bat’ command to run ‘mvn install’ as a Windows batch file. Finally, we use the ‘step’ command to utilize a step from a standard Jenkins plugin – in this case, the JUnitResultArchiver, to save and display the results of the unit tests.

All of the above is wrapped inside the ‘node’ command, to indicate that we want to run these commands in the context of a workspace running on one of Jenkins execution agents (or the master node if no agents are available).

7. Click on Save to save the changes and return to the project page.

8. Click on Build Now to start up a pipeline instance.

9. After a few moments, you should see the Stage View appear, and successive stages will appear as the build proceeds, until all three stages are completed.

Part 2 – Pipeline Definition in a ‘Jenkinsfile’

For simple pipelines or experimentation, it’s convenient to define the pipeline script in the web interface. But one of the common themes of modern software development is “If it isn’t in version control, it didn’t happen”. The pipeline definition is no different, especially as you build more and more complex pipelines.

You can define the pipeline in a special file that is checked out from version control. There are several advantages to doing this. First, of course, is that the script is version-controlled. Second, we can edit the script with the editor or IDE of our choice before checking it in to version control. In addition, we can employ the same kind of “SCM Polling” that we would use in a more traditional Jenkins job.

In the following steps, we’ll create a Jenkinsfile and create a pipeline job that uses it.

1. Open the Eclipse editor. If this lab is completed in the normal sequence, you should have the ‘SimpleGreeting’ project already in Eclipse’s workspace. If not, check out the project from version control (consult your instructor for directions if necessary).

2. In the Project Explorer, right-click on the root node of the SimpleGreeting project, and then select New → File.

3. Enter ‘Jenkinsfile‘ as the file name.

4. Click Finish to create the new file.

5. Enter the following text into the new file (Note: this is the same script that we used above, so you could copy/paste it from the Jenkins Web UI if you want to avoid some typing):

node {
    stage 'Checkout'
    git url: 'C:\\Software\\repos\\SimpleGreeting.git'
    
    stage 'Maven build'
    bat 'mvn install'
    
    stage 'Archive Test Results'
    step([$class: 'JUnitResultArchiver',
      testResults: '**/target/surefire-reports/TEST-*.xml'])
}

6. Save the Jenkinsfile by selecting File → Save from the main menu, or by hitting Ctrl-S.

7. In the Project Explorer, right-click on the SimpleGreeting node, and then select Team → Commit…

8. Eclipse will display the Commit Changes dialog. Click the check box next to Jenkinsfile (to include that file in the commit) and enter a commit message.

9. Click Commit and Push, and then click OK to dismiss the status dialog.

Now we have a Jenkinsfile in our project, to define the pipeline. Next, we need to create a Jenkins job to use that pipeline.

10. In the Jenkins user interface, navigate to the root page, and then click on New Item.

11. Enter ‘SimpleGreetingPipelineFromGit‘ as the name of the new item, and select Pipeline as the item type.

12. Click OK to create the new item.

13. Scroll down to the Build Triggers section.

14. Click on Poll SCM and enter ‘* * * * *’ as the polling schedule. This entry will cause Jenkins to poll once per minute.

15. Scroll down to the Pipeline section, and change the Definition entry to ‘Pipeline Script from SCM’

16. Enter the following:

SCM: Git

Repository URL: C:\Software\repos\SimpleGreeting.git

(Press the tab key)

17. The Pipeline section should look similar to:

18. Click Save to save the new configuration.

19. Click Build Now to launch the pipeline.

20. You should see the pipeline execute, similar to the previous section.

Part 3 – Try out a Failing Build

The pipeline that we’ve defined so far appears to work perfectly. But we haven’t tested it with a build that fails. In the following steps, we’ll insert a test failure and see what happens to our pipeline.

1. In Eclipse, go to the Project Explorer and locate the file ‘Greeting.java’. It will be under src/main/java in the package ‘com.simple’.

2. Open ‘Greeting.java’.

3. Locate the line that reads ‘return “GOOD”;’. Change it to read ‘return “BAD”;’

4. Save the file.

5. In the Project Explorer, right-click on Greeting.java and then select Team → Commit… (This is a shortcut for committing a single file).

6. Enter an appropriate commit message and then click Commit and Push.

7. Click OK in the results box, to close it.

8. Switch back to Jenkins.

9. In a minute or so, you should see a build launched automatically. Jenkins has picked up the change in the ‘Git’ repository and initiated a build. If nothing happens then click Build Now.

This time, the results are a little different. The ‘Maven Build’ stage is showing a failure, and the ‘Archive Test Results’ stage was never executed.

What’s happened is that the unit tests have failed, and Maven exited with a non-zero result code because of the failure. As a result, the rest of the pipeline was canceled. This behavior probably isn’t what you want or what you expect in most cases. We’d like to go ahead and archive the test results, even when there’s a failure. That way, we can see the trend including failed tests.

The solution here is to add a command-line parameter to the Maven invocation. If we add ‘-Dmaven.test.failure.ignore’ to the Maven command line, then Maven will continue with the build even if the tests fail.

10. Go back to Eclipse and open the ‘Jenkinsfile‘ if necessary.

11. Alter the ‘bat “mvn…” line to read as follows:

   bat 'mvn -Dmaven.test.failure.ignore install'

12. Save the ‘Jenkinsfile’. Commit and push the changes using the same technique as above.

13. After a minute or so, you should see a new Pipeline instance launched. If nothing happens then click Build Now.

This time, the pipeline runs to completion, and the test results are archived as expected. Notice that the build is now flagged as ‘unstable’ (indicated by the yellow color and the icon). The JUnit archiver noticed the failures and flagged the build unstable, even though Maven exited normally.

Part 4 – Add a Manual Approval Step

One of the interesting features of the Pipeline functionality is that we can include manual steps. This is very useful when we’re implementing a continuous deployment pipeline. For example, we can include a manual approval step (or any other data collection) for cases like ‘User Acceptance Testing’ that might not be fully automated.

In the steps below, we’ll add a manual step before a simulated deployment.

1. Go to ‘Eclipse’ and open the ‘Jenkinsfile‘ if necessary.

2. Add the following to the end of the file after the closing }:

stage 'User Acceptance Test'
def response= input message: 'Is this build good to go?',
 parameters: [choice(choices: 'Yes\nNo', 
 description: '', name: 'Pass')]
if(response=="Yes") {
 node {
  stage 'Deploy'
  bat 'mvn -Dmaven.test.failure.ignore install'
 }
}	

This portion of the script creates a new stage called ‘User Acceptance Test’, then executes an ‘input’ operation to gather input from the user. If the result is ‘Yes’, the script executes a deploy operation in a new ‘node’ step. (In this case, we’re repeating the ‘mvn install’ that we did previously. Only because we don’t actually have a deployment repository setup)

3. Save and commit ‘Jenkinsfile’ as previously.

When the pipeline executes, watch for a “paused” stage called ‘User Acceptance Test”. If you move your mouse over this step, you’ll be able to select “Yes” or “No”.

4. Move your mouse over the ‘User Acceptance Test’ step, select Yes and click Proceed.

You should see the job run to completion.

5. Run the pipeline again (click Build Now), but this time, select No on the ‘User Acceptance Test’ and click Proceed, you’ll see that the pipeline exits early and doesn’t run the ‘deploy’ stage.

What’s happened is that the final ‘Deploy’ stage was only executed when we indicated that the ‘User Acceptance Test’ had passed.

6. Close all.

Part 5 – Review

In this tutorial, we explored the Pipeline functionality in Jenkins. We built a simple pipeline in the Jenkins web UI, and then used a ‘Jenkinsfile’ in the project. Lastly, we explored how to gather user input, and then take different build actions based on that user input.

 

Create a Jenkins Job

In this tutorial, you will create and build a job in Jenkins.

Jenkins supports several different types of build jobs. The two most commonly-used are the freestyle builds and the Maven 2/3 builds. The freestyle projects allow you to configure just about any sort of build job, they are highly flexible and very configurable. The Maven 2/3 builds understand the Maven project structure, and can use this to let you set up Maven build jobs with less effort and a few extra features.

At the end of this tutorial, you will be able to:

  1. Create a Jenkins Job that accesses a Git repository.

Part 1 – Enable Jenkins’ Maven Plugin

1. Go to the Jenkins console:

http://localhost:8080

2. Login using your user and password.

3. Click on the Manage Jenkins link

 

4. Ignore any message regarding an update.

5. Click on Manage Plugins

 

6. Click on the Available tab.

 

7. In the filter box at the top-right of the window, enter ‘Maven Integration‘. Note: Don’t hit Return!

8. Entering the filter text will narrow down the available plugins a little. Scroll down to find the ‘Maven Integration‘ listing, and click on the check box next to it.

 

9. Click Install Without Restart.

10. Click Go back to the top page.

Part 2 – Create a Git Repository

As a distributed version control system, Git works by moving changes between different repositories. Any repository apart from the one you’re currently working in is called a “remote” repository. Git doesn’t differentiate between remote repositories that reside on different machines and remote repositories on the same machine. They’re all remote repositories as far as Git is concerned. In this lab, we’re going to start from a source tree, create a local repository in the source tree, and then clone it to a local repository. Then we’ll create a Jenkins job that pulls the source files from that remote repository. Finally, we’ll make some changes to the original files, commit them and push them to the repository, showing that Jenkins automatically picks up the changes.

1.Download the files Create A Jenkins Job and extract to C:\LabFiles\Create A Jenkins Job

2.Open to the folder C:\LabFiles\Create A Jenkins Job\SimpleGreeting.

3. Right click in the empty area and select Git Bash Here. The Git command prompt will open.

 

 

3. Enter the following command:

ls

4. Enter the following lines. Press enter after each line:

git config --global user.email "wasadmin@webagesolutions.com"
git config --global user.name "Bob Smith"

The lines above are actually part of the initial configuration of Git. Because of Git’s distributed nature, the user’s identity is included with every commit as part of the commit data. So we have to tell Git who we are before we’ll be able to commit any code.

5. Enter the following lines to actually create the Git repository:

git init
git add .
git commit -m "Initial Commit"

The above lines create a git repository in the current directory (which will be C:\LabFiles\Create a Jenkins Job\SimpleGreeting), add all the files to the current commit set (or ‘index’ in git parlance), then actually performs the commit.

6. Enter the following, to create a folder called repos under the C:\Software folder.

mkdir /c/Software/repos

7. Enter the following to clone the current Git repository into a new remote repository.

git clone --bar . /c/Software/repos/SimpleGreeting.git

At this point, we have a “remote” Git repository in the folder C:\Software\repos\SimpleGreeting.git. Jenkins will be quite happy to pull the source files for a job from this repo.

Part 3 – Configure Git

1. Go to Jenkins home.

http://localhost:8080

2. Click Manage Jenkins.

3. Click Global Tool Configuration.

4. Scroll to the Git section.

5. Make sure git path is the right one, if not fix it and save. (Your path could be different)

Part 4 – Create the Jenkins Job

1. Go to the Jenkins home.

2. Click on the New Item link.

 

 

3. Enter SimpleGreeting for the project name.

4. Select Maven Project as the project type.

5. Click OK, to add a new job.

After the job is created, you will be on the job configuration page.

6. Scroll down to the Source Code Management section and then select Git.

 

 

7. Under Repositories, enter C:\Software\repos\SimpleGreeting.git and press tab key.

 

 

8. Click Save.

9. You will see the Job screen. Click Workspace.

 

10. Click Build Now.

You should see the build in progress in the Build History area.

 

 

11. After a few seconds the build will complete, the progress bar will stop. Click on Workspace.

 

You will see that the directory is populated with the source code for our project.

 

12. Find the Build History box, and click on the ‘time’ value for the most recent build. You should see that the build was successful.

13. Click the Console Output from the left menu.

14. At the end of the console you will also see the build success and successful build finish.

You have created a project and built it successfully.

Part 5 – Enable Polling on the Repository

So far, we have created a Jenkins job that pulls a fresh copy of the source tree prior to building. But we triggered the build manually. In most cases, we would like to have the build triggered automatically whenever a developer makes changes to the source code in the version control system.

1. In the Jenkins web application, navigate to the SimpleGreeting project. You can probably find the project in the breadcrumb trail near the top of the window. Alternately, go to the Jenkins home page and then click on the project.

2. Click the Configure link.

3. Scroll down to find the Build Triggers section.

 

4. Click on the check box next to Poll SCM, and then enter ‘* * * * *’ into the Schedule text box. [Make sure there is a space between each *]

 

Note: The above schedule sets up a poll every minute. In a production scenario, that’s a higher frequency than we need, and it can cause unnecessary load on the repository server and on the Jenkins server. You’ll probably want to use a more reasonable schedule – perhaps every 15 minutes. That would be ‘H/15 * * * *’ in the schedule box.

5. Click Save.

Part 6 – Import the Project into Eclipse

In order to make changes to the source code, we’ll clone a copy of the Git repository into an Eclipse project.

1. If eclipse was closed then start Eclipse by running C:\Software\eclipse\eclipse.exe and use C:\Workspace as Workspace.

2. From the main menu, select File → Import…

3. Select Git → Projects from Git.

4. Click Next.

5. Select Clone URI and then click Next.

 

You might think that ‘Existing local repository’ would be the right choice, since we’re cloning from a folder on the same machine. Eclipse, however, expects a “local repository” to be a working directory, not a bare repository. On the other hand, Jenkins will complain if we try to get source code from a repository with a working copy. So the correct thing is to have Jenkins pull from a bare repository, and use Clone URI to have Eclipse import the project from the bare repository.

6. Click on Local File… and then navigate to C:\Software\repos\SimpleGreeting.git

7. Click OK.

8. Back in the Import Projects dialog, click Next.

9. Click Next to accept the default ‘master’ branch.

 

 

10. In the Local Destination pane, leave the defaults and click Next.

 

11. Select Import as a General Project and click Next.

 

12. Click Finish.

 

 

13. You should see the new project in the Project Explorer, expand it.

 

At this point, we could go ahead and edit the files, but Eclipse doesn’t understand the project’s layout. Let’s tell Eclipse what we know – that this project is built using Apache Maven.

14. Right-click on the SimpleGreeting project in the Project Explorer, and then select Configure → Convert to Maven Project.

 

15. After few seconds, you should now see the project represented as a Maven project in the Project Explorer.

Part 7 – Make Changes and Trigger a Build

The project that we used as a sample consists of a basic “Hello World” style application, and a unit test for that application. In this section, we’ll alter the core application so it fails the test, and then we’ll see how that failure appears in Jenkins.

1. In the Project Explorer, expand the src/main/java tree node.

 

2. Expand the com.simple package to reveal the Greeting.java file.

 

 

3. Double-click on Greeting.java to open the file.

4. Find the line that says ‘return “GOOD”;’. Edit the line to read ‘return “BAD”;’

 

5. Save the file by pressing Ctrl-S or selecting File → Save.

Now we’ve edited the local file. The way Git works is that we’ll first ‘commit’ the file to the local repository, and then we’ll ‘push’ the changes to the upstream repository. That’s the same repository that Jenkins is reading from. Eclipse has a short-cut button that will commit and push at the same time.

6. Right-click on SimpleGreeting in the Project Explorer and then select Team → Commit…

7. Enter a few words as a commit message, and then click Commit and Push.

8. Click OK in the status dialog that pops up.

9. Now, flip back to the web browser window that we had Jenkins running in. If you happen to have closed it, open a new browser window and navigate to http://localhost:8080/SimpleGreeting. After a few seconds, you should see a new build start up. You can launch a new build if it’s taking too long.

 

10. If you refresh the page, you should see that there is now a ‘Test Result Trend’ graph that shows we have a new test failure.

What happened is that we pushed the source code change to the Git repository that Jenkins is reading from. Jenkins is continually polling the repository to look for changes. When it saw that a new commit had been performed, Jenkins checked out a fresh copy of the source code and performed a build. Since Maven automatically runs the unit tests as part of a build, the unit test was run. It failed, and the failure results were logged.

Part 8 – Fix the Unit Test Failure

1. Back in eclipse, edit the file Greeting.java so that the class once again returns ‘GOOD’.

2. As above, save, commit and push the change.

3. Watch the Jenkins web browser window. After a minute or two you should see the build start automatically or you can click Build now, when the build is done then refresh the page.

 

4. Close all.

Part 9 – Review

In this tutorial, you learned

  • How to Set-up a set of distributed Git repositories

  • How to create a Jenkins Job that reads from a Git repository

  • How to configure Jenkins to build automatically on source code changes.

Version Control- Git

In this tutorial, you will learn to install, configure, and use Git on Linux.

Part 1 – Launch terminal

In this part you will launch the Linux terminal.

1. Open the Terminal window.

Alternatively, you can press Ctrl+Alt+T to access the terminal.

Part 2 – Install Git

1. Update APT repository

sudo apt-get update

Enter  password, if prompted

2. Install git.

sudo apt-get install git

Press Y, if prompted

3. In the terminal run following command to find git version number.

git –version
git help
git help -a

Part 3 – Using Git

In this part you will use Git to perform various operations, such as, check in, check status etc.

1. Switch to the “Documents” directory.

cd ~/Documents

2. Create a directory.

mkdir -p workspace/git

3. Switch to the “git” directory.

cd workspace/git

4. Initialize repository.

git init

5. Tell Git who you are.

git config --global user.name "Alice Smith"
git config --global user.email alice@smith.com

Note: One interesting aspect of Git is that it separates user identity in the repository from any sort of authentication or authorization. Because a distributed repository will generally be maintained by many separate individuals or systems, the identity of the committer must be contained in the repository – it can’t just be supplied as a user id when we do the commit. So, even if we’re not connected to any central repository, we need to tell Git who we are. The identity that we supply will be recorded whenever we commit to a repository.

6. Create a text file.

nano sample.txt

7. Enter following text.

First Version!

8. Press Ctrl+O to save the file and hit enter.

9. Press Ctrl+X to exit to the terminal.

10. Get Git status.

git status

Notice sample.txt is listed under untracked files.

11. Add the files to tracked.

git add .

Note: Here you are adding the current directory. You could also add the file using “git sample.txt”.

12. Get Git status again.

git status

Notice sample.txt is tracked.

13. Commit changes.

git commit

Notice it launched text editor automatically which lists operations that will get performed when files are committed. Here you can add detailed description that will get saved when you commit the changes.

14. Add following text in the first line.

Added sample.txt

15. Press Ctrl+O to save the file and hit enter.

16. Press Ctrl+X to exit to the terminal.

17. Get Git status.

git status

Notice it says there’s nothing to commit since you have already committed all changes.

18. Modify sample.txt.

nano sample.txt

19. Change “First Version!” to “Second Version!”

20. Press Ctrl+O to save the file and hit enter.

21. Press Ctrl+X to exit to the terminal.

22. Get Git status.

git status

Notice it says the file is modified.

23. View changes.

git diff

Notice it shows old text in red and new text in green.

24. Create another file.

nano another.txt

25. Add the following text.

Hello World!

26. Press Ctrl+O to save the file and hit enter.

27. Press Ctrl+X to exit to the terminal.

28. Add all files.

git add .

29. Get Git status.

git status

Notice it’s showing 1 file as modified and 1 file as a newly added file.

30. Commit changes.

git commit -m "Made 2 changes"

Notice when you pass -m switch, you can store a simple single line comment.

31. Get Git status.

git status

Notice there’s nothing to commit.

32. Delete sample.txt

rm sample.txt

33. Recover file.

git checkout sample.txt

34. View sample.txt

cat sample.txt

Note: It restored latest version by default.

35. Delete file again.

rm sample.txt

36. View all versions of a file.

git log

Notice it shows user, commit id, date time, and comment.

37. Copy the commit id for the older version.

38. View changes between current and the first version.

git diff <commit_id>

39. Restore the older version.

git checkout <commit_id> sample.txt

40. View file content.

cat sample.txt

Notice it’s the first version.

41. Close the terminal.

Part 4 – Review

In this tutorial,  you installed and used Git.