Previously on Jenkins Basics post we have been through the compressive illustration using the Freestyle project. However, we have several other ways of creating and configuring a job. The Pipeline is one of them. Going forward we will see a complete pipeline setup with the configuration type Pipeline.

Pipeline vs Freestyle

In Jenkins, a Freestyle project and a Pipeline are two different ways of creating and configuring a job.

A Freestyle project is a traditional type of job configuration in Jenkins, where you can configure various build steps, triggers, and post-build actions using a graphical user interface (GUI). It is a more flexible and customizable way of setting up your build process, allowing you to define each step in detail and providing more control over the job.

On the other hand, a Pipeline allows you to define your build process using code written in a Groovy-based domain-specific language (DSL). Pipelines are more structured and organized and provide better visibility into the entire build process. Pipelines also enable you to version control your build process, which makes it easier to manage changes and collaborate with other developers.

Overall, while a Freestyle project offers more flexibility and customization options, a Pipeline provides a more structured and scalable way of defining and managing your build process.

Types of Pipeline

There are 2 types of Pipelines available which are:

Declarative Pipeline

A Declarative Pipeline is a more opinionated syntax that provides a simpler way of defining your build process using a structured format. The syntax is based on a predefined set of directives and steps, which allows you to focus more on the high-level goals of your build process rather than the underlying implementation details. Declarative pipelines are best suited for simple to medium complexity builds, and provide better readability and maintainability.

Scripted Pipeline

A Scripted Pipeline is a more flexible way of defining your build process using a Groovy-based script. It provides more low-level control over your build process, allowing you to define your build steps in a more granular way. Scripted pipelines are best suited for complex builds, where you need more control and flexibility over the build process.

Both types of pipelines provide the ability to define a complete build process, including checkout, build, test, and deployment steps. They also allow you to define custom functions and share them across multiple builds. In general, Declarative pipelines are easier to read and maintain, while Scripted pipelines offer more flexibility and control.

Declarative Pipeline Illustration with Example

1.Using the 'Pipeline Script' option

Declarative Pipeline is a newer way of defining pipelines in comparison to scripted. Jenkins's declarative way of configuration provides a more structured and simplified syntax for defining the pipeline. It aims to provide a more "declarative" way of defining pipelines, where the focus is on what you want to achieve, rather than how to achieve it.

Below is a code snippet that will create an initial pipeline that further brings the code available on the git repository and will just echo the messages in all stages:

pipeline {
    agent any

    stages {
        stage('Code') {
            steps {

                git url: 'https://github.com/rkn1999/react_django_todo_app.git' , branch: 'main'
            }
        }
        stage('Build') {
            steps {
                sh 'docker build . -t ritesh1999/django-todo-cicd:latest'
            }
        }
        stage('Docker Push') {
            steps {

                withCredentials([usernamePassword(credentialsId: 'dockerHub', usernameVariable: 'dockerHubUser', passwordVariable: 'dockerHubPassword')]){
                sh "docker login -u ${env.dockerHubUser} -p ${env.dockerHubPassword}"
                sh 'docker push ritesh1999/django-todo-cicd:latest'
                }
            }
        }
        stage('Test'){
            steps {
                echo 'Testing the build'
            }
        }
        stage('Deploy'){
            steps {
                sh 'docker-compose down && docker-compose up -d'
            }
        }
    }
}

Code Explanation

Code: This stage pulls the codebase from the specified Git repository(https://github.com/rkn1999/react_django_todo_app.git( and branch(main).
Build: This stage builds the codebase, but in this script, it simply prints a message to the console indicating that the code is being built.
The withCredentials step is now wrapped in a steps block and contains a body that includes the sh steps that use the retrieved credentials.

Here's what this pipeline does:
1. The pipeline checks out the code from the specified git repository in the Code stage.
2. The pipeline builds a Docker image in the Build stage.
3. The pipeline logs in to Docker Hub using the specified credentials in the Docker Push stage.
4. The pipeline pushes the Docker image to Docker Hub in the Docker Push stage.
Test: This stage tests the built codebase, but in this script, it simply prints a message to the console indicating that the code is being tested.
Deploy: stage('Deploy') block is defining a deployment stage in the pipeline, which likely follows a build stage that produces a Docker image.

The steps block contains a single sh step, which is executing a shell command to bring down any existing containers defined in the docker-compose.yml file, and then bring up new containers in detached mode (-d) using the same file.

Overall, this code block is likely used to deploy a Dockerized application using Docker Compose.

NB: Whenever you are writing any shell script in Pipeline Script make sure the shell script begins with 'sh' just like : sh 'docker-compose down && docker-compose up -d' because it will make Jenkins aware that it's a shell script, not a regular Groovy script.

The "agent any" line at the top of the script specifies that the pipeline can be run on any available Jenkins agent.

Once the script is ready and you build it, the pipeline will look something like the below:

We can also see the logs right here by clicking on each stage whereas previously on freestyle we had to see the console:

Logs and Description

Started by user Ritesh Kumar Nayak
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/Django_todo_app_pipeline
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Code)
[Pipeline] git
The recommended git tool is: NONE
No credentials specified
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/Django_todo_app_pipeline/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/rkn1999/react_django_todo_app.git # timeout=10
Fetching upstream changes from https://github.com/rkn1999/react_django_todo_app.git
 > git --version # timeout=10
 > git --version # 'git version 2.34.1'
 > git fetch --tags --force --progress -- https://github.com/rkn1999/react_django_todo_app.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse refs/remotes/origin/main^{commit} # timeout=10
Checking out Revision 7aaa4f7f987d4603c5ccd1f2f9f1697b31ad6a82 (refs/remotes/origin/main)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 7aaa4f7f987d4603c5ccd1f2f9f1697b31ad6a82 # timeout=10
 > git branch -a -v --no-abbrev # timeout=10
 > git branch -D main # timeout=10
 > git checkout -b main 7aaa4f7f987d4603c5ccd1f2f9f1697b31ad6a82 # timeout=10
Commit message: "Merge pull request #7 from rkn1999/rkn1999-patch-5"
 > git rev-list --no-walk c3a8da920815b07733417da336fee4e090a46a46 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] sh
+ docker build . -t ritesh1999/django-todo-cicd:latest
Sending build context to Docker daemon  2.684MB

Step 1/6 : FROM python:3.9
 ---> 5407c5455ab9
Step 2/6 : WORKDIR app
 ---> Using cache
 ---> 21e15e2927b9
Step 3/6 : COPY . /app
 ---> 78cffaf129ae
Step 4/6 : RUN pip install -r requirements.txt
 ---> Running in e10b871fa2f2
Collecting asgiref==3.2.3
  Downloading asgiref-3.2.3-py2.py3-none-any.whl (18 kB)
Collecting Django==3.0.3
  Downloading Django-3.0.3-py3-none-any.whl (7.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.5/7.5 MB 32.9 MB/s eta 0:00:00
Collecting django-cors-headers==3.2.1
  Downloading django_cors_headers-3.2.1-py3-none-any.whl (14 kB)
Collecting djangorestframework==3.11.0
  Downloading djangorestframework-3.11.0-py3-none-any.whl (911 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 911.2/911.2 KB 47.1 MB/s eta 0:00:00
Collecting pytz==2019.3
  Downloading pytz-2019.3-py2.py3-none-any.whl (509 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 509.2/509.2 KB 48.6 MB/s eta 0:00:00
Collecting sqlparse==0.3.0
  Downloading sqlparse-0.3.0-py2.py3-none-any.whl (39 kB)
Installing collected packages: pytz, asgiref, sqlparse, Django, djangorestframework, django-cors-headers
Successfully installed Django-3.0.3 asgiref-3.2.3 django-cors-headers-3.2.1 djangorestframework-3.11.0 pytz-2019.3 sqlparse-0.3.0
[91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[0m[91mWARNING: You are using pip version 22.0.4; however, version 23.0.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
[0mRemoving intermediate container e10b871fa2f2
 ---> 4ae50e119375
Step 5/6 : EXPOSE 8001
 ---> Running in 3fe6f35096ab
Removing intermediate container 3fe6f35096ab
 ---> 56e19e287bb0
Step 6/6 : CMD ["python","manage.py","runserver","0.0.0.0:8001"]
 ---> Running in a1d864ebc472
Removing intermediate container a1d864ebc472
 ---> 240404b29313
Successfully built 240404b29313
Successfully tagged ritesh1999/django-todo-cicd:latest
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Docker Push)
[Pipeline] withCredentials
Masking supported pattern matches of $dockerHubPassword
[Pipeline] {
[Pipeline] sh
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
         Affected argument(s) used the following variable(s): [dockerHubPassword]
         See https://jenkins.io/redirect/groovy-string-interpolation for details.
+ docker login -u ritesh1999 -p ****
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /var/lib/jenkins/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[Pipeline] sh
+ docker push ritesh1999/django-todo-cicd:latest
The push refers to repository [docker.io/ritesh1999/django-todo-cicd]
c029b5aad2c5: Preparing
ce53e9e2be1b: Preparing
49033949e2e9: Preparing
ec085b3135eb: Preparing
f4e0f062ac99: Preparing
91cb66c337c1: Preparing
0baa05faf31a: Preparing
ac504d030134: Preparing
52ebb9a789db: Preparing
86d774dafc20: Preparing
da68f13652fc: Preparing
cf2e8433dbf2: Preparing
91cb66c337c1: Waiting
0baa05faf31a: Waiting
ac504d030134: Waiting
52ebb9a789db: Waiting
86d774dafc20: Waiting
da68f13652fc: Waiting
cf2e8433dbf2: Waiting
49033949e2e9: Layer already exists
ec085b3135eb: Layer already exists
f4e0f062ac99: Layer already exists
91cb66c337c1: Layer already exists
ac504d030134: Layer already exists
0baa05faf31a: Layer already exists
52ebb9a789db: Layer already exists
86d774dafc20: Layer already exists
da68f13652fc: Layer already exists
cf2e8433dbf2: Layer already exists
ce53e9e2be1b: Pushed
c029b5aad2c5: Pushed
latest: digest: sha256:c51d8e799d010720b72ee16dc7e31de47e137c772fa878435977928a64f63147 size: 2847
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] echo
Testing the build
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Deploy)
[Pipeline] sh
+ docker-compose down
Removing network django_todo_app_pipeline_default
Network django_todo_app_pipeline_default not found.
+ docker-compose up -d
Creating network "django_todo_app_pipeline_default" with the default driver
Creating django_todo_app_pipeline_web_1 ... 
Creating django_todo_app_pipeline_web_1 ... done
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

On the above log you can see that login to the docker hub is successful using the credentials we defined in Jenkins Credentials(Described in detail in the below section 'Jenkins Credentials') and the docker image has been pushed to the docker Hub being authenticated by my docker hub username and password.

When docker-compose up got executed, it again fetched the docker image from the docker hub and made the container up and running.

2.Using the 'Pipeline script from SCM' option

The pipeline script from SCM(Source Code Management) option does nothing but fetches the same Jenkins script from the Git repository.

The same Jenkins Groovy script will be pushed to the Git repo and then will be reused again and again to build the pipeline which is industry best practice.

SCM Configuration for Jenkinsfile

Below is the SCM configuration where we have given our git repo URL(https://github.com/rkn1999/react_django_todo_app.git) and branch name */main and Script path as Jenkinsfile which is the name of the Jenkins file itself.

Once we save the above configuration, Jenkins will point to the git repo to build the pipeline which will look something like the one below where you can see the "Declarative: Checkout SCM" column which means the pipeline is built from the SCM. And this is the ideal way which is used in Production.

App Deployed by Declarative Pipeline

After the pipeline has run successfully after the deployment step the app is now up and running on port number 8001 as defined in the docker-compose.yaml

Scripted Pipeline with Code

Scripted Pipeline is an older way of defining pipelines in Jenkins that provides more flexibility and control over the pipeline definition. It allows you to define the pipeline as a series of Groovy scripts, where you have full access to the Jenkins API and can define custom functions and logic.

NB: We are not using Scripted Pipeline much, so we won't go indept here. Our focus is on Declarative Pipeline.

Here's an example of a simple Scripted Pipeline:

node {
    stage('Build') {
        git url: 'https://github.com/rkn1999/java_todo_app.git' , branch: 'main'
    }
    stage('Test') {
        sh 'mvn test'
    }
    stage('Deploy') {
        sh 'mvn deploy'
    }
}

In the Build stage, the script uses the Git plugin to clone the source code repository of a Java todo application from the 'main' branch.

In the Test stage, the script runs the 'mvn test' command, which suggests that the project is a Maven project and this stage is used for running tests.

In the Deploy stage, the script runs the 'mvn deploy' command, which suggests that the project is also using Maven for deployment. However, the specific deployment steps would depend on the configuration and requirements of the project.

Overall, this pipeline script automates the build, testing, and deployment process for the given application, enabling faster and more consistent software delivery.

Jenkins Credentials

Jenkins credentials are used to securely store sensitive information, such as passwords, access tokens, SSH keys, and other secrets, that are required by Jenkins jobs or pipeline scripts.

Credentials can be used in various ways, such as:

Authenticating with external systems or services, such as Git repositories, Docker registries, or cloud providers.
Connecting to remote hosts via SSH or other protocols.
Providing authentication credentials for a Jenkins job or pipeline script to run as a specific user or service account.
Accessing secure configuration files, such as Maven or Gradle settings.xml files, that contain sensitive information.

Jenkins provides a wide range of credential types, including username and password, SSH username with the private key, secret text, and others. Credentials can be managed in Jenkins by going to the "Credentials" section in the Jenkins dashboard, where you can create, edit, delete, and organize them.

Here, we have added my Docker Hub credentials which will further be used in pushing the docker image to the docker hub. These credentials can also be used for logging in to Git and other external platforms if needed as discussed above.

Agent

What & Why 'Agents' in Jenkins?

In Jenkins, an agent is a machine or environment that can be used to execute builds or jobs. It can be a physical or virtual machine, a container, or even a cloud-based environment. Agents are essential in Jenkins because they allow for distributed builds, enabling a single Jenkins master to delegate work to multiple agents. This can greatly increase the speed of build execution and enable the parallelization of tasks.

Master will control the jobs by allocating the Jobs to different agents.

In order to establish the connection between Jenkins master and agents, agent servers need to have Java installed(as Jenkins is based on Java) and Jenkins user created in them.

Use case of Agents in Jenkins

Suppose you have a build job that takes a long time to execute, and you want to run it faster by running the work across multiple agents simultaneously. You can configure the job to use multiple agents, and Jenkins will distribute the work across them. For example, you can split the build job into three stages, and configure each stage to run on a different agent. This can greatly reduce the overall execution time of the job.
Suppose you have a Jenkins setup with a master node and three agent nodes - one on a physical machine, one on a virtual machine, and one on a cloud environment. You have a build job that requires a specific environment to run, say a Windows machine with a certain set of tools installed. You can configure the build job to run on an agent that meets these requirements.

To summarize, agents in Jenkins are machines or environments that can be used to execute builds or jobs. They are essential in enabling distributed builds and parallelization of tasks, which can greatly increase the speed and efficiency of build execution.

Advantages of implementing Jenkins Agents

Distributed Builds: Agents enable Jenkins to distribute build tasks across multiple machines or environments. This allows builds to be executed in parallel, greatly reducing the overall build time.
Scalability: With agents, Jenkins can easily scale up or down to meet the demands of different build tasks. For example, if a build requires a specific environment or toolset, an agent can be configured with those requirements and used only when needed.
Flexibility: Agents provide flexibility in terms of where and how builds can be executed. They can be physical or virtual machines, containers, or cloud-based environments. This allows Jenkins to execute builds in a variety of scenarios and environments.
Load Balancing: Jenkins can use agents to balance the load of build tasks across multiple machines or environments. This can help prevent overload and ensure that builds are executed efficiently.
Security: By using agents, Jenkins can ensure that builds are executed in isolated environments, reducing the risk of security vulnerabilities and ensuring that each build runs in a clean environment.

Setting up Agents in Jenkins

Step-1(Lunching Multiple EC2 Instances)

To set up Jenkins Agents, you need multiple EC2 instances to be connected to each other through SSH connection. So the first step will be to lunch multiple EC2 instances apart from Jenkins Master with the required amount of storage.

Step-2(SSH Key Generation)

To establish the connection between the master and agent through SSH, 2 keys such as the Private Key and the Public Key will be required. The Public Key will be given to the agent instance and the Private key will be used by Master Instance.

To generate the keys in the Master use ssh-keygen command which will generate the key pair as below:

In the above image, you can see that the key has been generated and saved in /home/ubuntu/.ssh .

By getting into the .ssh you can see the keys named as id_rsa and id_rsa.pub

The above one is the public key that we can see using cat id_rsa.pub .

Here id_rsa is the private key and id_rsa.pub is the public key. Now, this public key has to be given to the Agent server.

To do so, move to the /home/ubuntu/.ssh in the Agent server and there you will be seeing a file called authorized_keys as below:

And we have to edit that file with vim authorized_keys and paste the public key from the master node(id_rsa.pub) and save as below so that it can use this key while connecting to Master:

Step-3(Connecting using SSH)

NB: The Agent can only connect to the Master and it can't be vice-versa.

So, to connect the Agent to the Master we will be using the command sudo ssh ubuntu@ip-172-31-12-88

which simply means we are trying to establish a connection to the server ubuntu@ip-172-31-12-88

And it's normal to get the permission denied error for the first time.

Until now we were just trying to set up the connection between 2 EC2 instances however, our main goal is to set up an agent in Jenkins. Going forward we will be establishing the same connection from Jenkins.

Step-4(Creating Agent in Jenkins)

To set up an agent in Jenkins, go to Manage Jenkins and click on Set up agent:

Give a name to the and proceed further:

You can give any name and description as per your project.

The number of Executors: The number of executors on a Jenkins agent determines the number of jobs that can be run concurrently on that agent.

Remote Root Directory: In Jenkins, a remote root directory refers to the directory on the agent machine where Jenkins will execute the build. When you set up a Jenkins agent, you can specify a remote root directory on the agent machine where the build will be run. This directory will be used by Jenkins to store temporary files, logs, and other build artifacts during the build process.

Labels: They are used to organize and categorize agents based on their capabilities or attributes. For example, if you have a group of agents that have a specific set of tools or software installed, you can assign a label to that group to easily identify and select them for a particular job.

Usage:

Use this node as much as possible: This is the default setting.
In this mode, Jenkins uses this node freely. Whenever there is a build that can be done by using this node, Jenkins will use it.
Only build jobs with label expressions matching this node: In this mode, Jenkins will only build a project on this node when that project is restricted to certain nodes using a label expression, and that expression matches this node's name and/or labels.

Lunch method: this defines the way we want to connect the agent. For now, it's via SSH

Credentials: to lunch through SSH we need SSH keys so we have to add the Credentials as described below in the "Adding Credentials for SSH" section. Once you add the credentials to Jenkins it will be shown as an option to choose:

And we have choosen the highland one as our SSH key.

Availability: it defines when should the agent needs to be active and when to be offline. Here we have chosen to keep this online as much as possible.

Adding Credentials for SSH:

Kind: Here we have chosen kind as SSH Username with a private key because we will be using connecting the Agent to the Master with SSH.

Private Key: In the private key section we have to give the private key of the Master server because this Jenkins is hosted in the Master itself. We can't expose other servers' private keys in another server.

And then click on Add to the Credentials will be added.

Step-5(Agent Created & Installing Java in Agent)

After following all the above steps correctly you can see the agent has been created as below where we have Jenkins-Agent-1 created and listed with Built-in Node:

However currently, you can see the Jenkins-Agent-1 is not yet connected because to connect through Jenkins we need Java to be installed which is not yet done in the Agent server. In the logs also we can see :

So to Proceed further we have to install the exact version of Java in the Agent server with the command sudo apt install openjdk-11-jre .

Step-6(Establishing Connection)

Once Java is installed in the Agent server, you can go to the agent and click on Lunch Agent. Then the agent will start connecting and the logs will look like the below which shows that Agent successfully connected and online . You can go through the log to see the impact after Java installation:

SSHLauncher{host='54.237.66.8', port=22, credentialsId='Agent1-SSHKey', jvmOptions='', javaPath='', prefixStartSlaveCmd='', suffixStartSlaveCmd='', launchTimeoutSeconds=60, maxNumRetries=10, retryWaitTime=15, sshHostKeyVerificationStrategy=hudson.plugins.sshslaves.verifiers.NonVerifyingKeyVerificationStrategy, tcpNoDelay=true, trackCredentials=true}
[03/26/23 14:31:47] [SSH] Opening SSH connection to 54.237.66.8:22.
[03/26/23 14:31:47] [SSH] WARNING: SSH Host Keys are not being verified. Man-in-the-middle attacks may be possible against this connection.
[03/26/23 14:31:47] [SSH] Authentication successful.
[03/26/23 14:31:48] [SSH] The remote user's environment is:
BASH=/usr/bin/bash
BASHOPTS=checkwinsize:cmdhist:complete_fullquote:extquote:force_fignore:globasciiranges:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath
BASH_ALIASES=()
BASH_ARGC=([0]="0")
BASH_ARGV=()
BASH_CMDS=()
BASH_EXECUTION_STRING=set
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="5" [1]="1" [2]="16" [3]="1" [4]="release" [5]="x86_64-pc-linux-gnu")
BASH_VERSION='5.1.16(1)-release'
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
DIRSTACK=()
EUID=1000
GROUPS=()
HOME=/home/ubuntu
HOSTNAME=ip-172-31-50-64
HOSTTYPE=x86_64
IFS=$' \t\n'
LANG=C.UTF-8
LOGNAME=ubuntu
MACHTYPE=x86_64-pc-linux-gnu
MOTD_SHOWN=pam
OPTERR=1
OPTIND=1
OSTYPE=linux-gnu
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
PIPESTATUS=([0]="0")
PPID=16891
PS4='+ '
PWD=/home/ubuntu
SHELL=/bin/bash
SHELLOPTS=braceexpand:hashall:interactive-comments
SHLVL=1
SSH_CLIENT='44.197.212.28 42726 22'
SSH_CONNECTION='44.197.212.28 42726 172.31.50.64 22'
TERM=dumb
UID=1000
USER=ubuntu
XDG_RUNTIME_DIR=/run/user/1000
XDG_SESSION_CLASS=user
XDG_SESSION_ID=10
XDG_SESSION_TYPE=tty
_=']'
Checking Java version in the PATH
openjdk version "11.0.18" 2023-01-17
OpenJDK Runtime Environment (build 11.0.18+10-post-Ubuntu-0ubuntu122.04)
OpenJDK 64-Bit Server VM (build 11.0.18+10-post-Ubuntu-0ubuntu122.04, mixed mode, sharing)
[03/26/23 14:31:48] [SSH] Checking java version of home/ubuntu/jenkins-file/jdk/bin/java
Couldn't figure out the Java version of home/ubuntu/jenkins-file/jdk/bin/java
bash: line 1: home/ubuntu/jenkins-file/jdk/bin/java: No such file or directory

[03/26/23 14:31:48] [SSH] Checking java version of java
[03/26/23 14:31:48] [SSH] java -version returned 11.0.18.
[03/26/23 14:31:48] [SSH] Starting sftp client.
[03/26/23 14:31:48] [SSH] Remote file system root home/ubuntu/jenkins-file does not exist. Will try to create it...
[03/26/23 14:31:48] [SSH] Copying latest remoting.jar...
[03/26/23 14:31:48] [SSH] Copied 1,370,647 bytes.
Expanded the channel window size to 4MB
[03/26/23 14:31:48] [SSH] Starting agent process: cd "home/ubuntu/jenkins-file" && java  -jar remoting.jar -workDir home/ubuntu/jenkins-file -jar-cache home/ubuntu/jenkins-file/remoting/jarCache
Mar 26, 2023 2:31:49 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
INFO: Using home/ubuntu/jenkins-file/remoting as a remoting work directory
Mar 26, 2023 2:31:49 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging
INFO: Both error and output logs will be printed to home/ubuntu/jenkins-file/remoting
<===[JENKINS REMOTING CAPACITY]===>channel started
Remoting version: 3107.v665000b_51092
Launcher: SSHLauncher
Communication Protocol: Standard in/out
This is a Unix agent
NOTE: Relative remote path resolved to: /home/ubuntu/home/ubuntu/jenkins-file/home/ubuntu/jenkins-file
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jenkins.slaves.StandardOutputSwapper$ChannelSwapper to constructor java.io.FileDescriptor(int)
WARNING: Please consider reporting this to the maintainers of jenkins.slaves.StandardOutputSwapper$ChannelSwapper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Evacuated stdout
Agent successfully connected and online

Step-7(Agent Connected)

Now, in the above image, you can see that the Jenkins-Agent-1 which was offline previously is now connected and In Sync.

Deployment through Agent

As you know previously we have created two types of Jenkins Jobs:

Via Freestyle Project
Pipeline

Now we'll see how to configure both the types with Jenkins Agent and run the job on Agent.

Prerequisite for Agent server

Install docker in the agent so that it can identify docker while containerizing by docker file.
Install docker-compose in the agent so that it can identify docker-compose while running by docker-compose.yaml file.
Add docker as a user by using : sudo usermod -a -G docker $USER

1-Deployment of Freestyle Project to Jenkins-Agent

While working with the Freestyle project it's easier to configure with an agent.

To do so, select "Restrict where this project can be run" and add the Label Expression that was created for the agent during Agent creation as shown below:

Once we save the configuration, in the executor status at the left below corner you can see we have one Built-In Node and JenkinsAgent-1. However, under JenkinsAgent-1 there is 1 Idle which means on the agent one pipeline is tagged to run on JenkinsAgent-1.

Now, when we build the project by clicking on the Build Play button or by Git Push it will run the Job on the tagged server as below. Here as our Pipeline is tagged to JenkinsAgent-1 it is now running under the same:

Now, on the console output as well you can see that it is Building on JenkinsAgent-1 remotely:

Also the Agent Server you can see that the docker containers are up and running as below:

The app from Freestyle Project is Deployed in Jenkins-Agent:

2-Deployment of Declarative Pipeline Project to Jenkins-Agent

Now, running any declarative pipeline in Jenkins Agent is much easier. Here we only have to change the agent any to agent {label 'For-Django-App'} as below:

NB: Similarly, if you are fetching the Jenkins File from SCM then you can make the changes in JenkinsFile in Git and fetch it from there.

Now below you can see again it's running on JenkinsAgent-1

Now you can see this is the Declarative Pipeline project Django_TODO console where it is running on JenkinsAgent-1

Below you can see the Container is Up and Running about a minute:

The app from Declarative Pipeline is Deployed in Jenkins-Agent:

Conclusion

Here we have completed Jenkins E2E.

We have covered Freestyle Project vs Declarative Project

Jenkins as Code

We have implemented Jenkins Agents to run the Jobs Remotely on a different Server and many more.

Thanks For your Patience!!

Hope it Helps !! :)

Command Palette

Comments (1)

More from this blog