Linux for CI/CD — The Foundation of Continuous Integration and Continuous Delivery¶

Continuous Integration and Continuous Delivery (CI/CD) pipelines automate the process of building, testing, packaging, and deploying software. Nearly every modern CI/CD platform—including Jenkins, GitHub Actions, GitLab CI, Azure DevOps, CircleCI, and Tekton—runs its build agents on Linux. Understanding Linux is essential for creating reliable pipelines, troubleshooting build failures, managing build environments, and automating software delivery. Every DevOps engineer, Cloud Architect, Platform Engineer, Site Reliability Engineer (SRE), and Automation Engineer should master Linux for CI/CD.

Learning Path¶

Linux Mastery → Module 13: Linux for DevOps → Lesson 3

Difficulty: Beginner → Advanced

Reading Time: 120 Minutes

Course Progress

Course: Linux Mastery

Module: Linux for DevOps

Lesson: 3 of 10

What You'll Learn¶

After completing this lesson, you'll be able to:

Understand Linux's role in CI/CD
Learn how CI/CD pipelines work
Configure Linux build agents
Use shell scripting in pipelines
Manage environment variables
Build and manage artifacts
Troubleshoot pipeline failures
Apply production CI/CD best practices

Prerequisites¶

Complete:

Modules 1–12
Module 13 Lessons 1–2

Why Learn Linux for CI/CD?¶

Imagine releasing software manually.

Developer

↓

Manual Build

↓

Manual Testing

↓

Manual Deployment

↓

Production

Modern software delivery:

Developer

↓

Git Push

↓

CI/CD Pipeline

↓

Linux Runner

↓

Build

↓

Test

↓

Deploy

Linux is the operating system that powers most CI/CD runners and automation servers.

What is CI/CD?¶

Continuous Integration (CI)

Automatically:

Build applications
Run tests
Validate code
Package software

Continuous Delivery (CD)

Automatically:

Deploy applications
Promote releases
Roll back failed deployments
Deliver software consistently

CI/CD Architecture¶

Developer

↓

Git Repository

↓

CI/CD Platform

↓

Linux Runner

↓

Build

↓

Test

↓

Package

↓

Deploy

Why Linux is Used¶

Linux provides:

Shell scripting
Package management
Docker support
Kubernetes support
Automation tools
Stable environments
High performance
Open-source ecosystem

Linux Build Agents¶

A build agent (runner) executes pipeline jobs.

Examples:

GitLab Runner
Jenkins Agent
GitHub Actions Runner
Azure Pipelines Agent

Typical workflow:

Pipeline

↓

Linux Runner

↓

Execute Jobs

Linux Directory Structure in Pipelines¶

Common directories:

/workspace

/build

/tmp

/home/runner

/var/lib

Many CI systems create temporary workspaces for every job.

Shell Scripting¶

Most CI/CD pipelines execute shell commands.

Example:

#!/bin/bash

echo "Building application..."

make

echo "Build completed."

Shell scripting is one of the most valuable Linux skills for DevOps engineers.

Environment Variables¶

CI/CD platforms provide environment variables.

View variables.

env

Example:

echo $HOME

echo $PATH

Pipeline-specific examples:

CI=true

BUILD_ID

GITHUB_SHA

CI_COMMIT_SHA

File Permissions¶

Scripts must be executable.

chmod +x build.sh

Execute.

./build.sh

Package Management¶

Install dependencies.

Ubuntu:

sudo apt install git

RHEL:

sudo dnf install git

Pipelines often install required tools dynamically.

Artifact Management¶

Generated artifacts include:

Binaries
Docker images
JAR files
ZIP archives
Reports

Create an archive.

tar -czf app.tar.gz build/

Process Monitoring¶

View running jobs.

ps aux

Terminate a process.

kill PID

Useful when debugging stuck pipeline jobs.

Resource Monitoring¶

CPU:

top

Memory:

free -h

Disk:

df -h

Large builds often consume significant resources.

Networking¶

Test connectivity.

ping github.com

Download dependencies.

curl

wget

Verify listening ports.

ss -tuln

Logging¶

View build logs.

cat build.log

System logs.

journalctl

Logs are the primary source of information during pipeline failures.

Common Linux Commands in Pipelines¶

Working directory.

pwd

List files.

ls

Create directories.

mkdir

Copy files.

cp

Move files.

mv

Delete files.

rm

Real Production Examples¶

Build application.

make

Run tests.

pytest

Package application.

tar -czf release.tar.gz build/

View environment variables.

env

Monitor build resources.

top

Production Perspective¶

Linux powers CI/CD platforms including:

Jenkins
GitLab CI
GitHub Actions
Azure DevOps
CircleCI
Tekton
Buildkite
Argo Workflows

Strong Linux skills are essential for designing and maintaining reliable software delivery pipelines.

Hands-on Lab¶

Task 1¶

Display environment variables.

env

Task 2¶

Create a build script.

nano build.sh

Add:

#!/bin/bash

echo "Hello CI/CD"

Make it executable.

chmod +x build.sh

Run it.

./build.sh

Task 3¶

Monitor system resources.

top

free -h

df -h

Task 4¶

Create a build artifact.

tar -czf artifact.tar.gz .

Task 5¶

Check running processes.

ps aux

Task 6¶

Display network connections.

ss -tuln

Task 7¶

Test Internet connectivity.

curl https://example.com

Task 8¶

Write a shell script that:

Creates a workspace
Copies project files
Creates a compressed archive
Displays completion status

Command Deep Dive¶

Command	Purpose	Production Example
`env`	View environment variables	Pipeline debugging
`chmod +x`	Make scripts executable	Automation
`tar`	Package artifacts	Release builds
`ps aux`	Monitor processes	Build troubleshooting
`top`	Monitor resources	Runner monitoring
`curl`	Test connectivity	Dependency downloads

Common CI/CD Mistakes¶

Mistake	Solution
Hardcoding credentials	Use secure environment variables or secrets management
Ignoring file permissions	Ensure scripts are executable
Running builds as root unnecessarily	Use least privilege
Leaving temporary files behind	Clean workspaces after builds
Ignoring resource usage	Monitor CPU, memory, and disk consumption

Production Troubleshooting Scenario¶

Scenario

A pipeline fails during the build stage.

Investigation:

df -h

Result:

Disk Usage

↓

100%

Further analysis:

du -sh /tmp/*

Old build artifacts are consuming storage.

The administrator:

Removes unnecessary temporary files
Configures automatic workspace cleanup
Sets artifact retention policies

The pipeline runs successfully.

Root cause:

Insufficient Disk Space on Linux Runner

Best Practices¶

Keep build agents clean and updated.
Use shell scripts for repeatable automation.
Monitor runner CPU, memory, and disk usage.
Store secrets securely using CI/CD secret management.
Clean temporary files after builds.
Archive only required artifacts.
Use non-root users whenever possible.
Log every critical build step for easier troubleshooting.

Common Mistakes¶

❌ Hardcoding passwords or API keys.

✅ Avoid this mistake: hardcoding passwords or API keys.

❌ Ignoring Linux file permissions.

✅ Always review Linux file permissions.

❌ Leaving old artifacts on build servers.

✅ Do not leave old artifacts on build servers.

❌ Running resource-intensive builds without monitoring.

✅ Avoid running resource-intensive builds without monitoring.

❌ Writing large, difficult-to-maintain shell scripts.

✅ Avoid this mistake: writing large, difficult-to-maintain shell scripts.

Interview Questions¶

Beginner¶

Why is Linux widely used for CI/CD?
What is a build agent?
How do you make a script executable?
Which command displays environment variables?

Intermediate¶

How would you troubleshoot a failed Linux pipeline?
Why are environment variables important in CI/CD?
How do you package build artifacts?
Which Linux resources should be monitored during builds?

Architect Level¶

How would you design scalable Linux-based build runners?
How would you secure CI/CD pipelines running on Linux?
How would you optimize Linux runners for thousands of daily pipeline executions?

Summary¶

In this lesson, you learned:

Linux's role in CI/CD
Build agents and runners
Shell scripting
Environment variables
Package management
Artifact management
Resource monitoring
Production CI/CD best practices

Linux is the foundation of modern CI/CD automation. Nearly every build runner, deployment agent, and automation platform relies on Linux to execute pipelines, manage resources, and deliver software efficiently. Mastering Linux enables you to build reliable, secure, and scalable CI/CD pipelines for enterprise environments.

Key Takeaways¶

Linux powers most modern CI/CD platforms.
Shell scripting is a fundamental DevOps skill.
Build runners depend on Linux resource management.
Monitor CPU, memory, disk, and network during pipeline execution.
Use secure environment variables and proper file permissions.
Keep build environments clean, reproducible, and automated.

What's Next?¶

Linux for Git — Version Control in Linux Environments

You'll explore:

Git on Linux
Repository management
SSH authentication
Git configuration
Branching workflows
Linux file permissions with Git
Production Git best practices

By the end of the lesson, you'll understand how Linux and Git work together to support modern software development, collaboration, and DevOps workflows.