Variables, Secrets, and OIDC¶
Overview¶
Use CI/CD variables correctly (masked, protected, environment-scoped), avoid long-lived cloud keys where possible, and outline OIDC and Vault-style secret patterns for production.
Pipelines need configuration and credentials. GitLab provides CI/CD variables at project, group, and instance levels, plus predefined $CI_* variables. Production teams prefer short-lived cloud credentials via OpenID Connect (OIDC) and external secret managers over static keys in the UI.
This is a core tutorial in Module 6 · Variables & Secrets of the REBASH Academy GitLab CI/CD for Cloud & DevOps Engineers series — written for Cloud, DevOps, Platform, and SRE engineers.
Prerequisites¶
Learning Objectives¶
By the end of this tutorial, you will be able to:
- Choose YAML vs UI variables for non-secret config
- Apply masked and protected variable flags correctly
- Scope variables to environments
- Describe OIDC to AWS/GCP/Azure and Vault-style fetch patterns
Architecture¶
This topic’s control points and relationships are shown below.
Theory¶
What it is¶
CI/CD variables become environment variables in the job. Sources: predefined ($CI_JOB_TOKEN, $CI_REGISTRY, …), .gitlab-ci.yml variables:, and UI/API variables (optionally masked, protected, expanded, environment-scoped). Masked variables are redacted from job logs when they match masking rules. Protected variables are only available on protected branches and tags. Environments (environment:name) further scope deploy-time secrets.
OIDC lets GitLab mint a JWT for the job; the cloud provider trusts that JWT and returns temporary credentials — no static access key in GitLab. Vault (or cloud secret stores) patterns fetch secrets at job start using that identity or an authenticated agent.
Why it matters¶
Leaked long-lived keys are a top CI breach path. Feature-branch jobs that see production secrets violate least privilege. Masking is not encryption — it is log hygiene. Platform and DevSecOps standards: non-secrets in YAML, secrets in protected UI vars or external managers, cloud access via OIDC roles tied to project_path / ref claims.
How it works¶
- Prefer
$CI_*and UI/project variables for non-secret config (AWS_REGION, image names). - Store secrets in the GitLab UI (or external store); mark protected and masked when compatible.
- Restrict deploy jobs with
rules+environmentso only protected refs see production scopes. - For cloud: trust GitLab as an OIDC IdP; jobs mint a JWT via
id_tokens; STS / Workload Identity returns temporary creds. - For Vault: login with JWT/OIDC, fetch, use — never
echosecrets.
No paid GitLab or live cloud account is required to author the YAML; enable OIDC in a sandbox when ready.
Key concepts and comparisons¶
| Mechanism | Good for | Limit |
|---|---|---|
YAML variables | Non-secret defaults | Visible in Git |
| UI variable (masked) | Simple secrets | Masking rules; still stored in GitLab |
| Protected + environment | Production deploy secrets | Unprotected branches cannot read |
| OIDC to cloud | Temporary cloud API access | Needs cloud IdP config |
| Vault / secret manager | Central rotation, dynamic secrets | Extra runtime dependency |
| Anti-pattern | Prefer |
|---|---|
| Cloud access keys in YAML | OIDC role assumption |
| Same secret on all branches | Protected + environment scope |
echo $SECRET for debugging | Job traces with masking; short-lived tokens |
Common pitfalls¶
- Believing masked variables cannot be exfiltrated — a malicious job can still send them outbound.
- Unprotected runners + protected variables — understand runner privilege models.
- Forgetting
id_tokens/ audience config when migrating to OIDC. - Storing entire
.envfiles as a single variable without rotation owners.
Hands-on Lab¶
Create a workspace for this tutorial.
Focus: separate non-secrets from masked vars; shape OIDC job (file-only validation)
Step 1 – Author variables + id_tokens pipeline¶
cat > oidc-checklist.md << 'EOF'
- [ ] Non-secrets in YAML variables only
- [ ] Masked+protected secrets in CI/CD settings
- [ ] id_tokens with explicit aud
- [ ] Never echo tokens
EOF
cat > .gitlab-ci.yml << 'EOF'
stages: [verify, deploy]
variables:
APP_ENV: "ci"
show_predefined:
stage: verify
image: alpine:3.20
script:
- echo "project=$CI_PROJECT_PATH ref=$CI_COMMIT_REF_NAME"
- echo "APP_ENV=$APP_ENV"
oidc_ready_deploy:
stage: deploy
image: alpine:3.20
id_tokens:
GITLAB_OIDC_TOKEN:
aud: https://gitlab.com
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: manual
script:
- test -n "${GITLAB_OIDC_TOKEN:-}" && echo "OIDC token present (do not print)" || echo "Token only on GitLab runners"
- echo "Exchange JWT via cloud STS — not long-lived keys"
EOF
Step 2 – File-only validation¶
grep -A6 'id_tokens:' .gitlab-ci.yml
grep 'GITLAB_OIDC_TOKEN' .gitlab-ci.yml
test -f oidc-checklist.md
# No cloud API calls — structure only
Final step – Cleanup note¶
Validation¶
- Lab commands run under
~/rebash-gitlab/module-06/ - You can explain each Theory section in your own words
- You used modern tooling where it applies to this topic
- You can describe one production failure mode for this topic
Code Walkthrough¶
Production practice for Variables, Secrets, and OIDC always combines:
- Inspect before you change (status, plan, logs, dry-run)
- Prefer reversible, documented changes (Git, IaC, drop-ins, version pins)
- Capture evidence (command output, pipeline logs) for handovers
- Prefer current tools and APIs over legacy shortcuts
- Least privilege — escalate credentials only when required
Keep runbooks short enough to follow under pressure. Automate checks; keep humans for judgement.
Security Considerations¶
- Treat credentials and tokens for gitlab as privileged — never commit them
- Prefer short-lived auth (OIDC, roles, SSO) over long-lived keys
- Validate blast radius before apply/deploy/delete operations
- Restrict who can approve production changes
- Collect audit logs; limit who can read sensitive traces
Common Mistakes¶
Believing masked variables cannot be exfiltrated — a malicious job can still send them out
Validate assumptions against the Theory section and official docs before changing production.
Unprotected runners + protected variables — understand runner privilege models.
Lab shortcuts (open security groups, admin roles, skip approvals) must not ship unchanged.
Changing production without a rollback path
Always know how to revert (previous artefact, prior release, state rollback, DNS failback).
Best Practices¶
- Encode Variables, Secrets, and OIDC changes as code and review them in pull requests
- Pin versions (images, modules, actions, provider plugins)
- Separate environments with clear promotion gates
- Alert on symptoms with runbooks attached
- Destroy lab resources; tag everything with owner and expiry where possible
Troubleshooting¶
| Symptom | Likely cause | Fix |
|---|---|---|
| Auth / permission denied | Wrong identity, policy, or scope | Check caller identity, roles, and least-privilege policies |
| Timeout / no route | Network, DNS, security group, or endpoint | Trace path, DNS, and allow-lists before retrying |
| Drift / unexpected plan | Manual change or wrong state/workspace | Reconcile desired vs actual; avoid click-ops on managed resources |
| Pipeline/job red | Flaky step, cache, or missing secret | Read failing step logs; bisect recent workflow/config changes |
| Cost spike | Idle load balancer, NAT, oversized compute | Inventory billable resources; stop/delete labs promptly |
Summary¶
Variables, Secrets, and OIDC is essential for Cloud and DevOps engineers working with gitlab. Practise the lab until the inspection and change path is muscle memory, then continue the track.
Interview Questions¶
- Where should non-secret configuration live versus secret values?
- OIDC job cannot assume a cloud role — what claims and settings do you verify?
- What does masking actually guarantee in GitLab job logs?
- Why prefer OIDC over long-lived cloud access keys in CI?
- How do protected variables change merge request pipeline behaviour?
Sample answer — question 2
Verify id_tokens audience, the cloud identity provider trust policy (subject/ref/project), and that the job context may receive the token. Claim mismatches dominate.
Sample answer — question 4
OIDC issues short-lived credentials scoped by trust conditions, removing standing keys from GitLab variables. Never print the JWT.