KAHIBARO Bookshelves
Discord Login Register

18.1. Typical OpenShift workflows

From Idea to Running Application: A Day-in-the-Life View

This chapter focuses on how OpenShift is typically used in practice: the concrete sequences of steps teams follow day-to-day. Think of these as “recipes” or “patterns” that appear again and again, regardless of the specific application or organization.

We’ll look at common workflows from several perspectives:

  • Developer inner loop
  • Application onboarding into OpenShift
  • Configuration and secret changes
  • Deploying across environments (dev → test → prod)
  • Handling incidents and rollbacks
  • Scaling and maintenance
  • HPC-/batch-oriented workflows on OpenShift

Developer Inner Loop on OpenShift

The “inner loop” is what a developer does repeatedly during daily work: write code, run it, test it, and iterate. In OpenShift environments, this usually follows one of a few patterns.

Pattern 1: Source-to-Image (S2I) Driven Loop

Typical when the platform team encourages S2I and Git-based builds:

  1. Initial setup
    • Developer gets access to a project (namespace).
    • A BuildConfig and Deployment (or DeploymentConfig) are prepared, often via templates or GitOps.
  2. Code changes
    • Developer edits code locally.
    • Changes are pushed to a Git repository branch.
  3. Automatic build and image creation
    • A Git webhook triggers an OpenShift Build (BuildConfig).
    • S2I pulls the source, combines it with a builder image, and produces a new runtime image.
    • The image is pushed into the internal registry.
  4. Automatic deployment
    • The associated Deployment/DeploymentConfig detects the new image.
    • A rollout of the new application version occurs (usually rolling).
  5. Verification
    • Developer accesses the application via route/URL.
    • Uses OpenShift web console or oc to:
      • Inspect logs: oc logs
      • Check pod health: oc get pods
      • View events: oc describe pod ...
  6. Iterate
    • If the change is good: commit and push.
    • If there’s an issue: fix code, push again, repeat the build-rollout cycle.

Where CI/CD pipelines are used, some of these steps shift into Tekton or external CI tools, but the observable workflow (push → build → deploy → test) stays similar.

Pattern 2: Container-First Loop (Local Build, Cluster Run)

Common when teams prefer Docker/podman builds or need custom images:

  1. Developer builds and tests container images locally (docker build or podman build).
  2. Image is pushed to a registry that OpenShift can access.
  3. OpenShift Deployment/DeploymentConfig is updated to use the new image tag (manually or via automation).
  4. Rollout is triggered and verified.
  5. Developer iterates on both Dockerfile and code.

This pattern is also common in HPC workloads where custom MPI/GPU images are curated by specialized teams.

Onboarding an Application to OpenShift

When an existing (non-containerized or external) application is moved onto OpenShift, teams usually follow a structured onboarding workflow.

Step 1: Assess and Containerize

Typical tasks (often outside the cluster at first):

  • Identify app dependencies (language runtime, database, external services).
  • Define Dockerfile or choose an S2I builder.
  • Establish how configuration, secrets, and logging will be injected in-cluster.

Outcome: a runnable container image and a basic understanding of runtime requirements (ports, resources, storage).

Step 2: Define OpenShift Resources

In practice, this usually becomes a set of YAML definitions, often version-controlled:

  • Deployment/DeploymentConfig for pods and replicas.
  • Service for internal access.
  • Route/Ingress for external access.
  • ConfigMap and Secret stubs for configuration.
  • Optional PersistentVolumeClaim and StorageClass references for stateful needs.

Teams often create a “starter” YAML for new apps, or use templates/Helm/Kustomize or GitOps repositories.

Step 3: Create a Project and Apply Manifests

Typical sequence:

  1. Create or select the project (namespace) where the app will live.
  2. Apply manifests:
    • oc apply -f app.yaml
  3. Check object status:
    • oc get deployment
    • oc get pods
  4. Verify routing:
    • Confirm the app is reachable via its Route URL.

Step 4: Integrate with CI/CD

Most organizations standardize on:

  • A pipeline that:
    • Builds or triggers image build.
    • Runs tests.
    • Updates manifests or tags.
    • Deploys to a pre-production project.
  • Optionally, GitOps for promotion to higher environments.

The onboarding workflow typically ends when:

  • The app’s full lifecycle (build, test, deploy, update, rollback) is automated and repeatable on OpenShift.

Configuration and Secrets Change Workflow

Configuration and secret management changes are frequent and must be low-risk. Typical patterns:

Config Change (Non-Secret)

  1. Edit configuration values
    • Edit ConfigMap manifest in Git.
    • Or use oc edit configmap ... (less favored in GitOps setups).
  2. Apply updated ConfigMap
    • oc apply -f configmap.yaml
  3. Trigger config reload
    • If app reads config on start:
      • Restart pods (e.g., oc rollout restart deployment/myapp).
    • If app supports live reload:
      • It may watch mounted config files directly.
  4. Validate
    • Check pods status and logs.
    • Run basic functional checks.

In GitOps, the change is normally only made in Git, and an Operator reconciles it onto the cluster.

Secret Change (Credentials, Tokens, Keys)

  1. Change secrets in a secure system (vault, secret manager, or encrypted Git).
  2. Secret is synchronized or updated in OpenShift:
    • oc apply -f secret.yaml
  3. Restart or rollout application so it picks up new values (if not hot-reload capable).
  4. Confirm:
    • Connections still succeed (databases, APIs).
    • No authentication failures in logs.

A common rule is: never patch secrets directly from local files for production; use approved, audited workflows.

Multi-Environment Promotion Workflow

Applications rarely exist only in one project. A typical path is:

  • myapp-dev → myapp-test → myapp-prod

Different teams customize this, but the common elements are:

Separation of Concerns

  • Each environment lives in its own OpenShift project(s).
  • Shared patterns:
    • Config and secrets differ per environment.
    • Resource requests/limits and scaling policies are higher in test/prod.

Promotion Approaches

1. Image Tag Promotion

  1. Build image once (e.g., in dev pipeline).
  2. Test in dev.
  3. Promote by retagging image in a registry:
    • e.g., myapp:1.2.3-dev → myapp:1.2.3-test → myapp:1.2.3-prod.
  4. Env-specific deployments are wired to use different tags.

Benefits:

  • Single build artifact.
  • Clear traceability between environments.

2. GitOps-Based Promotion

  1. Separate Git repos or branches per environment.
  2. A successful dev/test result triggers:
    • A pull request to update manifests (or image tags) in the next environment’s repo/branch.
  3. A GitOps Operator watches the repo and applies changes to the respective cluster/namespace.
  4. Auditing and approvals happen via Git workflows.

This is increasingly common for OpenShift deployments.

Incident, Debug, and Rollback Workflows

When something goes wrong, teams follow repeatable operational workflows.

Observed Problem

  • Alert fires (from monitoring).
  • User reports high latency or errors.
  • A deployment triggers failing health checks.

Triage Workflow

Common initial steps:

  1. Identify failing components
    • oc get pods and filter by STATUS.
    • Check events: oc describe pod ....
  2. Gather logs and metrics
    • Application logs: oc logs (with -p for previous container).
    • Metrics and dashboards (from built-in monitoring stack).
    • Logs stack (e.g., Loki/Elastic) for cluster-wide logs.
  3. Reproduce issue
    • Use the app’s route URL.
    • For backend services, use oc rsh into a pod to run curl or other test clients.

Common Debug Actions

  • Check environment-specific configuration via ConfigMap and Secret.
  • Confirm correct image tag and version in Deployment.
  • Inspect recent deployments:
    • oc rollout history deployment/myapp
  • Compare working vs broken environment or version.

Rollback Workflow

Depending on the deployment mechanism:

Rolling Back a Deployment/DeploymentConfig

  1. Check rollout history.
  2. Roll back to a previous version:
    • For Deployment: update image tag, or use oc rollout undo if using native Kubernetes semantics.
    • For DeploymentConfig: oc rollout undo dc/myapp.
  3. Monitor the new rollout and validate.

GitOps Rollback

  1. Revert or restore a previous commit in Git.
  2. Wait for Operator to reconcile, or force sync.
  3. Confirm application behavior.

Rollbacks are often coupled with incident reports and post-incident reviews to improve future workflows.

Scaling and Resource Adjustment Workflows

Scaling is a core operational workflow for production and HPC-style workloads.

Manual Scale Up/Down

  1. Change number of replicas:
    • oc scale deployment myapp --replicas=5
  2. Confirm:
    • oc get pods to ensure all new pods are running.
  3. Monitor metrics (CPU, memory, latency) for impact.

This pattern is widely used during:

  • Flash traffic events.
  • Stress tests.
  • Batch window peaks (e.g., large MPI or GPU jobs).

Autoscaling Tuning

While separate chapters cover autoscaling details, the workflow around it typically includes:

  1. Analyze baseline consumption from metrics.
  2. Adjust HorizontalPodAutoscaler or other scaling policies.
  3. Validate that scale-out/in events happen as expected.
  4. Set guardrails (max replicas, scheduling constraints).

In HPC-like scenarios with GPU or large-memory pods, scaling also involves confirming resource availability and queueing behavior (possibly through custom controllers or Operators).

Maintenance and Change Windows

Routine maintenance (cluster-level or node-level) intersects application workflows.

Planned Changes

Typical steps for app teams:

  1. Receive notice of cluster maintenance window from platform team.
  2. Confirm apps are using:
    • Multiple replicas.
    • Proper readiness/liveness probes.
  3. Optionally:
    • Temporarily increase replicas to absorb disruptions.
    • Disable or adjust batch submissions.

After maintenance:

  • Perform health checks.
  • Validate critical workflows.
  • Review logs for anomalies.

OpenShift-Specific Maintenance for Apps

Common repetitive actions:

  • Rotating TLS certificates used in Routes.
  • Refreshing service account tokens or keys.
  • Reapplying quotas and limits when scaling environments.

Typical Workflows for Batch and HPC-Style Jobs

On OpenShift clusters supporting HPC and specialized workloads, additional patterns emerge.

Batch Job Submission Workflow

  1. Prepare container image with necessary tools/libraries (e.g., MPI, scientific stacks).
  2. Define Job or custom CR (via an Operator for batch/HPC).
  3. Submit job:
    • oc apply -f job.yaml
  4. Monitor:
    • oc get jobs
    • oc logs for job pods.
  5. Collect results from:
    • Persistent storage.
    • Object storage or external systems.

Repeat as needed, potentially via automation or workflow engines (e.g., Airflow, Argo Workflows).

Parallel/MPI Job Workflow

  1. Use specialized MPI container image.
  2. Define a job manifest that:
    • Requests appropriate resources (CPU, memory, GPUs).
    • Ensures pods are co-scheduled as required (via Operators or custom job controllers).
  3. Submit job and monitor its pods and logs.
  4. Validate intra-pod communication and performance metrics.
  5. Iterate on:
    • MPI configuration.
    • Node placement constraints.
    • Resource requests/limits.

These workflows are usually integrated with standard OpenShift practices (CI/CD for images, GitOps for manifests, metrics-based tuning), but tailored to HPC patterns like job queues and tightly coupled parallel runs.

Putting It Together: Composite Workflow Example

A realistic OpenShift usage often combines several of these workflows:

  1. Developer updates code and pushes to Git.
  2. CI/CD pipeline:
    • Builds container image.
    • Runs tests.
    • Updates GitOps manifests or image tags.
  3. GitOps promotion moves changes to dev → test.
  4. Configuration tweaks are applied via ConfigMaps/Secrets.
  5. Autoscaling policies are adjusted based on observed metrics.
  6. A run of batch/HPC jobs is triggered after deployment to generate data or perform heavy analysis.
  7. If an issue is observed:
    • Incident workflow kicks in (logs, metrics, debug).
    • Rollback occurs via deployment undo or Git revert.
  8. After stabilization:
    • The version is promoted to production, following the same systematic flows.

These patterns—rather than individual commands or YAML fields—are what define “typical OpenShift workflows” in day-to-day practice.

Views: 0

Comments

Please login to add a comment.

Don't have an account? Register now!

Where to Move

Move chapter:

  • ☰ 1. Course Overview
    • ☰ 1.1. What is OpenShift
      • ☰ 1.2. Why OpenShift is used in modern IT and cloud environments
        • ☰ 1.3. OpenShift vs Kubernetes
          • ☰ 1.4. Typical OpenShift use cases
            • ☰ 1.5. Structure and goals of the course
            • ☰ 2. Containers and Cloud-Native Fundamentals
              • ☰ 2.1. What is a container
                • ☰ 2.2. Containers vs virtual machines
                  • ☰ 2.3. Introduction to Docker and container images
                    • ☰ 2.4. Container registries
                      • ☰ 2.5. Basics of cloud-native applications
                      • ☰ 3. Kubernetes Fundamentals
                        • ☰ 3.1. What is Kubernetes
                          • ☰ 3.2. Kubernetes architecture
                            • ☰ 3.3. Control plane components
                              • ☰ 3.4. Worker node components
                                • ☰ 3.5. Kubernetes objects and resources
                                  • ☰ 3.6. Declarative configuration model
                                  • ☰ 4. OpenShift Architecture
                                    • ☰ 4.1. Overview of OpenShift architecture
                                      • ☰ 4.2. Control plane in OpenShift
                                        • ☰ 4.3. Nodes and machine roles
                                          • ☰ 4.4. OpenShift Operators
                                            • ☰ 4.5. OpenShift networking model
                                              • ☰ 4.6. OpenShift storage model
                                              • ☰ 5. OpenShift Installation and Deployment Models
                                                • ☰ 5.1. OpenShift deployment options
                                                  • ☰ 5.2. Installer-Provisioned Infrastructure
                                                    • ☰ 5.3. User-Provisioned Infrastructure
                                                      • ☰ 5.4. Managed OpenShift services
                                                        • ☰ 5.5. Single-node OpenShift
                                                          • ☰ 5.6. Cluster lifecycle management
                                                          • ☰ 6. Accessing and Using OpenShift
                                                            • ☰ 6.1. OpenShift web console
                                                              • ☰ 6.2. OpenShift CLI (oc)
                                                                • ☰ 6.3. Authentication and authorization
                                                                  • ☰ 6.4. Projects and namespaces
                                                                    • ☰ 6.5. Resource quotas and limits
                                                                    • ☰ 7. Building and Deploying Applications
                                                                      • ☰ 7.1. Application deployment concepts
                                                                        • ☰ 7.2. Source-to-Image (S2I)
                                                                          • ☰ 7.3. Container-based deployments
                                                                            • ☰ 7.4. DeploymentConfigs and Deployments
                                                                              • ☰ 7.5. Rolling updates and rollbacks
                                                                              • ☰ 8. Networking in OpenShift
                                                                                • ☰ 8.1. Services and service discovery
                                                                                  • ☰ 8.2. Routes and ingress
                                                                                    • ☰ 8.3. Network policies
                                                                                      • ☰ 8.4. Load balancing
                                                                                        • ☰ 8.5. External access to applications
                                                                                        • ☰ 9. Storage in OpenShift
                                                                                          • ☰ 9.1. Persistent storage concepts
                                                                                            • ☰ 9.2. Persistent Volumes
                                                                                              • ☰ 9.3. Persistent Volume Claims
                                                                                                • ☰ 9.4. StorageClasses
                                                                                                  • ☰ 9.5. Dynamic provisioning
                                                                                                    • ☰ 9.6. Stateful applications
                                                                                                    • ☰ 10. Configuration and Secrets Management
                                                                                                      • ☰ 10.1. ConfigMaps
                                                                                                        • ☰ 10.2. Secrets
                                                                                                          • ☰ 10.3. Environment variables
                                                                                                            • ☰ 10.4. Managing sensitive data
                                                                                                              • ☰ 10.5. Best practices for configuration
                                                                                                              • ☰ 11. Scaling and High Availability
                                                                                                                • ☰ 11.1. Horizontal pod autoscaling
                                                                                                                  • ☰ 11.2. Vertical scaling
                                                                                                                    • ☰ 11.3. Self-healing and pod restarts
                                                                                                                      • ☰ 11.4. High availability concepts
                                                                                                                        • ☰ 11.5. Multi-zone and multi-region clusters
                                                                                                                        • ☰ 12. Monitoring, Logging, and Observability
                                                                                                                          • ☰ 12.1. Built-in monitoring stack
                                                                                                                            • ☰ 12.2. Metrics and alerts
                                                                                                                              • ☰ 12.3. Logging architecture
                                                                                                                                • ☰ 12.4. Distributed tracing
                                                                                                                                  • ☰ 12.5. Troubleshooting applications
                                                                                                                                  • ☰ 13. Security in OpenShift
                                                                                                                                    • ☰ 13.1. Security model overview
                                                                                                                                      • ☰ 13.2. Role-Based Access Control
                                                                                                                                        • ☰ 13.3. Security Context Constraints
                                                                                                                                          • ☰ 13.4. Image security and scanning
                                                                                                                                            • ☰ 13.5. Network security
                                                                                                                                              • ☰ 13.6. Compliance and auditing
                                                                                                                                              • ☰ 14. CI/CD and DevOps with OpenShift
                                                                                                                                                • ☰ 14.1. DevOps concepts
                                                                                                                                                  • ☰ 14.2. CI/CD pipelines overview
                                                                                                                                                    • ☰ 14.3. OpenShift Pipelines (Tekton)
                                                                                                                                                      • ☰ 14.4. GitOps with Argo CD
                                                                                                                                                        • ☰ 14.5. Integrating OpenShift with external CI tools
                                                                                                                                                        • ☰ 15. Operators and Platform Services
                                                                                                                                                          • ☰ 15.1. What are Operators
                                                                                                                                                            • ☰ 15.2. Operator Lifecycle Manager
                                                                                                                                                              • ☰ 15.3. Installing and managing Operators
                                                                                                                                                                • ☰ 15.4. Common platform Operators
                                                                                                                                                                  • ☰ 15.5. Custom Operators
                                                                                                                                                                  • ☰ 16. Upgrades, Maintenance, and Operations
                                                                                                                                                                    • ☰ 16.1. OpenShift upgrade process
                                                                                                                                                                      • ☰ 16.2. Backup and restore
                                                                                                                                                                        • ☰ 16.3. Node maintenance
                                                                                                                                                                          • ☰ 16.4. Cluster troubleshooting
                                                                                                                                                                            • ☰ 16.5. Capacity planning
                                                                                                                                                                            • ☰ 17. OpenShift on HPC and Specialized Workloads
                                                                                                                                                                              • ☰ 17.1. Running batch workloads on OpenShift
                                                                                                                                                                                • ☰ 17.2. OpenShift and GPUs
                                                                                                                                                                                  • ☰ 17.3. MPI and parallel workloads in containers
                                                                                                                                                                                    • ☰ 17.4. Hybrid HPC and cloud workflows
                                                                                                                                                                                    • ☰ 18. OpenShift in Practice
                                                                                                                                                                                      • ☰ 18.1. Typical OpenShift workflows
                                                                                                                                                                                        • ☰ 18.2. Development to production lifecycle
                                                                                                                                                                                          • ☰ 18.3. Case studies from industry
                                                                                                                                                                                            • ☰ 18.4. Common pitfalls and best practices
                                                                                                                                                                                            • ☰ 19. Future Directions and Ecosystem
                                                                                                                                                                                              • ☰ 19.1. Kubernetes ecosystem overview
                                                                                                                                                                                                • ☰ 19.2. OpenShift roadmap
                                                                                                                                                                                                  • ☰ 19.3. Serverless and event-driven workloads
                                                                                                                                                                                                    • ☰ 19.4. AI and data platforms on OpenShift
                                                                                                                                                                                                    • ☰ 20. Final Project and Hands-On Exercises
                                                                                                                                                                                                      • ☰ 20.1. Deploying a containerized application
                                                                                                                                                                                                        • ☰ 20.2. Implementing CI/CD with OpenShift
                                                                                                                                                                                                          • ☰ 20.3. Securing and scaling an application
                                                                                                                                                                                                            • ☰ 20.4. Monitoring and troubleshooting report