Skybaer Labs logo
SKYBÆR LABS
Showcase Project

OpenShift & AWX

Red Hat CodeReady Containers, running as a single-node OpenShift platform underneath the lab’s AWX instance, and operated like a real platform instead of a VM to log into.

OpenShift / CRC oc CLI Container Platform Operations Custom Execution Environment Capacity Planning Red Hat Entitlements

Running CRC as a Platform

What’s actually true about this cluster day to day

rhel10-crc is its own RHEL 10 VM, running CodeReady Containers as a single-node OpenShift cluster nested inside it, alongside the Ansible RHCE lab VMs on the same physical host. AWX runs inside that cluster and manages this lab’s automation.

Managed like a bastion, not a VM

The oc CLI on control.lab reaches the cluster remotely; day-to-day work never means logging into rhel10-crc directly. There’s not even a fallback shell to lean on: crc ssh doesn’t exist in the current CLI version, and crc console only opens a browser URL.

A custom image, not the stock one

AWX’s stock Execution Environment (quay.io/ansible/awx-ee:24.6.1) doesn’t include everything the lab’s playbooks need. Built awx-ee-custom on top of it via an execution-environment.yml build definition: passlib (the password_hash Jinja2 filter needs it) and git, plus the community.general and community.windows collections pulled in from the lab’s own requirements.yml. Pushed to ghcr.io/jayberryman/skybaer-awx-ee-custom as a public image, so AWX pulls it without a registry credential.

Entitlements, checked not assumed

Red Hat Hybrid Cloud Console shows three active entitlements on the account: a 60-day OpenShift Container Platform trial (expiring Nov 8, 2026), Red Hat Beta Access, and a no-cost Developer Subscription for Individuals. Confirmed CRC authenticates through the account’s pull secret rather than the trial SKU specifically, so it keeps working past the trial’s expiration.

Related write-ups
  • Ansible / RHCE Lab: AWX’s job templates, Surveys, and role-based access control, the Ansible-facing side of this same AWX instance.
  • CRC + AWX Rebuild Runbook: the full rebuild procedure, including the operator image gotcha and the CRC node clock-drift investigation.

Real Troubleshooting

Capacity planning, with evidence

OOM-killed by its own host

Problem
Ran a full-lab load test: every lab VM, the new Foreman VM, and rhel10-crc powered on at the same time on the 32GB physical host.
Root cause
journalctl and dmesg on rhel10-crc showed a global OOM kill of the nested CRC VM’s qemu-kvm process. The host stayed up; CRC didn’t. It can’t sustain everything running at once.
Fix
Reduced CRC’s own memory allocation from 16GB to 12GB (crc config set memory 12288, still above CRC’s documented 9GB minimum) via crc delete + crc start, and settled on a standing rule: CRC/AWX runs in isolation or alongside only light guests, not as part of an all-VMs-on test. A RAM upgrade wasn’t an option without replacing all 4 already-populated DIMM slots. Still open: shrinking rhel10-crc’s own VMware-assigned memory (currently 19GB) down to return the freed headroom to the host.
Questions about the lab?
Contact ·