Based on : https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2/html/service_telemetry_framework_1.5/assembly-installing-the-core-components-of-stf_assembly list all needed operators and images and generate imageset-config.yaml
apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
storageConfig:
local:
path: ./
mirror:
operators:
- catalog: registry.redhat.io/redhat/redhat-operator-index:v4.10
packages:
- name: service-telemetry-operator
channels:
- name: stable-1.5
- name: openshift-cert-manager-operator
channels:
- name: tech-preview
- name: amq7-interconnect-operator
channels:
- name: 1.10.x
- name: smart-gateway-operator
channels:
- name: stable-1.5
- catalog: registry.redhat.io/redhat/certified-operator-index:v4.10
packages:
- name: elasticsearch-eck-operator-certified
minVersion: '2.6.2'
channels:
- name: stable
- catalog: quay.io/operatorhubio/catalog:latest
packages:
- name: prometheus
channels:
- name: beta
minVersion: '0.47.0'
- name: grafana-operator
channels:
- name: alpha
minVersion: '3.10.3'
- name: v4
minVersion: '4.8.0'
maxVersion: '4.8.0'
additionalImages:
- name: registry.redhat.io/openshift4/ose-oauth-proxy:v4.4
- name: gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0
- name: quay.io/prometheus-operator/prometheus-config-reloader:v0.47.0
- name: quay.io/prometheus/alertmanager:v0.21.0
- name: quay.io/prometheus/prometheus:latest
- name: docker.elastic.co/elasticsearch/elasticsearch-ubi8:7.16.1
- name: registry.redhat.io/rhel8/grafana:7
- name: quay.io/grafana-operator/grafana_plugins_init:0.1.0
- name: docker.io/grafana/grafana:8.4.11 # needed for polystat 2.4+
Based on https://github.com/openshift/oc-mirror/issues/538, we’re not able to sync non-default channel by itself; if it errors out, you need to also include at least one version from the default channel, had issue with last version 4.9 so used lower one.
From a server with access to internet, Download images locally
oc mirror --config imageset-config.yaml file://archives
Copy archives disconnected server, and populate images to mirror registry
oc mirror --from archives/ docker://quay.gnali.lab:8443
⚠️ Don’t forget to add quay rootCA to /etc/pki/ca-trust/source/anchors/, and quay authentication to .docker/config.json
Tag following quay image:
Image | Tag |
---|---|
prometheus-operator/prometheus-operator | v0.47.0 |
grafana-operator/grafana-operator | v4.8.0 |
Edit the generated catalog source file from mirroring to match the official source. Doing this we can just copy/paste commands from STF documentation.
$ grep metadata -A1 ./oc-mirror-workspace/results-1639608409/catalogSource-*
catalogSource-catalog.yaml:metadata:
catalogSource-catalog.yaml- name: operatorhubio-operators
--
catalogSource-certified-operator-index.yaml:metadata:
catalogSource-certified-operator-index.yaml- name: certified-operators
--
catalogSource-redhat-operator-index.yaml:metadata:
catalogSource-redhat-operator-index.yaml- name: redhat-operators
Disable default operator sources
oc patch OperatorHub cluster --type json -p '[{"op": "add", "path": "/spec/disableAllDefaultSources", "value": true}]'
Apply image content policy
oc apply -f ./oc-mirror-workspace/results-1639608409/
Connect to Openshift worker, and push oauth-proxy image
oc login -u kubeadmin -p FnJKS-hJTcm-A69VN-aNhDC https://api.ocp.gnali.lab:6443
podman login -u kubeadmin -p $(oc whoami -t) image-registry.openshift-image-registry.svc:5000
podman pull quay.gnali.lab:8443/openshift4/ose-oauth-proxy:v4.4
podman tag quay.gnali.lab:8443/openshift4/ose-oauth-proxy:v4.4 image-registry.openshift-image-registry.svc:5000/openshift/oauth-proxy:v4.4
podman push image-registry.openshift-image-registry.svc:5000/openshift/oauth-proxy:v4.4
Allow image to be pull via tag instead of digest (https://access.redhat.com/solutions/4817401)
$ cat << EOF > prometheus.conf
[[registry]]
prefix = ""
location = "quay.io/prometheus-operator"
mirror-by-digest-only = false
[[registry.mirror]]
location = "quay.gnali.lab:8443/prometheus-operator"
[[registry]]
prefix = ""
location = "quay.io/prometheus"
mirror-by-digest-only = false
[[registry.mirror]]
location = "quay.gnali.lab:8443/prometheus"
[[registry]]
prefix = ""
location = "docker.elastic.co/elasticsearch"
mirror-by-digest-only = false
[[registry.mirror]]
location = "quay.gnali.lab:8443/elasticsearch"
[[registry]]
prefix = ""
location = "gcr.io/kubebuilder"
mirror-by-digest-only = false
[[registry.mirror]]
location = "quay.gnali.lab:8443/kubebuilder"
[[registry]]
prefix = ""
location = "quay.io/grafana-operator"
mirror-by-digest-only = false
[[registry.mirror]]
location = "quay.gnali.lab:8443/grafana-operator"
[[registry]]
prefix = ""
location = "registry.redhat.io/rhel8"
mirror-by-digest-only = false
[[registry.mirror]]
location = "quay.gnali.lab:8443/rhel8"
[[registry]]
prefix = ""
location = "docker.io/grafana"
mirror-by-digest-only = false
[[registry.mirror]]
location = "sopra-quay.gnali.lab:8443/grafana"
EOF
$ base64_reg=$(base64 -w0 prometheus.conf)
$ cat << EOF > worker_registry.yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: master
name: 99-mirror-by-digest-registries
spec:
config:
ignition:
version: 3.1.0
storage:
files:
- contents:
source: data:text/plain;charset=utf-8;base64,${base64_reg}
filesystem: root
mode: 420
path: /etc/containers/registries.conf.d/99-mirror-by-digest-registries.conf
EOF
$ oc apply -f worker_registry.yaml
Important : it’s needed to create a separate machineconfig for each node role
Follow official documentation to install STF: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2/html/service_telemetry_framework_1.5/assembly-installing-the-core-components-of-stf_assembly#deploying-stf-to-the-openshift-environment_assembly-installing-the-core-components-of-stf
Download grafana polystat plugin from https://grafana.com/grafana/plugins/grafana-polystat-panel/?tab=installation
Unzip and copy plugins to grafana containers:
$ oc rsync grafana-polystat-panel grafana-deployment-68684776cc-ss28s:/var/lib/grafana/plugins/
Connect to grafana containers and reload grafana
$ oc exec grafana-deployment-68684776cc-ss28s -it -- /bin/bash
$ killall grafana-server
References:
- https://docs.openshift.com/container-platform/4.11/installing/disconnected_install/installing-mirroring-disconnected.html#installing-mirroring-disconnected
- https://github.com/openshift/oc-mirror
- https://zhimin-wen.medium.com/openshift-4-10-image-mirroring-for-airgap-environment-f6bed61ea719
- https://catalog.redhat.com/software/containers/openshift4/ose-oauth-proxy/5cdb2133bed8bd5717d5ae64?tag=v4.4.0-202101261542.p0&push_date=1612347059000>i-tabs=red-hat-login&container-tabs=gti
- https://cloud.redhat.com/blog/mirroring-openshift-registries-the-easy-way
- https://access.redhat.com/solutions/4817401