Troubleshooting

Start with the CR's Ready condition: its reason is listed in Reason codes. This page covers the failures that happen around the CRs.

The webhook is downCopy Anchor Link

Symptom. Creating or updating a namespaced CR fails with:

Error from server (InternalError): Internal error occurred: failed calling webhook
"validate.forgejo.weebo.io": failed to call webhook: Post
"https://weebo-forgejo.weebo-forgejo.svc:443/validate?timeout=5s": ...

With failurePolicy: Fail (the default), no replica answering means these writes are refused. Deletes are not affected.

kubectl -n weebo-forgejo get pods,endpoints
kubectl -n weebo-forgejo describe pod -l app.kubernetes.io/name=weebo-forgejo
kubectl -n weebo-forgejo logs -l app.kubernetes.io/name=weebo-forgejo --prefix --tail=100

kubectl -n weebo-forgejo get certificate,secret | grep webhook
kubectl get validatingwebhookconfiguration weebo-forgejo \
  -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | base64 -d | openssl x509 -noout -subject -enddate

Emergency: let writes throughCopy Anchor Link

If you must apply namespaced CRs now and can't fix the webhook, you can temporarily stop admission checks. The reconcilers still enforce the allow-list, so a disallowed CR is stored but never acted on (NamespaceNotAllowed).

kubectl patch validatingwebhookconfiguration weebo-forgejo --type json \
  -p '[{"op":"replace","path":"/webhooks/0/failurePolicy","value":"Ignore"}]'
# ...fix the webhook, then restore it (a helm upgrade does too):
kubectl patch validatingwebhookconfiguration weebo-forgejo --type json \
  -p '[{"op":"replace","path":"/webhooks/0/failurePolicy","value":"Fail"}]'

Never leave it on Ignore. Removing the operator for good? helm uninstall deletes the webhook configuration with it.

Nothing reconcilesCopy Anchor Link

CRs stay at their old status and nothing changes in Forgejo.

kubectl -n weebo-forgejo get lease weebo-forgejo-operator-leader -o yaml   # who leads, last renewal
kubectl -n weebo-forgejo logs <leader-pod> --tail=200

• No holder, or a holder that's gone: the Lease expires after 15 s and a replica takes it. If none does, check the replicas' logs for API errors (RBAC on leases in the release namespace).
• The leader restarts in a loop: its liveness probe (/livez) fails when its controllers stop making progress; the logs say which.
• CRs report InstanceUnreachable: the operator can't reach Forgejo (DNS, TLS, NetworkPolicy egress). See Connect a Forgejo.

A CR won't deleteCopy Anchor Link

A CR stuck in Terminating still has the operator's finalizer: the operator must clean up in Forgejo first. Look at its condition: an unreachable instance or a refused API call blocks the cleanup. Fix the cause; the finalizer is then removed on its own.

Only if the Forgejo object is already gone, or you accept leaving it behind, remove the finalizer by hand:

kubectl -n team-a patch fjrepo app --type merge -p '{"metadata":{"finalizers":null}}'

helm upgrade refuses the CRDsCopy Anchor Link

Error: UPGRADE FAILED: ... CustomResourceDefinition "forgejousers.forgejo.weebo.io"
exists and cannot be imported into the current release: invalid ownership metadata

The CRDs were installed outside this release (chart 0.1.0's crds/, or applied by hand). Hand them over to the release, see Upgrading from chart 0.1.0, or install with crds.install=false and keep managing them yourself.

Values are refusedCopy Anchor Link

Error: values don't meet the specifications of the schema(s) ...

The chart validates its values against values.schema.json: the message names the key. Check its spelling and allowed values with helm show values oci://ghcr.io/batleforc/charts/weebo-forgejo.

NextCopy Anchor Link

Backup & disaster recovery.