43 docs tagged with "operations"

Backup procedures and restore operations.

How to deploy and update services in RavenmaskOS.

Full recovery procedures for catastrophic failures.

Alert Details

If cleanup is insufficient, resize the Colima VM disk:

Alert Details

Alert Details

Approval workflow for high-risk automated operations in RavenmaskOS.

This runbook documents the CI/CD pipeline for n8n workflow management, enabling version control and automated deployment of workflows from development to pro...

Day-to-day operations and maintenance procedures for OpenBao.

Day-2 operational procedures for RavenmaskOS.

The RavenmaskOS wiki is the primary source of truth for all infrastructure, services, and operational documentation.

Create a new OIDC application in Zitadel and connect it to a service.

Diagnose and resolve SSO and login failures across RavenmaskOS.

If issues persist: 1. Check if upstream MCP servers are healthy 2. Review recent changes to Bifrost configuration 3. Check Grafana for anomalous metrics patt...

Handle TLS certificate renewal issues for Traefik/Let's Encrypt.

Remove old manual backups:

Run schema migrations safely for production services.

Restore PostgreSQL from a backup file.

Safely remove a service from RavenmaskOS, including all related infrastructure components, secrets, and tracking documentation.

Complete removal of Twenty CRM service from RavenmaskOS infrastructure. Twenty was installed for evaluation but replaced by custom CRM solution.

Update: - image: - Correct image and tag - container_name: - Match service name - volumes: - Map data directory - labels: - Traefik routing - environment: - ...

Resolve 403 Forbidden errors when Vidar/Bifrost attempts to access Docker network information through the docker-socket-proxy.

Diagnose connectivity problems between services or external clients.

If issues persist: 1. Check Langfuse traces for detailed error context 2. Review recent changes in GitLab: https://gitlab.ravenhelm.dev/agents/norns 3. Check...

Common Redis maintenance and recovery tasks.

Recover or reset Zitadel admin credentials.

Rotate the OAuth2-Proxy cookie secret used for SSO sessions.

If service won't recover: 1. Check Grafana for system-wide issues 2. Review recent changes 3. Consider rollback 4. Check disaster recovery options

Recover SPIRE agent attestation for identity workloads.

Upgrade Traefik with minimal downtime.

Procedure for unsealing OpenBao after container restart or server reboot.

Safely update a running service to a new image or configuration.

The pg library uses JavaScript's native URL parser to parse PostgreSQL connection strings. If the password contains special characters that aren't URL-encode...

Resolve 500 Internal Server Error on Vidar API endpoints caused by database column name mismatches between code and schema.

Resolve duplicate key constraint violations in the Vidar/Bifrost discovery scheduler when syncing entities from multiple sources.

Resolve OpenAI API rate limit errors (429) affecting the Vidar SRE agent's ability to investigate and remediate alerts.

Expected output shows both containers running: - vidar-api - vidar-admin

If issues persist: 1. Check Twilio service status: https://status.twilio.com 2. Review LiveKit logs for room/participant issues 3. Check if Whisper/Piper mod...

Step-by-step operational procedures for RavenmaskOS.

Managing credentials and sensitive configuration.

Status: Phase 3 Complete - Migrating secrets from 1Password to OpenBao for centralized secrets management.

Operational procedures for the Norns WhatsApp channel.