Deployment & Infrastructure¶

Production deployment on PM2 + host-level NGINX

Last Updated: 2026-03-09 (v4.3.0)

Infrastructure Overview¶

VitaraVox runs on two cloud instances: an OCI ARM64 server hosting the application, and an AWS instance running the OSCAR EMR development environment.

Production Server (OCI Toronto)¶

Component	Specification
Provider	Oracle Cloud Infrastructure (OCI)
Region	ca-toronto-1
Shape	VM.Standard.A1.Flex (ARM64)
OCPUs	2
Memory	24 GB
Storage	200 GB block volume
OS	Ubuntu (Linux 6.x, aarch64)
Cost	Free tier (Always Free eligible)

OSCAR Dev Instance (AWS Montreal)¶

Component	Specification
Provider	AWS
Region	ca-central-1 (Montreal)
Instance	t3a.medium
vCPU	2
Memory	4 GB
Storage	30 GB gp3
IP	15.222.50.48
Cost	~$30/month

Production clinics

In production, each clinic connects its own OSCAR instance. The AWS instance is for development and testing only. The SOAP adapter connects directly from the OCI server to the clinic OSCAR -- no sidecar or bridge is deployed on the clinic side.

Architecture Diagram¶

                        ┌─────────────────────────┐
                        │   Vapi Voice Platform    │
                        │  (squad of 9 agents)     │
                        └───────────┬─────────────┘
                                    │ HTTPS webhooks
                                    ▼
┌───────────────────────────────────────────────────────────┐
│  OCI Toronto — VM.Standard.A1.Flex (ARM64)                │
│                                                           │
│  ┌──────────────────────────────────────────────────┐     │
│  │  NGINX (host-level)                              │     │
│  │  :80 → redirect to :443                          │     │
│  │  :443 → proxy_pass localhost:3002                │     │
│  │  SSL: Let's Encrypt                              │     │
│  └──────────────────────┬───────────────────────────┘     │
│                         │                                 │
│  ┌──────────────────────▼───────────────────────────┐     │
│  │  PM2 → vitara-admin-api                          │     │
│  │  Node.js 18 + TypeScript (tsx)                   │     │
│  │  Port 3002                                       │     │
│  └──────────────────────┬───────────────────────────┘     │
│                         │                                 │
│  ┌──────────────────────▼───────────────────────────┐     │
│  │  PostgreSQL 16 (host-level)                      │     │
│  │  localhost:5432                                   │     │
│  │  Database: vitara_platform                       │     │
│  └──────────────────────────────────────────────────┘     │
│                                                           │
│  ┌──────────────────────────────────────────────────┐     │
│  │  NGINX (host-level) — vitdocs.vitaravox.ca       │     │
│  │  Static MkDocs site                              │     │
│  └──────────────────────────────────────────────────┘     │
└───────────────────┬───────────────────────────────────────┘
                    │ SOAP / OAuth REST
                    ▼
┌───────────────────────────────────────────────────────────┐
│  AWS Montreal — OSCAR EMR (t3a.medium)                    │
│  15.222.50.48                                             │
│  CXF SOAP endpoint + OAuth REST API                      │
└───────────────────────────────────────────────────────────┘

No Docker for the app server

The application server does not use Docker. The PM2 process, PostgreSQL, and NGINX all run directly on the host OS. The old Docker-based documentation is obsolete.

Prerequisites¶

Ensure the following are installed on the OCI server before deploying.

Required Software¶

Software	Version	Purpose
Node.js	18.x LTS	Application runtime
npm	9.x	Package management
TypeScript	5.3+	Compile-time type checking
tsx	4.x	TypeScript execution (used by PM2)
PM2	Latest	Process manager
PostgreSQL	16	Database
NGINX	Latest	Reverse proxy + SSL termination
Certbot	Latest	SSL certificate management
Git	Latest	Version control

Install Commands¶

# Node.js 18 (via NodeSource)
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
sudo apt-get install -y nodejs

# PM2 (global)
sudo npm install -g pm2

# PostgreSQL 16
sudo apt-get install -y postgresql-16

# NGINX
sudo apt-get install -y nginx

# Certbot
sudo apt-get install -y certbot python3-certbot-nginx

Environment Configuration¶

Location¶

The environment file lives at:

/home/ubuntu/vitara-platform/admin-dashboard/server/.env

Required Variables¶

# ===========================================
# VitaraPlatform Admin Dashboard - Environment
# ===========================================

# Server
PORT=3002
NODE_ENV=production
CORS_ORIGIN=https://dev.vitaravox.ca

# JWT Authentication
JWT_SECRET=<generate-secure-random-string>
JWT_REFRESH_SECRET=<generate-another-secure-random-string>
JWT_EXPIRES_IN=1h
JWT_REFRESH_EXPIRES_IN=7d

# PostgreSQL Database
DATABASE_URL=postgresql://vitara:<password>@localhost:5432/vitara_platform

# Vapi AI Voice Platform
VAPI_API_KEY=<vapi-api-key>
VAPI_BASE_URL=https://api.vapi.ai
VAPI_WEBHOOK_SECRET=<vapi-webhook-secret>

# OSCAR REST Bridge Connection (dev/fallback only)
OSCAR_BRIDGE_URL=http://15.222.50.48:3000/api/v1
OSCAR_BRIDGE_API_KEY=<oscar-bridge-api-key>

# Encryption
ENCRYPTION_KEY=<generate-32-byte-hex-key>

Never commit .env to version control

The .env file contains secrets (JWT keys, database credentials, API keys). It is gitignored. Copy from .env.example and fill in production values.

Generate Secrets¶

# JWT secrets (64-char base64)
openssl rand -base64 64

# Encryption key (32-byte hex)
openssl rand -hex 32

# Database password
openssl rand -base64 24

Optional Variables¶

# Debug mode (auto-expires after 4 hours, DO NOT leave on unattended)
# VITARA_DEBUG=false

# Log level override (trace, debug, info, warn, error, fatal)
# LOG_LEVEL=info

Middleware Stack¶

The Express server applies middleware in the following order (defined in src/index.ts):

Source: index.ts:33-79

#	Middleware	Purpose
1	`helmet()`	Security headers (CSP, HSTS, X-Frame-Options)
2	`requestLogger` (pino-http)	Request logging with `x-request-id` correlation
3	`cors({ origin: CORS_ORIGIN })`	CORS with configured origin
4	`express.json({ limit: '1mb' })`	JSON body parser
5	`auditMiddleware`	Global — logs POST/PUT/DELETE to `AuditLog` table
6	Rate limiters (per-route)	5/min auth, 100/min API, 300/min webhooks
7	`vapiWebhookAuth`	HMAC-SHA256 / API key / Bearer token on `/api/vapi`
8	`authMiddleware`	JWT verification on `/api/admin/` and `/api/clinic/`
9	`requireRole()`	RBAC enforcement (vitara_admin / clinic_manager)
10	`requireClinicAccess`	Tenant isolation (clinic managers see own clinic only)

EXPRESS MIDDLEWARE STACK (source: index.ts:33-79)

Incoming Request
       │
       ▼
┌──────────────────┐
│ helmet()         │ Security headers (CSP, HSTS, X-Frame)
├──────────────────┤
│ requestLogger    │ pino-http: request logging + x-request-id
├──────────────────┤
│ cors()           │ CORS policy enforcement
├──────────────────┤
│ express.json()   │ Body parser (1mb limit)
├──────────────────┤
│ auditMiddleware  │ Global: logs POST/PUT/DELETE to AuditLog
└────────┬─────────┘
         │
    Route matching
    ┌────┴────────────────────────────┐
    │                                 │
    ▼                                 ▼
/api/vapi/*                    /api/clinic/* & /api/admin/*
    │                                 │
    ▼                                 ▼
┌──────────────┐              ┌──────────────────┐
│webhookLimiter │ 300/min     │ apiLimiter       │ 100/min
├──────────────┤              ├──────────────────┤
│vapiWebhookAuth│             │ authMiddleware   │ JWT validation
│(HMAC/Bearer/ │              ├──────────────────┤
│ API Key)     │              │ requireRole()    │ Role check
└──────┬───────┘              ├──────────────────┤
       │                      │requireClinicAccess│ Tenant check
       │                      └────────┬─────────┘
       │                               │
       └───────────┬───────────────────┘
                   ▼
            Route Handler
                   │
                   ▼
              Response

JWT Authentication Flow¶

1. POST /api/auth/login → bcrypt compare → issue access (1h) + refresh (7d) tokens
2. Client includes Authorization: Bearer {access_token} on every request
3. authMiddleware verifies HS256 signature, extracts userId/role
4. On 401, client calls POST /api/auth/refresh with refresh token
5. Server issues new access token (refresh token unchanged)

No Token Revocation

JWT tokens cannot be revoked server-side — there is no blacklist. Tokens remain valid until expiry (1h access, 7d refresh). This is acceptable for the current single-instance deployment but needs a Redis-backed blacklist for HA or multi-instance setups.

Database Setup¶

Create the Database¶

# Switch to postgres user
sudo -u postgres psql

# Create role and database
CREATE USER vitara WITH PASSWORD '<secure-password>';
CREATE DATABASE vitara_platform OWNER vitara;
GRANT ALL PRIVILEGES ON DATABASE vitara_platform TO vitara;
\q

Run Prisma Migrations¶

cd /home/ubuntu/vitara-platform/admin-dashboard/server

# Generate the Prisma client
npx prisma generate

# Deploy migrations to production (non-interactive, does NOT reset data)
npx prisma migrate deploy

Migration commands

npx prisma migrate deploy -- Apply pending migrations (production-safe, non-interactive).
npx prisma migrate dev -- Create and apply migrations during development.
npx prisma migrate reset --force -- Drop and recreate the database. Destructive. Dev only.
npx prisma studio -- Launch the visual database browser on port 5555.

Seed Data (Optional)¶

cd /home/ubuntu/vitara-platform/admin-dashboard/server
npx tsx prisma/seed.ts

Deployment Procedure¶

PM2 Process Configuration¶

The PM2 ecosystem file is at:

/home/ubuntu/vitara-platform/admin-dashboard/server/ecosystem.config.cjs

module.exports = {
  apps: [{
    name: 'vitara-admin-api',
    script: 'npx',
    args: 'tsx src/index.ts',
    cwd: '/home/ubuntu/vitara-platform/admin-dashboard/server',
    env: {
      NODE_ENV: 'production',
      PORT: 3002,
      CORS_ORIGIN: 'https://dev.vitaravox.ca'
    },
    watch: false,
    max_memory_restart: '500M'
  }]
};

tsx, not compiled JS

The PM2 process runs npx tsx src/index.ts directly rather than compiling to JavaScript first. The tsx loader handles TypeScript transpilation at runtime. For manual TypeScript verification, you can still run npx tsc --noEmit.

First-Time Deployment¶

cd /home/ubuntu/vitara-platform/admin-dashboard/server

# Install dependencies
npm install

# Generate Prisma client
npx prisma generate

# Apply database migrations
npx prisma migrate deploy

# Start with PM2
pm2 start ecosystem.config.cjs

# Save the PM2 process list (survives reboot)
pm2 save

# Enable PM2 startup on boot
pm2 startup

Standard Deployment (Code Updates)¶

cd /home/ubuntu/vitara-platform/admin-dashboard/server

# Pull latest code
git pull origin main

# Install any new dependencies
npm install

# Apply pending database migrations
npx prisma migrate deploy

# Generate Prisma client (if schema changed)
npx prisma generate

# Restart the PM2 process
pm2 restart vitara-admin-api

Adapter Warm-Up on Startup¶

PM2 process startup triggers EMR adapter warm-up for all preferRest clinics via an IIFE in index.ts:

Queries database for clinics with preferRest: true
Creates adapter instances and awaits warmUp() for each
Prevents cold TLS handshake (~6s to Kai Cloudflare) from exceeding the 4s circuit breaker on first production call
Non-blocking: warm-up failures are logged but don't prevent server startup

PM2 STARTUP SEQUENCE

pm2 restart vitara-admin-api
         │
         ▼
┌────────────────────────────┐
│ index.ts loads             │
│  ├─ Express app setup      │
│  ├─ Middleware stack        │
│  ├─ Route registration     │
│  └─ server.listen(3002)    │
└────────────┬───────────────┘
             │
             ▼
┌────────────────────────────┐
│ IIFE: Adapter Warm-Up      │
│  ├─ Query DB for clinics   │
│  │   with preferRest=true  │
│  ├─ For each clinic:       │
│  │   ├─ Create adapter     │
│  │   ├─ await warmUp()     │
│  │   │   (TLS handshake    │
│  │   │    to Kai CF ~6s)   │
│  │   └─ Cache adapter      │
│  └─ Log: "X adapters warm" │
│                             │
│  ⚠ Non-blocking:           │
│  Failures logged, server   │
│  still accepts traffic     │
└────────────────────────────┘

Graceful Shutdown¶

Source: index.ts:169-188

When PM2 sends SIGTERM or SIGINT, the server drains in-flight requests before exiting. This is critical for update_appointment, which books a new slot then cancels the old one — interruption between those steps creates duplicate appointments.

SIGTERM/SIGINT received
       │
       ▼
server.close()             ← stop accepting new connections
       │
       ├── Wait for in-flight requests to complete
       │
       ├── All drained → exit(0)
       │
       └── 10s timeout → forced exit(1)

Drain timeout: 10 seconds (hardcoded in index.ts:182)
The 10s timer is .unref()'d so it doesn't prevent shutdown if all connections drain first
PM2 sends SIGTERM first; if the process doesn't exit, PM2 sends SIGKILL after its own timeout

Quick Build & Restart (No Schema Changes)¶

When deploying code changes that do not affect the Prisma schema or dependencies:

cd /home/ubuntu/vitara-platform/admin-dashboard/server && npx tsc && pm2 restart vitara-admin-api

TypeScript compilation

Running npx tsc is a type-check step (it compiles to dist/). The PM2 process itself uses tsx to run TypeScript directly, so the tsc step serves as a build-time validation that there are no type errors before restarting.

NGINX Configuration¶

NGINX runs at the host level (not in a Docker container). Configuration files are in /etc/nginx/.

API Proxy (api.vitaravox.ca)¶

Create or edit /etc/nginx/sites-available/api.vitaravox.ca:

# HTTPS Server
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name api.vitaravox.ca;

    # SSL (managed by Certbot)
    ssl_certificate /etc/letsencrypt/live/vitaravox.ca/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/vitaravox.ca/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers off;
    ssl_session_cache shared:SSL:10m;

    # Security headers
    add_header Strict-Transport-Security "max-age=31536000" always;
    add_header X-Frame-Options "DENY" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # Request size limit
    client_max_body_size 1M;

    # Proxy to PM2 process on port 3002
    location / {
        proxy_pass http://127.0.0.1:3002;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Connection "";

        proxy_connect_timeout 5s;
        proxy_send_timeout 30s;
        proxy_read_timeout 30s;
    }

    # Health check (no access log)
    location /health {
        proxy_pass http://127.0.0.1:3002/health;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        access_log off;
    }
}

# HTTP redirect to HTTPS
server {
    listen 80;
    listen [::]:80;
    server_name api.vitaravox.ca;

    location /health {
        proxy_pass http://127.0.0.1:3002/health;
        access_log off;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

Documentation Site (vitdocs.vitaravox.ca)¶

The docs site is served as static files. Config at /etc/nginx/sites-enabled/vitdocs.vitaravox.ca:

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name vitdocs.vitaravox.ca;

    ssl_certificate /etc/letsencrypt/live/vitaravox.ca/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/vitaravox.ca/privkey.pem;

    root /home/ubuntu/vitaravox-docs/site;
    index index.html;

    location / {
        try_files $uri $uri/ =404;
    }
}

server {
    listen 80;
    server_name vitdocs.vitaravox.ca;
    return 301 https://$host$request_uri;
}

Enable Sites and Reload¶

# Symlink to sites-enabled (if not already done)
sudo ln -s /etc/nginx/sites-available/api.vitaravox.ca /etc/nginx/sites-enabled/
sudo ln -s /etc/nginx/sites-available/vitdocs.vitaravox.ca /etc/nginx/sites-enabled/

# Test configuration
sudo nginx -t

# Reload
sudo nginx -s reload

SSL Setup (Let's Encrypt)¶

Initial Certificate¶

# Stop NGINX temporarily if port 80 is in use
sudo systemctl stop nginx

# Obtain certificate (standalone mode)
sudo certbot certonly --standalone \
  -d vitaravox.ca \
  -d "*.vitaravox.ca" \
  --email admin@vitaravox.com \
  --agree-tos \
  --non-interactive \
  --preferred-challenges dns

# Restart NGINX
sudo systemctl start nginx

Wildcard certificates

Wildcard certificates (*.vitaravox.ca) require DNS challenge verification. Certbot will prompt for a DNS TXT record. For automated renewal, configure a DNS plugin (e.g., Cloudflare).

Automatic Renewal¶

# Certbot installs a systemd timer by default. Verify it:
sudo systemctl status certbot.timer

# Or add a cron job:
# 0 3 * * * certbot renew --quiet --post-hook "sudo nginx -s reload"

Verify Certificate¶

# Check expiry date
sudo certbot certificates

# Or via OpenSSL
openssl s_client -connect api.vitaravox.ca:443 -servername api.vitaravox.ca < /dev/null 2>/dev/null | openssl x509 -noout -dates

Vapi GitOps Deployment¶

The Vapi voice agent configuration (assistants, tools, squad) is managed as code via the GitOps tool.

Location¶

/home/ubuntu/vitara-platform/vapi-gitops/

Environment¶

The GitOps tool requires a .env.dev file with the Vapi token:

# /home/ubuntu/vitara-platform/vapi-gitops/.env.dev
VAPI_TOKEN=<vapi-api-key>

Environment variable name

The Vapi GitOps tool uses VAPI_TOKEN (not VAPI_API_KEY). Using the wrong name will cause silent authentication failures.

Push Configuration to Vapi¶

cd /home/ubuntu/vitara-platform/vapi-gitops

# Push all assistant/tool/squad configs to Vapi dev environment
npm run push:dev

# Force push (overwrites remote even if unchanged)
npm run push:dev:force

Pull Configuration from Vapi¶

cd /home/ubuntu/vitara-platform/vapi-gitops

# Pull current state from Vapi into local YAML files
npm run pull:dev

Available Commands¶

Command	Description
`npm run push:dev`	Push local config to Vapi (dev)
`npm run push:prod`	Push local config to Vapi (prod)
`npm run pull:dev`	Pull remote config to local (dev)
`npm run apply:dev`	Apply changes (dev)
`npm run call:dev`	Make a test call (dev)
`npm run cleanup:dev`	Remove orphaned resources (dev)

Documentation Site Deployment¶

The documentation is built with MkDocs Material and served as static HTML.

Source and Output¶

Item	Path
Source files	`/home/ubuntu/vitaravox-docs/docs/`
MkDocs config	`/home/ubuntu/vitaravox-docs/mkdocs.yml`
Built site	`/home/ubuntu/vitaravox-docs/site/`
Public URL	`https://vitdocs.vitaravox.ca`

Build and Deploy¶

# Activate the MkDocs virtual environment
source /tmp/mkdocs-env/bin/activate

# Build the site to a temp directory, then copy to the serve path
mkdocs build --site-dir /tmp/vitdocs-site
cp -r /tmp/vitdocs-site /home/ubuntu/vitaravox-docs/site

# Reload NGINX to pick up any config changes
sudo nginx -s reload

One-liner

source /tmp/mkdocs-env/bin/activate && mkdocs build --site-dir /tmp/vitdocs-site && cp -r /tmp/vitdocs-site /home/ubuntu/vitaravox-docs/site && sudo nginx -s reload

Edit Docs¶

Only edit files under /home/ubuntu/vitaravox-docs/docs/. Never edit files in the site/ directory -- they are overwritten on every build.

Monitoring¶

PM2 Process Status¶

# Quick status table
pm2 status

# Expected output:
# ┌────┬──────────────────┬────────┬──────┬───────┬──────────┐
# │ id │ name             │ mode   │ pid  │ status│ mem      │
# ├────┼──────────────────┼────────┼──────┼───────┼──────────┤
# │ 0  │ vitara-admin-api │ fork   │ 1234 │ online│ ~76mb    │
# └────┴──────────────────┴────────┴──────┴───────┴──────────┘

Application Logs¶

# Stream real-time logs
pm2 logs vitara-admin-api

# Last 200 lines
pm2 logs vitara-admin-api --lines 200

# Error logs only
pm2 logs vitara-admin-api --err

Health Check¶

# Local health check
curl -s http://localhost:3002/health | jq .

# External health check
curl -s https://api.vitaravox.ca/health | jq .

Database Health¶

# Check PostgreSQL is accepting connections
pg_isready -h localhost -p 5432

# Check database size
psql -U vitara -d vitara_platform -c "SELECT pg_size_pretty(pg_database_size('vitara_platform'));"

NGINX Status¶

# Test configuration
sudo nginx -t

# Check process
systemctl status nginx

# Access logs
sudo tail -f /var/log/nginx/access.log

# Error logs
sudo tail -f /var/log/nginx/error.log

PM2 Monitoring Dashboard¶

# Interactive dashboard (CPU, memory, logs)
pm2 monit

Backup Procedures¶

Database Backup¶

# Full backup (plain SQL)
pg_dump -U vitara vitara_platform > backup_$(date +%Y%m%d).sql

# Compressed backup
pg_dump -U vitara vitara_platform | gzip > backup_$(date +%Y%m%d).sql.gz

# Custom format (supports parallel restore)
pg_dump -U vitara -Fc vitara_platform > backup_$(date +%Y%m%d).dump

No Docker exec

PostgreSQL runs on the host, not in a container. Use pg_dump directly -- do not use docker exec.

Automated Daily Backup¶

Add to crontab (crontab -e):

# Daily database backup at 2 AM, keep 30 days
0 2 * * * pg_dump -U vitara vitara_platform | gzip > /home/ubuntu/backups/vitara_$(date +\%Y\%m\%d).sql.gz
0 3 * * * find /home/ubuntu/backups -name "vitara_*.sql.gz" -mtime +30 -delete

Restore from Backup¶

# Plain SQL restore
psql -U vitara -d vitara_platform < backup_20260216.sql

# Compressed restore
gunzip -c backup_20260216.sql.gz | psql -U vitara -d vitara_platform

# Custom format restore
pg_restore -U vitara -d vitara_platform backup_20260216.dump

Vapi Configuration Backup¶

The Vapi GitOps config is version-controlled in Git. Additionally, manual backups exist at:

/home/ubuntu/vitara-platform/backups/vapi-20260210/

To create a fresh backup:

cd /home/ubuntu/vitara-platform/vapi-gitops
npm run pull:dev
# Configs are now saved as YAML in the gitops directory

Rollback Procedure¶

Quick Rollback (Code Only)¶

cd /home/ubuntu/vitara-platform

# See recent commits
git log --oneline -10

# Checkout previous version
git checkout HEAD~1

# Restart the PM2 process
cd admin-dashboard/server
pm2 restart vitara-admin-api

Rollback to a Specific Commit¶

cd /home/ubuntu/vitara-platform

# Checkout specific commit
git checkout <commit-sha>

# Install dependencies (in case they changed)
cd admin-dashboard/server
npm install

# Regenerate Prisma client
npx prisma generate

# Restart
pm2 restart vitara-admin-api

Rollback Database Migrations¶

Destructive operation

Rolling back migrations may cause data loss. Always take a backup first.

# Restore from a pre-migration backup
pg_dump -U vitara vitara_platform > pre_rollback_backup.sql
psql -U vitara -d vitara_platform < backup_before_migration.sql

Rollback Vapi Configuration¶

cd /home/ubuntu/vitara-platform/vapi-gitops

# Revert to previous Git commit
git checkout HEAD~1

# Push the old config back to Vapi
npm run push:dev:force

Service URLs¶

Service	URL	Notes
API (HTTPS)	`https://api.vitaravox.ca`	Production API endpoint
API Health	`https://api.vitaravox.ca/health`	Health check
Vapi Webhook	`https://api.vitaravox.ca/api/vapi`	Vapi webhook receiver
Legacy Webhook	`https://api.vitaravox.ca/vapi-webhook`	Legacy webhook path
Admin API	`https://api.vitaravox.ca/api/`	Admin dashboard API
Documentation	`https://vitdocs.vitaravox.ca`	MkDocs site
OSCAR Bridge	`http://15.222.50.48:3000/api/v1`	Dev EMR (REST bridge)
Vapi Phone	`+1 236-305-7446`	v3.0 voice agent

Firewall Configuration¶

# Allow SSH
sudo ufw allow 22/tcp

# Allow HTTP (for ACME/redirect)
sudo ufw allow 80/tcp

# Allow HTTPS
sudo ufw allow 443/tcp

# Enable firewall
sudo ufw enable

# Verify
sudo ufw status

OCI Security Lists

In addition to ufw, ensure the OCI VCN Security List allows ingress on ports 22, 80, and 443. OCI security lists operate independently of the OS firewall.

Troubleshooting¶

PM2 Process Won't Start¶

# Check logs for errors
pm2 logs vitara-admin-api --lines 50

# Common causes:
# - Missing .env file
# - DATABASE_URL incorrect or PostgreSQL not running
# - Port 3002 already in use
# - npm dependencies not installed

# Verify port availability
ss -tlnp | grep 3002

# Restart with full log output
pm2 delete vitara-admin-api
pm2 start ecosystem.config.cjs

Database Connection Failed¶

# Check PostgreSQL is running
systemctl status postgresql

# Test connection
psql -U vitara -d vitara_platform -c "SELECT 1;"

# Check pg_hba.conf allows local connections
sudo cat /etc/postgresql/16/main/pg_hba.conf | grep vitara

OSCAR SOAP Connection Failed¶

# Test SOAP endpoint reachability
curl -s http://15.222.50.48:8080/oscar/ws/services/DemographicService?wsdl | head -5

# Check server logs for SOAP errors
pm2 logs vitara-admin-api --lines 100 | grep -i soap

# Note: SOAP cold-start WSDL fetch can take 3-4s.
# Circuit breaker timeout is set to 4s (under Vapi's 5s tool timeout).

SSL Certificate Issues¶

# Check certificate expiry
sudo certbot certificates

# Force renewal
sudo certbot renew --force-renewal

# Reload NGINX after renewal
sudo nginx -s reload

High Memory / Restart Loop¶

# Check PM2 restart count (the ↺ column)
pm2 status

# If max_memory_restart (500M) is being hit:
pm2 logs vitara-admin-api --err --lines 200

# Check system memory
free -h

Production Checklist¶

Use this checklist when deploying to a new environment or onboarding a new developer.

[ ] OCI instance provisioned (ARM64, 2 OCPU, 24GB RAM)
[ ] Node.js 18, npm, PM2 installed
[ ] PostgreSQL 16 installed and running
[ ] Database and user created (vitara_platform / vitara)
[ ] NGINX installed and configured
[ ] SSL certificate obtained via Let's Encrypt
[ ] .env file created with all required variables
[ ] npm install completed in admin-dashboard/server/
[ ] Prisma client generated (npx prisma generate)
[ ] Migrations applied (npx prisma migrate deploy)
[ ] PM2 process started and saved (pm2 start && pm2 save)
[ ] PM2 startup hook enabled (pm2 startup)
[ ] Health check passing (curl https://api.vitaravox.ca/health)
[ ] Vapi webhook URL configured in Vapi dashboard
[ ] Vapi GitOps .env.dev configured with VAPI_TOKEN
[ ] Firewall rules applied (ufw + OCI Security List)
[ ] Automated database backups scheduled (cron)
[ ] SSL auto-renewal verified (certbot renew --dry-run)
[ ] Documentation site built and accessible

Key File Locations¶

File	Path
Application source	`/home/ubuntu/vitara-platform/admin-dashboard/server/src/`
PM2 ecosystem config	`/home/ubuntu/vitara-platform/admin-dashboard/server/ecosystem.config.cjs`
Environment variables	`/home/ubuntu/vitara-platform/admin-dashboard/server/.env`
Prisma schema	`/home/ubuntu/vitara-platform/admin-dashboard/server/prisma/schema.prisma`
Vapi GitOps config	`/home/ubuntu/vitara-platform/vapi-gitops/`
NGINX configs	`/etc/nginx/sites-enabled/`
SSL certificates	`/etc/letsencrypt/live/vitaravox.ca/`
Docs source	`/home/ubuntu/vitaravox-docs/docs/`
Docs built site	`/home/ubuntu/vitaravox-docs/site/`
MkDocs config	`/home/ubuntu/vitaravox-docs/mkdocs.yml`
Database backups	`/home/ubuntu/backups/`
Vapi backups	`/home/ubuntu/vitara-platform/backups/vapi-20260210/`