Cluster Deployment

GPT-Load high availability cluster deployment solution with master-slave architecture and horizontal scaling support

Cluster Architecture Overview

Distributed Master-Slave Architecture

Adopts a one-master-multiple-slave distributed architecture where the master node handles management functions and slave nodes focus on proxy services, achieving high availability and load distribution

Master Node

Web management interface, configuration management, statistics analysis

Slave Node

Focused on API proxy services, supports horizontal scaling

Shared Storage

Unified MySQL and Redis cluster

Deployment Requirements

Important Notice

Database and Redis are required for cluster deployment, used for distributed coordination and state synchronization

Infrastructure Requirements

Database Cluster

  • • MySQL 8.2+ high availability cluster
  • • Support master-slave replication or Galera cluster
  • • Recommended to configure read-write separation
  • • Regular backup and disaster recovery mechanisms

Cache Cluster

  • • Redis cluster or sentinel mode
  • • Used for distributed locks and state synchronization
  • • Configure persistence and failover
  • • Monitor memory usage and performance

Deployment Steps

1

Prepare Shared Infrastructure

Deploy MySQL Cluster (or PostgreSQL)

Recommend using cloud database cluster services or deploying independent MySQL/PostgreSQL clusters.

Deploy Redis Cluster

Recommend using Redis Sentinel or cluster mode to ensure high availability and data consistency.

2

Node docker-compose.yml Reference (Same for Master and Slave)

services:
  gpt-load:
    image: ghcr.io/tbphp/gpt-load:latest
    container_name: gpt-load
    ports:
      - "${PORT:-3001}:${PORT:-3001}"
    env_file:
      - .env
    restart: always
    volumes:
      - ./data:/app/data
    stop_grace_period: ${SERVER_GRACEFUL_SHUTDOWN_TIMEOUT:-10}s
    healthcheck:
      test: wget -q --spider -T 10 -O /dev/null http://localhost:${PORT:-3001}/health
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

Use Configuration to Distinguish Master and Slave Nodes

IS_SLAVE=true

Master node: not configured or set to IS_SLAVE=false. Slave node: set to IS_SLAVE=true.

Ensure only one master node in the entire cluster.

3

Configure Load Balancing

Use Nginx or other load balancers to distribute requests to different slave nodes

Nginx Configuration Example

# nginx.conf
upstream gpt_load_cluster {
    server master-node-ip:3001 weight=3;
    server slave-node1-ip:3001 weight=5;
    server slave-node2-ip:3001 weight=5;
    # Add more slave nodes as needed
}

server {
    listen 80;
    server_name your-domain.com;

    location / {
        proxy_pass http://gpt_load_cluster;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # If using CF or other proxy services, set real IP
        set_real_ip_from 0.0.0.0/0;
        real_ip_header X-Forwarded-For;
    }
}

Configuration Management

Unified Configuration Requirements

AUTH_KEYMust be the same (management key)
DATABASE_DSNMust be the same
REDIS_DSNMust be the same
IS_SLAVESet to true for slave nodes

Dynamic Configuration Synchronization

  • • System settings stored in MySQL auto-sync
  • • Group configurations pushed to all nodes in real-time
  • • Key status shared through Redis in real-time
  • • Configuration changes require no service restart

Monitoring and Operations

Health Check

Node Health Status

# Check node status
curl http://node:3001/health

# Response example
{
  "status": "healthy",
  "timestamp": "2025-07-15T12:56:00Z",
  "uptime": "11.822967ms"
}

Cluster Status Monitoring

  • • Monitor request volume and response time of each node
  • • Check database and Redis connection status

Scaling and Capacity Management

Horizontal Scaling

  • • Add slave nodes as needed
  • • Update load balancer configuration
  • • New nodes automatically sync configuration
  • • Seamlessly integrate with existing services

Graceful Shutdown

  • • Remove node from load balancer
  • • Wait for existing requests to complete
  • • Send stop signal to container
  • • Clean up related resources and logs

Best Practices

✅ Recommended Practices

  • • Use container orchestration tools (Docker Swarm/K8s)
  • • Configure high availability for database and Redis
  • • Set up comprehensive monitoring and alerting systems
  • • Regular backup of configurations and data
  • • Use configuration management tools for unified deployment
  • • Implement blue-green deployment or rolling updates

❌ Practices to Avoid

  • • Single point of failure for database or cache
  • • Inconsistent configuration between nodes
  • • Ignoring network latency and partition issues
  • • Lack of monitoring and log collection
  • • Manual management of large numbers of node configurations
  • • Direct modification of production environment configuration