Skip to main content

Load Balancer Component

The Load Balancer component distributes incoming traffic across multiple API Service instances, simulating how real-world systems scale to handle increased load and ensure high availability.

Overview​

In production systems, a single server can only handle so many requests before it becomes overwhelmed. Load balancers solve this by distributing traffic across multiple servers, ensuring no single server becomes a bottleneck.

Real-World Example

Think of a restaurant with one cashier vs. multiple cashiers:

  • One cashier (no load balancer): Long line, slow service, frustrated customers
  • Multiple cashiers with a host (load balancer): Host directs customers to available cashiers, faster service, happy customers

The load balancer is the "host" that decides which API Service (cashier) handles each request.

When to Use Load Balancer​

Use Load Balancer when you want to:

  • ✅ Simulate traffic distribution across multiple servers
  • ✅ Demonstrate horizontal scaling concepts
  • ✅ Test different load balancing algorithms
  • ✅ Show how systems handle increased load
  • ✅ Design high-availability architectures

You don't need Load Balancer if:

  • ❌ You only have one API Service
  • ❌ You're building a simple proof-of-concept
  • ❌ Your system doesn't need to scale horizontally

How Load Balancer Works​

Basic Flow​

User Request → Load Balancer → API Service #1 → Database
                             → API Service #2 → Database
                             → API Service #3 → Database
  1. User Request sends a request to the Load Balancer
  2. Load Balancer selects an API Service using its algorithm
  3. Selected API Service processes the request
  4. Database stores or retrieves data
  5. Response flows back through the same path

Architecture Example​

┌─────────────────┐
│  User Request   │
│   POST /users   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Load Balancer   │
│   (Random)      │
└────┬─────┬──────┘
     │     │
     │     └──────────┐
     │                │
     ▼                ▼
┌─────────┐      ┌─────────┐
│API Svc 1│      │API Svc 2│
│(Validate│      │(Validate│
│ Email)  │      │ Email)  │
└────┬────┘      └────┬────┘
     │                │
     └────────┬───────┘
              ▼
       ┌──────────┐
       │ Database │
       └──────────┘

Component Configuration​

Balancing Method​

Currently available: Random Selection

How it works:

  • Each request randomly selects one of the connected API Services
  • Every API Service has an equal chance of being selected
  • Simple but effective for demonstrating load distribution
info

Currently, Random Selection is the only available algorithm. We're exploring additional balancing strategies to enhance the learning experience.

Connected Servers​

The number of connected servers is automatically detected based on your connections:

  • 1 connection = Load Balancer forwards to 1 API Service
  • 3 connections = Load Balancer distributes across 3 API Services
  • 5 connections = Load Balancer distributes across 5 API Services
tip

The more API Services you connect, the better you can demonstrate horizontal scaling and traffic distribution patterns.

Setting Up Load Balancer​

Step 1: Add Components​

1. Add User Request component
2. Add Load Balancer component
3. Add multiple API Service components (2 or more)
4. Add Database component

Step 2: Create Connections​

User Request → Load Balancer
Load Balancer → API Service #1
Load Balancer → API Service #2
Load Balancer → API Service #3
API Service #1 → Database
API Service #2 → Database
API Service #3 → Database

Step 3: Configure Each API Service (Optional)​

You can give each API Service different code to see how the load balancer distributes work:

API Service #1: Tags requests as Server 1

def process_request(input_data):
    data = input_data.get('data', {})

    return {
        'operation': 'INSERT',
        'columns': [
            {'name': 'server', 'type': 'TEXT'},
            {'name': 'name', 'type': 'TEXT'}
        ],
        'data': {
            'server': 'Server-1',
            'name': data.get('name')
        }
    }

API Service #2: Tags requests as Server 2

def process_request(input_data):
    data = input_data.get('data', {})

    return {
        'operation': 'INSERT',
        'columns': [
            {'name': 'server', 'type': 'TEXT'},
            {'name': 'name', 'type': 'TEXT'}
        ],
        'data': {
            'server': 'Server-2',
            'name': data.get('name')
        }
    }

Step 4: Run Multiple Simulations​

Run your simulation multiple times and observe which API Service handles each request.

Simulation Behavior​

What You'll See in Logs​

When you run a simulation with a Load Balancer, the execution logs show:

🟡 Load Balancer: LB-1
ℹ️  Algorithm: random
ℹ️  Available Servers: 3
ℹ️          [1] API-1
ℹ️          [2] API-2
ℹ️          [3] API-3
ℹ️  Status: Selecting server...
✅ Selected: API-2
ℹ️  Flow: LB-1 --> API-2

Visual Feedback​

During simulation:

  1. Load Balancer component highlights when active
  2. Connection to selected API Service highlights
  3. Selected API Service component highlights
  4. Connection to Database highlights

This helps you visualize the request path through your system.

Load Balancing Algorithms Explained​

Random Selection (Currently Available)​

How it works:

Available servers: [API-1, API-2, API-3]
Request 1: Random(0-2) = 1 → API-2
Request 2: Random(0-2) = 0 → API-1
Request 3: Random(0-2) = 2 → API-3
Request 4: Random(0-2) = 1 → API-2

Characteristics:

  • ✅ Simple to understand and implement
  • ✅ Evenly distributes over many requests
  • ✅ No server state tracking needed
  • ⚠️ Can be uneven over small request counts
  • ⚠️ Doesn't consider server load or health

Best for:

  • Learning load balancing concepts
  • Stateless applications
  • When all servers have equal capacity

Example Distribution (10 requests):

API-1: ███ (3 requests)
API-2: ████ (4 requests)
API-3: ███ (3 requests)

Practical Examples​

Example 1: Basic Load Distribution​

Goal: Show how traffic distributes across servers

Setup:

User Request (POST /users)
  ↓
Load Balancer (Random)
  ↓ ↓ ↓
API-1  API-2  API-3
  ↓ ↓ ↓
Database

API Service Code (All servers):

def process_request(input_data):
    data = input_data.get('data', {})

    return {
        'operation': 'INSERT',
        'columns': [
            {'name': 'name', 'type': 'TEXT'},
            {'name': 'email', 'type': 'TEXT'}
        ],
        'data': data
    }

Test:

  1. Run simulation 10 times
  2. Check database to see which server handled each request
  3. Observe roughly even distribution

Example 2: Server-Tagged Requests​

Goal: Track which server processed each request

Setup: Same as Example 1

API Service #1 Code:

def process_request(input_data):
    data = input_data.get('data', {})

    return {
        'operation': 'INSERT',
        'columns': [
            {'name': 'server_id', 'type': 'TEXT'},
            {'name': 'name', 'type': 'TEXT'},
            {'name': 'processed_at', 'type': 'TEXT'}
        ],
        'data': {
            'server_id': 'server-1',
            'name': data.get('name'),
            'processed_at': '2024-01-15T10:00:00Z'
        }
    }

API Service #2 Code:

def process_request(input_data):
    data = input_data.get('data', {})

    return {
        'operation': 'INSERT',
        'columns': [
            {'name': 'server_id', 'type': 'TEXT'},
            {'name': 'name', 'type': 'TEXT'},
            {'name': 'processed_at', 'type': 'TEXT'}
        ],
        'data': {
            'server_id': 'server-2',  # Different server ID
            'name': data.get('name'),
            'processed_at': '2024-01-15T10:00:00Z'
        }
    }

Result: Query database to see server distribution:

server-1: 4 requests
server-2: 6 requests

Example 3: Simulating Server Failure​

Goal: Show what happens when one server is unavailable

Setup:

User Request
  ↓
Load Balancer
  ↓ ↓
API-1  API-2 (disconnected)
  ↓
Database

What happens:

  • Load Balancer only sees 1 connected server (API-1)
  • All requests route to API-1
  • System continues working despite "server failure"
Real-World Parallel

This demonstrates how load balancers provide high availability. If one server goes down, traffic automatically routes to healthy servers.

Current Limitations​

Single Request Mode

Currently, you can only send one request at a time, which limits the effectiveness of load balancing demonstrations. Each simulation shows one request being routed to one server.

We're exploring ways to better demonstrate load distribution patterns in future updates.

Best Practices​

  1. Connect Multiple API Services - At least 2-3 to demonstrate distribution
  2. Use Same Code for Realism - In production, all API Service instances run identical code. Use different code per server only if you want to explicitly track which server handled each request for demonstration purposes.
  3. Run Multiple Times - Random selection needs multiple runs to show distribution
  4. Document Your Setup - Use component descriptions to explain your architecture
  5. Start Simple - Begin with 2 servers, then scale up

Use Cases​

Learning System Design​

  • Understand horizontal scaling
  • Compare load balancing strategies
  • Demonstrate high availability

Interview Preparation​

  • Show how you'd scale a system
  • Discuss trade-offs between algorithms
  • Explain when to use load balancing

Architecture Planning​

  • Model your production system
  • Test different server configurations
  • Visualize traffic flow

Status Information​

The Load Balancer component displays:

Type: Load Balancer
Method: Random Selection (or your selected algorithm)
Servers: Number of connected API Services (auto-detected)

Integration with Other Components​

With User Request​

Connection:

User Request → Load Balancer

The User Request sends its HTTP request to the Load Balancer, which then decides which API Service should handle it.

With API Services​

Connection:

Load Balancer → API Service #1
Load Balancer → API Service #2
Load Balancer → API Service #3

Each connection represents an available server that can handle requests. The more connections, the better the load distribution.

With Database​

Indirect Connection:

Load Balancer → API Services → Database

The Load Balancer doesn't connect directly to the Database. Instead, it routes to API Services, which then interact with the Database.

Troubleshooting​

"No API Services connected"​

  • Cause: Load Balancer has no outgoing connections
  • Solution: Connect Load Balancer to at least one API Service component

All requests go to the same server​

  • Cause: Only ran simulation once, or only one server is connected
  • Solution: Run simulation multiple times with multiple API Services connected

Can't see load distribution​

  • Cause: Single request mode makes it hard to observe distribution patterns
  • Solution: Run simulation 10+ times and track results, or wait for batch request feature

Load Balancer shows 0 servers​

  • Cause: No connections from Load Balancer to API Services
  • Solution: Draw connections from Load Balancer to your API Service components

Future Enhancements​

We're taking a measured approach to building this platform, focusing on delivering features that truly enhance the learning experience. As we continue to develop the Load Balancer component, we're exploring ways to make simulations more realistic and educational.

We're committed to steady improvement while maintaining the quality and reliability of existing features.

Next Steps​