All articles

Model Musings

Modelling Servers in SimPy: From Single to Multiple

Published
Reading time
6 min

Servers are the workhorses of simulation. Tills, machines, staff, computers: anything that processes entities and takes time.

The Single Server

One resource, one capacity:

import simpy
import random

def customer(env, name, server):
    arrival = env.now
    print(f"{name} arrives at {arrival:.2f}")

    with server.request() as req:
        yield req
        wait = env.now - arrival
        print(f"{name} starts service at {env.now:.2f} (waited {wait:.2f})")

        service_time = random.expovariate(1/5)
        yield env.timeout(service_time)

    print(f"{name} leaves at {env.now:.2f}")

env = simpy.Environment()
server = simpy.Resource(env, capacity=1)

Classic M/M/1 queue territory.

Multiple Parallel Servers

Increase capacity:

# 3 identical parallel servers
servers = simpy.Resource(env, capacity=3)

Now up to 3 customers can be served simultaneously. Think: multiple tills at a supermarket.

Multiple Server Types

Different servers for different tasks:

class ServicePoint:
    def __init__(self, env, name, capacity, service_rate):
        self.name = name
        self.resource = simpy.Resource(env, capacity=capacity)
        self.service_rate = service_rate

    def serve(self, env, customer_name):
        with self.resource.request() as req:
            yield req
            service_time = random.expovariate(self.service_rate)
            yield env.timeout(service_time)

# Different service points
check_in = ServicePoint(env, "Check-in", capacity=4, service_rate=1/3)
security = ServicePoint(env, "Security", capacity=2, service_rate=1/5)
boarding = ServicePoint(env, "Boarding", capacity=1, service_rate=1/2)

def passenger(env, name):
    yield from check_in.serve(env, name)
    yield from security.serve(env, name)
    yield from boarding.serve(env, name)

Server with Setup Time

Some servers need setup between jobs:

def server_with_setup(env, name, job_queue, setup_time, process_time):
    last_job_type = None

    while True:
        job = yield job_queue.get()

        # Setup if job type changed
        if job['type'] != last_job_type:
            print(f"{name} setting up for {job['type']}")
            yield env.timeout(setup_time)
            last_job_type = job['type']

        # Process
        yield env.timeout(process_time)
        print(f"{name} completed job {job['id']}")

Server with Breaks

Staff take breaks, and the first instinct is to shrink the resource while they are away:

resource.capacity -= 1  # Raises AttributeError

SimPy will not have it. capacity is a read-only property, so that line stops the simulation with an AttributeError the moment the first break begins.

How, then, do you take a server away? You let the break queue for a server like everybody else, with one privilege: it goes to the front. Model the staff as a PriorityResource and have a break process request a server at a higher priority than any customer, then hold it for the length of the break:

import simpy

BREAK, CUSTOMER = 0, 1  # Lower number = higher priority

def customer(env, name, staff):
    arrived = env.now
    with staff.request(priority=CUSTOMER) as req:
        yield req
        print(f"{env.now:4.0f}: {name} served (waited {env.now - arrived:.0f})")
        yield env.timeout(5)

def arrivals(env, staff):
    for i in range(1, 8):
        env.process(customer(env, f"Customer {i}", staff))
        yield env.timeout(2)

def breaks(env, staff, work_duration, break_duration):
    while True:
        yield env.timeout(work_duration)
        print(f"{env.now:4.0f}: a break is due")
        with staff.request(priority=BREAK) as req:
            yield req  # Waits for a server to finish their customer
            print(f"{env.now:4.0f}: a server goes on break")
            yield env.timeout(break_duration)
        print(f"{env.now:4.0f}: the server is back")

env = simpy.Environment()
staff = simpy.PriorityResource(env, capacity=2)
env.process(arrivals(env, staff))
env.process(breaks(env, staff, work_duration=9, break_duration=6))
env.run(until=20)

Output:

   0: Customer 1 served (waited 0)
   2: Customer 2 served (waited 0)
   5: Customer 3 served (waited 1)
   7: Customer 4 served (waited 1)
   9: a break is due
  10: a server goes on break
  12: Customer 5 served (waited 4)
  16: the server is back
  16: Customer 6 served (waited 6)
  17: Customer 7 served (waited 5)

Read the log as a story. Customers arrive every 2 minutes and take 5 to serve, so the counter is already stretched. The break falls due at 9 while both servers are busy, so it waits for Customer 3 to finish at 10 rather than abandoning them mid-transaction. Customer 5 has been queueing since 8, yet the break goes first, because priority 0 beats priority 1; give the break the customers’ priority and Customer 5 is served at 10 while the break slips to 12. For six minutes the counter runs on one server and the waits climb; at 16 the with block hands the server back and normal service resumes. One break process takes one server at a time, so start one per member of staff (staggered, as any rota would be) if each needs a break of their own.

If the break cannot wait, as with a machine that fails mid-job rather than a teller who finishes the transaction, use a PreemptiveResource and request it with preempt=True. The customer being served then receives an Interrupt and must handle it; PreemptiveResource Explained walks through that pattern.

Server Pool with Load Balancing

Route to least busy server:

class ServerPool:
    def __init__(self, env, num_servers, service_rate):
        self.env = env
        self.servers = [
            simpy.Resource(env, capacity=1)
            for _ in range(num_servers)
        ]
        self.service_rate = service_rate

    def get_least_busy(self):
        """Return the server with the fewest customers, queueing or in service."""
        return min(self.servers, key=lambda s: len(s.queue) + s.count)

    def serve(self, customer_name):
        server = self.get_least_busy()
        with server.request() as req:
            yield req
            yield self.env.timeout(random.expovariate(self.service_rate))

pool = ServerPool(env, num_servers=3, service_rate=1/5)

def customer(env, name, pool):
    yield from pool.serve(name)

Server Utilisation Tracking

class TrackedServer:
    def __init__(self, env, capacity):
        self.env = env
        self.resource = simpy.Resource(env, capacity=capacity)
        self.busy_time = 0
        self.last_change = 0
        self.last_count = 0

    def request(self):
        return TrackedRequest(self)

    def update_stats(self):
        elapsed = self.env.now - self.last_change
        self.busy_time += self.last_count * elapsed
        self.last_change = self.env.now
        self.last_count = self.resource.count

    @property
    def utilisation(self):
        self.update_stats()
        if self.env.now == 0:
            return 0
        return self.busy_time / (self.env.now * self.resource.capacity)

class TrackedRequest:
    def __init__(self, tracked_server):
        self.server = tracked_server
        self.request = None

    def __enter__(self):
        self.request = self.server.resource.request()
        return self.request.__enter__()

    def __exit__(self, *args):
        self.server.update_stats()
        return self.request.__exit__(*args)

Server States

Model server states explicitly:

from enum import Enum

class ServerState(Enum):
    IDLE = "idle"
    BUSY = "busy"
    BREAKDOWN = "breakdown"
    MAINTENANCE = "maintenance"

class StatefulServer:
    def __init__(self, env, name):
        self.env = env
        self.name = name
        self.state = ServerState.IDLE
        self.resource = simpy.Resource(env, capacity=1)

    def process_job(self, job):
        self.state = ServerState.BUSY
        with self.resource.request() as req:
            yield req
            yield self.env.timeout(job['duration'])
        self.state = ServerState.IDLE

    def breakdown(self, repair_time):
        self.state = ServerState.BREAKDOWN
        yield self.env.timeout(repair_time)
        self.state = ServerState.IDLE

Speed-Dependent Servers

Faster service for higher utilisation:

def adaptive_server(env, queue, base_rate, speedup_threshold=5):
    while True:
        job = yield queue.get()

        # Speed up if queue is long
        if len(queue.items) > speedup_threshold:
            service_time = random.expovariate(base_rate * 1.5)  # 50% faster
        else:
            service_time = random.expovariate(base_rate)

        yield env.timeout(service_time)

Batch Servers

Process multiple items at once:

def batch_server(env, queue, batch_size, process_time):
    while True:
        batch = []

        # Collect batch
        while len(batch) < batch_size:
            item = yield queue.get()
            batch.append(item)

        # Process entire batch at once
        yield env.timeout(process_time)
        print(f"Processed batch of {len(batch)} at {env.now}")

Complete Example: Multi-Server System

import simpy
import random

class Server:
    def __init__(self, env, name, service_rate):
        self.env = env
        self.name = name
        self.service_rate = service_rate
        self.resource = simpy.Resource(env, capacity=1)
        self.customers_served = 0
        self.total_service_time = 0

    def serve(self, customer_id):
        with self.resource.request() as req:
            yield req
            service_time = random.expovariate(self.service_rate)
            yield self.env.timeout(service_time)
            self.customers_served += 1
            self.total_service_time += service_time
            return service_time

    @property
    def utilisation(self):
        if self.env.now == 0:
            return 0
        return self.total_service_time / self.env.now

def customer(env, customer_id, servers):
    arrival = env.now

    # Choose the server with the fewest customers (queue length alone
    # would send people to a busy server while another sits idle)
    server = min(servers, key=lambda s: len(s.resource.queue) + s.resource.count)

    service_time = yield from server.serve(customer_id)

    return {
        'id': customer_id,
        'arrival': arrival,
        'departure': env.now,
        'server': server.name,
        'wait': env.now - arrival - service_time,
        'service': service_time
    }

def arrivals(env, servers, results, arrival_rate):
    i = 0
    while True:
        yield env.timeout(random.expovariate(arrival_rate))
        result = env.process(customer(env, i, servers))
        i += 1

# Run simulation
random.seed(42)
env = simpy.Environment()
servers = [Server(env, f"Server{i}", 1/4) for i in range(3)]
results = []

env.process(arrivals(env, servers, results, arrival_rate=1/2))
env.run(until=1000)

# Report
for s in servers:
    print(f"{s.name}: served {s.customers_served}, utilisation {s.utilisation:.1%}")

Summary

Server modelling:

  • Single server: Resource(env, capacity=1)
  • Multiple parallel: Resource(env, capacity=n)
  • Different types: separate Resource instances
  • Track utilisation, states and performance

The server is the atom of simulation. Master it.

Next Steps

Ready to build a model of your own?

The free SimPy Handbook covers the foundations; the Simulation Bootcamp covers everything else, from first principles to models a business will act on.