Comparing the Response Time of Three Simple Queueing Systems

A shiny queue calculator

Tim Wise

A Quick Introduction to Queueing Theory

Queueing theory is the mathematical study of customers waiting in lines for services. A queueing system can be as simple as a single server, like waiting to get cash from an ATM machine. Or it can be a complex network of servers, like components flowing through a factory assembly line.

The simplest queue is a single server with a waiting line and called an M/M/1 queue. Customers arrive in the queue at rate \(\lambda\) and the server handles them at rate \(\mu\).

A generalization is an M/M/c queue that has a single line feeding C servers. The next person in line is served by the next available server. It's like a load balancer.

We're interested in comparing the response times of the two queueing systems for different values of \(C\).

Metrics of M/M/1 Queue and an Example

Typical configuration metrics of an M/M/1 queue are:

  • \(\lambda\), arrival rate of customers
  • \(S\), service time for a single customer

from those we can compute:

  • max server rate, or capacity, \(\mu = 1 / S\)
  • server utilization, \(\rho = \lambda / \mu\)
  • customer response time, \(R = 1 / (\mu - \lambda)\)

A queueing system is in steady-state when the arrival rate is less than the capacity of the server. For an M/M/1 queue, when \(\lambda < \mu \equiv \rho < 1\).

lambda <- 50    # tx per sec
S <- 0.010      # sec per tx
mu <- 1 / S; mu # max tx per sec

## [1] 100

# server utilization 
rho <- lambda / mu; percent(rho) 

## Error in eval(expr, envir, enclos): could not find function "percent"

# tx response time sec
R <- 1 / (mu - lambda); R        

## [1] 0.02

Consider Three Equivalent Queueing Systems

From Developing Our Intuition About Queuing Network Models:

Grocery Store Bank Teller Super Server
A set of six parallel M/M/1 queues An M/M/c queue with 6 servers An M/M/1 queue that is \(6x\) faster
each with an arrival rate \(\lambda\) with an arrival rate of \(6\lambda\) with an arrival rate of \(6\lambda\)
and each server works at rate \(\mu\) and each server works at rate \(\mu\) and the server works at rate \(6\mu\)

Which Queue Provides the Fastest Response Time?

Question 1

We've seen that response time of a queue is a function of the customer arrival rate and the service time. Which of the 3 queueing systems do you think provides the fastest response time under light load, i.e., slow customer arrival rate?

  1. Grocery Store
  2. Bank Teller
  3. Super Server

Hint: Under light load there is no waiting time. Response time is just the service time. Which system has the fastest server?

The Super Server is 6 times as fast as the other servers. So its service time \(1/6\)-th that of the other systems. Under low customer arrival rate, there is no wait time and the response time is just the service time. So the Super Server has the fastest response time under low load.

Question 2

Now, which system do you think provides better response time under very high load? To find out, go use our simple shiny app queue calculator.