Branch and Bound Algorithms

Branch and Bound (B&B) is a powerful exact algorithmic paradigm used to solve combinatorial optimization problems such as the Traveling Salesman Problem (TSP), Knapsack, Integer Programming, and many scheduling challenges. This tutorial walks you through the theory, practical implementation steps, and best practices, with code examples in Python and C++.

Table of Contents

  • Introduction to Branch and Bound
  • Core Concepts: Branching, Bounding, Pruning
  • Algorithmic Framework
  • Complexity Analysis
  • Step‑by‑Step Implementation (Python & C++)
  • Common Use Cases
  • Advantages & Limitations
  • Tips, Warnings, and Best Practices
  • Frequently Asked Questions
  • Quiz
  • References

1. Introduction to Branch and Bound

Branch and Bound is an exact search method that systematically explores the solution space of an optimization problem. Unlike heuristic methods, B&B guarantees an optimal solution if one exists, by partitioning the search space (branching) and discarding sub‑spaces that cannot contain a better solution (bounding).

Why Use Branch and Bound?

  • Provides provable optimality
  • Can exploit problem‑specific bounds for massive pruning
  • Works well when the search tree is sparse or when good bounds are easy to compute

2. Core Concepts

2.1 Branching

Branching splits a problem into smaller sub‑problems (nodes) by fixing decisions. For example, in the 0/1 Knapsack problem you can branch on whether to include a particular item.

2.2 Bounding

Bounding computes a lower (for minimization) or upper (for maximization) bound on the best possible solution within a node. Common bounding techniques include:

  • Relaxation (e.g., linear programming relaxation)
  • Heuristic estimates (e.g., greedy fill)
  • Problem‑specific inequalities

2.3 Pruning

If a node's bound is worse than the best solution found so far (the incumbent), the node is discarded. This step dramatically reduces the number of explored nodes.

3. Algorithmic Framework

  1. Initialize the incumbent solution (often using a heuristic).
  2. Create a priority queue (or stack) containing the root node.
  3. While the queue is not empty:
  4.  a) Extract the next node (best‑first, depth‑first, or breadth‑first).
  5.  b) Compute a bound for the node.
  6.  c) If the bound is worse than the incumbent → prune.
  7.  d) If the node represents a complete solution → update incumbent if better.
  8.  e) Otherwise, branch the node into child sub‑problems and push them onto the queue.

4. Complexity Analysis

In the worst case, B&B explores the entire exponential search tree, yielding O(b^d) time where b is the branching factor and d is the depth. However, effective bounding typically reduces the explored nodes dramatically, often giving practical runtimes far below the worst case.

5. Implementation Walkthrough

5.1 Python Example – 0/1 Knapsack

The following Python code demonstrates a simple B&B solver for the 0/1 Knapsack problem using a linear‑programming relaxation for bounding.

import heapq
from collections import namedtuple

Item = namedtuple('Item', ['value', 'weight'])

class Node:
    def __init__(self, level, profit, weight, bound, taken):
        self.level = level          # index of the item considered
        self.profit = profit        # profit so far
        self.weight = weight        # weight so far
        self.bound = bound          # upper bound of profit reachable
        self.taken = taken[:]       # list of booleans indicating taken items
    def __lt__(self, other):
        # heapq is a min‑heap; we want max‑bound first
        return self.bound > other.bound

def bound(node, capacity, items):
    if node.weight >= capacity:
        return 0
    profit_bound = node.profit
    j = node.level + 1
    totweight = node.weight
    # take items fractionally (fractional knapsack) to compute upper bound
    while j < len(items) and totweight + items[j].weight <= capacity:
        totweight += items[j].weight
        profit_bound += items[j].value
        j += 1
    if j < len(items):
        profit_bound += (capacity - totweight) * items[j].value / items[j].weight
    return profit_bound

def branch_and_bound_knapsack(items, capacity):
    # sort items by value/weight decreasing (helps bound quality)
    items = sorted(items, key=lambda i: i.value / i.weight, reverse=True)
    Q = []
    root = Node(-1, 0, 0, 0, [False]*len(items))
    root.bound = bound(root, capacity, items)
    heapq.heappush(Q, root)
    max_profit = 0
    best_taken = []
    while Q:
        node = heapq.heappop(Q)
        if node.bound <= max_profit:
            continue  # prune
        if node.level == len(items) - 1:
            continue
        # Branch: take next item
        nxt = node.level + 1
        taken_node = Node(nxt, node.profit + items[nxt].value,
                          node.weight + items[nxt].weight,
                          0, node.taken)
        taken_node.taken[nxt] = True
        if taken_node.weight <= capacity and taken_node.profit > max_profit:
            max_profit = taken_node.profit
            best_taken = taken_node.taken[:]
        taken_node.bound = bound(taken_node, capacity, items)
        if taken_node.bound > max_profit:
            heapq.heappush(Q, taken_node)
        # Branch: do NOT take next item
        not_taken_node = Node(nxt, node.profit, node.weight, 0, node.taken)
        not_taken_node.bound = bound(not_taken_node, capacity, items)
        if not_taken_node.bound > max_profit:
            heapq.heappush(Q, not_taken_node)
    return max_profit, best_taken

# Example usage
items = [Item(60, 10), Item(100, 20), Item(120, 30)]
capacity = 50
profit, selection = branch_and_bound_knapsack(items, capacity)
print(f"Maximum profit: {profit}")
print(f"Items taken: {selection}")

5.2 C++ Example – Traveling Salesman Problem (TSP)

Below is a concise C++ implementation that uses B&B with a lower‑bound based on the Minimum Spanning Tree (MST) of the remaining cities.

#include <bits/stdc++.h>
using namespace std;

struct Node {
    vector<int> path;      // visited cities in order
    double cost;           // cost so far
    double bound;          // lower bound (cost + MST of unvisited)
    vector<bool> visited; // city visited flags
    bool operator<(const Node& other) const {
        // priority_queue is max‑heap; we need smallest bound first
        return bound > other.bound;
    }
};

// Compute Euclidean distance matrix
vector<vector<double>> distanceMatrix(const vector<pair<double,double>>& pts) {
    int n = pts.size();
    vector<vector<double>> d(n, vector<double>(n, 0));
    for (int i=0;i<n;i++)
        for (int j=0;j<n;j++)
            d[i][j] = hypot(pts[i].first-pts[j].first, pts[i].second-pts[j].second);
    return d;
}

// Simple Prim's algorithm to compute MST weight of a set of vertices
double mstWeight(const vector<int>& vertices, const vector<vector<double>>& dist) {
    if (vertices.empty()) return 0.0;
    unordered_set<int> inMST;
    unordered_set<int> notInMST(vertices.begin(), vertices.end());
    int start = *notInMST.begin();
    inMST.insert(start);
    notInMST.erase(start);
    double total = 0.0;
    while (!notInMST.empty()) {
        double best = numeric_limits<double>::infinity();
        int bestV = -1;
        for (int u : inMST) {
            for (int v : notInMST) {
                if (dist[u][v] < best) {
                    best = dist[u][v];
                    bestV = v;
                }
            }
        }
        total += best;
        inMST.insert(bestV);
        notInMST.erase(bestV);
    }
    return total;
}

double lowerBound(const Node& node, const vector<vector<double>>& dist, int n) {
    // cost so far + MST of remaining + return to start
    vector<int> remaining;
    for (int i=0;i<n;i++) if (!node.visited[i]) remaining.push_back(i);
    double mst = mstWeight(remaining, dist);
    double minReturn = numeric_limits<double>::infinity();
    if (!node.path.empty()) {
        int last = node.path.back();
        for (int v : remaining) minReturn = min(minReturn, dist[last][v]);
    } else {
        minReturn = 0; // root node
    }
    double minToStart = numeric_limits<double>::infinity();
    for (int v : remaining) minToStart = min(minToStart, dist[v][0]);
    return node.cost + mst + minReturn + minToStart;
}

pair<double, vector<int>> branchAndBoundTSP(const vector<pair<double,double>>& points) {
    int n = points.size();
    auto dist = distanceMatrix(points);
    priority_queue<Node> pq;
    Node root;
    root.path = {0};
    root.cost = 0;
    root.visited = vector<bool>(n,false);
    root.visited[0]=true;
    root.bound = lowerBound(root, dist, n);
    pq.push(root);
    double bestCost = numeric_limits<double>::infinity();
    vector<int> bestPath;
    while (!pq.empty()) {
        Node cur = pq.top(); pq.pop();
        if (cur.bound >= bestCost) continue; // prune
        if (cur.path.size() == n) {
            // close the tour
            double tourCost = cur.cost + dist[cur.path.back()][0];
            if (tourCost < bestCost) {
                bestCost = tourCost;
                bestPath = cur.path;
                bestPath.push_back(0);
            }
            continue;
        }
        int last = cur.path.back();
        for (int nxt=0; nxt<n; ++nxt) if (!cur.visited[nxt]) {
            Node child = cur;
            child.path.push_back(nxt);
            child.cost += dist[last][nxt];
            child.visited[nxt]=true;
            child.bound = lowerBound(child, dist, n);
            if (child.bound < bestCost) pq.push(child);
        }
    }
    return {bestCost, bestPath};
}

int main(){
    vector<pair<double,double>> cities = {{0,0},{2,3},{5,4},{1,6},{7,2}}; // sample points
    auto [cost, tour] = branchAndBoundTSP(cities);
    cout << "Optimal tour cost: " << cost << "\nPath: ";
    for (int v: tour) cout << v << " ";
    cout << endl;
    return 0;
}

6. Common Use Cases

  • Traveling Salesman Problem (TSP)
  • 0/1 Knapsack and Multi‑dimensional Knapsack
  • Job Shop Scheduling
  • Vehicle Routing Problem
  • Integer Linear Programming (as a core component of branch‑and‑cut)
  • Maximum Clique and Graph Coloring

7. Advantages & Limitations

AspectAdvantageLimitation
OptimalityGuarantees global optimumMay require exponential time in worst case
FlexibilityCan incorporate problem‑specific boundsDesigning tight bounds can be complex
ScalabilityEffective on medium‑size instances (hundreds of variables)Large‑scale instances often need hybrid methods
ImplementationConceptually straightforwardMemory overhead for storing tree nodes

8. Tips for Effective Branch and Bound

💡 Tip: Sort variables/items by a heuristic that improves bound quality (e.g., profit/weight ratio for knapsack).
💡 Tip: Use a strong initial incumbent from a fast heuristic; a good incumbent dramatically improves pruning.
💡 Tip: Select the node‑selection strategy that fits your problem: best‑first for tighter bounds, depth‑first for low memory usage.
⚠ Warning: Avoid overly aggressive bounding that may discard optimal solutions due to rounding errors; always use safe (conservative) bounds.
📝 Note: Branch and Bound is often combined with other techniques such as cutting planes (branch‑and‑cut) or column generation for even larger integer programming problems.

9. Frequently Asked Questions

Q: Is Branch and Bound a heuristic?
A: No. B&B is an exact algorithm. Heuristics are sometimes used only to obtain an initial incumbent.


Q: What data structure is best for the node queue?
A: A priority queue (heap) for best‑first search, a stack for depth‑first, or a simple FIFO queue for breadth‑first. Choose based on memory and bound tightness.


Q: How does Branch and Bound differ from Branch and Cut?
A: Branch and Cut augments B&B with cutting planes that tighten the linear relaxation, often yielding far fewer nodes.


Q: Can B&B solve NP‑hard problems in polynomial time?
A: It cannot guarantee polynomial time for all instances. However, for many practical instances with strong bounds, it runs efficiently.


10. Quick Quiz

Q. Which of the following is NOT a typical component of a Branch and Bound algorithm?
  • Branching
  • Bounding
  • Pruning
  • Backpropagation

Answer: Backpropagation
Backpropagation is a term from neural networks; B&B uses branching, bounding, and pruning.

Q. In a minimization problem, a node can be pruned when its lower bound is:
  • Greater than the incumbent
  • Less than the incumbent
  • Equal to the incumbent
  • None of the above

Answer: Greater than the incumbent
If the best possible solution from that node (its lower bound) is already worse than the best known solution, the node cannot lead to an improvement.

11. References

References
📘 Summary: Branch and Bound is a versatile exact algorithmic framework that systematically explores decision spaces while discarding sub‑optimal regions through bounding. By integrating strong problem‑specific bounds, good initial incumbents, and appropriate search strategies, engineers can solve many NP‑hard problems efficiently in practice.