Branch and Bound (B&B) is a powerful exact algorithmic paradigm used to solve combinatorial optimization problems such as the Traveling Salesman Problem (TSP), Knapsack, Integer Programming, and many scheduling challenges. This tutorial walks you through the theory, practical implementation steps, and best practices, with code examples in Python and C++.
Table of Contents
- Introduction to Branch and Bound
- Core Concepts: Branching, Bounding, Pruning
- Algorithmic Framework
- Complexity Analysis
- Step‑by‑Step Implementation (Python & C++)
- Common Use Cases
- Advantages & Limitations
- Tips, Warnings, and Best Practices
- Frequently Asked Questions
- Quiz
- References
1. Introduction to Branch and Bound
Branch and Bound is an exact search method that systematically explores the solution space of an optimization problem. Unlike heuristic methods, B&B guarantees an optimal solution if one exists, by partitioning the search space (branching) and discarding sub‑spaces that cannot contain a better solution (bounding).
Why Use Branch and Bound?
- Provides provable optimality
- Can exploit problem‑specific bounds for massive pruning
- Works well when the search tree is sparse or when good bounds are easy to compute
2. Core Concepts
2.1 Branching
Branching splits a problem into smaller sub‑problems (nodes) by fixing decisions. For example, in the 0/1 Knapsack problem you can branch on whether to include a particular item.
2.2 Bounding
Bounding computes a lower (for minimization) or upper (for maximization) bound on the best possible solution within a node. Common bounding techniques include:
- Relaxation (e.g., linear programming relaxation)
- Heuristic estimates (e.g., greedy fill)
- Problem‑specific inequalities
2.3 Pruning
If a node's bound is worse than the best solution found so far (the incumbent), the node is discarded. This step dramatically reduces the number of explored nodes.
3. Algorithmic Framework
- Initialize the incumbent solution (often using a heuristic).
- Create a priority queue (or stack) containing the root node.
- While the queue is not empty:
- a) Extract the next node (best‑first, depth‑first, or breadth‑first).
- b) Compute a bound for the node.
- c) If the bound is worse than the incumbent → prune.
- d) If the node represents a complete solution → update incumbent if better.
- e) Otherwise, branch the node into child sub‑problems and push them onto the queue.
4. Complexity Analysis
In the worst case, B&B explores the entire exponential search tree, yielding O(b^d) time where b is the branching factor and d is the depth. However, effective bounding typically reduces the explored nodes dramatically, often giving practical runtimes far below the worst case.
5. Implementation Walkthrough
5.1 Python Example – 0/1 Knapsack
The following Python code demonstrates a simple B&B solver for the 0/1 Knapsack problem using a linear‑programming relaxation for bounding.
import heapq
from collections import namedtuple
Item = namedtuple('Item', ['value', 'weight'])
class Node:
def __init__(self, level, profit, weight, bound, taken):
self.level = level # index of the item considered
self.profit = profit # profit so far
self.weight = weight # weight so far
self.bound = bound # upper bound of profit reachable
self.taken = taken[:] # list of booleans indicating taken items
def __lt__(self, other):
# heapq is a min‑heap; we want max‑bound first
return self.bound > other.bound
def bound(node, capacity, items):
if node.weight >= capacity:
return 0
profit_bound = node.profit
j = node.level + 1
totweight = node.weight
# take items fractionally (fractional knapsack) to compute upper bound
while j < len(items) and totweight + items[j].weight <= capacity:
totweight += items[j].weight
profit_bound += items[j].value
j += 1
if j < len(items):
profit_bound += (capacity - totweight) * items[j].value / items[j].weight
return profit_bound
def branch_and_bound_knapsack(items, capacity):
# sort items by value/weight decreasing (helps bound quality)
items = sorted(items, key=lambda i: i.value / i.weight, reverse=True)
Q = []
root = Node(-1, 0, 0, 0, [False]*len(items))
root.bound = bound(root, capacity, items)
heapq.heappush(Q, root)
max_profit = 0
best_taken = []
while Q:
node = heapq.heappop(Q)
if node.bound <= max_profit:
continue # prune
if node.level == len(items) - 1:
continue
# Branch: take next item
nxt = node.level + 1
taken_node = Node(nxt, node.profit + items[nxt].value,
node.weight + items[nxt].weight,
0, node.taken)
taken_node.taken[nxt] = True
if taken_node.weight <= capacity and taken_node.profit > max_profit:
max_profit = taken_node.profit
best_taken = taken_node.taken[:]
taken_node.bound = bound(taken_node, capacity, items)
if taken_node.bound > max_profit:
heapq.heappush(Q, taken_node)
# Branch: do NOT take next item
not_taken_node = Node(nxt, node.profit, node.weight, 0, node.taken)
not_taken_node.bound = bound(not_taken_node, capacity, items)
if not_taken_node.bound > max_profit:
heapq.heappush(Q, not_taken_node)
return max_profit, best_taken
# Example usage
items = [Item(60, 10), Item(100, 20), Item(120, 30)]
capacity = 50
profit, selection = branch_and_bound_knapsack(items, capacity)
print(f"Maximum profit: {profit}")
print(f"Items taken: {selection}")
5.2 C++ Example – Traveling Salesman Problem (TSP)
Below is a concise C++ implementation that uses B&B with a lower‑bound based on the Minimum Spanning Tree (MST) of the remaining cities.
#include <bits/stdc++.h>
using namespace std;
struct Node {
vector<int> path; // visited cities in order
double cost; // cost so far
double bound; // lower bound (cost + MST of unvisited)
vector<bool> visited; // city visited flags
bool operator<(const Node& other) const {
// priority_queue is max‑heap; we need smallest bound first
return bound > other.bound;
}
};
// Compute Euclidean distance matrix
vector<vector<double>> distanceMatrix(const vector<pair<double,double>>& pts) {
int n = pts.size();
vector<vector<double>> d(n, vector<double>(n, 0));
for (int i=0;i<n;i++)
for (int j=0;j<n;j++)
d[i][j] = hypot(pts[i].first-pts[j].first, pts[i].second-pts[j].second);
return d;
}
// Simple Prim's algorithm to compute MST weight of a set of vertices
double mstWeight(const vector<int>& vertices, const vector<vector<double>>& dist) {
if (vertices.empty()) return 0.0;
unordered_set<int> inMST;
unordered_set<int> notInMST(vertices.begin(), vertices.end());
int start = *notInMST.begin();
inMST.insert(start);
notInMST.erase(start);
double total = 0.0;
while (!notInMST.empty()) {
double best = numeric_limits<double>::infinity();
int bestV = -1;
for (int u : inMST) {
for (int v : notInMST) {
if (dist[u][v] < best) {
best = dist[u][v];
bestV = v;
}
}
}
total += best;
inMST.insert(bestV);
notInMST.erase(bestV);
}
return total;
}
double lowerBound(const Node& node, const vector<vector<double>>& dist, int n) {
// cost so far + MST of remaining + return to start
vector<int> remaining;
for (int i=0;i<n;i++) if (!node.visited[i]) remaining.push_back(i);
double mst = mstWeight(remaining, dist);
double minReturn = numeric_limits<double>::infinity();
if (!node.path.empty()) {
int last = node.path.back();
for (int v : remaining) minReturn = min(minReturn, dist[last][v]);
} else {
minReturn = 0; // root node
}
double minToStart = numeric_limits<double>::infinity();
for (int v : remaining) minToStart = min(minToStart, dist[v][0]);
return node.cost + mst + minReturn + minToStart;
}
pair<double, vector<int>> branchAndBoundTSP(const vector<pair<double,double>>& points) {
int n = points.size();
auto dist = distanceMatrix(points);
priority_queue<Node> pq;
Node root;
root.path = {0};
root.cost = 0;
root.visited = vector<bool>(n,false);
root.visited[0]=true;
root.bound = lowerBound(root, dist, n);
pq.push(root);
double bestCost = numeric_limits<double>::infinity();
vector<int> bestPath;
while (!pq.empty()) {
Node cur = pq.top(); pq.pop();
if (cur.bound >= bestCost) continue; // prune
if (cur.path.size() == n) {
// close the tour
double tourCost = cur.cost + dist[cur.path.back()][0];
if (tourCost < bestCost) {
bestCost = tourCost;
bestPath = cur.path;
bestPath.push_back(0);
}
continue;
}
int last = cur.path.back();
for (int nxt=0; nxt<n; ++nxt) if (!cur.visited[nxt]) {
Node child = cur;
child.path.push_back(nxt);
child.cost += dist[last][nxt];
child.visited[nxt]=true;
child.bound = lowerBound(child, dist, n);
if (child.bound < bestCost) pq.push(child);
}
}
return {bestCost, bestPath};
}
int main(){
vector<pair<double,double>> cities = {{0,0},{2,3},{5,4},{1,6},{7,2}}; // sample points
auto [cost, tour] = branchAndBoundTSP(cities);
cout << "Optimal tour cost: " << cost << "\nPath: ";
for (int v: tour) cout << v << " ";
cout << endl;
return 0;
}
6. Common Use Cases
- Traveling Salesman Problem (TSP)
- 0/1 Knapsack and Multi‑dimensional Knapsack
- Job Shop Scheduling
- Vehicle Routing Problem
- Integer Linear Programming (as a core component of branch‑and‑cut)
- Maximum Clique and Graph Coloring
7. Advantages & Limitations
| Aspect | Advantage | Limitation |
|---|---|---|
| Optimality | Guarantees global optimum | May require exponential time in worst case |
| Flexibility | Can incorporate problem‑specific bounds | Designing tight bounds can be complex |
| Scalability | Effective on medium‑size instances (hundreds of variables) | Large‑scale instances often need hybrid methods |
| Implementation | Conceptually straightforward | Memory overhead for storing tree nodes |
8. Tips for Effective Branch and Bound
9. Frequently Asked Questions
Q: Is Branch and Bound a heuristic?
A: No. B&B is an exact algorithm. Heuristics are sometimes used only to obtain an initial incumbent.
Q: What data structure is best for the node queue?
A: A priority queue (heap) for best‑first search, a stack for depth‑first, or a simple FIFO queue for breadth‑first. Choose based on memory and bound tightness.
Q: How does Branch and Bound differ from Branch and Cut?
A: Branch and Cut augments B&B with cutting planes that tighten the linear relaxation, often yielding far fewer nodes.
Q: Can B&B solve NP‑hard problems in polynomial time?
A: It cannot guarantee polynomial time for all instances. However, for many practical instances with strong bounds, it runs efficiently.
10. Quick Quiz
Q. Which of the following is NOT a typical component of a Branch and Bound algorithm?
- Branching
- Bounding
- Pruning
- Backpropagation
Answer: Backpropagation
Backpropagation is a term from neural networks; B&B uses branching, bounding, and pruning.
Q. In a minimization problem, a node can be pruned when its lower bound is:
- Greater than the incumbent
- Less than the incumbent
- Equal to the incumbent
- None of the above
Answer: Greater than the incumbent
If the best possible solution from that node (its lower bound) is already worse than the best known solution, the node cannot lead to an improvement.