1. hobbies and interests
2. games
3. chess

Combinatorial Optimization at Work:
How to Find the Needle in the Haystack?
Martin Oellrich, T-Systems Nova GmbH, Berlin, Germany
martin.oellrich@t-systems.com
Abstract
Telecommunications network design most often aims at specified technical solutions at the lowest possible cost. Discrete
Mathematics can greatly help achieving this goal. The technical system is modelled in mathematical language and solved
by suitable methods. It lies in the nature of those problems that they have a vast multitude of possible solutions. Singling
out one with lowest cost among all of them is just like searching for a needle in a haystack.
As it is a typical example for a telecommunication design problem, we will work out step-by-step the solution process
of the Disjoint Routing Problem of Virtual Private Networks (VPN): a number of given demands must be routed through
a given network such that their paths do not share any common nodes or links, and cause minimal cost among all possible
solutions.
1
Introduction
A Virtual Private Network (VPN) is a network that appears
to be exclusively accessible and operated by a customer
company alone. In physical reality, it consists of a number of permanent lines that are leased from a carrier. So
control of its performance — in particular: its outages —
also lies with the latter. And for this reason, the carrier has
provide these lines subject to a reliability demand: one single failure in the underlying network may cause at most one
single failure on the VPN level, i.e., in the customer’s netFigure 2: Virtual Private Network
work. Technically speaking, the different leased lines must
be routed disjointly from each other.
In order to be competitive, the carrier must also consider
As the computed output data, we expect a list containthe cost side of this requirement. So naturally, a disjoint
routing at minimum cost is wanted. The problem of finding ing one simple routing path for each end-to-end pair of
the VPN, such that
this optimum is the topic of this tutorial.
To make things more precise, here are the needed input
• no two paths share a common trunk
data:
(trunk-disjoint routing), see figure 3, and
• the carrier’s network, see figure 1,
Figure 1: carrier’s network
• with a cost coefficient on every trunk,
• the Virtual Private Network, see figure 2.
Figure 3: disjoint routing
• the sum of the costs of all paths are at the possible
minimum.
Mathematically speaking, the problem is called
Minimum Cost Disjoint Embedding Problem. In stating it,
we have in mind to solve problem instances with a few thousand nodes and trunks. If node-disjoint routing is desired,
one only has to split all nodes into two nodes that are connected by an artificial trunk with zero cost, see figure 4.
Figure 4: node splitting technique
Our way through the solution process will start out with
a very simple but instructive example for a combinatorial
search.
2
and per column. Otherwise there would be a pair that could
attack each other.
The second goal is the optimization goal. It tells us
how to quantify (and thus to compare) the feasible objects
and which extremal property we want to achieve. The quantification is called the objective function. It may be minimized or maximized. In our case, the objective function is
the sum of the values the rooks are standing on. See figure 6
for a feasible solution and its objective function value.
9
7
8
2
2
3
7
4
4
1
5
6
4
1
6
3
Combinatorial Optimization
We leave the context of telecommunications planning for
the duration of this introduction. After we have understood
the ideas and techniques, solving our original problem will
be a straight-forward application. It is described in section 5.
9
2
4
4
7
3
1
1
8
7
5
6
2
4
6
3
Figure 5: a grid with numbers
Figure 6: a feasible solution with value 21
How many feasible solutions are there? Given the
empty grid, we have four distinct possibilities to place a
rook in the first row. Subsequently, this leaves three possibilities to place one in the second row, and so on. In total,
there are 4 · 3 · 2 · 1 = 24 feasible solutions. This particular
problem could still be solved by hand, but let us think big.
If we imagine that the setting is a real 8-by-8 chess board,
there are already 40320 feasible solutions. How about a side
length of 20, 50, 100? The numbers of feasible solutions explode: at a side length of about 35 we have roughly as many
feasible solutions as we have particles in our universe.
Almost all Combinatorial Optimization problems share
a common property: they possess a vast multitude of feasible solutions and the task is to seek out one of them with
the desired extremal property. This mostly involves comparing the objective values of all feasible solutions. This
demonstrates the necessity of special solution techniques.
Let us now consider the following optimization problem. We are given a 4-by-4 grid — one may imagine one
quarter of a chess board — where every cell carries a given 2.1 Enumeration tree
value (see figure 5). The task to solve: take four chess rooks As a first step, we organize the search for the optimal sound place them on the grid so as to simultaneously achieve lution in a systematic way. We enumerate all the solutwo goals:
tions without repetition as follows. We imitate the way of
counting the solution (see above) and start with an empty
1. no pair of the rooks may attack each other,
grid. On every cell in the top row, we can place our first
2. the sum of the numbers the rooks are standing on is a rook, yielding four mutually exclusive so-called partial solutions. With each of them, we can now iterate the process
minimum among all possible solutions.
and place another rook on the second row wherever it is not
The first goal is called a feasibility constraint, since it put below the first rook. This generates another 4 · 3 = 12
describes which combinatorial objects are considered per- mutually exclusive partial solutions with two rooks each.
missible or feasible. In our case here, it declares only such This goes on until all four rooks have been placed in all
solutions feasible where we have exactly one rook per row possible ways.
16
14
14
10
11
14
15
21
22
25
22
18
16
16
17
13
19
10
18
20
23
22
14
13
24
20
20
18
17
17
16
12
111111
000
000111
000
000111
111
000111
000
000111
111
000111
000
000111
111
000111
000
000111
111
000111
000
000111
111
000111
000
111111
000
000
000111
111
000
000111
111
000
000111
111
000
000111
111
000
000111
111
000
9
Figure 8: (partial) objective values in the tree
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
111111
000
000
000111
111
000
000111
111
000
000111
111
000
000111
111
000
000111
111
000
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
0001111
111
0000
1111111
000
0000
0001111
111
0000
0001111
111
0000
0001111
111
0000
0001111
111
0000
0001111
111
0000
0001111
111
0000
00000
11111
11111
00000
00000
11111
00000
11111
00000
11111
00000
11111
00000
11111
00000
11111
16
15
9
111
000
000
111
000
111
000
111
000
111
000
111
1111111
0000000
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
20
15
16
14
10
7
11
00
00
11
00
11
00
11
00
11
00
11
11
00
00
11
00
11
00
11
00
11
00
11
11
00
00
11
00
11
00
11
00
11
00
11
000
111
111
000
000
111
000
111
000
111
000
111
000
111
000
111
00
11
11
00
00
11
00
11
00
11
00
11
00
11
00
11
111
000
000
111
000
111
000
111
000
111
000
111
11
12
0
15
111
000
000
111
000
111
000
111
000
111
000
111
10
22
19
15
8
11
00
00
11
00
11
00
11
00
11
00
11
11
00
00
11
00
11
00
11
00
11
00
11
0000
1111
1111
0000
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
0000
1111
21
16
111
000
000
111
000
111
000
111
000
111
000
111
12
17
9
5
10
5
4
2
9
13
9
10
This generation process is illustrated in figure 7. The
empty grid is the root of a tree which branches out with the
possibilities of extending the partial solutions as said above.
Figure 7: full enumeration tree
2.2 Pruning the tree
As we see from figure 8, only very few vertices in the tree
are actually interesting. The optimum is located within a
small portion of the tree and several branches of it were a
successless effort in the enumeration. If we are able to direct the search better, we can save considerable time.
Keeping this in mind, we can rewrite the figure, showing
Here is a first idea how to identify a vertex — and with
only the objective function values of the (partial) solutions, it its entire subtree below — that does not justify further
see figure 8.
exploration. Remember: during the enumeration, the latSo basically, what we do is to generate the entire tree, est bold framed vertex indicates the best currently known
and to single out one of its bottom vertices with minimum solution. If we now produce a vertex with a value not less
value. Assuming that the bottom row of the tree is gener- than this solution value, we can stop extending the tree at
ated from left to right — a way of traversion called depth this point. As the values cannot decrease when more rooks
first search — figure 8 also shows the improvements made are placed, we will never find a better solution below this
in the search. Every time a vertex with lower value than all vertex. Figure 9 shows the tree when this rule is observed.
the ones to its left is encountered, it is marked by a bold
frame. The rightmost bold framed vertex, the one with the
value 10, is the desired optimum.
So the way of cutting off subtrees is governed by numbers that are available along the process. This idea is
strengthened in the next section.
10
11
10
14
15
16
12
17
5
9
5
4
15
9
10
13
9
2
8
For an example, consider figure 10. Since we know that
every row of the grid must host exactly one rook, this one
rook must stand on a cell carrying a value at least as high as
the minimum value in that row. So every feasible solution
must have an objective value at least the sum of these row
minima. In our 4-by-4 example, we compute a lower bound
of 6.
How can it help us? A simple calculation like the one in
figure 10 can be easily carried out in every new tree vertex.
So when deciding whether to explore or not, we have three
numbers at our disposal:
16
• the value curbest of the current best solution,
• the value lbound of the computed lower bound.
11
12
16
0
15
10
• the value parval of the partial solution reached, and
19
20
13
18
13
14
10
9
15
14
7
10
16
15
The lower bound guarantees that no feasible solution
has a lower value. So in the new vertex, no feasible solution extending the partial solution there has a lower value
less than parval + lbound. Comparing this sum to the best
known solution, we get an improved pruning criterion:
parval + lbound ≥ curbest ⇒ prune vertex.
20
18
24
20
12
17
17
9
16
22
2.4 Improved lower bounds
Figure 9: a first pruning idea
The pruning criterion applies to any lower bound, not only
the simple one presented in figure 10. How can we computer better ones? Surprisingly, there is a way of strengening the lower bound, often even up to the optimum value.
Consider the following algorithm, illustrated in figure 11.
2.3 Lower Bounds
The tree can be pruned in precisely that moment when we
are sure that the exploration of a subtree cannot yield a better solution than the best one currently known. This observation also holds true if we support the pruning decision
by a so-called lower bound. This is the name for a value
which offers a guarantee that no feasible solution can have
a smaller objective value.
9
2
4
4
7
3
1
1
8
7
5
6
2
4
6
3
2
2
1
1
6
row
minima
bound
Figure 10: a simple lower bound
3.
add
per
col.
9
2
4
4
0
9
2
4
4
7
3
1
1
2
9
5
3
3
8
7
5
6
−2
6
5
3
4
(n j = 1
2
0
2 original
grid
4
6
1. arbitrary
3 numbers
1 −1 2.tivenegasum
3 3 4. row
5 2 minima
7 3
4 3
1)
10
5. bound
Figure 11: a stronger lower bound
2.5 Correctness of the algorithm
Lower Bound Algorithm
1. Write an arbitrary number over each column of the
grid.
2. Add these numbers and write their negative sum on
their right.
This paragraph is more mathematical, as it has to prove the
correctness of the algorithm stated above. The more practical oriented reader may skip it.
In order to see why the algorithm always delivers a
lower bound, we need to formalize our rooks problem. We
give the value located in the grid cell (i, j) the name cij .
4. In each row, find the minimum value and write it Also, we introduce a variable xij that indicates whether
down on the right hand side, below the negative sum there is a rook being placed on cell (i, j) or not: xi j = 1
figure.
means “rook on (i, j)”, xi j = 0 means “no rook”. The objective function is the sum of all ci j , each value being selected
5. Add up alle row minima and the negative sum.
by its “rook factor” of xi j . The feasibility constraint can be
expressed as two sets of equalities: the sum over all x’s in
This is a generalization of the simple bound of figure 10
a row or column, resp., must equal one. In mathematical
in that we first choose the arbitrary numbers and add them
language, we write down a model:
to all cells in their respective columns. Then, like figure 10,
we seek the minimum values in all respective rows and add
them up, this time including the negative sum of the chosen
minimize
∑ ∑ ci j xi j
numbers. Setting these numbers all to zero would reproduce
i j
the calculation of figure 10.
subject to (i)
for all columns j
∑ xi j = 1
The calculation of lower bounds more than pays for iti
self. The effect of strong lower bounds can be seen in fig(ii)
for all rows i
ure 12. The upper numbers in the vertices are the sums
∑ xi j = 1
j
parval of the rook position values from figure 8, the lower
numbers are lower bounds lbound for the sums of the miss(iii) xi j = 0 or 1.
ing rooks. The total lower bounds are equal to their resp.
sums of parval + lbound.
These four model lines precisely capture the rooks problem, irrespective of the size of the grid or the values in
the cells. Remember: the feasibility constraint determines
0
parval
which objects are feasible to the problem, its expression are
10
lbound
the equalities stated after the “subject to” marker.
3. In each column, add the assigned number to all the
cell values.
9
7
8
2
10
6
10
8
12
16
13
9
5
4
8
4
6
4
9
6
Observation: if one removes (neglects) one or more
of these equalities, there are less restrictions left to define
feasibility. Thus, the set of all feasible solutions becomes
larger, as we allow more objects into it. Consequently, the
minimal objective value may at most decrease, this effect
possibly arising from new objects with smaller objective
values.
17
14
18
10
5
9
3
6
1
3
6
1
19
13
11
10
20
Figure 12: search tree with lower bounds
In mathematical terminology, the removal of feasibility
defining equalities (or inequalities) from a model is called
a relaxation. Its effect is always that the optimum value of
the relaxed problem is a lower bound to the desired optimum of the original problem.
In our example, we could for instance relax all equalities (i). That would allow two or more rooks to sit in the
same column while leaving intact the constraint “one per
row”. In fact, this is the model equivalent of the simple
lower bound from figure 10.
In order to understand the lower bound property of the
Now we only go down to an end vertex of the tree when
there is a better solution than the current one. A significant Lower Bound Algorithm, consider the following slightly
different model:
reduction in solution enumeration.
!
∑ ∑ ci j xi j + ∑ λ j ∑ xi j − 1
minimize
i
subject to
(i)
j
j
i
∑ xi j = 1
for all columns j
∑ xi j = 1
for all rows i
i
(ii)
The theory of the Bound Improvement Algorithm is beyond the scope of this paper. For an analysis and discussion
see, for instance, [1]. Here, we give the most widely used
formulation of it. The parameters are called λ j as above.
All of them together form the vector λ.
Bound Improvement Algorithm
j
(iii)
xi j = 0 or 1
(iv)
λ j arbitrary.
1. Set k = 0.
Start with arbitrary values λ(0) , e.g. λ(0) = 0.
It differs from the original rooks model only in the additional objective function term and the new arbitrary parameters λ j . Looking at the extra term more closely, it turns out
to be meaningless: since we have constraint (i), it must always equal zero. So we see that, independent of the choice
of the λ j , this second model has exactly the same set of feasible solutions and identical objective values for each solution. Thus, their respective optima are equal as well.
The power of the extra objective term unfolds, when
we now relax the constraint (i) as we did above. The
optimum of the relaxed second model (the so-called
Lagrangian relaxation of the original model) will now depend on the choice of the λ j . But since a relaxed optimum
can at most reach the unrelaxed one from below, our relaxed second model must always deliver a lower bound to
our unrelaxed original model. This completes the proof.
For the sake of completeness, it remains to show that
the algorithm really computes the optimum of the relaxed
second model. This can be readily seen by rearranging its
objective function:
!
2. Invoke the Lower Bound Algorithm with λ(k) , take
back the bound and the numbers n j of row minima
chosen in each resp. column j (cf. figure 11).
3. Define new vector λ(k+1) : let
(k+1)
λj
(k)
= λ j + sk (n j − 1) for all j.
4. Increase k by 1, repeat from step 2 as desired.
The algorithm produces a sequence of lower bound values which is not necessarily monotonic, see figure 13. So
one has to keep track of the best current bound value. Note
that the number ∑ j n j of found column conflicts tends to
decrease as the algorithm proceeds.
lower bound
number of conflicts
best possible bound value
∑ ∑ ci j xi j + ∑ λ j ∑ xi j − 1
i
=
j
j
i
∑ ∑(ci j + λ j ) xi j − ∑ λ j .
i
j
almost feasible solutions
200 iter.
j
The reader interested in the details of Lagrangian relaxation is referred to [1].
2.6 Choice of the parameters
Introducing the arbitrary numbers as parameters opens a
vast degree of freedom in the calculation. As figure 11
demonstrates, it is possible in our example to choose the
numbers in such a way that the lower bound equals the optimum. These are clearly the best bounds achievable, so
the method is quite powerful. But how can we make this
choice?
The answer: there is no formula for the best
values.
In practice today, an algorithm named
Subgradient Optimization is usually employed. It starts
with arbitrary values and updates them iteratively. In every iteration, the entire lower bound algorithm has to be
performed. This may be time comsuming, but is worth a
certain number of iterations, as the lower bounds increase
and save efforts in the search tree exploration process.
Figure 13: typical bound improvement sequence
The algorithm uses an auxiliary sequence sk which determines the length of the update step 3. Essentially any
such sequence works if it obeys the following conditions
(cf. [5]):
• all sk are nonnegative,
• sk converges to zero,
• ∑k sk diverges to infinity.
Examples for such sequences are Ck or √Ck where C is a
positive constant. Several refinements to the algorithm have
been suggested in the literature, for instance in [3, 2]. They
have the potential to significantly reduce the number of iterations needed to reach a satisfactory bound. Yet all of them
come at the expense of a rather tedious need to administer
more free parameters. Adapting the Bound Improvement
Algorithm to a specific problem is a substantial, yet worthwhile, subtask of the whole optimization project.
2.7 Upper Bounds
According to the pruning criterion of section 2.3, we can
best prune the search tree when we have strong lower
bounds and/or good known feasible solutions. Their values are always upper bounds for the optimum, since the
minimum objective value can be at most as high as that of
any feasible solution. So how can we achieve strong upper
bounds?
Since any feasible solution can serve us, the way to go
can be twofold. Either we produce many feasible solutions
fast and choose their best value. Or we can spend some
time on a more sophisticated way to construct a few “good”
solutions. We will discuss both approaches. An algorithm
that strives to compute a good, but not necessarily optimal,
feasible solution is called a heuristic.
Since feasible solutions to the rooks problem abound,
there is a simple way of getting many of them fast: random choice. Place one rook randomly and reduce the grid
by eliminating its row and column. Then do the same with
the reduced grid until all rooks have been placed. This can
be repeated several times in a row up to some prechosen
time-out. In terms of the search tree, this method follows
some randomly chosen path from the current vertex down
to a solution. It ignores any information along the way, see
figure 14.
start
vertex
An interesting feature of the Bound Improvement Algorithm of section 2.6 is it that it produces a sequence of
(mostly) infeasible solutions whose number of feasibility
conflicts more or less decreases over the iterations, see figure 13. Remember: according to the relaxation we chose,
we allowed objects as feasible solutions that have multiple
rooks in their columns. If rooks are placed in the same column, this is called a conflict. If now the number of conflicts
varies, we can pick one infeasible solution with the least
number of conflicts from this sequence. If its number is
zero, we even have a feasible solution ready at hand. If its
number is positive, but not too high, we can feed the solution into a heuristic that attempts to remove these conflicts.
A possible heuristic based on this idea could do the following.
Conflict Removal Heuristic
1. Take solution with minimal number of conflicts from
the Bound Improvement Algorithm.
2. Regard every rook in the solution as fixed when it
stands alone in its column. Eliminate both its row
and column from the grid.
3. Choose a column with more than one rook in it. For
every rook in the column, seek the minimum cell
value in its row without its current position in that
row. Calculate the difference between this minimum
and its current cell value.
4. From the rooks in the column, choose one with the
largest difference to be the one assigned to that column. Place the others on their minimum value cells
computed in step 3.
5. Go on with step 2 if there are still conflicts. Otherwise, output the found solution.
The Conflict Removal Heuristic alternatingly reduces
the problem size by accepting some rooks as being fixed,
and resolves a conflict by deciding which one rook gets to
stay in a column. The choice in step 4 is just one way of
deciding, any other rule will also do. For an example of the
Conflict Removal Heuristic, see figure 15.
9
13
20
16
18
22
25
Figure 14: random heuristic for solutions
2.8 Looking closer
An example for a slightly more sophisticated heuristic is the
following. Again we build up a solution rook by rook. We
no longer choose both the row and column positions randomly, but only the row. Once a row is given, we seek the
cell with minimum value and place the rook on it. After
that, we again reduce the grid and iterate.
8 2 +6
fixed
4
4
differences
5 6 −2
6 3 +1
rook in solution
next best position
Figure 15: step 3 of the Conflict Removal Heuristic
Note: a negative value difference may well occur, since bounds tend to be the lowest ones in the entire tree. Followthe Bound Improvement Algorithm uses modified cell val- ing them, we have a high chance of going straight into the
ues for its internal rook placement. The Conflict Removal “general area” of the optimum.
Heuristic only uses the given input cell values.
This is illustrated in figure 17 where the tree vertices
carry their total lower bound values. Following the minimal
bound value of 10, we are directly led to the optimum. In
2.9 Applicability
general, this is not as nicely the case. In large problem inAs easy as the design of a heuristic may seem, a word stances, the tree does in fact branch out somewhat, but does
should be said about the applicability. Not all problems are so in a more or less “focused” way. This strategy of tree
structured such that the feasibility constraint always allows exploration is called branch-and-bound.
us to construct a feasible solution in a more or less arbitrary
manner. This holds especially true when starting out with a
partial solution. figure 16 shows an example of an enumeration tree in which not all subtrees contain feasible solutions.
The problem setting is identical to the rooks problem, only
the rooks have been replaced by queens.
10
19
13
10
18
000000
111111
111111
000000
000000
111111
000000
111111
000000
111111
000000
111111
000000
111111
000000
111111
000000
111111
000000
111111
000000
111111
000000
111111
14
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
11
00
00
11
00
11
000
111
00
11
000
111
000
111
000
111
000
111
000
111
000
111
11
00
00
11
00
11
000
111
00
11
000
111
000
111
000
111
000
111
000
111
000
111
11
00
00
11
00
11
00
11
00 11
11
00
00
11
00
11
00
11
00
00 11
11
11
00
00
11
00
00 11
11
00
11
00
11
00
00 11
11
00
11
00
11
00
11
11
00
00
11
00
11
00
11
00
11
00
11
00
11
11
00
00
11
00
11
00
11
00
11
00
11
00
11
11
00
11
11
00
00
11
00
11
11
00
00
11
00
11
00
11
00
11
00
11
00
11
00
11
14
10
111
000
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
000
111
11
00
00
11
00
11
00
11
00
11
00
11
00
11
00
11
10
00
11
11
00
00
11
00
11
10
00
11
11
00
00
11
00
11
00
11
11
00
00
11
00
11
Figure 16: search tree with queens
In general, good knowledge about the problem structure
is necessary for designing a successful heuristic.
3
Vertex Sequence
There is another degree of freedom which can help us
keep the search effort under control: the vertex sequence in which we build up the enumeration tree. The
depth-first-search strategy we have used so far is canonical,
yet there is a better one. We let the lower bounds be our
guides. Whenever we have to choose another tree vertex to
explore, we choose one with a minimal lower bound value
among all vertices waiting to be explored.
The rationale behind this is the following observation:
consider the path from the tree root (the empty grid) down
to the optimal solution. Every vertex on it carries a lower
bound that cannot exceed the optimum objective value. But
since the optimum is the lowest possible value, also these
Figure 17: search tree with branch-and-bound
strategy
The branch-and-bound strategy possesses three important features:
• It requires possibly huge amounts of storage for all
the vertices that are candidates for exploration.
• Its performance relies heavily on the quality of the
lower bounds. If a problem does not offer the computation of strong bounds, this method may take prohibitive time (and storage) to proceed.
• In choosing always a vertex with minimal lower
bound, this bound is, in every iteration, a global lower
bound for the problem instance.
While the first two features rather represent limitations,
the third one makes branch-and-bound the more desirable:
the global lower bound steadily increases as the method proceeds. Using an upper bound heuristic (see section 2.7), one
can commonly observe that from time to time better feasible solutions, i.e. global upper bounds, are found. So both
global bounds converge monotonically towards each other,
eventually meeting at the optimum value. A typical trajectory of a branch-and-bound run is shown in figure 18.
objective value
actual search
optimality
proof
optimum found
time
Figure 18: a typical branch-and-bound trajectory
One can see another typical feature of combinatorial optimization in general: the optimal solution is often found
well before its optimality is proven by the lower bound.
This means that branch-and-bound can be used as an improvement heuristic itself: just let it run until the gap between the two global bounds is as small as is acceptable,
for instance maybe 10 per cent. Even with no more than
the accepted gap as an optimality guarantee, there is a good
chance that the optimum was already found.
Strictly speaking, we have only introduced the bound
step so far. The branch step is what we have silently used all
the time: the local enumeration of all possibilities of placing a rook within a grid row. In terms of the tree, branching
means the manner in which we branch out from a vertex.
We will not go into discussion here, as this way is rather
canonical for the rooks problem.
4
Summary of Combinatorial Optimization
The typical setting of a combinatorial optimization problem is an abundance of feasible solutions and the task is
to seek out one of them with minimum (or maximum) objective value. What makes such a problem hard is the fact
that it is often not possible to actually compare the objective
values of all feasible solutions. So special search techniques
must be employed that mostly work with partial solutions in
a clever way.
3. upper bound: a global value with the guarantee that
it is no less than the optimum value. In particular,
the objective value of any feasible solution is an upper bound. Upper bounds are strong when they are as
low as possible.
4. vertex sequence: the sequence in which the tree vertices are explored. Branch-and-bound is one popular
sequence, utilizing the lower bounds.
The effects of introducing the ideas 2.–4. can be seen
in figure 19 for the rooks problem. As the size of the grid
increases up to 100, the different curves indicate the computation times to optimum.
complete enumeration
time [s]
100000
10000
1000
100
10
1
0.1
0.01
Branch-and-Bound
respect best solution plus upper bounds
plus lower bounds
20
40
100
grid size
Figure 19: computation times over problem sizes
Three main features clearly come out:
• Complete enumeration without refinements is
quickly out of practical reach as sizes grow.
• The introduction of lower bounds is a major improvement that can possibly take out the strict (super-) exponential growth in time consumption. Thus, it is
usually worth investing quite some time into the calculation of strong bounds.
• The more sophisticated the search is, the more can
the algorithm exploit stochastic luck. So a more or
less accurate estimate of the remaining running time
is almost never possible.
4.2 Linear and integer programming
4.1 The ingredients
In the recent past, all-purpose LP/IP solvers have become
In order to make the solution process work, four ingredients strong on the optimization market. In principle, these tools
are able to solve combinatorial problems as well. Their adhave to be specified (here for a minimization problem):
vantage clearly is that one purchases excellent know-how
1. branching strategy: the manner in which a tree ver- with their running licenses and updates, and the modeler is
freed from thinking about the solution process.
tex branches out upon exploration.
Yet on the flip side, these software packages require that
2. lower bound: an estimate from below for the optimal the problem be modeled in their specific language which
objective values of all feasible solutions in a specified may not be an easy task, depending on the problem. In
subtree. Lower bounds are strong when they are as any case, it means one has to reorganize the given data into
some specified input format. Also, due to their all-purpose
high as possible.
nature, commercial solvers cannot exploit special advantageous features in the problem at hand. A common other
drawback is that they tend to consume huge amounts of storage for their execution, a possibly limiting factor for large
problem instances.
LP/IP software packages basically employ the same
method ingredients as listed above. They merely use different techniques in filling in the details. Tailor-made combinatorial solvers can normally be restricted to much less
memory consumption — possibly work directly on the data
at hand — and can be written so as to exploit special problem features. So there is a certain trade-off between buying
versus implementing a software solution in any case.
The reader interested in linear/integer programming is
referred to [4, 6] for an introduction.
In the next section, we apply the optimization knowhow presented to the Disjoint Paths Problem.
b
a
c
e
d
f
h
g
m
i
k
a b c d e f
g h i k m
Figure 20: VPN solution in grid representation
Figure 20 shows the feasible solution of figure 3 as a
grid pattern: the marked entries in a row indicate the trunks
that the row’s demand is routed over. Condition (a) is satWe now switch back to the VPN problem which was stated isfied, as one can see in the network. And, according to
in section 1. How does it fit into the combinatorial opti- condition (b), there is no column with more than one entry
mization framework?
in it. This is the desired disjointness. (Note that, for clarFirst, we need to identify the feasible objects. In this ity of presentation, no cost values are given. They normally
case, we expect as the output a routing of all given end-to- belong inside the grid, one per cell.)
In order to compute an optimal solution, we need to
end demand pairs of the VPN. A demand counts as routed
when we exhibit a simple path through the network con- specify the four “ingredients” identified in section 4.
necting its end nodes. Furthermore, the routing is feasible
if no two of these paths share a common trunk.
5.1 Branching strategy
Mathematically speaking, for each individual demand
pair d we seek a subset T d of all network trunks. The feasi- From the rooks problem, we know: a vertex in the search
tree always in principle looks like the entire problem. Placbility constraint then consists of two separate conditions:
ing a rook implied eliminating its row and column from the
grid, leaving a slightly smaller rooks problem.
(a) the trunks in each set T d must constitute a simple path
In the VPN case, there are always some demand pairs
in the network connecting the end nodes of demand d, to be disjointly routed through some network. The only difference to the entire given problem instance is that not nec(b) no two sets may intersect: T d1 ∩ T d2 = ∅ for all de- essarily all or the same demands are still unrouted, and that
mand pairs d1 , d2 .
not necessarily all trunks of the network are still available.
Several branching strategies are devisable. One strategy
has
proven to be especially well suited, see figure 21. It
Second, we need to set up an objective function. We
works
as follows:
are given a cost coefficient c on every trunk t, and this cost
5
Application to VPN planning
t
actually arises if one of the routing paths contains the resp.
trunk:
cost = ∑
∑ ct .
d t∈T d
The VPN problem can be visualized so that it looks similar to the rooks problem of section 2, see figure 20. The
rows of the grid correspond to the VPN demands, and the
columns correspond to the trunks of the network. All cells
in one and the same column carry identical cost values, as it
does not matter which demand path contains the column’s
trunk.
Branching Strategy
1. Choose an unrouted demand d = (mn) and one of
its end nodes n according to some rule (maybe randomly).
2. For all trunks t = (np) connected to node n, do:
3. reserve trunk t for the routing of demand d (i.e., put t
into the set T d )
4. form a new subproblem on the network less trunk t
and with demand d altered to d = (mp). If m = p,
remove demand d from the list of unrouted demands.
m
m
t
n
m
p
p
The Bound Improvement Algorithm (cf. section 2.6) is
readily applicable as well. Just two slight differences are to
be observed:
• A restriction applies to step 3: the λ-values must stay
nonnegative. This can be achieved by setting them to
zero if the update formula results in a negative value.
(The reason for this is that condition (b) restricts the
number of paths containing any one trunk to at most
one, not to exactly one, as in the rooks problem.)
m
• The numbers n j in the rooks problem are replaced by
numbers nt that for each trunk t in the network count
the number of routing paths containing it.
t
t
p
Figure 21: branching strategy
5.2 Lower bounds
The minimum cost paths needed in step 2 are to be computed by the well-known Dijkstra Algorithm. Presenting it
is beyond the scope of this paper, for an introduction see,
for instance, [1].
5.3 Upper bounds
Calculation of lower bounds works here almost the same
way as in the rooks problem, see also figure 22:
Lower Bound Algorithm
1. Assign to each column t (in the grid of figure 20) an
arbitrary nonnegative parameter value λt .
2. Now ignore the disjointness condition (b) and compute for every row a minimum cost path connecting
the end nodes of the row’s demand pair.
3. Add up the costs of all minimum cost paths and subtract the sum of the parameter values from it.
Unfortunately, the VPN problem has the property indicated
in section 2.9, namely that in general not all subtrees of
the entire enumeration tree contain feasible solutions. So
a heuristic may or may not be successful, depending on
the tree vertex it is run in. With this caveat, the following heuristic has proven to be very useful. It is a straightforward analogue of the algorithm given in section 2.8. A
conflict is a trunk which is contained in more than one routing path.
Conflict Removal Heuristic
1. Take solution with minimal number of conflicts from
the Bound Improvement Algorithm.
2. Regard every routing path in the solution as fixed
when none of its trunks is a conflict. Eliminate all
its trunks from the network.
3. Choose a conflict trunk t. For every demand whose
routing path contains t, compute the minimum cost
path in the network without t. Calculate the difference between its cost and the cost of the original path.
λa λb λc ≥ 0
...
λm
− ∑t λt
min
min
(nt = 0 2 1 1 0 0 1 1 0 0 2)
Figure 22: lower bounds
4. From the demands in step 3, choose one with the
largest cost difference to be the one allowed to “occupy” trunk t. Replace the other paths by their resp.
minimum cost paths computed in step 3.
5. Go on with step 2 if there are still conflicts. Otherwise output the found solution.
min
5.4 Vertex sequence
bound
Here the branch-and-bound sequence is appropriate. Note
that the subproblems generated upon vertex exploration (see
figure 21) are each specified by a subnetwork of the given
network and by demands different from the original ones
given. So every vertex kept track of necessitates a consider- References
able storage amount. A careful design of the data structures
[1] R. K. Ahuja, T. L. Magnanti, and J. B. Orlin: Network
used may reduce memory consumption significantly.
Flows, Prentice Hall, Englewood Cliffs (1993).
6
Conclusion
[2] M. S. Bazaraa and J. J. Goode: A Survey of Various
Tactics for Generating Lagrangian Multipliers in the
Context of Lagrangian Duality, in European Journal of
Operational Research 3, pp. 322–328.
[3] M. Held, P. Wolfe, and H. Crowder: Validation of
The techniques of Combinatorial Optimization offer a
Subgradient Optimization, in Mathematical Programbroad tool kit that is applicable to a vast variety of pracming 6 (1974), pp. 62–88.
tical problems. Optimizing over discrete objects has less
of a straightforward calculation, but more of a search pro- [4] G. L. Nemhauser and L. A. Wolsey: Integer and Comcess. We have presented and exemplified the ingredients of
binatorial Optimization, John Wiley and Sons, New
a typical such process.
York (1988).
The solution of the stated VPN routing problem has
been a successful project within T-Systems. Its outcome
was the efficient implementation of all the algorithms presented, plus several more refinements. Reliable VPN routing at minimum cost is now a service product available from
T-Systems, Dept. E46, Center of Technology, Darmstadt,
Germany.
[5] B. T. Polyak: A General Method for Solving Extremum
Problems, in Soviet Mathematics Doklady 8 (1967),
pp. 593–597.
[6] H. M. Salkin and K. Mathur: Foundations of Integer
Programming, Elsevier Science Publishing, New York
(1989).