Lecturer this time: Ioannis Caragiannis

Lecture notes is based on Karger’s Min Cut Algorithm by Sanjeev Arora, Sanjoy Dasgupta

Karger’s min cut algorithm and its extension. It’s simple for finding the minimum cut in a graph.

Clustering via Graph Cuts

The minimum cut of an undirected graph G=(V,E)G=(V,E) is a partition of the nodes into two groups V1,V2V_1,V_2 (s.t. V=V1V2V=V_1\cup V_2 and V1V2=V_1\cap V_2=\empty) so that the number of edges between V1V_1 and V2V_2 is minimized.

Variant of cut problem:

  1. Minimum global cut: 将图分成两不连通部分所需的最小割

  2. Minimum s-t cut: 分开源点 s 和汇点 t 所需的最小割

    size of minimum s-t cut = value of maximum s-t flow

    Algorithm: Ford-Fulkerson algorithm (compute then remove augmenting paths in a residual network 计算增广路径), Edmonds-Karp algorithm (O(nE2)O(n|E|^2))

一个很自然的计算 minimal global cut 的 deterministic 算法:

  1. 选择一个固定源点 s fix a node s

  2. 将其他 n1n-1 个点分别当作汇点 t,使用 EK 算法执行 n1n-1 次,分别计算 s 到这些 t 的最大流 the global should disconnect node s from any other node t.

  3. 返回这些最大流的最小值 return the minimum s-t cut in all these executions

    时间复杂度为 O(n2E2)O(n^2|E|^2) It’s the algorithm we wanna beat today.

    O(n4)O(n2E2)O(n6)O(n^4)\le O(n^2|E|^2)\le O(n^6)

Karger’s Algorithm

Randomized algorithm for finding the minimum cut:

  • Repeat until just two nodes remain:
    • Pick an edge of GG at random and collapse its two endpoints into a single node.
  • For the two remaining nodes u1,u2u_1,u_2, set V1={nodes that went into u1}V_1=\{\text{nodes that went into }u_1\} and V2={nodes that went into u2}V_2=\{\text{nodes that went into }u_2\}.

The algorithm has running time O(n2)O(n^2)

  • linear time to update node/edge information after a contraction
  • n2n-2 contraction in total.

Analysis

Some facts help to analyze the algorithm.

Fact 1

If degree(u)degree(u) denotes the number of edges touching node uu, then

uVdegree(u)=2E\sum_{u\in V}degree(u)=2|E|

Obviously, each edge contributes twice w.r.t. 2 nodes it connects.

Fact 2

If there are nn nodes, then the average degree of a node is 2En2|E|\over n.

A very straightforward conclusion.

When we randomly pick a node XX at random:

E[degree(X)]=uVPr(X=u)degree(u)=1nuVdegree(u)=2En\mathbb E[degree(X)]=\sum_{u\in V}\Pr(X=u)degree(u)={1\over n}\sum_{u\in V}degree(u)=\frac{2|E|}{n}

Fact 3

The size of the minimum cut is at most 2En2|E|\over n.

Consider the partition of VV into two pieces, one containing a single node uu, and the other containing the remaining n1n-1 nodes. The size of this cut is degree(u)degree(u). Since this is a valid cut, the minimum cut cannot be bigger than this. i.e.,

uV,mincutdegree(u)mincutminuV(degree(u))avguV(degree(u))=2En.\forall u\in V, mincut\le degree(u)\\ \Rarr mincut\le\min_{u\in V}(degree(u))\le\text{avg}_{u\in V}(degree(u))=\frac{2|E|}{n}.

Fact 4

If an edge is picked at random, the probability that it lies across the minimum cut is at most 2n2\over n.

This is because there are E|E| edges to choose from, and at most 2En2|E|\over n of them are in the minimum cut.

By facts above, we can easily analyze Karger’s algorithm.

Each time an edge is collapsed, the number of nodes decreases by 11. Therefore,

Pr(final cut is the minimum cut)=Pr(first edge not in mincut)×Pr(second edge not in mincut)×(12n)×(12n1)××(124)×(123)=n2nn3n1n4n22413=2n(n1)=(n2)1\Pr(\text{final cut is the minimum cut})=\Pr(\text{first edge not in mincut})\times\\ \Pr(\text{second edge not in mincut})\times\cdots\\ \ge(1-\frac{2}{n})\times(1-\frac{2}{n-1})\times\cdots\times(1-\frac{2}{4})\times(1-\frac{2}{3})\\ ={n-2\over n}\cdot{n-3\over n-1}\cdot{n-4\over n-2}\cdots{2\over4}\cdot{1\over3}\\ ={2\over n(n-1)}=\left(\begin{array}{c}n\\2\end{array}\right)^{-1}

Karger’s algorithm succeeds with probability p2n2p\ge {2\over n^2}. Therefore, it should be run Ω(n2)\Omega(n^2) times, after which the smallest cut found should be chosen.

Another way to implement Karger’s algorithm:

  • Assign each edge a random weight
  • Run Kruskal’s algorithm to get the minimum spanning tree
  • Break the largest edge in the tree to get the two clusters.

Why does Kruskal’s algorithm work?

Waiting for Iannis to tell me …

Run Multiple Times to Bound Error Probability

In order to boost the probability of success, we simply run the algorithm (n2)\ell\left(\begin{array}{c}n\\2\end{array}\right) times.

The probability that at least one run succeeds is at least

1(1(n2)1)(n2)1e.1-\left(1-(\begin{array}{c}n\\2\end{array})^{-1}\right)^{\ell(\begin{array}{c}n\\2\end{array})}\ge 1-e^{-\ell}.

Setting =clnn\ell=c\ln n, we have error probability 1nc\le{1\over n^c}.

It’s easy to implement Karger’s algorithm so that one run takes O(n2)O(n^2) time. Therefore, by running the algorithm O(n2logn)O(n^2\log n) times, we have an O(n4logn)O(n^4\log n) time randomized algorithm with error probability 1poly(n)1\over\text{poly}(n). Still better than EK algorithm for not very sparse graphs.

The Number of mincutmincut

Theorem: Any graph has at most n(n1)2n(n-1)\over2 distinct global cuts of mincutmincut.

Recall that we fixed a particular minimum cut and proved the probability 2n(n1)2\over n(n-1).

Denote by CiC_i the event that the ii-th minimum cut is returned in an execution by the contraction algorithm, Pr[Ci]2/n(n1)\Pr[C_i]\ge2/n(n-1).

Faster Version

Proposed by Karger and Stein.

Key idea

In the initial merging, it’s very unlikely we merged an edge in the mincutmincut. Towards the end of the algorithm, our probability of selecting an edge in the mincutmincut grows.

For a fixed minimum cut δ(S)\delta(S), the probability that this cut survives down to \ell vertices is at least C2/Cn2C_\ell^2/C_n^2. Thus, for =n/2\ell=n/\sqrt2 we have probability 12\ge{1\over2} of succeeding.

Detail in slides page 91.

Improved algorithm: From a multi-graph GG, if GG has at least 66 vertices, repeat twice:

  1. run the original algorithm down to n2+1{n\over\sqrt2}+1 vertices.
  2. recurse on the resulting graph.

Return the minimum of the cuts found in the two recursive calls.

The choice of 66 instead of other constant will only affect the running time by a constant factor.

T(n)=2T(n/2)+O(n2)=O(n2logn).T(n)=2T(n/\sqrt2)+O(n^2)=O(n^2\log n).

Since we succeed down to n/2n/\sqrt2 with probability 12\ge{1\over2}, we have the following recurrence for the probability of success, denote by P(n)P(n):

P(n)1(112P(n/2+1))2P(n)\ge1-\left(1-\frac{1}{2}P(n/\sqrt2+1)\right)^2

Inner term is probability that the sub-instance preserved mincutmincut AND recursive call found this optimum.

probability of success = 1 - probability of failure.

probability of failure = probability of both subroutines fails

probability of both subroutines fails = square of (probability of a subroutine fails)

probability of a subroutine fails = 1 - 1/2 * one successful recursive call

This solves to P(n)=Ω(1logn)P(n)=\Omega({1\over\log n}). WHY???

Set Q(h)=P(2h/2)Q(h)=P(2^{h/2})

This inequality becomes Q(h)Q(h1)Q(h1)24Q(h)\ge Q(h-1)-{Q(h-1)^2\over4}

It suffices to show inductively that Q(h)1h+1Q(h)\ge{1\over h+1}

Observe that the RHS of the inequality is increasing in Q(h1)Q(h-1).

By induction hypothesis, Q(h1)1hQ(h-1)\ge{1\over h}, the inequality implies that

Q(h)1h(1/h)241h1h(h+1)=1h+1Q(h)\ge{1\over h}-{(1/h)^2\over4}\ge{1\over h}-{1\over h(h+1)}={1\over h+1}

as desired.

Hence, similar to the earlier argument for the original algorithm, with O(log2n)O(\log^2n) runs of the algorithm, the probability of success is 11poly(n)1-{1\over\text{poly}(n)}.

There fore in O(n2log3n)O(n^2\log^3n) total time, we can find the minimum cut with probability 11poly(n)1-{1\over\text{poly}(n)}.