DifferentiableFrankWolfe

DifferentiableFrankWolfe

Differentiable wrapper for FrankWolfe.jl convex optimization routines.

DiffFW

Callable parametrized wrapper for the Frank-Wolfe algorithm to solve θ -> argmin_{x ∈ C} f(x, θ), which can be differentiated implicitly wrt θ.

Reference: https://arxiv.org/abs/2105.15183 (section 2 + end of appendix A).

Fields

• f: function f(x, θ) to minimize wrt x
• f_grad1: gradient ∇ₓf(x, θ) of f wrt x
• lmo: linear minimization oracle θ -> argmin_{x ∈ C} θᵀx from [FrankWolfe.jl], implicitly defines the convex set C
• alg: optimization algorithm from FrankWolfe.jl
• implicit: implicit function from ImplicitDifferentiation.jl

(dfw::DiffFW)(θ::AbstractArray, frank_wolfe_kwargs::NamedTuple)

Apply the Frank-Wolfe algorithm to θ with settings defined by the named tuple frank_wolfe_kwargs (given as a positional argument).

Return a couple (x, stats) where x is the solution and stats is a named tuple containing additional information (its contents are not covered by public API, and mostly useful for debugging).

DiffFW(f, f_grad1, lmo, alg=away_frank_wolfe; implicit_kwargs=(;))

Constructor for DiffFW which chooses a default algorithm and creates the implicit function automatically.

ConditionsFW

Differentiable optimality conditions for DiffFW, which rely on a custom simplex_projection implementation.

ForwardFW

Underlying solver for DiffFW, which relies on a variant of Frank-Wolfe.

simplex_projection(z)

Compute the Euclidean projection of the vector z onto the probability simplex.

This function is differentiable thanks to a custom chain rule.

Reference: https://arxiv.org/abs/1602.02068.

simplex_projection_and_support(z)

Compute the Euclidean projection p of z on the probability simplex as well as the indicators s of its support, which are useful for differentiation.

Reference: https://arxiv.org/abs/1602.02068.