Edit on GitHub

Currently supported operators

The operators currently supported are listed below. The operators with a check box have been subject to a large degree of scrutiny and have been implemented for both forward and reverse McCormick relaxations (Wechsung2015). Each McCormick object is associated with a parameter T <: RelaxTag which is either NS for nonsmooth relaxations (Mitsos2009, Scott2011), MV for multivariate relaxations (Tsoukalas2014, Najman2017), or Diff for differentiable relaxations (Khan2016, Khan2018, Khan2019). Conversion between MV, NS, and Diff relax tags is not currently supported. Convex and concave envelopes are used to compute relaxations of univariate functions.

Univariate McCormick Operators

Arbitrarily differentiable relaxations can be constructed for the following operators:

  • [x] Inverse (inv)
  • [x] Logarithms (log, log2, log10)
  • [x] Exponential Functions (exp, exp2, exp10)
  • [x] Square Root (sqrt)
  • [x] Absolute Value (abs)

Both nonsmooth and Whitney-1 (once differentiable) relaxations are supported for the following operators:

  • [x] Step Functions (step, sign)
  • [x] Trignometric Functions (sin, cos, tan)
  • [x] Inverse Trignometric Functions (asin, acos, atan)
  • [x] Hyperbolic Functions (sinh, cosh, tanh)
  • [x] Inverse Hyperbolic Functions (asinh, acosh, atanh)
  • [x] Common Activation Functions (relu, leaky_relu, param_relu, sigmoid, bisigmoid, softsign, softplus, maxtanh, pentanh, GeLU, ELU, SELU, Swish-1)
  • [x] Special Functions (erf)

Bivariate Operators: McCormick & McCormick

The following bivariate operators are supported for two MC objects. Both nonsmooth and Whitney-1 (once differentiable) relaxations are supported.

  • [x] multiplication (*)
  • [x] division (/)

Arbitrarily differentiable relaxations can be constructed for the following operators:

  • [x] addition (+)
  • [x] subtraction (-)
  • [x] minimization (min)
  • [x] maximization (max)

Common subexpressions

The following functions can be used in place of common subexpressions encountered in optimization and will result in improved performance (in each case, the standard McCormick composition rules are often more expansive).

McCormick.xexpax — Function

expax

The expax function is defined as expax(x, a) = x*exp(a*x).

Form defined in Najman, Jaromił, Dominik Bongartz, and Alexander Mitsos. "Relaxations of thermodynamic property and costing models in process engineering." Computers & Chemical Engineering 130 (2019): 106571.

McCormick.arh — Function

arh

The arrhenius function arh is defined as arh(x) = exp(-k/x).

McCormick.xlogx — Function

xlogx

The function xlogx is defined as xlogx(x) = x*log(x).

McCormick.mm — Function
mm(X,v,K) = v*X / (X + K)

A Michaelis-Menten rate function.

Bound Setting Functions

The following functions are used to specify that known bounds on a subexpression exist and that the relaxation/interval bounds propagated should make use of this information. The utility functions can be helpful in avoiding domain violations that arise due to the overly expansive nature of composite relaxations. Improper use of these functions may lead to cases in which the resulting relaxations are empty, so the user is encouraged to use discretion.

McCormick.positive — Function

positive(x::MC)

Sets the lower interval bound and the convex relaxation of x to a value of at least McCormick.MC_DOMAIN_TOL. (Sub)gradients are adjusted appropriately.

McCormick.negative — Function

negative(x::MC)

Sets the upper interval bound and the concave relaxation of x to a value of at most -McCormick.MC_DOMAIN_TOL. (Sub)gradients are adjusted appropriately.

McCormick.lower_bnd — Function

lower_bnd(x::MC, lb::Float64)

Sets the lower interval bound and the convex relaxation of x to a value of at least lb. (Sub)gradients are adjusted appropriately.

McCormick.upper_bnd — Function

upper_bnd(x::MC, ub)

Sets the upper interval bound and the concave relaxation of x to a value of at most ub. (Sub)gradients are adjusted appropriately.

McCormick.bnd — Function

bnd(x::MC, lb, ub)

Sets the lower interval bound and the convex relaxation of x to a value of at least lb. Sets the upper interval bound and the concave relaxation of x to a value of at most ub. (Sub)gradients are adjusted appropriately.

Specialized Activation Functions

McCormick.pentanh — Function

pentanh

The pentanh activation function pentanh(x) = x > 0.0 ? tanh(x) : tanh(0.25*x).

McCormick.leaky_relu — Function

leaky_relu

The leaky Rectified Linear Unit activation function leaky_relu(x) = max(x, 0.01x).

McCormick.param_relu — Function

param_relu

The parametric Rectified Linear Unit activation function param_relu(x, α) = (max(x, αx) with α in [0,1].

McCormick.maxtanh — Function

maxtanh

The maxtanh activation function maxtanh(x) = max(x, tanh(x)).

References

  • Khan KA, Watson HAJ, Barton PI (2017). Differentiable McCormick relaxations. Journal of Global Optimization, 67(4): 687-729.
  • Khan KA, Wilhelm ME, Stuber MD, Cao H, Watson HAJ, Barton PI (2018). Corrections to: Differentiable McCormick relaxations. Journal of Global Optimization, 70(3): 705-706.
  • Khan KA (2019). Whitney differentiability of optimal-value functions for bound-constrained convex programming problems. Optimization, 68(2-3): 691-711
  • Mitsos A, Chachuat B, and Barton PI. (2009). McCormick-based relaxations of algorithms. SIAM Journal on Optimization, 20(2): 573–601.
  • Najman J, Bongratz D, Tsoukalas A, and Mitsos A (2017). Erratum to: Multivariate McCormick relaxations. Journal of Global Optimization, 68: 219-225.
  • Scott JK, Stuber MD, and Barton PI. (2011). Generalized McCormick relaxations. Journal of Global Optimization, 51(4): 569–606.
  • Stuber MD, Scott JK, Barton PI (2015). Convex and concave relaxations of implicit functions. Optim. Methods Softw., 30(3): 424–460
  • Tsoukalas A and Mitsos A (2014). Multivariate McCormick Relaxations. Journal of Global Optimization, 59:633–662.
  • Wechsung A, Scott JK, Watson HAJ, and Barton PI. (2015). Reverse propagation of McCormick relaxations. Journal of Global Optimization, 63(1): 1-36.