# Currently supported operators

The operators currently supported are listed below. The operators with a check box have been subject to a large degree of scrutiny and have been implemented for both forward and reverse McCormick relaxations (Wechsung2015). Each McCormick object is associated with a parameter `T <: RelaxTag`

which is either `NS`

for nonsmooth relaxations (Mitsos2009, Scott2011), `MV`

for multivariate relaxations (Tsoukalas2014, Najman2017), or `Diff`

for differentiable relaxations (Khan2016, Khan2018, Khan2019). Conversion between `MV`

, `NS`

, and `Diff`

relax tags is not currently supported. Convex and concave envelopes are used to compute relaxations of univariate functions.

**Univariate McCormick Operators**

Arbitrarily differentiable relaxations can be constructed for the following operators:

- [x]
**Inverse**(inv) - [x]
**Logarithms**(log, log2, log10) - [x]
**Exponential Functions**(exp, exp2, exp10) - [x]
**Square Root**(sqrt) - [x]
**Absolute Value**(abs)

Both nonsmooth and Whitney-1 (once differentiable) relaxations are supported for the following operators:

- [x]
**Step Functions**(step, sign) - [x]
**Trignometric Functions**(sin, cos, tan) - [x]
**Inverse Trignometric Functions**(asin, acos, atan) - [x]
**Hyperbolic Functions**(sinh, cosh, tanh) - [x]
**Inverse Hyperbolic Functions**(asinh, acosh, atanh) - [x]
**Common Activation Functions**(relu, leaky_relu, param_relu, sigmoid, bisigmoid, softsign, softplus, maxtanh, pentanh, GeLU, ELU, SELU, Swish-1) - [x]
**Special Functions**(erf)

**Bivariate Operators: McCormick & McCormick**

The following bivariate operators are supported for two **MC** objects. Both nonsmooth and Whitney-1 (once differentiable) relaxations are supported.

- [x]
**multiplication**(*) - [x]
**division**(/)

Arbitrarily differentiable relaxations can be constructed for the following operators:

- [x]
**addition**(+) - [x]
**subtraction**(-) - [x]
**minimization**(min) - [x]
**maximization**(max)

### Common subexpressions

The following functions can be used in place of common subexpressions encountered in optimization and will result in improved performance (in each case, the standard McCormick composition rules are often more expansive).

`McCormick.xexpax`

— Functionexpax

The `expax`

function is defined as `expax(x, a) = x*exp(a*x)`

.

Form defined in Najman, Jaromił, Dominik Bongartz, and Alexander Mitsos. "Relaxations of thermodynamic property and costing models in process engineering." Computers & Chemical Engineering 130 (2019): 106571.

`McCormick.arh`

— Functionarh

The arrhenius function `arh`

is defined as `arh(x) = exp(-k/x)`

.

`McCormick.xlogx`

— Functionxlogx

The function `xlogx`

is defined as `xlogx(x) = x*log(x)`

.

`McCormick.mm`

— Function`mm(X,v,K) = v*X / (X + K)`

A Michaelis-Menten rate function.

### Bound Setting Functions

The following functions are used to specify that known bounds on a subexpression exist and that the relaxation/interval bounds propagated should make use of this information. The utility functions can be helpful in avoiding domain violations that arise due to the overly expansive nature of composite relaxations. Improper use of these functions may lead to cases in which the resulting relaxations are empty, so the user is encouraged to use discretion.

`McCormick.positive`

— Functionpositive(x::MC)

Sets the lower interval bound and the convex relaxation of `x`

to a value of at least `McCormick.MC_DOMAIN_TOL`

. (Sub)gradients are adjusted appropriately.

`McCormick.negative`

— Functionnegative(x::MC)

Sets the upper interval bound and the concave relaxation of `x`

to a value of at most `-McCormick.MC_DOMAIN_TOL`

. (Sub)gradients are adjusted appropriately.

`McCormick.lower_bnd`

— Functionlower_bnd(x::MC, lb::Float64)

Sets the lower interval bound and the convex relaxation of `x`

to a value of at least `lb`

. (Sub)gradients are adjusted appropriately.

`McCormick.upper_bnd`

— Functionupper_bnd(x::MC, ub)

Sets the upper interval bound and the concave relaxation of `x`

to a value of at most `ub`

. (Sub)gradients are adjusted appropriately.

`McCormick.bnd`

— Functionbnd(x::MC, lb, ub)

Sets the lower interval bound and the convex relaxation of `x`

to a value of at least `lb`

. Sets the upper interval bound and the concave relaxation of `x`

to a value of at most `ub`

. (Sub)gradients are adjusted appropriately.

### Specialized Activation Functions

`McCormick.pentanh`

— Functionpentanh

The `pentanh`

activation function `pentanh(x) = x > 0.0 ? tanh(x) : tanh(0.25*x)`

.

`McCormick.leaky_relu`

— Functionleaky_relu

The leaky Rectified Linear Unit activation function `leaky_relu(x) = max(x, 0.01x)`

.

`McCormick.param_relu`

— Functionparam_relu

The parametric Rectified Linear Unit activation function `param_relu(x, α) = (max(x, αx)`

with α in [0,1].

`McCormick.maxtanh`

— Functionmaxtanh

The `maxtanh`

activation function `maxtanh(x) = max(x, tanh(x))`

.

### References

**Khan KA, Watson HAJ, Barton PI (2017).**Differentiable McCormick relaxations.*Journal of Global Optimization*, 67(4): 687-729.**Khan KA, Wilhelm ME, Stuber MD, Cao H, Watson HAJ, Barton PI (2018).**Corrections to: Differentiable McCormick relaxations.*Journal of Global Optimization*, 70(3): 705-706.**Khan KA (2019).**Whitney differentiability of optimal-value functions for bound-constrained convex programming problems.*Optimization*, 68(2-3): 691-711**Mitsos A, Chachuat B, and Barton PI. (2009).**McCormick-based relaxations of algorithms.*SIAM Journal on Optimization*, 20(2): 573–601.**Najman J, Bongratz D, Tsoukalas A, and Mitsos A (2017).**Erratum to: Multivariate McCormick relaxations.*Journal of Global Optimization*, 68: 219-225.**Scott JK, Stuber MD, and Barton PI. (2011).**Generalized McCormick relaxations.*Journal of Global Optimization*, 51(4): 569–606.**Stuber MD, Scott JK, Barton PI (2015).**Convex and concave relaxations of implicit functions.*Optim. Methods Softw.*, 30(3): 424–460**Tsoukalas A and Mitsos A (2014).**Multivariate McCormick Relaxations.*Journal of Global Optimization*, 59:633–662.**Wechsung A, Scott JK, Watson HAJ, and Barton PI. (2015).**Reverse propagation of McCormick relaxations.*Journal of Global Optimization*, 63(1): 1-36.