Developer Notes

This section contains explanations for implementation details that might otherwise appear strange.

Limiting Memory Usage in JuMP

MPEC and GEL estimation modes use the JuMP package with the Ipopt solver. The JuMP model definitions in estimateBLP introduce many auxillary intermediate variables and constraints. JuMP is able to automatic generate efficient code to evaluate linear and quadratic expressions (and their derivatives). JuMP's handling of nonlinear expressions is less efficient (especially in terms of memory usage). A straightforward implementation of the model in JuMP using many nonlinear expressions results in memory usage that grows very quickly with the number of products per market. This can become prohibitive when the number of produces per market exceeds 50 or so.

To use JuMP with many products, it is useful to limit the presence of nonlinear expressions to involve as few constraints and variables as possible. This can be done by introducing auxillary varibles subject to linear and/or quadratic constraints. A good rule of thumb is that introducing auxillary variables will be beneficial when it makes the Hessian more sparse.

For example, rather than simply defining the share constraints as

\[s[j,t] = \frac{1}{S} \sum_{i=1}^S \frac{\exp(\delta[j,t] + x[:,j,t]'(\nu[:,i,t] .* \sigma)}{1 + \sum_{\ell} \exp(\delta[\ell,t] + x[:,\ell,t]'*\nu[:,i,t])}\]

the code defines intermediate variables, u, n, and d with extra constraints

\[u[i,j,t] = \delta[j,t] + x[:,j,t]'(\nu[:,i,t] .* \sigma)\]

\[n[i,j,t] = \exp(u[i,j,t])\]

\[d[i,t] = 1 + \sum_{j} n[i,j,t]\]

\[si[i,j,t]*d[i,t] = n[i,j,t]\]

\[s[j,t] = \frac{1}{S} \sum_{i=1}^S si[i,j,t]\]

The feasible set is unchanged, but now only the constraint involving n and u is not linear or quadratic.

Note that estimateRCIVlogit has not yet been modified in this way. Hence, estimateRCIVlogit with method =:MPEC or :GEL is not very efficient for models with a large number of products per market.