[WIP] Imbedded Laplace Approximation

[SteveBronder]

Summary

Code for the embedded laplace approximation. The tests are all passing but there is a few things still needed

• Update docs for all functions (@charlesm93 I added the basic arguments for a lot of the functions but could use your expertise to give the definitions for each argument)
• Add corresponding rngs for each specialized laplace approximation (and tests)
• Add tests for negative binomial

The file for laplace have been added to the mix folder since it uses higher order auto diff.

The current signature for the generalized laplace looks like the following in C++

inline auto laplace_marginal_lpdf(LFun&& L_f, LArgs&& l_args,
                                  const Theta0& theta_0, CovarFun&& K_f,
                                  std::ostream* msgs, Args&&... args)

which will translate to stan like the following

target += laplace_marginal_tol_lpdf(
  likelihood_functor,  make_tuple(data_arg1, data_arg2), 
  eta, theta_init,
  covariance_function, 
  1e-6, 10, 2, 1, 5, // tuning args 
  covar_fun_arg1, covar_fun_arg2);

Note that the first tuple used for the likelihood arguments must be data.

Instead of using a tuple for the first functor's inputs and variadic arguments for the covariance functors arguments I would rather have them both be tuples like the following

target += laplace_marginal_tol_lpdf(
  likelihood_functor, make_tuple(data_arg1, data_arg2), 
  covariance_function,  make_tuple(covar_fun_arg1, covar_fun_arg2)
  theta_init,
  eta,
  1e-6, 10, 2, 1, 5);

I think this is nice because it makes it makes the tolerance parameters always sit at the end and both functors have the same input scheme for their arguments. Does anyone have thoughts on this

---

Other additions related to this PR

• A filter_map function that applies a conditionally applies a lambda f to each input of a tuple given a type_trait i.e. the following code would print "fp detected" twice and increment the double elements of the tuple by 1.

std::tuple<double, int, double> tup{1.0, 2, 3.0};
std::tuple<double, int, double> tup2 = filter_map<std::is_floating_point>([](auto x) { 
  std::cout << "fp detected" << std::endl;
  return x + 1;
  }, tup);

The test_ad suite now has a compile time option for only running the tests with only prim and reverse mode with a new boolean template parameter to expect_ad. This is needed to use laplace with the test framework as the laplace impl here does not work with higer order autodiff (since it needs higher order autodiff)

Tests

Since the tests all seem very related I kept them in their own folder, is that alright? Or should I distribute them across the test folders like normal? While this PR is WIP I'm going to leave them in the same folder and if we don't want that then we can move them before we merge

./runTests.py -j20 ./test/unit/math/mix/laplace/ 

Side Effects

Release notes

Checklist

• Copyright holder: Simon's Foundation

  The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
  - Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
  - Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

• the basic tests are passing

  • unit tests pass (to run, use: ./runTests.py test/unit)
  • header checks pass, (make test-headers)
  • dependencies checks pass, (make test-math-dependencies)
  • docs build, (make doxygen)
  • code passes the built in C++ standards checks (make cpplint)

• the code is written in idiomatic C++ and changes are documented in the doxygen

• the new changes are tested