Add lognormal delay distribution

[essink]

Addressing #44

@morales-gregorio and I adapted some code that I hacked together previously.

As far as I know, there is no analytical expression for the Fourier-transform of the lognormal distribution. Thus the code uses a simple numerical integration of the Fourier-transform, integrating real and imaginary part separately. The choice of the integration times is chosen based on experience of how fast the lognormal distribution decays.

[moritzlayer]

@essink Thanks for the PR. I was surprised to learn that there is no known analytical form of the Fourier transform of the lognormal distribution. Have you checked the convergence properties of the naive implementation somehow? I found a few sources stating that convergence is often quite poor (e.g. https://ieeexplore.ieee.org/document/4471924).

[morales-gregorio]

Hi @moritzlayer ! Thanks for having a look. How would you suggest testing for convergence?
Best,
Aitor

[moritzlayer]

That is an excellent question. Beaulieu and colleagues have published quite a number of papers on the characteristic function of the lognormal distribution. In Saberali & Beaulieu (2012) they calculate the characteristic function empirically, by drawing samples from the lognormal distribution (see chapter V. Simulation Results). However, they seem to know what they are doing, and in Beaulieu & Xie (2004) they state that a FFT is not a good option. Instead they come up with a better integral form in Beaulieu (2008) (see Eqs. 6a & 6b). I would propose to simply trust them and implement these integrals instead.

[moritzlayer]

I just realized that you are not using a FFT. Nevertheless, I would go with their integral.

[moritzlayer]

BTW, could you please change the target directory for this PR to develop? We are merging all PRs to develop and once we are ready for a new release, we merge develop into master.

[morales-gregorio]

Hey @moritzlayer thanks for the input! I have cleaned up the working tree here, and looked into the Beaulieu paper. The only problem I see is that they use zero-mean distributions, which may not be our case, especially given that the lognormal distribution is only defined in $\mathbb{R}^+$.

Do you think we can do the same approximation as they did but without excluding the mean or is that necessary?

[morales-gregorio]

I use the following notebook for testing with a microcircuit model that has lognormal delay distribution

[moritzlayer]

Hey @morales-gregorio, I refactored the code a bit for faster integration and found two bugs in the implementation (the argument of cosine and sin was 1/y everywhere, but it needs to be y in two cases; additionally, variance and mean were interchanged in the calculation of the parameters of the underlying Gaussian). Furthermore, I wrote a test in nnmt/tests/unit/general/test_network_properties.py that checks whether the results of the integration makes sense. You can run it from the nnmt main directory using

pytest tests/unit/general/test_network_properties.py::Test_lognormal_characteristic_function -v

It shows that the results do make sense, but there are cases in which the numerical integration fails to give proper results, because the integrands are strongly oscillatory. I also tried some test cases which are relevant for your experiment, and for them the integration seems to work.

I have not cleaned up the code yet, so the old code is still there.

Next, I will work on the interface of the code, such that it nicely fits into the toolbox.

[moritzlayer]

Hey @morales-gregorio, I finished the implementation of the lognormal delay distribution matrix and implemented an integration test. Could you please do the following three things:

• Try running the code with your script. I hope everything works now.
• Have a look a the code, especially the unit test and confirm that this makes sense. Note that the unit tests still include cases in which the numerical integration fails.
• Change the target branch of this pull request to develop instead of master

Let me know when you are done. If you do not see any further problems, we can merge the PR.

[morales-gregorio]

Hi! I have downloaded the current implementation and can confirm it runs fine with my setup (the regular microcircuit that also Simon was running). Here is a figure of the eigenvalues at different frequencies for the different populations, notably the lognormal delays seem to not only rotate but also deforms them, especially at the higher frequencies (150-300 Hz).

I will have a closer look at the code in the following days