fixed autocorr + Lempel Ziv Complexity + Some failing tests

[abstractqqq]

The issue is that in lazy Polars, slice behaves differently. The problem lies somewhere in its length calculation. Try running the following code, which will give us error in the lazy case. Since in our original code, we are just returning one expression at the end, the error didn't occur, but something inside was wrong. You can see the issue here: pola-rs/polars#11594 (comment)

df = pl.DataFrame({
    "a": [1,2,1,2,1,2],
    "b": [None, None, 1,1,1,1]
})

df.select(
    pl.col("a").slice(offset = 0, length=pl.count() - 2).alias("1"),
    pl.col("a").slice(offset = 1, length=pl.count() - 2).alias("2")
)

df.lazy().select(
    pl.col("a").slice(offset = 0, length=pl.count() - 2).alias("1"),
    pl.col("a").slice(offset = 1, length=pl.count() - 2).alias("2")
).collect()

I fixed the bug by reverting back to using shift, which will be slower than .head() / tail() / slice(), but will avoid using slice for now. We actually do not need to drop nulls after shifting, because dot product will null will just result in 0.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
functime-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 9, 2023 2:35am

[topher-lo]

Just renamed value to threshold

[abstractqqq]

@MathieuCayssol Autocorrelation should be correct now. Please take a look

[MathieuCayssol]

Sure, I will have a look. For the unit tests, the things you've updated, I already did it in the branch #66