[BUG-REPORT]

[myloe00]

Thank you for reaching out and helping us improve Vaex!

Before you submit a new Issue, please read through the documentation. Also, make sure you search through the Open and Closed Issues - your problem may already be discussed or addressed.

Description
I have a dataset: 70G, 430 million rows and 6 columns.

And I try to use function export_hdf5 to export data. Then vaex running out of memory.

Finnally I found this line use all memory(code location: vaex.hdf5.writer.Writer.layout line:63):

    str_byte_length = {name: df[name].str.byte_length().sum(delay=True, progress=progressbar_strings) for name, dtype in dtypes.items() if dtype.is_string}

    arrow_null_count = {name: df.count(df[name], delay=True, progress=progressbar_count) for name, dtype in dtypes.items() if dtype.is_arrow or dtype.is_string}

And I try to remove param delay=True. everthing is OK.

I do not kown Whether it is an Issue.

Software information

• Vaex version (import vaex; vaex.__version__):

• {'vaex': '4.17.0', 'vaex-core': '4.17.1', 'vaex-viz': '0.5.4', 'vaex-hdf5': '0.14.1', 'vaex-server': '0.9.0', 'vaex-astro': '0.9.3', 'vaex-jupyter': '0.8.2', 'vaex-ml': '0.18.3'}

• Vaex was installed via: pip / conda-forge / from source

• OS: linux

Additional information
Please state any supplementary information or provide additional context for the problem (e.g. screenshots, data, etc..).