Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SprintCacheDataset issue with torch backend #1682

Open
robin-p-schmitt opened this issue Jan 28, 2025 · 0 comments
Open

SprintCacheDataset issue with torch backend #1682

robin-p-schmitt opened this issue Jan 28, 2025 · 0 comments
Assignees

Comments

@robin-p-schmitt
Copy link
Contributor

For the latest RETURNN, when using torch backend and SprintCacheDataset, I get this error.

File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/datapipes/datapipe.py", line 179, in IterDataPipe.__reduce_ex__
    line: return super().__reduce_ex__(*args, **kwargs)
    locals:
      super = <builtin> <class 'super'>
      __reduce_ex__ = <not found>
      args = <local> (4,)
      kwargs = <local> {}
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/datapipes/datapipe.py", line 354, in _DataPipeSerializationWrapper.__getstate__
    line: value = pickle.dumps(self._datapipe)
    locals:
      value = <not found>
      pickle = <global> <module 'pickle' from '/usr/lib/python3.10/pickle.py'>
      pickle.dumps = <global> <built-in function dumps>
      self = <local> _IterDataPipeSerializationWrapper
      self._datapipe = <local> BatchingIterDataPipe
TypeError: cannot pickle '_io.BufferedReader' object

The full error log and returnn config can be found here: https://gist.github.com/robin-p-schmitt/dca90f178d1bfc086f8faadb35211406

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants