Added a more representative training loop in the training interface tutorial with early stopping #1498
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1498 +/- ##
===========================================
- Coverage 89.60% 79.09% -10.51%
===========================================
Files 121 122 +1
Lines 9357 9394 +37
===========================================
- Hits 8384 7430 -954
- Misses 973 1964 +991
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Running this made no changes to the notebook |
There was a problem hiding this comment.
thanks for the updates. I left some minor comments. However, I am having trouble to see what this PR actually changes. Can you point me to the changes you made, e.g., just posting a comment describing the changes?
If you changed the training loop in the notebook for NPE, I think it would be good to change it for NLE and NRE accordingly as well. But please double-check with @michaeldeistler
| "id": "0abcbddf", | ||
| "metadata": { | ||
| "collapsed": false | ||
| }, | ||
| "source": [ | ||
| "Note: For the `DirectPosterior`, the batch dimension is optional, i.e., it is possible to sample for multiple observations simultaneously. Use `.sample_batch` in that case." |
There was a problem hiding this comment.
could you please fix this typo: it should be .sample_batched
| "source": [ | ||
| "### Custom Data Loaders\n", | ||
| "\n", | ||
| "One helpful advantage of having access to the training loop is that you can now use your own DataLoaders during training of the density estimator. In this fashion, larger datasets can be used as input to `sbi` where `x` is potentially an image or something else. While this will require [embedding the input data](https://sbi.readthedocs.io/en/latest/tutorials/04_embedding_networks.html), a more fine grained control over loading the data is possible and allows to manage the memory requirement during training.\n", | ||
| "One helpful advantage of having access to the training loop is that you can now use your own DataLoaders during training of the density estimator. In this fashion, larger datasets can be used as input to `sbi` where `x` is potentially an image or something else. While this will require [embedding the input data](https://sbi-dev.github.io/sbi/latest/tutorials/04_embedding_networks/), a more fine grained control over loading the data is possible and allows to manage the memory requirement during training.\n", |
There was a problem hiding this comment.
I think this link is outdate / points the old docs.
There was a problem hiding this comment.
it seems this is not fixed. Can you please fix the link @IzicTemi ?
IzicTemi
force-pushed
the
docs_tutorials
branch
from
f46f39c to
29b2075
Compare
There was a problem hiding this comment.
Thanks for the updates @IzicTemi!
Before merging this please:
- pull the most recent
mainfromsbi - checkout on your feature branch do another
git merge main - run
python tests/strip_notebook_outputs.py docs/tutorials. it will strip some outputs (I tested i locally) - run
git add docs/tutorials/18_training_interface.ipynbto add those changes - push to your remote branch.
- double check that your PR contains only your changes.
There was a problem hiding this comment.
the changes in this file are not part of this PR. probably due to a merge with main.
There was a problem hiding this comment.
please follow the steps in my comment above to fix this.
| "source": [ | ||
| "### Custom Data Loaders\n", | ||
| "\n", | ||
| "One helpful advantage of having access to the training loop is that you can now use your own DataLoaders during training of the density estimator. In this fashion, larger datasets can be used as input to `sbi` where `x` is potentially an image or something else. While this will require [embedding the input data](https://sbi.readthedocs.io/en/latest/tutorials/04_embedding_networks.html), a more fine grained control over loading the data is possible and allows to manage the memory requirement during training.\n", | ||
| "One helpful advantage of having access to the training loop is that you can now use your own DataLoaders during training of the density estimator. In this fashion, larger datasets can be used as input to `sbi` where `x` is potentially an image or something else. While this will require [embedding the input data](https://sbi-dev.github.io/sbi/latest/tutorials/04_embedding_networks/), a more fine grained control over loading the data is possible and allows to manage the memory requirement during training.\n", |
There was a problem hiding this comment.
it seems this is not fixed. Can you please fix the link @IzicTemi ?
The training notebook only has simple training examples.
The notebook was updated to show dataset splits and a training loop with early stopping after convergence adapted from the NPE base class train() method. A simple comparison with the inbuilt train method is also shown