[functorch] No-Op GradSampleModule #492
Conversation
|
@alexandresablayrolles has updated the pull request. You must reimport the pull request before landing. |
|
@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@alexandresablayrolles has updated the pull request. You must reimport the pull request before landing. |
|
@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
91dd689 to
4c40f1b
Compare
|
@alexandresablayrolles has updated the pull request. You must reimport the pull request before landing. |
|
@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
4c40f1b to
1286565
Compare
|
@alexandresablayrolles has updated the pull request. You must reimport the pull request before landing. |
|
@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@alexandresablayrolles has updated the pull request. You must reimport the pull request before landing. |
|
@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
| """ | ||
| ExpandedWeights-based implementation of AbstractGradSampleModule | ||
|
|
||
| Computes per-sample gradients using PyTorch built-in mechanism of ExpandedWeights. | ||
| See README.md for more details | ||
| """ |
|
@alexandresablayrolles has updated the pull request. You must reimport the pull request before landing. |
|
@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
55c8ff5 to
b385930
Compare
|
@alexandresablayrolles has updated the pull request. You must reimport the pull request before landing. |
|
@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
TL;DR: Adding a No-Op GradSampleModule in case the grad samples are computed by functorch. The CIFAR10 example has been updated to show a typical use-case for that.
The neat thing about functorch is that it directly gives the per-sample gradients with a couple of lines of code. These per-sample gradients are then manually given to
p.grad_sampleby the end-user.