Hi folks, How would you best recommend that I pass gradients through generate? below is a rough code snippet explaining the objective. I am thinking that I could take the hypo_ids directly from the model output (instead of from generate), but this is no longer natural because teacher-forcing is us…

EDIT: The only method seems to be to use RL to simulate the sampling that occurs. see https://papers.nips.cc/paper/8682-training-language-gans-from-scratch.pdf

@yjernite is also interested in this line of work. I would write a method similar to parlai’s decode_forced that forces the model to decode the tgt sequence and estimates its probability, then backprob the sum of the GT sequence. I’m not sure if that will lead to super similar results to the curre…

I just tried a simple ffnn to replicate argmax, but found that the gradients are almost always zero which makes sense I guess - changing other vector values will almost never change the maximum value.

Hello, I’m trying to do something similar. Did you manage to implement something working?

Carrying Gradients Through Generate

Research

patrickvonplaten November 2, 2020, 12:55pm 5

This should also be interesting: Big `generate()` refactor

Topic		Replies	Views
Disparity between output from `forward` and `generate` for greedy search (using Whisper) 🤗Transformers	2	1504	March 20, 2024
How to output loss from model.generate()? 🤗Transformers	16	6405	January 7, 2025
How to generate without decoding? 🤗Transformers	1	430	December 13, 2023
Language model gradients sensitive to target value/length Research	0	362	June 16, 2023
A potential in-place operation that caused an RuntimeError 🤗Transformers	1	2356	January 19, 2021

Carrying Gradients Through Generate

Related topics