GPT2 getting stuck on loop

Follow the full discussion on Reddit.
I'm relatively new to ML and have been running my own finetuning of GPT2 for a while. I'm not sure if this is the best or most appropriate place to ask this, so please forgive me if not (and feel free to redirect me elsewhere). With many prompts, GPT2 seems to get stuck in an infinite loop repeating the input phrase or part of the input phrase, particularly when I pass it titles. It will also randomly spit out the endtext token, or say things like Join my mailing list! Read the comments blow at the start of an "article".


There's unfortunately not much to read here yet...

Discover the Best of Machine Learning.

Ever having issues keeping up with everything that's going on in Machine Learning? That's where we help. We're sending out a weekly digest, highlighting the Best of Machine Learning.

Join over 700 Machine Learning Engineers receiving our weekly digest.

Best of Machine LearningBest of Machine Learning

Discover the best guides, books, papers and news in Machine Learning, once per week.