Ant and Dec almost quit Britain's Got Talent in 2012.
The famous hosting duo made the revelation in their new book, Once Upon A Tyne, the Sun has confirmed.
MORE: Amanda Holden delights fans with exciting BGT news
In pages seen by the paper, the Newcastle pair revealed that they felt that they "hardly seemed to feature at all" when watching footage back, leading them to feel as though they were being "side-lined" by the show.
WATCH: Ant and Dec reflect on their 39 awards
The famous pair wrote: "After filming interviews and interacting with every single act, as well as making jokes, giving reactions and everything else we always do, we found that when the show went out on TV, we hardly seemed to feature in it at all. The experience left us feeling like we were being side-lined."
Ant and Dec then flew to Los Angeles for a crisis meeting with Simon Cowell, adding that they "let him have it".
MORE: Simon Cowell's future on Britain's Got Talent finals revealed
Ant and Dec have hosted the show since its launch in 2007
MORE: Alesha Dixon reveals daughter's epic four-tiered first birthday cake
The music titan then promised that things would change, and the Saturday Night Takeaway hosts confirmed that doting dad Simon was "good to his word".
"We let him have it with both barrels," they explained.
"We told him everything - that we weren't being used, that we could be doing other stuff, that maybe someone else should take over.
"He made a promise to us that things would change and that we'd never feel like that again.
"He desperately wanted us to stay and we told him we'd do the next series and see how it went - and, to be fair to him, he was as good as his word and things did change."
It's also reported that Simon himself contributed to the book, writing that he appreciates the fact that if ever Ant and Dec are annoyed about something, they will always express their grievances to his face.
Like this story? Sign up to our newsletter to get other stories like this delivered straight to your inbox.