Photographs of then Crown Prince Frederik with Mexican socialite Genoveva Casanova in Madrid sparked salacious rumours when they were published by by Spanish gossip magazine, Lecturas, in November.
Philanthropist and model Genoveva, 47, denied having any romantic relationship with Frederik, 55, while the Danish royal household did not comment.
And now Genoveva's ex-husband has spoken out about the effect the images have had on the family.
Cayetano Martínez de Irujo, who is the current Count of Salvatierra and Duke of Arjona, appeared on Susanna Griso's show, Espejo Publico, on Tuesday.
The father-of-two, 60, said of the published photographs: "Well, that is an issue that will be seen in the courts, they have caused a lot of pain to [Genoveva] and my children, a lot of pain."
He said that the former couple's children, 22-year-old twins Luis and Amina, are working and that they are suffering a lot with this story. Cayetano explained that Genoveva has filed her own lawsuits through his lawyers.
Following publication of the images in Lecturas, HOLA! reported at the time that a mutual friend that Frederik and Genoveva have in common was unable to accompany the then Prince to visit an art exhibition due to illness and he asked Genoveva to go in his place.
After visiting the exhibition 'Picasso, the sacred and the profane', Frederik and Genoveva walked through El Retiro Park and had a very public dinner together at the restaurant 'El Corral de la Morería', in the heart of Madrid
In a statement to HOLA! at the time Genoveva said: "I categorically deny the statements that suggest a romantic relationship between Prince Frederik and me."
She added: "Any statement of this type not only completely lacks the truth but also misrepresents the facts in a malicious manner. This is already in the hands of my lawyers, who will take care of the pertinent steps to protect my right to honor, truth and privacy."
The mother-of-two also posted a legal letter on her personal Instagram page, but as of last week, she appears to have deleted her social media account.
Frederik succeeded his mother Queen Margrethe to the Danish throne on 14 January, following Margrethe's abdication.
The father-of-four is set to travel to Poland on Wednesday to host his first business delegation.
Meanwhile, Frederik's wife, Queen Mary, travelled to Germany last weekend to attend the Men's EHF Euro 2024 final.
LISTEN: How King Frederik really feels about Queen Mary's popularity