Agostini/AP, Fox

'American Idol' judge Simon Cowell tells the press that neither Madonna nor Howard Stern could replace him as a judge.

The acerbic British judge, who's leaving the hit Fox competition at the end of the season to helm a U.S. version of "The X Factor," was grilled by reporters yesterday about who could fill his shoes.

"You're never going to pick anyone as good as you are, are you?" Cowell joked. "That's why I don't think anybody [at Fox] really asks me that question."

He shrugged off talk about Stern and labeled Madonna a pipe dream, but said gossip blogger Perez Hilton, who has his own music label under Warner Music Group, might be a good fit.

"Perez would be funny," Cowell said. "He's got good taste in music, he's got a personality, that could work."

Cowell was "fairly certain" Stern hasn't been approached by producers, but also said he isn't sure the radio host would be up to the task.

"You've just got to know what you're talking about," he said. "We have to have people on these shows who actually know what they're talking about, rather than just guessing. They've really got to have experience, so that they can not just criticize, but offer constructive advice as well."

Another major factor is the time commitment. Cowell said he doesn't think a big name like Madonna could manage that.

"The good news for [the producers] is that there's no shortage of candidates," Cowell said.

In response to chatter that he and new judge Ellen DeGeneres don't get along, Cowell said there was "no fallout" after he arrived 20 minutes late to the set during Hollywood Week (he was at a press conference announcing his departure from the show) and that he thinks she was the right choice to replace Paula Abdul.

Speaking of Abdul, despite a contentious relationship on air, Cowell said he does miss the loopy choreographer.

"Even though we used to argue a lot," he said, "she is somebody who I got close to over the years. We'd hang out after the show, and she always made me laugh. I thought she was funny. It's like not having your friend on the show anymore."

Cowell predicted a win for the "Idol" ladies this season, saying most of the male contestants are on the same level and could cancel each other out. He added that the music industry at this moment is more favorable for females.

"You want somebody who represents what's going on at the moment," he observed. "I'd love to find a Taylor Swift. Somebody who's relevant, rather than just a contestant winner."

Along the same lines, Cowell would love to have Lady Gaga on the show as a mentor.

"She is the most relevant pop artist in the world right now," said Cowell. "And she's very smart. I like her."

Away from "Idol," Cowell did not discuss conflicting reports yesterday about a possible engagement to Mezhgan Hussainy, and a Fox spokeswoman said that there would be "no personal questions" taken in his chat with reporters. Cowell's spokesman told Hilton the judge wasn't engaged.

As for the rest of the season, Cowell vowed not to take a back seat to other judges just because it's his finale. He said he hopes the show remains a hit without him.

"I'm going to feel sad when it all ends ... but I want it to go out on a high," he said. "It's my last season. I want it to be successful, and I'll do everything I can to make it happen."

The "Simon 2.0" search should help with that, he said, because "there's a lot of interest to who's going to replace me."