Yemeni President Ali Abdullah Saleh (Photo via Creative Commons)

The remarks come in the face of increasing pressure to resign from the presidency.

"I tell those who appear in the media asking others to leave, that it is up to them to go," said Saleh on state news agency Saba.

Also on Tuesday, Saleh pulled his counerterrorism forces out of Abyan, a southern province plagued by Al Qaeda.

Global Post reports of Saleh's withdrawal from Abyan:

The move appeared to coincide with a deadly explosion at an ammunitions factory in Abyan on Monday that killed more than 100 people. The explosion took place outside the city of Jaar as someone lit a cigarette, igniting explosive materials inside the facility, witnesses said.

Yemen’s state news agency issued a statement claiming that Al Qaeda in the Arabian Peninsula was responsible for the explosion.

Saleh's opponents blame the president for Al Qaeda's resurgence in the country.

"We condemn this ugly crime and accuse the president and his people of involvement with Al Qaeda and armed groups to whom government institutions have been handed over in Abyan. The chaos was planned in advance," said the main coalition group of opponents in a statement.

Opposition groups demand Saleh step down immediately, rejecting a reported offer by Saleh to leave at the end of 2011. Saleh's ruling party says he should be allowed to stay through his current term, which ends in 2013.

Some fear that Saleh's defiance will lead to more bloodshed and a civil war. Many of the president's military generals have defected, including Saleh's right-hand man Major General Ali Mohsin.