"He has been freed but we have no details so far on the circumstances of his release," Mohammed Abdelaziz told.
Government spokesman Mohamed Kaabar told the state LANA news agency that the premier had been "freed, not released", without saying how.
He said Zeidan was "in good health" but did not elaborate on what he meant by his not being released.
Moments before news broke of Zeidan's release, Deputy Prime Minister Al-Seddik Abdelkarim had vowed that the government would not give into the demands of the perpetrators of a "criminal act".
"The government will not give in to blackmail by anyone," he said
The pre-dawn seizure of Zeidan came five days after US commandos embarrassed and angered Libya's government by capturing senior Al-Qaeda suspect Abu Anas al-Libi off the streets of Tripoli and whisking him away to a warship.
A source in the premier's office said Zeidan had been taken by gunmen from Tripoli's Corinthia Hotel, where he resides. A hotel employee confirmed a pre-dawn raid by "a large number of armed men".
The cabinet met in emergency session earlier in the morning.
A government statement said Zeidan had been taken "to an unknown destination for unknown reasons by a group" of men believed to be former rebels.
The Operations Cell of Libyan Revolutionaries, comprising former rebels and which had roundly denounced Libi's abduction and blamed Zeidan's government for it, said it had "arrested" Zeidan under orders from the public prosecutor.
But the cabinet said on its Facebook page that ministers were "unaware of immunity being lifted or of any arrest warrant" for the premier.
Thursday's government statement said it suspected both the Operations Cell of Libyan Revolutionaries and the Brigade for the Fight against Crime of being behind the raid that netted Zeidan.
Both groups loosely fall under the control of the defence and interior ministries but largely operate autonomously.
Two years after the revolution that toppled Kadhafi, Libya's new authorities are struggling to rein in tribal militias and groups of former rebels.
