Palestinian fighters and Syrian rebels initially pushed back the IS assault but they counter-attacked and have made fresh advances since Thursday inside the camp.
They have reportedly captured a square where the camp’s 18,000 residents usually gather to receive aid handouts.
The UN agency for Palestinian refugees UNRWA expressed concern about the situation, saying it had been unable to deliver aid to the camp since Wednesday.
“UNRWA remains acutely anxious about the humanitarian impact of continuing armed conflict between armed groups inside Yarmouk,” spokesman Chris Gunness said.
“With intense fighting continuing into a third day, the lives and safety of the 18,000 Palestinian and Syrian civilians inside Yarmouk are substantially threatened.”
The Observatory said government forces were shelling parts of Yarmouk intensively while Palestinian fighters battling IS militants were running low on ammunition.
The takeover marks another blow to refugees in the camp who have paid a heavy toll as a result of the conflict in Syria.
The conflict, which began as peaceful protests in March 2011 but developed into a civil war, has killed more than 230,000 people and prompted millions to flee their homes.
UNWRA reported last month that life for Palestinian refugees in Syria is becoming increasingly unsustainable, revealing that nine refugees drowned in March trying to reach Europe by boat.
Prior to the conflict, up to 600,000 Palestinian refugees lived in Syria, though the UN’s Palestine refugee agency UNRWA estimates that more than half have been forced to leave their homes due to violence in the country.
The Yarmouk camp has seen heavy fighting in recent years, including numerous sieges that have lasted months and led to widespread starvation.
Rebel groups took up positions in the camp more than a year ago. Over 200 camp residents have been killed since the army siege began, non-governmental groups say.
Syrian state media said the camp was under “the control of terrorist groups,” adding that there had been no army presence in Yarmouk “for a long time.”
The advance follows the capture of provincial capital Idlib by Al-Nusra Front and its allies on Saturday.
“Violent clashes have been underway since Thursday night between the army and Al-Nusra and its allies… around the Mastuma base,” the Britain-based Syrian Observatory for Human Rights said.
The base, seven kilometres (four miles) south of Idlib city, is the biggest regime base in the province, said Observatory director Rami Abdul Rahman.
The opposition groups had launched a “preventative attack on the base” where regime reinforcements were arriving ahead of a possible bid to recapture Idlib city, he said.
The Observatory added that clashes were underway between the Islamist forces and government troops in the area around the Shiite-majority village of Fuaa.
With the fall of Idlib city, regime forces in the province control only two towns and a handful of districts, along with the Abu Duhur military airport and five military bases.
