Amnesty International is urging the international community not to return refugees to Afghanistan while the country remains largely unstable.
At least one million Afghans have gone back since the fall of the Taliban last year.
The UN predicts two million more will return before year's end.
While the UN High Commission for Refugees (UNHCR) is facilitating returns for those who want to go, it has not actively encouraged it.
A spokesman for Amnesty said: "Afghanistan is far from stable as fighting continues, crime and banditry are rife, women and ethnic groups have been targeted for abuse, and there are thousands of unexploded land mines.
"While the situation remains fluid, governments should not encourage voluntary repatriation.
"Governments should take the lead from UNHCR. Encouraging returns or forcibly returning refugees at this tender stage in Afghanistan's transition is premature, irresponsible and unsustainable."
Since the ousting of the Taliban, officials from countries including Britain and Australia, have said they believe the situation has changed enough so that Afghan asylum seekers could be returned to Afghanistan.
"Those making decisions on Afghan asylum claims must be bound solely by international standards for protection, rather than any domestic political agenda," Amnesty International said.
Some 3.7 million refugees fled Afghanistan after two decades of war and drought.