Iranian boats aggressively approached three US naval ships in the Strait of Hormuz at the weekend and threatened that the ships would explode, US officials claim.
Iran dismissed US concerns about the incident in the strait, a main shipping route for Gulf oil, saying it was a routine contact.
But the Pentagon termed the Iranian actions "careless, reckless and potentially hostile" and said Tehran should provide an explanation.
"This is a very volatile area and the risk of an incident escalating is real," US Defence Secretary Robert Gates said. "It is a reminder that there is a very unpredictable government in Tehran."
The incident was the latest sign of tension between Washington and Tehran, at odds over a range of issues from Iran's nuclear programme to US allegations of Iranian support for terrorism and interference in Iraq.
President George W. Bush is due to travel to the Middle East this week on a trip he has said is partly aimed at countering Iranian influence.
In Tehran, the Iranian Foreign Ministry described the encounter as "ordinary" and said it had been resolved.