A 12-time Olympic medal winning swimmer wanted for questioning by Brazilian police returned to the United States before officers could seize his passport to ensure he was available for questioning, officials said Wednesday.

A Brazilian judge has issued a search and seizure warrant for US Olympic swimmer Ryan Lochte and teammate James Feigen as questions emerge about their account of being robbed at gunpoint early Sunday in Rio de Janeiro.

Two other swimmers, Jack Conger and Gunnar Bentz, were removed by Brazilian authorities on Wednesday night from their flight before it departed Rio de Janeiro to the United States, US Olympic Committee spokesman Patrick Sandusky told CNN Sports contributor Christine Brennan.

Federal police say Lochte left Brazil on Monday on a commercial flight, two days before the judicial order. Feigen is believed to still be in Brazil.

Lochte said he wouldn’t make up a story like this, NBC’s Matt Lauer reported Wednesday night. He said Lochte told him he and his three teammates were victims. NBC is broadcasting the Games in Rio.

Lochte compared the way police questioned him to a time in the United States when he talked with police after being robbed, Lauer said. Lochte said the Brazilian police asked few questions during what he thought was a casual interview. Lauer relayed that Lochte said he didn’t think police expressed any skepticism over his account.

Police arrived Wednesday morning at the Olympic Village to take the swimmers’ passports and gather further testimony from them about the reported robbery, Sandusky said earlier.

But the athletes were not there.

“The swim team moved out of the village after their competition ended, so we were not able to make the athletes available,” Sandusky said in a statement, adding that the committee would “continue to cooperate with Brazilian authorities.”

Lochte, Feigen, Bentz and Conger reported being robbed at gunpoint early Sunday in a taxi bound for the Olympic Village by individuals posing as armed police officers, the US Olympic Committee said Sunday.

The robbery — during which one of the bandits was said to have put a gun to Lochte’s forehead — reportedly occurred after the athletes left a party at the French hospitality house. Lochte told Lauer on Wednesday, however, that the gunman pointed the gun at him, but it wasn’t against his head.

Civil police say they are still looking for the taxi driver the athletes say they were with when they were robbed. Police say they have asked the driver to come forward.

Lochte’s lawyer: He would have stayed

The seizure warrant for the men’s passports, if served, would prevent them leaving the country.

Sandusky said that due to security protocols, he could not disclose the swimmers’ whereabouts. But Lochte, one of swimming’s biggest stars, has already returned to the United States, his attorney Jeff Ostrow told CNN.

Ostrow said Lochte, a gold medal winner in Rio, had “fully cooperated with the Brazilian authorities while in Rio.”

“He sat for a victim interview with the Brazilian Tourist Police, USOC Security, State Department, FBI, and anyone else that the Brazilian authorities requested to be present,” he said.

“He left for the United States as he was planning to after he completed his events. He was not asked by the Brazilian authorities to remain in Brazil for further investigation.

“Had they asked, he would have stayed and assisted. They still have not reached out to ask for additional information.”

Earlier Wednesday, the lawyer said that he and Lochte stood by the account provided to authorities.

“We stand by what he provided in that interview and signed off on.”

Judge’s questions

But questions remain.

Judge Keyla Blanc De Cnop said police still need more time to assess whether the US swimmers provided false information and if they filled a false crime report.

The judge said the athletes’ behavior upon arrival at the Olympic Village — combined with the inconsistencies in their statements — led police to question the veracity of their claim.

Judge Blanc De Cnop noted that Lochte had said a single robber approached the athletes and demanded all their money (400 real, or $124).

Feigen’s statement said a number of robbers targeted the athletes but only one was armed, the statement said.

Another potential issue highlighted by judge was the behavior of the athletes on arrival at the Olympic Village in the aftermath.

“It seems the alleged victims found their physical and psychological integrity were unshaken, even fooling about with each other,” said the judge, referring to footage from security cameras in the Olympic Village.

Video emerges

Public speculation has been fueled by the release of a video obtained by the Daily Mail, which it said was filmed shortly after the robbery.

The video appears to show the four swimmers returning to the Olympic Village looking relatively relaxed, and in possession of high-value items that might be expected to have been taken in an armed robbery.

CNN is unable to authenticate the video.

A spokeswoman for the Civil Police told CNN that the case had attracted attention among Brazilians because the victims said their phones and watches hadn’t been taken — items that would be a prized target for thieves in crime-plagued Rio.

She added that the issuance of search and seizure warrants was not a routine procedure but was sometimes done when concerns arose that information had been withheld.

She said that after Lochte’s statement to police, the three other swimmers were asked to present themselves at the police station to give their accounts. So far, only one had done so, she said.

Lochte’s harrowing account

Initial reports regarding the robbery were confusing, with an International Olympic Committee spokesman first claiming the reports were “absolutely not true” at a press conference Sunday.

He later backtracked and apologized, saying he was relying on initial information from the US Olympic Committee that was incorrect.

Speaking to NBC, Lochte described a harrowing encounter with the thieves.

“It’s been in Ohio as early as the mid-1850s at least, brought in as an ornamental plant because of its unique foliage and white flowers,” Gardner said. “It was actually planted in people’s landscaping, and it has been spreading.”

“We got pulled over, in the taxi, and these guys came out with a police badge, no lights, no nothing — just a police badge and they pulled us over,” he said.

“They pulled out their guns, they told the other swimmers to get down on the ground — they got down on the ground. I refused, I was like we didn’t do anything wrong, so — I’m not getting down on the ground.

“And then the guy pulled out his gun, he cocked it, put it to my forehead and he said, ‘Get down,’ and I put my hands up, I was like ‘whatever.’ He took our money, he took my wallet — he left my cell phone, he left my credentials.”