Virginia Senator Tim Kaine and Indiana Governor Mike Pence proved they weren't such staid gentlemen politicians after all in a wild free-for-all vice presidential debate with both slugging hard.

Pence was the calmer, slower-talker of the two, often sadly or disdainfully shaking his head, while Kaine was frequently on the attack — and interrupting.

The two clashed sharply on deportation, violence, insults, abortion and taxes.

Pence side-stepped several attacks on Trump, and also discussed some positions that seemed more like his than his boss's.

Many points were lost as the two talked loudly over one another repeatedly, often ignoring pleas from CNN moderator Elaine Quijano to "gentlemen, please" speak one at a time.

Quijano early in the debate pressed Pence about the New York Times report that Donald Trump may have used a billion-dollar business loss to avoid paying federal income taxes for 18 years.

"His tax returns showed he went through a very difficult time, but he used the tax code just the way it's supposed to be used," Pence responded. "And he did it brilliantly."

"How do you know that?" Kaine asked. "You haven't seen his tax returns."

Pence responded that Trump's business is worth billions. Kaine asked again, repeatedly pressing for Trump's release of his tax returns: "How do you know that?"

Kaine also snapped back about Trump's "brilliant" tax strategy: "So it's smart not to pay for our military? It's smart not to pay for our veterans? It's smart not to pay for teachers? I guess the rest of us who do pay for those things, I guess we're stupid."

The word likely repeated most often during the debate was "rapist," as Kaine continually came back to Donald Trump's insult during his primary campaign that Mexican immigrants in the US "are rapists" and "bringing crime."

The final time Kaine repeated it, Pence snapped: "Senator, you whipped out that Mexican thing again" which quickly became a favourite of tweeters.

Kaine responded: "Can you defend it?"

"Look, there are criminal aliens in this country," Pence said.

But Pence seemed to part ways with Trump over mass deportation of illegal immigrants.

Pence insisted that Trump would focus first on deporting "criminal aliens."

"Once we have accomplished that ... we'll deal with those that remain," he added, without offering details. Trump said repeatedly during the GOP primary that he would immediately deport 11 million immigrants in the US.

"Donald Trump believes in deportation nation," said Kaine. "You've got to pick your choice."

"That's nonsense," Kaine said. "It's nonsense." Though he didn't explain exactly what was nonsense.

Pence also differed sharply with Trump over his views of Russian President Vladimir Putin. He called Putin a "small and bullying leader," in contrast to Trump, who has heaped praise on Putin.

Kaine reminded Pence that Trump has called Putin a better leader than President Barack Obama, to which Pence replied: "That is absolutely inaccurate." Trump said it on a nationally televised forum.

Pence also called for US military strikes against Russian ally, the regime of Syrian leader Bashar al-Assad, in the event of a move against the rebel enclave around Aleppo.

"The provocations by Russia need to be met with American strength," Pence said in a far stronger position than one ever expressed by Trump. "The United States of America should be prepared to use military force to strike military targets of the Assad regime."

As for violence at home and police shootings of black men, Kaine pushed for community policing to nurture police trust in citizens and their trust in the police.

Pence backed the idea of community policing but also attacked "people that seize upon tragedy in the wake of police action shootings as a reason to use a broad brush to accuse law enforcement of implicit bias or institutional racism. And that really has got to stop."

The two men clashed over both Trump's and Clinton's Foundations with Pence indicating that donors to the Clinton organisation were attempting to purchase favours.

"I mean the reality is, when she was secretary of state, Senator, come on, she had a Clinton Foundation accepting contributions from foreign governments," said Pence.

Kaine defended the Clinton Foundation's good works and pointed out that the New York state attorney general has just barred Trump's charity from collecting money in the face of questionable donations, including a political contribution to a Florida state attorney general considering legal action against Trump University.

Kaine said that he and Clinton were firmly committed to the constitutional right to abortion for women, and asked Pence why Trump doesn't trust women to make the decision for themselves. He also asked why Trump wants to punish women who have abortions (Trump later walked back those comments).

"Donald Trump and I would never support legislation that would punish women who made the heartbreaking choice," said Pence.

Kaine asked: "Then why did he say that?"

"Look, he is not a polished politician like you and Hillary Clinton," said the governor.

"When Trump says women should be punished or Mexicans are rapists and criminals, or John McCain is not a hero — he is showing you who he is," said Kaine.