US and Canadian media reported that Carter had been battling prostate cancer.
Carter was a middleweight contender before he was convicted in the 1966 murders of three people who were shot and killed at a tavern in Paterson, New Jersey.
A fearsome fighter, Carter scored his biggest win in 1963, when he stopped past and future world champion Emile Griffith in the very first round of a non-title clash.
He lost a 15-round unanimous decision to Joey Giardello in a middleweight world title fight in 1964 -- his only world championship bout.
"He could have gone a long way," Griffith said. "I should know. He knocked me down and stopped me."
- An 'appeal to racism' -
However, Carter's ring career was abruptly curtailed by his triple murder conviction in 1967.
He denied the crime, and his story caught the attention of boxing great Muhammad Ali and inspired Bob Dylan's 1975 song "Hurricane."
Decades later, 1999 flick "The Hurricane," earned Denzel Washington an Oscar nomination and a Golden Globe for his portrayal of Carter, although some factual inaccuracies in the film provoked criticism.
The actor paid tribute to Carter in a statement to US media: "God bless Rubin Carter and his tireless fight to ensure justice for all."
Carter was convicted along with his friend John Artis, who was also black, by an all-white jury in the death of two white men and a white woman.
The convictions were gained in part with the testimony of two convicted felons who placed Carter and Artis at the scene, but later recanted.
Carter was given a second trial in 1976 and convicted yet again.
But, in an interview with PBS in 2011, Carter said he "wouldn't give up."
"Just because that misinformed jury found me guilty, did not make me guilty. And because I was not guilty, I refused to act like a guilty person.''
In 1985, Carter was exonerated by US district court judge, H. Lee Sarokin, who said the conviction had been "based on an appeal to racism rather than reason."
- 'Great deal of courage' -
After finally being released, Carter, a native of New Jersey, moved to Toronto.
From 1993 to 2005, he served as the executive director of the AIDWYC, and the organization's founder, Win Wahrer, told Canadian broadcaster CBC on Sunday that Carter was a "hero" for his work advocating for the wrongly imprisoned.
"It took a great deal of courage to speak out as he did on a consistent basis and relive his own experience," she said. "But he did it because he felt it was the right thing to do."
Carter continued his efforts on behalf of those he believed to be wrongly incarcerated even after he became ill.
In February, he penned an article in the New York Daily News seeking the release of David McCallum, who was convicted of a kidnapping and murder in 1985.
"I am now quite literally on my deathbed and am making my final wish to those with the legal authority to act," he said.
"My single regret in life is that David McCallum of Brooklyn... is still in prison."
Carter, whose formal education ended in the eighth grade, led a childhood dotted with unruly incidents before enlisting in the Army, where he first took up boxing.
With an honorable discharge, he returned home, overcoming a past that included petty crime to build his professional ring career.