In a statement, his family said he passed away "surrounded by family in his beloved Taos, New Mexico".
Current US Defense Secretary Lloyd Austin praised Mr Rumsfeld's "remarkable career", writing on Twitter that he was "propelled by his boundless energy, probing intellect, and abiding commitment to serve his country".
Mr Rumsfeld, who served in the US Navy in the 1950s, enjoyed a long and storied political career under four different presidents and was widely regarded as smart and combative, patriotic, politically savvy and a visionary of a modern US military.
However, this reputation was tainted by the long and costly war in Iraq.
He was the only person to serve twice as Pentagon chief, firstly as the youngest ever person to hold the position from 1975-77 and later as the oldest between 2001 and 2006.
In 2001, he began his second term as Defense Secretary under then-President George W Bush, but his initial plans were overshadowed by the 9/11 terror attacks.
Mr Rumsfeld went on to oversee the subsequent invasion of Afghanistan and the 2003 overthrow of Saddam Hussein in Iraq, where he was blamed for setbacks including the Abu Ghraib prisoner abuse scandal and for being slow to recognise a violent insurgency.
He twice offered his resignation to President Bush in 2004 amid disclosures that US troops had abused detainees at Iraq's Abu Ghraib prison — an episode he later referred to as his darkest hour as defense secretary.
Mr Rumsfeld held the position until November 2006 and was replaced by Robert Gates.
After retiring in 2008 he went on to head the Rumsfeld Foundation and focused on promoting public service, spending his time working with charities that provide services and support for military families and wounded veterans.
Becoming something of a TV star during the invasion of Afghanistan, Mr Rumsfeld would often appear at televised briefings on the war and was renowned for his blunt talk and uncompromising style.
When the Bush administration's attention later shifted to Iraq, the war effort in Afghanistan began to take a back seat, opening the way for the Taliban to reassert itself and preventing the US from sealing the success of the initial invasion.
He is survived by his wife Joyce, three children and seven grandchildren.
Cover image: Donald Rumsfeld speaks to soldiers in Iraq in 2004 (Picture: US Department of Defense).