Reports suggest the last of British Soldiers will leave the region within the next few days.
Speaking on Radio 4's Today Programme he called the decision a "'major strategic mistake".
Mr Tugendhat, who is currently the chair of the Foreign Affairs Committee, served as part of the Territorial Army and was based in Afghanistan for four years.
He said: "What we are demonstrating very publicly, very clearly, to many different adversaries and, indeed sadly, to allies, is that the US and her allies won't stay.
"If you don't have the ability to persist, you can forget about influencing others, nobody will care what you think if you aren't there tomorrow."
Speaking about the numbers of troops left in Afghanistan at the time the decision to withdraw was made, he said there were 10,000 NATO troops, 2,500 of them were Americans and fewer than 1,000 British.
"These are proportionately very small numbers," he said.
"We have more soldiers than that on operation in places like Cyprus. The Americans have more soldiers than that guarding their Capitol, in their own country.
"These are very, very small numbers of soldiers and if the decision is that you can't even endure that, then you can forget about influencing people over different parts of the world where the challenge may be greater.
"What you are doing by withdrawing is encouraging enemies and dissuading allies.
"That is dangerous," he added.
On a final note, the Conservative MP also said that he believes the British Army had "been stripped down too far to maintain these kinds of operations alongside the others that it is doing".
He added that this means the UK has "decided to withdraw from Global Britain and decided not to operate in such a way".
Reports are suggesting that more than 200 Black Watch soldiers will fly home over the next couple of days, ending the UK's 20-year deployment in Afghanistan.
This will coincide with the American Forces withdrawal, brought forward from the original 11 September date to 4 July to mark Independence Day in the US.
Cover image: MOD.