A quarter of homeowners were hit with a Stamp Duty bill of at least £7,500 when they bought property in the last tax year (2012/13).
In 2012/13, more than £4 billion was raised for the Treasury through residential Stamp Duty, of which £3.6 billion was paid at a rate of 3% or more - the rate levied on properties valued at more than £250,000.
The 3% burden falls heaviest on London, where property prices are highest and 60,329 homes were bought with a Stamp Duty rate of 3% or more in 2012/13. This was 65% of all of the transactions in the region.
Properties valued between £125,000 and £250,000 also incur Stamp Duty but do so at the lower rate of 1%.
Whatever the rate, Stamp Duty is stopping some people from getting on the property ladder in the first place as well as preventing others from moving when they need to, according to the Taxpayers' Alliance (TPA), which compiled the figures.
"The result is that people stay put when it would make sense for them to move for a variety of reasons, such as getting a new job; being closer to relatives; or having enough space for a growing family," said the TPA.
The organisation is campaigning to 'Stamp Out Stamp Duty' altogether but it particularly wants to see Chancellor George Osborne and Prime Minister David Cameron uphold a promise they made in 2007 to abolish Stamp Duty for almost all first-time buyers.
While Kate Faulkner, property expert and managing director of Designs on Property, agrees that first-timers should be rewarded for saving to get on the ladder, she's against axing Stamp Duty outright.
"We have to secure tax from somewhere and as people can buy as many properties as they like and not pay any tax on the gain, I think paying stamp duty is a fair way to tax property owners.
"Most people don't pay it, or pay just 1%, which is perfectly reasonable. Less than 10% of people pay over 3% in most areas," she said.
Indeed, the average house price across eight of the UK's 10 major cities are below the 1% Stamp Duty threshold, she added. These include Bradford, Leeds, Liverpool, Manchester, Nottingham, Peterborough and Sheffield.
However, Faulkner would like to see the £250,000 point at which the 3% tax kicks in increased. "This is the price for a three-bedroom family home across most decent areas of the UK and it's a big jump from 1%," she said. "The rest is, in my view, perfectly fair."