Customers of Barclays, Capital One, Lloyds Banking Group and MBNA may have been underpaid payment protection insurance (PPI) compensation.
An expert instructed by the BBC said the amount credit card customers could have been shortchanged by could be "somewhere in the region of £1 billion".
The BBC said the shortfall in compensation came about because "although these banks all refunded the premiums on their mis-sold PPI policies plus interest as regulators require, they have been failing correctly to refund additional charges which were triggered by the premiums of the mis-sold PPI policies".
It further explained: "Some of those premiums put people over their borrowing limits, meaning they were then charged an additional fee".
"This failure to include fees and charges in compensation calculations has resulted in dramatic reductions to the amounts some customers have received."
The Financial Conduct Authority said it was in talks with the banks involved about the issue.
Clive Adamson, the FCA's director of supervision, told the BBC: "If there are penalty fees or charges that arise from the mis-sale of the original PPI, then they should be refundable."
Barclays told the BBC that a system it previously used to calculate compensation did not fully piece together information about all the costs a customer may have incurred. It added that affected customers have since been refunded.
Lloyds Banking Group said it investigates whenever a customer informs it that they have incurred other costs because of their PPI policy and makes "an appropriate refund".
In March, the BBC reported that Lloyds Banking Group had been cutting the compensation to PPI claimants to save £60 million after a whistleblower came forward.
This, the broadcaster alleged, was because the bank had been deducting the cost of a cheaper, regular premium policy from the compensation package on the premise the customer could have bought one of them instead.
At the time, the banking group said that only a tiny fraction of the claims it handled were affected.