'Mansion tax police' will be sent to homes to decide whether they should pay more under Labour plans.
Ministers have admitted that valuation agents will be able to carry out 'internal inspections' of homes to check if they are worth more than £2 million.
Anyone who refuses a visit from the HMRC inspectors would be committing a criminal offence, punishable by a fine of up to £200.
The Tories described the move as a 'sinister assault on civil liberties' after uncovering the plan, which will hit London and the South East hardest.
At last year's Budget, then chancellor Rachel Reeves announced a 'mansion tax' on homes worth more than £2 million.
Under the plans, owners of high-value properties would be subject to an extra levy of between £2,500 and £7,500 a year.
HMRC agents will use third-party data and publicly available information to value properties but will also be able to carry out 'internal inspections'.
Shadow Chancellor Mel Stride said: 'We may have changed prime minister, but it's the same old story with Labour – how can we raise taxes to pay for more benefits?
HMRC will from next year conduct 'internal inspections' of houses to check whether they are worth more than £2 million
Now they have chosen a sinister assault on civil liberties as their latest method.
'Andy Burnham and (Housing Secretary) Angela Rayner are sending their council tax police to people's homes so they can drag them into paying their new family homes tax.
'This is a tax on aspiration, on working hard and investing well, and Labour will stop at nothing to make as many people as possible liable for it.'
Dan Tomlinson, a junior Treasury minister, confirmed that homeowners faced the prospect of the Valuation Office, which is part of HMRC, contacting them to 'arrange a visit' to their home.
Earlier this year, Jonathan Russell, chief executive of the Valuation Office, told MPs that houses previously valued at more than £1.5million would be reassessed to determine whether they now exceeded the £2million threshold.
Internal investigations would check on the size and architectural style of a house as well as the number of floors, rooms, bedrooms and bathrooms, according to reports.
Ministers are said to have been advised that 'regular' inspections would be required to ensure valuation data is up to date.
A Government spokesman said visits would take place by prior agreement with the property owner.