They do not have a statutary right of entry into your home. In theory they can get a search warrant, but they have to have a very good reason to get one.
It works like this - their computer in Bristol has the addresses of every home in Britain. It also has a list of every home with a registered TV licence. Because they assume that everyone has a TV set, they send out letters to those addresses that aren't registered as having a licence.
Even registering that you don't have a TV set either online, by phone or by post will not stop them from sending you threatening letters. The way they look at it is that they assume you are guilty and you have to prove to them that you don't have a TV set on the premises before they'll leave you alone.
Even if you do manage to prove to them that you don't have a TV, they'll likely give you a grace period of three years before they start hassling you again.
I won't even tell them my name and I most certainly will not let them in. It is not my place to do their job for them and certainly not to prove my innocence.
I know all of this because I used to work for the Post Office and the PO was an agent for the licencing company. Incidentally, despite what you might think, the money from the licences does not all go to the BBC. It goes to the Home Office who then pass on a portion to the BBC.