At work we use virtualised servers extensively. Our whole shared hosting/VPS platform is built on Virtuozzo. We have numerous other services which are virtualised in the the background using other technologies such as Xen, KVM, Hyper-V etc. It is a brilliant tool when deployed properly and has plenty of other benefits such as being able to move an virtual server to new hardware in a hurry.
However, if you are to believe the marketing hype, virtualisation will immediately save you X% where X is ridiculously large number like 70 or 80. What they always seem to fail to mention is that they're presuming that you're massively under utilising your current hardware.
This leads to a lovely self fulfilling prophecy. The people who move over are the ones underutilising their current hardware and they will see massive savings. These savings are due to bad planning and over speccing the hardware in the first place though, and virtualisation is the ideal technology to consolidate the hardware while keeping the outward facing infrastructure looking the same. This means there's a massive selection bias in the figures which virtualisation vendors quote, as they seem to only use these customers as examples.
If we then look to the other end of the spectrum, people properly utilising their existing infrastructure. Here virtualisation will still give plenty of benefits. For example, being able to move a virtualised server from physical server to physical server, often with no downtime. However, then you have to consider virtualisation overhead. As virtualisation is simple abstracting away the hardware, there is going to be an overhead in the translation. Depending on the technology used the overhead might be minimal or it might be large enough that new hardware is required to account for it.
There will also be no savings due to less hardware in this scenario as the virtualisation isn't being used for consolidation, but for ease of management. If it's a commercial virtualisation product such as VMWare, there's going to be extra cost involved. This cost might be offset in deceased administration time, but it's not going to be anything near the figures normally quoted for savings.
To go back to what started all this off, the 200 servers in the HSE. We have no way of knowing what the utilisation is like on these servers. For all we know, it's a fairly heavy Java based app running on them and the systems are well utilised. It's also possible that they are underutilised, but without knowing what they're actually doing, it's not possible to pull random figures like 75% out of the air.