Caving or cave diving? For caving, there aren't many lights that work especially well as dive lights and vice versa.
For cave diving you want a fairly long burn time, nothing less than two hours, preferably more. My own preference is for halogen, at 80p per 50W bulb as opposed to £80 for a 18W HID speaks for itself. Caves are harsh environments. I'm not a fan of LED for primary lighting, I find the light too diffuse and it doesn't penetrate as far. In cave diving you need to be able to focus the beam for different conditions, you can't do this with a LED. That said, all my back-ups are now LED.
My own preference is for AUL
www.dpvrepair.com I have used these for ten years without any trouble. I also like Sartek too but expensive.
I've had a lot of cheap lights and you get what you pay for. Lights I didn't like: Custom Divers (poorly fitting lids, dodgy reed switches and recently had one go boom), JMD (leaked in the shallows but fine deep), Halcyon (variable build quality), Niterider (some models have very unreliable switches in deep water)
Avoid anything that has a reed switch. They are a pain, they are fragile and unreliable. Fitting a new one is a nightmare. Similarly anything where you twist the light head. Again the reed switches are fragile and depending on the design you will kink the wires in the umbilical which will eventually fail. Waterproof toggle switches are best as on AUL & Sartek.
The battery should be accessible. Some cannisters are sealed and only have a charging port. When the battery eventually dies it makes changing it a problem. Carrying a spare battery pack is often easier than recharging on a trip. I check the voltage on mine often too so you need access to do it. Screw fitting lids can jam up if you take them deep (Dive Rite), they are also easily cross threaded (Scubapro's Speleolight).
SLA batteries are cheap (about £25 to make up a 14Ah pack) but big and heavy. They are however reliable and simple. NiMH is more expensive (about £90-100 for a pack to fit my Spectrum 14) but give very long burn times. They can be temperamental and don't give any warning that the battery is low on power -- they just switch off.
Cheers,
Stuart