very easy its just a split pin, which although tight will tap out, i have just replaced my handle with an alloy one. I used an old screw driver which i lightly tapped with a hammer to get it started then was able to pull it out with some vice grips. Tapped back in then finally squeezed with the vice grips again to fit flush.
Use a piece of timber to support the handbrake lever if you're replacing the handle in situ.
Tap the split pin out with a punch and the rubber handle just pulls off.
i've made some real nice ones out of Oak if you are interested. They are varnished with a glass-like finish. The pin is replaced with a Stainless dome head Allen bolt and they have to be bonded on with Araldite or similar to keep the strength. Had one on my bus for years and never seen another like it.
cheers
dave
Did the job today nice and easy. The piece of wood idea was great cheers. Any photos of your oak handles. Don't know if it would suit my bus but worth a look