Is there a trick or method I'm not aware of, to remove the plastic cover from behind driver side door handle, without breaking it? I've had it partially off, but the damn little thing is stubborn as hell. (the handle you pull on to close door) XC90

Attempted to remove plastic cover around interior driver side door handle, but am afraid of breaking it. I managed to pry it partly off, but the back seems to be very well atached. Please advise.