drivers side is your left side "short" axle?.. there is a circlip on the end that needs to be "clipped" in. If you can easily pull the shaft out means its not clipped in, give it a tap with a rubber mallet or heavy hammer with a piece of timber to protect the shaft.

Is the axle a replacement ? Some had the wrong clip size. Try pulling on the inner CV joint cup just before that shaft, see if it comes out, otherwise it should be secured properly. For the pull, how is the rear bushing on the control arm, when you jack the wheel ?

It's the small bushing at the rear of the control arm that goes bad (tears). For the axle, if it doesn't come out it's secure, but good to know it may not come out again if needed, if the clip was slightly larger (common problem)