Dosen't surprise me.
I waited until I got married and had a house and land payment before I decided to look for insurance (when I was by myself most of my life, I didn't feel like I had to leave money to anyone).
Being in your late 30's when you first look for an insurance policy one and, well, lets say not being perfect in my youth, holy **** batman! Doesn't matter how great of shape your in. I almost fell over dead on a whole life policy.
Had to settle for a 20 year term policy, but it didn't kill me on the premium (a lot less than what you mentioned now) and it's enough to ensure that the wife is taken care of per long term bills (house and land).
Luckily my parents got me a couple of whole life policies when I was younger, not much per premium now, and a little extra cash for the family if something happens to me. That was one of the first things I did for my son, get him a whole life policy. When he gets older, we can still control it, but he'll have something that will accumulate and the death benefit is nothing to sneeze at.
I'd suggest looking around. Something in your medical history has to be jacking that premium up isn't it?