One thing to remember about the mining issue is that coal mining is just as bad, and coal is often radioactive as well. More people have died from radiation poisoning due to coal power/mining than have died from radiation poisoning due to nuclear power, even when you include disasters like Three Mile Island and Chernobyl.

Of course, we've also been mining and using coal a lot longer, but the radioactive coal dust and possibly radioactive particles in the smoke from coal plants is something that many people are unaware of.

But, like you said, the big thing is to move away from fossil fuels entirely, and nuclear power has its own issues. It doesn't so much matter what we go with so long as we do actually go with something, and renewables are getting better and better all the time.