Division is defined as the inverse of multiplication. The answer to one divided by zero is the same as asking which number you would multiply by zero in order to get one. No number has that property, not even infinity. So the answer is undefined.

One divided by 'epsilon', where epsilon represents a very tiny number, approaches infinity for ever tinier epsilons, so in some maths contexts infinity makes sense. But in general it's a meaningless question, and so can only have a meaningless answer.