He's in the prime of his life. I don't know about the fertility of male tortoises, but female tortoises can lay viable eggs clear up to the day she dies...even if that's 100 years old.
Yeah, their life spans are similar to humans, so if they develop at a normal rate, they can reproduce early, but at 20 he is an eligible bachelor with decades of breeding to go. I don't know how old my two main breeding females are, but I am fairly certain that one is at least in her 20's, and the other probably in her teens. The male is 4, and he is breeding, but I expect him to become more fertile over the coming years.