How are you determining fertility? Do you candle after a diapause or use some other method? Its a great question, and I can only seek a better understanding of the problem. Maybe your colony set up is great and it's an incubation problem, or a single male has low motility?
Lets say you have looked into all these issue and the male is indeed not fertile enough, then the answer is simple, get another male.
If the incubation method is your suspect, then try different methods.
As all the above has been said. Is your female producing eggs?
How many males are there?
What medium are you incubating them on
What temp are you cooling them down to and for how long
What temp are you incubating for.
Without any of these questions answered, no help here