I've had a career in the pet trade dealing with tortoises and other animals since 1986. Been keeping them since1979. Much of my family and friends are medical professionals. And I'm a father. Given all of that, I have to say the risk is minimal and possibly nonexistent.
I'm just some guy on the internet, so if you are concerned, I would ask your doctor.
16 years ago, during the pregnancy for our first child, my wife had the same question. She is a Doctor and has a number of Doctor friends - she asked a neurologist, a GP and her OB - they all admitted they had no experience with tortoise specific concerns, but hypothesized that if there was a known issue that it would be public knowledge. I am also just some guy on the internet, so ask your doctor.
The answer is that we don't know. I can tell you that the odds are very low. But your health and your baby's health are paramount. Is there someone who might be able to help you?
Children who are subjected to the "dirt" of asnimals and the outdoors, usually have less allergies.
When cleaning up after your tort, wear a pair of gloves, the plastic, surgeony kind.
And I'm just a woman on the Internet with no kids.
Throughout my pregnancy I did everything that I normally do for my tortoise. I now have a 10 month old healthy daughter. I think it just comes down to how big you think the risk is. Personally I thought it was extremely low hence why I just carried on doing what I was doing. Congratulations by the way.