When ever I grow tort-edible flowers. I feed both leaves and flowers. I think you could miss out on some good nutrition if you don't.
I haven't tried feeding Geraniums yet, but I would offer leaves too. Fun side note...My tortoise likes pansie/viola leaves just as much as she likes the flowers