Offended by someone: get over it or say something?

Last week, I went to my local library. Which I do every other Saturday. The man at the information desk said to me, "Are you really going to read all these books?" with this look on his face like a disapproving dad. If he would have taken 2 seconds to look at my history he would have seen I regularly check out 10-14 books every 2 weeks. Now I have worked in a male dominated industry (construction) for the last 13 years so I really don't get offended by man vs. women stuff as it comes with the territory but this has me puzzled to say the least. I have to talk to this guy as I love to look in the historical records area while my son is doing his thing (computer, books, activities, whatever is going on) and he is the only one who can unlock the cabinets. Do I say something? Suck it up as an offhanded comment? I don't want to make waves, but I don't mind speaking up for myself either.