What happens if I don't swipe my Oyster card on leaving the tube?

I have a month's travelcard and just a little bit of top-up. Sometimes the barriers are left open at my station and I'm always tempted not to swipe my card as Oyster stores my travel patterns and I'm dead against this. Would I get charged for this?