I was able to come up with two stables in town -- Rivendell Riding Academy and Love Farm. The latter seems like it's more geared towards boarding for horses, though it does have a nice jump area. Rivendell is what it says it is, a riding academy. According to their current prices, it's about $35/lesson for an adult, but they offer all kinds of variations including group rates, private lessons, children's rates, etc.
RRA is in Clinton Corners on the Salt Point Turnpike, so it should be fairly easy to reach.