I can't think of any museums in town to be quite honest. Your best chance of finding one would be in the village of Wappingers Falls, but while some buildings are old and steeped in history, they're not museums. Some consider the Grinnell Library to be a landmark because it was built in the 19th Century (6th oldest library in the entire state). A few also say the same of Mesier Park, right next door to Grinnell -- that plot of land dates back to the 18th Century.
There is a Wappingers Historical Society, for those interested in learning more and taking an active part in preserving the town's history.