Well, there are some good things and some bad things about the area. For one, it's Harlem. Harlem isn't a dangerous neighborhood, per se, anymore. But, it definitely still has a largely lower class demographic. And, there isn't much to do in the realm of the bar and restaurant scene in Harlem aside from the Amersterdam area and what is directly surrounding Columbia University. It's also a pain in the ass to get downtown because Mt Morris is pretty far north and the further north you get, the less trains there are so you have to make a lot of changes which can add quite a bit to your commute. Also, Mt Morris is a historic district so the buildings are protected. You can't change the structure or facade at all --even if you own it.
That being said, the homes are gorgeous and you can't find brownstones anywhere else for a remotely reasonable price outside of this area. And, New York is famous for its changing neighborhoods. The Bowery was a crack den until about five years ago and now it's one of the trendiest and priciest areas downtown. This happens constantly in New York; so, if you find an amazing deal on a beautiful house, I would say, take it. It'll probably double in value once more people catch on and take over. You just have to tough out the period that the neighborhood is less than ideal. Which, honestly, won't be that long if you can get a brownstone for under $5 million.