The story starts with the question, "Am I mad?" The question re-appears throughout the work. In the story the main character murders the man he lives with, for what appears to be no reason at all.
The inference seems to be that we have to believe he's insane. Evil for evil's sake is so alien to our reasoning that maybe we have to believe he's insane, maybe even for our own sanity.
Even so, he seems to indicate his own insanity by saying: "The disease has sharpened my senses." In either case he kills him, and buries him in the floorboards. The police come, he hears the sound of what he thinks is the old man's beating heart, and confesses to the police in the hopes that the awful noise will stop.
It's fairly clear that the heart doesn't belong to the old man in all reality: him being dead, and all that jazz. It would seem that the heart is truly his own: giving guilt a sort of phenomena in nature. Because, only he can hear it, and the haunting beating noise starts right after he kills the old man.
The point is on human nature, and evil. The point on nature is that the human heart was made for good. It starts to malfunction, and rebel against its possessor's will when it has done evil. The point on evil is that, if you boil evil down to what it is it's madness, nothing more.
There's one last question: How many steps are we away from this. When will the reasons for evil simply dissolve, and when will evil get back to its true nature? When will it become evil for evil's sake?