Based on textual evidence in "The Fall of the House of Usher," what can you infer was wrong with the Ushers?

In truth, I find that very little can be inferred about what is wrong with the inhabitants of the House of Usher. There is a mild suggestion of opium use but the narrator never follows up on the suggestion, so that is ruled out as a cause of their malady. There are suggestions of what we would call clinical depression, but clinical depression is characterized as having no known cause or as continuing fro an extended periods of time past the end of a known cause, so that is probably ruled out.

There is a suggestion of catatonic schizophrenia but other characterizing symptoms are not noted, so that is ruled out. There is also strong suggestion of general hereditary madness but if this were the case, Madeline would not come through the door and the house would not have crumbled, so this too can logically be ruled out.

Two things are of significant note in trying to find the answer to what is wrong with the house of Usher. The first is that no daughters or second sons of Usher lived or, if they lived, married and had offspring. One of the first things the narrator says is that the family of Usher "put forth, at no period, any enduring branch" of the family. Familial branches develop when a daughter marries and has children or when a son other than the eldest son marries and has children.

The second is that Usher himself states a cause for the family malady. He states that what is wrong with the house of Usher is the arrangement of the stones of the house and the collocation of the vegetation and house. Usher unblushingly believes in the sentience of the vegetation.

"in the long undisturbed endurance of this arrangement, and in its reduplication in the still waters of the tarn. Its evidence—the evidence of the sentience—was to be seen, he said,..."

As the narrator says, this is not overly peculiar as many other people believed the same. However Usher distinguishes the thought by ascribing an maliciousness to the sentience of the vegetation.

Reasoning from Usher's pronouncements as the reader sketchily gleans them from the narrator, the vegetation appears to deem the land and stones of Usher as their rightful property and, after growing strong enough, began to sap the moral life out of the particularly susceptible family of Usher rendering them in the condition in which the narrator finds Madeline and Usher. The vegetation's final victory is the return of the stones of Usher to land, which is the stones' rightful inheritance.