In Narrative of the Life of Frederick Douglass, who are the protagonists and the antagonists?

In the novel, Frederick Douglass is of course the protagonist.  An argument that Mr. Covey is his primary antagonist can be made.  While all the slaveholders and the overseers are represented as corrupt, cruel, and evil, Douglass manages to deal with all of them in his own way.  Covey is another matter, however.  Douglass has not lead a charmed life by any stretch of the imagination, but it is when he is sent to stay with Mr. Covey that he truly fears for his life.

Covey is a notorious "slave-breaker."  Especially "troublesome" slaves are sent to him for a year, and within that year he does everything he can to break the slaves' spirits.  He is referred to as a snake by the slaves because of his hiding in the grass in order to catch the slaves taking a break from their labor.  He puts Douglass in charge of some oxen that run wild, almost killing Douglass in the process.

Finally, Douglass has enough.  Covey takes him to the barn, intending on beating him.  Douglass fights back, however, with all his will.  The fight lasts for two hours.  After the fight, both men are exhausted.  Covey leaves Douglass alone after the fight.  Douglass emerges the victor; his spirit was not broken.