Outgroup comparison

The method of outgroup comparison means looking at a closely related species which is known to be phylogenetically outside the group of species we are studying. The character state in that outgroup is likely to have been ancestral in the group under consideration.

In the abstract, there could be two species, species 1 and 3, sharing homology a and two others, species 2 and 4, with homology a' . We wish to know whether character a evolved into a' , or a' into a . We look at a closely related species and infer that the state there is ancestral in the group of four. If the outgroup had a we should infer that species 2 and 4 share a more recent common ancestor with each other than with any of the other species.

Outgroup comparison works on the assumption that evolution is parsimonious. In the figure, if the character in the outgroup a is ancestral in the group of species 1 - 4, there must have been at least one evolutionary event in the phylogeny: a transition from a to a' before the ancestor of species 2 and 4. If, having observed a in the outgroup, we had reasoned that a' was the ancestral state of species 1 - 4, we should need at least two events: a change from a' to a somewhere between the outgroup and species 1 - 4, and then a change from a' back to a in species 1 and 3. If the character state in the outgroup is ancestral, the fewest evolutionary events are required.

Outgroup comparison is fallible.

Sometimes, one possible outgroup will suggest that one character state is ancestral, but another possible outgroup will suggest that a different character state is ancestral. The result will then depend on which outgroup we rely on. The method is most reliable when the closely related species that could be used as outgroups all suggest the same inference; but it is possible to be led astray by the method in particular cases. The inference should be treated with caution, and if possible tested against other evidence.

Before we can use outgroup comparison, we need to know something about the phylogeny. We need to know that fish and amphibians were outside the Amniota in order to use them as 'outgroups'. In practice this is not a major problem. Outgroup comparison cannot be used when we are absolutely ignorant, but if we know something about the phylogeny of a group (for example that amphibians are not amniotes, but are closely related to them) we can build on that knowledge to find out more (in this case, more about the phylogeny within the amniotes).

Figure: in (a) species 1 - 4 have the character states as given. We wish to know if a or a' was the state in their common ancestor. In (b) we look at a closely related species, the outgroup. It has a state a, and we infer that was the state in the ancestor of species 1 - 4. The dotted lines indicate that branching relationships remain uncertain.