Two problems likely to arise in the transition from a rare new hybrid genotype to a full hybrid species:

1. Finding a mate.

When a fertile polyploid hybrid first arises, it is one hybrid (or perhaps one of a small number) within two large populations of the parental species. The hybrid can only mate with other hybrids like itself and so natural selection on the hybrid therefore has a kind of positive frequency dependence: when it is rare its fitness is lower because of the difficulty of finding a mate.

This helps explain why hybrid speciation has been much commoner in some groups of plants than others. It is much easier for a new hybrid to cross the difficult transition stage, in which it is rare, if it has alternative reproductive options besides sexual cross-fertilization. It has been shown that hybrid speciation is commoner in groups in which asexual reproduction or self-fertilization are possible.

2. Ecological competition.

Two species with identical ecological needs will usually not be able to coexist; if one of the species can survive on a lower level of resources, it will drive the other extinct. When the hybrid arises, it will be in the same place as the parental species, and it is likely to have ecological needs that overlap with them. It is therefore thought that the hybrid species that have become established are those that happened to be adapted to different ecological niches from the parental species.

Those that were ecologically too similar to the parental species would have been lost. The survival of Iris nelsonii (pictured opposite in its swamp land habitat) may have been helped by the way it lives in habitats in between the other three species.