Acoustic startle responses in fish are short latency (10 ms) evasive behaviours typically directed away from the eliciting sound stimulus. It is still unclear how fish nearly instantly determine sound source and escape directions. The phase comparison theory suggests that fish with a swim bladder distinguish between direct stimulation of the inner ear by particle motion and indirect stimulation by sound pressure via the swim bladder. The theory hypothesizes that fish interpret initial acceleration and rarefaction as movement towards and initial acceleration and compression as movement away from a sound source. By using a specially designed swing system and high speed video, we have examined the directionality of acoustic startle responses in different species of fish and predictions of the phase theory. The frequency range studied was 6.7 – 100 Hz. In the four species examined: roach (Rutilus rutilus), zebra fish (Danio rerio), two spotted goby (Gobiusculus flavescens) and three-spined stickleback (Gasterosteus aculeatus), infrasound was a potent trigger of typical C-type startle responses with acceleration thresholds of 0.02 – 0.08 ms-2. In the ostariophysin hearing specialists roach and zebra fish, startle responses were in the direction of stimulus acceleration during initial compression and oppositely directed during initial rarefaction. This was as predicted by the phase theory for sound source localization in fish. In the hearing generalists two spotted goby and three-spined stickleback, startle responses were in the direction of the initial stimulus acceleration irrespective of the sign of the sound pressure, and thus not in accordance the phase theory.