Differences in interaural phase configuration between a target and a masker can lead to substantial binaural unmasking. This binaural benefit decreases with increasing interaural time difference (ITD) of the masking noise. Combining two noises with opposite ITDs (double-delayed noise) further reduces the binaural benefit. Thus far, modeling of these detection thresholds required both a mechanism for internal ITD compensation and a larger binaural than monaural bandwidth. However, neither has yet been demonstrated in mammals. We propose an alternative model which evaluates the complex-valued correlation coefficient in each frequency band, i.e. only one complex number at lag zero, instead of the established correlation functions. The reduced binaural benefit in double-delayed noise is modeled by an incoherence interference across frequency channels. This mechanism differs from wider filters since it has no effect when the masker coherence is constant across frequency channels, e.g., with only a single masking noise. The model precisely accounts for binaural unmasking for a wide range of conditions. It helps resolving the inconsistency that previously simulation of some data required wide filters while others required narrow filters. As it does not require internal delay compensation, it appears to be well compatible with the physiology of the mammalian binaural system.