et al. 1986; Underwood 1990, 1993). As Underwood (1990) pointed out, studies lacking replication cannot be logically interpreted. In the marine reserve context there are many reasons why researchers might have limits on their
sampling designs. However, a critical evaluation of the experimental designs employed by many published studies brought to light the following problems with replication and lack of control sites:
(1) insufficient sample replication (for example only one site sampled inside and outside a reserve, or no control sites sampled at all);
(2) spatial confounding (for example all control sites located only at one end of the reserve, so that comparisons are confounded by unknown location effects);
(3) lack of temporal replication (most studies consist of surveys done at only one time);
(4) lack of replication at the reserve level limiting the generality of results (although in many cases this reflects the number of reserves available); and
(5) non-random placement of reserves, i.e. often reserves are sited to include ‘special’ or unique features, which causes difficulties in selecting valid control sites (this is obviously no fault of the researchers).
To date, there are no well-designed studies that avoid the above problems as well as possessing a time series of ‘before’ and ‘after’ data.
How many studies unambiguously demonstrate significant within-reserve increases in the density of exploited species? Edgar and Barrett (1997) recognized that, with a sufficiently large sample size, a statistically significant difference between two sites (separated either spatially or temporally) can almost always be obtained due simply to true natural biological variability between the sites. That is, the null hypothesis of no difference between two biological entities is necessarily false. They therefore proposed a 100% increase in density as a minimum criterion for claiming the existence of a ‘reserve effect’. This type of approach is more generally known as bio-equivalence testing, in which an effect is not considered biologically significant unless it exceeds a pre-specified threshold (McBride 1999). If we use the 100% threshold, and ignore flaws in sampling design, then there were only a handful of instances where differences in density of individual species between reserve and fished areas can be regarded as biologically significant (Polunin & Roberts 1993; Francour 1994; Harmelin
et al. 1995; Russ & Alcala 1996; Edgar & Barrett 1997, 1999; Willis et al. 2003). In many other cases, slight trends towards higher reserve densities were described, but these were of insufficient magnitude to confidently attribute them to reserve
effects, rather than real biological variability at the spatial or temporal level (Roberts & Polunin 1992; Chapman & Kramer 1999; Paddack & Estes 2000). If we consider only those studies that are replicated in both time and space, to our knowledge there are only a few that establish increases in excess of 100%.
Also it is one thing demonstrating a benifit inside the reserve - another to show fishery wide benfits:
Several theoretical studies have indicated that marine reserves can provide increases or equivalence in yield under the assumed model and parameter values (Polacheck 1990; DeMartini 1993; Attwood & Bennett 1995; Sladek Nowlis & Roberts 1999). However, if management decisions are based upon models built on unquestioned assumptions then we may find ourselves making costly errors. We reinforce this point by noting that the model of Parrish (1999) produces a contrary result; it suggests that the large reserves that are believed to be required to contribute to the Californian groundfish fishery might actually be to the detriment of the fishery, due to the displacement of fishing effort onto the remaining fishing grounds. In contrast, Horwood et al. (1998) conclude that reserves will have little effect on fishery yield. Yet, the model of Hastings and Botsford (1999) concludes that, even with arbitrarily high fishing effort outside of large reserves, marine reserves will return fisheries yields equivalent to traditional fisheries management for a wide variety of groundfish. Taken together, the conflicting conclusions from various plausible models lead us back to the beginning, where we must admit that, at present, we cannot predict what the effects of marine reserves might be.