The oft-scrutinized Comparative Survival Study team's methods and aims received mostly positive reviews in a scientific report released last week by the Northwest Power and Conservation Council.
The joint review by two NPCC science advisory panels focused on the 264-page (675 pages when including appendices) "Comparative Survival Study (CSS) of PIT-Tagged Spring/Summer Chinook and Steelhead In the Columbia River Basin's: Ten-year Retrospective Analyses Report."
The full ISAB/ISRP document can be found at http://www.nwcouncil.org/library/isab/isabisrp2007-6.htm
The retrospective "utilizes a holistic approach to describe study methods and assess the data available from ten years of juvenile salmon and steelhead PIT-tagging and PIT tag detections" for the period 1996 through 2005, according to its introduction.
"The Ten Year Summary Report aggregates the available data within years regarding environmental variability and migration timing for the assessment of juvenile reach survivals and Smolt to Adult Return (SAR). The data are additionally aggregated across years to assess trends and variability in SARs. These new analyses provide an additional perspective for the CSS results that increases the utility of the CSS data for making management decisions."
The retrospective was completed at the end of May and offered for public review through June 29.
Detailed comments, including numerous that were critical of CSS methods and interpretation of its analysis, were offered by the Bonneville Power Administration, U.S. Fish and Wildlife Service, the NOAA Fisheries Service and its Northwest Fisheries Science Center, the U.S. Army Corps of Engineers and University of Washington professor James Anderson.
The final retrospective report released Aug. 31 includes a 145-page appendix that lists the comments and responses to those comments by the Comparative Survival Study Oversight Committee and the Fish Passage Center. The FPC staff and representatives of the USFWS, Idaho, Oregon and Washington fish management agencies, and Ecologic, compiled the report. The final can be found at http://www.fpc.org/documents/CSS/FINAL%20COMPLETE%2010%20YEAR%20CSS%20REPORT-8-31-07withfrontpage.pdf
The Independent Scientific Review Panel and the Independent Scientific Advisory Board, in their review released last week, "only briefly reviewed the comments from others on the CSS report in the context of our chapter reviews. We do not address the comments point by point, and we did not re-analyze any of the CSS or commenters' specific data analyses due to the short time available for our review."
The document was produced at the request of the NPCC at the end of the fiscal year 2007-2009 project selection process for its Columbia River Fish and Wildlife Program. The Council recommended $790,000 in 2007 funding for the CSS project, and up to $125,000 for the compilation of the summary report.
The Council did not earmark any funding for the project in 2008-2009, saying it would forestall those decisions until the summary report was completed and had been reviewed by the ISAB and ISRP. The contract for the work was extended into fiscal year 2008, from Oct. 1 through February, according to Tony Grover, director of the NPCC's fish and wildlife program, which is funded by the BPA.
A discussion of the CSS funding issue is scheduled for the Council's Dec. 11-12 meeting in Portland, he said.
The 27-page ISAB/ISRP review dated Nov. 19 found the 10-year report "clear, thorough, responsive to past ISAB comments, and was completed in a retrospective style, a major accomplishment for which we commend the CSS investigators."
The ISRP, which reviews project proposals for technical merit, said that three of the project's four biological objectives meet the Council program's scientific review criteria.
A fourth objective in the 2007-2009 CSS proposal, a planned comparison of SARs of upriver chinook that were transported through the federal Columbia/Snake river hydro system with downriver indicator stocks, did not, "because of inevitable confounding from other factors in establishing cause(s) of upriver/downriver differences that may be detected, regardless of sample size and detection power that could be achieved," the ISAB/ISRP review says.
The Fish Passage Center's Michele DeHart said the retrospective authors are reviewing the ISAB/ISRP report and would likely seek follow-up discussions with the science panels on some issues.
"We want to talk the ISAB about what that means," DeHart said of the science panels' conclusions on the upriver/downriver proposal. She predicted a positive scientific end overall.
"They've been very helpful to us in the past," DeHart said. "This project's probably been reviewed more than any other project."
"Overall, the CSS Ten-Year Retrospective was effective in answering the concerns posed by the ISAB's review of the CSS 2005 Annual Report (ISAB 2006-3)" and by the Council, the ISAB/ISRP review says. "The Retrospective provided improved clarity in the presentation and explanation of the sophisticated methodologies used in analyses of CSS data.
"The scope of CSS investigations resulted in an extensive report, containing many detailed summaries of past and present work, and the report presents key data and data summaries in support of their major conclusions. The CSS team has responded very well in a short time frame to the difficult challenge of including enough details to allow scientific review, while avoiding obfuscation by sheer volume of material," according to the review's executive summary.
The Council had asked earlier whether the design, implementation, and interpretation of the CSS statistical analyses were based on the best available methods and what weight the analyses, "taking into account whatever scientific criticisms of the analyses that the ISAB decides are valid, if any," should be given by policy makers.
The methods are "very good," according to the review. "The CSS constitutes a successful implementation of a large-scale tagging program" that benefits from the incorporation of PIT-tag information from other tagging programs. The ISAB/ISRP recommended that increased tag data sharing would be helpful in addressing critical scientific uncertainties and improve reliability of survival estimates. It also gave detailed advice on how the study design and analysis might be improved.
The review said the CSS results are "based on carefully considered and applied methods."
"It is inevitable that there are other methods of analysis and that there can be other interpretations of results. We support the CSS efforts to refine analytical methodology, analyze other data, and design additional studies to collect more data to answer important questions for the region," the review says.
"Caution is always needed in interpreting results, and the assumptions that are used in interpretation, as well as measures of uncertainty, must be taken into account in deciding the application of any interpretation.
"For instance, current conclusions that transportation provided, or did not provide, benefit to a species or wild/hatchery group requires qualification with the possibility of selection bias of fish for transportation due to size, condition, location in the water column, etc.
"Similarly, conclusions about mortality or delayed mortality of transported fish, relative to in-river fish, are not equivalent to saying that mortality or delayed mortality are DUE to transportation, unless all other factors can be discounted. Similarly, statements that trends are consistent with a specific hypothesis are most useful where alternative explanations are examined and discounted," according to the ISAB/ISRP review.
"We also find many well-supported interpretations in the CSS Retrospective that should be carefully considered by Council and other decision-makers," the science panels said.
The CSS is a field study of the survival of PIT-tagged spring/summer chinook and PIT-tagged summer steelhead through the Snake/Columbia river hydro system from smolts through returning adults, with a focus on relative survival of fish that traveled as smolts by alternative routes (e.g., in river, transported, different routes of dam passage, and different numbers of dams passed).
"The CSS is important because it is one of the few organized attempts to systematically release PIT-tagged hatchery-reared and wild smolts into the Columbia River for the purpose of comparative monitoring and evaluation," the review says. "Most aspects of the study, from its design and methods to the analytical results, continue to be strongly debated in the Region because the relative survival rates of salmonids under different hydrosystem operations and environmental constraints is a central concern of water and fish management policies."
The science panels in the most recent CSS review admit to changing their mind about the upriver/downriver comparison, an attempt to determine how the juvenile fishes' hydro system passage experience might affect their ability to survive in the ocean and return as adults. In past reviews they have suggested that more downriver stocks be tagged to improve the statistical validity of that prong of the CSS study.
"We understand this is a critical change in our recommendation and reach this conclusion after significant internal discussion based on findings from our latent mortality report and evaluation of the CSS Retrospective Report," the new review says of recommendation to drop the upriver/downriver comparison.
"The core reason a contrast of salmon survival between upriver and downriver locations is not advised is that the populations in tributaries downriver of the dams are not replicates of the upper Snake River populations," the review says. "Moreover, there is no parallel river system, so the challenge is more difficult than simply finding populations that could serve as replicates. There is inevitable confounding of all differences between downriver and upriver stocks and their environments, precluding clear attribution of cause for any upriver/downriver differences that might be shown.
"Finally, the absence of a clear measure indicating the cause for differences in upriver-downriver SARs does not mean that a real effect is absent.
"The ISAB's Latent Mortality Report (ISAB 2007-14) expresses in detail the concerns about interpretation of cause-effect from the upriver-downriver statistical comparison and notes also the lack of a needed appropriate baseline, the damless reference, in the context of attempts to measure latent mortality."
"The sponsors have presented evidence suggestive of a hydrosystem effect on differences in SARs between upriver and downriver sites, but little may be gained from further analysis of differences in SARs. The major conclusions of the research are already available for scrutiny by scientists and managers in peer-reviewed scientific literature and reports including the retrospective summary."