Synthesizing results from more than one plausible assessment model removes the pressure to find “the best" model, but introduces additional complexities and decisions that must be considered. In this session, we will compare the status quo assessment process (based on selecting a single best model) to the ensemble model process, highlighting and discussing both advantages and caveats.
This network session will summarize the status quo and ensemble modeling processes, highlighting the characterization of uncertainty in both assessments and forecasts, model performance metrics, and the practicalities of reviewing the assessment and providing management advice. Participants are invited to share their assessment experience with uncertainties in their model structure, performance of their model over time, and sensitivities of management advice to model structure. Focused questions will lead participants through the important steps of the model ensemble process, and feedback about feasibility, and impacts on the review and advice aspect will be summarized.
Background
There are uncertainties associated with every phase of the stock assessment process, ranging from the collection of data, assessment model choice and assumptions, interpretation of risk, and implementation of management advice. The dynamics of the modeled fish populations are very complex, and our incomplete understanding of those dynamics (and limited observations of important mechanisms) necessitate that our models are simpler than nature. The aim is for the model to capture enough of the dynamics to accurately estimate trends and abundance, and to provide advice to managers about sustainable harvest. The status quo approach to assessment modeling has been to identify the 'best' model, based on diagnostics and model selection criteria, and to generate advice from that single model. This procedure essentially ignores advice from other model congurations regardless of how closely they performed relative to the 'best' model. The ensemble modeling paradigm proposes to more fully capture uncertainty in the assessment model selection and advice provision—but is this the case?
|
