STEP 1 fitting a variety of models to a selection of actual assessment data sets, some models may not even be optimised (i.e. applied with standard settings only for features such as shrinkage, rather than fine-tuned in the spirit of a real stock assessment).
STEP 2 demands more rigour in demonstrating that the model provides a reasonable fit to the data, standard statistical diagnostic tests will be reviewed.
STEP 3 introduces simulation data sets with observation error. These approaches can be applied at two levels: within models and across models. The former explores the estimation properties of the model and the latter explores model robustness.
STEP 4 again uses simulated data sets but this time incorporates both observation and process error in the simulation of pseudo data sets.
STEP 5 aims, aided by actual assessment data sets, to investigate "grand questions" e.g. utility of ageing data, dome selectivity, changes in selectivity over time, importance of contrast in index time series, retrospective patterns, etc. Data sets may be "tweaked" to create more contrast or challenge models.
The rationale and more detail can be found in section 3.3 of the 2012 ICES Working group for stock assessment methods.
