TIM CARTER

A.1.3.1 Introduction 

Any source of information will have some limitations. The better the design of any investigation, the better understood these will be. As noted, most studies to date have made use of information collected for other purposes. This is likely to mean that aspects such as the standardisation of information on patterns of exposure, and on the determination of adverse effects will be less than perfect.

A.1.3.2 Key tests of validity

There are several facets of validity. Any information or analysis needs to be assessed in terms of these.

Is the data from a population that is representative of the one you propose to assess risk or to specify interventions on?

The ideal is to have information about a similar population of seafarers to the one you are concerned with. If the results come from an ethnically different population they may reflect patterns of ill health from the home country of the study population rather than those for your target group. If they do not include those who have left seafaring they will not provide a full picture of long-term disease. A study based on a single clinic will simply reflect those who attend, it will not take account of any selection factors that determine who is seen there.

Is the data from a large enough sample to exclude effects that may come from chance variations among small groups?

This is termed the ‘power’ of a study and is an important aspect of good study design.

Is the population used to determine the prevalence or the incidence of a condition clearly defined, with details given?

Risk can be under or overstated if the population is not defined. Age effects may predominate if the two populations used for any comparison are not age matched.

Are well-defined diagnostic criteria used to identify the cases that are used to determine prevalence or incidence rates?

If several people decide on these they are likely to differ from one another. Often when available data is used as the basis for studies there will be little or no control over the diagnostic labels attached to each case of ill-health.

If the study relies on the results from clinical, laboratory or psychological tests, are the tests used valid and relevant to the risk you are assessing?

There are many instances of invalidated or poorly quality controlled tests being used to take decisions on risk or to decide on likelihood of future illness. These are discussed in chapter 4.8.11.