Data Analysis

Once the data are collected they must be analysed and interpreted. The steps to be followed in preparing the data for analysis and interpretation differ, depending on the type of data. The interpretation of qualitative data may in some cases be limited to descriptive narratives, but other qualitative data may lend themselves to systematic analyses through the use of quantitative approaches such as thematic coding or content analysis. Analysis includes several steps:

Check the raw data and prepare data for analysis
Conduct initial analysis based on the evaluation plan
Conduct additional analyses based on the initial results
Integrate and synthesize findings.

The first step in quantitative data analysis is the checking of data for responses that may be out of line or unlikely. Such instances include: selecting more than one answer when only one can be selected; always choosing the third alternative on a multiple-choice test of science concepts; reporting allocations of time that add up to more than 100 percent; inconsistent answers, etc. Where such problematic responses are found, it is frequently necessary to eliminate the item or items from the data to be analysed.

After this is done, the data are prepared for computer analysis; usually this involves coding and entering (keying) the data with verification and quality control procedures in place.

The next step is to carry out the data analysis specified in the evaluation plan. While new information gained as the evaluation evolves may well cause some analyses to be added or subtracted, it is a good idea to start with the set of analyses that seemed to be of interest originally. For the analysis of both qualitative and quantitative data there are computer programs currently available that make the data analysis task considerably easier today than it was 25 years ago. These should be used. Analysts still need to be careful, however, that the data sets they are using meet the assumptions of the technique being used. For example, in the analysis of quantitative data, different approaches may be used to analyse continuous data as opposed to categorical data. Using an incorrect technique can result in invalidation of the whole evaluation project.

It is very likely that the initial analyses will raise as many questions as they answer. The next step, therefore, is conducting a second set of analyses to address these further questions. If, for example, the first analysis looked at overall teacher performance, a second analysis might want to subdivide the total group into subunits of particular interest - i.e., more experienced versus less experienced professionals – and examine whether any significant differences were found between them. These reanalysis cycles can go through several iterations as emerging patterns of data suggest other interesting avenues to explore. Sometimes the most intriguing of these are results which emerge from the data; ones that were not anticipated or looked for. In the end, it becomes a matter of balancing the time and money available, against the inquisitive spirit, in deciding when the analysis task has been completed.