As reported earlier, achievement items for the 1995 British Columbia Assessment of Mathematics and Science were drawn from three sources: the 1990 Provincial Mathematics Assessment, the 1991 Provincial Science Assessment, and the Third International Mathematics and Science Study (TIMSS). As a result, there was an opportunity to make student achievement comparisons over time, given the extensive use made of items from earlier assessments.

In this section, comparisons by grade level are made between student achievement in 1995 and that which occurred in 1990 for mathematics and in 1991 for science. Change categories correspond to the 1995 tables of specifications (see Chapter 2) and include at least six items for reporting purposes .

Results are discussed by subject and, within each subject, by grade level.

7.1 Mathematics Achievement Changes

At all three grade levels, sufficient items from the 1990 assessment were used to enable a substantial number of comparisons. These comparisons are reported in the next four sections.

Student results were examined for changes in performance on 61 items common to both the 1990 and 1995 assessments. Of these, 52 appeared in sufficient numbers by reporting category to be discussed as content reporting categories in this section. At least six items are needed to comprise a reporting category. A summary of reporting categories with item locations is shown in Table 7.1.

Table 7.1 Grade 4 Mathematics Change 1990 to 1995 (5K)

7.1.1.1 Numeration

Student achievement improved on five of the nine items on Numeration, resulting in an increase of 1.4 percentage points in the Mean Percent Correct for this reporting category.

Students improved on items involving conversion between word and numeral forms, and on those involving ordering numbers by magnitude. Results dropped somewhat on items involving more than one step.

7.1.1.2 Whole Numbers

Performance dropped on nine items, improved on five, and remained the same on two. Overall, there was a decrease of 1.4 percentage points in the Mean Percent Correct for this reporting category.

Results decreased on several questions involving basic operations of multiplication and division. Word problems showed mixed results, with both increases and decreases.

7.1.1.3 Fractions

Overall achievement on Fractions was about the same as in 1990, with results showing a net decrease of about half a percentage point in the Mean Percent Correct for this category. Improvement was shown on three items; performance decreased on six.

Students showed improvement on two of the items involving the pictorial representation of fractions. Decreases occurred on items involving conversions among forms of rational numbers and on those involving placement of fractions on the number line.

7.1.1.4 Geometry

Results in the area of Geometry showed improvement over 1990. Achievement improved on six of the nine items in this category. Overall, the Mean Percent Correct increased by 1.3 percentage points within this set of items.

Students improved on items involving lines of symmetry and congruent figures. Results dealing with spatial relationships, however, were mixed, with both increases and decreases shown on items.

7.1.1.5 Measurement

Improvement was shown on eight of the nine items on Measurement. Overall, there was an increase of 3.2 percentage points in the Mean Percent Correct for items in this category. The greatest increases were on items involving estimating the area of a shape, comparing the areas of two figures, and selecting the correct units of measure for capacity and mass.

7.1.1.6 Miscellaneous Change Items

The nine change items not part of a reporting category dealt with a variety of concepts, including Place Value and Data Representation. Students showed improvement on five of these items, decline on two, and similar results on two. There was an overall gain on these items of 2.0 percentage points.

7.1.1.7 Summary for Grade 4 Mathematics

Improvement was shown on three of the five reporting categories. Among the areas of improvement were problem solving, knowledge of relationships among geometric figures, determining the area of figures, and use of terminology. Declines were evident in the basic calculation of whole numbers and fractions. Overall, student performance on the 61 items common to the 1990 and 1995 assessments was 0.7% higher in 1995.

There were 61 items for which results were available from both the 1990 and 1995 assessments. Given the requirement of a minimum of six item for a reporting category, 50 of these are discussed by topic. A summary of these, showing item locations, is shown in Table 7.2.

Table 7.2 Grade 7 Mathematics Change 1990 to 1995 (5K)

7.1.2.1 Decimals

Results on eight of the nine Decimal items were lower in 1995 than in 1990. These items included all of the basic operations of addition, subtraction, multiplication, and division of decimals. Results were lower both for computation items and for word problems dealing with these concepts. The only item showing an increase was item A49 which involved converting a decimal to a percent. Overall, there was a decrease of 6.0 percentage points in the Mean Percent Correct for this category.

7.1.2.2 Fractions

Four of the six items dealing with common fractions showed declines over 1990. Overall, the net decline across the items was 0.7 of one percentage point.

Items on which performance dropped included operations with mixed numbers, subtraction of a fraction from a whole number, and the addition of fractions. The two items showing increases were word problems, one involving addition and subtraction, and the other multiplication of fractions.

7.1.2.3 Ratio/Proportion/Percent

Results for Ratio, Proportion, and Percent were also down. Students attained lower scores in 1995 on all six of the items in this category. Overall, results showed a net decrease of more than four percentage points.

The items dealt with equivalent ratios, the use of proportions to solve equations and word problems, finding the percent of a number, and conversion of a fraction to a percent.

7.1.2.4 Geometry

Student achievement on the 13 Geometry items increased on four items, remained the same on two, and decreased on seven. Overall, there was a net decrease of 1.9 percentage points in the Mean Percent Correct.

Items on which increases were shown dealt with measures of congruent angles, lines of symmetry, and the rotation of figures and objects. Items involving similar triangles, coordinates of points, reflections, and the diameter of a circle registered decreases in performance.

7.1.2.5 Measurement

Results showed a decrease in achievement on five of the six Measurement items. There was a net decline among items of 4.5 percentage points.

Outcomes measured by these items included the perimeter of a figure, the volume of a box, and the area of a classroom. These results show that students are less able to deal with the concepts of area, perimeter, and volume than their counterparts in 1990.

7.1.2.6 Algebra

There were ten change items in Algebra, and results showed increases on five items and decreases on five. Overall, there was no change in Mean Percent Correct across items.

Items on which students showed improved performance included evaluating algebraic expressions, translating from number sentences to equations, and solving a linear equation. Decreases in achievement were shown on items involving the solution of an equation containing fractions, and recognition of patterns involving variables.

7.1.2.7 Miscellaneous Change Items

The 11 change items not part of a reporting category dealt with a variety of concepts, including Numeration, Whole Number Operations, Integers, and Data Analysis. Students showed improvement on two of these items, and decline on nine. There was an overall decline on these items of 0.5 percentage points.

7.1.2.8 Summary for Grade 7 Mathematics

Achievement declined on the majority of items contained in five of the six reporting categories. These results show significant drops in achievement on items involving calculation with whole and rational numbers. In addition, students were less successful than they were five years ago when it came to answering questions involving area, perimeter, and volume. Performance was also lower on items requiring knowledge and/or application of various geometric concepts. Over all of the 61 change items, there was a decrease of more than two (2.3) percentage points in the level of student achievement.

Some concerns were expressed by the Interpretation Panel during its deliberations on changes for a number of sample items. Panelists questioned whether the increased use of calculators in classrooms resulted in lower scores on calculation questions, given that their use was not permitted on this assessment. The panel was also concerned about the poor problem-solving skills demonstrated by students.

Overall, it is clear that much greater attention needs to be paid to the development of suitable mathematics understanding in the intermediate years. The results are so generally decreased from those in the 1990 assessment that it is recommended:

Recommendation 7.1

That the Ministry of Education investigate the reasons for the marked decline in students' achievement in mathematics at the Grade 7 level and institute measures designed to raise students' understanding of mathematics to a more satisfactory level.

Part of the investigation might address the lack of a common mathematics textbook or small number of textbooks at the Grade 7 level and the relationship that such a lack has to the current situation. In addition, attention should be focussed on in-service activities and resource packages for teaching integer concepts, given that this topic is a relatively recent addition to the curriculum and may not yet be taught appropriately.

Change in mathematics achievement at the Grade 10 level is examined over two populations, students enrolled in Mathematics 10 and students enrolled in Mathematics 10A.

7.1.3.1 Mathematics 10

Thirty-three Math 10 items were repeated from the 1990 assessment. Twenty-seven of these were contained in reporting categories for change, where a minimum of six items per category is required. A summary of the items, with their locations, is shown in Table 7.3.

Table 7.3 Mathematics 10 Change Items (5K)

7.1.3.1.1 Geometry

Of the 10 items in the Geometry category, scores improved on four, remained the same on one, and declined on five. Overall, there was an approximate half percentage point decline for the category.

Improvement was shown on items involving lines cut by a transversal and proving triangles to be congruent. Declines were shown on items involving the point of intersection of two lines, and on items requiring knowledge of the slope of a line. Items involving similar triangles showed both increases and decreases.

7.1.3.1.2 Algebraic Expressions

Students improved their performance on five of the nine items testing Algebraic Expressions. There was an overall increase of one percentage point in the Mean Percent Correct for this category.

Among the outcomes showing improvement were evaluating algebraic expressions by substituting values, factoring a binomial, and simplifying an expression using laws of exponents. Items that showed declines tested greatest common factor and the division of monomials with variables raised to powers.

7.1.3.1.3 Equations and Inequalities

Students showed improved performance on seven of the eight items involving Equations and Inequalities. Overall, a net improvement of 2.6 percentage points was shown on these items.

Improvement was shown on items measuring a number of outcomes in this area, among them solving a linear equation containing fractions, solving equations in terms of literals, and solving a linear inequality.

7.1.3.2 Miscellaneous Change Items

The six change items not part of a reporting category dealt with concepts involving Irrational Numbers and Data Analysis. Two questions asked students to write expressions in simple radical form, one asked which type of graph would be most appropriate to represent a set of data, one involved plotting a set of data as points in a plane, and two required determining the likelihood of an event.

Students showed improvement on three of these items, decline on two, and similar results on one. There was an overall gain on these items of 0.1 percentage points.

7.1.3.3 Summary of Change Over Time for Mathematics 10

Students in Math 10 improved their achievement between 1990 and 1995 on two of the three reporting categories. In particular, they did better on items involving the simplification of algebraic expressions and the solution of equations. In contrast, they did less well on items in the area of analytic geometry. These items included finding the slope of a line and the point of intersection of two lines in a plane. Over all of the change items, there was an increase in Mean Percent Correct of just under one percentage point.

Table 7.4 shows the three change categories for Mathematics 10A. They include a total of 22 items, of which six measure Whole Numbers, seven involve Rational Numbers, and nine deal with concepts of Geometry. Another 10 items, not constituting a change category, were spread across Data Analysis, Measurement, and Algebra.

Table 7.4 Mathematics 10A Change Items (5K)

7.1.4.1.1 Whole Numbers

Results improved on five out of the six items measuring Whole Numbers and remained the same on one. There was an overall increase of almost five percentage points in Mean Percent Correct across the six items. A major outcome on which improvement was shown was simplifying numerical expressions. Several items tested this skill: some involved order of operations, and others included whole number exponents.

7.1.4.1.2 Rational Numbers

Achievement improved on five out of seven items involving Rational Numbers. There was an average gain of 1.4 percentage points across items in this category.

Students showed gains on items measuring conversions from percent to a decimal and from a fraction to a decimal, and on word problems involving basic operations with fractions and decimals. A decrease in performance was shown on an item involving the subtraction of mixed numbers.

7.1.4.1.3 Geometry

Achievement improved on six of the items in the Geometry reporting category. It remained the same on one item and declined on two. The overall change for this category showed an improvement of 1.2 percentage points.

Students improved on items involving the coordinates of points in a plane and basic trigonometric ratios. Achievement dropped in the area of similar triangles.

7.1.4.1.4 Miscellaneous Change Items

Of the 10 items not included in the change categories, three tested Data Analysis, three Measurement, and four Algebra. Six showed increases over 1990, one was the same, and three showed declines. Overall, there was a net increase of 2.0 percentage points across these items.

7.1.4.1.5 Summary of Change Over Time for Mathematics 10A

Mathematics 10A students showed improvement in all of the change categories. There were increases in performance on 22 items and declines on only seven, across all categories. This positive result could be due to a number of factors, among them the availability of a textbook which was not available in 1990 and the expanded use of calculators in the classroom. Overall performance showed an increase of 1.9 percentage points in Mean Percent Correct.

.2 Science Achievement Changes

At all three grade levels, it was possible to use a number of science items which had been used in 1991. This report will deal with changes at objective and goal levels, whenever there are sufficient items to draw conclusions (a minimum of six items is required to make a meaningful comparison).

At the Grade 4 level, there were 75 items from the 1991 assessment: 50 from Goal B (Science Processes); 19 from Goal C (Scientific Knowledge); and six from Goal D (Cognitive Processes). Table 7.5 displays the results for the Grade 4 change items.

Table 7.5 Grade 4 Science Achievement Change 1991 to 1995 (5K)

7.2.1.1 Goal B: Science Processes

Fifty items measuring eight objectives of Goal B (Science Processes) were common to both the 1991 and 1995 assessments. Overall achievement on Goal B decreased by about three percentage points, and results declined on 34 of the 50 items.

There were four objectives with sufficient items to draw conclusions about change in student performance over time: Observation, Classification, Quantification, and Communication. It can be seen from Table 7.5 that six of the eight items dealing with Observation, six of the nine items dealing with Classification, eight of the 11 items dealing with Quantification, and seven of the 11 items dealing with Communication showed declines over 1991. Of the 11 items distributed across the four objectives of Goal B not reported in the table (Inference, Safe Use of É, Information, and Solve Problems), achievement declined on seven items and improved on four items.

7.2.1.2 Goal C: Scientific Knowledge

Nineteen items measuring Goal C (Scientific Knowledge) were used in both the 1991 and 1995 assessments. Of these, results improved on seven items and declined on 12 items. There was a decrease of nearly two percentage points in the Mean Percent Correct for this goal.

7.2.1.3 Goal D: Cognitive Processes

Results for Goal C (Cognitive Processes) were also down. The Mean Percent Correct showed a decrease of 4.3 percentage points, and students attained lower scores in 1995 on all six items in this category.

7.2.1.4 Summary for Grade 4 Science

Clearly, compared to the 1991 assessment, there has been an overall decrease (of nearly three percentage points in Mean Percent Correct) in student achievement as measured by these multiple-choice items. It was reported in the 1991 assessment that there had been a decrease in achievement levels between 1986 and 1991. It appears that the decline in student achievement that was noted in 1991 has not been stopped, and primary students in British Columbia continue to lose ground in science.

At the Grade 7 level, there were a total of 75 items from the 1991 assessment: 39 from Goal B (Science Processes), 25 from Goal C (Scientific Knowledge); and 11 from Goal D (Cognitive Processes). Table 7.6 displays the results for the Grade 7 change items.

Table 7.6 Grade 7 Science Achievement Change 1991 to 1995 (5K)

7.2.2.1 Goal B: Science Processes

There were 39 items measuring Goal B (Science Processes) common to both the 1991 and 1995 assessments. Of these, results improved on 17 items and declined on 22. Overall achievement on Goal B decreased by approximately one percentage point.

Although these items were used to measure student performance on nine objectives within this goal, only two objectives, Prediction and Identify and Control Variables, had enough items to draw conclusions about change in student performance since the 1991 assessment. Table 7.6 shows that results improved on three of the six Prediction items and declined on three, and that results improved on only one of the six Identify and Control Variables items.

7.2.2.2 Goal C: Scientific Knowledge

Twenty-five items measuring Goal C (Scientific Knowledge) were common to the 1991 and 1995 assessments. Results on 10 of the items improved and 15 declined. The Mean Percent Correct for this goal showed a decrease of almost three percentage points.

The objective Knowledge of Facts had enough items common to both assessments to draw conclusions about change in student performance. It should be noted that there are three sub-objectives within this objective: Knowledge Earth/Space, Knowledge Life Sciences, and Knowledge Physical Science. Of the 12 items measuring this objective, results improved on six items and declined on six; the overall Mean Percent Correct went down nearly two percentage points.

7.2.2.3 Goal D: Cognitive Processes

Results for Goal C (Cognitive Processes) were also down. The Mean Percent Correct showed a decrease of 1.5 percentage points, and students attained lower scores in 1995 on eight of the 11 items in this category.

7.2.2.4 Summary for Grade 7 Science

As in Grade 4, there was decrease in student achievement in 1995 compared to the 1991 assessment as measured by these multiple-choice items. In the 1991 assessment, a decrease from the level of achievement demonstrated in the 1986 assessment was also reported. It appears that the decline in student achievement at Grade 7 that was noted in 1991 has also not been stopped, and intermediate students in British Columbia continue to score lower in science assessments of this type, although the decline (1.7 percentage points in Mean Percent Correct) does not appear to be as great as in the primary area. While there appears to be little cause for concern at the junior secondary level, the fact that student achievement in science at the elementary level has continued the decline noted in the 1991 assessment is alarming. The 1991 assessment report discussed some of the possible reasons for the decline noted in 1991; these were:

• greater integration of special needs students
• changing emphasis of the curriculum and increased demands to accommodate other programs such as safety, cancer, drug and alcohol awareness, Family Life, etc.
• an increasing number of interruptions to the "normal" school day
• an increased number of ESL students
• increased awareness of the Year 2000 (interpretations of the new Primary and Intermediate Programs have resulted in a perception of less class time being devoted to science)
• less funding available for in-service in science
• reduced funds for the purchase and replenishment of science equipment and supplies (Bateson et al., 1992)

There is no reason to believe that those comments do not still apply. These factors were offered without value judgments, and suggestions were made that further study is required to substantiate or refute these factors.

At the Grade 10 level, there were a total of 49 items from the 1991 assessment: nine from Goal B (Science Processes), 32 from Goal C (Scientific Knowledge); and eight from Goal D (Cognitive Processes). Table 7.7 displays the results for the Grade 10 change items.

Table 7.7 Grade 10 Science Achievement Change 1991 to 1995 (5K)

7.2.3.1 Goal B: Science Processes

Nine items from the 1991 assessment were used in 1995 to measure student performance on Goal B (Science Processes). Student results improved on six of the items, and the Mean Percent Correct went up by about six percentage points.

7.2.3.2 Goal C: Scientific Knowledge

Thirty-two items measuring Goal C (Scientific Knowledge) were common to the 1991 and 1995 assessments. Results improved on seven of the items and declined on 25. The Mean Percent Correct for this goal showed a decrease of almost three percentage points.

The objective Knowledge of Facts had sufficient items common to both assessments to draw conclusions about change in student performance since 1991. It should be noted that there are six sub-objectives within this objective: Astronomy/Space, Earth Science, Life Functions, Ecology/Resource, Changes in Matter, and Energy. Of the 24 items measuring the objective Knowledge of Facts, results improved on five items and declined on 19, and the overall Mean Percent Correct went down nearly three percentage points.

7.2.3.3 Goal D: Cognitive Processes

The Mean Percent Correct for the eight items measuring Goal D (Cognitive Processes) used in both the 1991 and 1995 assessments declined by 1.5 percentage points. Student performance improved on three of the items and declined on five.

7.2.3.4 Summary for Grade 10 Science

There has been an overall very slight decrease in student achievement compared to 1991, as measured by these multiple-choice items. It appears that the overall substantial decline in student achievement that has been noted in Grade 4 and 7 has not continued into the junior secondary level. There appears to be a substantial increase (over six percentage points in Mean Percent Correct) in the Goal B (Science Processes) area, and a decline (nearly three percentage points) in the Goal C (Scientific Knowledge) areas, and a decline of more that one percentage point in Goal D (Cognitive Processes) areas.