|
MASSACHUSETTS COMPREHENSIVE ASSESSMENT SYSTEM (MCAS)
Overview of the MCAS Program
WHAT ARE THE STATE TESTING REQUIREMENTS?
MCAS was implemented in response to the Education Reform Law of 1993,
which required that MCAS be designed to
- test virtually all public school students across the Commonwealth, including students
with disabilities and students with limited English proficiency
- be administered annually in at least grades 4, 8, and 10
- measure performance based on the Massachusetts Curriculum Framework learning
standards
- report on performance of individual students, schools, and districts
- serve as one basis of accountability for students, schools, and districts (for example,
beginning in 2001, grade 10 students must pass the grade 10 tests as one condition of
eligibility for a high school diploma)
WHAT IS TESTED ON MCAS?
Learning standards in the Massachusetts Curriculum Frameworks
MUST ALL STUDENTS PARTICIPATE?
Yes. All public school students in grades 4, 8, and 10 must participate in
MCAS testing.
 | Students with disabilities will be allowed to use testing accommodations as necessary.
|
|
| The Department is currently designing testing alternatives for students with
disabilities who cannot be tested using accommodations, as well as for students with
limited English proficiency who have attended public schools in the United States for
three or fewer years. |
The Education Reform Law provides for no exemptions from taking the tests.
This policy assures that all Massachusetts public school students are provided an
opportunity to learn the materials covered by the Curriculum Framework learning
standards.
WHAT TYPES OF QUESTIONS APPEAR ON MCAS?
Multiple-choice questions
|
| used in all content area tests
|
|
| students select an answer from four options |
Open-response questions
|
| used in all content area tests
|
|
| students create a one- or two-paragraph response in writing or in the form of a chart,
table, diagram, illustration, or graph, as appropriate |
Short-answer questions
|
| used in Mathematics tests only
|
|
| students generate a brief response, for example, a short statement or computation
leading to a numeric solution |
Writing prompts
|
| used in English Language Arts tests only
|
|
| students write a composition based on the writing prompt, which may relate to a reading
passage |
HOW ARE TEST RESULTS REPORTED?
Results are reported for individual students, schools, and districts
according to four performance levels defined by the Board of Education:
|
| Advanced
|
|
| Proficient
|
|
| Needs Improvement
|
|
| Failing |
Content-specific performance level definitions are available from the
Department of Education.
Reports will include results for each content area as well as for
sub-areas within each content area, and will also show comparative results by school,
district, and state.
HOW ARE TEST RESULTS USED?
Improvements in teaching and learning
|
| Parents and students will use the results to monitor how students are progressing.
|
|
| Local educators will use results to help identify strengths and weaknesses in their
curriculum and instruction. |
School and district accountability
|
| In the near future, the Board of Education will establish standards for performance for
districts that improve or fail to improve student academic performance, as required by the
Education Reform Law. |
Student accountability
|
| Beginning with the Class of 2003, students will be required to pass the MCAS grade 10
tests to receive a high school diploma. Students will be given multiple opportunities, if
necessary, to pass the tests. Students must also meet local graduation requirements for
high school graduation, for example, completion of required coursework. |
WHAT IS THE MCAS SCHEDULE?
MCAS is administered in May of each school year, with test results reported
the following fall. The first MCAS administration took place in May 1998.
MASSACHUSETTS COMPREHENSIVE
ASSESSMENT SYSTEM (MCAS)
The MCAS Standard-Setting Process
OVERVIEW
Standard-setting is the process of determining the minimum total test score
(threshold score) for a particular performance level. Standard-setting should not be
confused with scoring, which is the process of assigning points to student responses.
Scoring must occur before standard-setting can begin.
In August of 1998, the Massachusetts Department of Education convened
panels, whose members included both educators and non-educators, to participate in the
standard-setting process for MCAS. The Board of Education has previously approved four
MCAS performance level categories: Advanced, Proficient, Needs Improvement, and Failing,
along with content- and grade-specific descriptors for each performance level. The panels
used the performance level descriptors to evaluate student performance on the MCAS tests.
Panelists were arranged into content- and grade-specific groups, for
example, grade 4 mathematics. Each panel made judgements about student performance based
on the criteria given in the performance level descriptors for their assigned content area
and grade level. The panelists used a standard-setting method referred to as the
Student-Based Constructed Response (SBCR) Method, in which judges/panelists examine
complete student response sets (student responses to multiple-choice questions and
samples of actual student work on open-response questions) and match each student response
set to one of the performance level categories.
THE SBCR METHOD
The SBCR Method of standard-setting involves three major procedures:
- Training/Calibration
- Range-Finding
- Pinpointing Threshold Scores
Each MCAS panel was assigned a facilitator who guided panelists through these three
procedures.
- Training/Calibration
During this first phase of the MCAS standard-setting process, panelists
reviewed, for their assigned content area and grade level, (1) all MCAS test questions and
(2) content- and grade-specific descriptors for each performance level. Panelists were
given the opportunity to discuss and comment on test questions and descriptors. Next, to
ensure that panelists attained a common interpretation of performance descriptors and the
relationship of those descriptors to student work, panel members individually assigned
performance levels to a set of six sample student responses. Panelists then compared their
individual results and discussed at length how the performance level descriptors supported
their conclusions.
- Range-Finding
During the range-finding phase of standard-setting, identical sets of
student work were provided to each panelist. Panelists were asked to independently
categorize the sets as Advanced, Proficient, Needs Improvement, or Failing, based
on the performance level descriptors, and without any discussion. This process revealed
which sets of student work generated the most agreement and which generated the
most disagreement between panelists. The results were documented, and the sets of
work which generated the most disagreement defined the score intervals in which the
threshold scores must fall. Many more student response sets from those intervals were
selected for further evaluation in the next phase.
- Pinpointing Threshold Scores
In order to determine final minimum scores for each performance level,
panelists examined sets of work about which they disagreed in the range-finding phase,
along with additional student response sets representing those same score intervals.
Panelists assigned performance levels to these sets of responses. The minimum score for
each performance level was precisely "pinpointed" by determining the score
around which there was, collectively, the maximum disagreement between panelists. This is
the point that best represents the transition from response sets at a higher level to
those at a lower level.
MASSACHUSETTS COMPREHENSIVE ASSESSMENT
SYSTEM (MCAS)
1998 Requirements for Student Participation
WHO MUST PARTICIPATE IN MCAS?
All public school students in grades 4, 8, and 10, including
students
- in charter schools
- in institutional school programs
- in educational collaboratives
- receiving publicly funded special education in 766-approved private schools
- with disabilities who either have an Instructional Education Program (IEP) or receive
Section 504 instructional accommodations
- who are limited English proficient (unless they have been enrolled in United States
schools for three or fewer years AND who are ineligible for the Spanish-language
version of MCAS)
Home-schooled students are not enrolled in the public school system and are
therefore not required or entitled by law to participate in MCAS. After MCAS has been
fully implemented, the Department of Education plans to consider whether and under what
circumstances it is feasible to permit privately educated students, including those being
home-schooled, to participate in the MCAS testing program.
WHY MUST THESE STUDENTS PARTICIPATE IN MCAS?
|
| The Education Reform Law of 1993 requires their participation. This law ensures that all
students are provided with an opportunity to learn the material covered by the
Massachusetts Curriculum Frameworks' academic learning standards.
|
|
| Beginning with the Class of 2003, all students will have to pass the grade 10 tests as
one condition for receiving a high school diploma. |
WHAT IF, AS A PARENT, I DO NOT WANT MY CHILD TO TAKE THE MCAS TESTS?
|
| Parents may not legally refuse their child's participation in MCAS. Massachusetts
General Laws chapter 76, Sections 2 and 4, establish penalties for truancy as well as for
inducing unlawful absence of a minor from school. In addition, school discipline codes
generally define local rules for school attendance and penalties for unauthorized absence
from school or from a required part of the school day. |
|
| Beginning with the graduating class of 2003, any student who does not pass the grade 10
MCAS tests will not receive a high school diploma. |
HOW DO STUDENTS WITH DISABILITIES PARTICIPATE IN MCAS?
|
| The student's IEP or Section 504 Team determines how the student will participate
in MCAS. If the Team determines that it is appropriate and allowed within the guidelines,
certain testing accommodations are permitted. If the Team determines that the student's
disability prevents the student from taking the MCAS tests, even with accommodations, the
Team must locally develop an alternative assessment for that student which is appropriate
to the student's academic development. The Department is in the process of developing a
system of alternative assessments for these students that will be available for future
MCAS administrations. |
HOW DO LIMITED ENGLISH PROFICIENT (LEP) STUDENTS PARTICIPATE IN MCAS?
LEP students are defined in the following ways:
- students who are enrolled in a Transitional Bilingual Education (TBE) program or who
receive English as a Second Language (ESL) support at school; OR
- students who were not born in the United States whose native language is a
language other than English and who are currently not able to perform ordinary classroom
work in English; OR
- students who were born in the United States to non-English-speaking parents and who are
not currently able to perform ordinary classroom work in English.
LEP students, including those in two-way bilingual programs, must take the English-language
version of MCAS if they meet any of the following definitions:
|
| The student has been enrolled in school in the United States for more than three years; OR |
|
| The student is currently enrolled in a TBE program or receives ESL support and has
been/will be recommended for regular education classes for the school year following MCAS
testing; OR |
|
| The student is in a two-way bilingual program and has been in school in the
United States for three or fewer years, but would likely be transitioned
into regular education classes, were such a program offered at the student's school. |
LEP students who are native-Spanish speakers must take the Spanish-language
version of MCAS if they meet all of the following requirements:
|
| The student has completed three or fewer years of school in the United States; AND
|
|
| The student is currently enrolled in a TBE program or receives ESL support and will not
be recommended for regular education classes for the following school year; AND
|
|
| The student does not have adequate English-language skills to participate in the
English-language version of MCAS; AND
|
|
| The student possesses reading and writing skills in Spanish appropriate to his or her
grade level. |
The Spanish-language version of MCAS includes tests in Mathematics
and Science & Technology only.
LEP students who meet the following requirements are not required
to participate in MCAS at this time:
|
| Spanish-speaking LEP students who have been enrolled for three or fewer years of school
in the United States AND will not be recommended for regular
education classes in the following school year, AND who do not possess sufficient
reading and writing skills in Spanish to permit their participation in the
Spanish-language version of MCAS.
|
|
| Non-Spanish-speaking LEP students who have been enrolled for three or fewer years of
school in the United States AND who will not be recommended for
regular education classes in the following school year. |
MASSACHUSETTS COMPREHENSIVE ASSESSMENT SYSTEM
(MCAS)
Facts on Scoring of Student Answers to Open-Response Questions and
Writing Prompts
WHAT WAS SCORED?
Over 6 million total student responses
- to open-response questions in English Language Arts (Language and Literature
Component), Mathematics, and Science & Technology
- to writing prompts for English Language Arts short and long compositions
HOW WAS SCORING DONE?
Use of scoring rubrics
|
| Responses to open-response questions are scored using a scoring guide, or rubric. MCAS
rubrics indicate what knowledge and skills students must demonstrate to earn 1, 2, 3, or 4
score points.
|
|
| Students' writing is evaluated on two criteria:
 | topic development, based on a 1-6 score point scale
|
|
| standard English conventions, based on a 1-4 score point scale |
|
Scorer training
Scorer training sessions require that scorers
- answer each test question or writing prompt to be scored
- review the content covered by the question or prompt and discuss its scoring rubric and
benchmark student responses
- score a set of responses (training pack)
- discuss training pack responses and scores assigned to them
- score another set of responses (qualifying pack)
- be retrained, as necessary, until the scorer meets established standards for score
reliability for every test question or writing prompt to be scored
Scoring system
- (All but long compositions) Scorers view electronic copies of digitally scanned images
of student responses on a computer monitor and assign scores electronically. Use of this
computerized scoring system assures that student responses are randomly assigned to
scorers, second readings are truly "blind," and supervisors have immediate
access to information about scorer accuracy.
- Students' actual long compositions are each scored twice (by two separate scorers) for
both topic development and mechanics usage.
WHO DID THE SCORING?
Professional scorers (approx. 700) and Massachusetts teachers (approx. 700
selected applicants from across the Commonwealth)
|
| Professional scorer qualifications/requirements
- A minimum of two years of college coursework in the discipline to be scored
- Approximately 70 percent of Advanced Systems' scorers hold bachelor's or more advanced
degrees (Advanced Systems is the state's testing contractor.)
|
|
| Massachusetts teachers
- Currently teaching in the Commonwealth
|
WHERE DID SCORING TAKE PLACE?
Massachusetts teachers
- Department of Education Writing Institutes in Mansfield, Holyoke, and Worcester
Professional scorers
- Professional Scoring Centers in Albany, NY; Cape Elizabeth, ME; and Dover, NH
WHEN DID SCORING TAKE PLACE?
July and August, 1998
|
