PERFORMANCE AND PORTFOLIO ASSESSMENT FOR LANGUAGE MINORITY STUDENTS

PORTFOLIO ASSESSMENT

Portfolios present a practical approach to assembling student work, interpreting evidence of student performance, and assessing student performance relative to instructional objectives. The concept of portfolios has been adopted from the arts where students maintain evidence of their best work to illustrate their accomplishments (Jongsma, 1989). In classroom instruction, portfolios are used in a similar manner, but the contents of the portfolio may represent work in progress, formal products, and ratings or other evidence of student knowledge relative to specific objectives or purposes (Valencia, 1990).

There is no "right" way to design portfolios. Each classroom, school district, and state will reflect a unique approach to authentic assessment, and in this sense, each student's collection of documents will differ somewhat, depending on the purpose of the assessment (Tierney, Carter, & Desai, 1991). Creating and maintaining student portfolios require that a variety of teacher and staff decisions be made concerning the instructional goals or objectives to be assessed, who will be involved in the portfolio design and interpretation, what instruments will be selected and how student performance will be demonstrated, how portfolio information will be used, and how the results will be conveyed to others. Because the entire portfolio process can be complex, systematic review and evaluation of the process should be conducted on a periodic basis.

Steps to portfolio development should include designing the portfolio, planning for and collecting the necessary data, analyzing the portfolio contents, and using the results (Moya & O'Malley, in press). Each of these points will be described in the following sections.

Designing Portfolios

For the purposes of assessment, the material in a student portfolio is most useful when each piece collected reflects progress toward particular learning goals. To this end, portfolios can be designed following a multi-step process that involves:

• setting the purpose of the portfolio;
• focusing on specific learning goals;
• identifying performance tasks and/or selecting appropriate instruments;
• setting criteria;
• selecting students to be assessed;
• collaborating with other teachers and staff;
• conducting staff development; and
• involving students and parents in the portfolio development process.

Each of these steps is discussed below.

Purpose

Before collecting any samples of student work, the first step in planning a portfolio is to determine the purpose for conducting the assessment, and how the results will be used (Moya & O'Malley, in press; Navarrete et al., 1990). Will the results be used for making decisions related to classroom instruction? Will they be used to determine whether a student is ready to move out of a special English language support program, such as ESL, ESL-content, or bilingual education? Will they be used to aid in assigning a student grade? Specifying how the results of the portfolio assessment are to be used will assist in determining the goals to be assessed and the samples of student work to be collected.

Specific Focus

The second step in portfolio design is focusing the portfolio on specific learning goals. Each portfolio should have a specific focus determined by school staff. The focus may be on oral or written language skills or on content area skills such as those required in mathematics, science, or social studies. Objectives may also be selected from goals contained in local curriculum frameworks, state guidelines, program objectives, or consensus among ESL/bilingual and mainstream teachers concerning important goals for learning. While it may be possible to collect student work for all content areas as well as for English language skills in a single student folder, attempting to do this for purposes of assessment could prove to be rather unwieldy. Providing a focus, on the other hand, avoids having to go through an overwhelming amount of information in each portfolio.

Performance Task/Instrument Selection

Once learning goals and performance objectives have been identified, portfolio designers need to identify performance tasks and instruments to be used to measure whether learning goals are being attained. School staff should strive to combine traditional and performance assessment measures in order to get multiple indicators of a student's ability level. Standardized tests are often required for district accountability needs. Using results obtained on standardized achievement tests together with anecdotal records, rating scales, teacher observation checklists, and writing samples to assess literacy skills provides much more information than standardized test results alone. Furthermore, having multiple indicators of student performance enables teachers to cross-check one type of information against another.

Each portfolio should also contain items which are required to assess progress on particular instructional goals and others which are optional. Required items might include those which are necessary to communicate a student's progress to other teachers or to administrators and can include a student's "best work," while optional items could include drafts of work in progress, ongoing ratings of performance, and occasional pieces selected by the student (Valencia, 1990). The use of required items introduces an element of consistency in the evaluation of student portfolios. By making certain items obligatory and others optional, teachers get the information they need for making instructional decisions while also encouraging students to participate actively in portfolio design and use.

Setting Criteria

Teachers or school staff should determine criteria (performance standards) for interpreting portfolio contents before collecting any student data. Performance criteria must be established in order to determine the degree to which a student has attained the objectives each task/instrument is designed to assess. Teachers need to identify and establish a minimum number of specific objectives that illustrate attainment of the instructional goals. One way to set criteria is to require students to perform tasks either independently or with assistance. Another possibility is to define expected student performance in narrative or anecdotal form. The narrative can specify what the students should be able to do to meet the criterion for performance or growth over time.

Staff Collaboration

If portfolio assessment is to be undertaken by a school-based team, it will be essential to identify school staff willing to participate in the assessment process. Ideally, a cross-section of teachers, staff, and administrators at each school who serve the same student(s) could become members of a portfolio assessment team. For example, a team at the upper elementary levels might consist of an ESL or bilingual education teacher, the grade level classroom teacher, a reading specialist, and the school principal. At the middle school level, the team might consist of a student's ESL or bilingual education teacher, content area teachers, and perhaps the school counselor. If portfolio assessment is a totally new experience for school staff, it is probably a good idea to pilot test the approach with a small number of staff and students before using it on a school-wide or district-wide basis.

Staff Development

All staff involved in the portfolio process should receive information and training on how to plan, implement, and interpret portfolios, especially when portfolio assessment is to be conducted at the school-building or district-wide level. Staff preparation not only enables staff to collaborate with and support each other, it also builds critical support for the portfolio process itself. Staff should receive training on how to design portfolios, how to target specific learning objectives and select students (if portfolios are limited to only a part of the student population), and how to set criteria for each portfolio. Staff development will also be essential to planning individual portfolio contents and to designing, administering, and scoring holistic, performance-based measures such as oral interviews, teacher observation checklists, rating scales, and writing samples. Performance and portfolio assessment hold great promise for improving assessment, but they can only reach their potential when teachers master their use (Stiggins, 1990).

Student Selection

Portfolio teams or individual teachers need to consider several factors when deciding whether to implement portfolio assessment with one or more students. If the classroom teacher is acting on his/her own to gather the information (without any support from other school staff), initially it may be advisable to limit the number of portfolios to only a few students. This can prevent teachers from being overwhelmed by the data collection and analysis effort and giving up before experiencing the benefits of portfolio assessment. On the other hand, if portfolio assessment is to be a school-or district-wide initiative, and if more than a few teachers are going to be involved and provided staff development in its systematic implementation, then many or all students can be included in the procedure.

Questions to ask regarding which students will participate in the portfolio process are: Will the portfolio be used in only one classroom? Will it be used only with students participating in a particular program (such as ESL, bilingual education, or Chapter 1)? Will the process be limited to a single or multiple grade levels? Will all students in each classroom be assessed using a portfolio? Will the procedure be limited to only those individuals needing frequent monitoring?

Student/Parent Involvement

The teacher and/or portfolio assessment team should encourage the active involvement of both students and parents in the assessment process. A key element in portfolio assessment is student self-evaluation. Students are asked to reflect on their progress toward learning goals and encouraged to select samples of their work which they believe illustrate progress toward these goals (Baron, 1992a; Palmer Wolf et al., 1992; Tierney, Carter, & Desai, 1991). Teacher/student/parent conferences can be scheduled at times convenient for the parents so that they can be informed of their child's progress. Portfolio contents provide much more information to parents about their child's learning than the percentiles represented on standardized achievement tests. Furthermore, parents appreciate being given an opportunity to respond to examples of student work, particularly items that show progress and substantiate grades given to the student.

Planning for and Collecting the Data

Data collection for portfolio assessment consists of identifying information to be entered into the portfolio, determining the frequency of data collection, creating a system to record and monitor the frequency of data collection, and setting guidelines for the removal and updating of portfolio material. These guidelines may vary depending on the purpose of the portfolio.

At the elementary and middle school levels, portfolios can be updated on a semester basis or at each grading period. Some schools are experimenting with portfolios which present increasingly comprehensive information on students by beginning with an initial portfolio every nine weeks, a semester portfolio drawn from these, and a year-end portfolio (Palmer Wolf et al., 1992). However, if the purpose is to closely monitor student progress, assessments should take place approximately every four to six weeks. Occasional items can be placed in the portfolio on a more frequent basis and removed when they have been superseded by more recent work or have become redundant. The decision to remove or maintain portfolio materials is a collaborative one to be made between the student and the teacher.

Analyzing Portfolio Contents

To determine whether a portfolio's contents reflect a student's progress toward learning goals, the teacher or portfolio assessment team can match contents to specific learning goals and objectives on a cover sheet, as illustrated in the Sample Portfolio Analysis Form in Figure 5 (adapted by Pierce from Moya & O'Malley, in press). Note that student objectives are placed in the left-most column of Figure 5, followed by illustrations of student progress, and a specific citation or page reference to materials that support each objective. When additional evidence of student progress for each objective is entered or found in the portfolio, relevant page citations can be added.

Summary comments, interpretations, and recommendations can be added at the bottom of the Portfolio Analysis Form. These anecdotal notes help interpret and integrate the results of student performance across various measures or skill areas. Forms such as the one in Figure 5 offer several advantages: teachers can indicate the relationship between each item in the portfolio and the objectives being assessed; they can make specific suggestions for instructional adaptations to student needs; and they can sift through portfolio contents periodically to remove materials that, although interesting, have no use in evaluating student progress.

To determine how students acquiring English as their second language are progressing in comparison to other students, performance assessments can be administered to native or proficient English-speaking grade-level peers in mainstream classes and average ratings calculated for these students as well as for the English language learners. Administering any of the performance assessments described in this publication to a locally selected, "average" group of English-speaking peers will provide the most meaningful basis for comparison. This will inform the teacher regarding both the English learner's progress and his/her preparation for functioning at an independent/average level in a typical mainstream classroom.

FIGURE 5. Sample Portfolio Analysis Form

DATE: 5/1/92

STUDENT: Marisel A.

TEACHER: Jones

GRADE: 4

EDUCATIONAL GOAL: Student demonstrates ability on variety of writing tasks

PERFORMANCE TASK	CONTENTS ILLUSTRATING STUDENT PROGRESS	DATE
* Demonstrates interest and ability in variety of writing	Literacy development Checklist	3/20/92
* Writes a short story	Writing Sample: Dog Story	4/22/92
* Writes to communicate with others	Letter Dialog Journal	4/10/92 3/31/92
* Expresses writing Preferences	Self-Assessment of Writing	4/24/92
* Shares writing with others	Anecdotal record	4/6/92
Summary Comments: _____________________________________________________________ _____________________________________________________________ _____________________________________________________________

Using Portfolio Results

There are a variety of ways in which portfolio results can be used. The Sample Portfolio Analysis Form shown in Figure 5 is an essential component in many of these uses:

• diagnosis and placement- student strengths and needs are examined with regard to major curriculum objectives;
• monitoring student progress- growth in learning over the course of the semester or school year can be monitored;
• feedback on the effectiveness of instruction- if individual students are not progressing, the instructional approach should be re-evaluated and appropriate adaptations made to meet each student's needs. One possible conclusion is that a student needs instructional support beyond the services provided by the classroom(s) in which the portfolio has been maintained;
• communication with other teachers- this includes other members of the portfolio team and those at other schools to which students may transfer;
• student feedback- portfolios enable students to comment and reflect on their progress and plan what they would like to do to maintain or change it; and
• communication with parents- portfolios provide parents with concrete evidence which supports instructional decisions.

PORTFOLIOS IN PRACTICE: SOME COMMON CONCERNS

As a result of conducting workshops with teachers and administrators on portfolio assessment, it has been our experience that, while the use of portfolios has many advantages, it is not without its limitations. In this section we will describe some of the more commonly identified obstacles to implementing portfolios and suggest procedures for overcoming them. We will also look briefly at how portfolios have been used in some states and school districts and as part of ESL/bilingual programs to gain insights into real problems and possible solutions.

"Portfolios take too much time."

Teachers and administrators frequently express concerns that designing and using portfolios are time consuming processes. This is commonly expressed by teachers in the initial stages of portfolio assessment, but appears to become of lesser concern as teachers begin to establish a classroom environment that encourages students to be increasingly independent and responsible for assessing their own progress using portfolios (Kolls, 1992; Tierney, et al., 1991).

There are several ways to reduce the time involved in implementing portfolio assessment:

1. make the data collection part of daily instructional routines;
2. make students responsible for collecting information on a regular basis;
3. identify specific items that go into the portfolio and list them on a portfolio analysis form;
4. initially, use portfolios with only two or three students who need intensive monitoring;
5. use staggered data collection cycles where assessment data are collected from only a few students daily or weekly;
6. share responsibilities of data collection and interpretation with other school staff so that individual teachers do not become overwhelmed by the process; and
7. create common planning times for teachers and other staff involved in portfolio development.

"How do we decide what goes into the portfolio?"

The teacher or portfolio assessment team determines the contents of each portfolio by identifying learning goals and specifying minimal levels of student performance that show whether students have attained these goals. Instruments are then selected for eliciting the desired student performance and criteria set for determining performance levels.

"How do we interpret the contents of a portfolio?"

A major concern in portfolio assessment is what to do with portfolio contents once they have been collected. Prior to attempting to interpret this information, the teacher or portfolio assessment team should identify learning goals, student performance levels that demonstrate attainment of these goals, the tasks or instruments to be used to elicit student performance, the criteria for determining student progress, and whether specific portfolio items will be required or optional. Next, the teacher or assessment team must determine how much weight to give each item in the portfolio. As with the portfolio design process, guidelines for interpreting portfolio results should be established in advance of data collection.

"How are others using performance-based assessment and portfolios?"

Most of the portfolios in practice that we know of in language arts, ESL, and bilingual programs are used to assess literacy skills (see Figure 6). We know of few which focus on content area skills or oral language proficiency. In Figure 6 we provide a sampling of contents suggested for reading/writing portfolios by two ESL programs in two school districts in Virginia (Arlington and Fairfax) and two language arts programs in one county in Florida (Orange County) and one elementary school in New Hampshire (Stratham Memorial School).

What becomes immediately evident from Figure 6 is that while Arlington County and Stratham Memorial School distinguish between reading and writing entries, Orange and Fairfax Counties indicate core (required) and optional items, as we have recommended in this publication. In addition, all of the portfolios indicated in Figure 6 incorporate many of the items we have described in this publication, including test results, lists of books read, writing samples, learning logs, reading/writing checklists, and student self-assessments.

At the state level, more and more states are moving to performance assessment to ensure that students graduate with more than basic skills and to get a more complete picture of students' abilities. A 1990 survey by the Center for Research on Evaluation, Standards, and Student Testing (CRESST) at the University of California-Los Angeles found that nearly half of state testing programs either had performance assessments already in place, were planning to implement them, or were exploring their use (O'Neil, 1992). During 1990-91 the State of Vermont piloted the nation's first statewide portfolio assessment, which focused on mathematics and writing in Grades 4 and 8. The Vermont project was designed by teachers and fully implemented in 1991-92. It involves teachers in reviewing the actual classwork of students. It is important to note that major emphasis is being given to staff development and building resources that support portfolio assessment implementation efforts (Vermont State Department of Education, 1990).

In Connecticut, the State Department of Education has developed the Connecticut Common Core of Learning Assessment Project in Science and Mathematics to design performance-based assessment tasks for high school students that can be used by both teachers and policy makers to determine what students know and can do. This is a low-stakes project (the results are not being used for student promotion or graduation) which allows Connecticut educators time to examine their curricular, instructional, and assessment strategies and bring them into closer alignment with the new vision of science and mathematics education (Baron, 1992b).

In Kentucky, a state school reform mandate calls for the implementation of the nation's first completely performance-based statewide assessment system by 1995. The system, which has already begun to be implemented, will rely heavily on teacher assessment of student performance at Grades 4, 8, and 12. The statewide assessment system will have three components: an accountability assessment and two voluntary assessments-formal and informal-of student progress in classrooms. The state calls for teachers and other school staff to collect student performance task results in an "accountability portfolio" (Rothman, 1991).

SUMMARY AND CONCLUSION

School-based student assessment procedures are beginning to change to keep pace with shifts in instructional paradigms. As instruction for language minority students has moved in the direction of teaching language and content-area skills in context, assessment has begun to incorporate a wider variety of measures that more closely reflect the types of tasks that students are asked to perform in classrooms or in real-life settings. In this sense, school-based assessment procedures are becoming more authentic. Changes in assessment are especially important for language minority students and those acquiring English as their second language.

In addition to shifts in school-based assessment practices, state and national education initiatives are moving toward higher standards of student performance and school accountability. Whether or not students and schools are prepared to meet standards will depend upon the nature of the standards, whether students are being assisted in meeting them, and what instruments are used to measure student progress. Clearly, these shifts toward higher performance standards will have an impact on language minority students.

To be able to effectively monitor the progress of language minority students, assessment needs to be conducted on an ongoing basis with procedures that promise to yield the most useful information for classroom instruction. While standardized achievement tests cannot provide this type of information, performance-based assessment can. Performance tasks and instruments must be carefully designed, administered, and scored by teachers or assessment teams who have been trained on the assessment procedures used. In addition, performance criteria need to be set and raters trained on appropriate scoring procedures to ensure the reliability and validity of results.

Portfolio assessment has the potential for becoming an effective management system for performance assessment results. It represents a focused assessment of learning goals or objectives in English, native language arts, and/or the content areas. It can be conducted on a classroom, school-wide, or district-wide basis. At all levels, specific steps are recommended for implementing portfolio assessment, including: identifying the purpose and focus of the assessment, designing the portfolio, collecting data, analyzing portfolio contents, and using analysis of the results to make adaptations in instruction.

In this publication, we have addressed how performance and portfolio assessment can be used to monitor the classroom progress of upper elementary and middle school students. Portfolio assessment has the potential for being used in other ways, as well, such as:

(1) For students who are moving from one teacher or school to another, portfolios can be used to pass along critical information on their strengths and needs so that the new teacher does not have to duplicate assessments which have already been conducted;

(2) For students who are being considered for placement at different levels within an ESL or bilingual education program, portfolio results can be used to determine their ability to function at various levels;

(3) For students who are being considered for transition from an ESL or bilingual education program to a mainstream, English-only program, portfolio results can be used to measure performance relative to grademates in the mainstream; and

(4) For students who are being considered for prereferral to special education programs, portfolio results can be used to determine whether performance is related to language proficiency, including both native language and English literacy skills.

Portfolio assessment is currently being used on a limited basis and at various levels of implementation with language minority students and English language learners in school districts and states throughout the country. A large number of teachers and staff, while valuing the nature of the information generated by student portfolios, nevertheless feel that the constraints placed on their time and resources by portfolio assessment preclude its use on a regular or expanded basis. We have discussed some of the concerns expressed by practitioners and have made a few suggestions for overcoming them. Key to addressing concerns about portfolios are an adequate program of staff development and opportunities for teacher collaboration in planning and collecting student performance samples. We hope that teachers and school staff continue to share with us their successes and failures in implementing portfolios as we move toward gathering more information on this innovative assessment approach.

ABOUT THE AUTHORS

Lorraine Valdez Pierce is a Research Associate for the Center for Applied Linguistics and Coordinator of Publications for NCBE. She has been Senior Evaluation Specialist at the Georgetown University Evaluation Assistance Center (EAC)-East, Director of National Origin Desegregation Programs at American University's Mid-Atlantic Equity Center, and an ESL teacher of adolescents and adults for over a decade. Her latest publication is Effective Schools for National Origin Language Minority Students.

J. Michael O'Malley is Supervisor of Testing and Accreditation at Prince William County Public Schools in Manassas, Virginia. He was Director of the Georgetown University Evaluation Assistance Center (EAC)-East from 1985 to 1991 and has served as the Principal Investigator for Georgetown University's Language Research Projects on language learning strategies. His latest publication (with A. U. Chamot) is Learning Strategies in Second Language Acquisition.