Report # 2 on Theme I, Issue No. 5: Student Outcomes Assessment at CSUSB, 1993-2003

Status Report on Outcomes Assessment at CSUSB, 1993-2003

What is outcomes assessment? The American Association for Higher Education (AAHE) describes outcomes assessment as:

Outcomes Assessment examines the quality and effectiveness of academic programs examination of student learning. This report is an overview of our undergraduate, graduate and General Education Outcomes Assessment Program since 1994. The guiding principles of CSUSB in the development of outcomes assessment is that outcomes assessment should be practical and useful, should be cost effective and sustainable, respecting Faculty workload. Outcomes Assessment must help academic departments and programs

The CSU System's Commitment to Assessment

Initiated by the California legislature and the CSU trustees, the CSU began its involvement in Outcomes Assessment in 1985 with the attendance of CSU representatives at a national Conference on outcomes assessment. In 1987, the CSU held a system-wide faculty-administrator workshop on outcomes assessment and the CSU Academic Senate appointed an assessment specialist and established a system-wide advisory committee to the Chancellor on student outcomes assessment. The Chancellor's office also consulted with the faculty, students, and the administration, held subject area workshops on assessment, panel discussions, and regional meetings, and provided exploratory grants for assessment pilot projects. In 1989, the Chancellor's Advisory Committee On Outcomes Assessment submitted its final report and recommendations for CSU policy on student learning outcomes assessment. In 1990, the CSU Academic Senate endorsed the recommendations of the advisory committee. (The Chancellor's Advisory Committee's report on CSU Trustee's Action on Student Learning Outcomes can be found at: http://www.calstate.edu/AcadAff/Sloa/archives/trustee_action.shtml)

CSUSB's Commitment to Assessment

In 1993 CSUSB began active involvement in Outcomes Assessment with the appointment of the Task Force on Quality Assurance appointed by the Vice President Of Academic Affairs. The committee's charge was to examine our current efforts in quality assurance and to develop a plan for campus-wide involvement in outcomes assessment activities. The overall goal was to insure the quality and effectiveness of our programs. This committee studied the current level of assessment activities on campus, reviewed the growing literature on outcomes assessment, and developed a plan to phase in an assessment program. The 1994 taskforce's report can be found at our website at: http://gradstudies.csusb.edu/outcome/report.html)

In 1994, the Task Force on Quality Assurance presented its report to the CSUSB Faculty Senate, which formally resolved: "to proceed in developing concepts of outcomes assessment as an important step for providing the effectiveness of our academic programs" (FSD 94-01). In its resolution, the Faculty Senate cited a number of factors influencing its decision to accept the Task Force's report. These factors included (1) the call by various national reports and public policy organizations for greater emphasis on the effectiveness of higher education, (2) the plan by the California Legislature, Post-Secondary Education Commissions and CSU Trustees that institutions of higher education should be more accountable and provide evidence of their effectiveness, (3) the call by the Western Association of Schools and Colleges (WASC), as well as many specialized, national accrediting bodies in the professions, for clear goals and objectives, and proof of the effectiveness of all areas of the university including academic programs and general education; (4) CSUSB's final accreditation report from the WASC Commission on Senior Colleges that identified assessment as a significant weakness of the campus and recommended a coordinated effort in this area; and (5) the Senate's acknowledgement that a strong system of planning and budgeting require a data collection and evaluation component. (See 1994 Faculty Senate Resolution on Assessment at: http://gradstudies.csusb.edu/outcome/resolution.html)

Guided by the 1994 Faculty Senate Resolution Concerning Outcomes Assessment, the campus joined a nationwide movement to develop systematic processes focused on assessing student learning, processes that require an institution to develop a culture of evidence, whereby in a planned and careful way they ask, "Are students learning what they should be learning? Are they able to apply specialize knowledge and skills in the workplace, or as they seek higher education? Are students able to communicate well and solve problems? How do we know these things? What evidence do we have and what methods do we use to find out whether or not our students know what we want them to know?" By asking and seeking answers to these questions, the campus addressed the call by the California legislature, the CSU system, and the WASC accreditation standards for evidence of the effectiveness and outcomes of academic programs and the institution as a whole.

The campus further solidified its commitment to assessing student learning when it developed and approved its strategic plan in 1998. In order to achieve Goal One, "Become a learning community that excels in creating, applying and exchanging knowledge," the university has committed itself to "Improve student learning through assessment of both programs and student learning." See: Strategic Plan 1998-99 http://www.csusb.edu/president/strategicplan.html

Assessment Infrastructure at CSUSB

Outcomes Assessment is supported at the highest levels of both the CSU system and the University, which have formally affirmed their commitment to the concept of accountability in student learning outcomes.

The assessment program is built into CSUSB's administrative and academic structure, and is a multipartite, collaborative structure designed to:

Guided by the Faculty Senate Resolution Concerning Outcomes Assessment (FSD 9401), the development of an outcomes assessment infrastructure at CSUSB began in 1994 with the formation of the University Outcomes Assessment Committee (UOAC). The UOAC is a Faculty Senate-appointed committee made up of one representative from each of the five colleges (Arts & Letters, Business & Public Administration, Education, Social & Behavioral Sciences, and Natural Sciences) and the five College Outcomes Assessment Coordinators. Additional members include the AVP for Assessment, Dean of Undergraduate Studies, Assoc. Provost for Academic Programs, and the Director of Institutional Research. The committee's responsibilities include reviewing assessment plans and providing guidance and feedback to departments and programs. It also reviews and approves funding requests from departments for outcomes assessment activities.

Since 1994, the assessment infrastructure at CSUSB has expanded to include a large segment of the university. As more and more departments developed and implemented their assessment plans, the administration, recognizing the need for central coordination, appointed an Associate Vice President for Assessment and Planning in 1999. The AVP oversees and coordinates the assessment process for the campus as a whole. Under the guidance and supervision of the AVP for Assessment and Planning, the Academic units plan, implement, and annually report the assessment of their own programs.

All components of the assessment infrastructure are designed to insure that assessment at CSUSB is faculty-driven and collaborative and provides useful information for ongoing program improvement. Each of the five colleges, Arts and Letters, Business and Public Administration, Education, Natural Sciences, and Social and Behavioral Sciences has an Assessment Coordinator who works with the departmental assessment coordinators, the UOAC, and the AVP for Assessment and Planning. College Assessment Coordinators serve on the UOAC. They also work with the Assessment AVP, and the department committees and provide feedback and guidance to department committees.

In addition to the five College Assessment Coordinators, each academic department and program, depending on its size, has either a Department Outcomes Assessment Coordinator or a Department Outcomes Assessment Committee. Department Coordinators, usually one or two faculty representatives from each department and graduate program, collaborate with the AVP for Assessment, College Coordinators, and the department committees in the development of their outcomes assessment plans. Department Assessment Committees work collaboratively with their department members, the Department and College Coordinators, and the AVP for Assessment to develop outcomes assessment strategies for their departments.

The university began discussions on the development of an assessment plan for the General Education Program in fall 1995. The UOAC formed a Sub-Committee on Assessment of General Education in winter 1997. In fall 1999, the UOAC established formal sub-committees to develop and implement assessment methodology for general education. These committees lead the assessment of general education, under the guidance of the AVP for Assessment and Planning. The University General Education Outcomes Assessment Committee is charged with working with the faculty to develop strategies for the assessment of general education at CSUSB and implement a sustainable assessment program. Its primary focus has been the development of assessment methodologies for the four general education basic skills areas of written communication, oral communication, critical thinking, and mathematics. The committee is composed of one representative from each of the colleges, the Chair of the Univ. General Education Committee, the AVP for Assessment, the Dean of Undergraduate Studies, the Assoc. Provost for Academic Programs, and the Director of Institutional Research.

There are three College General Education Outcomes Assessment Committees, one each from the colleges that offer general education courses, Arts and Letters, Social and Behavioral Sciences, and Natural Sciences. These committees are charged with working with the faculty to develop strategies for the assessment of general education breadth areas in Humanities, Social and Behavioral Sciences, and Natural Sciences at CSUSB and implement a sustainable assessment program. They are composed of faculty appointed by the dean of the college. The AVP for Assessment and a representative from the University General Education Outcomes Assessment committee serve as consultants to the committees.

Besides committees devoted to the assessment of general education and the academic departments, CSUSB has also developed specialized committees as needed to focus on special issues and assist programs in the development of their assessment methodologies. These programs are often large, multi-concentration departments, sometimes involving thousands of students. For example, in winter quarter 2001, we formed an MBA Outcomes Assessment Committee to work with the faculty to develop an assessment plan. The plan was completed, approved, and implemented in winter 2002. In fall 2001, we formed an outcomes assessment committee to develop outcomes assessment for the Liberal Studies Program, which involves more than a dozen departments and has two thousand majors. (A full outline of CSUSB's assessment infrastructure can be found at: http://gradstudies.csusb.edu/outcome/infrastructure.html )

Funding for Outcomes Assessment

The administration at CSUSB has been devoted to the full support of assessment. Funds for outcomes assessment are provided by the Provost, and VP for Academic Affairs and administered by the AVP for Assessment and the University Outcomes Assessment Committee. At the present time, there is no set amount of money designated for outcomes assessment. Instead, funds are allocated as needed to enhance the development of an assessment infrastructure designed to:

(For a detailed description of the funding available to support faculty and programs, the application for funding for assessment activities, and the criteria for evaluation of funding requests, please see: http://gradstudies.csusb.edu/outcome/funding.html )

Format and Approval Process for Assessment Plans

In 1994, academic departments and programs began the development of formal statements of their knowledge and skill goals for the graduate and undergraduate majors. They also developed their initial assessment plans. In that same year, the faculty senate established the University Outcomes Assessment Committee (UOAC) whose charge was to review, provide feedback and university level approval of programs' assessment plans. In spring 1995, the UOAC began the development of guidelines, format and approval criteria to assist programs in developing their assessment plans, and in fall 1995, established formal procedures and criteria for review of programs' mission, goals, and objectives. All program assessment plans must include the following components:

The required format for assessment plans was developed to achieve several primary purposes. The first goal was to achieve university-wide consistency based on the assumption that the assessment plans are public documents, available for viewing not just by faculty and students across the university, but also for a wide array of stakeholders. Second, the UOAC found that the format specified has an inherent logic and pattern that guides the program in developing a clear, organized, and concrete set of assessment measures. Third, while the format establishes a uniform structure, it also allows programs the flexibility to devise their own mission statements, learning goals, objectives, and methods of assessment.

Programs do not develop their assessment plans in isolation. All programs have designated assessment coordinators or an assessment committee comprised of faculty in the department who consult with all faculty in the department as the plan is developed. At CSUSB, the development of assessment plans is faculty-driven and collaborative with guidance and funding available from the college assessment coordinators, the UOAC and the AVP for Assessment and Planning.

The UOAC's approval of assessment plans for implementation is carried out in the same spirit of collaboration as the development of the plans. At CSUSB, we have put in place an approval process designed to provide support and guidance to programs at every stage of development. In the development process, programs are encouraged to focus first on articulating their mission statements and learning goals and objectives. The UOAC evaluates the mission statements and learning goals and objectives according to the following criteria:

After evaluating the mission, goals and objectives according to the criteria, the UOAC informs the program that either it may move forward and develop the criteria, methods and implementation structure or requests revision and resubmission and provides it with written guidance as to how to strengthen and complete the plan. As more and more programs have had their assessment plans approved, providing models for other programs still engaged in the development process, in recent years, programs have begun to submit complete plans for review. The UOAC evaluates these assessment plans according to the following criteria:

The UOAC provides written comments on the assessment plan and suggestions for improvement or clarification, and rates the plan either (a) acceptable as is, ready to carry out assessments, or (b) needs minor revision, or (c)) is unacceptable in its present form and requires major revision. (The format and criteria for reviewing and approving assessment plans is located on our assessment website under Guidelines Formats and Criteria for Review of Assessment Plans at: http://gradstudies.csusb.edu/outcome/guidelines.html)

Since 1994, CSUSB has been working to ensure that programs have clear academic standards for student achievement in place. As of Spring 2003, 83% of the undergraduate and graduate programs at CSUSB (excluding Interdisciplinary programs) have submitted outcomes assessment plans to the University Outcomes Assessment Committee. Of those submitted, 69% have been approved for implementation by the committee. Overall, 58% of the undergraduate and graduate programs at CSUSB (excluding Interdisciplinary programs) have university-approved outcomes assessment plans. (Approved assessment plans are available upon request in the office of the AVP for Assessment and Planning and may be viewed by clicking on individual program names on CSUSB's outcomes assessment website at: Assessment Plans by College and Program: http://gradstudies.csusb.edu/outcome/bycollege.html)

What does the assessment plans tell us about outcomes assessment at CSUSB?

Mission Statements

Assessment at CSUSB is tied to the University mission. The University mission statement states that:

Paralleling that of the University, many programs have developed mission statements that express similar values and aspirations. The Department of Music states that in addition to providing opportunities and instruction for music majors to develop their intellectual and artistic musical abilities, the department seeks also to provide students the "opportunities to gain professional experience through public performance in ensembles and solo recitals that, in turn, contribute to the cultural environment of the campus and community." Similarly, The mission of the Master of Social Work Program is "to make graduate training available and accessible within the Inland Empire …(and) provide agency and community leadership in social service institutions, communities, and political arenas." The Geography Department provides "a curriculum, faculty, learning environment and community relationships, which foster curiosity, scholarship, enhanced teaching and learning, and service to the region."

Another aspect of the university mission is to provide a sound program of general education in the arts and sciences for all undergraduates. A number of programs identify the same goal. In addition to its goal to prepare geology majors for advanced studies in the geological sciences or related disciplines, the Geology Department also has a mission "to educate non-majors to make informed decisions on matters pertaining to interactions between geology and society." The mission of the Anthropology Department is to serve both majors and non-majors: "the curriculum for majors emphasizes the theories, methods, concepts, ethics, and findings of anthropology and is designed to prepare students for careers that can take advantage of anthropological perspectives or for graduate education in anthropology (and) the curriculum for non-majors includes anthropology courses that are part of the requirements of other majors and general education courses, and these courses expose students to anthropological perspectives through the examination of particular topics; this exposure is intended to promote tolerance and understanding while broadening students' horizons."

To foster excellence and anticipate the needs of our service region, the university has identified six programmatic areas beyond its traditional liberal arts curricula for continued emphasis and expansion. One of these areas is the preparation of teachers to serve a multicultural society. The M. A. in Education, Teaching English as a Second Language Option program has as its overall mission "the improvement the teaching of English to speakers of other languages, with respect for the integrity and maintenance of the diverse languages and cultures of the learners." The master's program in Educational Administration seeks to provide "academically rigorous instruction in traditional, contemporary, and emerging educational administration theory, research and practice to prepare students to be effective leaders in private and public education …and) serve students by recognizing the diverse populations they represent, responding to their needs and involving them and practitioners in the decision-making process for the program."

Another one of the six programmatic areas is to prepare "health and helping professions to support the burgeoning population in our service area." The primary mission of the Department of Nursing is the liberal and professional education of students to prepare them for careers of leadership and service in nursing. Reflecting the university's service region, the nursing department prepares its Baccalaureate graduates to provide "care to men and women of all ages and various socioeconomic and cultural backgrounds."

Another one of the six programmatic areas is to offer "applied science and engineering programs to support technology development in southern California." The Master of Science in Computer Science program is designed to help students develop "analytical, problem solving and practical skills in computer science through advanced coursework, directed software development and supervised research and independent studies. Students should acquire sufficient and requisite knowledge in the discipline as well as critical thinking, oral and written communication skills to qualify them for highly technical positions in the industry and to enable them to be accepted in a doctoral program in computer science. Students experience software development on equipment, operating systems, networks and in languages, that are currently in use in industry."

Another of the six programmatic areas is to offer "management and administration programs to graduate future business and public agency leaders." The mission of the Master in Business Administration program is to provide post-baccalaureate students with a high quality master's level education in the field of business administration. The program is designed to prepare promising students for positions of increasing responsibility and leadership through education in the broad scope of business and through in-depth knowledge in one or more specialized areas of business. Likewise, the Master in Public Administration is a broad program designed to prepare students for careers in administration in the public sector. The major goal is to provide students with a generalist perspective of public administration, including tools of decision-making, an understanding of the total public administrative system, and a capacity for working within the governmental and nonprofit sector environments. The MPA degree is intended for graduate students and career employees seeking to develop and strengthen academic and professional skills for admission and advancement in the professional practice of public administration.

Goals and Objectives

CSUSB's outcomes assessment plans contain goals and objectives that address a variety of learning outcomes. Since 1994, faculty in the academic departments and programs worked collaboratively to develop formal statements of the cognitive, affective and skill learning goals and objectives expected of their majors, accompanied by descriptions of the methodology and criteria they will use to determine whether their students are achieving these learning goals. The most effective statements of learning goals and objectives are those that employ action verbs, identify, and describe what students will learn rather than identifying instructor behavior (Palomba and Banta, 36). Many departments have used as a guide in the development of their objectives, Bloom's taxonomy of cognitive objectives containing six levels of thinking skills, arranged in order of complexity. The first three levels in order of difficulty are knowledge, comprehension, and application, followed by the "higher order" thinking skills of analysis, synthesis, and evaluation.
Bloom's Taxonomy (http://faculty.washington.edu/krumme/guides/bloom.html)

Departments and programs across the university require their graduates to possess a variety of cognitive behaviors. The master's in National Security Studies Program includes among its learning objectives for its graduates the requirements that be able to explain (comprehension) the major security features of a region, in either Eurasia or the Middle East, compare and contrast (analysis) the theories of international relations that pertain to security studies, and evaluate major security policy challenges of the United States in the world today.

The Department of Biology faculty expects students who complete the bachelors degree to be able to demonstrate (application) an understanding of cell structure, cell physiology and the molecular processes of cells; perform appropriate statistical analysis of experimental data and draw valid conclusions from their analysis; and critically evaluate a journal article from the primary literature. By requiring its graduates to demonstrate the proper laboratory practice through the use of equipment and the ability to use basic and advanced techniques in several areas of biology, the biology department also expects a number of skill-based behaviors of its graduates, including the use of sensory cues (perception) to guide actions, guided response (the knowledge of the steps required to complete a task), and adaptation (the ability to perform a task and also modify actions to account for new or problematic situations).

Faculty in the Department of Art expect their graduates to display a range of cognitive, and skill learning behaviors. They expect their graduates to demonstrate cognitive behaviors, ranging from knowledge to the higher order thinking skills of analysis and evaluation. Art department faculty expect their graduates to know the works and intentions of major artists/designers and movements of the past and present, both in Western and non-Western worlds, discuss a major art work and analyze it using appropriate language for the media discussed and evaluate the issues regarding ethical standards, for example, gender issues, censorship the use of human subjects, plagiarism, copyright infringement. The skill based-goals the department requires of its graduates include mechanism, the ability to perform a task or objective in a somewhat confident, proficient, and habitual manner; organization, the ability to create new tasks or objectives, incorporating learned ones. The department expects its students to learn how to see and interpret the world as an artist and use their ideas and the tools of their discipline to create art or design product.

A number of programs have recognized the importance of its graduates to possess the ability to communicate in writing and orally. These departments have set as learning outcomes for their graduates, the effective use of verbal communication skills, the ability to document data effectively, clear communication through scholarly papers and use of communication technology. For example, the Department of English expects its graduates to be able to analyze, interpret, and compare literary works, and to write about literature in a clear, coherent, literate way that demonstrates a high level of understanding both of a text's technical merits and of its emotional impact. The faculty in the Department Of Philosophy expects its graduates to be able to write a coherent essay, demonstrating the ability to effectively analyze, assess, and construct rigorous arguments for and/or against philosophical positions and reasoning. Upon graduation, students in the Department of Chemistry will know how to use computer technology to learn, gather, display, and analyze chemical information; communicate scientific information effectively through written reports and oral presentations. Other departments that have stated similar learning goals for its graduates are the Departments of Spanish and French, Department of Political Science, Departments of Management and Marketing, the MA in Mathematics, and the Designated Subjects Credential in College of Education.

Methodology

At CSUSB each department selects its own assessment tools to determine whether or not its graduates are meeting its learning objectives. In the selection and development of their assessment methodology, programs are expected to measure their students' achievement by using multiple measures that include both direct and indirect measures, tailored to the discipline/program. The techniques employed to measure student achievement at CSUSB are broad and varied, illustrating the fact that there is no one "best" or most appropriate model.

At CSUSB, portfolio evaluation is the most common direct indicator of student learning utilized by programs, followed by, in order of most frequent use, locally developed pre-tests or post-tests for mastery of knowledge, the senior thesis or major project, and capstone courses designed to measure student mastery of the discipline.

The most commonly used indirect indicators of student learning are student satisfaction surveys, alumni surveys reporting satisfaction with the degree program and career success, exit interviews, and internship evaluations. (The types of direct and indirect indicators used by individual departments can be viewed at our assessment website at: Types of Assessment at CSUSB http://gradstudies.csusb.edu/outcome/types.html and Assessment Methods at: http://gradstudies.csusb.edu/outcome/assessment_methods.html)

Annual Assessment Status Reports from academic departments and programs

The annual status reports on outcomes assessment activities are integral to CSUSB's commitment to continuous program improvement and student learning. Each program is required to submit an update on its outcomes assessment activities for the prior academic year. A report outlining the assessment findings, results, and implementation activities is due each fall quarter. Programs whose plans have not yet been approved by the University Outcomes Assessment Committee submit a report detailing its current status, e.g. the state of development of the assessment plan and what remains to complete it.

In the annual report, programs are asked to describe their assessment activities for the year. Based on their assessment activities, they describe their assessment results, i.e. what they found out about their students' learning based on the results from their assessment activities, and in the context of their goals and objectives. We do not require the submission of the raw data; we want to know the important findings and how they are being used for program revision/improvement. Next, programs are asked to explain the changes they have made in their program or are in the process of making based on what they have learned about their program and students from their assessment activities. Finally, they are asked to provide a time-line and implementation strategies for both a one-year and five-year departmental commitment. (See Guidelines and Format for Submission of Annual Status Reports at: http://gradstudies.csusb.edu/outcome/guide_status.html)

Programs are asked to prepare their reports according to the following criteria. Their reports should be presented in an organized, detailed manner geared toward informing an audience comprised of colleagues in your program and faculty across the campus. An assessment report should present results in a way that is useful to the audience who will read it, and should examine results as well as the assessment process, itself. The report should clarify what was done, why it was done, and what was found. Program strengths and weaknesses should be identified in a balanced way, and the focus should be on program improvement, not on identifying scapegoats. Annual status reports are available upon request in the office of the AVP for Assessment and Planning and may be viewed on CSUSB's outcomes assessment website. (See Annual Status Reports by College and Program at: http://gradstudies.csusb.edu/outcome/statusrpt.html) (Use mouse to highlight and click on individual reports shown on the chart.)

Results: What we have learned about student learning from Outcomes Assessment?

Overview

The annual status reports have begun to provide us with specific information on student learning outcomes programs and how departments are using what they have learned for program improvement. The majority of departments with approved assessment plans use assessment information for curricular purposes, to (1) facilitate curriculum discussions at faculty meetings, curriculum committee meetings, and faculty retreats, (2) revise course content, (2) add and/or delete courses, (4) change course sequences, (5) revise degree requirements or degree sheet options, and (6) guide changes in degree programs and the development of new degree program options. Information gained from their outcomes assessment activities have led some departments to make changes in services they provide for their students. Specifically, they have modified their advising processes, and developed new career exploration and career services for students. Based on information gained from their outcomes assessment activities, departments have made changes in the instructional emphasis for current faculty, and in the emphasis for new or vacant faculty positions. Reflecting their awareness of the campus' commitment as stated in its mission statement and strategic plan that we will connect with community, some departments have shared their assessment information with alumni and industrial review boards. Over the course of CSUSB's decade-long involvement in Outcomes Assessment, departments have begun to use their assessment findings to justify past curriculum changes and show program improvement resulting from those changes. The annual status reports reveal that outcomes assessment at CSUSB is ongoing and evolving: departments routinely use assessment information to further refine their assessment methodology or to implement new assessment methods.

Closing the Feedback Loop: The use of outcomes assessment findings to improve curriculum

Outcomes Assessment at CSUSB has helped departments by providing them with data that confirms the strengths in their curriculum and also has helped programs identify changes that might be needed. About 54% of the academic programs are doing serious outcomes assessment and have identified areas for improvement. Almost all of these departments have either acted on this data and as a result made changes in their curriculum or teaching practice, or have planned future changes. Below are snap-shot descriptions of curricular changes made by some departments as a result of their outcomes assessment findings.

Department of English

The English Department's status reports from 1997-2002 not only highlight the curricular changes they have made as a result of their outcomes assessment data, but also demonstrate the value of longitudinal information gained from assessment in shaping those changes. The English department expects its graduates to be able to "know that literature can be studied in a variety of ways, and to be familiar with some of these critical approaches." In 1997, as a result of their evaluation of their seniors' portfolios, the department observed that students were not meeting this learning goal according to its expectations. As a result, the members of the department began ongoing discussions about introducing explicit instruction in critical approaches in its courses and also considered the possibility of recruiting a specialist in critical theory.

In 1998, the faculty decided to include a wider variety of critical approaches in its analysis and survey courses, and senior seminars. The department's review of senior portfolios in 1999 showed that more students were familiar with critical approaches, but student performance in this area still did not meet department expectations. After forming an ad hoc group to look into the issue the English, the department added English 385-- Literary Theory and Criticism--as a course requirement in 2000. This year 2002, the students' portfolios show that English majors are much more adept in describing their understanding of literary theories than students were several years ago, but there remains considerable variation in this understanding. While the outcomes assessment results corroborate the English department's decision to require English 385, faculty are watching carefully to see the effects of this requirement on students' understanding of literary criticism.

Department of Communication Studies

In its 2001/2002 annual report, the Communication Department explained how two of its assessment tools, the senior portfolio, and survey of internship evaluations guided them in making curriculum decisions. The faculty processes and discusses assessment data at its annual retreat in September. One of the Department's expectations is that its graduates will "apply theoretical/critical communication perspectives in everyday life." Their analysis confirmed their belief that students were meeting this goal successfully, according to the department's criteria. Their assessment also sensitized them to the fact, that while their students are able to apply theoretic principles to demands arising in their personal and professional lives, they were slightly stronger in their mastery of the functions of theory and its relationship to methodology.

Their assessment also enabled them to target some areas they wanted to change. Based on their assessment of student portfolios, faculty decided that they wanted their students to be more familiar with the program's learning goals and objectives, and elected to attach copies of the goals and objectives to syllabi for each course in their program, explain to students how the particular course fits into those standards, and engage in a careful reexamination of course assignments in terms of their "goodness-of-fit" with the goals and objectives for the program. Based on information gained from supervisors' evaluations of interns, they have decided to monitor the correlation between the internships and their expectations for students' achievement in their classes and the quality of the feedback they give them on their performance in those classes.

General Education Outcomes Assessment

GE Outcomes Assessment Infrastructure

Currently, the GE outcomes assessment infrastructure consists of four committees comprised of faculty and administrative representatives. The University General Education Outcomes Assessment Committee, formed in fall 1999, is charged with working with the faculty to develop strategies for the assessment of general education at CSUSB and implement a sustainable assessment program for basic skills. The committee is made up of one representative from each of the colleges; the chair of the University GE Committee; AVP for Assessment; Dean of Undergraduate Studies, Assoc. Provost for Academic Programs, and the Director of Institutional Research.

There are three General Education Breadth Area Outcomes Assessment Committees, formed in June 2001. The committees, one each from the colleges of Arts and Letters, Social and Behavioral Sciences, and Natural Sciences, are composed of faculty appointed by the Dean of the college. The AVP for Assessment and a representative from the University GE Outcomes Assessment committee serve as consultants to the committees. These committees are charged with working with the faculty to develop strategies for the assessment of the general education breadth areas in the Humanities, Social and Behavioral Sciences, and Natural Sciences at CSUSB and implement a sustainable assessment program. The AVP for Assessment meets and consults with the University General Education Committee regarding the development of assessment for the GE program.

Funding for GE Outcomes Assessment

Paralleling its commitment to funding assessment for the majors, the administration at CSUSB has been devoted to the full support of GE assessment. Funds for GE outcomes assessment are provided by the Provost, and VP for Academic Affairs and administered by the AVP for Assessment. At the present time, there is no set amount of money designated for outcomes assessment. Instead, funds are allocated as needed to enhance the development of a GE assessment infrastructure. The administration has provided funds to support

Development of Learning Goals and Objectives for GE Outcomes Assessment

Since 1999, CSUSB has made a number of major steps in the development of assessment for the General Education Program. In fall 1999, the UOAC formed a sub-committee to develop and implement assessment methodology for general education. This committee is titled the University General Education Outcomes Assessment Committee. In winter 2000, CSUSB a faculty team attended the system-wide General Education Assessment Conference held at CSU Fullerton, and the AVP for Assessment and Planning conducted an outcomes assessment workshop for the faculty of the College of the Education.

In spring 2000, the University General Education Outcomes Assessment Committee began the development and preliminary distribution of learning goals and objectives for the general education basic skills areas in written communication, oral communication, mathematics, and critical thinking. The committee consulted with department chairs, the University General Education Committee, university and college curriculum Committees, College Outcomes Assessment Coordinators, and faculty who teach general education courses. They carefully aligned them with the University General Education Policy. (See http://gradstudies.csusb.edu/outcome/Alignment.html.) The AVP for Assessment and Planning presented the committee's report to the Faculty Senate Executive Committee.

In fall 2000, the University General Education Outcomes Assessment Committee began university-wide distribution of draft learning goals and objectives for the general education basic skills areas to the faculty for feedback and approval. At the end of fall 2000, faculty approved the learning goals and objectives for the four General Education Basic Skills areas (Written Communication, Oral Communication, Mathematics, and Critical Thinking). (See http://gradstudies.csusb.edu/outcome/BasicSkills.html.) The committee researched and selected a commercial assessment test to pilot. During this same quarter, the committee also proposed a committee structure to develop goals and objectives for the GE Breadth areas in Arts and Letters, Natural Sciences, and Social and Behavioral Sciences.

In spring 2001, the college deans appointed faculty to serve on the three General Education Breadth area committees, which begin drafting learning goals and objectives for the breadth areas in Arts and Letters, Natural Sciences, and Social and Behavioral Sciences. That summer, the General Education Breadth Area Committees continued drafting learning goals and objectives for the breadth areas. In fall 2001, the General Education Breadth Area Committees began university-wide distribution of draft goals and objectives for faculty review and approval. (See http://gradstudies.csusb.edu/outcome/BreadthArea.html.) They carefully aligned them with the University General Education Policy. (See http://gradstudies.csusb.edu/outcome/Alignment.html.)

In winter, 2002 The General Education Breadth Area Committees researched and began the development of assessment tools, and members of the committees attended the General Education Embedded Assessment Conference at CSU Fullerton.

In spring and summer 2002, the General Education Breadth Area Committees consulted with faculty teaching the breadth area courses and began developing embedded assessment tools to be piloted in GE courses. From fall 2002 through spring 2003, the GE breadth area committees piloted locally-developed embedded assessment questions in the final exams in general education courses in Arts and Letters, Natural Sciences, and Social and Behavioral Sciences.

GE Outcomes Assessment Pilot Projects

Pilot Project in Basic Skills

During the 2000-2001 and 2002-2002 AY, CSUSB began the assessment of the basic skills areas by administering the Academic Profile (ETS). We selected the AP from among other national tests, because it tests general academic knowledge and skills, and includes material usually covered in courses taken during the first two years of college, the "general education requirements." The exam, which yields criterion-referenced and norm-referenced scores, focuses on skills developed in introductory courses in the humanities, social sciences, and natural sciences. It concentrates on issues, themes, and ideas and measures three clearly defined factors, writing, mathematics, and reading/critical thinking. Questions do not ask for recall of specific information but instead, test a student's ability to read carefully, make judgments about clarity, correctness, or organization of material, think critically about issues and arguments, and work effectively with mathematics. No specific courses or course patterns are assumed.

We administered the Academic Profile to more than 1,200 students in eight of upper-division GE capstone classes, from each of the three breadth areas, Arts and Letters, Natural Sciences, and Social and Behavioral Sciences. We chose to pilot the AP in the upper-division GE capstone classes, because junior or senior status is required for enrollment in them, thus assuring that the majority of students would have completed most or all of their basic skills courses.

Our purpose in adopting the Academic Profile was twofold purpose: to provide us with evidence about what we are doing well in GE and at the same time assist us in improving the quality of our general education program and the student learning that results, so that CSUSB, as an institution can begin to move beyond mere anecdotal information about our students' academic achievement and construct a culture of evidence. However, several issues emerged from our use of this instrument. Because it did not focus on our curriculum, specifically our learning outcomes goals and objectives, the information we received from the Academic Profile was too general and less useful than we had hoped as an indicator of the effectiveness of our GE program. We were unable to align any of the data from the Academic Profile with our GE learning goals. This is a problem inherent in such surveys. In several classes, students completed the exam as an "extra credit" option, resulting in little incentive to perform well. In addition, our campus' scores were below the mean.

One major outgrowth of our experience with the Academic Profile was our realization that we need to develop local instruments that will more readily assess our particular constellation of courses. Therefore, in Fall 2002, the GE assessment committees developed and began piloting campus-based embedded assessment of student learning in the breadth areas. To date they have piloted questions in fifteen general education breadth area courses, involving nearly 2,000 students.

Pilot Project in the GE Breadth Areas

In fall, 2002 we began piloting essay and objective questions embedded in the final exams of courses in the GE breadth areas. Specifically, we have tested students in B2-Life Sciences and B3-Physical Sciences categories of the Natural Sciences breadth area, the C1-Arts category of the Humanities breadth area, and the D2-American Institutions, D3-World Cultures, D4-Discipline Perspectives, and D5-Integrative Capstone categories of the Social and Behavioral Sciences breadth area. In spring, 2003 we have scheduled additional pilots in the B2-Life Sciences, B4-Special Topics in Science and Technology, C2-Literature, C3-Foreign Literature and Literature in Translation, D1-American History and Civilization.

In spring 2003 the breadth area committees began their analysis of the results from the pilot project with particular focus on what student responses tell us about (1) the alignment between the goals and objectives and types of embedded exam questions, (2) how students are meeting the goals and objectives, (3) the usefulness of the data in closing the assessment loop and shaping curriculum, and (4) the feasibility of developing a faculty-supported sustainable assessment infrastructure for CSUSB's General Education program. We have learned that although our exam questions and learning goals parallel one another closely, we need to develop mechanisms to ensure consistent alignment, especially as we assess more and more courses. Our preliminary data suggests that our students' performance is adequate, ranging from 70 to 76 percent in correct responses to the embedded exam questions, and over time, additional findings will be valuable in helping us shape and improve our GE curriculum. The pilot has shown us that our biggest challenge lies in the development of a sustainable assessment infrastructure for CSUSB's General Education program. This challenge stems from two characteristics of our GE program: its size, students must complete 82 quarter units, and, even more problematic, the range and diversity of the program's more than 150 courses from which students may choose. The results of our pilot project suggest that a "one-size-fits-all" structure and the same methodology are not practicable for our program.

Besides our efforts to develop embedded assessment for the GE program, CSUSB engages in a number of other GE assessment activities, especially in the area of basic skills. We screen the math and writing skills of all our incoming freshmen (EPT and ELM exams), and require them to take remedial classes if their skills are below a set standard. Our upper-division writing requirement requires students to achieve proficiency in critical thinking, organizational skills, and the ability to use evidence. The upper-division writing requirement is also a means by which we assess the writing skills of students entering our master's programs.