3.4. Program Outcomes

This section will guide the program review team to develop/review the program outcomes to deliver key program competencies identified earlier in section 3.2.

After completing this section, the program review team may determine new or updated program level outcomes are needed.  Any changes to program outcomes (add new, amend, or delete current outcomes) should be documented in the recommendations section of the self-study report.

Definitions of Program Outcomes

A program outcome is “an intended effect of the program educational experience that has been stated in terms of specific, observable, and measurable student performance” (Veltri, Webb, Matveev & Zapatero, 2011). In other words, program outcomes are statements that describe key competencies, capabilities students can demonstrate at the end of the program.

Program outcomes essentially are the building blocks of the program and provide graduates with the terminology to describe the capabilities they can bring to a workplace or place of further education. What should your students know and be able to do by the time they graduate? In other words, program outcomes translate the competencies of the program into concrete expectations as outcomes.

Difference between Program Competencies and Program Outcomes

Program competencies and outcomes do sound almost the same. But there is a fundamental difference between them.  Here is the difference between a program competency and a program learning outcome.

Competency: A general statement that describes the desired knowledge, skills, and behaviors of a student graduating from a program (or completing a course). Competencies commonly define the applied skills and knowledge that enable people to successfully perform in professional, educational, and other life contexts” (Hartel and Foegeding, 2004).

Outcome: A very specific statement that describes exactly what a student will be able to do in some measurable way. There may be more than one measurable outcome defined for a given competency (Hartel and Foegeding, 2004).

Competencies may require multiple outcomes to achieve it. See example below.

Does your Program have Program Level Outcomes?

At this stage of the program review, the purpose is to evaluate whether the current program outcomes reflect and deliver the program competencies and other aspects of the program. Some KPU programs may not have defined program outcomes. Since program level outcomes acts as the glue that holds all course level outcomes together, it is paramount that you develop program outcomes before you proceed further. Answer the following questions and take the necessary step accordingly.

If your program does NOT have program outcomes, then go to section 3.4.1

If your program does HAVE program outcomes, then go to section 3.4.2

3.4.1. Developing Program Outcomes 

Content in this section will assist the program review team to develop program outcomes given the current program does not have any documented program level learning outcomes.

See below instructions to write program level outcomes

a) How to Write a Program Outcome Stem

Before presenting the list of program outcomes, a stem that refers to the outcomes needs to be developed. See below examples of different program outcome stems.

• Students will be expected to:
• Upon successful completion of the program, students will be able to:

b) General Strategies to Write Program Outcomes

Here are some general strategies to consider are

• Start with program competencies. The program outcomes developed for the program should translate all identified program competencies (section 2) into specific measurable statements.
• Refer to individual industry competencies and 21st century skills. In section 3.2.1, step 3, you clustered several industry competencies and 21st century skills into several program competency buckets. Go back and review these individual industry competencies and 21st century skills as you write program outcomes.
• Work from your program description. The program description is a summary of the outcomes the program intends to deliver.
• If you have course outcomes already written, adapt them into to more global statements. After all, your course outcomes are the delivery vehicle of the overall program outcomes.
• Use accreditation requirements of the program to guide program outcomes.
• Examine PLOs used for similar programs at other institutions

Source:  Dyjur, Grant, & Kalu, 2019. University of Calgary, Taylor Institute, Program-Level Learning Outcomes

c) How Many Program Outcomes?

A general guideline, 8 – 12 program outcomes are ideal for a program. In section 3.6, the program review team will map out the new/current program level outcomes to course level outcomes. If there are too few or too many program outcomes, the curriculum mapping process will become challenging in many ways.

d) Specific Steps when Writing and Aligning PLOs

Program outcomes should answer the following questions: a) what knowledge, skill or ability will the student have upon completion of the program; and b) how will the learner demonstrate this competency?

 STEP A: Identify the HOW of the Program Outcome

Select a present tense, active verb that most clearly describes what students must do to demonstrate successful attainment of the program  outcome. Avoid verbs that are vague or cannot be observed (e.g., “know,” “learn” or “understand”). Pick verbs that are specific.

Caution: Do not use BLOOM’S Taxonomy verbs to answer the how part of the program outcome.

Blooms taxonomy provides a comprehensive list of very specific action verbs that can be used to denote multiple levels of complexity of learning outcomes. For example, action verbs such as “list, define” will highlight less complex tasks to more complex tasks such as “design, produce”. Blooms taxonomy verbs are useful when writing course level outcomes but are too specific for program level outcomes. While program outcomes need to be specific, they should be global and provide a 360-degree overview of the program deliverables.

STEP B: Determine the WHAT of the Program Outcome

Consider the “what” of the program outcome. What skill set, knowledge or ability will students be expected to walk away upon the completion of program? What assets will they be developing?

To avoid overly vague or generalized program outcomes, acknowledge the frame of reference within which the program operates, or assessment will occur.            

Appendix 5: Examples of Program Outcomes

STEP C – Align Newly Developed Program Outcomes with Program Competencies

Having developed program outcomes, this step will assist the team to ensure that the newly developed program outcomes are aligned well with the program competencies developed/reviewed in section 3.2.

Refer to the example, template below as you follow these steps.

Appendix 6: Program Outcomes Alignment Template

Step 1 – In column A vertically, list program competencies you developed/updated in section 3.2

Step 2 – List your current or newly developed program outcomes horizontally in columns B to columns I

Step 3 – Now map program outcomes with program competencies by marking “X” in the corresponding row/column

If you identify any gaps in the matrix above (Notice in the above example, program outcome 5 & 8 are not representing any of the program competencies identified), go back and revise program outcomes as required.

3.4.2 Evaluating Current Program Outcomes 

Content in this section will assist the program review team to review the currency of the existing program outcomes and align them with program competencies that was updated in section 3.2.

a) Criteria to Evaluate the Currency of Program Outcomes

Revisiting the current program-level outcomes will provide the team an opportunity to review the currency of the program. Inevitably program competencies (section 3.2) will be updated due to changes in industry requirements and 21st century skills. Thus, these changes will have to be addressed in the program outcomes. Even though the current program outcomes may seem to accomplish the key program competencies set in the program, taking a second look at them would nevertheless help the team to gain new understanding about the changing nature of the program, shifts in perspectives in individual courses to what faculty is contributing collectively to the program (Dyjur, Grant & Kalu, 2019).

Here are some guiding questions for the program review team to consider when evaluating the currency of program outcomes:

• Do they convey the purpose of the program?
• Do they convey what is important or unique about the program?
• Do they outline the critical industry & 21st century competencies, skills and knowledge that students are expected to learn by the end of the program?
• What do you value about the program? What is special or innovative about it? Are they captured in the PLOs?
• What would the program need to be like for it to be the most sought after of its kind in BC or in Canada?
• Is anything missing?

Source:  Dyjur, Grant, & Kalu, 2019. University of Calgary, Taylor Institute, Program-Level Learning Outcomes

b) Align Current Program Outcomes with Program Competencies

Having reviewed the currency of existing program outcomes, this step will assist the team to align these program outcomes with program competencies (new or current) developed/reviewed in section 3.2.

Refer to the example, template below as you follow these steps.

Appendix 6: Program Outcomes Alignment Template

Step 1 – In column A vertically, list program competencies you developed/updated in section 3.2

Step 2 – List your current or newly developed program outcomes horizontally in columns B to columns I

Step 3 – Now map program outcomes with program competencies by marking “X” in the corresponding row/column

3.4.3 Next Steps

Refer to the Self-Study Guide (KPU Program Review Guide Series #4) Chapter 3, section 3.4 (Program outcomes). Follow the instructions provided in that section.