1. education
2. graduate school
3. college

INSTITUTIONAL
EFFECTIVENESS
MANUAL
A GUIDE FOR ASSESSMENT AND CONTINUOUS IMPROVEMENT PUBLISHED BY THE DIVISION OF INSTITUTIONAL RESEARCH,
PLANNING AND EFFECTIVENESS
January 2013
BROWARDCOLLEGEISACCREDITEDBYTHECOMMISSIONONCOLLEGESOFTHESOUTHERNASSOCIATION
OFCOLLEGESANDSCHOOLSTOAWARDASSOCIATEANDBACCALAUREATEDEGREES.V ISITTHE
COMMISSIONONCOLLEGESATHTTP://SACSCOC.ORGFORQUESTIONSABOUTTHEACCREDITATIONOF
BROWARDCOLLEGE.
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
TABLEOFCONTENTS
1 Preface ...................................................................................................................................pg 4
2 About this Manual..................................................................................................................pg 5
3 Terminology ...........................................................................................................................pg 6
4 Accreditation and Institutional Effectiveness ........................................................................pg 7
5 Institutional Effectiveness at Broward College
5.1 Description ....................................................................................................................pg 8
5.2 Institutional Effectiveness Planning vs. Strategic Planning .........................................pg 9
5.3 Institutional Effectiveness Model.. .............................................................................. pg 10
5.4 Important IE and SACS Deadlines for the 2012-13 IE Cycle .....................................pg 10
5.5 Institutional Effectiveness Staff ....................................................................................pg 11
6 TaskStream Institutional Effectiveness Tool
6.1 Defining Units: Who Needs to Complete an IE Plan? .................................................pg 12
6.2 Using TaskStream .........................................................................................................pg 13
6.3 Institutional Effectiveness and Outcomes Assessment Template .................................pg 14
6.4 Template for Assessing Educational Programs ............................................................pg 15
6.5 Template for Assessing Administrative, Student Support, Community Units .............pg 16
7 Quality Assurance Review Process
7.1 Reviewers: Who Needs to Review My IE Plan? ..........................................................pg 17
7.2 Review Rubric: Academic Program Assessment Plan and Report ...............................pg 18
7.3 Review Rubric: Administrative, Student Support, Community/Public Svc Units .......pg 19
8 Best Practices: Academic Areas
8.1 Developing a Mission Statement ..................................................................................pg 20
8.2 Identifying, Writing and Prioritizing Academic Outcomes ..........................................pg 21
8.3 Assessing Academic Outcomes ....................................................................................pg 22
9 Best Practices: Administrative, Student Support, Community Units
9.1 Developing a Mission Statement ..................................................................................pg 23
9.2 Identifying, Writing and Prioritizing Unit Outcomes ...................................................pg 24
9.3 Assessing Unit Outcomes .............................................................................................pg 25
Page2
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
Table of Contents
10 Using Findings for Continuous Improvement ..............................................................pg 26
11 Training ..........................................................................................................................pg 27
12 FAQ................................................................................................................................pg 28
13 Appendices
A: Example Educational Program IE Plan...................................................................pg 30
B: Example Administrative, Student Support, Community Unit IE Plan....................pg 36
Page3
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
1. Preface
Dear Colleagues:
Thank you for your time and effort in helping make our Institutional Effectiveness process
at Broward College a success. The Office of Institutional Effectiveness prepared this manual
specifically to assist college staff, administrators and faculty in developing, implementing
and assessing strong, integrated, college-wide plans that add to the quality of Broward
College’s teaching, learning and overall institutional effectiveness.
Several people contributed to the development and completion of this manual: Drs. Gail
Wisan, Amy Gregory, Mrs. Mildred Foster and Mr. Gregg Sekscienski. I also would like to
extend my appreciation to Mrs. Laurie Heacock for her diligent and thoughtful work on the
overall document. To them, I am very grateful for their contributions.
If you need any assistance or have questions about institutional effectiveness or planning
assessment processes, please contact me. Suggestions for improving this manual are always
welcome.
Sincerely,
Rigoberto J. Rincones-Gómez, Ph.D.
Associate Vice President
Division of Institutional Research, Planning and Effectiveness
Email: rrincone@broward.edu | Office: 954- 201-7857 | Fax: 954-201-7322
Page4
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
2. About this Manual
This manual provides a conceptual and hands-on guide for the Broward College community.
This guide is intended to assist each and every academic program, educational and student
support area, administrative unit, and community and public service area to identify its expected
outcomes, measure the extent to which the program or unit achieves its outcomes, and provide
data and evidence of improvement based on an analysis of its assessment results. The evaluation
of institutional effectiveness (IE) is essential for continuous improvement in institutional quality
and for demonstrating the college is effectively accomplishing its mission.
Users of this manual can expect to understand:

Commonly used institutional effectiveness terminology

The college’s comprehensive, institutional effectiveness process

How to develop institutional effectiveness plans for educational programs

How to develop institutional effectiveness plans for administrative, student support and
community/public service areas

How to input institutional effectiveness plans into TaskStream (the college’s IE tool)

How institutional effectiveness supports SACS-COC accreditation

Important dates and deadlines for the IE cycle

How to use institutional effectiveness results for continuous improvement
Page5
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
3. Terminology
Term
Common Definition
Accountability
The obligation of a college to account for its activities, accept
responsibility for them and disclose the results in a transparent manner to
the public including the state, federal government and accrediting body.
Action
Specific change that will be implemented to improve programs or units
based on results/findings from a previous assessment cycle.
Action Status
The status of a program or unit’s actions when measures have been
updated or findings have been added. This would typically be entered
when preliminary finding are reported.
Goal
A statement of a desired future state or condition, stated more generally
than an outcome.
Learning Assessment
The process of determining the extent to which students have mastered
some instructional objective or competency.
Outcomes
The short-, medium- and long-term desired effects or changes.
Administrative Outcomes
Benefits that departments or administrative units receive or the college
receives as a whole.
Learning Outcomes
Changes in knowledge, attitudes, behaviors, values and opportunities that
occur as a result of learning. Learning outcomes may be identified at the
student, course and program level. Educational and student support areas
should also identify learning outcomes.
Program Outcomes
Benefits that students receive as a result of the completion of an entire
program of study.
Mapping
Aligning outcomes to a strategic goal, core value or to the college’s
mission.
Mission
A clear description of the purpose of the program or unit and who it
serves.
Unit
An academic degree-granting program, an academic or student support
area, an administrative support area or a community/public service area.
Workspace
Designated area in TaskStream where IE plans and findings are entered.
Page6
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
4. Accreditation & Institutional Effectiveness
The Southern Association of Colleges and Schools Commission on Colleges (SACS-COC) is the
regional body for the accreditation of degree-granting higher education institutions in the South.
In their 2012 edition of The Principles of Accreditation: Foundations for Quality Enhancement,
institutional effectiveness is addressed in Core Requirement 2.5 and is defined as follows:
“The institution engages in ongoing, integrated, and institution-wide research-based
planning and evaluation processes that (1) incorporate a systematic review of institutional
mission, goals, and outcomes; (2) result in continuing improvement in institutional
quality; and (3) demonstrate the institution is effectively accomplishing its mission.”
Whereas the Core Requirements are foundational requirements, The Principles of Accreditation:
Foundations for Quality Enhancement also contains Comprehensive Standards, which are
operational and represent best practices. The expectation for Institutional Effectiveness is
addressed in Comprehensive Standard 3.3:
“The institution identifies expected outcomes, assesses the extent to which it achieves these
outcomes, and provides evidence of improvement based on analysis of the results in each of
the following areas:
3.3.1.1 educational programs, to include student learning outcomes
3.3.1.2 administrative support services
3.3.1.3 academic and student support services
3.3.1.4 research within its mission, if appropriate
3.3.1.5 community/public service within its mission, if appropriate”
Page7
Institutionaal Effectivenesss Manual
Division of Innstitutional Reesearch, Planninng and Effectivveness
5. IE
E at Brow
ward Colleege
5.1 Description
he Commissiion on Colleeges, Browarrd College m
must complyy with
To mainttain accreditation with th
the core requirement
r
s and standaards containeed in the Prinnciples of Acccreditation: Foundations
for Qualiity Enhancem
ment and witth the policies and proceedures of thee Commissioon on Collegges.
In the con
ntext of institutional effeectiveness, Broward
B
Colllege strives to be the beest in class
amongst all Florida peer
p collegess. The Strateegic, Strateggic Implemenntation and D
Detailed
Implementation Plan
ns provide th
he roadmap to
o reach this end vision. E
Each membeer of the college
community works as part of a teaam within a division, offfice or deparrtment to translate these
o actions to drive Browaard College forward.
f
plans into
To advan
nce success through
t
orgaanizational structure, thee Division off Institutionaal Research,
Planning
g and Effectiv
veness (IRPE
E) is organizzed into fourr areas or off
ffices: Instituutional Plannning,
Institutio
onal Effectiveness, Institu
utional Reseearch and Buusiness Intellligence.
Page8
Institutionaal Effectivenesss Manual
Division of Innstitutional Reesearch, Planninng and Effectivveness
5.2 Instittutional Efffectiveness Planning
P
vs. Strategic P
Planning
It is usefu
ful to distingu
uish between
n strategic planning and institutionall effectiveneess planning.
Strategic planning is a systemic process
p
of en
nvisioning a desired futuure, and trannslating that
vision intto broadly defined
d
goalss or objectivees and a sequuence of stepps to achieve them. Thee
Broward College Strategic Plan, for instancee, extends thee college’s vvision througgh 2017.
In contraast, institutional effectiveeness plannin
ng operates on an annuaal cycle wherre each
program//unit examin
nes its purpose, identifiess expected ooutcomes, sellects assessm
ment tools annd
strategiess to collect data,
d
and anaalyzes and reeports findinngs for continnuous qualitty improvem
ment.
Innstitutional E
Effectivenesss
Plannning
Sttrategic Plan
nning
•
•
•
•
•
Collegee level
Vision focused
Wide view
v
Flexiblle means
Begins with desired
d end
•
•
•
•
•
Unnit level
Miission focuseed
Naarrow view
Sppecific meanss
Beegins with reeview of pastt
asssessment cyccle or review
w of
currrent status ((if new)
5.3 Instittutional Efffectiveness Model
M
Institutio
onal effectiveeness at Brow
ward Colleg
ge is missionn-driven and includes a sset of ongoinng
and systeematic, instittutional proccesses and prractices that encompass::

The
T assessmeent of academ
mic, adminisstrative, studdent support and commuunity/public
seervice prograams and serv
vices.

The
T developm
ment and imp
plementation
n of program
m/unit assesssment and evvaluation plaans.

The
T use of ev
valuation resu
ults to fosterr a culture off evidence aand continuous improvem
ment.
Page9
Institutionaal Effectivenesss Manual
Division of Innstitutional Reesearch, Planninng and Effectivveness
Overall Institutio
onal Effecttiveness M
Model
5.4 Impo
ortant IE an
nd SACS Deeadlines forr the 2012–113 Cycle
7/2/2012
2012-13 IEE
Cycle Startts
A
Aug 12
7/2/2012
10/5/12
Dead
dline for units
to en
nter IE plans
Sep 12
2
12/19/12
2
Deadline for u
units
to enter IEE
preliminary fin
ndings
Aug 12 - Dec 12
Fall TTerm
Oct 12
N
Nov 12
IEE Training
S
Sessions
9/6/1
12 - 9/28/12
Dec 12
Jan 13
IE Status
Report
1/15/2013
Page10
1/22/2013
IE Status
Presented
to BC BOT
F
Feb 13
5/10//13
Deadline ffor units
to enter finaal findings
for the 20
012-13
IE Cycle
Jan 13 - May 13
M
Winter Term
Mar 13
Apr 13
Maay 13
Dead
dline for units
to uplo
oad IE evidence IEE Status
R
Report
5/24/13
6/1
11/2013
6/28/2013
20
012-13 IE
Cyycle Ends
Jun 13
6/28/2013
IE Staatus
Presented
to BC BOT
6/18/2
2013
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
5.5 Institutional Effectives Staff
Dr. Rigoberto Rincones-Gómez, Associate Vice President of Institutional Research, Planning
and Effectiveness
Dr. Gail Wisan, District Director of Institutional Effectiveness
Dr. Amy Hyman Gregory, Interim Director of Institutional Effectiveness
Ms. Mildred Foster, Accreditation and Document Analyst of Institutional Effectiveness
Page11
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
6. TaskStream: Institutional Effectiveness Tool
The college is committed to using TaskStream as an IE tool through 2014.
6.1 Defining Units: Do I Need to Complete an IE Plan?
As stated in the Procedure Manual shown below, “all College administrators are responsible for
the implementation and execution of institutional effectiveness plans for the organizational
departments and/or programs for which they are assigned administrative responsibility.”
Page12
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
6.2 Using TaskStream
Broward College has implemented TaskStream’s customizable Accountability Management
System (TaskStream’s AMS) to support continuous improvement projects and manage college
wide and program level accountability processes.
To request access to TaskStream and the creation of a workspace, please contact Dr. Amy
Hyman Gregory at 954-201-7543 or ahymangr@broward.edu.
The TaskStream User Guide is accessible from the IRPE Division SharePoint web site in
Workplace.
To assist you in completing your online IE plans in TaskStream, paper templates are provided in
section 6.3.
Page13
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
6.3 Institutional Effectiveness & Outcome Assessment Template
Note:

If you have already entered your 2012-13 assessment plan into TaskStream, you do not need
to fill out a paper template or follow the steps below.

If your unit is an educational program, use Template 1. If your unit is an administrative,
student support or community/public service unit, use Template 2.
STEPS:
1. Please enter your unit’s mission statement in Item 1 of the Template.
2. Please define new individual outcomes or use/select outcomes from your 2011-12
assessment plan.
3. Please map each of your outcomes to the appropriate strategic goal(s), core value(s),
and/or the college’s mission.
4. Please identify the action(s)/activities based on your 2011-12 assessment findings and
link them to specific outcomes. Then complete the following for each action/activity:







Action details
Implementation plan (timeline)
Responsible personnel
Resources needed
If budget approval is required
Budget request amount
Priority (low, medium or high)
5. Please select one of the four options (not started, in progress, completed or not
implemented) at the time you report your findings.
6. Please follow the directions included in Item 6 of the template.
7. Assessment findings will be completed after data have been gathered and analyzed.
8. Please follow the directions included in Item 8 of the template.
Page14
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
Page15
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
Page16
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
7. Quality Assurance Review Process
7.1 Reviewers: Who Needs to Review My IE Plan?
To provide quality assurance, institutional effectiveness planning includes a review process.
Review rubrics are designed to facilitate this process, promoting quality control, providing
guidance to reviewers and ensuring consistency in standards. Once an assessment plan or report
is completed, the TaskStream workspace manager must submit their plan/report for review.
The assessment plans for academic programs are typically reviewed by Associate and Academic
Deans followed by final reviews by the Provost and Campus President. The assessment plans for
student support programs are reviewed by the Vice President of Student Affairs. The assessment
plans for administrative and community/public service units are reviewed by the appropriate
Vice President.
Page17
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
7.2 Review Rubric: Academic Program Assessment Plan and Report
Page18
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
7.3 Review Rubric: Administrative, Student Support, Community/Public Service Units
Page19
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
8. Best Practices: Academic & Educational Programs
8.1 Developing a Purpose or Mission Statement
A purpose or mission statement is a clear description of the purpose of a program and who it
serves. A mission statement guides the practices of the program.
Ideally, a mission statement should:
 Concisely state the purpose of the program
 Indicate the primary function or activities of the program
 Specify who the program serves
 Clearly support the college’s broader mission
 Showcase what makes the program unique or distinctive
8.2 Identifying, Writing and Prioritizing Academic Outcomes
Academic outcomes are comprised of student learning outcomes and program outcomes. Student
learning outcomes are typically included in a course syllabus and answer what students should
know and be able to do at the time of course completion. Program learning outcomes answer
what students should know and be able to do at the time of program completion.
To identify academic outcomes, focus on the key skills a student gains from completing a course
(student learning outcomes) or program (program outcomes) in terms of:

Knowledge

Attitudes

Values

Behaviors

Opportunities
Once you have identified a list of academic outcomes, evaluate them based on the following
criteria to ensure they are:
 Learner centered versus teaching centered
 Central to the program’s mission
Page20
Institutionaal Effectivenesss Manual
Division of Innstitutional Reesearch, Planninng and Effectivveness
 Im
mportant and
d meaningfu
ul for both faaculty and stuudents
 Connected
C
an
nd mapped to
o a goal in th
he 2012-17 B
Broward College Strateggic Plan, to a
co
ore value or to the colleg
ge mission
 Actionable:
A
you
y can take action as a result
r
of the outcome
 SMART: Sp
pecific, Meassurable, Attaainable, Releevant, Timely
Modelfo
orWritingO
Outcomes: A
Asaresultoofstudents
completin
ng<insertprogram>,<iinsert%>w
willbeable
to<inserttactionverb
b>.
Example
e:Asaresulltofstudentsscompletinggthe
NursingR
R.N.program
m,95%willbeabletoppassthe
NCLEXliccensingexam
montheirfi
firstattemptt.
Next, priioritize the outcomes.
o
Do
D not try to cover every domain. Foocus on a small number oof
outcomess (top three) that are:
 Widely
W
relevaant
 Highly
H
valued
d by faculty and students
 Reflective
R
of higher orderr critical skillls
 Most
M useful for
f informing
g decisions that
t can leadd to improvem
ment
Page21
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
8.3 Assessing Academic Outcomes
Once you have defined your top three academic outcomes, the next step is to choose an
assessment method. There are two types of assessment methods: direct and indirect. Direct
assessment methods measure student performance by examining samples of actual student work
at the course, program or institutional level. Indirect assessment methods measure outcomes
related to student learning—such as student perceptions—versus evidence of direct student
learning.
Examples of Direct and Indirect Measures of Academic Learning Outcomes
Direct Measures
Course
Indirect Measures
 Research project
 Course evaluation
 Tests and quizzes
 Student hours spent on service
learning
 Arts performance
 Lab assignment
Program/Unit
 Employer rating of student
performance
 Focus groups with students, faculty
and employers
 Licensure exam pass rate
 Program review data
 Capstone project
 Job placement
 Internship supervisor rating
 Employer or graduate surveys
 Portfolio
Institutional
 Performance on achievement
tests
 Rubric scores for general
education courses or other
courses required of all students
 Performance on tests of writing,
critical thinking or general
knowledge
 Locally developed, commercial or
national satisfaction and
engagement surveys, e.g.,
Community College Survey of
Student Engagement (CCSSE),
Noel-Levitz

Graduation rates
 Retention rates
 Transcript analyses that examine
patterns of course selection and
grading
Adapted from the Middle States Commission on Higher Education. Student Learning Assessment: Options and
Resources. Chapter 3, Evaluating Student Learning: 27-53; 2007.
Page22
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
9. Best Practices: Administrative, Student Support, Community Units
9.1 Developing a mission statement
A mission statement is a clear description of the purpose of a unit and who it serves. A mission
statement guides the practices of the unit.
Ideally, a mission statement should:
 Concisely state the purpose of the unit
 Indicate the primary function or activities of the unit
 Specify who the unit serves
 Clearly support the college’s broader mission
 Showcase what makes the unit unique or distinctive
9.2 Identifying, Writing and Prioritizing Outcomes for Administrative, Student Support,
Community/Public Service Units
In addition to the academic outcomes described in Section 8, administrative, student support and
community/public service units also include administrative outcomes. Administrative outcomes
align with unit mission statements and answer how the college—including students, faculty and
staff-—benefits as a result of utilizing the services or functions provided by these units. Ideally,
these outcomes:

Address performance: efficiency, effectiveness, quality, service, cost

Inform decision-making and planning for innovation and
change

Focus attention on what matters the most: key functions,
services, processes
Key questions to ask when identifying administrative outcomes
include:

To achieve our mission how should we appear to students,
faculty, staff, other units, the college as a whole?
Page23
“Efficiencyisdoingthingsright;
effectivenessisdoingtheright
things.”
PeterF.Drucker
Institutionaal Effectivenesss Manual
Division of Innstitutional Reesearch, Planninng and Effectivveness

What
W businesss processes must we exccel at to bestt benefit thosse we serve??

How
H is our un
nit planning for innovatiion and in m
meeting the fuuture needs oof those we
seerve?

How
H do we develop
d
and manage
m
reso
ources (humaan and fiscall)?

What
W is the most
m highly valued
v
servicce we providde institutionnally?
Once you
u have identiified a list off administrattive outcomees, evaluate them based on the following
criteria to
o ensure they
y are:
 Central
C
to you
ur area’s misssion
 Under
U
your purview
 Distinctive
D
to
o your unit
 Connected
C
an
nd mapped to
o a goal in th
he 2012-17 B
Broward College Strateggic Plan, to a
co
ore value or to the colleg
ge mission
 Actionable:
A
you
y can take action as a result
r
of the outcome
 SMART: Sp
pecific, Meassurable, Attaainable, Releevant, Timely
TipforWrritingOutco
omes:Usea
actionverbs suchasincrrease,decreease,
improve,reeduce,minim
mize,maximize,complette,promote,,provide,deesign,
develop,offfer,conduct,,deploy,pro
ocess,implem
ment,expan
nd….
Example:A
Asaresulto
ofmeetingw
withanadvis
isor,95%off allnewstud
dents
developeda
aneducation
nalplandurringtheirfirrstsemesterr.
Next, priioritize the outcomes.
o
Do
D not try to cover every domain. Foocus on a small number oof
outcomess (top three) that are:
 Widely
W
relevaant
 Highly
H
valued
d by the stafff in your uniit
Page24
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
 Indicative of what your target audience knows, can do or values as a result of using your
service
 Most useful for informing decisions that can lead to improvement.
9.3 Assessing Administrative, Student Support, Community/ Public Service Outcomes
Once you have defined your top three outcomes, the next step is to choose an assessment
method. See Section 7.3 on page 19 of this guide for a description and examples of direct and
indirect assessment methods for academic areas.
Typically the methods most useful for assessing administrative outcomes include:

Faculty/staff and student satisfaction and engagement surveys (e.g., % students student
satisfied with Advising and Enrollment Services, % of faculty and staff satisfied with
service from the Help Desk)

Focus group sessions targeted to the policies, processes and procedures of a unit (e.g.,
admission process, financial aid process, testing and placement policy)

Evaluation forms from attendees of a program or training conducted by your unit (e.g., %
of new students satisfied with orientation, etc.)

Unit statistics (e.g., call abandonment rate, resolution rate, % of requests processed on
time, training expenditures/employee, student/staff ratios, % transcripts evaluated within
x days of receipt, etc.)
Page25
Institutionaal Effectivenesss Manual
Division of Innstitutional Reesearch, Planninng and Effectivveness
10.Using
U
Find
dings for Continuou
C
s Improveement
The goal of assessmeent is continu
uous improv
vement. Eachh outcome m
measure asseessed should
result nott only in traccking and monitoring, bu
ut in actions designed too yield improovement. Thhe use
of resultss is crucial in
n closing thee feedback lo
oop and for ssuccessful asssessment.
with all stakkeholders,
To facilittate use, the results shou
uld be broadlly shared andd discussed w
including
g faculty, adm
ministrators, staff, studeents, advisoryy boards andd the commuunity, as
appropriaate. These diiscussions sh
hould guide the developm
ment of speccific strategiies that are
sustainab
ble, feasible and designed to improvee program, uunit or studennt learning ooutcomes.
a
may require
r
resouurces, it is crritical that im
mprovementt
Because the implemeentation of actions
strategiess requiring funds
fu
be link
ked to the bu
udget processs by means oof a budget rrequest. For
example,, an improveement strateg
gy may require investmeents in technnology, profeessional
developm
ment or the hiring
h
of add
ditional staff.. If funds aree not availabble, then the implementaation
strategy may
m require the reallocaation of resou
urces. Somee improvemeent strategiess do not requuire
fiscal ressources but in
nvestments in
i time and collaboration
c
n which requuire planningg such as
changes to
t policies, course
c
redesign, schedulling changess or program
m modificatioons.
Page26
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
11. Training
An Institutional Effectiveness training program was launched at Broward College in September
2012. The training program has three components:
1. Assessment Planning
2. TaskStream
3. Reviewer Training
Group and individual training sessions are held during the yearly assessment cycle. Each year,
assessment cycle timelines are posted on the IRPE SharePoint web site in WorkPlace and widely
shared with the Broward College community.
1. Assessment Workshop: How to Create Better Assessment Plans and Write Better
Outcomes
Description: This workshop will explain the components of an Institutional Effectiveness Plan
(mission, program and learning outcomes, mapping, actions, strategies and measures) and using
your findings/results for improvement. Participants will learn: how to write better outcomes,
create good assessment plans, plan for assessment, how to make assessment useful for your
department/unit.
2. TaskStream Training Workshop
Description: This workshop will cover the basics of entering institutional effectiveness
assessment information into TaskStream. Participants will develop the knowledge necessary to
create/revise assessment plans, map outcomes to specific goals and initiatives, and enter
assessment plans, findings, and supporting documentation into their area’s Workspace(s). A
TaskStream user account is needed to enroll in this workshop.
3. Reviewer Training Workshop
Description: Reviewer training includes suggested reviewer structure, the purpose and use of
rubrics, example rubrics and information on how to review plans and reports in TaskStream.
Page27
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
12. FAQS
Q. Are all college administrators required to complete an institutional effectiveness plan?
A. Yes, all College administrators are responsible for the implementation and execution of
institutional effectiveness plans for the organizational departments and/or programs for which
they are assigned administrative responsibility.
Q. How do I input my IE plan?
A. Via TaskStream. Paper templates are available to help you organize your plan before entering
into TaskStream.
Q. How do I acquire a TaskStream account and the creation of a workspace?
A. Contact Dr. Amy Hyman Gregory in IRPE at 954-201-7543 or ahymangr@broward.edu
Q. Who has to review my IE plan?
A. The assessment plans for academic programs are typically reviewed by Associate and
Academic Deans followed by final reviews by the Provost and Campus President. The
assessment plans for student support programs are reviewed by the Vice President of Student
Affairs. The assessment plans for administrative and community/public service units are
reviewed by the appropriate Vice President.
Q. How many outcomes should I include in my IE plan?
A. Three is recommended
Q. What are the criteria for determining if I developed a good outcome measure?
A. Specific, Measurable, Attainable, Relevant, Timely (SMART)
Q. What do outcomes have to map and connect to?
A. A goal in the strategic plan, a core value or to the college’s mission
Q. Who do I contact for questions regarding TaskStream use, training or troubleshooting?
A. Contact Dr. Amy Hyman Gregory in IRPE at 954-201-7543 or ahymangr@broward.edu
Q. Who do I contact regarding assessment workshops, assistance writing outcomes, developing
rubrics or curriculum maps?
A. Contact Dr. Gail Wisan in IRPE at 954-201-7482 or gwisan@broward.edu
Q. Who do I contact to schedule meetings, trainings or workshops?
A. Contact Mildred Foster in IRPE at 954-201-7480 or mfoster@broward.edu
Page28
Institutionaal Effectivenesss Manual
Division of Innstitutional Reesearch, Planninng and Effectivveness
Ap
ppendix A
ACT
TUAL TASK
KSTREAM
M EXAMPLE
E
Institutiona
al Effectiven
ness & Outccome Assesssment for E
Educational Program
Fiscal Yeear: _7/1/20
012-6/30/201
13________
______________
Educatio
onal Progra
am: Radiatio
on Therapy
y A.S. degreee (Health Sciences) Specific Coursse (if
applicab
ble):
Steps to
o Creating an
a IE Plan iin TaskStreeam
1. Missio
on Statemen
nt/Academicc Program Goals
G
(Whaat are studentts expected tto know andd be
able to
o do at the en
nd of the pro
ogram/course? What skiills should sttudents havee acquired baased
on thee program off study and/o
or specific co
ourse?)
The misssion of the Radiation
R
Theerapy Prograam is to preppare competeent, ethical aand culturallly
sensitive entry-level therapists who
w are capab
ble of functiooning in an ever-changinng health caare
delivery system as a result of beiing dedicated
d to lifelong self-directed learning. T
The program
m is
plishing its mission
m
throu
ugh the use of effective and diverse instructionaal
committeed to accomp
methods that encomp
pass both traaditional as well
w as technnology-basedd strategies.
2. Learn
ning/Program
m Outcomees and Meassures (Studeent Learnin
ng Outcomes are specific,
measu
urable, and reesults oriented changes within
w
studeents that can be evaluatedd based on
classro
oom learning
g. Program
m Outcomes are measuraable end resuults studentss are expecteed to
achiev
ve after comp
pleting the entire
e
prograam.):
Performaance
The grad
duate will bee able to effeectively applly
the skillss and knowleedge related
d to the
radiation
n therapy field
Career Development
D
t
The grad
duate will bee able to partticipate in
career deevelopment based upon selfassessmeent, perform
mance appraissals, work
setting and
a special in
nterests to fo
oster a sense of
life-long
g learning
The Presscription
The grad
duate will bee able to perfform safe
radiation
n therapy treatments com
mpetently
based on
n the prescrip
ption so as to
o minimize
risk to th
he patient/cliient, self, and others.
Page29
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
See measures below in Assessment Plan.
3. Mapping (Outcomes should be mapped to the appropriate strategic goal of the 2012-2017 BC
strategic plan, core values, and/or to the specific aspect of the BC mission):
Outcomes are mapped to Broward College Strategic Plan 2012-2017, student success Goal 1.
4. Actions (Actions are specific changes used to improve your program based on the previous
cycle’s assessment findings):
Increase the variety of teaching methods utilized
To begin utilizing clicker technology, web enhancing of lectures and labs
Action: Continue web enhancement of the Radiation Therapy
Program
This Action is associated with the following Findings
Finding on student performance.
1.2 To Begin Web
Enhancement of
Lecture Courses
To enhance the students
access to material by
providing different
styles and methods of
delivery. Also allowing
the students access to
the educational material
24/7
Action details: Develop one or more blended course for the
program
Implementation Plan (timeline): To have the first blended
course up and running during fall 20141 and survey the results
in 20142.
Key/Responsible Personnel: faculty, program manager,
associate dean
Resources needed (IT,IR, etc.): student satisfaction and
increasing academic and clinical performance
Budget approval required? (Describe): course and training
necessary are offered through the professional development.
Budget request amount: $0.00
Priority: Medium
Page30
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
5. Action Status (Action Status indicates the current status of each action taken, whether it is
completed, is still in progress, etc. Action Status may be reported later in the year as you
record progress or at the same time you report your current assessment cycle findings):
Report status in preliminary report, December 2012 and final report, 2013
6. Assessment Plan (Input your current assessment plan carrying over outcomes and measures
from the previous cycle and/or including new outcomes and measures implemented. Include
a description of each measure, an acceptable and ideal target, an assessment timeline, and the
responsible personnel for each):
Measure: Performance
Program level; Direct - Other
Performance
The graduate will be
able to effectively apply
the skills and knowledge
related to the radiation
therapy field
Details/Description: At the end of the program, the graduate will
be able to perform safe radiation therapy treatments competently
based on the prescription so as to minimize risk to the
patient/client, self, and others as observed in the classroom, lab and
clinical arenas.
Measurement will entail direct observation of student providing
simulated or actual treatment.
Acceptable Target: 85% of the graduates will demonstrate the
above.
Ideal Target: 100% of the graduates will demonstrate the above.
Assessment Timeline: Annually - at the end of Term III.
Key/Responsible Personnel: Faculty
Clinical Instructors
The Prescription
The graduate will be
able to perform safe
radiation therapy
treatments competently
based on the
prescription so as to
minimize risk to the
patient/client, self, and
others.
Measure: The Prescription
Program level; Direct - Other
Details/Description: At the end of the program, the graduate will
be able to perform safe radiation therapy treatments competently
based on the prescription so as to minimize risk to the
patient/client, self, and others as observed in the classroom, lab and
clinical arenas.
Acceptable Target: 85% of the graduates will demonstrate the
above.
Page31
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
Ideal Target: 100% of the graduates will demonstrate the above.
Assessment Timeline: Annually at the end of term III.
Key/Responsible Personnel: Faculty
Clinical Instructors
Measure: Data Collection
Program level; Direct - Other
Data Collection
The graduate will be
able to demonstrate
competence in
performing specific
data-collection
techniques related to the
prescription.
Health Care Delivery
Systems
The graduate will be
able to use human and
material resources and
services to provide highquality, efficient, and
cost effective radiation
therapy services based
on patient/client goals
and outcomes
Details/Description: At the end of the program, the graduate will
be able to demonstrate competence in performing specific datacollection techniques, as delegated by the supervising radiation
therapist as observed in the classroom, lab and clinical arenas.
Acceptable Target: 85% of the graduates will demonstrate the
above.
Ideal Target: 100% of the graduates will demonstrate the above.
Assessment Timeline: Annually - at the end of Term III.
Key/Responsible Personnel: Faculty
Instructors
Measure: Health Care Delivery System
Program level; Direct - Other
Details/Description: At the end of the program, the graduate will
be able to use human and material resources and services to
provide high-quality, efficient, and cost effective radiation therapy
services based on patient/client goals and outcomes as observed in
the classroom, lab and clinical arenas.
Acceptable Target: 85% of the graduates will demonstrate the
above.
Ideal Target: 100% of the graduates will demonstrate the above.
Assessment Timeline: Annually - at the end of Term III.
Key/Responsible Personnel: Faculty
Instructors
7. Assessment Findings (Summarize the findings for each outcome and measure, including
supporting evidence such as surveys, data, tables, and indicate specifically how these findings
will be used for improvement.):
Page32
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
Evidence must be provided on each outcome.
Measure: Performance
Program level; Direct - Other
Details/Description: At the end of the program, the graduate will
be able to perform safe radiation therapy treatments competently
based on the prescription so as to minimize risk to the
patient/client, self, and others as observed in the classroom, lab and
clinical arenas.
Acceptable Target: 85% of the graduates will demonstrate the
above.
Ideal Target: 100% of the graduates will demonstrate the above.
Assessment Timeline: Annually - at the end of Term III.
Key/Responsible Personnel: Faculty
Clinical Instructors
Findings for Performance
Performance
The graduate will be
able to effectively apply
the skills and knowledge
related to the radiation
therapy field
Summary of Findings: At the end of Term III all of the
graduating students were able to perform safe radiation therapy
procedures. This was based on their ability to read the
prescriptions and perform the treatments accurately and safely.
This was documented on the Rotational evaluations, the therapist
final evaluations and the photon and electron treatment
evaluations.
Results: Acceptable Target Achievement: Exceeded; Ideal
Target Achievement : Exceeded
How do you plan to use these findings for continuous
improvement? We plan to continue to monitor and evaluate the
graduating students in all aspects of a radiation therapy, including
reading prescription. room preparations and delivery.
Substantiating Evidence:
Electron treatment eval prescription.pdf (Adobe Acrobat
Document)
Areas utilized for performance
grades Summer 2011.pdf (Adobe Acrobat Document)
Monthly-Rotational Eval performance.pdf (Adobe Acrobat
Document)
Areas utilized for performance
Photon Evaluation performance docx.pdf (Adobe Acrobat
Document)
Areas utilized for performance
Sample of electron Eval...PDF (Adobe Acrobat Document)
Sample of Monthly rotational evals.PDF (Adobe Acrobat
Page33
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
Document)
Sample of photon eval.PDF (Adobe Acrobat Document)
Sample of Therapist eval.PDF (Adobe Acrobat Document)
Therapists Final Evaluation Performance.pdf (Adobe
Acrobat Document)
Areas utilized for performance
Measure: The Prescription
Program level; Direct - Other
The Prescription
The graduate will be
able to perform safe
radiation therapy
treatments competently
based on the
prescription so as to
minimize risk to the
patient/client, self, and
others.
Details/Description: At the end of the program, the graduate will
be able to perform safe radiation therapy treatments competently
based on the prescription so as to minimize risk to the
patient/client, self, and others as observed in the classroom, lab and
clinical arenas.
Acceptable Target: 85% of the graduates will demonstrate the
above.
Ideal Target: 100% of the graduates will demonstrate the above.
Assessment Timeline: Annually at the end of term III.
Key/Responsible Personnel: Faculty
Clinical Instructors
Findings for The Prescription
Summary of Findings: At the end of Term III all of the
graduating students were able to perform safe radiation therapy
procedures. This was based on their ability to read the
prescriptions and perform the treatments accurately and safely.
This was documented on the Rotational evaluations, the therapist
final evaluations and the photon and electron treatment
evaluations. The students were also able to safely and accurately
run quality assurance on the machines to make sure that the
correct dose per prescription was maintained, and this was
demonstrated in the clinic during warm up procedures.
Results: Acceptable Target Achievement: Exceeded; Ideal
Target Achievement : Exceeded
How do you plan to use these findings for continuous
improvement? We plan to continue to monitor and evaluate the
graduating students in all aspects of a radiation therapy, including
reading prescription. room preparations and delivery and warm
up procedures.
Substantiating Evidence:
CT Simulation Evaluation Form prescription docx.pdf
(Adobe Acrobat Document)
Page34
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
Areas utilized for prescription skills
Electron treatment eval prescription.pdf (Adobe Acrobat
Document)
Areas utilized for prescription skills
grades Summer 2011.pdf (Adobe Acrobat Document)
Monthly-Rotational Eval prescription.pdf (Adobe Acrobat
Document)
Areas utilized for prescription skills
Photon Evaluation prescription.pdf (Adobe Acrobat
Document)
Areas utilized for prescription skills
Sample of electron eval..PDF (Adobe Acrobat Document)
Sample of Monthly rotational evals.PDF (Adobe Acrobat
Document)
Sample of photon eval.PDF (Adobe Acrobat Document)
Sample of Therapist eval.PDF (Adobe Acrobat Document)
Sample Warm up Eval..PDF (Adobe Acrobat Document)
Samples of CT Simulation evals..PDF (Adobe Acrobat
Document)
Therapists Final Evaluation Prescription.pdf (Adobe
Acrobat Document)
Areas utilized for prescription skills
Warm-up Competency. eval prescription.pdf (Adobe
Acrobat Document)
Areas utilized for prescription skills
8. Budget Request (If funding is needed to support an outcome you must complete a Budget
Request Evaluation Form indicating the status of your budget request, whether it has been
100% approved, partially approved, or not approved).
Page35
Institutionaal Effectivenesss Manual
Division of Innstitutional Reesearch, Planninng and Effectivveness
Ap
ppendix B
MODE
EL EXAMP
PLE
Institu
utional Effecctiveness & Outcome Assessment
A
ffor Administrative, Stu
udent Support
and
d Communiity/Public Seervice Unit
Fiscal Yeear: 7/1/201
12-6/30/2013
3
Administraative Unit: ☐ Yes ☐ N
No
Student Support Seervices Unit:: X Yes ☐ No
N
Com
mmunity/Pu
ublic Servicee Unit: ☐ Y
Yes
☐ No
me: Disabiliity Services (Division of
o Student A
Affairs)
Unit Nam
Steps to
o Creating an
a IE Plan iin TaskStreeam
1. Missio
on Statemen
nt (What serrvices do you
u provide? T
To whom aree theses servvices provideed?
For stu
udent support, how do you
y plan to im
mpact studennt learning ooutcomes?)
Disability
y Services (D
DS) helps en
nsure equal educational
e
aaccess and oopportunity ffor all studennts
with docu
umented disabilities. In
n addition, DS collaborattes with facuulty, adminisstrators, and staff
to ensuree the facilitattion of reaso
onable and ap
ppropriate acccommodatiions for studdents with
documen
nted disabilitties in accord
dance with Section
S
504 oof the Federral Rehabilitaation Act off
1973 and
d Title III of the Americaans with Dissabilities Actt.
2. Outco
omes and Measures
M
(Ad
dministrativee and Studen
nt Learningg Outcomes are specificc,
measu
urable, and reesults oriented):
Outcomee 1 example: Disability Services
S
(DS
S) will providde accurate aand timely innformation tto
studen
nts in their fiirst semesterr at Broward
d College so tthat they cann self-identiffy as needing DS
if apprropriate.
Examplles of direct measures: 1,)counts
1
of workshops
w
pprovided durring student orientation;
2.) Inforrmation on DS
D website evaluated
e
witth rubric; 3.)) Hits on thee Broward D
DS website; 44.)
Countts of studentss who are prrovided one on
o one counnseling regarrding DS serrvices duringg the
first month
m
of scho
ool;
Outcomee 2 example:
Disability
y Services will
w provide faculty
f
with timely inforrmation abouut the studennts in their
classes who are seelf-identified
d as needing accommodaations and sppecify the naature of the
needed
d appropriatte accommod
dation(s).
Examples of direct measures:
m
1.)) time line do
ocument creaated for students, facultty and staff
regard
ding accomm
modations;
2.) Direct evidence of
o distribution of time lin
ne to studentts, faculty, annd staff (e.g., meeting
minutees, email disstributions; student
s
newsspaper adverrtisements.
Page36
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
3. Mapping (Outcomes should be mapped to the appropriate strategic goal of the 2012-2017 BC
strategic plan, core values, and/or to the specific aspect of the BC mission):
Disability Services outcome about providing accurate and timely information to students maps to
the Broward College Strategic Plan student success goal 1 and also Broward College's
mission.
4. Actions (Actions are specific changes used to improve your unit based on the previous cycle’s
assessment findings):
1.) Each semester, Disability Services will provide monitoring reports regarding the distribution
of accommodation information to faculty and information about the accommodations their
students will need.
2.) Disability Services will provide information to faculty one week earlier than the prior
semester.
3.) Disability Services will provide training for volunteer note takers who will then provide
service for students with documented disabilities who require assistance with note taking.
5. Action Status (Action Status indicates the current status of each action taken, whether it is
completed, is still in progress, etc. Action Status may be reported later in the year as you
record progress or at the same time you report your current assessment cycle findings): When
the preliminary report is filed (December 2012), the Action Status needs to be completed; Is
Action "in process," "completed," or "not started."
6. Assessment Plan (Input your current assessment plan carrying over outcomes and measures
from the previous cycle and/or including new outcomes and measures implemented. Include
a description of each measure, an acceptable and ideal target, an assessment timeline, and the
responsible personnel for each):
For example, carry over (i.e., continue from the prior year) Outcome 2 :
Disability Services will provide faculty with timely information about the students in their
classes who are self-identified as needing accommodations and specify the nature of the
needed appropriate accommodation(s). Direct Measure: number of information sessions
provided for faculty with a rubric evaluating information content; Indirect Measure: survey of
faculty who report what they learned and their satisfaction.
Acceptable target: provide information at faculty orientation; Ideal target: provide information at
information session on each campus. Targets related to information learned at information
sessions.
Responsible person: Director of Disability Services;
Budget: budget for coffee, copying, and printing of information material.
Add new outcome: Disability Services will recruit and train note takers in classes with students
with documented disabilities. Direct Measures: count of recruitment activities; number of note
Page37
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
takers recruited; number of note takers trained;
Add new outcome: Disability Services will compare the learning outcomes (including test
performance) of students with disabilities with access to note takers as compared with classes
without note takers for students with documented disabilities.
7. Assessment Findings (Summarize the findings for each outcome and measure, including
supporting evidence such as surveys, data, tables, and indicate specifically how these findings
will be used for improvement.):
Findings completed in the preliminary report by December 2012 and again by June 2013.
Summarize findings for each outcome and then provide evidence of findings.
For Example, analyze success data for students with documented disabilities who have access to
note takers in their class as compared with students with documented disabilities who do not
have access to note takers. Compare both to success data on non-disabled students. If note-taking
action/strategy is working, students using the service should exhibit the same level of success
(e.g., grade performance) as students without disability in the same class.
8. Budget Request (If funding is needed to support an outcome you must complete a Budget
Request Evaluation Form indicating the status of your budget request, whether it has been
100% approved, partially approved, or not approved).
Budget: budget for coffee, copying, and printing of information material.
Page38
Institutional Effectiveness Manual
Division of Institutional Research, Planning and Effectiveness
This page intentionally left blank
Page39