• Print

Academic Planning & Alignment and Academic Administrative Reviews FAQ's

Process

1. Why are we doing this?
2. Why are Academic and Alignment reports due after reallocations have been identified?
3. There have been so many changes in the process. What are these changes and why were they made?
4. Who is the audience for my report?
5. What are the changes to the process?
6. How will the reports be used?
7. How will the Comprehensive Academic Rating Team (CART) be trained and what rubric will be used?
8. If Comprehensive Academic Rating Team (CART) identifies inconstancies in ratings how will this be handled?
9. Will the Academic Planning and Alignment Process be used for reallocations?
10. Can the academic reports be rated without raters knowing the program identity?
11. What are the potential recommendations that can be made by the Dean to the Provost and the Provost to the President as a result of the program's academic planning and alignment report?

Program Data

12. When I review the data I was given there seem to be some gaps. Why do I not have IR data for all of the sub-criteria?
13. Some of the data from IR don't relate to my program--what do I do with them?
14. What do I do if I do not agree with the data the IR provided?
15. Can I provide additional data that the IR office does not have?
16. Do I benchmark my data to that of programs internal to the University or to external peers?

Academic Administrative Reports

17. I have an (institute, center, clinic, admin department)--what data will I get and what rubric should I write to?
18. Who is engaged with the review of the academic administrative reports?

Report Writing

19. My department has a BA and BS program--do I need to write a separate report for each?
20. What do I do with criteria that are not relevant to my program?
21. Who do I go to for help or questions?
22. As a report writer what is the best way to get assistance with the report?


Process

1. Why are we doing this?
The Academic Division is undertaking the Academic Planning and Alignment process to ensure that Pacific programs are aligned with Pacific 2020, are high quality and sustainable and, importantly, provide an outstanding learning environment for Pacific's students.  The vision for these efforts is that "Pacific will become known as one of the best teaching-focused universities in California - the first choice for students." to Pacific 2020.

The Academic Planning and Alignment process is one of several parts of Focusing On Our Future. The University is developing an evidence based evaluation process that is critical for our long-term strength.

  • The Academic Planning and Alignment process provides a snapshot of our overall program mix and is a starting point in an ongoing process to monitor our progress toward institutional goals and to consciously build strong programs in alignment with our strategic vision.
  • The academic reports and ratings will be essential for the Deans and the Provost to make recommendations for the University's program mix.

2. Why are Academic and Alignment reports due after reallocations have been identified?
The Deans recommended that the budget reallocations be separated from the academic reviews.  The two processes, reallocation and Academic Planning and Alignment, have different purposes.

  • Reallocation is intended to create a fund to allow for quick action on strategic plans that come forward.
  • Academic Planning and Alignment is a part of a bigger and longer-term process that aims to create a program mix that strengthens Pacific's reputation and long-term sustainability.
  • Separation of the Academic Planning and Alignment process and reallocation allows faculty to focus on generating thoughtful, honest and useful reports and ratings without being influenced by reallocation decisions.

3. There have been so many changes in the process.  What are these changes and why were they made?
The overall process for rating has remained the same from its inception. Improvements have been made to allow more time for reflection, strengthen the role of faculty in the process, and improve the rating system.

  • The timeline for reports was extended to allow faculty more time to analyze the data and write the reports.
  • The composition of Comprehensive Academic Rating Team (CART) was modified.  Instead of being the Strategic Planning Committee, CART will be comprised of members from existing groups experienced in assessment and program review -- Academic Affairs, Committee of Graduate Studies, Institutional Effectiveness Committee, and the Assessment Working Group -- plus two members from Academic Council.
  • The rubric and scoring were modified to address concerns. For more details click here (See question #5)

4. Who is the audience for my report?
This report has multiple audiences within the University and, as such, should communicate clearly with those inside and outside of your particular school/college.  The following groups and individuals are the audiences for your report:  the school/college faculty rating team, Comprehensive Academic Review Team (CART- see below), school/college dean, Provost and President.  Their role in reviewing the reports is described below.

5. What are the changes to the process?
As we have worked with faculty, deans and report writers, we have listened to your suggestions and made the following improvements to the Academic Planning and Alignment process:

  • The scoring on the rubric has been modified to address concerns about the rating scale. It now reflects a 1-5 rating scale for each criterion.
  • The President has agreed that holistic scoring of the program is not needed and acknowledges the importance of scoring each independent criterion.
  • The report has been shortened. The two financial criteria, 7 and 8, do not need to be addressed in the written report. This change enables the unit to begin to analyze these complex data with your dean. Your dean will discuss the data for criteria 7 and 8 with individual programs, rather than disseminating university wide, to ensure these data remain restricted to those within the Pacific community. Therefore criteria 7 and 8 will not be rated. The page limit remains the same.

6. How will the reports be used?
The faculty rating team in your unit will be the first to review the reports.  They will use the rubric to rate each of the seven criteria related to your program.  The criteria are rated individually and ratings will not be totaled into a composite score for the program.

The school/college ratings will then go to Comprehensive Academic Rating Team (CART), which is comprised of members from existing faculty bodies/committees with roles in assessment and program review -- Academic Affairs, Committee of Graduate Studies, Institutional Effectiveness Committee, and the Assessment Working Group -- plus two members from Academic Council.  CART will review the reports for cross-school consistency in applying the rubric; they will not be conducting ratings. 

Subsequently, your dean will use the reports and ratings to consult with faculty in the school/college about the future direction of the unit including its overall program mix. Your dean will make program recommendations to the Provost to enhance, maintain, consolidate/reorganize, reduce or eliminate to align unit level strategic plans with Pacific 2020 and the Academic Division Priorities.

The Provost will review the reports and recommendations from the Deans.  Based on this information the Provost will draft recommendations and share with the leadership team and the university community.  With input from these constituents, the Provost will make final recommendations to the President. 

The President will be interested in the overall portfolio of the University's academic offerings. She expects that meaningful conversations about program quality, impact and future sustainability are happening across the University, as well as conversations about how the schools/college will align with Pacific 2020.

7. How will the Comprehensive Academic Rating Team (CART) be trained and what rubric will be used?
The members of CART of will participate in live trainings (training via actual ratings of reports) with the unit report raters in January and will receive a second round of training during its live readings in February.  They will use the same rubric as the unit rating teams in order to check for consistency across units.

8. If Comprehensive Academic Rating Team (CART) identifies inconstancies in ratings how will this be handled?
Inconsistencies in ratings will be referred back to the unit rating teams for review.

9. Will the Academic Planning and Alignment Process be used for reallocations?
No.  These processes have been separated.  The units are aware that the Academic Division must contribute a total of 6% to the Strategic Investment Fund. The deans have received their percentage and are working on those plans currently. Draft plans for the contributions from the academic units are due to the Provost on November 1, with final recommendations due on December 1, 2013. 

10. Can the academic reports be rated without raters knowing the program identity?
Ratings will only be done by school-based teams. Since the data in the academic reports are program/degree specific, it is not possible to redact the reports for the rating process.  
Note that Comprehensive Academic Rating Team (CART) will not be rating program reports, only checking for consistency in ratings.

11. What are the potential recommendations that can be made by the Dean to the Provost and the Provost to the President as a result of the program's academic planning and alignment report?
There are five recommendations that can be made; enhance, maintain, consolidate/reorganize, reduce or eliminate. The complete discussion of the actions is available here.


Program Data

12. When I review the data I was given there seem to be some gaps.  Why do I not have IR data for all of the sub-criteria?

  • The Academic Planning and Alignment reports include seven criteria for report writers to discuss their program from a variety of perspectives, from history to future opportunities.  There are 23 sub-criteria and IR has provided quantitative data for nine of these. Academic units will provide data, as available, for the remaining 14 sub-criteria, in many cases using qualitative evidence to discuss their respective programs.
  • Data (either from IR or the academic programs) may be lacking in some areas, but that is something to note in your report and work on together in the future.  No single data point will be the sole basis for program recommendations; rather recommendations are based on the complex interplay of many data points.

13. Some of the data from IR don't relate to my program--what do I do with them?
You should review all the data that has been made available to you and use those that are relevant to your program.  If you are not sure if some data is relevant, contact IR since there may be ways of incorporating data that have not yet been considered. The important part is to connect with IR to have a conversation about any data about which you have questions.  Most programs have already engaged in individual conversations and have found these extremely helpful for writing their reports. 

14. What do I do if I do not agree with the data the IR provided?
Again, contact Institutional Research and talk with them about the differences. Sometimes the differences in the data depend on the semester the data is reported from or the method of calculation of certain data points.  Whatever the reason for the difference, it is important that you are comfortable with the data and understand what they represent.

15. Can I provide additional data that the IR office does not have?
Yes, additional data can be appended. There are no limitations on length and number of appendices.  Be sure to append only that data which is important to illustrating a criterion and is cited explicitly in your narrative.

16. Do I benchmark my data to that of programs internal to the University or to external peers?
Benchmarking against university wide figures Overview of the Data for Academic Program Reports informs you of the success of your program in relation to other University of the Pacific programs.  External data is valuable and should be used when available. Knowing how your program is performing in relation to external peers will help you define the strength of your program.  External benchmarks may not be readily available for all programs but are important to include when available.  Programs may develop their own peer list specific to the discipline. If you have developed your own list of external peers, please append it to this report, along with any additional external benchmarking data you cite in your report.


Academic Administrative Reports

17. I have an (institute, center, clinic, admin department)--what data will I get and what rubric should I write to?
The reports for Institutes, centers, clinics and admin departments will receive basic budget and FTE data from institutional research.  These reports will be completed using criteria for the academic administrative reports. Check the program list for your specific institute, center, clinic or admin department.

18. Who is engaged with the review of the academic administrative reports?
The process is the same that was followed by the administrative reviews completed during the summer. See the Academic Administrative Review Process


Report Writing

19. My department has a BA and BS program--do I need to write a separate report for each?
Yes. In all cases the BS and the BA degrees must be reported independently.  Even though some of the data for each may overlap, the discussions of the two degrees may be different for many of the criteria.

20. What do I do with criteria that are not relevant to my program?
When the criteria are not relevant, simply indicate so in the template with a statement explaining why the particular criterion is not relevant.

21. Who do I go to for help or questions?
Please contact Berit Gundersen (bgundersen@pacific.edu or x63158) for general questions and IR directly for data related questions (Mike Rogers, mrogers@pacific.edu). 

22. As a report writer what is the best way to get assistance with the report?
The Office of the Provost and the Office of Institutional Research (IR) have worked closely with many report writers throughout the Fall semester.

  • IR has met with report writers in each unit and has met with all individual report writers who have requested follow up assistance.
  • Those who have taken advantage of drop in sessions held in the Office of the Provost have found them useful.
    November 25,  2:00 - 3:00 pm   
    December 4,  11:00 - 12:00 pm
    December 16,  10:00 - 11:00 am
  • Staff in the Office of the Provost are also available to review drafts of reports.
  • Some report writers have already completed drafts of their reports, and we might be able to pair you with these writers so you can see examples of completed reports.   

Please take advantage of these opportunities to assist you!  Contact Victoria Oliva (voliva@pacific.edu) who will coordinate an appointment to assist you with data interpretation or any other dimension of report writing.  All are welcome!