university place home | contact
Evaluation Plan and Techniques

University Place Assessment Plan

The University Place assessment is designed to document project goal attainment, provide formative feedback to aid in project management, and provide documentation to help disseminate project resources in the field. This assessment will utilize a multi-trait, multi-method case study design utilizing both project outputs and outcomes for each stakeholder group. This assessment plan is designed to provide documentation of the extent to which major project objectives have been fulfilled.

The goal of the University Place Project is to develop a partnership model demonstrating (1) how university-generated content is adapted to expand and enhance local program and educational services; (2) how the content could be shared among stations; and (3) how the practice could be promoted to other public broadcast stations.

This assessment will focus on the following levels of inquiry:

  1. Content Distribution and Use

    Objective: University Place content will be provided and utilized through a variety of platforms.

    The University Place Assessment will track both content distribution and platform use through the following indicators and data sources:

    Distribution System Participation Indicators Data Source
    A. Website    
       1. PreK-12 Content Page Views Hosting Station
       2. Other Content Page Views, Requests for Media Hosting Station
    B. Toolkit Perceived Quality
    Anticipated Toolkit Use
    PBS Station Survey
    PBS Station Survey
    C. Video On Demand Views
    Most popular views
    Hosting Station
    Hosting Station
    D. On-Air Broadcast Number of Viewers
    Distribution of Viewers
    *Nielsen Ratings
    *Nielsen Ratings

    *Ratings: Currently, Nielsen Ratings are only available for the primary broadcast channel at each PBS station. Partnership stations are broadcasting University Place content on additional, secondary channels. However, the secondary digital broadcast channels do not yet have an industry wide rating system, such as the Nielsen's. It is unclear if this will change within the grant period.

  2. The WPSU, WPT and WOSU Partnership

    Objective: WPSU, WPT and WOSU will form a collaborative partnership to increase access to educational and informational content.

    The collaboration among WPSU, WPT and WOSU will be assessed to identify a set of effective practices that have been developed throughout the course of this project. To this end, the following outcomes will be identified and reported through a series of staff interviews:

    1. Partnership Management - Ability to develop and maintain an effective collaboration with PBS project partners.
    2. Technology Management - Ability to select, utilize and maintain appropriate technology.
    3. Content Management - Ability to develop and maintain an effective collaboration with content experts to provide quality control in content selection; and ability to provide a variety of content platforms to maximize information access.
    4. Project Management - Ability to manage the project from initiation to completion.
    5. Staff Skills - Development of production staff skill set relevant to utilizing new technology in the distribution of video content.
  3. Collaboration Assessment

    Objective: WPSU, WPT and WOSU will form collaborative partnerships with their affiliated universities, to increase university-generated content for local programming and educational services.

    To this end, the assessment will track the content collaborations of stakeholder groups (university participants, K-12 and communities) as well as identify the following outcomes:

    Outcome Method
    Expanded and enhanced adaptations of university-generated content Host Station Interviews
    Shared content among stations Host Station Interviews
    Content available on University Place web site
  4. Communication and Marketing Plan

    Objective: Develop a marketing and communication plan to increase the awareness of the University Place Project throughout each Grantee's broadcast area.

    The effectiveness of the University Place Communication and Marketing Plan will be assessed through tracking the individuals and organizations who have been provided access to University Place documentation and resources through conferences, community engagement events, and other awareness raising outreach activities, to determine depth and reach. Additional outcome measures:

    Outcome Method
    Contributions of Content created by non-public television entities Host Station Interviews
    Content Accessed by non-partner PBS stations Host Station Interviews
  5. Sustainability

    Objective: Develop a resource that is sustainable and provide a set of effective practices that will facilitate project sustainability in similar projects.

    Project sustainability is dependent on the presence of the proper technology, the necessary skill set for University Place staff, reliable, timely and relevant content, and a management infrastructure to coordinate the collaborative partnerships. This assessment will examine sustainability potential through a series of interviews with relevant WPSU, WPT and WOSU staff in each of the following areas:

    • Technology Audit - Identify technology needs for building sustainable resources.
    • Staff Skills Audit - Identify staff skill needs for continued resource management.
    • Content Provision - Identify content management practices likely to lead to sustainability.
    • Partnership Management - Identify partnership management practices associated with sustainability
    • Additional Resources Generated - Identify additional resources generated through the course of this project that can contribute to project sustainability.

Evaluation Schedule

The evaluation of the University Place project occurred in the following three phases:

Phase I - Assessment Design and Planning

  • Establish the criteria for effectiveness for the University Place Project;
  • Design a research methodology to accurately assess this project;
  • Review all baseline data relevant to program effectiveness; and
  • Produce an assessment plan to be included in the Mid-term Project Assessment Report to CPB.

Phase II - Assessment Data Collection and Analysis

  1. Compose and pilot test all data collection instruments;
  2. Coordinate with key staff at all partner organizations to facilitate data collection from representative samples of the target audience;
  3. Provide all data analyses for the project; and
  4. Provide an executive summary of the assessment data to the Project Director.

Phase III - Evaluation Report

  • Compose a detailed summary report of all data obtained in this assessment with an outline for future program assessment;
  • Provide a summary of program strengths and weaknesses; and
  • Provide a set of recommendations for program improvement.