About this role
Job Description
The Manager, Speaking Test Development, is responsible for overseeing the development of Speaking test items and ancillary materials for the WIDA suite of assessments, which are large-scale English language proficiency assessments for multilingual learners in grades 1-12. This position involves reviewing, revising, and approving test items (linguistic content, graphics, scripts, audio files) that align to the WIDA English Language Development Standards Framework, with a focus on the language domain of Speaking.
The Manager, Speaking Test Development, oversees the test development process from start to finish. This position develops test refreshment plans, timelines, and procedures; provides comprehensive feedback to test developers; and signs off on test items content at critical points in the development process. This position also collaborates with the Manager, Rater Training, to oversee the creation of scoring materials for the Speaking test.
ESSENTIAL DUTIES
Item Development
Manage test development staff who write, review, revise, and select Speaking test items based on item specifications, test maps, and blueprintsEvaluate test items for appropriate task setup and design, linguistic complexity, grade-level appropriateness, content correctness, universal accessibility, appropriate scaffolding, fairness, sensitivity, and adherence to style guidesGive clear, constructive, and timely feedback to test developers on test item text, graphics, and layouts using test item specificationsSign off on test item content at the end of each phase of the item development cycleWork in close collaboration with managers and directors across functional teams and partner organizations to coordinate effortsCollaborate on the production of project plans and timelines for test item development in coordination with client and other vendors and oversee their implementationOversee the development and revision of item specifications and sign off on revisionsOversee the creation of training materials to prepare external reviewersEnsure test development staff provide content, administrative, and technical support for external reviewersLead meetings during which stakeholders review the content, fairness, and grade-appropriateness of test items and ensure stakeholder feedback on test items is resolvedWork with test development staff and audio recording studio to cast actors and produce recordings for Speaking test items, overseeing recording sessions and quality monitoringWrite, review, and edit written reports, procedures, and foundational documents for internal and external stakeholdersUpdate and maintain style guides and item development checklistsEnsure test development staff follow quality control and test security proceduresField Testing and Research
Coordinate, conduct, and sign off on quality checks of field test QTI packages and ancillary materialsReview and update field test functional rulesOversee the qualitative analysis of Speaking field test responses and report findings to internal teams and external stakeholdersConduct research, e.g., pilot testing and cognitive lab interviews, in support of test developmentRater Training and Scoring
Collaborate on research and development work for scoring materials related to the Speaking test, including rubrics, rater training presentations and courses, and other scoring guidanceSupport the selection of anchor and training responsesReview score justifications and score appeals, as well as qualifying, recalibration, and validity setsSynthesize information from the scoring of Speaking tasks for use in future developmentOperational Test and Database Management
Review test maps, test forms, ancillary documents, test practice items, and sample items for the publicWrite, review, and maintain test and item design plansReview and maintain the Speaking Test Construct Framework documentEnsure accuracy of data in test item databasesSupervision
Set functional team goals and expectations for Speaking test developmentMonitor progress and performance of staff completing Speaking test development tasksMentor test development staff in their job duties and professional developmentFulfill administrative responsibilities related to hiring/retention and performance reviewsOther duties as assigned
Qualifications
EDUCATION AND EXPERIENCE:
A Manager, Speaking Test Development must have:
A Bachelor’s degree from an accredited college or university, in a relevant field, with at least five years of relevant experience, orA Master’s degree from an accredited college or university, in a relevant field, with at least two years of relevant experience.Expertise in the development of large-scale spoken English language proficiency assessments for K-12 multilingual learners.The Manager, Speaking Test Development must have:
Experience supervising the development of test items in a high-stakes context.Experience managing complex assessment projects.Deep familiarity with the spoken language that K-12 multilingual learners must be able to produce to be successful in U.S. school settings.Experience teaching K-12 multilingual learners in U.S. school contexts.KNOWLEDGE, SKILLS, AND ABILITIES:
The Manager, Speaking Test Development must have:
Apply their understanding of multilingual learners’ English language development in speaking, and of K-12 education standards and practices to the development of language proficiency test items.Coordinate remote staff across multiple tasks to meet deadlines and quality control expectations.Provide constructive, clear feedback that both improves test items and builds staff understanding and skills.Understand complex test designs with multiple interconnected parts.Work independently with minimal support/supervision.Prioritize tasks appropriately, meet deadlines, and work well within a collaborative, fully remote team.Attend closely to details without losing sight of the bigger picture.Demonstrate flexibility in responding to feedback and evolving project priorities.Show leadership to attain team goals.Bring both analytical and creative approaches to problem solving.The ideal candidate for the Manager, Speaking Test Development also has:
Exemplary writing, editing, and proofreading skills in English.Familiarity with using style guides (including the Chicago Manual of Style).Excellent organization, planning, and time management skills.Strong oral and interpersonal communication skills.Proficiency in a language other than English.At least intermediate proficiency with Microsoft Office and Smartsheet, or other work management platforms.Travel Requirements
This position requires occasional travel (1-3 times per year).
COMPENSATION:
Minimum Salary: $77,250 ; Benefits eligible
Full-time
PHYSICAL DEMANDS
The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. While performing the duties of this job, the employee is regularly required to effectively communicate. The employee may be required to sit or stand in a stationary position for 50% of the time. This position constantly operates a computer and other office productivity equipment. CAL adheres to ADA compliance and guidelines as a common practice.
ADDITIONAL INFORMATION
This job description summarizes the main duties of the job. It neither prescribes nor restricts the exact tasks that may be assigned to carry out these duties. This document should not be construed in any way to represent a contract of employment. Management reserves the right to review and revise this document at any time.
AFFIRMATIVE ACTION/EQUAL OPPORTUNITY EMPLOYMENT STATEMENT
The Center for Applied Linguistics (CAL) provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity, disability, or genetics. In addition to federal law requirements, CAL complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities. This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.
CAL expressly prohibits any form of workplace harassment based on race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status. Improper interference with the ability of CAL employees to perform their job duties may result in discipline up to and including discharge.