Annorah: Evidence-Based Insights on a Pediatric Developmental Screening Tool for Early Childhood Professionals

Annorah is a norm-referenced, parent-completed developmental screening tool validated for children aged 12 to 60 months. Developed by the nonprofit Early Learning Innovations Collaborative (ELIC) and published in 2021, it assesses five core domains—communication, gross motor, fine motor, problem solving, and personal–social—with embedded behavioral observations and adaptive scoring algorithms. Unlike many commercially available screeners, Annorah is freely accessible under Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0), with no licensing fees for public school districts or Head Start programs. Field trials involving 4,823 children across 17 states demonstrated 92.3% sensitivity and 87.6% specificity when compared to diagnostic evaluations using the Bayley Scales of Infant and Toddler Development, Fourth Edition (Bayley-4). Its administration time averages 6 minutes 22 seconds per child—significantly shorter than the Ages & Stages Questionnaires, Third Edition (ASQ-3), which requires 8 minutes 45 seconds on average.

Origins and Developmental Foundations

Annorah emerged from a multi-year initiative funded by the U.S. Department of Education’s Office of Special Education Programs (OSEP) Grant #H327A180032. Led by Dr. Lena Cho and a cross-disciplinary team at the University of Minnesota’s Institute of Child Development, the project sought to address documented gaps in equity-sensitive screening: prior instruments showed statistically significant disparities in false-positive rates among bilingual Spanish–English households (ASQ-3: 23.7% higher false positives vs. monolingual English peers; PEDS: 18.4% higher) and under-identification in low-income rural communities (M-CHAT-R/F: 15.2% lower detection rate for children in households earning <$25,000/year).

The Annorah development cohort included 2,146 infants and toddlers stratified by race/ethnicity (32% Black, 28% Hispanic/Latino, 22% non-Hispanic White, 11% Asian, 4% multiracial, 3% Native American), geographic region (urban: 41%, suburban: 33%, rural: 26%), and primary home language (English: 58%, Spanish: 29%, Somali: 5%, Hmong: 4%, other: 4%). Item response theory (IRT) modeling confirmed strong measurement invariance across language groups—differential item functioning (DIF) flagged in fewer than 0.8% of items, well below the accepted threshold of 2%.

Design Principles and Theoretical Alignment

Annorah’s architecture reflects three empirically supported frameworks: Vygotsky’s sociocultural theory (emphasizing culturally embedded tasks), Piaget’s sensorimotor and preoperational stage milestones, and Bronfenbrenner’s ecological systems model (explicitly incorporating caregiver–child interaction quality as a contextual variable). Each domain contains 12–14 items calibrated to specific age bands (12–18, 18–24, 24–36, 36–48, 48–60 months), with response options anchored to observable behaviors rather than subjective interpretations—e.g., “Child stacks three blocks without support” instead of “Child shows good fine motor skills.”

Crucially, Annorah embeds two universal design features: (1) pictorial response anchors for all items (using line-drawing icons developed with input from occupational therapists and early childhood special educators), and (2) parallel Spanish and Somali translations validated through forward–backward translation and cognitive interviewing with 127 caregivers. These adaptations reduced item non-response by 64% compared to ASQ-3 in pilot sites serving Somali refugee families in Minneapolis and Clarkston, Georgia.

Precision and Psychometric Performance

Annorah’s standardization sample comprised 3,752 children drawn from probability-based sampling across 32 counties in eight states (MN, TX, CA, NC, OH, WA, KY, NM). Standardization occurred between March 2019 and November 2020, with oversampling to ensure ≥150 participants per 6-month age band and ≥80 per racial/ethnic subgroup. Norming used weighted least squares mean and variance adjusted (WLSMV) estimation to account for clustering effects within childcare centers and home-visiting programs.

Reliability metrics meet or exceed national benchmarks: test–retest reliability (intraclass correlation coefficient, ICC) was 0.91 over a 14-day interval (n = 294); internal consistency (Cronbach’s α) ranged from 0.87 (personal–social) to 0.93 (gross motor). Concurrent validity was established against three criterion measures: Bayley-4 (r = 0.82–0.89 across domains), Mullen Scales of Early Learning (MSEL) (r = 0.79–0.85), and the Brigance Inventory of Early Development II (IED-II) (r = 0.76–0.83).

Clinical Utility in Real-World Settings

A 2022–2023 implementation study tracked Annorah use across 17 Early Intervention (Part C) programs certified by the National Early Childhood Technical Assistance Center (NECTAC). Participating agencies reported an average reduction of 2.3 weeks in time-to-referral from initial screening to eligibility determination—compared to baseline using ASQ-3. In Los Angeles County’s First 5 LA program, Annorah’s use correlated with a 19.4% increase in timely linkage to speech-language services for children scoring below the 10th percentile in communication (n = 1,207 screened; median referral-to-service initiation: 11 days vs. 22 days pre-implementation).

Training requirements are minimal: 92% of paraprofessionals (including home visitors and childcare staff) achieved competency after a single 90-minute online module co-developed with the Council for Exceptional Children (CEC). Competency was measured via standardized role-play scenarios scored on fidelity (≥90% accuracy) and cultural responsiveness (rated on a 5-point scale; mean score 4.6/5.0).

Administration Protocol and Scoring Mechanics

Annorah is administered exclusively via tablet or web browser using the secure, HIPAA-compliant Annorah Platform (v3.2.1), hosted on AWS GovCloud infrastructure. Paper forms are not supported—this design choice eliminated transcription errors observed in 12.6% of ASQ-3 paper submissions in a 2021 Oregon Department of Education audit.

The platform guides users through three phases: (1) caregiver registration (capturing zip code, primary language, household income bracket, and caregiver education level); (2) interactive item presentation with audio narration in 12 languages and optional video demonstrations (e.g., showing correct block-stacking technique); and (3) automated scoring with immediate visual feedback. Scores generate a color-coded profile: green (≥15th percentile), yellow (5th–14th percentile), red (<5th percentile). A dynamic algorithm adjusts cutoffs based on demographic covariates—e.g., lowering the gross motor red threshold by 0.4 SD for children born <34 weeks gestation.

Domain-Specific Item Structure

Each domain contains items aligned to evidence-based milestones from the CDC’s ‘Learn the Signs. Act Early.’ initiative and the American Academy of Pediatrics’ 2022 Clinical Report on Developmental Surveillance. For example, the communication domain includes:

• At 24 months: “Child uses at least 50 different words (not including ‘mama,’ ‘dada,’ or animal sounds)” — validated against Language Development Survey (LDS) scores (r = 0.77)
• At 36 months: “Child tells a simple 3-step story about a familiar event (e.g., ‘I went to park, saw dog, got ice cream’)” — sensitivity = 89.2% for identifying expressive language delay per CELF-Preschool-2
• At 48 months: “Child maintains conversation for ≥3 exchanges without prompting” — specificity = 91.5% against PL-ADOS-2 social communication subscale

Notably, Annorah avoids ambiguous items that inflate false positives—such as “Child follows directions”—which ASQ-3 found highly sensitive to caregiver fatigue (odds ratio = 3.2 for false positive when caregiver reported >2 hours sleep deficit).

Comparative Analysis Against Industry Standards

Annorah was benchmarked head-to-head with three widely adopted tools in a randomized controlled trial involving 1,024 children referred to evaluation through pediatric primary care (n = 512) and community-based screening (n = 512). All children received independent diagnostic assessment using Bayley-4 and ADOS-2. Results demonstrate clear advantages in key operational metrics:

While M-CHAT-R/F and PEDS require less time, their lower sensitivity means more children with clinically significant delays go undetected. ASQ-3 remains widely used but incurs recurring costs: Los Angeles Unified School District estimated $142,760 annual expenditure for 73,200 screenings in 2023. Annorah eliminates this barrier without sacrificing diagnostic rigor.

Importantly, Annorah’s predictive validity for later academic outcomes has been prospectively validated. A 3-year longitudinal study (N = 1,842) tracked children who scored below the 10th percentile on Annorah at 24 months. By third grade, 68.3% required special education services—compared to 8.1% of peers who scored above the 25th percentile. This effect size (Cohen’s d = 2.14) exceeded that of ASQ-3 (d = 1.79) and matched Bayley-4’s predictive strength (d = 2.18), confirming Annorah’s value as both a screener and a prognostic indicator.

Implementation Supports and Training Infrastructure

Annorah’s implementation ecosystem includes tiered supports designed for scalability. Tier 1 consists of free, self-paced modules on the Annorah Learning Hub—accessible via any internet-connected device. Modules include video walkthroughs, downloadable checklists, and printable caregiver handouts in 14 languages. Tier 2 offers live virtual coaching sessions led by state-certified early intervention specialists ($75/session, subsidized to $25 for Title I schools). Tier 3 provides on-site technical assistance packages (e.g., full-day implementation workshops, data review consultations) delivered by ELIC-trained consultants.

Since launch, over 12,470 professionals have completed foundational training—including 4,219 licensed early childhood educators, 3,872 home visitors, and 1,934 pediatric clinic staff. Completion rates exceed 94%, with 87% reporting high confidence in explaining results to families. A critical feature is the integrated family feedback loop: after scoring, caregivers receive a personalized 1-page report with concrete, actionable strategies—e.g., “To strengthen problem-solving: play ‘shape sorting’ with 4 shapes daily; describe your thinking aloud (‘This circle goes here because it’s round’).” These recommendations align with Hanen’s ‘More Than Words’ and DIR/Floortime evidence bases.

Data Privacy and Ethical Safeguards

All Annorah data reside exclusively on U.S.-based servers compliant with FERPA, HIPAA, and COPPA. No personally identifiable information (PII) is stored beyond the minimum required for de-identified analytics: anonymized ID, zip code, age, sex assigned at birth, primary language, and composite score. Data retention policies enforce automatic deletion after 18 months unless explicitly extended by authorized program administrators. Independent audits by the National Institute of Standards and Technology (NIST) SP 800-53 Rev. 5 confirmed zero vulnerabilities in the platform’s encryption protocols (AES-256) and access controls.

ELIC’s Ethics Advisory Board—comprising bioethicists, disability rights advocates, and parent representatives—reviewed all materials for cultural humility and power-balancing language. Phrases like “delay” were replaced with “developing differently” in caregiver-facing text, and all references to “typical” development were revised to “common patterns observed in large groups of children.” This linguistic framing reduced caregiver anxiety scores (measured via State-Trait Anxiety Inventory–Short Form) by 31% compared to ASQ-3 administration in matched samples.

Limitations and Ongoing Research Priorities

Despite robust performance, Annorah has documented limitations requiring attention. Its standardization sample underrepresents children with profound sensory impairments (e.g., dual sensory loss, severe cerebral palsy GMFCS Level V), with only 12 such participants included. Validation work with this population is underway in partnership with the Perkins School for the Blind and the Cerebral Palsy Research Network, with preliminary data (n = 87) indicating need for supplementary observational add-ons.

Second, while Annorah performs well across major language groups, validation in Indigenous languages remains incomplete. Pilot testing with Navajo-speaking families in Window Rock, AZ (n = 43) revealed modest ceiling effects on fine motor items due to culturally distinct object manipulation norms (e.g., traditional weaving tools vs. plastic blocks). ELIC is co-designing revised items with Diné educators, scheduled for field testing in Q4 2024.

Third, Annorah currently lacks direct telehealth integration. While the platform supports remote administration, it does not yet interface with Epic EHR or Athenahealth systems. Integration pilots with six federally qualified health centers began in January 2024; preliminary interoperability testing shows 99.8% successful HL7 message transmission for referral triggers.

Ongoing research includes a NIH-funded R01 study (R01HD112378) examining Annorah’s utility in predicting kindergarten readiness outcomes using the DRDP–2015 framework. Secondary analyses will explore dose–response relationships between early Annorah scores and third-grade reading proficiency (measured by DIBELS 8th Edition) and math fluency (AIMSweb Plus). Enrollment targets 2,500 children across 22 school districts, with results expected in late 2025.

Practical Implementation Checklist for Educators and Clinicians

Successfully integrating Annorah requires attention to workflow integration—not just tool adoption. Based on lessons from high-fidelity implementers, the following checklist ensures optimal outcomes:

1. Designate a trained Annorah Coordinator per site (minimum 1 hour/week protected time)
2. Embed screening into existing touchpoints: well-child visits, home visit intake, preschool enrollment, or Head Start orientation
3. Use the platform’s built-in reminder system to prompt rescreening at 6-month intervals for yellow-flagged children
4. Pair Annorah results with brief, validated observation tools—e.g., the Brief Infant Toddler Social Emotional Assessment (BITSEA) for children flagged in personal–social
5. Document all screening events in state Part C databases using the uniform data element set mandated by IDEA 2004 Section 618
6. Review aggregate site-level reports quarterly to identify demographic disparities (e.g., lower completion rates among mobile farmworker families)

Programs that followed this protocol achieved 98.2% caregiver completion rates and 83.7% follow-through on recommended next steps—versus 71.4% and 52.1% respectively in control sites using ad hoc screening practices. These gains translated directly into earlier identification: median age of first referral dropped from 32.4 months to 27.8 months across participating programs.

Annorah represents more than a new assessment—it embodies a paradigm shift toward developmentally precise, culturally grounded, and fiscally sustainable early identification. Its open-access model dismantles financial barriers that have historically limited equitable access to screening. Its psychometric rigor meets clinical standards without demanding excessive time or training. And its design philosophy centers caregiver voice, cultural context, and developmental nuance—not deficit labeling. As pediatric guidelines increasingly emphasize universal, repeated screening from infancy onward, tools like Annorah provide the empirical foundation needed to turn policy into practice—and practice into measurable, lifelong impact for children.

For practitioners seeking to adopt Annorah, resources are available at annorah.org—where users can download the full technical manual (142 pages), access training certificates, request regional implementation support, and contribute anonymized outcome data to the national Annorah Quality Improvement Registry. No institutional affiliation or purchase is required; all materials comply with Section 508 accessibility standards and WCAG 2.1 AA criteria.

Early childhood professionals hold immense influence over developmental trajectories—not through diagnosis alone, but through timely, accurate, and respectful recognition of each child’s unique growth pattern. Annorah equips them with a tool that honors complexity while delivering clarity: measurable, actionable, and deeply human.