How the Department Made Determinations — Part C — 2022

H OW THE D EPARTMENT M ADE D ETERMINATIONS UNDER S ECTIONS 616( D) AND 642 OF THE INDIVIDUALS WITH D ISABILITIES E DUCATION A CT IN 2022: P ART C REVISED 06/22/ 2022 INTRODUCTION In 202 2, the U.S. Department of Education (Department) is continuing to use both results and compliance data in making our determination for each State under sections 616(d) and 642 of the Individuals with Disabilities Education Act (IDEA ) for each State’s early intervention program under Part C of the IDEA. We considered the totality of the information we have about a State, including information related to the State’s Federal fiscal year (FFY) 2020 State Performance Plan (SPP)/Annual Performance Report (APR), Indicator C3 Child Outcomes data (Outcomes data) and other data reported in each State’s FFY 2020 SPP/APR; information from monitoring and other publicly available information, such as Specific Conditions on the State’s grant award under Part C; the impact of COVID-19 on the State’s ability to collect and report valid and reliable data, and other issues related to a State’s compliance with the IDEA. In examining each State’s Outcomes data, we sp ecifically considered the following results elements: (1) Data quality by examining — (a) the completeness of the State’s data, and (b) how the State’s FFY 2020 data compared to four years of historic data to identify data anomalies; and (2) Child performance by examin ing— (a) how each State’s FFY 2020 data compared with all other States’ FFY 2020 data, and (b) how each State’s FFY 2020 data compared with its own FFY 2019 data. Below is a detailed description of how the Office of Special Education Programs (OSEP) evaluated St ates’ data using the Results -Driven Accountability (RDA) Matrix. The RDA Matrix is individualized for each State and consists of: (1) a Compliance Matrix that includes scoring on SPP/APR Compliance Indicators and other compliance factors; (2) Results Components and Appendices that include scoring on Results Elements; (3) a Compliance Score and a Results Score ; (4) an RDA Percentage based on both the Compliance Score and the Results Score; and (5) the State’s 2022 Determination . The scoring of each of the above evaluation criteria is further explained below in the following sections: A. 202 2 Part C RDA Matrix and Results Score B. 202 2 Part C Compliance Matrix and Compliance Score; and C. 20 22 RDA Percentage and 202 1 Determination In making the 202 2 determinations based on FFY 20 20 APR data, OSEP specifically considered whether and to what extent States and Entities included in the narrative for each impacted indicator: (1) a 3 description of the impact on data completeness, validity, and/or reliabilit y for the indicator; (2) an explanation of how COVID -19 specifically impacted that State’s or Entity’s ability to collect or verify the data for the indicator; and (3) a description of any steps the State or Entity took to mitigate the impact of COVID -19 o n the data collection and verification. OSEP appreciates States’ and Entities’ level of transparency regarding the impact of COVID -19 on the data reported in the FFY 20 20 SPP/APR. For 2022 determinations, no State or Entity received a determination of “Needs Intervention” due solely to data impacted by COVID -19. A. 202 2 Part C RDA Matrix and Results Score In making each State’s 2022 determination, the Department used the FFY 2020 early childhood outcomes data reported by each State under SPP/APR Indicator C3 by considering the following results elements: 1. Data Quality (a) Data Completeness: Data completeness was calculated using the total number of Part C children who were included in each State’s FFY 20 20 Outcomes data and the total number of children the State reported exiting during FFY 20 20 in its FFY 2020 IDEA Section 618 Exiting data; and (b) Data Anomalies: Data anomalies were calculated by examining how the State’s FFY 20 20 Outcomes data compared to four years of historic data. 2. Child Performance (a) Data Comparison: How each State’s FFY 2020 Outcomes data compared with all other States’ FFY 2020 Outcomes data; and (b) Performance Change Over Time: How each State’s FFY 2020 Outcomes data compared with its own FFY 201 9 Outcomes data. Calculation of each of these results elements and scoring is further described below: 1. Data Quality (a) Data Completeness: The data completeness score was calculated using the total number of Part C children who were included in your State’s FFY 20 20 Outcomes data and the total number of children your State reported exiting during FFY 20 20 in its FFY 20 20 I DEA Section 618 Exiting data. Each State received a percentage, which was computed by dividing the number of children reported in the State’s FFY 2020 Outcomes data by the number of children the State reported exited during FFY 20 20 in the State’s FFY 2020 IDEA Section 618 Exiting Data. This yielded a percentage such that each State received a data completeness score of ‘2’ if the percentage was at least 65% 1; a data 1 In determining the data completeness score, the Department will round up from 64.5% (but no lower) to 65%. Similarly, the Department will round up from 33.5% (but no lower) to 34%. 4 completeness score of ‘1’ if the percentage was between 34% and 64%; and a data completene ss score of ‘0’ if the percentage were less than 34%. For the State with an approved sampling plan, the State received a ‘2’. (Data Sources: FFY 2020 APR Indicator C3 data and EDFacts School Year (SY) 20 20-20 21 ; data extracted 5/26/22.) (b) Data Anomalies: The data anomalies score for each State represents a summary of the data anomalies in each State’s FFY 2020 Outcomes data. Publicly available data for the preceding four years reported by and across all States for each of 15 progress categ ories under Indicator 3 (in the FFY 2016 – FFY 201 9 APRs) were used to determine an expected range of responses for each progress category under Outcomes A, B, and C. 2 For each of the 15 progress categories, a mean was calculated using this publicly available data. A lower and upper scoring percentage was set at one standard deviation above and below the mean for category a and two standard deviations above or below the mean for categories b through e. In any case where the low scoring percentage set from one or two standard deviations below the mean resulted in a negative number, the low scoring percentage is equal to 0. If your State's FFY 20 20 Outcomes data reported i n a progress category fell below the calculated "low percentage" or above the "high percentage" for that progress category for all States, the data in that particular category are statistically improbable outliers and considered an anomaly for that progres s category. If your State’s data in a particular progress category was identified as an anomaly, the State received a ‘0’ for that category. A percentage that is equal to or between the low percentage and high percentage for each progress category received 1 point. A State could receive a total number of points between 0 and 15. Thus, a point total of 0 indicates that all 15 progress categories contained data anomalies and a point total of 15 indicates that there were no data anomalies in all 15 progress ca tegories in the State's data. An overall data anomalies score of ‘0’, ‘1’, or ‘2’ is based on the total points awarded. Each State received a data anomalies score of ‘2’ if the total points received in all progress categories were 13 through 15; a data ano malies score of ‘1’ for 10 through 12 points; and a data anomalies score of ‘0’ for zero through nine points. (Data Sources: States’ FFY 2016 through FFY 2019 SPP/APR Indicator C3 data and each State’s FFY 20 20 Outcomes data) 2 The three Child Outcome areas are: Outcome A (Positive social -emotional skills (including social relationships); Outcome B (Acquisition and use of knowledge and skills (including early language/communication)); and Outcome C (Use of appropriate behaviors to meet their need). The five Progress Categories under SPP/APR Indicato r C3 are the following: a. Percent of infants and toddlers who did not improve functioning b. Percent of infants and toddlers who improved functioning but not sufficient to move nearer to functioning comparable to same -aged peers c. Percent of infants and toddlers who improved functioning to a level nearer to same- aged peers but did not reach it d. Percent of infants and toddlers who improved functioning to reach a level comparable to same- aged peers e. Percent of infants and toddlers who maintained functioning at a level comparable to same- aged peers Outcomes A, B, and C under SPP/APR Indicator C - each contain these five progress categories for a total of 15 progress categories 5 2. Child Performance (a) Data Compar ison: The data comparison overall performance score represents how your State's FFY 20 20 Outcomes data compares to other States' FFY 2020 Outcomes data. Each State received a score for the distribution of the 6 Summary Statements (SS) for that State compared to the distribution of the 6 Summary Statements in all other States. 3 The 10th and 90th percentile for each of the 6 Summary Statements was identified and us ed to assign points to performance outcome data for each Summary Statement. Each Summary Statement outcome was assigned ‘0’, ‘1’, or ‘2’ points. If a State’s Summary Statement value fell at or below the 10th percentile, that Summary Statement was assigned a score of ‘0’. If a State’s Summary Statement value fell between the 10th and 90th percentile, the Summary Statement was assigned ‘1’ point, and if a State’s Summary Statement value fell at or above the 90th percentile, the Summary Statement was assigned ‘2’ points. The points were added across the 6 Summary Statements. A State can receive total points between 0 and 12, with the total points of ‘0 ’ indicating all 6 Summary Statement values were below the 10th percentile and a total points of 12 indicating all 6 Summary Statements were above the 90th percentile. An overall comparison Summary Statement score of ‘0’, ‘1’, or ‘2’ was based on the total points awarded. The data comparison Overall Performance Score for this results element of ‘0’, ‘1’, or ‘2’ for each State is based on the total points awarded. Each State received an Overall Performance Score of: ‘2’ if the total points across SS1 and SS 2 were nine through 12 points; score of ‘1’ for five through eight points; and score of ‘0’ for zero through four points. (Data Sources: All States’ SPP/APR Indicator C3 data from FFY 20 20 and each State’s FFY 20 20 SPP/APR Indicator C3 data.) (b) Performance Change Over Time: The Overall Performance Change Score represents how each State’s FFY 20 20 Outcomes data compared with its FFY 2019 Outcomes data and whether the State’s data demonstrated progress. The data in each Outcome Area is assigned a value of 0 if there was a statistically significant decrease from one year to the next, a value of 1 if there was no significant change, and a value of 2 if there was a statistically significant increase. The specific steps for each State are described in the State’s RDA Matrix. The scores from all 6 Outcome Areas were totaled, resulting in total points ranging from 0 – 12. The Overall Performance Change Score for this results element of ‘0’, ‘1’, or ‘2’ for each State is based on the total points awarded. Each State r eceived an Overall Performance Change Score of: ‘2’ if the total points were eight or above; a score of ‘1’ for four through seven points; and score of ‘0’ for three points or below . Where OSEP has approved a State’s reestablishment of its Indicator C3 Outcom e Area baseline data as 3 Each of the three Chil d Outcome Areas (A, B, and C) are measured by the following two Summary Statements: 1. Of those infants and toddlers who entered or exited early intervention below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turned 3 years of age or exited the program. 2. The percent of infants and toddlers who were functioning within age expectations in each Outcome by the time they turned 3 years of age or exited the program. 6 its data for FFY 20 20, because the State has changed its methodology for collecting this outcome data, the State received a score of ‘N/A’ for this element since determining performance change based on the percentages across these t wo years of data would not be a valid comparison. The points are not included in either the numerator or denominator in the overall calculation of the results score. (Data Source: SPP/APR Indicator C3 data from FFY 2019 and 2020) B. 202 2 Part C Compliance Matrix and Compliance Score In making each State’s 2022 determination, the Department used a Compliance Matrix, reflecting the following compliance data: 1. The State’s FFY 20 20 data for Part C Compliance Indicators 1, 7, 8A, 8B, and 8C (in cluding whether the State reported valid and reliable data for each indicator); and whether the State demonstrated correction of all findings of noncompliance it had identified in FFY 2019 under such indicators; 2. The timeliness and accuracy of data reporte d by the State under sections 616, 618, and 642 of the IDEA ; 3. The State’s FFY 20 20 data, reported under section 618 of the IDEA , for the timeliness of State complaint and due process hearing decisions; 4. Longstanding Noncompliance: The Department considered : a. Whether the Department imposed Specific Conditions on the State’s FFY 20 21 IDEA Part C grant award and those Specific Conditions are in effect at the time of the 2022 determination, and the number of years for which the State’s Part C grant award has bee n subject to Specific Conditions; and b. Whether there are any findings of noncompliance identified in FFY 2018 or earlier by either the Department or the State that the State has not yet corrected. The Compliance Matrix indicates a score of ‘0’, ‘1’, or ‘2’ for each of the compliance indicators in item one above and for each of the additional factors listed in items two through four above. Using the cumulative possible number of points as the denominator, and using as the numerator the actual points the Stat e received in its scoring under these factors, the Compliance Matrix reflects a Compliance Score, which is combined with the Results Score to calculate the State’s RDA percentage and determination. 7 1. Scoring of the Matrix for Compliance Indicators 1, 7, 8A, 8B, and 8C In the 2022 Part C Compliance Matrix, a State received points as follows for each of Compliance Indicators 1, 7, 8A, 8B, and 8C: 4 • Two points, if either: o The State’s FFY 20 20 data for the indicator were valid and reliable, and reflect at least 95% 5 compliance; or o The State’s FFY 2020 data for the indicator were valid and reliable, and reflect at least 90% compliance; and the State identified one or more findings of noncompliance in FFY 201 9 for the indicator, and has demonstrated correction of all findings of noncompliance identified in FFY 2019 for the indicator. Such full correction is indicated in the matrix with a “Yes” in the “Full Correction of Findings of Noncompliance Identified in FFY 2019 ” column. 6 • One point, if the State’s FFY 20 20 data for the indicator were valid and reliable, and reflect at least 75% compliance, and the State did not meet either of the criteria above for two points. • Zero points, under any of the following circumstances: o The State’s FFY 20 20 data for the indicator reflect less than 75% compliance; or o The State’s FFY 20 20 data for the indicator were not valid and reliable; 7 or o The State did not report FFY 20 20 data for the indicator. 8 4 A notation of “N/A” (for “not applicable”) in the “Performance” column for an indicator denotes that the indicator is not applicable to that particular State. The points for that indicator are not included in the denominator for the matrix. 5 In determining whether a State has met t he 95% compliance criterion for these indicators (1, 7, 8A, 8B, and 8C), the Department will round up from 94.5% (but no lower) to 95%. Similarly, in determining whether a State has met the 90% compliance criterion discussed below, the Department will roun d up from 89.5% (but no lower) to 90%. In addition, in determining whether a State has met the 75% compliance criterion discussed below, the Department will round up from 74.5% (but no lower) to 75%. The Department will also apply the rounding rules to the compliance criteria for 95% and 75% for: (1) the timeliness and accuracy of data reported by the State under sections 616, 618, and 642 of the IDEA; (2) the State’s FFY 2020 data, reported under section 618 of the IDEA, for the timeliness of State complaint and due process hearing decisions. 6 A “No” in that column denotes that the State has one or more remaining findings of noncompliance identified in FFY 2019 for which the State has not yet demonstrated correction. An “N/A” (for “not applicable”) in that column denotes that the State did not identify any findings of noncompliance in FFY 2019 for the indicator. 7 If a State’s FFY 2020 data for any compliance indicator are n ot valid and reliable, the matrix so indicates in the “Performance” column, with a corresponding score of “0.” The explanation of why the State’s data are not valid and reliable is contained in the OSEP Response to the State’s FFY 20 20 SPP/APR in the EMAPS SPP/APR reporting tool . 8 If a State reported no FFY 2020 data for any compliance indicator, the matrix so indicates in the “Performance” column, with a corresponding score of 0. 8 2. Scoring of the Matrix for Timely and Accurate State -Reported Data In the 2022 Part C Compliance Matrix, a State received points as follows for Timely and Accurate State- Reported Data 9: • Two points , if the OSEP -calculated percentage reflects at least 95% compliance. • One point, if the OSEP -calculated percentage reflects at least 75% a nd less than 95% compliance . • Zero points, if the OSEP -calculated percentage reflects less than 75% compliance. 3. Scoring of the Matrix for Timely State Complaint Decisions and Timely Due Process Hearing Decisions In the 2022 Part C Compliance Matrix, a State received points as follows for timely State complaint decisions and for timely due process hearings, as reported by the State under section 618 of the IDEA : • Two points, if the State’s FFY 2020 data were valid and reliable , and reflect at least 95% compliance. • One point, if the State’s FFY 2020 data reflect at least 75% and less than 95% compliance. • Zero points, if the State’s FFY 2020 data reflect less than 75% compliance. • Not Applicable (N/A), if the State’s data reflect less than 100% compliance, and there were fewer than ten State complaint decisions or ten due process hearing decisions. 4. Scoring of the Matrix for Long -Standing Noncompliance (Includes Both Uncorrected Identified Noncompliance and Specific Conditions) In the 2022 Part C Compliance Matrix, a State received points as follows for the Long -Standing Noncompliance component: • Two points, if the State has: o No remaining findings of noncompliance identified by OSEP or the State; in FFY 2018 or earlier, and o No Specific Conditions on its FFY 20 21 grant award that are in effect at the time of the 202 2 determination . 9 OSEP used the Part C Timely and Accurate Data Rubric to award points to states based on the timeliness and accuracy of their 616 and 618 data. A copy of the rubric is contained in the OSEP Response to the State’s FFY 2020 SPP/APR in the EMAPS SPP/APR reporting tool . On page one of the rubric, entitled “Part C Timely and Accurate Data” S tates are given one point for each indicator with valid and reliable data and five points for SPP/APRs that were submitted timely. The total points for valid and reliable SPP/APR data and timely submission are added together to form the APR Grand Total. T he State’s 618 data is scored based on information provided to OSEP on 618 data timeliness, completeness and edit checks from EDFacts. T he percentage of Timely and Accurately Reported Data is calculated by adding the 618 Data Grand Total to the APR Grand Total and dividing this sum by the total number of points available for the entire rubric. This percentage is inserted into the Compliance Matrix. 9 • One point , if either or both of the following occurred: o The State has remaining findings of noncompliance, identified by OSEP or the State, in FFY 2018 , FFY 2017, and/or FFY 2016 , for which the State has not yet demonstrated correction (see the FFY 2020 OSEP Response to the State’s FFY 20 20 SPP/APR in the EMAPS SPP/APR reporting tool for specific information regarding these remaining findings of noncompliance); and/ or o The Department has imposed Specific Conditions on the State’s FFY 2021 Part C grant award and those Specific Conditions are in effect at the time of the 202 2 determination. • Zero points, if e ither or both of the following occurred: o The State has remaining findings of noncompliance identified, by OSEP or the State, in FFY 201 5 or earlier, for which the State has not yet demonstrated correction (see the OSEP Response to the State’s FFY 20 20 SPP/APR in the EMAPS SPP/APR reporting tool f or specific information regarding these remaining findings of noncompliance); and/or o The Department has imposed Specific Conditions on the State’s last three (FFYs 2019 , 20 20, and 20 21 ) IDEA Part C grant awards, and those Specific Conditions are in effect at the time of the 202 2 determination. C. 202 2 RDA Percentage and 2022 Determination Each State’s 2022 RDA Percentage was calculated by adding 50% of the State’s Results Score and 50% of the State’s Compliance Score. The State’s RDA Determination is defined a s follows: 1. Meets Requirements A State’s 2022 RDA Determination is Meets Requirements if the RDA Percentage is at least 80%, 10 unless the Department has imposed Specific Conditions on the State’s last three IDEA Part C grant awards (for FFYs 2019 , 20 20, and 2021 ), and those Specific Conditions are in effect at the time of the 2022 determination. 2. Needs Assistance A State’s 2022 RDA Determination is Needs Assistance if the RDA Percentage is at least 60% but less than 80%. A State would also be Needs Assistance if its RDA Determination percentage is 80% or above, but the Department has imposed Specific Conditions on the State’s last three IDEA Part C grant a wards (for FFYs 2019, 20 20, and 2021), and those Specific Conditions are in effect at the time of the 2022 determination. 3. Needs Intervention A State’s 2022 RDA Determination is Needs Intervention if the RDA Percentage is less than 60%. However, if a State’s RDA percentage is less than 60% based on data that were impacted by the COVID -19 pandemic, the State’s 2022 RDA determination is Needs Assistance instead of Needs 10 In determining whether a State has met this 80% matrix criterion for a Meets Requirements determination, the Department will round up from 79.5% (but no lower) to 80%. Similarly, in determining whether a State has met the 60% matrix criterion for a Needs Assistance determination discussed below, the De partment will round up from 59.5% (but no lower) to 60%. 10 Intervention. The Department did not make a determination of Needs Intervention for any State in 2022. 4. Needs Substantial Intervention The Department did not make a determination of Needs Substantial Intervention for any State in 202 2.