How the Department Made Determinations — Part C 2021

H OW THE D EPARTMENT M ADE D ETERMINATIONS UNDER S ECTIONS 616( D) AND 642 OF THE INDIVIDUALS WITH D ISABILITIES E DUCATION A CT IN 2021 : P ART C REVISED 06 /2 2/2021 INTRODUCTION In 2021 , the U.S. Department of Education (Department) is continuing to use both results and compliance data in making our determination for each State under sections 616(d) and 642 of the Individuals with Disabilities Education Act ( IDEA ) for each State’s early intervention program under Part C of the IDEA . We considered the totality of the information we have about a State, including information related to the State’s Federal fiscal year (FFY) 2019 State Performance Plan (SPP)/Annual Performance Report (APR), Indicator C3 Child Outcomes data (Outcomes data) and other data reported in each State’s FFY 2019 SPP/APR; information from monitoring and other publicly available information, such as Specific Conditions on the State’s grant award under Part C; the impact of COVID -19 on the State’s ability to collect and report valid and reliable data, and other issues related to a State’s compliance with the IDEA. In examining each State’s Outcomes data, we sp ecifically considered the following results elements: (1) Data quality by examining — (a) the completeness of the State’s data, and (b) how the State’s FFY 201 9 data compared to four years of historic data to identify data anomalies; and (2) Child performance by examin ing— (a) how each State’s FFY 2019 data compared with all other States’ FFY 2019 data, and (b) how each State’s FFY 2019 data compared with its own FFY 2018 data. Below is a detailed description of how the Office of Special Education Programs (OSEP) evaluated St ates’ data using the Results -Driven Accountability (RDA) Matrix. The RDA Matrix is individualized for each State and consists of: (1) a Compliance Matrix that includes scoring on SPP/APR Compliance Indicators and other compliance factors; (2) Results Components and Appendices that include scoring on Results Elements; (3) a Compliance Score and a Results Score ; (4) an RDA Percentage based on both the Compliance Score and the Results Score; and (5) the State’s 2021 Determination. The scoring of each of the above evaluation criteria is further explained below in the following sections: A. 2021 Part C RDA Matrix and Results Score B. 2021 Part C Compliance Matrix and Compliance Score; and C. 202 1 RDA Percentage and 2021 Determination In making the 2021 determinations based on FFY 2019 APR data, OSEP specifically considered whether and to what extent States and Entities included in the narrative for each impacted indicator: (1) a 3 description of the impact on data completeness, validity, and/or reliability for the indicator; (2) an explanation of how COVID -19 specifically impacted that State’s or Entity’s ability to collect or verify the data for the indicator; and (3) a description of any steps the State or Entity took to mitigate the im pact of COVID -19 on the data collection and verification. OSEP appreciates States’ and Entities’ level of transparency regarding the impact of COVID -19 on the data reported in the FFY 2019 SPP/APR. For 2021 determinations, no State or Entity received a det ermination of “Needs Intervention” due solely to data impacted by COVID -19. A. 2021 Part C RDA Matrix and Results Score In making each State’s 2021 determination, the Department used the FFY 2019 early childhood outcomes data reported by each State under SPP/APR Indicator C3 by considering the following results elements: 1. Data Quality (a) Data Completeness: Data completeness was calculated using the total number of Part C children who were included in each State’s FFY 2019 Outcomes data and the total number of children the State reported exiting during FFY 2019 in its FFY 2019 IDEA Section 618 Exiting data; and (b) Data Anomalies: Data anomalies were calculated by examining how the State’s FFY 2019 Outcomes data compared to four years of historic data. 2. Child Performance (a) Data Comparison: How each State’s FFY 201 9 Outcomes data compared with all other States’ FFY 201 9 Outcomes data; and (b) Performance Change Over Time: How each State’s FFY 201 9 Outcome s data compared with its own FFY 2018 Outcomes data. Calculation of each of these results elements and scoring is further described below: 1. Data Quality (a) Data Completeness: The data completeness score was calculated using the total number of Part C children who were included in your State’s FFY 2019 Outcomes data and the total number of children your State reported exiting during FFY 2019 in its FFY 2019 IDEA Section 618 Exiting data. Each State received a percentage, which was computed by dividing the number of children reported in the State’s FFY 201 9 Outcomes data by the number of children the State reported exited during FFY 2019 in the State’s FFY 201 9 IDEA Section 618 Exiting Data. This yielded a percentage such that each State received a data completeness score of ‘2’ if the percentage was at least 65% 1; a data 1 In determining the data completeness score, the Department will round up from 64.5% (but no lower) to 65%. Similarly, the Department will round up from 33.5% (but no lower) to 34%. 4 completeness score of ‘1’ if the percentage was between 34% and 64%; and a data completeness score of ‘0’ if the percentage were less than 34%. For the two States with approved samplin g plans, the State received a ‘2’. (Data Sources: FFY 2019 APR Indicator C3 data and EDFacts School Year (SY) 2019 -2020; data extracted 5/26/2021.) (b) Data Anomalies: The data anomalies score for each State represents a summary of the data anomalies in each State’s FFY 201 9 Outcomes data. Publicly available data for the preceding four years reported by and across all States for each of 15 progress categories under Indic ator 3 (in the FFY 2015 – FFY 2018 APRs) were used to determine an expected range of responses for each progress category under Outcomes A, B, and C. 2 For each of the 15 progress categories, a mean was calculated using this publicly available data. A low er and upper scoring percentage was set at one standard deviation above and below the mean for category a and two standard deviations above or below the mean for categories b through e. In any case where the low scoring percentage set from one or two stand ard deviations below the mean resulted in a negative number, the low scoring percentage is equal to 0. If your State's FFY 201 9 Outcomes data reported in a progress category fell below the calculated "low percentage" or above the "high percentage" for that progress category for all States, the data in that particular category are statistically improbable outliers and considered an anomaly for that progress category. If your State’s data in a particular progress category was identified as an anomaly, the Sta te received a ‘0’ for that category. A percentage that is equal to or between the low percentage and high percentage for each progress category received 1 point. A State could receive a total number of points between 0 and 15. Thus, a point total of 0 indi cates that all 15 progress categories contained data anomalies and a point total of 15 indicates that there were no data anomalies in all 15 progress categories in the State's data. An overall data anomalies score of ‘0’, ‘1’, or ‘2’ is based on the total points awarded. Each State received a data anomalies score of ‘2’ if the total points received in all progress categories were 13 through 15; a data anomalies score of ‘1’ for 10 through 12 points; and a data anomalies score of ‘0’ for zero through nine po ints. (Data Sources: States’ FFY 2015 through FFY 2018 SPP/APR Indicator C3 data and each State’s FFY 2019 Outcomes data) 2 The three Child Outcome areas are: Outcome A (Positive social -emotional skills (including socia l relationships); Outcome B (Acquisition and use of knowledge and skills (including early language/communication)); and Outcome C (Use of appropriate behaviors to meet their need). The five Progress Categories under SPP/APR Indicator C3 are the following: a. Percent of infants and toddlers who did not improve functioning b. Percent of infants and toddlers who improved functioning but not sufficient to move nearer to functioning comparable to same -aged peers c. Percent of infants and toddlers who improved functionin g to a level nearer to same-aged peers but did not reach it d. Percent of infants and toddlers who improved functioning to reach a level comparable to same -aged peers e. Percent of infants and toddlers who maintained functioning at a level comparable to same -aged peers Outcomes A, B, and C under SPP/APR Indicator C - each contain these five progress categories for a total of 15 progress categories 5 2. Child Performance (a) Data Comparison: The data comparison overall performance score represents how your State's FFY 2019 Outcomes data compares to other States' FFY 2019 Outcomes data. Each State received a score for the distribution of the 6 Summary Statements (SS) for that State compared to the distribution of the 6 Summary Statements in all other States. 3 The 10th and 90th percentile for each of the 6 Summary Statements was identified and us ed to assign points to performance outcome data for each Summary Statement. Each Summary Statement ou tcome was assigned ‘0’, ‘1’, or ‘2’ points. If a State’s Summary Statement value fell at or below the 10th percentile, that Summary Statement was assigned a score of ‘0’. If a State’s Summary Statement value fell between the 10th and 90th percentile, the Summary Statement was assigned ‘1’ point, and if a State’s Summary Statement value fell at or above the 90th percentile, the Summary Statement was assigned ‘2’ points. The points were added across the 6 Summary Statements. A State can receive total points between 0 and 12, with the total points of ‘0 ’ indicating all 6 Summary Statement values were below the 10th percentile and a total points of 12 indicating all 6 Summary Statements were above the 90th percentile. An overall comparison Summary Statement score of ‘0’, ‘1’, or ‘2’ was based on the total points awarded. The data comparison Overall Performance Score for this results element of ‘0’, ‘1’, or ‘2’ for each State is based on the total points awarded. Each State received an Overall Performance Score of: ‘2’ if the total points across SS1 and SS 2 were nine through 12 points; score of ‘1’ for five through eight points; and score of ‘0’ for zero through four points. (Data Sources: All States’ SPP/APR Indicator C3 data from FFY 201 9 and each State’s FFY 201 9 SPP/APR Indicator C3 data.) (b) Performance Change Over Time: The Overall Performance Change Score represents how each State’s FFY 201 9 Outcomes data compared with its FFY 201 8 Outcomes data and whether the State’s data demonstrated progress. The data in each Outcome Area is assigned a value of 0 if there was a statistically significant decrease from one year to the next, a value of 1 if there was no significant change, and a val ue of 2 if there was a statistically significant increase. The specific steps for each State are described in the State’s RDA Matrix. The scores from all 6 Outcome Areas were totaled, resulting in total points ranging from 0 – 12. The Overall Performance C hange Score for this results element of ‘0’, ‘1’, or ‘2’ for each State is based on the total points awarded. Each State received an Overall Performance Change Score of: ‘2’ if the total points were eight or above; a score of ‘1’ for four through seven poi nts; and score of ‘0’ for below three points. Where OSEP has approved a State’s reestablishment of its Indicator C3 Outcome Area baseline data as its 3 Each of the three Chil d Outcome Areas (A, B, and C) are measured by the following two Summary Statements: 1. Of those infants and toddlers who entered or exited early intervention below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turned 3 years of age or exited the program. 2. The percent of infants and toddlers who were functioning within age expectations in each Outcome by the time they turned 3 years of age or exited the program. 6 data for FFY 2019 , because the State has changed its methodology for collecting this outcome data, the Sta te received a score of ‘N/A’ for this element since determining performance change based on the percentages across these two years of data would not be a valid comparison. The points are not included in either the numerator or denominator in the overall ca lculation of the results score. (Data Source: SPP/APR Indicator C3 data from FFY 2018 and 2019 ) B. 2021 Part C Compliance Matrix and Compliance Score In making each State’s 2021 determination, the Department used a Compliance Matrix, reflecting the following compliance data: 1. The State’s FFY 2019 data for Part C Compliance Indicators 1, 7, 8A, 8B, and 8C (including whether the State reported valid and reliable data for each indicator); and whether the State demonstrated correction of all findings of noncompli ance it had identified in FFY 2018 under such indicators; 2. The timeliness and accuracy of data reported by the State under sections 616, 618, and 642 of the IDEA ; 3. The State’s FFY 2019 data, reported under section 618 of the IDEA , for the timeliness of State complaint and due process hearing decisions; 4. Longstanding Noncompliance: The Department considered: a. Whether the Department imposed Specific Conditions on the State’s FFY 2020 IDEA Part C grant award and those Specific Conditions are in effect at the time of the 2021 determination, and the number of years for which the State’s Part C grant award has been subject to Specific Conditions; and b. Whether there are any findings of noncompliance identified in FFY 2017 or earlier by either the Department or the State that the State has not yet corrected. The Compliance Matrix indicates a score of ‘0’, ‘1’, or ‘2’ for each of the compliance indicators in item one above and for each of the additional factors listed in items two thr ough four above. Using the cumulative possible number of points as the denominator, and using as the numerator the actual points the State received in its scoring under these factors, the Compliance Matrix reflects a Compliance Score, which is combined wit h the Results Score to calculate the State’s RDA percentage and determination. 7 1. Scoring of the Matrix for Compliance Indicators 1, 7, 8A, 8B, and 8C In the 2021 Part C Compliance Matrix, a State received points as follows for each of Compliance Indicators 1, 7, 8A, 8B, and 8C: 4 • Two points, if either: o The State’s FFY 2019 data for the indicator were valid and reliable, and reflect at least 95% 5 compliance; or o The State’s FFY 2019 data for the indicator were valid and reliable, and reflect at least 90% compliance; and the State identified one or more findings of noncompliance in FFY 2018 for the indicator, and has demonstrated correction of all findings of noncompliance identified in FFY 2018 for the indicator. Such full correction is indicated in the matrix with a “Yes” in the “Full Correction of Findings of Noncompliance Identified in FFY 2018 ” column. 6 • One point, if the State’s FFY 2019 data for the indicator were valid and reliable, and reflect at least 75% compliance, and the State did not meet either of the criteria above for two points. • Zero points, under any of the following circumstances: o The State’s FFY 2019 data for the indicator reflect less than 75% compliance; or o The State’s FFY 2019 data for the indicator were not va lid and reliable; 7 or o The State did not report FFY 2019 data for the indicator. 8 4 A notation of “N/A” (for “not applicable”) in the “Performance” column for an indicator denotes that the indicator is not applicable to that particular State. The points for that indicator are not included in the denominator for the matrix. 5 In determining whether a State has met t he 95% compliance criterion for these indicators (1, 7, 8A, 8B, and 8C), the Department will round up from 94.5% (but no lower) to 95%. Similarly, in determining whether a State has met the 90% compliance criterion discussed below, the Department will roun d up from 89.5% (but no lower) to 90%. In addition, in determining whether a State has met the 75% compliance criterion discussed below, the Department will round up from 74.5% (but no lower) to 75%. The Department will also apply the rounding rules to the compliance criteria for 95% and 75% for: (1) the timeliness and accuracy of data reported by the State under sections 616, 618, and 642 of the IDEA; (2) the State’s FFY 2019 data, reported under section 618 of the IDEA, for the timeliness of State compl aint and due process hearing decisions. 6 A “No” in that column denotes that the State has one or more remaining findings of noncompliance identified in FFY 2018 for which the State has not yet demonstrated correction. An “N/A” (for “not applicable”) in that column denotes that the State did not identify any findings of noncompliance in FFY 2018 for the indicator. 7 If a State’s FFY 2019 data for any compliance indicator are not valid and reliable, the matrix so indicat es in the “Performance” column, with a corresponding score of “0.” The explanation of why the State’s data are not valid and reliable is contained in the OSEP Response to the State’s FFY 2019 SPP/APR in the EMAPS SPP/APR reporting tool . 8 If a State repor ted no FFY 2019 data for any compliance indicator, the matrix so indicates in the “Performance” column, with a corresponding score of 0. 8 2. Scoring of the Matrix for Timely and Accurate State -Reported Data In the 2021 Part C Compliance Matrix, a State received points as follows for Timely and Accurate State -Reported Data 9: • Two points , if the OSEP -calculated percentage reflects at least 95% compliance. • One point, if the OSEP -calculated percentage reflects at least 75% and less than 95% compliance . • Zero points, if the OSEP -calculated percentage reflect s less than 75% compliance. 3. Scoring of the Matrix for Timely State Complaint Decisions and Timely Due Process Hearing Decisions In the 2021 Part C Compliance Matrix, a State received points as follows for timely State complaint decisions and for timely due process hearings, as reported by the State under section 618 of the IDEA : • Two points, if the State’s FFY 2019 data were valid and reliable, and reflect at least 95% compliance. • One point, if the State’s FFY 2019 data reflect at least 75% and less than 9 5% compliance. • Zero points, if the State’s FFY 2019 data reflect less than 75% compliance. • Not Applicable (N/A), if the State’s data reflect less than 100% compliance, and there were fewer than ten State complaint decisions or ten due process hearing decisions. 4. Scoring of the Matrix for Long -Standing Noncompliance (Includes Both Uncorrected Identified Noncompliance and Specific Conditions) In the 2021 Part C Compliance Mat rix, a State received points as follows for the Long -Standing Noncompliance component: • Two points, if the State has: o No remaining findings of noncompliance identified by OSEP or the State; in FFY 2017 or earlier, and o No Specific Conditions on its FFY 202 0 grant award that are in effect at the time of the 2021 determination . 9 OSEP used the Part C Timely and Accurate Data Rubric to award points to states based on the timeliness and accuracy of their 616 and 618 data. A copy of the rubric is contained in the OSEP Response to the State’s FFY 2019 SPP/APR in the EMAPS SPP/APR reporting tool . On the second page of the rubric, entitled “Part C Timely and Accurate Data SPP/APR Data” states are given one point for each indicator with valid and reliable data and five points for SPP/APRs that were submitted timely. The total points for valid and reliable SPP/APR data and timely submission are added together to form the APR Grand Total. On page t hree of the rubric, the State’s 618 data is scored based on information provided to OSEP on 618 data timeliness, completeness and edit checks from EDFacts. On page four of the rubric, the percentage of Timely and Accurately Reported Data is calculated by adding t he 618 Data Grand Total to the APR Grand Total and dividing this sum by the total number of points available for the entire rubric. This percentage is inserted into the Compliance Matrix. 9 • One point , if either or both of the following occurred: o The State has remaining findings of noncompliance, identified by OSEP or the State, in FFY 2017 , FFY 2016 , and/or FFY 2015 , for which the State has not yet demonstrated correction (see the FFY 2019 OSEP Response to the State’s FFY 201 9 SPP/APR in the EMAPS SPP/APR reporting tool for specific information regarding these remaining findings of noncompliance); an d/or o The Department has imposed Specific Conditions on the State’s FFY 2020 Part C grant award and those Specific Conditions are in effect at the time of the 2021 determination. • Zero points, if e ither or both of the following occurred: o The State has remai ning findings of noncompliance identified, by OSEP or the State, in FFY 201 4 or earlier, for which the State has not yet demonstrated correction (see the OSEP Response to the State’s FFY 2019 SPP/APR in the EMAPS SPP/APR reporting tool for specific informa tion regarding these remaining findings of noncompliance); and/or o The Department has imposed Specific Conditions on the State’s last three (FFYs 2018 , 2019 , and 20 20) IDEA Part C grant awards, and those Specific Conditions are in effect at the time of the 2021 determination. C. 2021 RDA Percentage and 2021 Determination Each State’s 2021 RDA Percentage was calculated by adding 50% of the State’s Results Score and 50% of the State’s Compliance Score. The State’s RDA Determination is defined as follows: 1. Meets Requirements A State’s 2021 RDA Determination is Meets Requirements if the RDA Pe rcentage is at least 80%, 10 unless the Department has imposed Specific Conditions on the State’s last three IDEA Part C grant awards (for FFYs 2018 , 2019 , and 2020), and those Specific Conditions are in effect at the time of the 2021 determination. 2. Needs Assistance A State’s 2021 RDA Determination is Needs Assistance if the RDA Percentage is at least 60% but less than 80%. A State would also be Needs Assistance if its RDA Determination percentage is 80% or above, but the Department has imposed Speci fic Conditions on the State’s last three IDEA Part C grant awards (for FFYs 2018 , 2019 , and 2020), and those Specific Conditions are in effect at the time of the 2021 determination. 3. Needs Intervention A State’s 2021 RDA Determination is Needs Intervention if the RDA Percentage is less than 60%. 10 In determining whether a State has met this 80% matrix criterion for a Meets Requirements determination, the Department will round up from 79.5% (but no lower) to 80%. Similarly, in determining whether a State has met the 60% matrix criterion for a Needs Assistance determination discussed below, the Department will round up from 59.5% (but no lower) to 60%. 10 4. Needs Substantial Intervention The Department did not make a determination of Needs Substantial Intervention for any State in 2021 .