Under Section 616(d) of the
Individuals with Disabilities Education Act in 2023:
Freely Associated States, Outlying Areas, and the
Bureau of Indian Education
Part B

Revised 06/23/2023

Introduction

In 2023, the U.S. Department of Education (Department) is continuing to use both results and compliance data in making our determination for each freely associated State, outlying area, and the Bureau of Indian Education (BIE) (Entities) under Section 616(d) of the Individuals with Disabilities Education Act (IDEA). We considered the totality of the information we have about an Entity, including information related to the participation of children with disabilities (CWD) on regular Statewide assessments; exiting data on CWD who dropped out and CWD who graduated with a regular high school diploma;[1] the Entity’s Federal fiscal year (FFY) 2021 State Performance Plan/Annual Performance Report (SPP/APR); information from monitoring and other public information, such as Specific Conditions on the Entity’s grant award under IDEA Part B; and other issues related to the Entity’s compliance with the IDEA. Below is a detailed description of how the Office of Special Education Programs (OSEP) evaluated the Entities’ data using the Results Driven Accountability (RDA) Matrix.

The RDA Matrix consists of:

1. a Compliance Matrix that includes scoring on SPP/APR Compliance Indicators and other compliance factors;
2. a Results Matrix that includes scoring on Results Elements;
3. a Compliance Score and a Results Score;
4. an RDA Percentage based on the Compliance Score and the Results Score; and
5. the Entity’s Determination.

The scoring of each of the above evaluation criteria is further explained below in the following sections:

1. 2023 Part B Compliance Matrix and Scoring of the Compliance Matrix
2. 2023 Part B Results Matrix and Scoring of the Results Matrix
3. 2023 RDA Percentage and 2023 Determination

A. 2023 Part B Compliance Matrix

In making each Entity’s 2023 determination, the Department used a Compliance Matrix, reflecting the following data:

1. The Entity’s FFY 2021 data for applicable Part B Compliance Indicators[2] 4B, 9, 10, 11, 12, and 13 (including whether the Entity reported valid and reliable data for each indicator); and whether the Entity demonstrated correction of all findings of noncompliance it had identified in FFY 2020 under such indicators;
2. The timeliness and accuracy of data reported by the Entity under Sections 616 and 618 of the IDEA;
3. The Entity’s FFY 2021 data, reported under Section 618 of the IDEA, for the timeliness of State complaint and due process hearing decisions;
4. Longstanding Noncompliance:

The Department considered:

1. 1. Whether OSEP imposed programmatic Specific Conditions on the Entity’s FFY 2022 IDEA Part B grant award and those Specific Conditions are in effect at the time of the 2023 determination, and the number of years for which the Entity’s Part B grant award has been subject to programmatic Specific Conditions; and
   2. Whether there are any findings of noncompliance identified in FFY 2019 or earlier by the Entity that the Entity has not yet corrected.

Scoring of the Compliance Matrix

The Compliance Matrix indicates a score of 0, 1, or 2, for each of the compliance indicators in item one above and for each of the additional factors listed in items two through four above. Using the cumulative possible number of points as the denominator, and using as the numerator the actual points the Entity received in its scoring under these factors, the Compliance Matrix reflects a Compliance Score, which is combined with the Results Score to calculate the Entity’s RDA Percentage and Determination.

Scoring of the Matrix for Compliance Indicators 4B, 9, 10, 11, 12, and 13

In the attached Entity-specific 2023 Part B Compliance Matrix, an Entity received points as follows for each of the Compliance Indicators 4B, 9, 10, 11, 12, and 13:[3]

• Two points, if either:
  • The Entity’s FFY 2021 data for the indicator were valid and reliable, and reflect at least 95%[4] compliance (or, for Indicators 4B, 9, and 10, reflect no greater than 5% compliance);[5] or
  • The Entity’s FFY 2021 data for the indicator were valid and reliable, and reflect at least 90% compliance (or, for Indicators 4B, 9, and 10, reflect no greater than 10% compliance); and the Entity identified one or more findings of noncompliance in FFY 2020 for the indicator, and has demonstrated correction of all findings of noncompliance identified in FFY 2020 for the indicator. Such full correction is indicated in the matrix with a “Yes”) in the “Full Correction of Findings of Noncompliance Identified in FFY 2020” column.[6]
• One point, if the Entity’s FFY 2021 data for the indicator were valid and reliable, and reflect at least 75% compliance (or, for Indicators 4B, 9, and 10, reflect no greater than 25% compliance), and the Entity did not meet either of the criteria above for two points.
• Zero points, under any of the following circumstances:
  • The Entity’s FFY 2021 data for the indicator reflect less than 75% compliance (or, for Indicators 4B, 9, and 10, reflect greater than 25% compliance); or
  • The Entity’s FFY 2021 data for the indicator were not valid and reliable;[7] or
  • The Entity did not report FFY 2021 data for the indicator.[8]

With the FFY 2021 SPP/APR submissions, OSEP conducted a comprehensive analysis of each Entity’s methodology for Indicators 4A and 4B to ensure it is reasonably designed to determine if significant discrepancies are occurring in the rate of long-term suspensions and expulsions of children with disabilities among LEAs in the Entity or compared to the rates for nondisabled children within those LEAs. In OSEP’s response to the FFY 2021 SPP/APR submissions, OSEP provided Entities with detailed feedback about their methodology for Indicators 4A and 4B.[9] In light of this detailed analysis and feedback on Entities’ FFY 2021 SPP/APR submissions, OSEP determined that, while Indicator 4B data continued to be a factor in each Entity’s 2023 Part B Compliance Matrix, no Entity’s 2023 determination was negatively impacted due solely to its Indicator 4B data.

Scoring of the Matrix for Timely and Accurate Entity-Reported Data

In the attached Entity-specific 2023 Part B Compliance Matrix, an Entity received points as follows for Timely and Accurate Entity-Reported Data[10]:

• Two points, if the OSEP-calculated percentage reflects at least 95% compliance.
• One point, if the OSEP-calculated percentage reflects at least 75% and less than 95% compliance.
• Zero points, if the OSEP-calculated percentage reflects less than 75% compliance.

Scoring of the Matrix for Timely State Complaint Decisions and
Timely Due Process Hearing Decisions

In the attached Entity-specific 2023 Part B Compliance Matrix, an Entity received points as follows for timely State complaint decisions and for timely due process hearing decisions, as reported by the Entity under Section 618 of the IDEA:

• Two points, if the Entity’s FFY 2021 data were valid and reliable, and reflect at least 95% compliance.
• One point, if the Entity’s FFY 2021 data reflect at least 75% and less than 95% compliance.
• Zero points, if the Entity’s FFY 2021 data reflect less than 75% compliance.
• Not Applicable (N/A), if the Entity’s data reflect less than 100% compliance, and there were fewer than ten State complaint decisions or ten due process hearing decisions.

Scoring of the Matrix for Longstanding Noncompliance
(Includes Both Uncorrected Identified Noncompliance and Specific Conditions)

In the attached Entity-specific 2023 Part B Compliance Matrix, an Entity received points as follows for the Longstanding Noncompliance component:

• Two points, if the Entity has:
• No remaining findings of noncompliance identified, by the Entity, in FFY 2019 or earlier; and
• No programmatic Specific Conditions on its FFY 2022 IDEA Part B grant award that are in effect at the time of the 2023 determination.
• One point, if either or both of the following occurred:
• The Entity has remaining findings of noncompliance identified, by the Entity, in FFY 2019, FFY 2018, and/or FFY 2017, for which the Entity has not yet demonstrated correction (see the OSEP Response to the Entity’s FFY 2021 SPP/APR in the EMAPS SPP/APR reporting tool for specific information regarding these remaining findings of noncompliance); and/or
• OSEP has imposed programmatic Specific Conditions on the Entity’s FFY 2022 IDEA Part B grant award and those Specific Conditions are in effect at the time of the 2023 determination.
• Zero points, if either or both of the following occurred:
• The Entity has remaining findings of noncompliance identified, by the Entity, in FFY 2016 or earlier, for which the Entity has not yet demonstrated correction (see the OSEP Response to the Entity’s FFY 2021 SPP/APR in the EMAPS SPP/APR reporting tool for specific information regarding these remaining findings of noncompliance); and/or
• OSEP has imposed programmatic Specific Conditions on the Entity’s last three (FFYs 2020, 2021, and 2022) IDEA Part B grant awards, and those Specific Conditions are in effect at the time of the 2023 determination.

B. 2023 Part B Results Matrix

In making each Entity’s 2023 determination, the Department used a Results Matrix reflecting the following data:

1. The percentage of CWD participating in regular Statewide assessments across all available grade levels (3 through 8);
2. The percentage of fourth-grade CWD scoring at basic or above on the NAEP;[11]
3. The percentage of fourth-grade CWD included in NAEP testing;
4. The percentage of eighth-grade CWD scoring at basic or above on the NAEP;
5. The percentage of eighth-grade CWD included in NAEP testing;
6. The percentage of CWD exiting school by dropping out; and
7. The percentage of CWD exiting school by graduating with a regular high school diploma.

The Results Elements for participation in regular Statewide assessments are scored separately for reading and math. When combined with the exiting data, there are a total of four Results Elements for the Entities. The Results Elements are defined as follows:

Percentage of CWD Participating in Regular Statewide Assessments

This is the percentage of CWD who took regular Statewide assessments in School Year (SY) 2021–2022 with and without accommodations by averaging the assessment participation percentages across all available grade levels (3 through 8) where a regular assessment was administered, for reading and math separately. The numerator for calculating the participation percentage of CWD who took regular Statewide assessments with and without accommodations for each grade level with available data is the number of CWD participating with and without accommodations in regular Statewide assessments in SY 2021–2022, and the denominator is the number of all CWD participants and non-participants in regular and alternate Statewide assessments in SY 2021–2022, excluding medical emergencies. The calculation is done separately by subject (math and reading). The numerator for calculating the percentage of CWD who took regular Statewide assessments in SY 2021–2022 with and without accommodations is the sum of the participation percentages for each grade level in SY 2021–2022, and the denominator is the number of grade levels where the test was administered. The calculation is done separately by subject (math and reading).
(Data source: EDFacts SY 2021–2022; data extracted 4/5/23)

Percentage of CWD Scoring at Basic or Above on the NAEP

This is the percentage of CWD, not including students with a Section 504 plan, by grade (4 and 8) and subject (math and reading), who scored at or above basic on the NAEP in the most recently administered NAEP. As explained in footnote 11, given that data from the most recently administered NAEP for the BIE (i.e., 2019) was not comparable to the data from the most recently administered NAEP for the 50 States, the District of Columbia, and Puerto Rico (i.e., 2022), OSEP did not use NAEP data in making the BIE’s 2023 determination. Therefore, this results element was not a factor in 2023 determinations for the Entities.

Percentage of CWD Included in NAEP Testing

This is the reported percentage of identified CWD, by grade (4 and 8) and subject (math and reading), who were included in the NAEP testing in the most recently administered NAEP. As explained in footnote 11, given that data from the most recently administered NAEP for the BIE (i.e., 2019) was not comparable to the data from the most recently administered NAEP for the 50 States, the District of Columbia, and Puerto Rico (i.e., 2022), OSEP did not use NAEP data in making the BIE’s 2023 determination. Therefore, this results element was not a factor in 2023 determinations for the Entities.

Percentage of CWD Exiting School by Dropping Out

This is a calculation of the percentage of CWD, ages 14 through 21, who exited school by dropping out. The percentage was calculated by dividing the number of students ages 14 through 21 served under IDEA Part B, reported in the exit reason category dropped out for SYs 2020–2021, 2019–2020, and 2018‑2019, by the total number of students ages 14 through 21 served under IDEA Part B, reported in the six exit-from-both-special education-and-school categories (graduated with a regular high school diploma, graduated with an alternate diploma, received a certificate, dropped out, reached maximum age for services, and died) for SYs 2020–2021, 2019–2020, and 2018–2019, then multiplying the result by 100.[12]
(Data source: EDFacts SYs 2020–2021, 2019–2020, and 2018–2019; data extracted 5/25/22, 5/26/21, 5/27/20)

Percentage of CWD Exiting School by Graduating with a Regular High School Diploma

This is a calculation of the percentage of CWD, ages 14 through 21, who exited school by graduating with a regular high school diploma. The percentage was calculated by dividing the number of students ages 14 through 21 served under IDEA Part B, reported in the exit reason category graduated with a regular high school diploma for SYs 2020–2021, 2019–2020, and 2018–2019, by the total number of students ages 14 through 21 served under IDEA Part B, reported in the six exit-from-both-special education-and-school categories (graduated with a regular high school diploma, graduated with an alternate diploma, received a certificate, dropped out, reached maximum age for services, and died), exiting school in SYs 2020–2021, 2019–2020, and 2018–2019, then multiplying the result by 100.
(Data source: EDFacts SYs 2020–2021, 2019–2020, and 2018–2019; data extracted 5/25/22, 5/26/21, 5/27/20)

Scoring of the Results Matrix

In the attached Entity-specific 2023 Part B Results Matrix, an Entity received points as follows for the Results Elements:

• An Entity’s participation rates on regular Statewide assessments were assigned scores of ‘2’, ‘1’ or ‘0’ based on an analysis of the participation rates across all States and Entities. The participation rates for the Entities were calculated based on an average of participation rates across all available grade levels (3 through 8) in which the assessment was administered. The calculation is done separately by subject (math and reading). A score of ‘2’ was assigned if at least 90% of CWD in the Entity participated in the regular Statewide assessment; a score of ‘1’ if the participation rate for CWD was 80% to 89%; and a score of ‘0’ if the participation rate for CWD was less than 80%.
• Each Entity’s data on the percentage of CWD who exited school by dropping out were rank-ordered, and the top, middle, and bottom thirds were determined using tertiles. The exiting percentages for the Entities were calculated using the percentage of CWD exiting school by dropping out in SYs 2020–2021, 2019–2020, and 2018–2019, and points were assigned. The percentages that fell in the top tertile of States and Entities (e., those with the lowest percentage) received a score of ‘2’, percentages that fell in the middle tertile of States and Entities received a ‘1’, and percentages that fell in the bottom tertile of States and Entities (i.e., those with the highest percentage) received a ‘0’.
• Each Entity’s data on the percentage of CWD who exited school by graduating with a regular high school diploma were rank-ordered and the top, middle, and bottom thirds determined using tertiles. The exiting percentages for the Entities were calculated using the percentage of CWD exiting school by graduating with a regular high school diploma in SYs 2020–2021, 2019–2020, and 2018–2019, and points were assigned. The percentages that fell in the top tertile of States and Entities (e., those with the highest percentage) received a score of ‘2’, percentages that fell in the middle tertile of States and Entities received a ‘1’, and percentages that fell in the bottom tertile of States and Entities (i.e., those with the lowest percentage) received a ‘0’.

The following table identifies how each of the Results Elements was scored:

Using the cumulative possible number of points as the denominator, and using as the numerator the actual points the Entity received in its scoring under the Results Elements, the Results Matrix reflects a Results Score, which is combined with the Compliance Score to calculate the Entity’s RDA Percentage.

C. 2023 RDA Percentage and 2023 Determination

The Entity’s RDA Percentage was calculated by adding 40% of the Entity’s Results Score and 60% of the Entity’s Compliance Score. As explained on page five above, while Indicator 4B data was a factor in each Entity’s 2023 Compliance Matrix, no Entity’s 2023 determination was negatively impacted due solely to its Indicator 4B data — specifically, if an Entity’s RDA Percentage is in the Needs Assistance range instead of Meets Requirements range based solely on Indicator 4B data, the Entity’s 2023 RDA Determination is Meets Requirements; likewise, if an Entity’s RDA Percentage is in the Needs Intervention range instead of Needs Assistance range based solely on Indicator 4B data, the Entity’s 2023 RDA Determination is Needs Assistance. The Entity’s RDA Determination is defined as follows:

Meets Requirements
An Entity’s 2023 RDA Determination is Meets Requirements if the RDA Percentage is at least 80%,[13] unless the Department has imposed Specific Conditions on the Entity’s last three (FFYs 2020, 2021, and 2022) IDEA Part B grant awards, and those Specific Conditions are in effect at the time of the 2023 determination.

Needs Assistance
An Entity’s 2023 RDA Determination is Needs Assistance if the RDA Percentage is at least 60% but less than 80%. An Entity’s determination would also be Needs Assistance if its RDA Determination percentage is 80% or above, but the Department has imposed Specific Conditions on the Entity’s last three (FFYs 2020, 2021, and 2022) IDEA Part B grant awards, and those Specific Conditions are in effect at the time of the 2023 determination.

Needs Intervention
An Entity’s 2023 RDA Determination is Needs Intervention if the RDA Percentage is less than 60%.

Needs Substantial Intervention
The Department did not make a determination of Needs Substantial Intervention for any State or Entity in 2023.

[1] When providing exiting data under Section 618 of the IDEA, Entities are required to report, among other things, on the number of students with disabilities who exited an educational program through receipt of a regular high school diploma. These students meet the same standards for graduation as those for students without disabilities. As stated in 34 C.F.R. § 300.102(a)(3)(iv), in effect June 30, 2017, “the term regular high school diploma means the standard high school diploma awarded to the preponderance of students in the State that is fully aligned with State standards, or a higher diploma, except that a regular high school diploma shall not be aligned to the alternate academic achievement standards described in Section 1111(b)(1)(E) of the ESEA. A regular high school diploma does not include a recognized equivalent of a diploma, such as a general equivalency diploma, certificate of completion, certificate of attendance, or similar lesser credential.”

[2] The U.S. Virgin Islands report data for Indicators 4B, 9, 10, 11, 12, and 13. Guam, American Samoa, and the Commonwealth of the Northern Mariana Islands report data for Indicators 11, 12, and 13. The Federated States of Micronesia, the Republic of the Marshall Islands, the Republic of Palau, and the BIE report data on Indicators 11 and 13.

[3] A notation of “N/A” (for “not applicable”) in the “Performance” column for an indicator denotes that the indicator is not applicable to that particular Entity. The points for that indicator are not included in the denominator for the matrix.

[4] In determining whether an Entity has met the 95% compliance criterion for Indicators 11, 12, and 13, the Department will round up from 94.5% (but no lower) to 95%. In determining whether an Entity has met the 90% compliance criterion for these indictors, the Department will round up from 89.5% (but no lower) to 90%. In addition, in determining whether an Entity has met the 75% compliance criterion for these indicators, the Department will round up from 74.5% (but no lower) to 75%. Similarly, in determining whether an Entity has met the 5% compliance criterion for Indicators 4B, 9, and 10, the Department will round down from 5.49% (but no higher) to 5%. In determining whether an Entity has met the 10% compliance criterion for these indicators, the Department will round down from 10.49% (but no higher) to 10%. In addition, in determining whether an Entity has met the 25% compliance criterion for these indicators, the Department will round down from 25.49% (but no higher) to 25%. The Department will also apply the rounding rules to the compliance criteria for 95% and 75% for:

• the timeliness and accuracy of data reported by the Entity under Sections 616 and 618 of the IDEA; and
• the Entity’s FFY 2021 data, reported under Section 618 of the IDEA, for the timeliness of State complaint and due process hearing decisions.

[5] For Indicators 4B, 9, and 10, a very high level of compliance is generally at or below 5%.

[6] A “No” in that column denotes that the Entity has one or more remaining findings of noncompliance identified in FFY 2020 for which the Entity has not yet demonstrated correction. An “N/A” (for “not applicable”) in that column denotes that the Entity did not identify any findings of noncompliance in FFY 2020 for the indicator.

[7] If an Entity’s FFY 2021 data for any compliance indicator are not valid and reliable, the matrix so indicates in the “Performance” column, with a corresponding score of 0. The explanation of why the Entity’s data are not valid and reliable is contained in the OSEP Response to the Entity’s FFY 2021 SPP/APR in the EMAPS SPP/APR reporting tool.

[8] If an Entity reported no FFY 2021 data for any compliance indicator (unless the indicator is not applicable to the Entity), the matrix so indicates in the “Performance” column, with a corresponding score of 0.

[9] In OSEP’s July 19, 2022, guidance titled Questions and Answers: Addressing the Needs of Children with Disabilities and IDEA’s Discipline Provisions, at Question L-6, OSEP reiterated that factors it may consider in determining reasonableness of a State’s methodology include whether none, or a very low percentage of, the State’s LEAs are being examined for significant discrepancy under the State’s chosen methodology, and whether statistically sound alternative methodologies exist or are being used by similarly-situated States.

[10] OSEP used the Part B Timely and Accurate State-Reported Data Rubric to award points to Entities based on the timeliness and accuracy of their Sections 616 and 618 data. A copy of the rubric is contained in the OSEP Response to the Entity’s FFY 2021 SPP/APR in the EMAPS SPP/APR reporting tool. On page one of the rubric, entitled “Part B Timely and Accurate Data,” Entities are given one point for each indicator with valid and reliable data and five points for SPP/APRs that were submitted timely. The total points for valid and reliable SPP/APR data and timely SPP/APR submission are added together to form the APR Grand Total. The Entity’s Section 618 data is scored based on information provided to OSEP on Section 618 data timeliness, completeness, and edit checks from EDFacts. The percentage of Timely and Accurate State-Reported Data is calculated by adding the 618 Data Grand Total to the APR Grand Total and dividing this sum by the total number of points available for the entire rubric. This percentage is inserted into the Compliance Matrix.

[11] The BIE is the only Entity for which NAEP data are applicable as a results element. However, OSEP did not use NAEP data in making the BIE’s 2023 determination because the NAEP data available for the BIE were not comparable to the NAEP data available for the 50 States, the District of Columbia, and Puerto Rico; specifically, the most recently administered NAEP for the BIE is 2019, whereas the most recently administered NAEP for the 50 States, the District of Columbia, and Puerto Rico is 2022.

[12] The Department will make these calculations using unsuppressed data. However, due to privacy concerns, the Department has chosen to suppress calculations made with small cell counts in the public document.

[13] In determining whether an Entity has met this 80% matrix criterion for a Meets Requirements determination, the Department will round up from 79.5% (but no lower) to 80%. Similarly, in determining whether an Entity has met the 60% matrix criterion for a Needs Assistance determination discussed below, the Department will round up from 59.5% (but no lower) to 60%.