Part I. Primary and Secondary Education in the Humanities Section A. National Measures of Achievement

NOTE TO READERS: Please include the following reference when citing data from this page: "American Academy of Arts and Sciences, Humanities Indicators, http://HumanitiesIndicators.org". Print Section

Indicator I-1 Reading Competency among School-Age Children

Indicator I-2 Writing Proficiency

Indicator I-3 Knowledge of U.S. History

Indicator I-4 Knowledge of Civics

Indicator I-5 Performance on SAT Verbal/Critical Reading and Writing Exams

The most readily available measures of student achievement in the humanities are standardized test scores in humanities subjects. Consistent trend data are available from two major sources, the College Board’s SAT (formerly known as the Scholastic Aptitude Test and Scholastic Assessment Test) critical reading and writing examinations, and the National Assessment of Educational Progress (NAEP). The NAEP, also known as “The Nation’s Report Card,” has been regularly administered by the U.S. Department of Education’s National Center for Education Statistics (NCES) in several subject areas since 1969. Although both data sets have limitations as measures of humanities knowledge—only the NAEP is based on a national, representative sample of students, and even in combination the examinations do not provide information for the full range of humanities-related subjects taught in the nation’s schools—such scores offer a window on the effectiveness of humanities education at the primary and secondary levels.

Indicator I-1	Reading Competency among School-Age Children
Print I-1

Updated and augmented (6/10/2011).

See the

The NAEP includes two assessments in reading. The first, currently administered every two years and usually referred to as the “main” NAEP reading assessment, changes in response to the current state of curricula and educational practices. The second test is specifically designed to generate long-term trend data. Administered every two to five years, this examination has remained essentially unchanged since it was first given to students in 1971; it features shorter reading passages than the main NAEP assessment and gauges students’ ability to locate specific information, make inferences, and identify the main idea of a passage. (For a detailed comparison of the two assessments, see http://nces.ed.gov/nationsreportcard/about/ltt_main_diff.asp.)

The NAEP long-term trend exam (LTT) is taken by a nationally representative sample of students in each of three different age groups: 9-year-olds, 13-year-olds, and 17-year-olds. The percentages indicated on the graphs displaying LTT data (Figures I-1a, I-1b, and I-1c) are cumulative totals; they indicate the percentage of students in each grade level scoring at or above each performance level. (The LTT performance thresholds are constructed at 50-point intervals and range from 150 to 350; see the Note on the Difference between NAEP "Achievement" and "Performance" Levels.) The performance levels are also cumulative in the sense that students performing at each level also display all the competencies associated with the level(s) below it. (See NAEP’s descriptions of the skills demonstrated by students scoring at each performance level.) Also, although the performance levels at which the majority of students score are different for each of the age groups (a result of their cumulative nature), the color-coding of the levels is consistent across the LTT graphs. Blue represents the percentage of students scoring at or above the most basic performance level for that age group. Red represents the percentage scoring at or above the intermediate performance level. Gold represents the percentage scoring at or above the advanced performance level.

In 2004, the LTT was updated in several ways. Content and administration procedures were revised, and, for the first time, accommodations were made for English language learners and students with disabilities that would allow these students to be included in the assessment (they have been included in the main NAEP reading assessment since 1996). Both the original and revised formats were administered in 2004 so NCES could investigate the effects of the new format on scores. This "bridge" study indicated that differences in student scores between the two formats were solely attributable to the inclusion of students with disabilities and English language learners in the testing population. On the basis of these findings, NCES concluded, “bearing in mind the differences in the populations of students assessed (accommodated vs. not accommodated), future assessment results could be compared to those from earlier assessments based on the original version.”¹

Among 9-year-olds, reading performance increased steadily between the early 1970s and 1980 (Figure I-1a). Reading performance then began declining and by 1990 had largely returned to its original level (though somewhat more students were assessed at the highest performance level for this grade than in 1971). The 1990s were another period of improvement, with incrementally greater percentages of students attaining Levels 150 and 200, which represent the basic and intermediate performance levels. (Nonetheless, the percentage of students scoring at the highest performance level did drop slightly.) The period between 1999 and 2008 was one of more marked progress. In 2008, a higher percentage of students than in any previous assessment year—96%, up five percentage points from 1971—demonstrated the ability to perform simple, discrete reading tasks associated with the most basic performance level (Level 150). At the high end of the performance spectrum (Level 250), students demonstrated gains of a similar magnitude, with an increase from 16% to 21% in the proportion of students exhibiting the ability to interrelate ideas and make generalizations. The greatest gains among 9-year-olds from 1999 to 2008 were realized in the middle of the performance spectrum (Level 200). In 2008, 73% of students—up nine percentage points from 1999 (and up 14 points from 1971)—demonstrated the ability to:

	locate and identify facts from simple informational paragraphs, stories, and news articles;
	combine ideas and make inferences based on short, uncomplicated passages; and
	understand specific details or sequentially related information.

This increase is the most striking development observed in any age group to date. At the same time, however, the increase since 1980, the peak of the early upswing in scores, is a more modest five percentage points. Thus, the story of 9-year-olds’ reading performance since 1971 is one of recovery of lost gains rather than continuous progress.

Figure I-1a, Full Size Supporting Data

The story for early adolescents, on the other hand, is one of stasis. Although the early 1990s saw an increase in the percentage of 13-year-old students scoring at the middle and high performance levels (Levels 250 and 300), little movement in scores was observed until 2008 (Figure I-1b). In this year, the percentage of students scoring at or above the intermediate performance level increased by four percentage points to 63%. In 2008, nearly all students (94%) displayed at least partially developed skills and understanding (i.e., scored at least 200), and 13% demonstrated the ability to understand complicated information (i.e., scored 300 or better).

Figure I-1b, Full Size Supporting Data

For 17-year-olds, the percentage of students achieving at least basic competency (Level 250) rose from 79% in 1971 to a high of 86% in 1988 (Figure I-1c). Subsequently, however, this trend reversed, and by 2008 the percentage had returned to that recorded in 1971. The trend in midlevel achievement was similar: an increase followed by reversion to the original level. Thus in 2008, as in 1971, 39% of students left high school (most of the 17-year-olds tested were secondary-school seniors) able to understand complicated literary and informational passages (Level 300). The share of students exiting with the ability to extend and restructure ideas drawn from specialized or complex texts (Level 350) was 6% in 2008, having changed little over the previous 37 years.

Figure I-1c, Full Size Supporting Data

The LTT is an assessment of basic skills. In contrast, the main NAEP assessment evaluates fourth-, eighth-, and 12th-grade students’ ability to tackle more-demanding reading tasks, including the reading of longer passages or pairs of passages. According to NCES, the main assessment “measures a range of reading skills, from identifying explicitly stated information, to making complex inferences about themes, to comparing multiple texts on a variety of dimensions”. As Figure I-1d indicates, on the 2009 main NAEP assessment, 33% of fourth graders demonstrated reading skills at or above the “proficient” level, while a similar proportion displayed “below basic” skills. (The main NAEP includes four achievement levels: “below basic,” “basic,” “proficient,” and “advanced”. See http://nces.ed.gov/nationsreportcard/reading/achieveall.asp for a description of the skills associated with each achievement level.) The performance of eighth and 12th graders on the reading assessment was similar to that of their younger counterparts with respect to the proportion of students demonstrating at least proficiency, but in the upper grades a smaller share, approximately one-quarter, demonstrated below basic skills (Figures I-1e and I-1f).

Figure I-1d, Full Size Supporting Data

Figure I-1e, Full Size Supporting Data

Figure I-1f, Full Size Supporting Data

For purposes of comparison, Figures I-1d through I-1f also depict student performance on the NAEP math and science assessments. At the fourth-grade level, students’ performance on the 2009 science assessment was similar to their demonstrated abilities in reading, though slightly fewer were below basic and slightly more were at basic or above. Students also performed better on the math assessment than on the reading, with a somewhat larger share demonstrating at least proficiency and a share demonstrating below basic skills that was 15 percentage points smaller than that for reading. Among eighth graders, reading performance was almost identical to that in math in 2009. Students had more difficulty in science, with 37% demonstrating below basic skills, compared with 25% on the reading assessment. At the 12th-grade level, students did better in reading than in either science or math. The commonality among the three assessments, at all three grade levels, is that on none of them did a majority of students demonstrate proficiency.

Another notable difference between the subject areas is that at the two lower grade levels the improvement in student achievement between the early 1990s and 2009 was considerably greater in math than in reading (the results of the 2009 science assessment cannot be compared with those from previous years because of major changes in the assessment framework). The increase in the percentage of fourth graders scoring at the proficient level or higher on the math assessment was 26 points. The increase in the eighth-grade math percentage over the same period was 19 points. In contrast, the improvement between 1992 and 2009 in reading was four points for fourth graders and three points for eighth graders. At the other end of the achievement spectrum, the share of fourth-grade students demonstrating below basic skills in math shrunk from 50% to 18% over the two decades. Developments at the eighth-grade level in math were also dramatic, with the share of students performing at a below basic level decreasing 21 percentage points, from 48% to 27%. The percentage of students performing at a below basic level in reading decreased five percentage points for fourth graders and six points for eighth graders. Among 12th graders reading achievement declined between 1992 and 2009 (these developments cannot be compared with those in math, because the earliest year for which appropriate math assessment data are available is 2005).

(The NAEP Data Explorer permits analysis of both the long-term trend and main NAEP data sets by gender, ethnicity, and other key variables. With Explorer one can also obtain results of recent reading assessments for individual states and compare these with student outcomes in other parts of the country. For both an overview of Explorer and tips for its effective use, see http://nces.ed.gov/nationsreportcard/pdf/naep_nde_final_web.pdf. The Explorer itself is located at http://nces.ed.gov/nationsreportcard/naepdata/.)

Data from the Organisation for Economic Co-operation and Development’s (OECD) Programme for International Student Assessment (PISA) reveal that while American 15-year-olds demonstrated levels of reading literacy similar to those of students in several other Western industrialized nations—such as France, Germany, the United Kingdom, and Sweden² —they scored measurably lower, on average, than their counterparts in nine jurisdictions (14% of those that participated in PISA; Figure I-1g). In 2009, the United States’ average score on the PISA combined literacy scale3 was statistically indistinguishable from the OECD average but was lower than that of several Asian jurisdictions, as well as Australia, Canada, Finland, and New Zealand. American adolescents did best on the reading literacy test items meant to gauge students’ ability to reflect on and evaluate what they had read (Figure I-1h). They did less well on tasks that involved access and retrieval of information. But even on the higher-order reading tasks on which they tended to do better, American students were outperformed by students in China (Shanghai and Hong Kong), Korea, and Canada, among other jurisdictions.

Figure I-1g, Full Size Supporting Data

Figure I-1h, Full Size Supporting Data

With respect to the distribution of reading literacy proficiency among the mid-adolescent population, 30% of U.S. 15-year-olds were capable of difficult reading tasks (i.e., scored at Level 4 or higher). Ten jurisdictions (15% of those participating) had measurably greater shares of students with such capability (Figure I-1i; see http://nces.ed.gov/pubs2011/2011004.pdf, page 10, for a detailed description of the types of tasks associated with each PISA proficiency level). In top-ranked Shanghai and Korea, 54% and 46% of students were able to complete such tasks. Eighteen percent of American 15-year-olds demonstrated reading literacy at sub-basic levels (i.e., scored at Level 1 or below). Six jurisdictions (9% of those participating) had measurably lower shares of students demonstrating such minimal reading literacy.

Figure I-1i, Full Size Supporting Data

Figure I-1j compares the United States’ international standing in reading literacy to its performance in math and science literacy. The data suggest that the United States’ relative performance was stronger in reading than in math or science. The United States was outperformed by fewer nations on the reading assessment than on the other two examinations. Moreover, while the average differential between the United States’ and higher-scoring jurisdictions’ average scores on the reading literacy assessment was comparable to that for the science exam (approximately 30 points), the differential for math was closer to 40 points. Additionally (not pictured on the figure), the students of Shanghai, the top-ranked jurisdiction on all three literacy assessments, outscored U.S. students in science and math by larger margins, on average, than they did in reading.

Figure I-1j, Full Size Supporting Data

---

Note

Belgium
Chinese Taipei
Denmark
Estonia
France
Germany
Hungary
Iceland
Ireland
Liechtenstein
Netherlands
Norway
Poland
Sweden
Switzerland
United Kingdom

Indicator I-2	Writing Proficiency
Print I-2

Updated (6/25/2010).

See the

In addition to reading, writing is a core humanistic competency measured by the NAEP (see also U.S. history and civics). Students in the fourth, eighth and 12th grades are evaluated on the basis of essays they write in response to standardized prompts (see examples of prompts and student responses).

In the most recent assessment year, 2002 for fourth graders and 2007 for eighth graders, close to 90% of students in these grades demonstrated at least basic writing competency, a level that reflects modest growth since 1998 (Figure I-2). This rise is attributable mostly to an increase in the share of students demonstrating higher-order writing skills.

Figure I-2, Full Size Supporting Data

Twelfth-grade performance has been somewhat lower and more dynamic. Seniors’ performance slipped between 1998 and 2002, with the percentage of students scoring at the basic achievement level or better declining from 78% to 74%. But over the next five years that lost ground was recovered. In 2007, the percentage of 12th graders scoring at the basic level or higher was 82%. This gain among 12th graders reflected a growing share of students who demonstrated basic writing competence. There was no growth between 2002 and 2007 in the share of high school seniors exhibiting true writing proficiency. Since 1998, fewer than one in four soon-to-be high school graduates have been assessed as writing at the proficient level or higher (within the NAEP framework, a proficient writer is one who demonstrates a grasp of writing skills that are essential for success in most walks of life; these skills include the use of transitional elements and the ability to select language appropriate for the intended audience).

Recent cohorts of young people have been more successful than their predecessors in maintaining their writing competence as they transition from middle to high school. Although the NAEP is not a longitudinal assessment (i.e., it does not follow the same students over time), the spacing of the assessments permits tracking of cohorts’ performance as they move through the educational system. The sample of eighth graders assessed in 1998 was from the same cohort of students from which the 2002 sample of 12th graders was drawn. Between early and late adolescence, this cohort’s writing ability declined, with 26% of them failing to demonstrate at least basic competency as high schoolers, compared with only 16% of the sample drawn from this same group when they were in middle school. However, for those students who started eighth grade in 2002, the drop off in high school does not seem to have been so precipitous. Fifteen percent of this later cohort scored below the basic achievement level in middle school, while a comparable 18% did so in 12th grade (the latter percentage is an approximation of this cohort’s high school performance based on 2007 data—the NAEP was not administered in 2006, the year these students would have been seniors). For both cohorts, the 12th graders who were assessed as part of NAEP did not include those students who dropped out of school before reaching their senior year. Had these individuals remained in school long enough to be tested, they would presumably have increased the percentage of students demonstrating less than basic achievement in writing.

(The NAEP Data Explorer permits analysis of these assessment data by gender, ethnicity, and a number of other key variables. With this tool one can also obtain results of recent writing assessments for individual states and compare these with student outcomes in other parts of the country. For both an overview of Explorer and tips for its effective use, see http://nces.ed.gov/nationsreportcard/pdf/naep_nde_final_web.pdf. The Explorer itself is located at http://nces.ed.gov/nationsreportcard/naepdata/.)

Indicator I-3	Knowledge of U.S. History
Print I-3

Updated (8/15/2011).

Although it was introduced later and is given less frequently than the reading assessment, the NAEP for U.S. history also supplies data describing change over time in students’ knowledge of a core humanities subject. Figure I-3 depicts fourth, eighth and 12th graders’ U.S. history achievement in 2010 compared to that in 1994. Over this period, the percentage of students in the two lower grades demonstrating at least basic achievement in the subject increased by a statistically significant margin. No measurable difference exists between the 1994 and 2010 U.S. history achievement of high school seniors.

Figure I-3, Full Size Supporting Data

The greatest change over time was observed between the two cohorts of fourth graders. In 2010, 73% of all students in this grade demonstrated at least basic knowledge of U.S. history, up from 64% in 1994. In addition, fourth graders proved to be the most knowledgeable, in that a greater proportion of these students demonstrated at least basic achievement in U.S. history than did students in either of the two other age groups (in both testing years).

Figure I-3 shows that in both 1994 and 2010 the older the cohort the lower the proportion of students demonstrating at least basic knowledge. U.S. history achievement is most dramatically lower in high school. In 2010, for example, 69% of middle schoolers demonstrated at least a basic understanding of U.S. history, while only 45% of high schoolers did so.

In addition to examining the trends both over time and across school grades, the cross-sectional picture should be considered: in both 1994 and 2010, the majority of American school-age children demonstrated minimal knowledge of the nation’s history. The absence of long-term trend data prevents a systematic evaluation of how recent a phenomenon this is, but research reveals that young people’s ignorance of U.S. history has been a source of public concern since the beginning of the 20th century.1

(The NAEP Data Explorer permits analysis of these assessment data by gender, ethnicity, and a number of other key variables. For both an overview of Explorer and tips for its effective use, see http://nces.ed.gov/nationsreportcard/pdf/naep_nde_final_web.pdf. The Explorer itself is located at http://nces.ed.gov/nationsreportcard/naepdata/.)

Indicator I-4	Knowledge of Civics
Print I-4

Updated (8/12/2010).

The NAEP civics examination is designed to gauge students’ proficiency in three civics areas: knowledge, intellectual skills, and dispositions. In 2010, 77% of all fourth graders scored at or above the basic achievement level (Figure I-4a). As was true of the NAEP assessments in other humanities subjects, lower levels of competency were observed among older students, with only 64% of 12th graders demonstrating at least basic achievement in 2010. (For the percentages of students in different grades who demonstrated particular civics competencies, see Graphic I-4a).

Figure I-4a, Full Size Supporting Data

Graphic gI-4a, Full Size

Two possible explanations can be advanced for the lower levels of achievement in the higher grades. The first of these is a “cohort-based” explanation. This asserts that, in the case of students who took the 2010 NAEP civics examination, students born in the early 1990s are for some reason less receptive to civics instruction than their counterparts born in the early 2000s. The other type of explanation focuses on “age effects.” This explanation asserts that something about late adolescence—either the developmental process or high school education in the United States—is less conducive to civics learning. (See the memo from the NAEP governing board describing how the timing of the high school assessment may be resulting in an underestimation of 12th graders’ achievement in civics and other areas.)

The spacing of the NAEP civics assessments permits an investigation of these issues. Because a particular cohort of students can be followed over time (the sample of eighth graders who took the exam in 2006 was drawn from the same cohort as the sample of 12th graders who took the exam in 2010), researchers can “control” for cohort effects (i.e., reduce the possibility that observed differences between younger and older students’ performance is attributable to differences between grade cohorts).

As in the case of writing, the data provide some support for the second type of explanation; that is, student performance is linked to age. As students in the cohort progressed through their educational careers, the percentage demonstrating at least basic achievement decreased. However the picture was not one of unambiguous decline in civics achievement but of polarization, with students becoming increasingly concentrated at the two ends of the performance spectrum. In eighth grade, 30% of the cohort demonstrated less than basic achievement in civics. (The results of the 2006 exam are not depicted here, but they are available for downloading at http://www.eric.ed.gov/PDFS/ED496659.pdf.) Upon reaching 12th grade, as indicated in Figure I-4a, 36% failed to demonstrate at least basic competency. At the same time, the share of students scoring at the high end of the achievement scale (performing at the “proficient” or “advanced” levels) was somewhat larger in 12th grade than in eighth grade.

Figure I-4a also reveals that the nation has made progress over time with respect to fourth graders’ civics achievement. Between 1998 and 2010 the share of students at this grade level who demonstrated less than basic achievement decreased from 31% to 23%. Gains were seen not only in the share of students demonstrating at least basic achievement but also in the proportion displaying true proficiency in the subject. No statistically significant change over the same time period was found in the performance of eighth or 12th graders.

(The NAEP Data Explorer permits analysis of these assessment data by gender, ethnicity, and a number of other key variables. For both an overview of Explorer and tips for its effective use, see http://nces.ed.gov/nationsreportcard/pdf/naep_nde_final_web.pdf. The Explorer itself is located at http://nces.ed.gov/nationsreportcard/naepdata/.)

The only source of data that permits international comparison of young people’s civics achievement is the International Association for the Evaluation of Educational Achievement’s (IEA) periodic assessment, first administered in 1971 and again in 1999 and 2009. Because the United States opted not to participate in the latest study, the most recent data available on how U.S. 14-year-olds perform relative to their counterparts elsewhere in the world are from the late 1990s. In 1999, the IEA assessment consisted of two components. The first focused on civics content knowledge (Graphic I-4b), or theoretical knowledge about democratic institutions and practices, such as the purpose of political parties (25 items). The second component examined students’ civics skills (Graphic I-4c); that is, interpretive abilities important in understanding political material, such as the ability to distinguish between facts and opinions or being able to critically read a political cartoon or pamphlet (13 items). The two scores were then averaged, with civics content knowledge scores weighted somewhat more heavily, to produce a total civics knowledge score for each nation.

Graphic gI-4b, Full Size

Graphic gI-4c, Full Size

On the civics skills portion of the exam, the United States outperformed all of the OECD nations, as well as the non-OECD nations, that participated in the 28-country study (Figure I-4b). The United States did not score as well on the civics content portion of the test, coming in behind several other OECD countries. In terms of its 14-year-olds’ total civics knowledge, the United States ranked fourth, though the difference between its average score and that of the three OECD leaders—Poland, Finland, and Greece—was not statistically significant.

Figure I-4b, Full Size Supporting Data

Back to Top

---

Indicator I-5	Performance on SAT Verbal/Critical Reading and Writing Exams
Print I-5

Updated (1/19/2012) with data for 2010 and 2011.

Although controversy over the SAT persists on a number of fronts, the verbal portion of the SAT (renamed “critical reading” in 2005) is a valuable measure of college-bound seniors’ linguistic skills, because the test has been administered for several decades and thus permits comparison over an extended period of time. The SAT data reveal a steep decline between 1967 and the early 1980s in mean verbal scores,1 followed by a leveling off, with mean scores ranging between approximately 500 and 510 ever since (Figure I-5a; scores have been adjusted to take into account the 1995 recentering of the scoring system).

Figure I-5a, Full Size Supporting Data

As was widely reported at the time, SAT verbal scores declined relatively sharply in 2006, with the combined male and female scores showing their furthest drop since 1975. The College Board, the body that administers the SAT, attributed the drop to a decline in the number of students who retook the exam (examinees’ scores tend to increase substantially the second time they take the test). Observers had speculated that this drop in scores was due to the increased length of the exam, which included a new writing section. College Board officials responded that an analysis of approximately 700,000 tests produced no evidence to support this theory.2 The average verbal score in 2009 was 501, down two points from 2006.

The average SAT math score also declined over the course of the 1970s. But unlike the SAT verbal average, which has remained fairly constant over the last two decades, the mean score on the quantitative portion of the math exam has risen steadily since its low point in 1980. Between 1967 and the late 1980s, average math scores were consistently lower than those on the verbal exam. But with the steady improvement of math scores and the stagnation of verbal scores, American students were demonstrating somewhat stronger math than verbal skills by 1990. The gap has grown since then, so that by 2006 the mean math score was 15 points higher than the verbal score. Things have changed little since that year; in 2009 the difference was 14 points. This is a profound reversal from the state of affairs in 1967, when the average verbal score exceeded the math score by 27 points.

Male students’ average verbal SAT score has been consistently higher than that of female students since the early 1970s. Initially the gap was small, but the disparity grew, and during the 1980s the gender gap in verbal scores ranged from 10 to 13 points. The gap has narrowed since then, with female examinees’ scores coming within five points of male scores in 2009.

The difference in mean scores between white and minority groups is more pronounced (Figure I-5b). Mean verbal scores for most ethnic groups increased between 1992 and 2009. The greatest gains, by a substantial margin, were made by students who self-identified as Asian, Asian American, or Pacific Islander. In 2009, these students scored 29 points higher, on average, than they had in the early 1990s. However, white students still performed better than all other groups. The gap between African American and white scores has been particularly large, with white examinees scoring 99 points higher, on average, in 2009.

Figure I-5b, Full Size Supporting Data

High school seniors graduating in 2006 were the first to take the SAT’s new writing test, which includes both multiple-choice questions and a student-written essay. The mean score on the writing test for all college-bound seniors was 497 in that year (Figure1-5c), with female students scoring higher than their male counterparts, on average (502 v. 491).3 Three years later, the average score for all college bound seniors had declined (four points), as had the average for every ethnic/national group, with the exception of Asians. The latter group’s average score was eight points higher in 2009 than in 2006, bringing it above that for whites. Every other minority group’s average was lower than that for white students. The magnitude of the performance gap between each of these groups and whites was similar to that for the verbal exam. Between 2006 and 2009 the gap between average female and male performance on the writing examination increased from 11 to 13 points.4

Figure I-5c, Full Size Supporting Data

---

Notes