In 1795, Friedrich Schiller, in his famous poetological treatise On Naïve and Sentimental Poetry, claims that “ancient” and “modern” poetry differ in their degree of reflection. While the naïve poet moves us by imitating nature, “by sensuous truth, by living presence” (Schiller [1795] 1985, 194), 1

This poetological distinction is linked in Schiller’s treatise with a philosophy of history in such a way that naïve poetry is possible in the present but “latently anachronistic” (Prill 1994, 521): Under the conditions of modernity, in which a “correspondence between […] feeling and thinking” is hardly possible any more,3 poetry must increasingly become sentimental poetry, that is, a poetry that is moved “through ideas” (Schiller [1795] 1985, 194, 197).4

More than 220 years after Schiller formulated this influential thesis, which has found a diverse echo especially in discourses on the “reflexivity” of the modernist novel (see Beebe (1976), Orr (1981)), computational philological methods offer the possibility to study inner-literary reflections on a broad empirical basis. Using the example of German-language narrative fiction, the present paper will investigate whether literature indeed became more and more “sentimental” – as Schiller put it –, i.e. whether it exhibits an increasing degree of reflectiveness.

Of course, the concept of “literary reflectiveness” or – maybe more wide-spread – “literary reflexivity” is till today a very complex one and there is no direct route from Schiller’s concept of sentimental (reflective) poetry, which is embedded in an entire anthropology and philosophy of history, to an annotation-based and narratologically underpinned approach like ours. The concept of “literary” or “narrative reflexivity” (Williams 1998) belongs to a whole semantic field of (often interchangeably used) ‘big concepts’ like “metatextuality”, “metafiction”, “self-reflexivity” on the on hand (see Julie Tanner (2022)), and rather text-passage oriented concepts like “authorial intrusions”, “commentary” or “digression” on the other hand. This may be one of the reasons why there is little consens about the historical development of literary reflectiveness: While it is evident from a number of case studies that at least some early-modern works of literature exhibit significant traits of reflectiveness (see Zapf et al. (2005, 8), Henke (2005)), it is by no means clear how this phenomenon developed in the context of a rapidly growing book market in the 19^th century and a mass market in the 20^th century.

Our approach aims at measuring the degree of reflectiveness of a narrative by identifying so-called “reflective passages”. In the next section, we will introduce our concept of a reflective passage and illustrate how we collaboratively annotated three different subtypes of reflective passages. Section 3 will present a questionnaire that was used to empirically assess the contribution of each of these subtypes (and their interplay) to readers’ perception of a textual passage being a reflection. Based on the statistical analysis of the results of this questionnaire, we introduce the notion of perceived reflectiveness of a given text passage, which is measured by the reflection score. Section 4 will describe two neural classifiers: a multi-label and a binary classifier for identifying reflective passages. In section 5, we will present a diachronic analysis of reflective passages as well as perceived reflectiveness in German fiction based on these two classifiers, that allows for evaluating the hypothesis of a gradual increase of reflectiveness in the modern period. Finally, we will summarise our results and sketch prospects for future research.

When speaking of reflective passages in the context of fictional literature, one may think of various things. Without a doubt, fictional narrative texts regularly stimulate reflections in readers. Authors of such texts also often engage in extensive reflection before or during writing. Reflective passages, in contrast, refer to those reflections that are present on the surface of the text in fictional narrative texts (Gittel 2022). The broad and complex field of the phenomenon of reflective passages becomes clear from the fact that they are referred to in research by many terms that are by no means synonyms, such as “authorial intrusion” (Dawson 2016), “commentary” (Chatman 1980, 226–252), “digression” (Esselborn 2007), “factual discourse” (Konrad 2017), “serious speech acts in fictional works” (Klauk 2015), “gnomic statement” (Mäkelä 2017), “narrator’s comment” (Zeller 2007), or Sentenz (“aphorism”, Reuvekamp 2007). Although reflective passages have been much discussed recently in connection with their specific manifestations in essayistic and encyclopaedic narrative (Ercolino 2014; Gittel 2015; Herweg et al. 2019), they are not a clearly delimited phenomenon either in narratology or in literary history. For a definition of reflective passage, however, one can draw on considerations of two more established terms in literary theory – ‘comment’/‘commentary’ and ‘non-fictional speech’ – and one in linguistics, namely ‘generalisation’. We consider a reflective passage as a textual passage that is either a comment, non-fictional speech, a generalisation or a combination of these three phenomena. Reflective passages greatly differ regarding their length, ranging from one clause to several sentences or whole paragraphs. The minimal length of a reflective passage being a clause, we will focus in our quantitative diachronic analysis (see section 5) on reflective clauses as the minimal unit of a reflective passage. Since the details of our annotation of these phenomena can be found elsewhere (cf. Barth et al. 2021, Gödeke et al. 2022, Weimer et al. 2022, Barth et al. 2022), we will introduce these phenomena through examples in the following and use the corresponding tags comment, non-fictional speech, and generalisation henceforth.

“Comment” is listed in narrative theory alongside “report”, “description” and “speech” as a fourth so-called “narrative mode” (Bonheim (1975, 329), see also Bonheim (1982)). These four modes, which can overlap, are sufficient for a classification of all passages in a narrative text according to Bonheim. Comments express an evaluative attitude of the speaker towards diegetic state of affairs, illuminate his relationship to the diegesis, or the representation of the events. Thus, they can reveal the narrator’s attitude towards characters or events, or his interpretations and explanations of them, as well as his relation to the concrete representation or to narration/fictionality in general. To illustrate what this main type of within-fiction reflections may look like, we use the beginning of Goethe’s Elective Affinities as example (the corresponding passages are marked in square brackets here and in the following5):

The account of Edward’s April afternoon is interrupted here by a (metafictional) comment that identifies the speaker as an entity that exercises power of designation over the entities of the narrated world. Overall, however, “comment” is a relatively heterogeneous concept. In research, for example, comments on the story, which can have an interpretive, judgemental or generalising character, are distinguished from comments on the discourse (Chatman (1980, 226–252), see also the term “non‑mimetic judgements” in Martinez-Bonati and Silver (1981, 32–33)). Because of this heterogeneity, two criteria are often involved in the identification of comments, one formal and one content-related: According to the formal criterion, comments are those passages of text that are neither speech, report nor description. Like descriptions, they belong to the static mode according to Stanzel and are accompanied by narrative pauses (Stanzel (1988, 66), Martínez and Scheffel (2007, 46)). One often speaks of “pure comment” in reference to such ex negativo identifiable passages (Bonheim 1975, 337). According to the criterion of content, these are passages that express an evaluative attitude of the speaker, his relationship to the event or the representation of the event. If this criterion is taken as a basis, comments can also occur within descriptions, character speech or narrator’s report, so-called “integral comments” (Bonheim 1975, 337). The following dialogue in Theodor Fontane’s The Stechlin can serve as an example, in which Woldemar, the son of the old Stechlin, expresses his astonishment:

Please note that comment is a relatively heterogeneous annotation category that comprises different sub-phenomena: attitude is annotated whenever the speaker comments on fictional events, characters, objects or itself. This type of comment can be easily interwoven with report, for example through certain adverbs (e.g. “Unfortunately, she had already left the house”). interpretation is annotated when explanations or interpretations are provided in a passage through which the diegesis can be understood anew. metacomment is annotated whenever the narrator comments on the fictionality of the story or the process of writing or telling the story.

In addition to comment, there is a second phenomenon relatively well described in literary theory that can be used to formalise the concept of reflective passages: the phenomenon of non-fictional speech in fictional texts. According to many theorists, fictional texts consist not only of fictional speech, which – according to a common characterisation – serves to construct the fictional world but also of non-fictional speech (Searle 1975, Klauk 2015).6 The typical case of non-fictional speech with an assertive character (in the speech act theoretical sense) is relevant to the question of reflections in literature. Characteristic of this phenomenon is that (1) an assertion/hypothesis about the real world is suggested in a clearly delimitable text passage and (2) the propositional content of the assertion/hypothesis can be read off from this text passage itself.7 Corresponding (popular) examples are the following:

Example (4) – more precisely the third sentence of the Handke quote – demonstrates that non-fictional speech does not always have to take the form of generalisation, even though this is the case most often discussed in research (e.g. Vesper 2014).

Third, the phenomenon of generalisation may be regarded as a subtype of reflective passages in its own right. Although generalisation is considered to be an indicator for ‘non-fictional speech’ and ‘comment’ (see Chatman (1980) and Vesper (2014)), its appearances in narrative fiction are much less explored than ‘comment’ and ‘non-fictional speech’ (see Gödeke et al. (2022) for a first attempt). As generalisation, we annotate any statements not made about specific objects, individuals, time periods, or spaces, but about whole classes or groups of entities.

As in this example, non-fictional speech often co-occurs with generalisation. However, generalisations can be about all sorts of entities (characters, spaces, events) in the fictional world as well. Generalisations and non-fictional speech (as comments) can also occur within characters’ speech: Characters can make statements about whole classes or groups of entities, and characters can suggest in a clearly delimited text passage a hypothesis about the real world whose propositional content (e.g. “there is no pure knowledge”) can be read off from this text passage itself.

Having examined the three reflection-constituting phenomena, we will give a brief overview of our annotation results. Our annotation corpus consists of 34 texts with more than 16,000 annotated sentences covering the time period from 1616 to 1942 (cf. https://gitlab.gwdg.de/mona/korpus-public/-/releases/v5.1, data publication (see section 7) and Table 1). In general, the first approximately 400 sentences of each text were annotated by two annotators with a background in German Philology. Two to three experts (authors of this paper) created gold standards for all texts and collaboratively adjudicated the initial annotations (i.e. review, accept, correct or delete).

We computed inter-annotator agreement on clause-level based on Fleiss’ Kappa (κ, Fleiss 1971) and Mathet’s Gamma (γ, Mathet et al. 2015, cf. Table 2). κ calculates agreement based on the differences for each clause, while γ respects the individual annotated passages as units in a continuum, and also partially overlapping passages are compared as units instead of disjointed clauses. We therefore consider that γ better represents the errors made by annotators for a category with rather long passages such as reflection.8 Using Landis and Koch (1977)’s guideline for interpreting the results of κ, we achieve moderate values for comment and substantial values for both, generalisation and non-fictional speech for κ (see Table 2). In our perception, γ generally tends to yield more conservative values compared to κ.

So far, we have presented the theoretical background for and our operationalisation of ‘reflective passages’ and the associated phenomena of ‘comment’, ‘non-fictional speech’ and ‘generalisation’ as well as our annotation results. We stipulated that whenever at least one of these three phenomena is present, such a passage is a reflective passage. In the following section, we will introduce the second central term for the envisioned diachronic analysis: perceived reflectiveness, represented by the ‘reflection score’.

We tested the perception of reflectiveness in a reception experiment conducted via a survey. In particular, we were interested in the contribution of individual phenomena (generalisation, comment, non-fictional speech) to the overall reflectiveness of a text passage and whether the passages that were not annotated with any of the above-mentioned phenomena can be perceived as reflective. Our objective is to quantify the contribution of the three phenomena and their combinations to the perception of a textual passage as reflective.

The survey was designed as follows: First, we extracted passages from our corpus, more precisely, from texts after 1850 (because we assumed that our participants would more readily understand the language in these more modern texts than in many of the earlier texts). The extracted passages consisted of one sentence and were annotated with the tags generalisation, comment, non-fictional speech or their combinations.

Second, we manually chose ten sentences for each of the following groups:

Additionally, we extracted passages that do not carry any of these tags as negative examples. Altogether there were 100 passages in the survey.

For a better understanding of the passage, we provide the survey participants with the context of one sentence before and one sentence after the passage. The passage in question is highlighted (see Figure 1). Each passage is accompanied by the following question and the corresponding answer options on a scale of 1 to 5:

For our experiment, we used the web-based survey tool LimeSurvey (LimeSurvey 2024). It allows us to give the participants 30 randomly selected passages. We chose 30 passages as a good trade-off between obtaining sufficient coverage for each passage in the survey while at the same time limiting the experimentation time for the participants. In total, we received 118 complete answers, in which the participants provided their assessments for all 30 passages.

For a statistical analysis, we averaged the ratings from all participants for each passage. When we speak of “reflection ratings”, we refer to these averages. The left column in Table 3 shows that all three phenomena correlate with the reflection ratings, but to a varying degree. Using Dancey and Reidy 2004’s naming convention, the correlation is weak for non-fictional speech and generalisation, and moderate for comment. This illustrates that none of our phenomena is perfectly congruent to (perceived) reflection.

In a next step, we created a logistic regression model to gain insights into the interplay between the phenomena. As features, we used the three phenomena as main effects as well as all combinations as interaction effects. We ran both forward selection and backward elimination to determine the best model in terms of the Akaike Information Criterion (AIC), both leading to the same result: a model using all main effects and the interaction effect generalisation*comment. The model’s coefficients are shown in the right column of Table 3. Note that the regression coefficients of the main effects are sorted in the same way as their correlation coefficients.

Using the regression coefficients, we can calculate a reflection score r for any passage with known labels for generalisation, comment or non-fictional speech as follows:

σ⁢(x) denotes the logistic sigmoid function 11+e–x. This means that, for example, a passage that is annotated as comment but neither generalisation nor non-fictional speech receives the following reflection score:

The value of r lies between 0 and 1. Since 0.64 > 0.5, the reflection score for comment-only passages can be interpreted as “reflective”. Table 4 shows that:

Generally, the presence of each of our phenomena increases the reflection score, which runs between 0.33 and 0.73.

While further research would be necessary to understand why certain combinations tend to be perceived as reflective more often than others, another question is whether the perception of a reflective passage actually triggers reflection on the part of the reader. We have to leave such intriguing questions to (psychological) researchers, but we may emphasize two more general insights from our experiment: On the one hand, we can assume that our ‘flexible’ operationalisation of a “reflective passage” captures basic intuitions about what it is “to reflect upon something”. On the other hand, this results in a hierarchisation of the sub-phenomena we examined, which have a varying degree of influence on whether a certain passage is perceived as reflective.

So far, we developed a basic definition of “reflective passage” and a more complex reflection score in order to analyse literary reflection. Since both rely on the identification of the three reflective sub-phenomena (generalisation, comment and non-fictional speech), we trained two neural classifiers for the automatic tagging of these phenomena: one multi-tagger and, additionally, one binary tagger (reflective vs. non-reflective passage). To our knowledge, this has not been tried before. Each classifier takes a text span of three sentences as input, where one clause of the inner sentence is marked, and was trained to predict the categories of the marked clause.10 We split our corpus text-wise into training, development and test set so that the distribution of generalisation, comment and non-fictional speech is similar in all sets. Wieland’s The History of Agathon and Seghers’ The Seventh Cross are held out for the evaluation of the models, and Fontane’s The Stechlin and Mann’s The Magic Mountain serve as development set, while the other texts are used for training.11 The classifiers are available through the software package (Dönicke et al. 2022).12

We followed the approach of Schomacker et al. 2022. The multi-label classifier has three output neurons, where each neuron corresponds to one tag (generalisation, comment, non-fictional speech), and the binary classifier has one (reflective). Both classifiers are based on a large BERT model, that was pre-trained on German data (Chan et al. 2020)13 and were trained for 20 epochs with a batch size of 8. To increase the convergence speed, we used the LAMB optimiser with a learning rate of 10^–4 (You et al. 2020). Furthermore, we set the hidden dropout to 0.3 and the attention dropout to 0.0.

Table 5 shows the Precision, Recall and F-score of our classifiers on the test texts (cf. Sokolova and Lapalme 2009). For generalisation, the multi-label reflection classifier performs with 61% F1 like the binary generalisation-only classifier from Schomacker et al. 2022, which illustrates that the other two phenomena can be learned in addition without performance loss. The same classifier achieves with 69% F1 the best results for comment, hereby outperforming the statistical comment-only classifier from Weimer et al. (2022) by 10%. Overall, the multi-label reflection classifier achieves a micro-averaged F1 score of 66% and the binary reflection classifier adds 3% on top of that. While the multi-label classifier achieves similar performance on both test texts (± 7%), the binary classifier shows a greater variation in F1 (± 18%).

This section will first introduce our diachronic corpus “KOLIMO-selection” (1700–1945, see subsection 5.1). In a second step, we report the results of our diachronic corpus analysis (see subsection 5.2). In addition to the “reflection score”, we analysed the presence of the three subtypes of reflective passages (comment, generalisation, non-fictional speech) that, according to our initial definition, constitute a “reflective passage”. In a third step, we took into account potential covariates that may relate to the distribution of reflective passages in literary history, like text length, canonisation status and authors’ sex (see subsection 5.3).

A so far unfulfilled promise of Computational Literary Studies is to write a more empirically saturated history of literature. Our aim in this paper was to contribute to this new literary history through a diachronic analysis of the narratological phenomenon of reflective passages. Our approach illustrates how many different elements have to come together to get closer to this goal: After 1) a resource-intensive annotation of more than 16,000 sentences for the phenomenon of reflection, we were able 2) to build a multi-label and a binary classifier for reflective passages. 3) We studied how different types of reflective passages are perceived by actual readers and introduced the reflection score as a measure of the perceived reflectiveness of a textual passage. 4) Through a complex filtering process, we build a suitable diachronic corpus of 5,200 original German language fiction texts from the much larger KOLIMO corpus and 5) enriched their metadata regarding fictionality status, canonisation status and authors’ sex. Finally, we were able to analyse the frequency of reflective passages over 250 years of literary history. Our findings suggest three boom periods of reflective passages: around 1755, 1835 and 1920. generalisation is the most common phenomenon (M=17.6% of all clauses), comment the second common (M=15.6%), while non-fictional speech is rather rare (M=5.6%). In terms of perceived reflectiveness, all sub-phenomena contribute to a textual passage’s reflectiveness, while comment is the best indicator, generalisation plus non-fictional speech also indicate reflectiveness. Important covariates of the proportion of reflective clauses are text length, canonisation status and authors’ sex. On average, longer texts, canonised texts, and texts from male authors contain more reflective clauses than their respective counterparts. Since our diachronic corpus itself is only a (small) sample from the literary production in German language (Gittel 2021, 5), and – due to limited metadata – does allow to control only a few potential covariates that steer literary production, our results should be regarded as motivation for further quantitative research in the future. Nevertheless, our research represents a step forward towards an empiricisation of literary studies. It demonstrates that quantitative research can underpin existing hypotheses in literary studies (like the one from a boom of essayism at the beginning of the 20^th century) and set new questions on the agenda (e.g. about the nature of the boom of reflection in the Vormärz period). To answer such questions, Computational Literary Studies and hermeneutic research need to go hand in hand in our opinion. Quantitative research may in the future shed light on the thematic contents of the different subtypes of reflection and their combinations – a question deliberately put aside in the present paper – and hermeneutic research may formulate justified hypotheses about the functions of different types of reflective passages in specific contexts. In this way, literary studies may advance towards an empirically saturated functional literary history.

Data can be found here: https://zenodo.org/records/10246193, and here: https://doi.org/10.5281/zenodo.11164190.

Software can be found here: https://doi.org/10.5281/zenodo.11163719, and here: https://doi.org/10.5281/zenodo.11164036.