Who Knows What in German Drama? A Composite Annotation Scheme for Knowledge Transfer. Annotation, Evaluation, and Analysis

1. Introduction

“A play should lead up to and away from a central crisis, and this crisis should consist in a discovery by the leading character which has an indelible effect on his thought and emotion and completely alters his course of action,” (Anderson 1965, p. 116) stated American playwright Maxwell Anderson (1888–1959) in an essay titled The Essence of Tragedy (1939). In his essay, Anderson was in search of a formula for writing a successful play. After producing a number of what he called accidentally successful plays and some box office failures, he wondered “whether or not there were general laws of governing dramatic structure which so poor a head for theory as my own might grasp and use,” (Anderson 1965, pp. 114–115) in a bid to reduce “some of the gamble […] of play-writing.” (Anderson 1965, p. 115) He found his answer in Aristotle’s Poetics. To be precise, he found it in Aristotle’s discussion of recognition scenes, i. e., “a change from ignorance to knowledge,” (Aristotle 1995, p. 65) which Anderson transferred into a poetology of his own. With regard to Aristotle’s remarks, Anderson characterised scenes of recognition as “essential to tragedy.” (Anderson 1965, p. 115) He stated that a playwright must “follow the ancient Aristotelian rule: he must build his plot around a scene wherein his hero discovers some mortal frailty or stupidity in himself and faces life armed with a new wisdom.” (Anderson 1965, p. 120) In Anderson’s view, then, recognition scenes, which lead to a central crisis, play a major role in shaping the course of action and the play’s impact on the audience.¹ Although we are studying recognition scenes in plays, they are a common feature not only of tragedy or drama, but of literature as a whole. They are neither limited to high, middle or low brow literature nor to certain genres or literary periods (cf. Cave 1988, pp. 1–9). The revelation of the perpetrator in a crime novel, and how they are found guilty, can be seen as similar to recognition scenes in plays. In the following, we will limit the scope of our discussions to typically permanent relations between characters such as family relations.

An instructive example for recognition scenes, which we will use not only to illustrate the phenomenon, but also to explain our methodological approach, is Karoline von Günderrode’s two-act play Udohla (1805). The play revolves around effects that, according to Terence Cave, are substantial for discovery: “knowledge and the means of acquiring it, with secrets, disguises, lapses of memory, clues, signs and the like.” (Cave 1988, p. 2) Günderrode and her writings were virtually forgotten until Christa Wolf published selected works by her in the late 1970s (cf. Lipinski 2011, p. 113). Udohla, one of ten plays Günderrode authored, is set in a palace and its adjacent garden in Delhi. The play’s constellation of characters is presented as a “familial muddle” (Engelstein 2004, p. 281), i. e., family relations that are at first not transparent – neither for the audience nor for the characters appearing in the play – and later turn out to be different than expected. The play’s plot is initially marked by two important moments, both of which concern the reigning Sultan of the Mughal Empire. First, members of the Sultan’s staff, namely the vizier Mangu, the Hindu Sino, and the Dervish, argue about whether the Sultan is going to marry his recently reappeared sister Nerissa. Intrafictionally, a sibling marriage would violate Mongolian Muslim law, but not that of the hierarchically subordinate Hindu population (cf. Günderrode 1990, pp. 204–205).² The Sultan himself is seemingly ambivalent about his desires and questions the motives of God when asking (cf. Licher 1996, p. 189): “Warum o Schicksal, muß ich diese lieben? / Die Einzige die du mir hast versagt.” (Günderrode 1990, p. 209) (“Why oh fate, must I love her? / The only one you have denied me”).³ Second, the Sultan is also told that the death sentence against Bahadar, a Hindu rebel and political traitor, has been carried out, but Bahadar’s two children have escaped. Both pieces of information have implications for the further course of the plot. Over several steps of knowledge transmission, it turns out that Nerissa is not the Sultan’s long-lost sister. Instead, she is the daughter of the previously executed Bahadar. At the same time, it becomes clear that Nerissa is the sister of the titular character Udohla. Pretending to be a relative of the Nawab,⁴ Udohla attempts to outsmart the Sultan in a bid to free his father from captivity, which – as the audience already knows – is certain to fail from the beginning. As we can see, just as in Aristotle’s prime example Oedipus Rex, the scenes of recognition in Günderrode’s play focus primarily on family relations.

In our article, we will extend this small-scale example on the connection of family relations, the knowledge about them and a central discovery to a larger corpus of plays. For this purpose, we present a framework for the formal modelling and quantitative analysis of family-related knowledge transfers in German plays of the eighteenth and nineteenth century. By means of (manual) annotation we will operationalise⁵ knowledge transfers and thereby intertwine a content-focused approach with already established procedures of quantitative drama analysis concentrating on structural properties of theatre plays.⁶ We use annotation as a method that enriches texts or text segments with certain information, whereby the annotations takes on different functions (cf. Pagel et al. 2020, pp. 125–141). On the one hand, we employ it to further develop and refine established quantitative methods of text analysis. This way, the annotations become part of the analysis of a play or a corpus of plays, and can support the interpretation. On the other hand, the annotations will serve as training or test data for future machine learning procedures.

In our article, we will first set forth our theoretical framework from a literary studies perspective drawing upon Aristotle’s Poetics (section 2). Secondly, we will introduce our annotation scheme in detail (section 3). In doing so, we will illustrate how to identify text passages that include a transfer of knowledge concerning family relations and how to label them with our annotation scheme. We include knowledge changes for characters present on stage as well as the audience. Thirdly, we go on to discuss the calculation of inter-annotator agreement for our annotated data (section 4). As there is no standardised procedure yet to convincingly measure the agreement of our annotators, we will discuss some of the options and challenges when measuring agreement from a theoretical and practical point of view. Lastly, we will analyse the data we obtained during our annotation process (section 5). We will thus focus on three different perspectives. Analysing our corpus of 20 plays, we will examine at what point in drama new knowledge about family and love relations is distributed and how the internal and external communication systems are involved. Secondly, we will focus on one key piece of information and present a visualisation of knowledge flow based on our annotations. Our third perspective concentrates on a methodological question: Can we use our annotation data to employ new, more content-based ways of literary network analysis? Can this approach help in identifying important characters for the play’s action and contribute to improving the integration of quantitative network analysis with qualitative close readings, thus bridging the perceived gap of quantitative and qualitative methods (cf. Mueller 2012)? We will discuss these questions with regard to Günderrode’s Udohla.

2. The Distribution of Knowledge in Plays

The interplay of internal and external communication systems in drama, i. e. the communication of the fictional characters on the one hand and the perception of this communication by the audience on the other, is considered one of the central “qualities necessary for identifying dramatic communication” (Pfister 1988, p. 49). As Bernhard Asmuth points out in his introduction to drama analysis, a play as a whole is not only a sequence of actions, but also a multi-perspectival processing of knowledge (cf. Asmuth 2016, p. 114).⁷ In the light of events that may have taken place before the actual plot of the play’s main text, characters are – potentially – already set apart from each other by a different degree of knowledge. Herein, we employ a broad understanding of knowledge that is not strictly limited to the classical notion of propositional knowledge as “justified true belief” (e. g., Pollock and Cruz 1999, p. 13 or Ichikawa and Steup 2018) which originated from Plato.⁸ As it is not uncommon for literature to deliberately play with knowledge, facts, beliefs, hearsay, and rumours,⁹ we opt for a more “lightweight sense of knowledge” (Ichikawa and Steup 2018). In our case, this includes beliefs that are both justified and depicted to be true, but might later turn out to be false, e. g., through scenes of recognition.

A character’s level of knowledge can change continuously in the course of the play. At the same time, the relationship between the audience’s level of information and that of the individual characters in the play is constantly adjusted. The exposition, e. g., reduces the knowledge gap between the audience and the characters that prevails at the beginning of a play (cf. Asmuth 2016, p. 122). The disparities in the “levels of awareness” (Pfister 1988, p. 49) can be attributed primarily to two causal differences between the internal and external communication systems: While the audience in its observer role perceives every scene of the play and can thus compare and aggregate partial knowledge of the characters, it sometimes remains unclear what prior knowledge the characters actually have. This also applies to possible time leaps, for instance between two acts of the play. Furthermore, it might not be clear to what extent the statements of a character correspond to the “facts” of the fictional world, i. e., whether the statements are credible (cf. Jeßing 2015, pp. 50–51). Depending on the course of the plot, the audience can either have an information advantage or an information disadvantage over the characters acting on stage at different times. The relative level of being informed between the audience and a character can change from scene to scene. The same applies to the internal communication system of the plays’ characters, when comparing the degree of knowledge different characters have in a certain scene. For this phenomenon, Bertrand Evans coined the term “discrepant awareness” (Evans 1960, p. VIII). This “discrepant awareness” between two characters can thus lead to rather different evaluations of the same action or situation. If we think of Günderrode’s Udohla, a character’s judgement of the supposed marriage between the Sultan and Nerrisa would greatly depend on whether the character knows that the Sultan and Nerrisa are not siblings and on the character’s religious views, i. e., them being Hindu or Muslim. In this situation, the lack of knowledge or a perceived, but actually incomplete awareness will influence the judgement in one way or the other.

The gap between the characters’ level of knowledge and that of the audience can be seen as an important element of suspense in drama, as it ensures sustained attention and emotional excitement (cf. Anz 2007, p. 464). This applies to both the suspense felt when one is curious about what is going to come up next and the suspense arising in respect to how something that is already known to be happening is going to happen.¹⁰ In this respect, the device of dramatic irony is particularly important, as it is based precisely on this gap of being informed. The audience’s knowledge advantage with respect to an upcoming action is, thus, a prerequisite for dramatic irony. In understanding a remark that is innocuous from the perspective of the speaking character, the audience can interpret the utterance as an allusion to the catastrophe that is later actually realised.¹¹ Consequently, elements such as dramatic irony are closely linked to the play’s effect on the audience: Is the play supposed to convey a moral theorem? Is it meant to purify the audience’s affects? Should it educate the audience? Or is it simply meant to entertain? In his Poetics, Aristotle defines the (cathartic) effect as the central concern of tragedy.¹² He considers reversal (peripeteia) and recognition (anagnorisis) as important building blocks to evoke pity and fear, the desired affects caused by a tragedy.¹³ Recognition is directly related to Evans’ concept of “discrepant awareness,” for Aristotle defines recognition as “a change from ignorance to knowledge, leading to friendship or to enmity, and involving matters which bear on prosperity or adversity.” (Aristotle 1995, p. 65) Since such scenes of recognition are ideally linked to the reversal, i. e., “a change to the opposite direction of events” (Aristotle 1995, p. 65), they represent central moments of knowledge transmissions that can be decisive for understanding and interpreting a play. The examples Aristotle used to illustrate recognition and reversal “are taken solely from the field of familial philia” (Destrée 2020, p. 117). This is one reason why our annotation experiments focus on knowledge about family relations.

3. Annotating Knowledge Transfers

The aim of our research is to model knowledge transfers in German plays by means of annotation. This enables us to empirically analyse the textual implementation of the theoretical considerations described in the previous chapter. While knowledge is a broad phenomenon, we restrict our annotation to the domain of knowledge about familial character relations. We employ a wide understanding of knowledge that does not imply that the information at hand must be correct. We therefore also include beliefs. In this section, we will present our current annotation scheme. We developed the guideline by annotating 16 plays in the course of roughly a year. The full (German) guideline as used by our annotators is available online.¹⁴ The annotation is performed using the tool CorefAnnotator¹⁵ (Reiter 2018).

Our annotation scheme targets text sections in which knowledge transfers take place. More precisely, we annotate a text section if:

• a)  the knowledge concerning character relations of at least one of the characters or the audience is changed, or

• b)  a character’s knowledge about the knowledge of another character is changed.

A case of a) would be a text section in which character A learns that B and C are siblings. An example for b) is a section in which B learns that A knows that B and C are siblings. The latter can be understood as knowledge about knowledge, or meta-knowledge.

Annotation spans are not fixed to a specific length. Knowledge transfers can happen in one sentence or even a word, but can also be extended over a whole paragraph, especially when knowledge is distributed implicitly. However, our annotators are encouraged to identify a span that is as short as possible. When a relevant text span is identified, it is annotated with a label that uses this pattern:¹⁶

The SOURCE is usually a character that provides a piece of information, but can also be an object or an action that allows for inferences about character relations, for instance when Saladin recognises the handwriting of his brother in Lessing’s Nathan der Weise (1779). The TARGET is always a character or a group of characters (and/or possibly the audience) whose knowledge is changed. The item KNOWLEDGE is restricted to knowledge about character relations and, more precisely, to the set of relations presented in Table 1. Optionally, ATTRIBUTEs can be added, for instance to mark the information as a lie or as uncertain. The latter is especially frequent as many dramatic texts play with strong allusions to a fact that is ultimately confirmed only at the end.

In Udohla, by the playwright Karoline von Günderrode, the vizier Mangu lets the audience know that Nerissa is the Sultan’s sister (which turns out to be wrong). This is annotated as follows:

Characters are referenced by the identifier they receive in the Drama Corpora Project (DraCor, Fischer et al. 2019). Characters that do not have dialogue, do not have such an identifier. Instead, they are given an identifier by our annotators. Frequently, characters are not introduced by name and their identity is (partly) unclear. We annotate such character mentions as a variable in capital letters. In the play Magie und Schicksal (1805) (“Magic and Destiny”) by Günderrode, the character Cassandra mentions a son whom the audience did not hear about before. At first, we do not have any additional knowledge about this son and therefore annotate him as a variable:

Later in the play, it is revealed that the character Ligares is in fact the mentioned child of Cassandra. We can now annotate that the variable CHILD[CASSANDRA] and Ligares are identical:

Note that it is also possible to fill any of the positions in the annotation label with a list of several characters by enclosing them in square brackets. This is used extensively, for instance, when Nerissa (in Udohla) reveals in the final scene that she is the daughter of the Sultan’s enemy Bahadar (who was just killed on his order):

As mentioned above, we restrict our annotations to the domain of knowledge about character relations, i. e. family and love relations. Table 1 gives an overview of all character relations that are included in the annotation scheme. Formally, we differentiate directed relations such as parent_of(PARENT, CHILD), where the position of characters is important because of the asymmetry of the relation, from undirected relations. When annotating undirected, symmetric relations such as siblings(SIBLING-A, SIBLING-B), the order of the characters is irrelevant. Semantically, the relations form three groups, the biggest of which are family relations and love relations.¹⁷ The last group of identity relations is not about relations between two characters in the strict sense, but includes a) cases where we learn a (first, or additional) name of a character, and b) cases where two characters are revealed to be the same, as in example 4. All relations in the table can be negated by adding a ! at the beginning, e. g., !siblings(nerissa, sultan) to express that Nerissa and the Sultan are not siblings.

While the annotation guideline covers most of the knowledge transfers happening in the plays, some challenges remain. This is related to the rather simplistic communication model that underlies the scheme. Although we try to include rules of pragmatic communication in our annotation decision, we formally conceptualise knowledge distribution as transfer: The knowledge of one character is transferred to another character. This is motivated by the structure of plays, which is characterised by the alternation of character speech. This view assumes, however, that the communicated information is understood in the intended way. While this might be true in many cases, there are exceptions. There can be misunderstandings, pieces of information can be interpreted in different ways, and different prior knowledge or values can influence the understanding. Currently we also assume that the communicating characters are transparent to all characters involved. This is not true for text passages where characters transfer knowledge to a character whose identity is unclear to the speaker. For instance, in Schiller’s Braut von Messina (1803), Don Cesar confesses his love to Beatrice without even knowing her name – not to mention that she is also his brother’s lover and his sister. The annotation shown in example 6 captures the view of the audience but does not conform to the perspective of Don Cesar. While the guidelines do provide solutions for such scenes, we want to improve their generalisability in future versions, once more texts with similar constellations have been annotated.

All plays are annotated by two student annotators independently, then all deviations between the two versions are discussed with one of the authors. Afterwards, each of the annotators produces a revised version. Contrary to many other annotation projects, the aim of this step is not to create one consensus version of the annotation. The annotation task is complex and many text passages can be interpreted in more than one way. In addition, the annotation scheme sometimes allows for different ways of modelling a knowledge transfer. Therefore, the revision focuses on plausibility, consistency and formal correctness of the annotations. However, additional consensus versions were created for analyses that require one single reference version, because the focus does not lie on annotation variation (as in section 5.3).

Table 2 gives an overview of the current state of our corpus of annotated plays. Our choice of texts was guided by the goal to cover a broad range of phenomena, as this will allow us to:

1. create an annotation guideline that covers as many textual phenomena as possible and is largely applicable to unseen texts without requiring additions and modifications, and

2. in the future, work on the automation of the annotation and train a robust machine learning model that is able to generalise across different epochs, authors, and genres.

We therefore combined plays in which we knew that knowledge about character relations is important for the plot (based on prior readings and secondary literature) with canonical plays with a less obvious focus on family relations. We included tragedies as well as comedies and did not restrict ourselves to a specific epoch, but annotated a broad mix of plays from the eighteenth and nineteenth century.

In the process of developing the annotation guideline, we annotated 16 dramatic texts. Once the guidelines were consolidated, we began tracking the initial versions of our annotators for the calculation of inter-annotator agreement. The annotation of this second round is ongoing. We have chosen four of these plays to develop a suitable way of determining inter-annotator agreement for a complex annotation task as this. The next section will present and discuss our current measure.

4. Calculating Inter-Annotator Agreement

For manual annotation and coding tasks in a wide range of disciplines, measuring inter-annotator agreement (IAA, sometimes also called “inter-coder reliability”) is a standard procedure (cf. Artstein and Poesio 2008; Krippendorff 2004), much like the evaluation of automatic predictions based on machine learning. The goal of this metric is to have a quantitative view on the agreement between annotators, and ultimately to evaluate the quality of the annotation guidelines, the annotation process, or the annotations themselves. Unlike an evaluation of automatic predictions, there is no ‘gold standard,’ i.e., no set of annotations is considered to be true. Instead, IAA ‘only’ measures the agreement. A corner stone of IAA metrics is to take into account expected agreement (also called ‘chance agreement’), i.e., agreement that is achieved by random annotation decisions. This is done to compensate for the difficulty of the task; if there are more classification categories, the task is considered to be more difficult because there are more options to choose from, and the expected agreement decreases. On structurally simple tasks such as part-of-speech tagging, measuring IAA is well established and understood: Fleiss’ Kappa (Fleiss 1971), for instance, can be used to calculate the IAA between n annotators, who assigned one of k categories to each of N items.¹⁸

The annotation task we discuss in this article, however, is more complex: i) Annotation decisions are not made in isolation, but depend on the textual context as much as on previously made decisions. As we are only annotating the transfer of new information to the target, a subsequent mention of the same information by and to the same character is not annotated. Consequently, each annotation label may only appear once in a text.¹⁹ ii) After having decided that a knowledge transfer takes place (and selecting the exact boundaries), annotators need to make decisions about the SOURCE and TARGET of the transfer, the participants of the character relation and its direction, and, finally, about potential attributes of the annotation (e.,g., the transfer being a lie). iii) The annotation is not done on fixed, pre-defined units, but the annotation spans can be defined freely. All three properties make measuring IAA difficult.

The metric Gamma (Mathet et al. 2015) has been proposed as a versatile, highly adaptable metric for various tasks. It has several properties that make it promising for our use case: i) To calculate expected agreement, it samples a large number of random annotations from the existing annotations. Based on these random annotations, we can compute expected agreement in the same way we calculate observed agreement. This way, expected agreement can be measured empirically instead of theoretically, which makes it less dependent on assumptions and more widely applicable. ii) Equality between annotation categories can be graded. Instead of only recognising that transfer(X, Y, parent_of(P, C)) is different from transfer(X, Z, parent_of(P, C)), we can provide a function to express the similarity of the two annotations as a value between zero and one. This allows us to define the similarity of the annotations above to be less than one, but larger than zero. iii) For measuring observed agreement, Gamma first establishes an alignment between the different annotators’ annotations. This alignment can also be visualised and inspected, which is a helpful tool in the annotation process. Figure 1 shows an example for the established alignments. The overall Gamma score is calculated based on pairwise similarity functions between two (or more) annotations that are aligned. Since Gamma is computed over disagreements instead of agreements, we will discuss the calculation of disagreements in the following.²⁰ The final Gamma score, however, can be interpreted in the same way as other metrics: The higher the score, the better the agreement.

Figure 1

An alignment between the annotations in Günderrode’s Magie und Schicksal as established by Gamma.

The final Gamma value is a weighted combination of two aspects of disagreement. Positional disagreement expresses how different the annotations’ positions are, while categorical disagreement compares the labels that the annotators have assigned. The exact calculation of positional and categorical disagreement, as well as the weighting of these two components, can be customised. The two values are not fully independent of each other, as the alignment of the annotations takes the labels into account, i.e., Gamma attempts to align annotations with the same label.

5. Analysing Annotated Knowledge Transfers

The following section is dedicated to analysing our annotated corpus, with a focus on three different aspects of our annotations, as a showcase of how our annotations can be further used. As a first investigation, we concentrate on the quantitative properties of the annotated relations and provide an overview of the annotations (5.1): How many knowledge transfers are annotated per play? Which character relations are the most frequent? And when in a play is knowledge distributed in the inner and outer communication system? We discuss the results with regard to established views in drama theory. Second, we focus on how one key piece of information is distributed among the characters, and present a new visualisation for such knowledge flows (5.2). Third, we further explore the potential of the annotations in a network analysis of Günderrode’s Udohla. Dramatic network analysis is currently based mostly on so-called configurations (cf. Pfister 1988, pp. 171–176). Using a more content-based form of character networks by exploiting our annotations, we attempt to chart a path to better integrate quantitative analysis and interpretative reading. We thus not only visualise the annotated knowledge transfers as a network, but also compare different characters in view of centrality measures (5.3). As we have argued in the previous section, there can be more than one way of interpreting a text and possibly also more than one way of modelling knowledge transfers in our annotation scheme. While we use both versions of the annotations for our statistical analyses in Section 5.1, we have created a consensus version for the analysis of Günderrode’s Udohla for simplicity.

5.1 Quantitative Overview of the Annotations

In total, our analysed corpus consists of 20 plays (see Table 2 for an overview) annotated by two annotators. It contains 551 transfer annotations for Annotator 1 and 506 transfer annotations for Annotator 2. Averaged over both annotations, there are 26.4 (±11.5) annotations per play and 1.06 (±0.56) annotations per 1000 tokens. The standard deviations indicate a substantial variation between the plays. Table 4 shows how these values are distributed in detail. Table 5 displays the ten most frequently annotated relations for each annotator. Overall, the ranking is fairly similar for both annotators, and two adjacent relations switch ranks only twice. The relation in_love_with is by far the most frequent, with its negation following shortly after. In contrast to most family relations, love relations can change over time. They can be hinted at, be the content of rumours, or trigger an important conflict for a play’s rising action. Hence, they are talked about more often than other relations. The identity relation occupies the second rank. It is most frequently used for characters that are at first mentioned without name and therefore annotated by a variable that is later unified with their character id. Unsurprisingly, the relations child_of and parent_of are also frequently used. These mark the importance of the core family for the plot of our selected plays.

Table 4

The number and density of transfers for both annotations.

	Annotation 1	Annotation 2
No. of transfers	551	506
Average no. of transfers per play	26.24±11.58	26.63±11.65
No. of transfers per 1000 tokens	1.11±0.54	1.01±0.58

Table 5

Ten most frequently annotated relations per annotator.

Annotation 1		Annotation 2
Relation	Count	Relation	Count
in_love_with	114	in_love_with	112
child_of	72	identity	74
identity	72	child_of	54
parent_of	43	parent_of	45
!in_love_with	42	has_name	44
has_name	36	!in_love_with	36
engaged	32	engaged	32
siblings	26	siblings	25
lovers	16	spouses	16
spouses	16	lovers	11

When combining annotation 1 and 2, around 50% of the characters are involved in knowledge transfers, with 38% being the source and around 43% being the target of knowledge transfers at least once. In 45% of the cases, the SOURCE transfers a relation involving themselves. In contrast, in only 5% of all cases, the TARGET learns about a relation concerning themselves. It is evident that characters possess the most knowledge about their own relations and can therefore pass on this knowledge reliably. For the same reason, learning about one’s own family or love relations is rather rare, but might point to especially interesting passages of the plot, as Udohla has exemplified. Table 6 shows a detailed breakdown of the numbers per annotator.

Table 6

Overview of characters and their involvement in knowledge transfers.

	Annotation 1		Annotation 2
	Total	Percent	Total	Percent
No. of characters	289	NA	289	NA
No. of characters involved in transfer	134	46.37	127	43.94
… as SOURCE	105	36.33	95	32.87
… as TARGET	118	40.83	111	38.41
Character relays information about themself	258	0.47	215	0.42
Character receives information about themself	32	0.06	24	0.05

Additionally, we also investigate when in a play knowledge transfers happen. Figure 2 shows the number of annotations over the relative position at which they occur and whom they are directed at: other characters in the internal communication system, the audience, or both. The position in this analysis encompasses the entire text, including the dramatis personæ. We bin the number of annotations, so that each bar covers a range of 5%. Thus, 55 annotations were made by Annotator 1 in the first 5% of all plays with the audience as the target, 26 annotations were made in the next 5 %, and so on. We can see that the segments right at the beginning and end of a play are the ones with the highest number of annotations. The remaining segments of the plays have a more or less similar distribution of annotations with increases in the middle of the plays and in the final quarter. At the beginning of the plays, the majority of information is transferred to the audience (blue bars), while this focus shifts to the characters towards the middle and end of the plays (red bars).

Figure 2

Distribution of annotations by Annotator 1(2a) and Annotator 2(2b) over the relative positions of the 20 plays. The annotations are separated by the target of the knowledge transfer: (i) Only the audience is the target, (ii) only characters are the target, but not the audience, or (iii) the audience and one or more characters are the target. Note that the latter is displayed twice to get the correct total for both audience and characters.

This observation can be explained conclusively as it supports established drama theory. The beginning and end of a play are central places for the transmission of knowledge, both in the internal and the external communication system. “What we understand as the transmission of information at the beginning of a play largely coincides with the classical theoretical concept of the exposition,” acknowledges Manfred Pfister (1988, p. 86). He goes on to define the exposition as forwarding of information concerning “events and situations from the past that determine the dramatic present”.²⁵ With regard to the audience, the transmission of information in the character’s internal communication system or the dramatis personæ fulfils at least two functions: On the one hand, it is intended to instigate the audience’s attention. On the other hand, the audience is provided with the knowledge necessary to understand the subsequent actions (cf. Asmuth 2016, pp. 103–105). As Figure 2 illustrates for our annotated corpus, most of the knowledge about family and love relations that is transferred at the beginning of a play is indeed directed at the audience – oftentimes even solely. Similar to the exposition at the beginning, the resolution at the closure of a play is a common section for transmitting unknown information, e. g., through recognition. In such closed endings, deviations in knowledge between characters and the audience are typically dissolved: “as a result of either intrigue, self-deception or lack of information”, a character or even a group of characters have gotten into trouble. “This situation then culminates in either a happy or a tragic ending, after additional information has been introduced” (Pfister 1988, p. 95). The values in Figure 2 show a shift of direction towards the end of the annotated plays. About halfway through the plays, the number of annotations directed at characters in the internal communication system increases relatively to those directed at the audience. The knowledge that is transmitted in the resolution is frequently addressed to the plays’ characters. The audience, in turn, already possesses the information necessary to deduce the probable outcome. Thus, the suspense felt by the audience at the end – at least in our corpus – seems to be in respect to how an information they possess influences the characters’ actions.

5.2 Tracking Knowledge Flow

We can exemplify these theoretical considerations that relate to the communication system when inspecting the distribution of a specific piece of information as presented in Figure 3. The two visualisations of knowledge transfers in Günderrode’s Udohla present an instructive way to display the flow of knowledge within the play. By discussing the possible marriage between the Sultan and Nerissa at the beginning of the play, Sinu, Mangu and the Dervish indirectly pass on their knowledge to the audience. In doing so, the audience is also informed that Nerissa and the Sultan are siblings – at least according to the current beliefs of the present characters. As Figure 3a visualises, it is Nerissa herself that corrects this wrong piece of information for the audience while she is talking to Elpa. From there on the audience has an information advantage over most of the fictional characters. For the other characters, it takes until the middle of the second act, where Mangu receives a letter of the Sultan’s actual sister, to learn of this fact. Udohla, in the meantime, passes along the information that he is Bahadar’s son to Sino, who then passes the knowledge to Mangu between the scenes (see Figure 3b). Mangu in turn tells the sultan. The resolution at the end of the play then brings together the knowledge acquired by the various characters in the course of the play. Nerissa reveals that she is the daughter of Bahadar. She is the last character to learn that Udohla, too, is Bahadar’s child and thus her brother. The visualisations concisely depict the flow of these pieces of information and how they are transmitted from character to character or to the audience.

Figure 3

Knowledge flow over the two acts of Günderrode’s Udohla.

5.3 Networks of Knowledge Transfer

As a third kind of analysis, we use the annotated knowledge transfers to construct character networks. Networks, which are based on the knowledge about family relations and its dissemination in a play, can help to identify key characters that propel the dramatic plot either by gaining new information or by distributing it. In these networks, each node represents a character (or other sources of information such as letters, observations, etc.) and edges between nodes signify that one or more family-related knowledge transfers between two nodes have taken place.²⁶ They can therefore be used to complement the information gathered by established configuration based networks that focus on co-presence of characters. Since there is a SOURCE and a TARGET for each knowledge transfer, the networks are directed. The edges can be weighted with the total number of knowledge transfers that have taken place between two nodes. An example of such a network is shown in Figure 4 for Günderrode’s Udohla. The nodes are scaled according to their weighted degree (Barrat et al. 2004), which is a measure that calculates the sum of the weights of all incoming and outgoing edges for each node.

Figure 4

Network of knowledge transfer in Günderrode’s Udohla. Based on the consensus version.

The visualisation in Figure 4 shows Udohla, Sino, who is a Hindu staff member of the Sultan, and the vizier Mangu, to be the central characters of the network according to their weighted degree. At first glance, it might seem surprising that Sino and Mangu are two of the most central characters in the network. For the plot and its resolution, there are more important characters, mostly Nerissa,²⁷ Udohla and the Sultan. How can the central position of Sino and Mangu, i. e., their high weighted degree then be explained? For Sino, there are mainly two reasons. The first reason concerns the intra-fictional progression of the plot. Günderrode conceptualised Sino as Udohla’s only confidant within the Sultan’s palace (cf. Obermeier 1996, pp. 106–107). Both Sino and Udohla are Hindus and they are linked through a mutual close acquaintance. Naturally, then, Sino is the only character in the play Udohla could trust to share his real identity with, which is important for the play’s final resolution, as Sino is able to confirm to the Sultan that Udohla is Bahadar’s son. The second reason is that Sino’s role is used to transmit knowledge from the internal communication system to the audience. Herein, Sino becomes the recipient of new information, while primarily the audience is “the intended receiver of the information given.” (Pfister 1988, p. 89) To that effect, Mangu takes on a different role in the network. As he receives the letter of the Sultan’s real sister, he is then able to pass on the information that Nerissa is not the Sultan’s sister to other characters of the play. The audience, however, already knows this fact from an earlier conversation of Nerissa and Elpa.

To further track the development of knowledge in the course of Udohla, we bin the play’s text into 10 equal-length segments and create a network in each of these segments. In these networks, we calculate in- and out-strength. While the strength metric to scale the nodes in Figure 4 uses both incoming and outgoing edges, in-strength only considers incoming, and out-strength only considers outgoing edges for the calculation. Figure 5 shows cumulative curves for the development of both in-strength and out-strength in Udohla. Here, cumulative means that the networks of each bin are constructed by taking the annotations of the current bin and all previous bins into account. In this way, we can see which character received and transferred knowledge about family relations at what point in the play. There are some instructive observations that are in need of interpretation. Firstly, Udohla’s high out-strength value is mostly linked to a single scene in the middle of the play, where he introduces himself as Achmed pretending to be the Nawab’s herald. As he passes this false information to five other characters, it has a big impact on his central position in the network. Secondly, the roles of Sino and Mangu in the knowledge transfer network seem to be roughly comparable. Both receive information about family relations that they in turn pass on to other characters. While Sino can be described as a confidant of Udohla, Mangu, being a Muslim, takes on a similar role with regard to the Sultan. All of the important information that the Sultan receives before the final resolution comes from Mangu. Thirdly, the Sultan’s role in view of knowledge distribution is strikingly passive. He is only TARGET of knowledge transfers, never the SOURCE. This underlines a different conceptualisation of the Sultan’s character. Although he does indeed receive some information in the course of the play, oftentimes he is the last character to be reached by this knowledge. Looking at the play’s resolution, this makes sense. Being at the centre of the final recognition scene, the Sultan has to be unaware that Nerissa and Udohla are the children of Bahadar up to this point. Sino and Mangu, on the other hand, accumulate new knowledge throughout the play and serve as middlemen, bridging the knowledge either to the audience or to the main characters.

Figure 5

Cumulative in-strength and out-strength in the course of Günderrode’s Udohla for all the involved entities.

Following this, we investigate the so-called betweenness centrality (Freeman 1977) of the network. Betweenness centrality measures how often a node is part of a shortest path between two other nodes (cf. Freeman 1977, p. 37). Since betweenness centrality can be seen as a measure for the flow of communication in a network and how single nodes control the flow of communication, it appears to be especially suited for networks of knowledge transfer. Its “use seems natural in the study of communication networks where the potential for control of communication by individual points may be substantively relevant” (Freeman 1977, p. 40), as Linton Freeman states in his pioneering study. Figure 6 shows the development of the betweenness centrality for Udohla. We can see that Sino and especially Mangu are the characters with the highest betweenness centrality in the play. This further corroborates their role as middlemen in the play. Moreover, the visualisation illustrates that in Udohla, knowledge transfers responsible for betweenness centrality mostly occur in the second half or even the end of the play. Looking at the structure of a theatre play, this seems conclusive from a conceptual point of view. As a node has to be both TARGET and SOURCE of at least one knowledge transfer to be part of a shortest path, it is not surprising to find this realised only towards the end of Udohla. As shown above, the beginning and end of a play are key segments for the transmission of knowledge. In order to have a play’s resolution resulting directly from a recognition scene – as is demonstrated in Günderrode’s Udohla – the characters involved must possess a different knowledge base right until that moment.

Figure 6

Betweenness centrality in the course of Günderrode’s Udohla for all the involved entities. Three characters received a betweenness centrality value of 0 for all positions and were omitted from the graph: Derwisch, Elpa and the Sultan.

In summary, the analyses have raised a promising perspective for more extensive investigations on a bigger corpus. Our annotation data can provide insights into different structural principles of German plays. Sino’s and Mangu’s central position in the network and their values for in- and out-strength as well as betweenness centrality further show the potential of our methodological approach. As exemplified in Udohla, we can detect characters that have a key role for the flow of knowledge in the course of the play, without being considered main characters themselves. Although our networks are based on the transmission of knowledge about family relations, they depend on co-presence networks. Thus, they can be described as second-order networks. I. e., if in the course of a play two characters are not present on stage together, it is highly unlikely that new information circulates between them. Therefore, we consider a systematic comparison between co-presence networks and knowledge transfer networks as an especially insightful task for future research.

6. Conclusions

In this article, we have presented a composite scheme for the annotation of knowledge transfers about family relations in German plays. As illustrated throughout our article, annotating these knowledge transfers is a complex task, which gives rise to a number of challenges. Our scheme is based on considerations of drama theory on knowledge distribution. As our results are prospectively also intended to be of relevance for research in traditional literary studies, we have refrained from an operationalisation that overly simplifies concepts in light of computation. Instead, we chose an operationalisation that purposefully connects to terms and concepts of drama theory. As a consequence, the scheme is situated at the intersection between annotation and modelling.

At the same time, this project is (to our knowledge) the first that attempts to measure inter-annotator agreement for such a complex annotation task by employing the metric Gamma. We have discerned a number of intricacies that make the application of Gamma tricky and might be relevant for other annotation projects in computational literary studies. While the ability to provide a custom similarity function makes Gamma versatile, this also requires us to make a high number of design decisions that influence the results and decrease comparability with other applications of Gamma. Conceptually, the definition of what we want to consider a positional and/or categorical agreement is not always straightforward because of the (sometimes) vague nature of the target phenomenon, the compositionality of the annotation labels, and dependencies between its components.

As our preliminary analyses have shown, a systematic annotation of knowledge transfers about family relations allows for investigations that go beyond structural features of the play’s surface. Herein, we made use of our annotation data to propose an extension to the widely utilised co-presence networks. In specifying the edges as a directed knowledge transmission, networks can be interpreted in light of more tailored research questions as we have hinted at with Günderrode’s Udohla. The analyses have also revealed clear perspectives for larger corpus studies. This gives rise to future questions concerning literary history. Do patterns of family related knowledge distribution emerge for different dramatic genres? Is it possible to characterise the scenes where changes of knowledge occur in more detail? How many characters are on stage in these scenes? How many of them are actively involved in passing on knowledge? What kind of characters do pass on the knowledge?

Our future work mostly focuses on two aspects. Firstly, we are currently implementing a system to automatically infer all deducible family relations from our annotations. As the annotations only cover the transmission of new information from one character to another (or to the audience), this inference system is needed to have a full account of what all characters and the audience know at all times during the drama. Having this knowledge base would both benefit the measuring of the IAA – as it would solve certain problems such as using different predicates for the same relation – and the subsequent analysis. Secondly, we are working on automating certain aspects of the annotation process by creating transformer-based machine learning models which learn to predict the positions in a text where knowledge is transferred and the type of family or love relation that is transferred. Applying these models on new data will facilitate the annotation of new texts. Evaluating the performance of the models on existing data can give additional insights into the complexity of the annotation task.

7. Data availability

Data can be found here: https://github.com/quadrama/jcls2022

8. Software availability

Software can be found here: https://github.com/quadrama/jcls2022

9. Acknowledgements

The research in this article has been conducted in the Q:TRACK project (https://quadrama.github.io/index.en), which is part of the priority programme SPP 2207 Computational Literary Studies and funded by the German Research Foundation (DFG). We thank Jonas Hirner and Christian Lantzinger for their annotations and Erik Ketzan, Trinity College Dublin, for proofreading.

10. Author contributions

Melanie Andresen: Annotation Supervision, Guideline Development, Inter-Annotator Agreement, Writing

Benjamin Krautter: Theoretical Framework, Analysis and Interpretation, Writing

Janis Pagel: Corpus Statistics, Network Analysis, Writing

Nils Reiter: Methodology, Inter-Annotator Agreement, Writing