Visualization as Defamiliarization. Mixed Methods Approaches to Historical Book Reviews

1. Background

In the study “The Order of Criticism: Swedish Book Reviews in 1906, 1956, 2006” (Kritikens ordning: Svenska bokrecensioner 1906, 1956, 2006) from 2013, literary scholar Lina Samuelsson analyzed what characterized literary criticism as an institution and practice, mapping dominant themes, values, and discourses, at different points in time. Combining a sociological and historical perspective with Foucauldian discourse analysis, the study traces what has historically constituted a literary book review and what norms literary reviewers have followed at different points in time.¹

The current research project “The New Order of Criticism: A Mixed Methods Study of 150 Years of Book Reviews in Sweden”, repeats, extends, and challenges the original study (Samuelsson being a member of the project team), drawing upon data-driven approaches to explore how ‘traditional’ and ‘digital’ methods can contribute to enhancing each other, both in practical and epistemological terms.² Thus, the project ties into the ongoing critical discussion in digital humanities about the need for integrative interdisciplinary approaches and to reflect on the positivist claims made within the field (Jockers 2013; Moretti 2013). As digital historian Jo Guldi argues, without the insights of the humanities, data-driven approaches risk producing analyses that are empty or misleading. According to Guldi, data-intensive analysis lacks a historical sensibility and an awareness of the data’s original context often raises more questions than it answers (Guldi 2023, 1,27,83). Turning the argument around on proponents of the presumed scientificity of distant reading and macro analysis, digital literary historian Katherine Bode suggests that an exclusive focus on textual signals could be understood merely as an enactment of a de-contextualized understanding of text as data, emphasizing that aggregating text data involves a stripping of context (Berry and Fagerjord 2017; Bode 2018; Dobson 2019). Consequently, Bode argues for the importance of an interpretative and contextual understanding of both the data and the results.³

In this paper, we revisit the review material that the original study, “The Order of Criticism”, was based on from a mixed methods perspective to discuss the possibility of an analytical interplay between data visualization and close reading. Rather than engaging in the debate concerning the prerequisites of data as evidence or the need for criticality when creating data visualizations, we explore the possibility of discovering alternative ways of looking at a particular material through a dialectical mixed methods approach. Thus, in this particular context, we are less interested in evaluating the original study or interrogating the creation of the visualizations (nor the methodology of the original discourse analysis) than in exploring how data-driven and interpretative methods can provide complementary analytical perspectives on a text collection, focusing on significant data patterns that emerge in visualizations and comparing them to the original analysis. Essentially, our discussion will emphasize performative and interpretative affordances of the visualizations rather than computational aspects (Bode 2020).

In total, the original study, “The Order of Criticism”, was based on 700 book reviews, which can be considered a rather substantial material for a ‘traditional’ literary history study, even though it can be considered a small dataset in a digital humanities context.⁴ However, in digital humanities, data-driven analyses of literary criticism and reception have been performed on less extensive but more curated datasets and, notably, the collection used for “The Order of Criticism” exceeds for instance the two corpora of English and German historical book reviews (605 and 547, respectively) from the long 18^th and 19^th centuries created by Brottrager et al. (2022) for automated sentiment detection.

To delineate our approach, we begin by situating our study within the field of mixed methods and highlighting our dialectical approach, emphasizing that while so-called quantitative and qualitative methods tend to generate different results, they can nevertheless be intermingled, making the answer to a research question more complex and flexible. We then describe the process of generating text data visualizations based on the book reviews originally investigated in “The Order of Criticism”, using TF-IDF (Term Frequency – Inverse Document Frequency) and an interface developed within our current project.⁵ Turning to the analysis, we examine data visualizations of word frequencies, publication years, and genre categorizations respectively in the review material from the original study, focusing on results that raise questions in relation to the prior results concerning the literary discourse in 1906, 1956, and 2006. The analysis leads up to a concluding discussion about the criticality of a contextual sensibility for understanding how we can analyze text data visualizations, but also the possibility of attributing an estranging quality to them. Drawing upon Russian formalist Viktor Shklovksy, we suggest the concept of “defamiliarization” (“priëm ostraneniya”) as a conceptual framework for understanding the process of being able to look anew at a seemingly familiar research material (“the already analyzed”) through the lens of visualizations, potentially turning the analytical gaze toward overlooked aspects (Shklovsky 1929 1990).

2. Mixed Methods – Pragmatic and Dialectical Approaches

In digital humanities, there is a growing interest in critical reflection on “what is happening” or “what should happen” at the concrete intersections between data-driven and interpretative methods (Ahnert et al. 2023). Concerning data-intensive studies of newspaper data and literary criticism, the discussion has primarily revolved around the future potential of computational methods and productive approaches, rather than the very nature of interdisciplinary syntheses (Piper 2020; Underwood 2018). Only in recent years, there has appeared a clearly articulated theoretical interest within digital humanities in developing a more organic interdisciplinarity with integrated workflows and there remains a lack of systematic reflection on the relationship between different interdisciplinary and methodological syntheses (Oberbichler et al. 2021).

However, such modes of reflection can be found within the field of mixed methods that centers on the creation and reflection of syntheses between quantitative and qualitative approaches (J. W. Creswell and J. D. Creswell 2022; Johnson et al. 2007). Much of the research practices associated with mixed methods are, of course, not necessarily ‘new’, but the field has nevertheless come to serve as a distinct space for self-reflexive discussion. According to philosopher Yafeng Shan, the heterogeneous field of mixed methods can be discussed at various levels in scientific practice, including material selection, method selection, research purpose, and epistemology (a method’s epistemological implications) (Shan 2023). Shan further identifies a number of fundamental approaches to mixed methods, including a “pragmatic” and a “dialectical” approach, which can be used to frame our study (Shan 2023, 3–4).

From a pragmatic standpoint, researchers (individually or in groups) are free to use the method – quantitative or qualitative – that they believe best suits their task, without considering one method a priori better than the other. Shan (2023, 6–8) sees this as a “weaker” category insofar as the pragmatic position is open to the possibility of integrating quantitative and qualitative methods without necessitating their combination. Somewhat akin to the pragmatic stance is the dialectical one. Here, the different epistemological approaches underlying quantitative and qualitative methods are also accepted, but it is emphasized that they lead to different results. Thus, it is not just about choosing the method that “works best”, but also about accepting that different methods complement each other due to their distinct epistemological consequences. Adopting different perspectives makes the answer to a research question more complex and flexible. Therefore, Shan (2023, 8) understands the dialectical approach as a “strong” category of mixed methods because it starts from the premise that research questions cannot be answered by only one quantitative or qualitative method, but are better understood by combining them.

3. Data Visualizations

Emphasizing the rhetorical power of data visualizations, Johanna Drucker asserts that they always involve calculations that are graphically represented to communicate specific aspects of the underlying data (Drucker 2021, 86). In our case, data visualizations create a multi-dimensional ‘map’ of various relationships between book reviews based on their linguistic characteristics at both the word and sentence levels. By studying these visualizations, we can explore the potential of a quantifying method to elucidate significant patterns in the texts in comparison with a prior study based on the same material. Consequently, we are primarily interested in patterns in the visualizations that go against our expectations based on previous results. In this, we are inspired by Andrew Piper and Mark Algee–Hewitt’s work on the creation of topological models for visualizing the lexical relationality between Goethe’s The Sorrows of Young Werther and the author’s œvre, bringing into view textual relationships through the form of the diagram (Piper and Algee-Hewitt 2014). Reading “words in space”, rather than within sentences, as Piper and Algee–Hewitt put it, allows them to bring to light “the latency of the lexically manifest” or the potential “meaning of the distributed recurrences of language that can easily escape our critical consciousness”, provoking new close readings of Goethe’s texts (Piper and Algee-Hewitt 2014, 157 and passim).

In “The Order of Criticism”, 700 literary book reviews from newspapers and periodicals were examined to provide a systematic and fairly representative sample of literary criticism for the years 1906, 1956, and 2006. Each year was studied through two delimited samples that provided the study with roughly the same number of reviews from each year (198, 272, and 230 reviews from 1906, 1956, and 2006, respectively). In 1906, the samples were based on one month in spring and one month in autumn, and in 1956 and 2006, on one week each in spring and autumn. While one of the aims of our current research project is to determine whether this sampling of book reviews is in fact representative (using text mining of reviews in the newspaper collection of the National Library of Sweden (Kungliga Biblioteket, KB)), in the present paper we will stick with the original selection for comparative purposes.⁶

Methodologically, the study took inspiration from the so-called year study method, meaning that the reviews were analyzed from a synchronic rather than a diachronic perspective, without aligning them into a continuous historical account or ‘narrative’, primarily comparing what could be analytically distinguished through peepholes into the past (Samuelsson 2013, 18; North 2001; Gumbrecht 1997). Notably, as part of the work process, the reviews were transcribed by hand, primarily from newspapers on microfilm, creating a collection, and compiled as a rudimentary database in the form of a spreadsheet containing metadata on publication year, reviewed author, reviewed work, work’s publication year and language, as well as reviewer and organ of publication. Information about the gender of authors and reviewers was also included when available (in some cases, the name of an author or a reviewer is lacking because they wrote anonymously or used an unfamiliar pseudonym or signature).⁷

In generating data visualizations based on the original text material, we opted for quantifying the differences between the transcribed reviews, expressed as a form of distance, leading to the placement of texts closer or farther apart. More specifically, the text in each review was lemmatized (i.e., different inflectional forms of a word were combined) and transformed using TF-IDF, a method that emphasizes words that are unique to a specific text and downplays words that are common to all texts (e.g., ‘the’, ‘it’, ‘that’, ‘be’) (Spärck Jones 1972), while at the sentence level we use the Sentence Transformer model trained by the National Library of Sweden (Rekathati 2021), in an approach similar to e.g. Van Cranenburgh et al. (2019). In these representations, some texts appear more similar than others – for simplicity, we refer to them as neighbors (“grannar”) – based on vocabulary or sentence structure. The similarity between the texts was then visualized as distances in the form of a “map”⁸, where reviews appear as a cloud of dots, each dot corresponding to a review whose metadata (publication year, reviewed author, etc.) is displayed when the user activates the dot with a click in the interface, the size of the dots in the visualization being determined by the length of the review texts (Figure 1). The positioning or embedding of the reviews is calculated at the word level from the TF-IDF representation and at the sentence level using the Sentence Transformer representation with UMAP (Uniform Manifold Approximation and Projection) as an approximation of the aforementioned distance between the review texts (akin to e.g. multidimensional scaling, MDS), being based solely on linguistic factors and independent of the metadata in the spreadsheet (McInnes et al. 2020; Borg and Groenen 2005).

Figure 1: The “map”, showing 700 book reviews, here presented by year (“årtal”) and word level (“ordnivå”).

In these visualizations, the embedding is projected onto a two-dimensional plane, which means that the distance between reviews is not reproduced exactly. Rather, this relationship is multidimensional and complex (comparable to a map of the Earth, a body that, due to its spherical shape, cannot be accurately represented on a flat map) or, as Drucker would put it, “any point or mark used as a specific node in a humanistic graph is assumed to have many dimensions to it – each of which complicates its identity by suggesting the embeddedness of its existence in a system of co-dependent relations” (Drucker 2011, §20). The true embedding distance is displayed in the “neighbors” column (“Grannar” in Figure 2), which may be used to confirm which reviews are actually close to each other locally. While it is indeed possible to globally quantify inter- and intra-group dispersion as in Van Cranenburgh et al. (2019), we judge that a local neighborhood of reviews remains more interpretable for a reader. In our interface, the visualizations display how the reviews position themselves in relation to each other based on factors such as year of publication, genre categorization, critic, publishing organ, and author of reviewed work (Figure 3). Unlike other explorative methods, such as topic modeling, this study is mainly interested in the characterization of reviews per the existing metadata.

Figure 2: Some of the neighbors (“grannar”) to the review by the signature “H. J.” of Vilhelm Ekelund’s poetry collection Hafvets stjärna (“The Star of the Sea”).

Figure 3: The interface for choosing parameters in the visualization, in this example based on media type (“medietyp” – newspaper or journal), and sentence level (“meningsnivå”).

On a more abstract level, our approach to visualization ties into the discussion of “performative materiality” to counteract an overestimation of the truth value of data representations. Since data involves simplifications of the phenomena they describe, Katherine Bode stresses that in data-rich literary research, we should consider the fact that the qualities of computational analysis are performative rather than representative. Bode describes this performative dimension in data representations as “sites – or apparatuses – for engaging with literary texts as emergent events, always arising from and altering how the literary past is (re)configured” (Bode 2020). A way to affirm this performative dimension on a technical level is, as advocated by Bode, to incorporate a self-reflective function into an interface. However, our approach to the visualizations rather raises another performative issue: a certain defamiliarizing quality.

In a discussion of Roberto Busa’s pioneering work in computer-driven text processing through the Index Thomisticus that began in 1946, Stephen Ramsay writes that the indexing of words in Thomas Aquinas’s collected works in the form of punch cards gave rise to a particular effect, “not the immediate apprehension of knowledge, but instead what the Russian Formalists called – the estrangement and defamiliarization of textuality. One might suppose that being able to see texts in such strange and unfamiliar ways would give such procedures an important place in the critical revolution the Russian Formalists ignited” (Ramsay 2011, 3). The concept of defamiliarization has been associated with various meanings in literary theory, but one can say that the concept is generally associated with aesthetic effects that create a distance between a work and its observer to provoke reflection. Notably, defamiliarization has traditionally been linked to modernist thought, which is characterized by the idea that consciously complex formal language somehow paves the way for a deeper understanding of reality. While our study obviously does not concern art in this sense or the imperative to stimulate a deeper reflection on the world, it is nevertheless crucial that data visualizations may not only provide an abstracted and modeled overview of a certain material, but also create a distance between us, as observers, and the material, thereby making it possible to speak of a defamiliarizing quality.⁹

4. Comparative Re-reading

Turning to our analysis, we have chosen to focus on three factors – word and sentence levels, year of publication, and genre categorization – to show how data visualizations can inspire re-readings and provide complementary perspectives on familiar material.

4.1 Word and Sentence Levels

In “The Order of Criticisism”, Samuelsson (2013, 155) writes: “As a genre, reviews have not undergone major changes over the past hundred years. In 1906, as well as in 1956 and 2006, descriptions, interpretations, and evaluations of one or more works constitute the core of criticism. Different functions may be more or less dominant, criteria and rhetoric may vary, but the genre of the review remains stable”.¹⁰ Other literary scholars of Swedish book reviews have made similar observations. For instance, Tomas Forser calls reviews “a genre of great durability” (Forser 2002, 155), and Per Rydén describes it as “a traditional, almost static genre” (Rydén 1987, 33). However, although the genre as a whole exhibits striking similarities over time, it is clear that the content has changed over the course of a century to the extent that a data-driven analysis distinguishes a clear difference between reviews from different time periods.

If we return to Figure 1, we can see that reviews tend to group together based on differences and similarities at the word level, predominantly according to the year of publication. Furthermore, there is a clear distance between them. The differences between 1906 (blue) and 2006 (green) are more significant than those between 1956 (orange) and 1906 or 2006, indicating some form of chronological change.¹¹ In short, the visualization shows that reviews from, for example, 1906 in terms of word choice are as similar to each other as they are different from texts from 1956 and 2006. For the middle year 1956, reviews are slightly more dispersed in the visualization, with some ending up with reviews from 2006 and others from 1906. A few reviews from 2006 are placed among the reviews from 1906: Jim Kelly’s detective novel Måntunneln ((Moon Tunnel)) and the children’s books Skämmarkriget (The Shaming War) by Lene Kaaberbøl, Min syster flygande Flavia (My Sister the Flying Flavia) by Helena Öberg, and När Johan vaknar upp en morgon är han stark (When Johan Wakes Up One Morning He is Strong) by Petter Lidbeck and Lisen Adbåge, which we will return to below.

Notably, one should pay attention to which words determine a text’s placement in a particular year cluster. While it is not possible to draw any conclusions about this solely based on the most represented words in an individual text (since positioning is determined by a complex system of relative occurrences among the reviews), it is relevant to take into account which words are over- or underrepresented for each individual year in groupings. Over- and underrepresentation are calculated here using Dunning’s log-likelihood method, a familiar algorithm in corpus and discourse analysis, which quantifies how unexpected a word is in a text given the words in all other texts within a certain group, such as years (Dunning 1993). One possible explanation for reviews grouping so clearly by year may, of course, be language changes over time. For instance, words that are particularly characteristic of specific years, according to data analysis, include “skald” (“poet”) and “författarinna” (“female author”), as well as the word form “äro” (“are”) for 1906. However, such words seem outdated in 2006 when terms like “fiktiv” (“fictional”), “identitet” (“identity”), and “relation” (“relationship”) are prominent.¹²

One way to get closer to the factors that determine the placement of reviews in the visualization is to compare the words that vary most in frequency between the years, i.e., those that are over- or underrepresented for a specific year.¹³ Other words that are particularly characteristic of appearing in a 1906 review include “han” (“he”), “hon” (“she”), “djup” (“depth”), “akt” (“act”), “förf” (“auth”, abbreviation for “author”), and “öfrig” (“other”). The latter (“öfrig”) can be related to the spelling reform, while “akt” is probably connected to more plays being reviewed in 1906 than in the other years. The use of “förf” (“auth”) likely results from it being a common abbreviation for “författare” (“author”) at that time. Furthermore, the more frequent use of “hon” (“she”) and “han” (“he”) in 1906 than in later years could be explained by how reviews at the time dedicated significant space to content summaries, often focused on describing and explaining characters and their actions.

Equivalent typical words for reviews from 1956, for example, are “roman” (“novel”), “social” (“social”), “urval” (“selection”), “miljö” (“setting”), “analys” (“analysis”), “avsnitt” (“section”), “fin” (“fine”), “politisk” (“political”), “höst” (“autumn”), “spela” (“play”), “uppleva” (“experience”), “människa” (“human”), “diktare” (“poet”), and “beroende” (“dependence”). The presence of some of these words can probably be explained by the topics and themes of the literary works that were most frequently reviewed, as well as by the fact that the term “diktare” replaced “skald” (“skald”, a Medieval poet). The interest in formal features and close reading that has been associated with New Criticism during this period can be noted in the use of terms such as “analysis” and “section” (Samuelsson 2013, 76–77). The high-frequency words also testify to a certain societal engagement in the criticism, as evidenced by the presence of words like “political”, “environment”, and “social”. This is also noted in “The Order of Criticism”, where it is related to the reflections of the time, in the aftermath of World War II, on “humanity”, “mankind”, and the human psyche, something that can also be seen in the recurring use of the term “human” (Samuelsson 2013, 84,88).

For 2006, on the other hand, the most distinctive words are “jag” (“I”), “skriva” (“write”), “text” (“text”), “språk” (“language”), “roman” (“novel”), “bli” (“become”), “berättelse” (“story”), “läsa” (“read”), “mamma” (“mom”), “pappa” (“dad”), “barn” (“child”), “far” (“father”), “handla” (“act”), and, as mentioned above, “relation” (“relationship”), “identitet” (“identity”), and “fiktiv” (“fictional”). Here, we observe several words that can be related to the fact that the discussed works – and perhaps in some cases reflections on the critics’ own lives – revolve around relationships and family dynamics (“mom”, “dad”, “child”, “father”, “relationship”). Other words are indicative of how literature is discussed and described (“write”, “language”, “novel”, “story”, “fictional”, “act”). The distinguishing words confirm the prior observations in “The Order of Criticism” about a more present and subjective critical subject, as well as a significant interest in identity issues (Samuelsson 2013, 125–127,134–136,145–148).¹⁴

A visualization at the sentence level (Figure 4) provides a much more heterogeneous result, which can support the above argument that the form of criticism has not changed significantly, while the visualization at the word level in Figure 1 indicates that the content expressed or valued has changed over time.¹⁵ In this way, one can say that the data-driven analysis actually seems to confirm the earlier assumptions of literary critics that literary criticism as a whole is a relatively stable – or, if you will, conservative – genre of text.

Figure 4: Visualization of the material by year (“årtal”) based on the sentence level (“meningsnivå”).

4.2 Publication Year

As a distinct example of the defamiliarizing qualities of the vizualisation, we can compare the reviews that end up far from others within the same group (i.e., outlier dots) to study common distinguishing features. For example, the review of Moon Tunnel by Jim Kelly, reviewed in Sydsvenska Dagbladet in 2006, can be seen on the map surrounded by reviews from 1906. Looking at the neighbors, they are indeed reviews from different years, but a significant number of them are from 1906 (Figure 5). Since this text, unlike most of the others from 2006, has neighbors from 1906, there is a reason to consider why this is the case.

Figure 5: The neighbors to Jan Broberg’s 2006 review of Jim Kelly’s Moon Tunnel, four of them being from 1906.

The review of Moon Tunnel is part of a collective review where Kelly’s work is discussed in pair with Peter Robinson’s En bit av mitt hjärta (Piece of My Heart), but the text is clearly divided in the sense that the first half deals with Robinson’s work and the second with Kelly’s. The visualization is based on the database, which treats these texts as two separate segments (as mentioned above). The review of Robinson’s work, unlike the review of Kelly’s, is located near the cluster of 2006 reviews but is also surrounded by reviews from 1956. It’s worth noting that these reviews, even though they appear in the same article, were separated in the original study for analytical purposes and are thus treated as separate texts in the database. This makes the collective review particularly interesting for our purposes, as the same text gives rise to two different placements in the visualization leading to the question whether they differ significantly.

Starting with the review that landed in the center of the 1906 review cluster, Moon Tunnel by Jim Kelly, the words that the computational analysis has identified as significant, aside from those related to the plot, include words like “obestridd” (“undisputed”), “lättköpta” (“easily bought”), “återigen” (“again”), “elegi” (“elegy”), “udda” (“odd”), “mästerskap” (“mastery”), “lansera” (“launch”), “lovande” (“promising”). In this context, significant means the weighting an individual word has on the placement of the work in the visualization. Words like “promising”, as well as others listed further down like “nå” (“achieve”), “steg” (“step”), and “författare” (“author”), are terms that could be related to the typical characteristics of literary criticism around 1906 and a tendency to assess how well the author has developed artistically, and to determine if an author is worthy of the title as true author.¹⁶ Clear evaluative words like “undisputed” and “mastery” could be linked to this discourse, which becomes evident upon closer examination of the text.

The presence not only of individual words but also how evaluative words function in the review of Moon Tunnel that resemble the order of criticism in 1906 becomes apparent when one considers the review as a text rather than as text data. The review begins with: “Jim Kelly does not reach the now undisputed mastery of Robinson, but his latest detective novel, Moon Tunnel, is still a step forward for this promising English author”.¹⁷ Here, one can observe stylistic features that are described in The Order of Criticism as characteristic of 1906. The critic’s evaluation is evident – Kelly is considered “inferior” to Robinson, who is described as a “master”. Similarly, the development of the author’s work is assessed and the reviewer believes that the novel is “a step forward for this promising English author”. This can be compared to reviews from 1906 where a critic might praise aspects such as “an unusually straightforward developmental trajectory”, while another critic laments a poetry collection that is “all too similar to its older siblings” (Broberg 2006, 33).¹⁸

Looking at the reviews of The Shaming War and My Sister the Flying Flavia, which also have neighbors from a century ago, both stand out for consisting of plot summaries, concluding with a clear assessment from the critic. “With My Sister the Flying Flavia, copywriter Helena Öberg has created a sympathetic and easily readable story for those between seven and nine”, writes Sydsvenska Dagbladet, and the critic from Upsala Nya Tidning concludes the review of Lene Kaaberbøl’s The Shaming War with the judgment that: “The Shaming series is not a complicated fantasy work, rather a fairly simply told saga, with not too large a cast of characters or an advanced structure. But due to some truly scary scenes, it is still not suitable reading for very young fantasy fans”.¹⁹ Helena Öberg’s When Johan Wakes Up One Morning he is Strong is also reviewed in Upsala Nya Tidning, alongside another illustrated chapter book. This text is also relatively short and primarily focused on the plot.

The reason why these children’s book reviews are close to the 1906 cluster likely lies in the significant use of words describing the content of the literary works, which is also typical of early 20^th century criticism, along with words declaring a clear concluding judgment.²⁰ Furthermore, the critics do not refer to themselves in the above-mentioned reviews of Öberg’s, Kaaberbøl’s, and Kelly’s books: There are no “I”, “my”, “mine”, or other references to the critic as a person. This distinguishes these reviews from the descriptions of literary criticism in 2006 encountered in “The Order of Criticism”, which highlights the presence of the critical subject, while the absence of reference to the writing subject is typical of critics from a hundred years earlier.

But, returning to the crime fiction review discussed above: How do the texts about Robinson’s and Kelly’s detective novels differ from each other? After all, the books are reviewed in the same review but end up in different places in the visualization (Broberg 2006). Why does the text about Robinson’s detective novel end up among reviews from 1956 but much closer to other 2006 reviews than the later part of the text discussing Kelly’s?

Of the words listed as significant for the placement of the Robinson review (among those not related to the plot), we can note terms such as “förtjänst” (“merit”), “höstbok” (“autumn book”), “engelsk” (“English”), “deckararena” (approx. “detective genre”), “roman” (“novel”), “konststycket” (“the feat”), “komplexitet” (“complexity”), “mysterium” (“mystery”), “täthet” (“density”), “eminent” (“eminent”), “levandegöra” (“bring to life”), “förbrylla” (“baffle”), “personteckning” (“characterization”), “invända” (“object”), “nyanserad” (“nuanced”), “parentes” (“parenthesis”), “händelseförlopp” (“sequence of events”), “invändning” (“objection”), “ovänta[d]” (“unexpected”), and “bidra” (“contribute”). One can also note more words related to the critic and their task, such as “recensera” (“review”), “recension” (“critique”), “läsare” (“reader”). Furthermore, several evaluative expressions are present, such as “ny” (“new”), “bra” (“good”), “favorit” (“favorite”), “positiv” (“positive”), which aligns more with the literary critical discourse of 1956 and 2006 than 1906 (Samuelsson 2013, 134–135). Looking at the actual review, it also starts with a clear focus on the critic himself: “That Peter Robinson belongs to my favorites in the detective genre today, has surely become apparent from my reviews over the years”, [authors’ emphasis]. This is followed by a reservation, typical of reviews in 2006, which at the same time emphasizes the qualities of the work: “It could possibly be argued that the author does not play entirely fair with the reader in a certain respect, but it is still an objection that carries little weight considering all the other merits of the novel” (Broberg 2006). The critic talks about the novel as dense and complex, the characterization nuanced, and the setting vivid.

Primarily, the Robinson review focuses on evaluation, and it is a positive one. Despite recurring phrases related to the plot of the novel, there is not a direct description of the plot, but rather, they serve as summaries: It is in the vividly depicted English landscape where “the events unfold”, and it is the “portrayal of the youth culture that plays a significant role in the plot” that makes the novel complex. We do not learn much more about what is being depicted. This brevity in plot summaries is more characteristic of 1956 and 2006 reviews than of the 1906 reviews, where we have seen that the course of events can be described in some detail. However, the Robinson review ends in the spirit of the 1906 critics with an assessment of the author’s progress: “Yes, Robinson has certainly developed since entering the detective genre”.

Thus, there are clear differences in language use at the word level between reviews from 1906, 1956, and 2006, but somewhat less at the sentence level, which in this case could be interpreted as the rhetoric and typical genre features of the criticism. Some discursive features noted to apply to the different years are supported by the data-driven analysis, but there is also room to discover other patterns, such as how different literary categories are reviewed. This will be the focus of the next observation about the defamiliarizing quality of our visualizations.

4.3 Genre Categorization

During the writing process of “The Order of Criticism”, the data were compiled regarding the genres in which reviewed works were categorized according to the National Library of Sweden’s catalog Libris: prose, poetry, drama, children’s literature, and “other” (which includes among other audiobooks and comic books). However, literary genres are far more complex and ambiguous than what these categories reflect. Institutionalized classifications are just one part of the networks of cultural meaning-making and historical processes that contribute to our understanding of which genre a particular book can be understood in relation to. Genres consist of a constantly changing, multifaceted, and contradictory palette of aesthetic traditions and labels, where libraries are one actor, and the audience, the book industry, reviewers, and researchers are others. Nevertheless, the Libris catalog can be used to create a rudimentary perspective on the relationships between different literary works and their reception, since computerized analysis can easily track differences and similarities at the text level based on attributed genres.

To avoid delving into a complex genre theoretical discussion, for the sake of simplicity, we choose to refer to these variables as ‘genre categorizations’. Even though the Libris catalog might be considered an authority in this context, there are plenty of indications that library classifications can be discussed. For example, “children’s literature”, rather than being a more distinct genre, should be seen as a collective term for literature written by adults for a child audience, which can encompass both prose and poetry as well as plays for children. Nevertheless, in critical practice, there is a tendency for different reviewers to be assigned works from different genres: One critic reviews prose, another reviews drama, a third reviews poetry, and someone else writes about children’s literature.²¹

In Figure 6, where the visualization is color-coded at the word level based on assigned genres in Libris, we can see that the reviews, as in the case of publication years, are clearly grouped by category. The same is true at the sentence level, as shown in Figure 7.²² At the word level, almost all poetry (orange) is concentrated on the left. Likewise, drama (green) forms a distinct cluster. Similarly, prose (blue), which constitutes the largest category, is cohesive. The most dispersed category is children’s literature (red), both at the word and sentence levels, which can likely be explained by the fact that children’s literature, as mentioned earlier, encompasses a range of forms of expression. It may also be due to significant variations within children’s literature criticism. An indication of this is that the ‘other’ category, which includes among other comic books and essays, can also be described as heterogeneous and scattered in the visualization.

Figure 6: Visualization on the word level, based on the reviewed work’s genre. “Prosa” = Prose; “Lyrik” = Poetry; “Drama” = Play; “Barn” = Children’s literature; “Annat” = Other.

Figure 7: Visualization on the sentence level based on the reviewed work’s genre.

As in the case of publication years, it is reasonable to make some observations about noteworthy placements. In Figure 6, we can note that a limited number of poetry reviews ended up among prose reviews, while there are no prose works in the poetry section on the left. In this sense, one can speak of a significant consistency within poetry criticism. Some of the prose reviews that are placed near the poetry reviews (and have several poetry neighbors) are reviews of Vendela Fredricsson’s Landar (Landing) from 2006. In this context, it is relevant to mention that Landing is a prose-lyric short novel that made Expressen’s critic wonder “if the alleged debut novelist […] actually wants to write semi-surrealistic poetry”.²³ The colleague in Helsingborgs Dagblad noted that “[a]t times, Landing feels more like poetry than a novel” (Lingebrandt 2006). Landing was also reviewed by Göteborgs-Posten, but its critic, unlike the others, did not focus on the work’s lyrical aspect, but rather discussed its plot (a love triangle) in some detail. This review is also placed far from the other reviews of the same book.

The ‘drama cluster’ in Figure 6 includes a limited number of works that were reviewed in several newspapers, mainly in 1906. However, we find some drama reviews placed further away together with prose, including Cecilia Nelson’s Öknen (The Desert), reviewed in Norrländska Socialdemokraten in 2006, as well as a collective review in the magazine Perspektiv in 1956 of four comedy plays. It should be mentioned in this context that only a few plays were reviewed during the examined periods of 1956 and 2006. The fact that these are placed far from the others indicates possible historical changes and differences in both the drama category and the criticism of drama. In the review of The Desert, there is actually no discussion about the genre itself – that is, the play – except that it mentions that it is Nelson’s “debut play”. Among the words that have influenced the review’s placement in the visualization are those related to the work’s plot, including “kamel” (“camel”) and “möte” (“meeting”), and adjectives like “politisk” (“political”) and “verklig” (“real”).

Another indication that the reviewed works have more influence on the groupings than the reviewer or the category is that the reviews from 1956 of Erland Josephson’s drama Sällskapslek (Party Games), Jean Anouilh’s Ornifle eller Luftgästen (Ornifle: A Play), Hans Hergin’s O, sköna Tasmanien (O, Beautiful Tasmania), and Bo Widerberg’s Skiljas (Divorce) are included in the same collective review, but are not placed next to each other. Although works in the same category often become neighbors in the visualization, this is not surprising in itself. The content of a work is reflected in the text that deals with it, often through quotes and plot summaries. However, it is still worth noting that even though the visualization does not take metadata into account, it creates a striking pattern.

For instance, there are reviews from 1906: Anders Österling’s play Nattens röster (Voices of the Night). Reading the reviews, it becomes clear that they are remarkably similar to each other. This is evident not least through the words that are most significant for the placement of the reviews in the visualization. Several of the recurring words are related to the play’s form and content, such as “akt” (“act”), “musik” (“music”), and “mor” (“mother”).²⁴ Other recurring words are related to the genre itself, such as “dramatisk” (“dramatic”), “drama” (“drama”), “vers” (“verse’), and “lyrisk” (“lyrical”).

When it comes to the prose category, reviews of the same book also group together. In Figure 8, we have sorted out the works that were reviewed at least five times in 1906 and marked them in different colors. Here we see that although some reviews of the same work are so close that they overlap, while others have a wider spread, reviews of the same title are usually neighbors. Essentially the same is true for 1956 and 2006. In short, reviews tend to group with their peers in terms of both categories, publication years, and titles.

Figure 8: Visualization on the word level of the reviews where the same title has been reviewed more than five times in 1906. Different colors mean different literary works.

5. Conclusion – Contextualization and Defamiliarization

Initially, we described our use of a mixed methods approach to the study of literary criticism in terms of what Shan (2023, 8) refers to as a “dialectical position”, which means that the investigation does not prioritize a quantitative method over a qualitative method, and vice versa. Rather, we recognize that different approaches generate different results, which, taken together, can nevertheless enrich the understanding of what has characterized the norms of literary criticism at different points in time, as analyzed in a previous study. According to Shan, mixed methods can be applied at different levels in scientific practice, including method selection and epistemology, which has a bearing on our analysis of data patterns emerging in visualizations of a corpus of book reviews previously examined in a study in comparative literature. Methodologically, we have combined a quantification of differences and similarities between book review texts with close re-reading, taking the historical context of the texts into account. Epistemologically, following Piper and Algee-Hewitt (2014), we have explored how dialectically combining traditional and digital analysis may contribute to new knowledge about a particular research material.

Therefore, there is a point in discussing the results on both a concrete and abstract level. Concretely, our visualizations of overrepresented and underrepresented words in literary criticism from different periods confirm assumptions made in the original study, for example, that reviews in 1906 devoted more space to plot summaries and evaluation of authorship, while reviews in 1956 reflected a different societal engagement, and those in 2006 tended to emphasize the ‘I’ of the critic. However, by visualizing linguistic characteristics in relation to publication year, we found not only that reviews grouped themselves into clusters roughly in line with our expectations, but also that reviews sharing strong thematic similarities challenged chronological expectations and grouped together regardless of significant historical distances. An example is a review from 2006 of a detective novel that contained a rhetoric very similar to how reviews in 1906 tended to evaluate authors based on their perceived artistic development toward ‘mastery’. Our visualizations of genre categorizations also called for closer examination. The fact that a review of a prose-lyrical short novel ended up near the cluster of poetry reviews, rather than prose reviews, was likely due to how the reviewers tended to emphasize the book’s fusion of prose and poetry. At the same time, a single review of the novel in question that did not touch upon this aspect ended up far from the others. Thus, here the visualization directed our attention to the extent to which reviews foreground genre characteristics, a critical aspect not discussed in “The Order of Criticism”. Notably, these results point to the importance of a contextual approach when analyzing our text data visualizations. Without knowledge of the historical contexts of literary criticism, it would be hard to make such observations about the clustering and breaks in the expected pattern.

Furthermore, our analysis highlights the usefulness of the concept of defamiliarization in our analytical context. Here, we can specifically turn to Victor Shklovsky’s conceptualization of how defamiliarization slows down or de-automates perception, allowing familiar assumptions to be renegotiated. Analyzing Shklovsky’s notion of defamiliarization and the perceptual processes that a work sets in motion, literary scholar Beata Agrell makes an important distinction (Agrell 1997b, 26–58; Agrell 1997a, 87–89). Agrell argues that, according to Shklovsky’s theory, the work in question “is thus not autonomous but directed towards a certain type of observation, which it simultaneously invokes through its built-in devices” (Agrell 1997b, 28).²⁵ Hence, in a transferred sense, one may say that our data visualizations de-automatize the perception of the text material and also defamiliarize the original conclusions in “The Order of Criticism”. The fact that our results confirm many of the conclusions of the prior study can be viewed as a strength in this context in that it indicates that the visualizations can indeed capture significant patterns in the material. Perceiving something in a radically different way does not necessarily mean seeing radically different things. Rather, a key point in thinking about visualizations in terms of Shklovsky’s concept of defamiliarization is that they offer a ‘double vision’ or a shift between different positions from which to study the texts. Arguably, one may talk about the potential to evoke shifts in perspective and to direct analytical attention to overlooked aspects of a specific material. Thus, rather than ultimately leading to a ‘better’ path to truth, visualizations could potentially generate new research questions about familiar materials. Which seems significant enough.

Thus, we conclude that digital methods should not be regarded as quick fixes to make things easier. Rather, they should be hailed as what they are: tools for making our endeavors actually more complicated. This is a very proud tradition within the Humanities.

6. Data Availability

Data not subject to copyright restrictions can be found here: https://zenodo.org/records/13742526.

7. Software Availability

Software can be found here: https://dh.gu.se/kno/.

8. Acknowledgements

This paper has been written with the financial support of Riksbankens Jubileumsfond, funding the project The New Order of Criticism: 150 years of Book Reviews in Sweden (2020–2024). Special thanks to Aram Karimi, GRIDH (Gothenburg Reserach Infrastructure in Digital Humanities) at the University of Gothenburg, for helping out with the LaTeX implementation.

9. Author Contributions

Daniel Brodén: Supervision, Writing - original draft, Writing - review & editing

Jonas Ingvarsson: Supervision, Writing - original draft, Writing - review & editing

Lina Samuelsson: Conceptualization, Methodology, Writing - original draft, Writing - review & editing

Victor Wåhlstrand Skärström: Visualization, Formal Analysis, Writing - original draft, Writing - review & editing