30^th November 2019

We are happy to announce that the first special issue of our shared task has now been published. The individual articles are:

• Introduction (Evelyn Gius Nils Reiter Marcus Willand)
  • Foreword to the Special Issue “A Shared Task for the Digital Humanities: Annotating Narrative Levels”
  • Chapter 1: Introduction to Annotation, Narrative Levels and Shared Tasks
  • Chapter 2: Evaluating Annotation Guidelines
  • Chapter 3: Description of Submitted Guidelines and Final Evaluation Results
• Guidelines
  • Guideline I
    • Narrative Boundaries Annotation Guide (Joshua Eisenberg, Mark Finlayson)
    • Review of (Meredith A. Martin)
  • Guideline II
    • For Annotating Anachronies and Narrative Levels in Fiction (Edward Kearns)
    • Review (Tilmann Köppe)
  • Guideline III
  • Guideline IV
    • Annotating Narrative Levels in Literature (Nora Ketschik, Benjamin Krautter, Sandra Murr, Yvonne Zimmermann)
    • Review (J. Berenike Herrmann)
  • Guideline V
    • Annotation Guidelines for Narrative Levels and Narrative Acts (Florian Barth)
    • Review (Jan Horstmann)
  • Guideline VI
    • SANTA 6 Collaborative Annotation as a Teaching Tool Between Theory and Practice (Matthias Bauer, Miriam Lahrsow)
    • Review (Natalie M. Houston)
  • Guideline VII
    • Guidelines for annotation of narrative structure (Mats Wirén, Adam Ek, Anna Kasaty)
    • Review (Gunther Martens)
  • Guideline VIII
    • Annotation Guidelines for Narrative Levels (Adam Hammond)
    • Review (Tom McEnaney)

24^th September 2018

This after-workshop post summarizes some interesting findings during our workshop. The post focusses on subjective impressions of the workshop, it will take some more time to deliver the actual results.

31^st August 2018

This post outlines our plans for the upcoming workshop. It includes a first look at (not yet finalized) agenda and describes what material we generate beforehand, and how.

16^th June 2018

A first look at the submitted guidelines.

6^th June 2018

This post provides information about the submission of the guidelines.

23^rd May 2018

This article details how we plan to evaluate/compare annotation guidelines during the workshop. The evaluation will be structured along of three dimensions: i) Relation of guidelines and theory, ii) applicability, and iii) usefulness. To guide the workshop discussion(s) and to explicate the dimensions more clearly, we created a brief questionnaire.

23^rd May 2018

Updated: Workshop dates, evaluation procedure. We would like to invite you to participate in the 1st Shared Task on Narrative Level Annotation. It is an adaptation of the shared task-format established in the field of Computational Linguistics and Informatics to the field of Literary Studies. The goal of the first phase of this (two-phased) shared task is the collaborative creation of annotation guidelines, which in turn will serve as a basis for the second phase, an automatisation-oriented shared task.

20^th January 2018

The corpus has been compiled to cover as much relevant phenomena as possible. It is heterogeneous with respect to genre, publication date and text length. Still, representativity (whatever that means for literature) was not a guiding principle. All texts are available in English and German. Some texts are translations from a third language.

The maximal length of the texts in this corpus is 2000 words. Since this limitation entails a bias with respect to the use of narrative levels, we also have included longer texts, which we make available in a shortened version. For the latter we removed passages that do not affect the overall narrative level structure in a substantial manner.

The corpus is freely available on github.

An overview of the authors and texts can be found in this CSV file.

6^th October 2017

We would like to invite you to participate in the 1st Shared Task on Narrative Level Annotation. It is an adaptation of the shared task-format established in the field of Computational Linguistics and Informatics to the field of Literary Studies. The goal of the first phase of this (two-phased) shared task is the collaborative creation of annotation guidelines, which in turn will serve as a basis for the second phase, an automatisation-oriented shared task.

1^st October 2017

This article describes where to start and how to proceed when developing annotation guidelines. It focuses on the scenario when you’re creating new guidelines for a phenomenon that has been described mostly theoretically.

17^th July 2017

This post details how we envision the first phase of the shared task on systematic analysis of narrative texts through annotation (SANTA) concretely. We hope this gives potential participants a clearer picture of what is expected of them and how long it would take.

We have now set up a mailing list. The list serves both as a simple registration tool as well as for communication with the organizers and discussions among the participants. The list is hosted at Stuttgart University.

Please visit listserv.uni-stuttgart.de/mailman/listinfo/sharedtasksinthedh to register.

The lack of systematic annotation for literary concepts is major roadblock for large-scale studies of literature. We propose to conduct a series of workshops inspired by shared tasks in NLP to foster the development of annotation guidelines for certain literary concepts. This in turn enables systematic annotation and development of automatisation options.

This text has been accepted for the next Digital Humanities Conference, to be presented as a short talk in Montreal 2017. We have put it online here as a reference point for discussion and information.