What is TEI all about?
Text Encoding and markup for the Humanities

Objectives

(c) Edward Vanhoutte 1 of 1 [96]

• Discuss the importance of text encoding for the humanities
• Explain what TEI stands for
• Assess the value of TEI for your project
• Start on a TEI encoding project

Overview

(c) Edward Vanhoutte 1 of 1 [96]

• Part 1: History and Theory
  • Textual Modelling
  • History
  • XML
  • TEI
• Part 2: Demonstration
  • Electronic scholarly editions
  • Digital Archive of Letters in Flanders
  • James Joyce Timeline: Finnegans Wake Notebooks
  • TEI by Example

Application: Where can a bike be used?

(c) Edward Vanhoutte 9 of 28 [96]

• Geographically
• Topographically

Application: When can a bike be used?

(c) Edward Vanhoutte 10 of 28 [96]

• Time
• Age
• Purpose

Use: Who can use a bike?

(c) Edward Vanhoutte 11 of 28 [96]

• Gender
• Race
• Religion
• Ability

Application: Where can a text be used?

(c) Edward Vanhoutte 22 of 28 [96]

• Geographically
• Topographically

Application: When can a text be used?

(c) Edward Vanhoutte 23 of 28 [96]

• Time
• Age
• Purpose

Use: Who can use a text?

(c) Edward Vanhoutte 24 of 28 [96]

• Gender
• Race
• Religion
• Ability

Computing in the Humanities: 1949-1970

(c) Edward Vanhoutte 1 of 10 [96]

• ad hoc programming in projects
• project-specific markup languages
• some sort of standardization: e.g. COCOA

Computing in the Humanities: 1970-1985

(c) Edward Vanhoutte 2 of 10 [96]

• method oriented program packages: e.g. SPSS (Statistical Package for the Social Sciences)
• OCP (Oxford Concordance Program) takes COCOA as input

Computing in the Humanities: 1985-1993

(c) Edward Vanhoutte 3 of 10 [96]

• PC revolution
• standard software
• dBase, Access
• SGML ISO 8879:1986 (Standard Generalized Markup Language)

Computing in the Humanities: 1993-1999

(c) Edward Vanhoutte 5 of 10 [96]

• 1993: launch of the WWW
• computer as presentational medium
• computer as web tool
• HTML technology
• SGML technology

Computing in the Humanities: 1999-2009

(c) Edward Vanhoutte 8 of 10 [96]

• 1999: XML (eXtensible Markup Language)
• computer as presentational medium
• computer as web tool
• distributed computing
• XML technology

TEI Short History

(c) Edward Vanhoutte 10 of 10 [96]

• 1987: Vassar College, Poughkeepsie
• 1990: P1 : SGML
• 1992: P2 : SGML
• 1994: P3 : SGML : 600 + elements
• 1995: TeiLite : SGML : 131 elements
• 1999: P3rev : SGML
• 2000: TEI Consortium
• 2001: P4 : XML comp
• 2001: TeixLite : XML
• 2005: P5 : XML
• 2007: P5 version 1

Why would you want to learn about XML?

(c) Edward Vanhoutte 4 of 29 [96]

• It's a good approximation
• It works (fairly well)
• It's widely supported
• It's a W3C recommendation

Why would you want to learn about XML?

(c) Edward Vanhoutte 5 of 29 [96]

• Migration will be supported
• Migration will create more jobs

XML: Five Essential Components

(c) Edward Vanhoutte 8 of 29 [96]

1. Processing Instructions: <?xml version="1.0" ?>
2. Elements: <title> or </title>
3. Attributes (optional): <title type="journal" >
4. Entity References: &eacute; = &#233; = é
5. CDATA: <![CDATA[This text escapes the XML processor]]>

XML comes in 2 flavours

(c) Edward Vanhoutte 10 of 29 [96]

1. Well-formed XML
2. Valid XML

Well-formed XML

(c) Edward Vanhoutte 11 of 29 [96]

1. XML is case sensitive
2. At least one element
3. There is always a root element
4. All logical and physical structures nest properly
5. Correspondence of element names in start- and end-tags
6. Attribute names only appear once in a start-tag
7. Attribute values are quoted
8. Attribute values do not refer to external entities
9. Entities are declared
10. No entity reference contains the name of a non-parsed entity

Valid XML

(c) Edward Vanhoutte 12 of 29 [96]

• Well-formed XML
• Conforms to a Document Type Definition (DTD)
• Or to a valid Schema

Document Type Definition

(c) Edward Vanhoutte 13 of 29 [96]

<?xml version="1.0" ?>
<greeting>
    <salutation>hello</salutation>
    <target>world</target>
</greeting>
               

Document Type Definition

(c) Edward Vanhoutte 14 of 29 [96]

<?xml version="1.0" ?>
<greeting>
    <salutation>hello</salutation>
    <target>world</target>
</greeting>
               

<!ELEMENT greeting (salutation, target)>
<!ELEMENT salutation (#PCDATA)>
<!ELEMENT target (#PCDATA)> 

Document Type Definition

(c) Edward Vanhoutte 15 of 29 [96]

• names for all your elements
• names and default values for their attributes
• rules about how elements can nest
• names for re-usable pieces of data (entities)
• and a few other things

Defining an element in a DTD

(c) Edward Vanhoutte 16 of 29 [96]

<!ELEMENT name contentModel>
               

• name is the name of the element (GI)
• contentModel defines valid content for the element:
  • #PCDATA
  • EMPTY
  • other elements
  • mixed content combining PCDATA and other elements

Theoretical problem with the DTD

(c) Edward Vanhoutte 20 of 29 [96]

• DTDs are written in a formal language different from the XML documents.

• A self-declarative way of documenting the formal specification for the structure of an XML document
• Expressed in XML itself

Problems with Schema

(c) Edward Vanhoutte 21 of 29 [96]

• W3C XML Schema: http://www.w3.org/XML/Schema
• Relax NG: http://www.relaxng.org
• XML-Data (XDR)
• Document Content Description (DCD)
• Schema for Object-oriented XML (SOX)
• Document Definition Markup Language (DDML)
• Schematron
• Datatypes for DTDs (DT4DTD)
• Document Structure Description (DSD)
• Regular Language Description for XML (RELAX)
• TREX (Tree Regular Expressions for XML)
• Examplotron
• Hook
• Document Schema Definition Language (DSDL)
• STEP/EXPRESS and XML

Book DTD

(c) Edward Vanhoutte 22 of 29 [96]

<!ELEMENT   book    (chapter+)>
<!ELEMENT   chapter (section+)>
<!ELEMENT   section (p+)>
<!ELEMENT   p       (#PCDATA)>

               

Book Schema (Relax NG Simple notation)

(c) Edward Vanhoutte 23 of 29 [96]

<?xml version="1.0" encoding="UTF-8"?>
<grammar xmlns="http://relaxng.org/ns/structure/1.0"
xmlns:a="http://relaxng.org/ns/compatibility/annotations/1.0"
>

 <start>
      <ref name="book" />
  </start>

  <define name="book">
    <element>
      <name>book</name>
      <oneOrMore>
        <ref name="chapter" />
      </oneOrMore>
    </element>
  </define>

 <define name="chapter">
    <element>
      <name>chapter</name>
      <oneOrMore>
        <ref name="section" />
      </oneOrMore>
    </element>
 </define>

 <define name="section">
    <element>
      <name>section</name>
      <oneOrMore>
        <ref name="p" />
      </oneOrMore>
    </element>
 </define>

 <define name="p">
    <element>
      <name>p</name>
      <text />
    </element>
 </define>

</grammar>

               

Why a DTD/Schema?

(c) Edward Vanhoutte 25 of 29 [96]

• A DTD/Schema is very useful at data preparation time: validating editors only allow you to input correct markup
• Useful for consistent encoding in projects
• Guarantees longevity and interchangeability of semantics and structure in encoded texts
• We need it for validation
• Useful for software development and operability

Valid XML

(c) Edward Vanhoutte 26 of 29 [96]

<!DOCTYPE TEI PUBLIC "-//TEI//DTD TEI Lite 1.0//EN"
 "../dtd/teixlite.dtd" [ ]>
               

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified">

Validating XML

(c) Edward Vanhoutte 27 of 29 [96]

• an XML document
• a DTD or Schema
• a validating parser

Writing DTDs or Schema

(c) Edward Vanhoutte 28 of 29 [96]

• Difficult
• Time Consuming
• Expensive

Writing DTDs or Schema

(c) Edward Vanhoutte 29 of 29 [96]

• Difficult
• Time Consuming
• Expensive

• The work has been done for you

What does the TEI offer you?

(c) Edward Vanhoutte 2 of 25 [96]

• Vocabulary (elements)
• Tuning tools (attributes)
• Syntax (content models and nesting rules)
• Modification and extension guidelines

For which purposes?

(c) Edward Vanhoutte 3 of 25 [96]

• Edit texts (e.g. word processors, syntax-directed editors)
• Edit, display, and link texts in hypertext systems
• Format and print texts using desktop publishing systems, or batch-oriented formatting programs
• Load texts into free-text retrieval databases or conventional databases
• Unload texts from databases as search results or for export to other software
• Search texts for words or phrases
• Perform content analysis on texts
• Collate texts for critical editions
• Scan texts for automatic indexing or similar purposes
• Parse texts linguistically
• Analyze texts stylistically
• Scan verse texts metrically
• Ink text and images

How does the TEI offer this to you?

(c) Edward Vanhoutte 4 of 25 [96]

• Website http://www.tei-c.org
• On-line reference documentation: Guidelines
• Print reference documentation: Guidelines
• On-line schema/DTD generator: ROMA
• On-line tutorials: TEI by Example http://www.teibyexample.org
• Free software
• Free stylesheets
• Community of practitioners: TEI-L maillist
• Experience of projects
• Opportunity to participate through SIG's
• Conferences

TEI Modules

(c) Edward Vanhoutte 5 of 25 [96]

1. TEI Infrastructure
2. TEI Header
3. Elements Available in All TEI Documents
4. Default Text Structure
5. Representation of Non-standard Characters and Glyphs
6. Verse
7. Performance Texts
8. Transcriptions of Speech
9. Dictionaries
10. Manuscript Description
11. Representation of Primary Sources
12. Critical Apparatus
13. Names, Dates, People, and Places
14. Tables, Formulae, Graphics
15. Language Corpora
16. Linking, Segmentation, Alignment
17. Simple Analytical Mechanisms
18. Feature Structure
19. Graphs, Networks, Trees
20. Certainty and Responsibility
21. Documentation Elements

<teiHeader>

(c) Edward Vanhoutte 7 of 25 [96]

• bibliographic control and identification
• resource documentation and
• processing (see later)

Textual Phenomena

(c) Edward Vanhoutte 10 of 25 [96]

1. Structural
2. Renditional
3. Logical & Semantic
4. Analytical

Structural Features

(c) Edward Vanhoutte 11 of 25 [96]

• Prose: paragraphs <p>, divisions <div>, headings <head>, lists <list>, listitem <item>, quotations <q>, page breaks <pb>, segments <seg>, figures <figure>, and tables <table>.
• Verse: linegroups <lg> and lines <l>.
• Drama: divisions <div>, speeches <sp>, paragraphs <p>, linegroups <lg>, lines <l>. and segments <seg>.

Logical Features

(c) Edward Vanhoutte 17 of 25 [96]

• Emphasis <emph>, foreign words <foreign> and other linguistically distinct uses <distinct> of highlighting
• The use quotation marks in the representation of speech and thought <said>, quotation <quote>, cited quotation <cit>. words or phrases mentioned <mentioned> and words or phrases for which the author or narrator indicates a disclaiming of responsibility <soCalled>.
• Technical terms <term>, glosses <gloss> or documentation of XML elements, attributes and classes with <altIdent>, <desc>, <equiv/>

Analytical Features

(c) Edward Vanhoutte 19 of 25 [96]

• Ommissions, deletions, additions: <gap>, <del>, <add>, <supplied>, <unclear>, <damage>
• Regularization: <reg>, <orig>
• Editorial interventions: <corr>, <sic>

The multiple hierarchy

(c) Edward Vanhoutte 20 of 25 [96]

• XML allows only one hierarchy at a time
• Is a document
  • chapter-paragraph-phrase
  • gathering-page-leaf
  • or both?
• discontinuous segments
• links and milestones

Losing the Pedals? → TEILite

(c) Edward Vanhoutte 23 of 25 [96]

• one of many possible views of the TEI DTD
• small and simple
• designed to meet 90% of the needs of 90% of the TEI user community
• 128 elements
• was devised as a didactic stepstone to the full flavour TEI, but began a life of its own
• realistic for existing texts and for document production

Webliography: TEI

(c) Edward Vanhoutte 25 of 25 [96]

• TEI Consortium: http://www.tei-c.org
• TEI P5 Guidelines: http://www.tei-c.org/P5/
• TEILite: http://www.tei-c.org/Lite/
• TEI software: http://www.tei-c.org/Software/index.html
• TEI Roma: http://tei.oucs.ox.ac.uk/Roma/
• Projects using the TEI: http://www.tei-c.org/Applications/index-subj.html