Best Practices for TEI in Libraries

From TEIWiki
Revision as of 20:17, 27 January 2009 by Syd (talk | contribs) (→‎LEVEL 5: Scholarly Encoding Projects)
Jump to navigation Jump to search

TEI in Libraries: Home

Introduction

These recommendations are for libraries using the Text Encoding Initiative’s Guidelines for Text Encoding and Interchange (P5). They are intended for use in large, library-based digitization projects, but may be useful in other scenarios as well.

There are many different library text digitization projects, for different purposes. With this in mind, the Task Force has attempted to make these recommendations as inclusive as possible by developing a series of encoding levels. These levels are meant to allow for a range of practice, from wholly automated text creation and encoding, to encoding that requires expert content knowledge, analysis, and editing.

Recommendations for Levels 1-4 are intended for projects wishing to create encoded electronic text with structural markup, but minimal semantic or content markup. Also, the encoding levels are cumulative: encoding requirements at each level incorporate the requirements of lower levels. Levels 1-4 allow the conversion and encoding of texts to be performed without the assistance of deep content knowledge and can be enriched with more markup at any time. Level 5, in contrast, requires scholarly analysis.

For a history of this document, see Appendix A.

General Recommendations

Note: all recommendations that follow are intended to conform to TEI P5 version 1.2 (released 2008-11); however, as we convert from the previous version (based on P4:2004), there are likely to be exceptions.

  • The encoding level (as described in this document) should be recorded in the <editorialDecl> in the TEI header, along with an explanation of any deviation from the recommendations.
  • When reformatting to digital media using any level of encoding, the electronic text should begin with the transcription of the first word on the first leaf of the original work. It may be impractical or undesirable to transcribe and encode certain features of the text, such as publisher's advertisements or indexes, but if at all possible, they should be included as links to page images. Any omissions of material found in the original work should be noted in the <editorialDecl> in the TEI header.
  • File naming should follow ISO 9660 conventions: 8-character filenames, 3-character extensions, using A-Z, a-z, 0-9, underscores and hyphens. The rationale behind this suggestion is that when moving texts across different platforms (DOS for instance), some systems will truncate beyond the eighth character. File naming section re-written to take into account all the comments; see http://wiki.tei-c.org/index.php/Talk:TEI_in_Libraries:_Guidelines_for_Best_Practices#General_Recommendations_section. (Mdalmau)]
  • We recommend the use of numbered divisions throughout the electronic text. Numbered divisions present advantages to search and indexing software by explicitly communicating the hierarchical level of the section described. Texts at all levels should include at least one <div1>. Numbered division section re-written to take into account all the comments; see http://wiki.tei-c.org/index.php/Talk:TEI_in_Libraries:_Guidelines_for_Best_Practices#Numbered_Divs (Mdalmau)]
  • Page breaks should be encoded using the <pb> element, which should demark the top of a page (i.e. the text of page seven should immediately follow <pb n="7"/>), and should always be contained within a division. E.g., a page break that occurs between chapters 2 and 3 should be encoded near the top of the <div> that holds chapter 3 (rather than near the bottom of the <div> that holds chapter 2). Page break recommendation is ok as-is. See http://wiki.tei-c.org/index.php/Talk:TEI_in_Libraries:_Guidelines_for_Best_Practices#Page_Breaks for an overview of the original issue (Mdalmau).
  • Be consistent in your encoding, both within a single document and across your collection.

The TEI Header

[We should link to corresponding sections of the P5 guidelines when relevant and throughout. We could create a "References" section after discussion of each level, but we should also provide in-line links as well. Rationale posted in the discussion tab. (Mdalmau)]

The TEI(emcaulay) header may be used to(emcaulay) describe a collection of documents, a single item, or a portion of an item. Variances in TEI header content can result from making different choices of what is being described. [ I don't understand.(emcaulay)] [Within the library domain, a TEI Header is often perceived as similar to or at least related to a MARC record. However, (emcaulay) a TEI header does not have to (emcaulay) have a one-to-one correspondence with a MARC record: one TEI header may be described by multiple MARC analytic records, or one MARC record may be used to describe a collection of TEI documents with individual headers.

A TEI header serves several purposes. It may contain an historical background on how the file has been treated. It can extend the information of a classic catalog record. The text center or cataloging agency can act as the gatekeeper for creators by providing standards for content. A TEI header can serve many publics: headers can be created in a text center and reflect the center's standards, or they can serve as the basis for other types of metadata system records produced by other agencies. Headers can function in detached form as records in a catalog, as a title page inherent to the document, or as a source for index displays.

Chief Sources of Information for Creating the TEI Header

Does the TEI header act as the electronic title page for the encoded document (part of the item) or as a catalog record for it (pure metadata)? Is it integral to the document it describes or independent? Depending on the community being served, the TEI elements will reflect the interest of that community. Nonetheless, it is possible to describe a set of "best practices" that will produce compatible content while accommodating this variety of purposes. Compatibility of content encourages a more understandable set of results when information about assorted items is displayed as a set of search results, a contents list, or an index, and it allows for more reasonable conversion of content information from TEI elements to elements of other metadata sets when this action seems advisable.

[the preceding paragraph is hard to understand and I want answers to those nice questions at the beginning!] (emcaulay)

It is a traditional practice of librarianship to agree upon which location(s) in a document and in what order of preference one should look to identify the title, author, etc., of that document. This practice permits a certain consistency in terminology and allows for a certain amount of authentication of content. We recommend the following preferences to those who create headers and to those who attempt to use headers to create traditional catalog records that are compliant with AACR2 Anglo-American Cataloging Rules 2nd Edition and ISBD(ER) International Standard Bibliographic Description for Electronic Resources rules.

As a member of the academic community, the header creator or editor has a responsibility to verify, whenever humanly possible, the intellectual source for an electronic document that presents itself without any information regarding its source or authorship.

Who Should Create and / or the TEI Header

Every group will have its own method for creating and editing TEI Headers. Generally, the person who creates the TEI Header is familiar with TEI and is also familiar with bibliographic description. (emcaulay)


Chief Sources of Information for TEI documents

  1. For an electronic document with a digitized title page, prefer
    1. Chief source of information = information coded as title page
    2. Use added information from an originating paper document if absolutely certain it is the source
  2. If no title page is present and there is no evidence of a source document, the header creator
    1. May assign a title and author if appropriate
    2. Enclose the information in brackets, using the standard English language convention for editorial interjections
  3. If neither header nor title page is present but the header creator has satisfactory evidence of an originating source, that document should be used as the chief source of information for the title and author of the header. If the source cannot be fully verified as to edition, authorship, etc., this fact should be clearly indicated in a note in the <fileDesc>. Shouldn't this be clarified in sourceDesc?. (Mdalmau)]

Element Recommendations for the <teiHeader>

<teiHeader type="____"> [Syd and Melanie don't like this use of type, finding it semantically ambiguous and suggesting there are many possible uses for the type attribute.] QUOTED FROM TEI P5 GUIDELINES, CHAPTER TWO: <teiHeader> (TEI Header) supplies the descriptive and declarative information making up an electronic title page prefixed to every TEI-conformant text. attribute:type specifies the kind of document to which the header is attached, for example whether it is a corpus or individual text. [END QUOTE] I think this confirms Syd and Melanie's concern about the value being assigned to the type attribute (if I understand what they don't like). --Emcaulay 17:02, 1 December 2008 (EST)

Standards which apply to the header, e.g., <teiHeader type="ISBD(ER)">, <teiHeader type="AACR2">

<fileDesc>

The fileDesc element is used to describe both the electronic TEI document and the original source document from which the electronic text was created. TEI P5 Guidelines--Emcaulay

Again, I've always seen this as entering the document, not the encoding. (Emcaulay )
<title type="____"> [TEI P5 requires use of @level, but TEI Lite does not. So we'll probably want to require @level. (Kshawkin)]

Title given to the TEI document being created. If the TEI Header refers to multiple documents, then the title is constructed according to standard library practice. If the TEI Header refers to only one document, the title is usually the title of the original source, possibly with some indication that the TEI file is an electronic edition.(Emcaulay ) [See pp. 50-51 in PDF version of P5 guidelines for possible clarification. (Mdalmau)] [We should prescribe use of this tag: one or more title tags? With which values of @type? (Kshawkin)]

<author> Author of the TEI document being created. Rarely applicable. [Melanie and Kevin thought this was rarely applicable since they thought of a TEI document as an edition edited by encoders (whose names go in the editor element) but without any true author. However, new editions are thought to be "by" the original author, not the editor, so maybe this should be reconsidered. (Kshawkin)] I've always seen this as the bibliographic citation for the electronic edition of the book (or other document). So for a TEI file that is an electronic edition of Charles Dickens's A Christmas Carol, the author in this part of the header would be Charles Dickens. Worth discussing. (Emcaulay ) Whenever possible, establish or use the form of the name from a national name authority file.
<editor> Editor of the TEI document should be entered into the <editor> element before the <respStmt>. Whenever possible, establish or use the form of the name from a national name authority file.
<respStmt> Record the names of other persons or organizations not covered by <author> and <editor>. Whenever possible, establish or use the form of the name from a national name authority file. [specify order of resp and name child elements? (Kshawkin)]
<editionStmt> This element contains information about the edition of the the TEI document produced, not the source document.
<publicationStmt> Use the child elements below rather than <p> for a prose description.
<publisher> The publisher is the party responsible for making the file (the TEI document, not the source document) public.
<distributor> The distributor is the party from whom copies of the file (the TEI document, not the source document) can be obtained. Often the same as <publisher>, in which case no <distributor> element should be specified.
<authority> Only used for a text (the TEI document, not the source document) that is not formally published, but is nevertheless made available for circulation, in which case the party who makes it available should be recorded here.
<idno> Any unique identification number for the TEI document determined by the publisher.
<availability status="___"><p> [It turns out that the status attribute is optional. Since its three legal values are semantically vague, I suggest we no longer recommend its use. (Kshawkin)] Provide a rights statement for the TEI document. Use a standard license, such as one from Creative Commons, if possible.
<date> Refers to the date of the first publication of the TEI document. For most purposes, the year alone (yyyy) will be adequate. If greater precision is required, enter dates as yyyy-mm-dd.

<seriesStmt><title> [TEI P5 requires use of @level, but TEI Lite does not. So we'll probably want to require level="s". (Kshawkin)]

Whenever possible, establish or use the form of the name from a national name authority file for the electronic series. [We should prescribe use of this tag: one or more title tags? With which values of @type? (Kshawkin)]

<notesStmt> Optional.
<sourceDesc> In order to effectively represent the source(s) when many documents are represented by the TEI header in the absence of structures identifying parent-child and component relationships, multiple source descriptions should be employed with relationships described in free text. Relationships also could be useful in other portions of the TEI header. Cataloger may need to do research to establish the original source.
<bibl> Metadata for the source document is automatically generated from a MARC record. Use <bibl> with child elements listed, in the order below, for ease of display according to ISBD: <author>, <title type="245a">, <title type="245b">, <respStmt>, <edition>, <pubPlace>, <publisher>, <date>, <extent>, <title type="series">, <note>, <idno type="isbn-13">, <idno type="isbn-10">. [This may not be standard practice (auto-generation of header from MARC). But for me the issue is book-centricity. I think we either provide a general example but emphasize structured biblio. metadata or multiple examples that cover a range of content types (the latter being more difficult to ascertain and document. (Mdalmau)] [Melanie and I didn't mean this to be book-centric. Would be happy to have examples of manuscripts or other non-book material. But I do believe we should allow for generation from MARC. (Kshawkin)] [Now that we plan to give up on the fiction of using only the TEI Lite schema, we could use biblStruct instead of bibl. (Kshawkin)]
<author> Name authority heading for the main entry of the source document when this entry is a personal author or corporate body. [Maybe this should instead be "name authority headings for main and added entries of the source document when these entries are personal authors or corporate bodies"? That is, is there any reason to continue with the "main entry" nonsense that we've been living with since card catalogs? (Kshawkin)] It is possible to have multiple <author> elements. Whenever possible, establish or use the form of the name from a national name authority file.

<title type="245a"> [TEI P5 requires use of @level, but TEI Lite does not. So we'll probably want to require @level. (Kshawkin)]

Title proper, according to the national cataloging code.

<title type="245b"> [TEI P5 requires use of @level, but TEI Lite does not. So we'll probably want to require @level. (Kshawkin)]

Other title information, according to the national cataloging code, if present.
<respStmt> Statement of responsibility, according to the national cataloging code. [If metadata is generated automatically from a MARC record, we can't automatically determine what text should go in resp and name elements! (Kshawkin)]
<edition> Edition statement (if present).
<pubPlace> Place of publication from the original source (if present)
<publisher> First publisher etc. from the original source (if present)
<date when="____"> [I suggest not requiring the when attribute. AACR2 practice is to transcribe only the year of publication in Arabic numerals, so there won't be any need to add something more machine-readable. (Kshawkin)] Date of publication etc. from the original source (if present).
<extent> Extent of item, according to national cataloging code.
<title type="series">

[Or just level="s" if we give up TEI Lite compatibility (Kshawkin)]

Series statement, according to national cataloging code.
<note> Notes about the source document, according to national cataloging codes.
<idno> In this location, <idno> refers to identification numbers for the source document. They can be used to indicate the source's location in an individual institution's collection. If a formal standard location system is being used, indicate the nature of the system, e.g., <idno type="LC call number">. [Syd thinks we should recommend a specific abbreviation for LC call numbers. (Kshawkin)] Use type="isbn-13" and type="isbn-10" if applicable.

<encodingDesc>

<projectDesc><p> Enter a description of the purpose for which the electronic file was encoded.
<editorialDecl n="_"><p> Record editorial decisions made during encoding. Record encoding level as an arabic numeral in the n attribute.
<classDecl> <taxonomy xml:id="____"> <bibl>

[Needs to be rewritten. (Kshawkin)] If used, identify the appropriate taxonomy definitions or descriptive sources in the <taxonomy> element followed by xml:id, e.g.,

<taxonomy xml:id="LCSH"> <bibl>Library of Congress Subject Headings</bibl> </taxonomy>, <taxonomy xml:id="AAT"> <bibl> Art & Architecture Theasaurus</bibl> </taxonomy>.

<profileDesc>

<textClass> The elements below are contained within this element.
<classCode scheme="___"> True classification numbers as opposed to call numbers can be entered here.

<keywords>
<term>

Use for uncontrolled terms.

<keywords scheme="____">
<term>

Use for controlled vocabulary as specified in <encodingDesc> taxonomy xml:id. Example: scheme="#LCSH"

<revisionDesc>

<change when="YYYY-MM-DD" who="URI">

Create a <change> element to record each significant change to the TEI document, in reverse chronological order (i.e., most recent first).

when="YYYY-MM-DD"

The date of the change in ISO 8601 form (YYYY-MM-DD) should be recorded on the when= attribute of <change>.

who="URI"

The person who is responsible for making the change is indicated by the who= attribute of <change>. Its value is a URI that points to a <respStmt> or <person> element that encodes information about the responsible party. Note that this reference is a URI reference and not an ID/IDREF reference, and thus is not checked by validation software. Small projects sometimes take advantage of this by putting information into the URI itself, and not having a <respStmt> or <person> element. E.g., who="#Kevin_Hawkins".

content

A prose description of the change is recorded as the content of each <change> element. This prose may contain lists for organization, and phrase-level markup (like <gi>, <ptr>, or <date>), but not paragraphs.

Minimal TEI Header Recommendation

[This minimal header will need to be revised once the element recommendations above are agreed upon. But I think we should replace this minimal header with a full sample header. (Kshawkin)] 

  <teiHeader>
    <fileDesc>
      <titleStmt>
        <title> </title>
        <author> </author>
      </titleStmt>
      <publicationStmt>
        <publisher> </publisher>
        <pubPlace> </pubPlace>
        <idno> </idno>
        <availability status="">
          <p> </p>
        </availability>
        <date when=""/>
      </publicationStmt>
      <sourceDesc>
        <biblFull>
          <titleStmt>
            <title> </title>
            <author> </author>
          </titleStmt>
          <publicationStmt>
            <publisher> </publisher>
            <pubPlace> </pubPlace>
            <date when=""/>
          </publicationStmt>
        </biblFull>
      </sourceDesc>
    </fileDesc>
  </teiHeader>

Repeat the <biblFull> field, as appropriate, if there is more than one source for the electronic item. See some examples in context …


Acknowledgments and Bibliography

The header portion of this document was prepared by Judy Ahronheim, Thomas Champagne, Lynn Marko, Kelly Webster, and Chris Wilcox of the University of Michigan Library and Jackie Shieh of the University of Virginia Library in October 1998. The source documents were the cataloging guides prepared by those two institutions (Virginia and Michigan). In addition, documentation from the Oxford Text Archive, Arts and Humanities Data Service of the United Kingdom also was made available to assist in this effort.

This text was heavily revised in 2008 by Melanie Schlosser and Kevin Hawkins, with input from other members of the SIG on Libraries.

Encoding Levels

LEVEL 1: Fully Automated Conversion and Encoding

Purpose: To create electronic text with the primary purpose of keyword searching and linking to page images. The primary advantage in using the teixlite DTD at this level is that a TEI header is attached to the text file.

Rationale: The text is subordinate to the page image, and is not intended to stand alone as an electronic text (without page images).

Texts at Level 1 can be created and encoded by fully automated means, using uncorrected OCR of page images ("dirty OCR"), exporting from existing electronic text files, or actually not including any text at all. Only those elements that are necessary to divide the text from the header and facilitate linking to page images are used. Encoding is performed automatically based on artifacts of the OCR or other document creation process (page breaks, for example) and metadata collected during the imaging or preparation process. This encoding is both minimal and reliable, and does not typically require extensive review of each page of each text.

Level 1 texts are not intended to be adequate for textual analysis; they are more likely to be suited to the goals of a preservation unit or mass digitization initiative. Though their encoding is minimal, Level 1 texts are fully valid XML texts. In addition to taking advantage of the TEI header, using the teixlite DTD--with the consistency suggested by these guidelines--allows Level 1 texts to be compatible with more richly encoded teixlite texts (that also follow these guidelines) for searching, for example. Further encoding based on document structures or content analysis can be added to a Level 1 text at any time.

Level 1 is most suitable for projects with the following characteristics:

  • a large volume of material is to be made available online quickly
  • a digital image of each page is desired
  • no manual intervention will be performed in the text creation process
  • the material is of interest to a large community of users who wish to read texts that allow keyword searching
  • sophisticated search and display capabilities based on the structure of the text are not necessary
  • extensibility is desired; that is, one desires to keep open the option for a higher level of encoding to be added at a later date


<div1> If no type= attribute is specified, a type= value of "section" should be presumed.
<p> At least one "container" element per div is required (while <ab> is another option for this case, the Task Force suggests using <p> in order that the document be open to being extended to other encoding levels).I don't remember this discussion. It doesn't seem very difficult, once the decision is made to upgrade, to transform all ab's to p's. Or? [pwillett] See comments in http://wiki.tei-c.org/index.php/Talk:TEI_in_Libraries:_Guidelines_for_Best_Practices#Paragraphs_or_Anonymous_Block [Mdalmau]
<pb> Required in Level 1. Page images can be linked to the text by specifying a jpeg or other image file as the value of the facs= attribute. Page numbers can be supplied with the n= attribute to record the number that is on the page. The Task Force sees the use of METS here as having a tremendous advantage. METS/TEI page turning documentation will be included in the near future.

Basic Structural Example:

<TEI xml:id="someid" xmlns="http://www.tei-c.org/ns/1.0">
  <teiHeader>
    [Source and processing information goes here]
  </teiHeader>
  <text>
    <body>
      <div1>
        <p>
          <pb xml:id="p00000001" n="1"/>
          [main body of the unmarked up plain text begins here]
          <pb xml:id="p00000002" n="2"/>
          [more plain text goes here with appropriate page breaks interspersed] ...
          <pb xml:id="p00000145" n="145"/>
          [more plain text]
          <pb xml:id="p00000146" n="146"/>
          [text ends here]
        </p>
      </div1>
    </body>
  </text>
</TEI>

See an example in context…

LEVEL 2: Minimal Encoding

provisional revisions made 12/1/08(emcaulay)

Purpose: To create electronic text for full-text searching, linking to page images, and identifying simple structural hierarchy to improve navigation. (For example, you can create a table of contents from such encoding.)Emcaulay

Rationale: The text is mainly (Emcaulay) subordinate to the page image, though navigational markers (textual divisions, headings) are captured. However,(Emcaulay) the text could stand alone as electronic text (without page images) if the accuracy of its contents is suitable to its intended use and it is not necessary to display low-level typographic or structural information. Level 2 requires a set of elements more granular than those of Level 1, including bibliographic or structural information below the monographic or volume level. One of the motivations for using Level 2 is avoiding expensive analysis of textual elements and/or expensive double-keying or OCR plus proofreading.(Emcaulay)

Though texts at Level 2 can be created and encoded by automated means, based on the typographic elements in the electronic file (for example, bold centered text at the top of the page surrounded by whitespace indicates a new chapter heading, and thus a new division), it is not likely to be absolutely reliable across a large body of material, especially if the materials are from earlier than 1900. Level 2 encoding requires some human intervention to identify each textual division and heading. Level 2 texts do not require any special knowledge or manual intervention below the section level.

[REMOVE (CONTRADICTS FIRST PARAGRAPH OF RATIONALE: Level 2 texts are not intended to be displayed separately from their page images.] (emcaulay) Level 2 encoding of sections and headings provides greater navigational possibilities than Level 1 encoding, and enables searching to be restricted within particular textual divisions (for example, searching for two phrases within the same chapter). I don't see how this contradicts the first paragraph of rationale. [pwillett]

Level 2 is most suitable for projects with the following characteristics [delete "with the following characteristics"; replace with "in which" (Wisneski):

  • a large volume of material is to be made available online quickly
  • a digital image of each page is desired
  • the material is of interest to a large community of users who wish to read texts that allow keyword searching
  • rudimentary search and display capabilities based on the large structures of the text are desired
  • each text will be [delete "will be"; replace with "is" (Wisneski)] checked to ensure that divisions and headers are properly identified
  • extensibility is desired; that is, one desires to keep open the option for a higher level of encoding to be added at a later date

All elements specified in Level 1 plus the following:

<front>, <back> Optional
<div1> type="section" is the default attribute value. It is recommended that the n= attribute be included to record the sequence of divisions. [I prefer not to do sequencing ... it's a pain in the butt to keep track of in long documents and I'm not sure what the benefit is. Looking for others' thoughts on this. (Emcaulay)] I also think sequencing is a pain, and have never done it. [pwillett]
<head> Required if present
<p> One "container" element per div is required. [I'm not an expert on tag abuse, but this is perhaps an example of it. The <p> element is used to encode paragraphs, but in the Level 2 approach, it is used to contain all material within a division (regardless of paragraph breaks). I WOULD LIKE TO DISCUSS ON 12/02/08 (Emcaulay) I think we should be recommending AB instead. [pwillett]

Basic Structural Example:

NEW EXAMPLE FRAGMENT

<TEI xmlns="http://www.tei-c.org/ns/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.tei-c.org/ns/1.0 http://www.tei-c.org/release/xml/tei/custom/schema/xsd/teilite.xsd">
 <teiHeader type="text">[See above for an example of a TEI Header]</teiHeader>
 <text>
  <front>[titlepage information, table of contents, prefaces, etc.][optional]</front>
  <body>
   <div type="section" n="1">
    <pb xml:id="p21198-zz0002mpqr" n="1"/>
    <head>A DISSERTATION UPON Religious Worship.</head>
    <p>[a whole section is contained within this paragraph tag; interspersed with <pb> elements pointing to page
        images]<pb xml:id="p21198-zz0002mpwb" n="2"/></p>
   </div>
   <div type="section" n="2">
    <pb xml:id="p21198-zz0002mq0c" n="27"/>
    <p>
     <figure xml:id="ill005">
      <graphic url="imag1.jpg"/>
     </figure>
    </p>
    <div type="subsection" n="1">
      <head>CHAP. I. The Origin of the Customs and Ceremonies of the Jews. their federal Divisions;
      and the various Particulars wherein they differ.</head>
        <p>[all the paragraphs of chapter one go here with page breaks inserted]</p>
    </div>
   </div>
  </body>
  <back> [optional] </back>
 </text>
</TEI>

[I do have a full example [very long!!!], but it's not live on the UCLA DLP site yet, let's discuss this part tomorrow? (Emcaulay)

OLD EXAMPLE (Emcaulay) 

<TEI xmlns="http://www.tei-c.org/ns/1.0" xml:id="someid">
  <teiHeader> [Source and processing information goes here] </teiHeader>
  <text xml:id="someotherid">
    <front> [titlepage information, table of contents, prefaces, etc.] </front>
    <body>
      <div1 type="chapter" n="1">
        <head>Chapter 1</head>
        <p>[text of Chapter 1 goes here interspersed with <pb> elements pointing to page
        images]</p>
      </div1>
      <div1 type="chapter" n="2">
        <head>Chapter 2</head>
        <p>[text of Chapter 2 goes here interspersed with <pb> elements pointing to page
        images]</p>
      </div1>
      <div1 type="chapter" n="3">
        <head>Chapter 3</head> [text of Chapter 3 goes here interspersed with <pb> elements
        pointing to page images] </div1>
      <div1 type="chapter" n="4">
        <head>Chapter 4</head> [text of Chapter 4 goes here interspersed with <pb> elements
        pointing to page images] </div1>
    </body>
    <back> [optional text of backmatter, appendices, etc.] </back>
  </text>
</TEI>

See an example in context…

LEVEL 3: Simple Analysis

Reference: Default Text Structure, P5 Guidelines

Purpose: To create text that can stand alone as electronic text and identifies hierarchy and typography without content analysis being of primary importance.

Rationale: Level 3 texts can be created from scratch or by the relatively easy conversion of existing HTML or word-processing documents. Encoding offers the advantage of the TEI header, interoperability with other TEI collections, and extensibility to higher levels of encoding. Level 3 generally requires some human editing, but the features to be encoded are determined by the appearance of the text and not specialized content analysis.

Level 3 texts identify front and back matter, and all paragraph breaks. The finer granularity of encoding these features, as well as figures, notes, and all changes of typography, allows a range of options for display, delivery, and searching. For example, one has the option of identifying and, therefore, specifying the display charactersitics of different typographic styles, and regularizing the display and placement of note text.

Level 3 texts can stand alone as text without page images and, therefore, can be uploaded, downloaded and delivered quickly, and require less storage space than digital collections with page images. However, the simple level of structural anaylsis and absence of specialized content analysis reflected in Level 3 encoding may make it desirable for some, depending on project priorities, to include page images in order to provide users with a fuller set of resources.

Level 3 is most suitable for projects with the following characteristics:

  • the material is of interest to a large community of users who wish to read texts that allow keyword searching
  • some sophistication of display, delivery, and searching based on structure of the text is desired
  • each text will be checked to ensure that encoding decisions have been made appropriately
  • the users of the texts may have limited storage or display capabilities
  • the creator of the texts has limited or no ability to provide content expertise to analyze, tag, or review texts
  • extensibility is desired; that is, one desires to keep open the option for a higher level of encoding to be added at a later date

All elements specified in Levels 1 and 2, plus the following:


<front>, <back> Required if present.
<p> Required for paragraph breaks in prose.
<lg> and <l> Required for identifying stanzas and line breaks in verse.
<list> and <item> May be used in this level to indicate ordered and unordered list structures.
<table>, <row>, and <cell> May be used to indicate table structures.
<figure> Required to indicate figures other than page images.
<hi> Required to indicate changes in typeface; rend attribute is optional.
<note> All notes must be encoded. It is also recommended that notes that extend beyond one page be combined into one <note> element. Marginal notes, without reference, should occur at the beginning of the paragraph to which they refer, with the value of the place attribute as "margin".
<lb> May be used to force line breaks. REMOVE?


NOTE ON <note>:

It may be desirable to move footnotes from their original location in the text. If left at the bottom of a page, a note may become included in another paragraph or section of the encoded text, and thus separated from its reference. There are options for placement of footnotes if they are moved:

  • Inline. The note is inserted at the point of reference. An n attribute records the value of the note reference if there is one.
  • End-of-Division. Notes moved to the end of the division

Basic Structural Example forthcoming ...

LEVEL 4: Basic Content Analysis

Purpose: To create text that can stand alone as electronic text, identifies hierarchy and typography, specifies function of textual and structural elements, and describes the nature of the content and not merely its appearance. This level is not meant to encode or identify all structural, semantic or bibliographic features of the text.

Rationale: Greater description of function and content allows for:

  • flexibility of display and delivery
  • sophisticated searching within specified textual and structural elements
  • combining the broadest range of uses and audiences

Texts encoded at Level 4 are able to stand alone as part of a library collection, and do not require page images in order for them to be read by students, scholars and general readers. This level of TEI encoding allows them to be displayed or printed in a variety of ways suitable for classroom or scholarly use.

Level 4 texts contain elements and attributes that describe content. For example, lines of verse are tagged with <l>; the <p> element is reserved for true paragraphs. Features of the text that may contribute to meaning, such as indentation of verse lines and typographic change, are preserved. These are textual features that are not encoded at lower levels and that allow the text to be used and understood fully independent of images.

The ability to stand alone as text means that Level 4 texts are more nimble and robust for exercises such as format repurposing and textual analysis.

Finally, functionally accurate encoding in Level 4 texts allows them to be searched or displayed in sophisticated ways. For example, a searcher could limit his or her search in a dramatic text to stage directions or in a verse text to only first lines. In a politicall tract published by subscription, a search could be confined to names that appear in lists, thus limiting a search to names of people who subscribed to a particular volume. This ability to limit searches becomes more significant as textbases become larger, and thus is of great importance to the library community as it attempts to build into the initial design and implementation of textbases the features needed to enhance interoperability.

Level 4 is most suitable for projects with the following characteristics:

  • sophisticated search and retrieval capabilities are desired
  • the texts will be used for textual analysis
  • extensibility is desired; that is, one desires to keep open the option for a higher level of encoding to be added by the scholarly community at a later date
  • the users of the texts may have limited storage or display capabilities


General Level 4 Recommendations

  • Typographically distinct text should be encoded as appropriate, e.g. with <term>, <q>, <gloss>, <mentioned>, <soCalled>, <foreign>, <title>, or <emph>. Any ambiguous emphasized text should be encoded as <hi> (e.g. <hi rend="bold">).
  • It is recommended that the <sic> element be used to indicate typographic errors, with corrections (if any) noted as the value of the corr= attribute.
  • <titlePage> should include the verso if present, divided by <pb n="verso"/>. Tables of contents, errata, subscription lists, "other titles by the same author" should be included in a separate numbered division, as a <list> with <item>s. Frontispieces should be encoded as a <figure>, within a separate numbered division and <p>.

See some examples in context…

Level 4 Prose

  • Letters that occur within the text body provide some challenges. It is recommended that quoted letters that occur as part of a text (and not collections of letters themselves) be encoded within <q><text><body><div1 type="letter">, with <opener>, <dateline>, <salute>, <signed>, <closer> included as appropriate.
  • Quotations that do not occur inline, but are set off typographically in some way, should be encoded as <q>.
  • Notes are to be encoded as described in Level 3.
  • <argument>, <opener>, <epigraph>, <closer>, <trailer>, <add>, <del>, <unclear> as appropriate.

See some examples in context…

Level 4 Drama

  • Cast lists should be encoded as <list>s, with <item>s.
  • Speeches are encoded as <sp>, with speakers identified within <speaker> elements; stage directions are encoded as <stage> and enclose block level content describing scenery, etc.


See some examples in context…

Level 4 Oral History

  • Speakers in interviews can be identified in the <teiHeader> in several ways.
    • In the <profileDesc>, in the <particDesc> in a <list>, with <name> inside of <item>s.
    • As a list of author <name>s within <fileDesc><titleStmt>
  • In either method, use an xml:id= on the <name> element to uniquely identify the individual participant
  • Questions and answers from interviewees and interviewers are encoded as <sp>, with speakers identified within <speaker> elements with a who= attribute the value of which corresponds to the xml:id= in the list of interview participants.

See some examples in context…

Level 4 Verse

  • All verse, even poems without separate stanzas or verse paragraphs, should be contained within a line group element <lg>. This will assist with automated processing and retrieval.
  • It is common to see informal divisions within poems, noted by a string of asterisks or periods. These should be encoded as <milestone/>s with attribute values of unit="typography" and n="()" indicating the character used and its occurrence, <milestone unit="typography" n="******"/>.
  • <l> It is recommended that indentation be recorded and that the rend attribute be used to do this.

See some examples in context…

Level 4 Front and Back Matter

  • It is recommended that all prefaces, tables of contents, afterwords, appendices, endnotes and apparatus be encoded. For publisher's advertisements, indexes, and glossaries or other front or back matter that isn't considered of primary importance to the text, there are three options:
    • Fully transcribe and encode
    • Link to page images (may include an unencoded transcription)
    • Omit, noted in <editorialDesc>

See some examples in context…

LEVEL 5: Scholarly Encoding Projects

Level 5 texts are those that require subject knowledge, and encode semantic, linguistic, prosodic, or other elements beyond a basic structural level.

General Guidelines for Attribute Usage

Some general advice on the use of particular attributes follows.

  • type=: Constructing a list of acceptable attribute values for type that could find wide agreement is impossible. Instead, it is recommended that projects describe the type= attribute values used in their texts in the project ODD file or other documentation and that this list be made available to people using the texts. See ABC for Book Collectors by John Carter (7th edition, New Castle, DE:Oak Knoll Books, 1995) for a list of standard names and definitions of bibliographic features of printed books. For those elements where type is not required, such as <head> and <title>, use the attribute values for subtitles and additional titles, but not main titles.
    Example: <div1 type="volume">
  • n=: Sometimes an n= (number) attribute can be used by itself. For instance in the case of pagebreaks:
    Example: <pb n="456"/>
  • xml:id=: If you are in a situation that requires you to uniquely identify an element that will be used to reference another specific location in one or more texts, use an xml:id= attribute. The value of this attribute must be unique within a document, and must be composed of alphanumeric characters, dots, hyphens, and underscores, and must start with a letter.
    Example: <note xml:id="n5" n="5">
  • target=: follows the same syntactic rules as the xml:id= attribute value. In fact, target= and xml:id= are often used in conjunction with one another as in the case of footnotes where the <anchor xml:id="n5" n="5"> is at a specific place in the text and is referred to by the <note target="#n5" n="5"> which contains the actual information of the footnote itself elsewhere.
  • rend=: Difficulty using rend= attributes occurs when it is desirable to record more than one rendition feature. With this in mind, it is recommended that projects employ the following adaptation of "rendition ladders", a concept developed at the Brown University Women Writers Project. This system allows for sets of multiple renditional features to be included in one rend= value. Rendition ladders consist of categories of renditional features with values of each of those features enclosed in parentheses.
    rend= should only be used to override a default value. For instance, if all text encoded as <hi> is defined as being rendered in italics, there is no reason to encode text as <hi rend="font(italics)"> Combining renditional features would result in a tag with attributes such as <l rend="font(italics)align(right)">
    • font
      italics, bold, fsc (full and smallcaps), smallcap, underlined, gothic
    • align
      right, left, center, block
    • indent
      Values in parentheses should indicate the number of tabstops to be indented, e.g., <l rend="indent(1)">
  • lang=: Use ISO639-2 three-character language codes. Note that this recommendation is slightly different from that of the TEI P5 Guidelines, which recommends the BCP 47 language codes.

References: <references/>

Appendix A: History of this Document

The Text Encoding Initiative Guidelines for Electronic Text Encoding and Interchange (referred to as the TEI Guidelines) were first published in 1994 and represent a tremendous achievement in electronic text standards by providing a highly sophisticated structure for encoding electronic text. Digital librarians have benefited greatly from the standardization provided by these guidelines, and the potential for interoperability and long-term preservation of digital collections facilitated by their wide adoption.

In 1998, the Digital Library Federation (DLF) sponsored the TEI and XML in Digital Libraries Workshop at the Library of Congress to discuss the use of the TEI Guidelines in libraries for electronic text, and to create a set of best practices for librarians implementing them. From this workshop, three working groups were formed the members of which represented some of the largest and most mature digital library programs in the U.S.

Group 1 was charged to recommend some best practices for TEI header content and to review the relationship between the Text Encoding Initiative header and MARC. To this end, representatives of the University of Virginia Library and the University of Michigan Library gathered in Ann Arbor in early October 1998 to develop a recommended practice guide. This work was assisted by similar efforts that had taken place in the United Kingdom under the auspices of the Oxford Text Archive the previous year. The section on the header below is based on a draft of those recommended practices. It was submitted to various constituencies for comment. In mid-2008, Melanie Schlosser and Kevin Hawkins heavily revised this section for discussion by the TEI SIG on Libraries and the DLF TEI Task Force.

Group 2 was charged with developing a set of recommendations for libraries using the TEI Guidelines in electronic text encoding. This group included the following representatives from six libraries:

  • LeeEllen Friedland, Library of Congress
  • Nancy Kushigian, University of California, Davis
  • Christina Powell, University of Michigan
  • David Seaman, University of Virginia
  • Natalia Smith, University of North Carolina at Chapel Hill
  • Perry Willett, Indiana University (chair)

At the ALA mid-winter (January 1999), the DLF task force revised a draft set of best practices, called TEI Text Encoding in Libraries: Guidelines for Best Practices (referred to as TEI in Libraries Guidelines). The revised recommendations were circulated to the conference working group in May 1999 and presented at the joint annual meeting of the Association of Computers and the Humanities and Association of Literary and Linguistic Computing in June 1999. Version 1.0 was circulated for comments in August 1999. These guidelines were endorsed by the DLF, and have been used by many digital libraries, including those of the task force members, as a model for their own local best practices. Libraries, museums and end-users have benefitted from a set of best practices for electronic text in a number of ways, including better interoperability between electronic text collections, better documented practices among digital libraries, and a starting point for discussion of best practices with commercial publishers regarding electronic text creation.

Written in 1998, this first iteration of TEI in Libraries Guidelines made no mention of XML, XSLT, or any of the other powerful tools that have now become common parlance and practice in creating digital documents and collections. Based on these important changes in markup technology, it came to the attention of the DLF and members of the original Task Force that the TEI in Libraries Guidelines required substantial revision. In 2002, the TEI Consortium published a new edition of the complete TEI Guidelines that conformed to XML specifications. In order to remain useful, the TEI in Libraries Guidelines had to be updated to reflect these developments.

Furthermore, librarians need more guidance than the original TEI in Libraries Guidelines provided. There are many library-specific encoding issues which need to be addressed and documented to ensure consistency. The intention of this document is to provide recommended paths of encoding for these issues.

In addition, these library guidelines have the potential to be much more useful if they can serve as a training document from which librarians can learn about text encoding and addressing particular encoding challenges. To fulfill this role, the guidelines require more examples and detailed explanations, giving documentation of the use of TEI in a library context. Librarians also need a set of standards and best practices for vendors and publishers who create electronic text for digital libraries, so that these collections adhere to the same archival standards as locally-created electronic text collections. With detailed guidelines that could serve as an encoding specification, librarians might encourage vendors to follow the principles in these standards, to facilitate the long-term preservation of commercially published electronic text collections, and more readily allow for cross-collection searching.

In order to facilitate the evolution of this document, another DLF-sponsored Task Force—some of the representatives of which were on the original Task Force—met on October 24-25, 2003 at the Cosmos Club in Washington, D.C.:

  • Richard Gartner, Oxford University Library
  • Matthew Gibson, University of Virginia Library
  • Kirk Hastings, California Digital Library
  • Christina Powell, University of Michigan
  • Merrilee Proffitt, RLG
  • David Seaman, Digital Library Federation
  • Natalia Smith, University of North Carolina at Chapel Hill
  • Perry Willett, Indiana University (chair)

These representatives met to revise the original TEI in Libraries Guidelines in order that they:

  • reflect changes occuring within the text encoding world generally and within the TEI community specifically
  • further illuminate the different levels of encoding by offering clearer and more robust examples.

After producing Version 2.0 of the Guidelines, this group (with some changes in membership) met again at the Cosmos Club on February 13-14, 2006. Those in attendance were:

  • Syd Bauman, The TEI Consortium
  • Richard Gartner, Oxford University Library (by phone)
  • Matthew Gibson, Virginia Foundation for the Humanities (chair)
  • Merrilee Proffitt, RLG
  • Chris Powell, The University of Michigan
  • David Seaman, Digital Library Federation
  • Natasha Smith, University of North Carolina at Chapel Hill
  • Perry Willett, The University of Michigan
Retrieved from "https://wiki.tei-c.org/index.php?title=Best_Practices_for_TEI_in_Libraries&oldid=5077"