Internationalization Best Practices: Handling Right-to-left Scripts in XHTML and HTML Content

How to: Attributes in HTML 4.01 that have values of right and left are align and clear.

align is used with the elements hr, div, hX, p, col, colgroup, tbody, td, tfoot, th, thead and tr. clear is used with br.

For example, to right align a paragraph you could use the CSS rule in Example 6:

(Note that this best practice does not refer to the values rtl and ltr that are used with the dir attribute.)

Discussion: Values of right and left in attributes need to be converted when translating the document into a language using the Arabic or Hebrew scripts. It can save a lot of time and risk to use CSS style sheets to achieve the same effect. One should expect the style sheet to be converted as part of the translation process.

How to: Use the dir attribute when you need to indicate directionality. For more information about using this attribute see the rest of this document.

Discussion: Because directionality is an integral, and needs to be a persistent part of the document structure, markup should always be used to set the directionality for a document or chunk of information, or to indicate places in the text where the Unicode bidi algorithm is insufficient to achieve desired inline directionality.

For XHTML or HTML served as text/html (ie. treated as HTML by the user agent), the expected behavior of the dir attribute and its values is clearly defined in the HTML specification, so CSS is not needed. The CSS2 specification recommends the use of markup for bidi text in HTML. In fact it goes as far as to say that conforming HTML user agents may ignore CSS bidi properties. This is because the HTML specification clearly defines the expected behavior of user agents with respect to the bidi markup.

See CSS vs. markup for bidi support for a fuller explanation.

How to: Use a separate link element to link to a style sheet containing only CSS statements to support the dir attribute.

Example 7 shows part of an XHTML 1.1 document head where the style sheet bidi.css contains all the properties needed to control the directionality set by the dir attribute values:

Example 8 shows what the file bidi.css might contain:

Discussion: XHTML served as application/xhtml+xml, application/xml or text/xml is treated by user agents as XML, not HTML.

Normally a user agent will not automatically recognize or know what to do with any bidi markup you use in XML documents. CSS properties should therefore be used to indicate the expected visual behavior of text in your document. (Note that this is not the case for documents served as HTML, see Best Practice Best Practice 2: Do not use CSS styling for HTML.)

Keeping the relevant CSS in a separate style sheet is not essential, but it may help in a number of ways:

• this single style sheet can be used with all the other style sheets on your site

• the text is less likely to be accidentally changed or omitted when changes are made to the style sheet or a new style sheet is created

• if the user turns off style, switches styles or overrides the styling, it is sometimes possible to do so selectively - in which case the bidi styling can be left alone.

The CSS, however, should always be linked to dedicated bidi markup in the text.

See CSS vs. markup for bidi support for a fuller explanation.

How to: Add dir="rtl" to the html tag any time the overall document direction is right-to-left.

This will cause block elements and table columns to start on the right and flow from right to left. All block elements in the document will inherit this setting unless it is explicitly overridden.

No dir attribute is needed for documents that have a base directionality of ltr, since this is the default.

Having established the directionality at level of the html tag, you should not use the dir attribute on other elements unless you want to change the directionality for that element. Unnecessary use of the dir attribute impacts bandwidth and potentially creates unnecessary additional work for page maintenance (see Best Practice Best Practice 7: Use bidi markup only when necessary.).

Discussion: Note that in Internet Explorer, applying a right-to-left direction in the html or body tag will affect the user interface, too.

The scroll bar will appear to the left side of the window, and JavaScript alert message boxes such as the one shown on the slide will be mirror imaged (see the tests). (Note how the yellow icon on the slide appears on the right, and the logical order of the text, <arabic> W3C <hebrew>, is displayed from right to left.) This behavior does not occur in other browsers.

Some speakers of languages that use right-to-left scripts prefer the directionality of the user interface to be associated with the desktop environment, not with the content of a particular document. Because of this, they may prefer not to declare the document directionality on the html or body tag. To avoid this without tagging every block element in the document you could add a div element immediately inside the body element that surrounds all the other content in the document, and apply the dir attribute to that. The directionality will then be inherited by all other block elements in the body of the document, but will not set off the changes to the browser. If you do this, you must ensure that you add a dir attribute to the head element also, to cover its title element, attribute values, etc.

UA issues: The following summarizes support for this feature in the user agents tested for this document at the time of writing. See the test results page for more details and latest results.

In Internet Explorer adding the dir attribute to the html tag also moves the scroll bar to the left of the browser window. See the discussion immediately above for the implications of this.

According to the Microsoft article Authoring HTML for Middle Eastern Content, the following behaviors can only be expected in Internet Explorer 5 if the dir attribute is on the html element, rather than the body element.

• The OLE/COM ambient property of the document is set to AMBIENT_RIGHTTOLEFT

• The document direction can be toggled through the document object model (DOM) (document.direction="ltr/rtl")

• An HTML Dialog will get the correct extended windows styles set so it displays as a RTL dialog on a Bidi enabled system.

• If the document has vertical scrollbars, they will be used on the left side if dir="rtl".

How to: Create and store your Hebrew content in logical order (ie. usually as you would pronounce it), not the order you expect to see it displayed. It is usually best to use an Unicode encoding, such as UTF-8. If, for some reason, you choose to serve your Hebrew page in an ISO encoding instead, then specify ISO-8859-8-I, not ISO-8859-8.

Discussion: 'Visual ordering' of text was common for old user agents that didn't support the Unicode bidirectional algorithm. Text was stored in the source code in the same order you would expect to see it displayed. This also involved such things as disabling any line wrapping, explicit right-alignment of text in paragraphs and table cells, and reverse-ordering of table columns when translating from English to a language using a bidi script. For example, if you want to add a few words in the middle of a paragraph, you would have to move text to and from every line that followed it in the paragraph (see the tutorial Creating (X)HTML Pages in Arabic & Hebrew for an example).

Note, too, that if you have in-line markup, such as emphasis or link text, that spans more than one line, you will need to mark up the text runs on both lines separately. Again, adding text before such markup in a paragraph would mean that you have to carefully change this markup to reflect the new position of the text.

The result is very fragile code that is difficult to maintain. In addition, all the extra tags needed to manage the text would bloat your code and impact not only authoring time, but also bandwidth. Visually ordered bidirectional HTML does not conform to the HTML specification.

Using logically ordered text, on the other hand, makes it almost trivial to create long paragraphs of flowing text that automatically wrap to the width of the block element. It also makes it much easier to address accessibility, using such things as screen readers.

With 'logical ordering' text is stored in memory in the order in which it would normally be typed (and usually pronounced). The Unicode bidirectional algorithm is then applied by the browser to render the correct visual display.

Certain encodings are associated with visual vs. logical ordering of text. Text in a Unicode encoding, such as UTF-8, is always logical.

According to RFC1555 and RFC1556, there are special conventions for the use of charset parameter values to indicate bidirectional treatment in MIME mail, in particular to distinguish between visual, implicit, and explicit directionality. 'Visual' refers to the practice of typing in the Hebrew characters in reverse order and preventing automatic line breaks. Formatting the document visually in this way is typically done to ensure reasonable display on older user agents that do not handle bidirectionality. Such documents do not conform to the HTML specification. 'Implicit' is also called logical ordering, and refers to an approach where all characters in memory in the order in which it would normally be typed. Correct ordering for display is then done by a special algorithm (this is the preferred approach). 'Explicit' refers to the use of explicit markers in the text to indicate directional changes.

The charset parameter value ISO-8859-8 for Hebrew denotes visual ordering, ISO-8859-8-I denotes implicit bidirectionality, and ISO-8859-8-e denotes explicit directionality.

Because HTML uses the Unicode bidirectional algorithm, conforming documents encoded using ISO 8859-8 must be labeled as ISO-8859-8-I. Explicit directional control is also possible with HTML, but cannot be expressed with ISO 8859-8, so "ISO-8859-8-e" should not be used.

Contrary to what is said in RFC1555 and RFC1556, ISO-8859-6 (Arabic) is not visual ordering.

Note, also, that ISO encodings don't include diacritics - if you want these use a logical encoding such as a Unicode encoding or Windows-1255.

How to: Add the dir attribute to a block level element where you want to change its directionality. Example 10 shows how you might mark up a blockquote element to render a left-aligned English quote in a right-to-left page.

If you want to align a table to the left or right of a page, rather than just change the order of its columns and contents, you need to put the table in a div element, and add the dir attribute to that, rather that put the attribute on the table element.

Note also that you should only use the dir attribute on block elements when you need to change the directionality from the current default (see Best Practice Best Practice 7: Use bidi markup only when necessary.).

Discussion: The following example illustrates the effect of applying a change in directionality to a block level element using the dir attribute.

Example 11 inherits the LTR directionality of this page, and its source contains some Hebrew text, followed by punctuation, followed by a graphic.

Example 12 is exactly the same code, but with an explicit dir="rtl" added to the paragraph tag to turn this into a right-to-left paragraph embedded in this left-to-right page.

Note, in particular, that the positions of the image and punctuation in the example above change relative to the text, because the overall directional flow has been changed. Note also, however, that the Hebrew characters are still read in the same direction. Their sequence is determined by the Unicode bidirectional algorithm, not by the dir attribute.

The content of all nested block elements will inherit directionality (unless of course a nested element explicitly changes its directionality using dir). Remember that the base directionality for a document should already be established by the html element. There is no need to add dir attributes to block level elements unless you want to apply a different direction to that set by the html tag or an explicit setting on a parent block element.

Visual user agents that support bidirectional display will typically right-align block elements in a rtl context, and vice versa. (See the example above.)

The dir attribute setting also affects the flow of columns in a table.

The table element in Example 13 has a dir attribute set to rtl.

Example 14 shows the same table element with the dir attribute removed. The directionality of the columns is now set by the next ancestor element that specifies directionality - in this case the default ltr setting of the html tag of this document.

Note how the cells inherit the directionality set for the table. This produces the alignment of text in the cell, the order of text relative to the number, and the position of the question mark.

Note also that in most browsers, unlike other block elements, adding a dir attribute to the table will not cause the table to be aligned differently. It will only affect the order of columns and table content. If you want the table to be aligned with the other side of the content area you will need to wrap the table in another block element (eg. a div) that carries a dir attribute.

UA issues: [Ed. note: dir causes problem on DT element in IE6]

How to: Once you have established the appropriate directionality for the html element you will only need to apply bidi markup to a block element if you want that element's directionality to be different. The same applies for inline markup. Do not use inline bidi markup unless the Unicode bidi algorithm is insufficient on its own.

The following Arabic example shows bad usage. None of the dir attributes are needed if dir="rtl" was added to the html element. Removing them will significantly simplify the document and reduce bandwidth requirements.

The Unicode Bidirectional Algorithm is applied to text that is stored in logical order, and determines the appropriate display direction of a sequence of characters. It does this on the basis of semantics associated with those characters by the Unicode Standard.

The following Arabic text in example 16 contains the number 1996 that runs left to right within the overall right to left flow of the Arabic letters. No special markup or styling is needed to achieve this. The bidirectional algorithm alone is enough.

Occasionally the Unicode bidirectional algorithm is not sufficient to correctly order chunks of embedded text. Alternatively, you may want to override the effects of the bidirectional algorithm for a part of the page. In these cases you can apply additional markup to produce the ordering you want.

How to: Where punctuation or other 'neutral characters' occurring between LTR and RTL text do not display in the expected location, place a RIGHT-TO-LEFT MARK (Unicode character U+200F) or a LEFT-TO-RIGHT MARK (Unicode character U+200E) alongside the misplaced character or characters to produce the desired result. (For explanations with examples see the discussion below.)

These characters can be added as characters or as escapes. (But see the issues associated with escapes at Section 3.2: Adding escapes to the content.)

Discussion: You need to be familiar with the concepts in What you need to know about the bidi algorithm and inline markup to understand this best practice.

Unfortunately, the bidirectional algorithm may not always produce the desired result with regard to the placement of punctuation. For instance, the overall context of the example below is LTR. If we introduce some punctuation between the Arabic and Latin letters it will produce the (undesirable) result in Example 17.

The exclamation mark is part of the Arabic phrase and should have appeared to its left. It appears to the right because it is between an Arabic and Latin character and the overall paragraph direction is LTR. It is therefore treated as part of the English text.

An easy way to fix this is to insert the Unicode character U+200F, called the RIGHT-TO-LEFT MARK (RLM), after the exclamation mark.

Now with two strong RTL characters on either side, the exclamation mark too will be treated as part of the RTL directional run and we will get the following (correct) result.

You could encounter a similar problem in an Arabic paragraph that included English text followed by an exclamation mark. There is a similar character, U+200E, called the LEFT-TO-RIGHT MARK (LRM) which would solve the problem in this case.

How to: This best practice is relevant when the placement of some punctuation or other neutral characters in between two runs of text with the same directionality causes the text to be incorrectly ordered. (For explanations with examples, see the discussion below.)

If the overall directional context is right-to-left, place a RIGHT-TO-LEFT MARK (Unicode character U+200F) alongside the neutral characters to produce the desired result. Otherwise use a LEFT-TO-RIGHT MARK (Unicode character U+200E).

These characters can be added as characters or as escapes. (But see the issues associated with escapes at Section 3.2: Adding escapes to the content.)

Note that the dir attribute is not appropriate to resolve this case.

Discussion: You need to be familiar with the concepts in What you need to know about the bidi algorithm and inline markup to understand this best practice.

In Example 19 there is a comma between two of the state names written in Arabic script. With a strongly typed right-to-left (RTL) character on either side, the bidirectional algorithm sees the neutral comma as part of the Arabic text. Therefore, the first two Arabic words are treated as a single directional run, going right to left. This leads to a reading sequence of:

1. (ltr) The names of these states in Arabic are

2. (rtl) مصر, البحرين [nirHbla ,rSm]

3. (ltr) and

4. (rtl) الكويت [tiukla]

5. (ltr) respectively.

The comma, however, is really part of the English text. So what is wanted is this order, as shown in Example 20:

1. (ltr) The names of these states in Arabic are

2. (rtl)مصر [rSm]

3. (ltr) ,

4. (rtl) البحرين [nirHbla]

5. (ltr) and

6. (rtl) الكويت [tiukla]

7. (ltr) respectively.

The correct result was obtained by simply placing a LEFT-TO-RIGHT MARK (Unicode character U+200E) immediately after the comma. This has the effect of placing the neutral comma between two strongly typed characters, one left-to-right and the other right-to-left. Because neutral characters in this position take on the directionality of the overall context (here that of the paragraph), the bidi algorithm will now see it as part of the English left-to-right flow and will see the two Arabic words as separate.

In Example 21, this time in a right-to-left Hebrew paragraph, the beginning (ie. the right hand side) of the sentence looks a real mess. This is because the text from "W3C" to "Consortium" is seen as a single directional run of LTR characters. The second parenthesis from the right falls between LTR and RTL characters, so assumes the directionality of the paragraph - RTL.

It is very simple to obtain the correct result. Simply put a RIGHT-TO-LEFT MARK (Unicode character U+200F) immediately after the hyphen. This causes the hyphen and the nearby parenthesis to be seen as part of the paragraph's text flow, as shown in Example 22.

It is probably worth adding one more example to this discussion. Where lists are composed from separate items, separated by punctuation or spaces, you may also need to think of adding an RLM or LRM.

The list of translated versions at the top right of the Hebrew page in Example 23 needs to read in the order Arabic, English and French (from right to left). On the English page this list is created by simply concatenating the names of the language in the desired order. Do that on the Hebrew or Arabic pages, however, and the order of display will be Arabic, French, English - since the bidirectional algorithm treats the two Latin words as a single directional run. The PHP code that produces the list needs to insert an RLM character between each Latin language name in order to achieve the appropriate ordering. Note how this requires a change to the logic of the script in this case.

How to: Add the dir attribute to an element surrounding the embedded text. If there is no element surrounding the text, use a span element.

Set the value of the dir attribute to either ltr or rtl, depending on the main direction of the embedded text.

For examples and more detailed explanations see the discussion that follows.

Discussion: You need to be familiar with the concepts in What you need to know about the bidi algorithm and inline markup to understand this best practice.

This best practice is particularly useful where embedded text, such as a quote, is bidirectional. At a simple level the Unicode bidirectional algorithm takes care of the reordering of inline text, but where there is nesting of directionality the dir attribute may need to be used.

The Unicode bidirectional algorithm organizes characters into directional runs - sequences of characters with the same directionality. Directionally neutral characters such as spaces and punctuation take on the directionality of surrounding characters, allowing directional runs to span several words. In Example 24 there are three directional runs - English, Arabic, and English. These are ordered according to the prevailing directionality of the paragraph - in this case left-to-right.

Unfortunately, the bidirectional algorithm alone does not produce the desired result if one of the directional runs contains mixed direction text, as can be seen in the following example.

The incorrect line of text in Example 25 is coded as a simple sequence of characters without any inline markup. Note that the order of the two Hebrew words is correct, but the text "W3C" should appear on the left hand side of the quotation and the comma should appear between the Hebrew text and "W3C", as shown in Example 26.

To get the correct result we have to create a new 'embedding level' by surrounding the text within the quote marks with a span element and setting its dir attribute to rtl as shown in Example 27. (The language attributes have been omitted to make the example clearer. See also the issues associated with representing source text at Section 3.3: Example source text in this document)

This causes the comma to take on the same RLT directionality as the whole span, and orders the Hebrew directional runs appropriately.

Note that we have used a span element to carry the dir attribute in this case. If the quote had already been surrounded by an element, the dir attribute should be attached to that. A span element should only be used where there is nothing else available.

Note also that we placed the span element inside the quotation marks, since these are a part of the English text.

How to: In (X)HTML do not use the Unicode bidi formatting code characters where markup is available. To show the limits of embedded text with a different default direction, use the dir attribute, and to override the bidirectional algorithm use the bdo element.

On the other hand, attribute text and element text that allows no internal markup, ie. the title, textarea and option elements, cannot support use of dir on a span or other element to label part of its content.

In these cases you can use Unicode characters to do the same job. The following table shows correspondences between markup and Unicode control codes:

These characters can be added as characters or as escapes. (But see the issues associated with escapes at Section 3.2: Adding escapes to the content.)

Discussion: The HTML 4 specification specifically warns against mixing the two approaches because of the increased likelihood of improper nesting. It also recommends the use of markup because it "offers a better guarantee of document structural integrity and alleviates some problems when editing bidirectional HTML text with a simple text editor". It does not proscribe the use of Unicode bidi formatting codes.

The joint Unicode Technical Report #20 and W3C Note, Unicode in XML and other Markup Languages goes further. It explicitly recommends that only the markup be used. It also recommends that the Unicode bidi formatting codes should be ignored if detected in a browser context, and replaced by appropriate markup when received in an editing context.

Of course, in attribute values or for the three elements listed above markup cannot be used, so the Unicode control characters are the only option available.

[Ed. note: Summarize and point to the bidi space Q&A]

The shape of the glyphs used for a pair of mirrored characters will be determined at run time according to the directional context in which they appear.

bdo stands for 'bidirectional override'. This inline element can be used to override the Unicode bidirectional algorithm if the dir attribute doesn't produce the desired result or if you want to produce a different result.

You can illustrate the order of characters as stored in memory in examples by simply applying a bdo tag to the example text. This causes the characters to flow in one direction, regardless of the directionality of the characters involved.

For instance, in the normal case the Unicode bidirectional algorithm would operate on Hebrew text to produce the result shown in example 28. Note how the characters in the Hebrew words are read right to left.

If, however, you wanted to produce an example showing the sequence in which characters are stored in the computer's memory such as in example 29 ...

... you can do so using the following underlying code