These Conformance tests are implemented using the JavaScript class xbTestFrame. xbTestFrame is designed to test hierarchal API's defined in JavaScript. These tests can be run either with Netscape 6, a recent build of Mozilla or Internet Explorer 5.5.
The test frame represents the API to be tested as a hierarchal collection of names with possible tests defined
for each name. When testing, the test frame will first evaluate the API Name to see if it exists. If the API Name
does not exist, all tests are skipped for that API Name. If the API Name does exist, a Pre-Condition
expression is evaluated for each test to see if the context required by the test is met. If the
Pre-Condition is met, the individual test is performed. An example of where a Pre-Condition is used
in the DOM Core Level 2 Test is when testing the attributes property of a Node.
attributes is a NamedNodeMap, but is only non-null for Element Nodes. Therefore, a
Pre-Condition for applying the NamedNodeMap tests to an attributes property is that the Node
must be an Element Node.
In the reports, Test results are color coded as passed all tests, failed some tests, failed all tests, and not tested.
The API style of test report consists of two sections:
The Report By Test Case lists each Test Case performed and provides a link to display all of the steps during the setup and execution of the Test Case. Additionally, a link to a report showing the state of the objects used during the test is available.
The Report by API Name lists the names defined in the API in the left frame. If an API Name has tests associated
with it, the left hand column of the table will appear with a plus sign +. Clicking on the plus sign
will change the + to a - and will display the tests for that API Name as well as for any
of it's descendant API names. To view an individual report, click on it's link. To hide the tests for an API Name,
click on the - sign.
The pass/fail/skipped totals for each API name are arranged in a hierarchal fashion. The grand totals can be found in the first row of the Report by API Name table. For actual API Names, the totals include any tests directly performed for that API name as well as any descendant API names. For example, the totals for Document will also include the totals for Document.nodeName and so on.
Some interfaces may exist in the API but are never instantiated. In the DOM Core, CharacterData is such an interface. These interfaces will show the passed/failed/skipped counts for all of the interfaces that are descendants but will not have test reports associated with them.
The Current Test Reports over count the total number of tests reported at the highest level of the report. This is due to the populating of ancestor API Name's test counts with counts from their descendants. For example, a test on a Comment will contribute to the totals for CharacterData. This works as expected. The problem is that the totals for CharacterData will contribute to the grand totals which results in their being over counted. The overall percentage of the passed vs. failed tests is not changed however. This problem will be resolved as soon as possible.
You may inspect the Source code used in the test frame at xbTestFrame. We will be adding more documentation on how the test frame is implemented in the future. We invite everyone to become involved in helping us make these tests the most definitive possible.