Basic XML Parsing via XMLTable

In Basic XML Parsing via PL/SQL, I looked at "How do I parse this XML?" when the XML resides in PL/SQL. Oftentimes, the XML resides in the database and so Oracle provides another option for parsing the XML, the XMLTable command. This command became available with the release of 10.2. On versions <= 10.1, you have to use TABLE(xmlsequence(extract(...))). I might cover that at some point in the future but will see. While reading that documentation may make XMLTable seem complex, which it can be, it is very easy to use. Let us start with putting our XML into the DB via:

Connected to Oracle Database 11g Enterprise Edition Release 11.1.0.6.0

SQL>
SQL> CREATE TABLE your_table
 2  (xml_col   XMLTYPE);

Table created

SQL>
SQL> INSERT INTO your_table
 2  VALUES
 3  ('<employees>
 4   <emp>
 5    <name>Scott</name>
 6    <favorites>
 7     <color>red</color>
 8     <color>orange</color>
 9    </favorites>
10   </emp>
11   <emp>
12    <name>John</name>
13    <favorites>
14     <color>blue</color>
15     <color>green</color>
16    </favorites>
17   </emp>
18  </employees>');

1 row inserted

SQL> commit;

Commit complete

Now let's get out the same data we did in the original PL/SQL example. For this we will only be doing Methods 3 - 5 since they cover Methods 1 and 2 as well.

SQL> -- Method 3
SQL> SELECT xt.nam
 2    FROM your_table yt,
 3         XMLTable('employees/emp'
 4                  PASSING yt.xml_col  -- defines source of XMLType, can define a join
 5                  COLUMNS
 6                  nam  VARCHAR2(20) PATH 'name') xt;

NAM
--------------------
Scott
John

SQL> --
SQL> -- Method 4
SQL> SELECT xt.nam
 2    FROM your_table yt,
 3         XMLTable('employees/emp[2]'
 4                  PASSING yt.xml_col
 5                  COLUMNS
 6                  nam  VARCHAR2(20) PATH 'name') xt;

NAM
--------------------
John

SQL> --
SQL> -- Method 5.a
SQL> SELECT xt.color
 2    FROM your_table yt,
 3         XMLTable('employees/emp/favorites/color'
 4                  PASSING yt.xml_col
 5                  COLUMNS
 6                  color  VARCHAR2(10) PATH '.') xt;

COLOR
----------
red
orange
blue
green

SQL> --
SQL> -- Method 5.b
SQL> SELECT xt.nam, xt2.color
 2    FROM your_table yt,
 3         XMLTable('employees/emp'
 4                  PASSING yt.xml_col
 5                  COLUMNS
 6                  nam      VARCHAR2(20) PATH 'name',
 7                  color_t  XMLTYPE      PATH 'favorites') xt,  -- path to the node that repeats
 8         XMLTable('favorites/color'
 9                  PASSING xt.color_t  -- define input XMLType as output of above, aka a join
10                  COLUMNS
11                  color  VARCHAR2(10) PATH '.') xt2;

NAM                  COLOR
-------------------- ----------
Scott                red
Scott                orange
John                 blue
John                 green

SQL> --
SQL> -- Method 5.c
SQL> SELECT xt.nam, xt2.color
 2    FROM your_table yt,
 3         XMLTable('employees/emp'
 4                  PASSING yt.xml_col
 5                  COLUMNS
 6                  nam      VARCHAR2(20) PATH 'name',
 7                  color_t  XMLTYPE      PATH 'favorites/color') xt,  -- path to the node that repeats
 8         XMLTable('/color'
 9                  PASSING xt.color_t  -- define input XMLType as output of above, aka a join
10                  COLUMNS
11                  color  VARCHAR2(10) PATH '.') xt2;

NAM                  COLOR
-------------------- ----------
Scott                red
Scott                orange
John                 blue
John                 green

Methods 3 - 5.a here extract the same information as we did via PL/SQL.

Breaking down the Oracle documentation syntax, I will cover XMLnamespaces_clause in another post. The XQuery_string can be a simple XPath as shown above or more complex XQuery statements to be demonstrated later. The XML_passing_clause is just the PASSING line where the input XML is defined. This can join the XMLTable to another table/XMLTable, a variable, or be XML generated right there. The XML_table_column section following COLUMNS simply lists how to find the information referenced by the XQuery_string. The information is relative to the results of the XQuery_string.

Trouble shooting tips:

• If you are unsure of what the XMLType looks like at some point within an XMLTable, simply create a column of XMLType and include it in the SELECT list. In example 5.c, adding xt.color_t to the select list allows us to see what Oracle was passing into the second XMLTable.

Oracle Documentation

One thing I have noticed while being on the OTN forums is that not everyone knows about the online documentation for the version of Oracle they are using. Thankfully, Oracle has the handy Oracle Documentation to start from.

XML Parsing with Default Namespaces via PL/SQL

In my previous post I covered parsing of XML where a namespace prefix existed for each namespace. What happens when you have to parse XML where some of the XML is in a default namespace? Make up a namespace prefix. No W3C complaint parser will care about what the prefix is in the XML and the prefix in the XPath is as long as the URI (the stuff within the " quotes) is the same for the node in question. Here is a simple example based off the ongoing example

DECLARE
  l_doc       VARCHAR2(2000);
  l_domdoc    dbms_xmldom.DOMDocument;
  l_nodelist  dbms_xmldom.DOMNodeList;
  l_node      dbms_xmldom.DOMNode;
  l_value     VARCHAR2(30);
  l_a_ns      VARCHAR2(30);
  l_both_ns   VARCHAR2(60);

  l_xmltype   XMLTYPE;
  l_empx      XMLTYPE;
  l_index     PLS_INTEGER;
  l_col_ind   PLS_INTEGER;
BEGIN
  l_doc := '<employees xmlns="abc.com/123" xmlns:b="xyz.net/456">
 <emp>
  <name>Scott</name>
  <b:favorites>
   <b:color>red</b:color>
   <b:color>orange</b:color>
  </b:favorites>
 </emp>
 <emp>
  <name>John</name>
  <b:favorites>
   <b:color>blue</b:color>
   <b:color>green</b:color>
  </b:favorites>
 </emp>
</employees>';

  l_a_ns := 'xmlns:a="abc.com/123"';  -- assign random prefix to namespace
  -- assigning a different prefix to this namespace
  l_both_ns := l_a_ns || ' xmlns:c="xyz.net/456"'; -- separated by a space

  l_xmltype := XMLTYPE(l_doc);

  -- Method 4
  dbms_output.new_line;
  dbms_output.put_line('Method 4');
  l_value := l_xmltype.extract('/employees/emp[2]/name/text()', 'xmlns="abc.com/123"').getStringVal();
  dbms_output.put_line('Emp Name: '||l_value);

  -- Method 5
  dbms_output.new_line;
  dbms_output.put_line('Method 5');
  l_index := 1;
  WHILE l_xmltype.Existsnode('/a:employees/a:emp[' || To_Char(l_index) || ']', l_a_ns) > 0
  LOOP
   l_empx := l_xmltype.extract('/a:employees/a:emp[' || To_Char(l_index) || ']', l_a_ns);
   l_col_ind := 1;
   WHILE l_empx.Existsnode('/a:emp/c:favorites/c:color[' || To_Char(l_col_ind) || ']', l_both_ns) > 0
   LOOP
      l_value := l_empx.extract('/a:emp/c:favorites/c:color[' || To_Char(l_col_ind) || ']/text()', l_both_ns).getStringVal();
      dbms_output.put_line('Color: '||l_value);
      l_col_ind := l_col_ind + 1;
   END LOOP;
   l_index := l_index + 1;
  END LOOP;
END;

In the XML, we have a default namespace of "abc.com/123" for the elements and the previous "xyz.net/456" namespace. In the code, I randomly picked a prefix of "a" for the default namespace and so anytime an element in the default namespace is referenced, the corresponding prefix is used.

As Method 4 shows, you could just provide the default namespace as is for the third parm and then there is no need to include a prefix in the XPath itself. I prefer to go the route of method 5 as I have encountered XML where the "default" namespace has changed several nodes deep in the XML. Since the default changes, you need to specify a namespace prefix to keep things in order with the namespace each node resides in.

XML Parsing with Namespaces via PL/SQL

My previous post covered the basics for parsing simple XML in PL/SQL, but most XML has one or more namespaces associated with it. So how do you deal with that? Simple pretty much. Oracle provides a third parm (usually it is the third) to pass namespace URI information along to the underlying parser. I've taken the previous XML and added two namespaces to it and updated the appropriate calls to make this example.

DECLARE
l_doc       VARCHAR2(2000);
l_domdoc    dbms_xmldom.DOMDocument;
l_nodelist  dbms_xmldom.DOMNodeList;
l_node      dbms_xmldom.DOMNode;
l_value     VARCHAR2(30);
l_a_ns      VARCHAR2(30);
l_both_ns   VARCHAR2(60);

l_xmltype   XMLTYPE;
l_empx      XMLTYPE;
l_index     PLS_INTEGER;
l_col_ind   PLS_INTEGER;
BEGIN
l_doc := '<a:employees xmlns:a="abc.com/123" xmlns:b="xyz.net/456">
 <a:emp>
  <a:name>Scott</a:name>
  <b:favorites>
   <b:color>red</b:color>
   <b:color>orange</b:color>
  </b:favorites>
 </a:emp>
 <a:emp>
  <a:name>John</a:name>
  <b:favorites>
   <b:color>blue</b:color>
   <b:color>green</b:color>
  </b:favorites>
 </a:emp>
</a:employees>';

l_domdoc   := dbms_xmldom.newDomDocument(l_doc);
l_a_ns := 'xmlns:a="abc.com/123"';
l_both_ns := l_a_ns || ' xmlns:b="xyz.net/456"'; -- separated by a space

-- Method 1
dbms_output.put_line('Method 1');
-- third parm for namespace info
l_nodelist := dbms_xslprocessor.selectNodes(dbms_xmldom.makeNode(l_domdoc),'/a:employees/a:emp/a:name', l_a_ns);
FOR cur_emp IN 0 .. dbms_xmldom.getLength(l_nodelist) - 1 LOOP
 l_node := dbms_xmldom.item(l_nodelist, cur_emp);
 l_value := dbms_xmldom.getnodevalue(dbms_xmldom.getfirstchild(l_node));
 dbms_output.put_line('Emp Name: '||l_value);
END LOOP;

-- Method 2
dbms_output.new_line;
dbms_output.put_line('Method 2');
l_nodelist  := dbms_xmldom.getelementsbytagname(l_domdoc, 'name');  -- This doesn't care about namespaces
-- get first item from list, could loop as shown above
l_node      := dbms_xmldom.item(l_nodelist, 0);
l_value    := dbms_xmldom.getnodevalue(dbms_xmldom.getfirstchild(l_node));
dbms_output.put_line('Emp Name: '||l_value);

-- Done with DOMDocument examples, setup for XMLType based examples
dbms_xmldom.freeDocument(l_domdoc);
l_xmltype := XMLTYPE(l_doc);

-- Method 3
dbms_output.new_line;
dbms_output.put_line('Method 3');
l_index := 1;
WHILE l_xmltype.Existsnode('/a:employees/a:emp[' || To_Char(l_index) || ']', l_a_ns) > 0
LOOP
 l_value := l_xmltype.extract('/a:employees/a:emp[' || To_Char(l_index) || ']/a:name/text()', l_a_ns).getStringVal();
 dbms_output.put_line('Emp Name: '||l_value);
 l_index := l_index + 1;
END LOOP;

-- Method 4
dbms_output.new_line;
dbms_output.put_line('Method 4');
l_value := l_xmltype.extract('/a:employees/a:emp[2]/a:name/text()', l_a_ns).getStringVal();
dbms_output.put_line('Emp Name: '||l_value);

-- Method 5
dbms_output.new_line;
dbms_output.put_line('Method 5');
l_index := 1;
WHILE l_xmltype.Existsnode('/a:employees/a:emp[' || To_Char(l_index) || ']', l_a_ns) > 0
LOOP
 l_empx := l_xmltype.extract('/a:employees/a:emp[' || To_Char(l_index) || ']', l_a_ns);
 l_col_ind := 1;
 WHILE l_empx.Existsnode('/a:emp/b:favorites/b:color[' || To_Char(l_col_ind) || ']', l_both_ns) > 0
 LOOP
    l_value := l_empx.extract('/a:emp/b:favorites/b:color[' || To_Char(l_col_ind) || ']/text()', l_both_ns).getStringVal();
    dbms_output.put_line('Color: '||l_value);
    l_col_ind := l_col_ind + 1;
 END LOOP;
 l_index := l_index + 1;
END LOOP;
END;

Running this script produces

Method 1
Emp Name: Scott
Emp Name: John
 
Method 2
Emp Name: Scott
 
Method 3
Emp Name: Scott
Emp Name: John
 
Method 4
Emp Name: John
 
Method 5
Color: red
Color: orange
Color: blue
Color: green

which is exactly the same output as the previous example. This is what we expected. The only call that did not have to change was the one for Method 2. DBMS_XMLDOM is pretty much ignores namespaces when performing operations so this does have useful situations. As Methods 1, 3, and 4 show, the namespace parm only needs to contain the URI information for namespaces referenced in the Xpath, not in the XML. All the namespaces information can be passed along, but it is not needed. Pretty simple, excluding the whole "wait, what namespace is that node in again?" issue.

Basic XML Parsing via PL/SQL

One question that comes up with some frequency on various OTN forums is "How do I parse this XML?" Depending upon where the XML is (PL/SQL or DB) and what version of Oracle, there are different options that can be used. In starting simple, I show some examples for parsing some basic XML via PL/SQL. This example shows some basic ways to parse XML.

DECLARE
l_doc       VARCHAR2(2000);
l_domdoc    dbms_xmldom.DOMDocument;
l_nodelist  dbms_xmldom.DOMNodeList;
l_node      dbms_xmldom.DOMNode;
l_value     VARCHAR2(30);

l_xmltype   XMLTYPE;
l_empx      XMLTYPE;
l_index     PLS_INTEGER;
l_col_ind   PLS_INTEGER;
BEGIN
l_doc := '<employees>
 <emp>
  <name>Scott</name>
  <favorites>
   <color>red</color>
   <color>orange</color>
  </favorites>
 </emp>
 <emp>
  <name>John</name>
  <favorites>
   <color>blue</color>
   <color>green</color>
  </favorites>
 </emp>
</employees>';

l_domdoc   := dbms_xmldom.newDomDocument(l_doc);

-- Method 1
dbms_output.put_line('Method 1');
l_nodelist := dbms_xslprocessor.selectNodes(dbms_xmldom.makeNode(l_domdoc),'/employees/emp/name');
FOR cur_emp IN 0 .. dbms_xmldom.getLength(l_nodelist) - 1 LOOP
 l_node := dbms_xmldom.item(l_nodelist, cur_emp);
 l_value := dbms_xmldom.getnodevalue(dbms_xmldom.getfirstchild(l_node));
 dbms_output.put_line('Emp Name: '||l_value);
END LOOP;

-- Method 2
dbms_output.new_line;
dbms_output.put_line('Method 2');
l_nodelist  := dbms_xmldom.getelementsbytagname(l_domdoc, 'name');
-- get first item from list, could loop as shown above
l_node      := dbms_xmldom.item(l_nodelist, 0);
l_value    := dbms_xmldom.getnodevalue(dbms_xmldom.getfirstchild(l_node));
dbms_output.put_line('Emp Name: '||l_value);

-- Done with DOMDocument examples, setup for XMLType based examples
dbms_xmldom.freeDocument(l_domdoc);
l_xmltype := XMLTYPE(l_doc);

-- Method 3
dbms_output.new_line;
dbms_output.put_line('Method 3');
l_index := 1;
WHILE l_xmltype.Existsnode('/employees/emp[' || To_Char(l_index) || ']') > 0
LOOP
 l_value := l_xmltype.extract('/employees/emp[' || To_Char(l_index) || ']/name/text()').getStringVal();
 dbms_output.put_line('Emp Name: '||l_value);
 l_index := l_index + 1;
END LOOP;

-- Method 4
dbms_output.new_line;
dbms_output.put_line('Method 4');
l_value := l_xmltype.extract('/employees/emp[2]/name/text()').getStringVal();
dbms_output.put_line('Emp Name: '||l_value);

-- Method 5
dbms_output.new_line;
dbms_output.put_line('Method 5');
l_index := 1;
WHILE l_xmltype.Existsnode('/employees/emp[' || To_Char(l_index) || ']') > 0
LOOP
 l_empx := l_xmltype.extract('/employees/emp[' || To_Char(l_index) || ']');
 l_col_ind := 1;
 WHILE l_empx.Existsnode('/emp/favorites/color[' || To_Char(l_col_ind) || ']') > 0
 LOOP
    l_value := l_empx.extract('/emp/favorites/color[' || To_Char(l_col_ind) || ']/text()').getStringVal();
    dbms_output.put_line('Color: '||l_value);
    l_col_ind := l_col_ind + 1;
 END LOOP;
 l_index := l_index + 1;
END LOOP;
END;

Running this script produces

Method 1
Emp Name: Scott
Emp Name: John
 
Method 2
Emp Name: Scott
 
Method 3
Emp Name: Scott
Emp Name: John
 
Method 4
Emp Name: John
 
Method 5
Color: red
Color: orange
Color: blue
Color: green

In PL/SQL, you basically have two methods. The first, and older method, is via dbms_xmldom/dbms_xslprocessor and the second is via XMLType. The first method uses DOMDocuments, DOMNodes, DOMNodelist, etc to slowly break down the XML and parse/traverse it. This is based on the W3C definition of a DOM. The second method uses XPath 1.0 syntax to quickly parse an XMLType and return either another XMLType fragment or a basic data type. My preference for parsing XML is via an XMLType as the code seems cleaner to write, understand, and maintain.

Methods 1 and 2 use the old style to parse the XML.
Method 1 used dbms_xmlprocessor to select all the nodes that match the XPath /employees/emp/name and then iterates over the list that is returned to print out all the names.
Method 2 uses dbms_xmldom to get all the nodes called "name" and simply prints the first one. It too could easily loop through all the name nodes since both operations return a DOMNodeList.

Methods 3 through 5 use XMLType to parse the XML.
Method 3 looks for and returns the same information as Method 1. This difference is that instead of returning a list of nodes that match the criteria, it goes out and gets each occurrence one at a time.
Method 4 is like Method 2 in that it returns the contents of a specific node directly, in this case, the name of the second employee "emp[2]/name"
Method 5 shows the name for each employee and their favorite colors. As the code shows, it extract each emp as an XMLType and then parses that XMLType to extract all the colors.

Gotchas:

• /text() refers to the text contents of the given node. Leaving this off returns the node itself (an XMLType).

• .getStringVal() and others, along with many examples can be found in the XMLType Operations section within the online Oracle Documentation.
  

• "ORA-30625: method dispatch on NULL SELF argument is disallowed" means your XPath returned returned nothing aka NULL and further operations failed. This is common when doing a .get*Val() on a .extract() that returned NULL. One way is to do a .ExistsNode before the .extract().get*Val()