XQuery tutorial

How to query a native XML database with XQuery, how to copy files to a native XML database. This tutorial doesn't intend to be an XQuery tutorial, but a tutorial for submitting XQuery queries with RefleX in batch mode and within a Web application.

In RefleX, native XML databases are accessible via XML:DB. In this tutorial, we are using the eXist database.

Preparing the server

This section explain how to install eXist locally for running the tutorial.

You can either download and install yourself eXist, or use the libraries available in the RefleX repository :

  • Using the eXist distribution :
    • download eXist
    • install as indicated on the manual
    • set the environment variable EXIST_HOME on the relevant directory :
       linux$ export EXIST_HOME=/path/to/exist/
    • launch the server with an XMLRPC service on localhost:8181/xmlrpc/* (the following examples will use that database); alternatively, you can launch the server with the start.sh script like shown below.
  • Using the libraries of eXist available in the RefleX repository :
    • checkout the files available in the SVN repository :
       linux$ svn checkout svn://scm.gforge.inria.fr/svn/reflex/root/libext/exist
      or download them at https://gforge.inria.fr/plugins/scmsvn/viewcvs.php/root/libext/exist?root=reflex and https://gforge.inria.fr/plugins/scmsvn/viewcvs.php/root/libext/exist/libext?root=reflex; you can also browse the SVN repository to reach that files.
    • set the environment variable EXIST_HOME on the relevant directory (where are the .jar files and the libext/ directory) :
       linux$ export EXIST_HOME=/path/to/exist/
    • run the server :
       linux$ cd doc/tutorial/xquery/server/
       linux$ ./start.sh
      The end of the output should be :
      Server launched ...
      Installed services:
      -----------------------------------------------
      xmlrpc: localhost:8181/xmlrpc/*
      Do not cancel the task; you can of course run the server in background; if needed, you can relaunch the server in the same manner.
    • in another terminal, run the examples

Troubleshooting

  • If you get the following error:

    Exception in thread "main" java.lang.NoClassDefFoundError: org/xml/sax/ext/DefaultHandler2

    ...ensure that the additional .jar files are present in the [EXIST_HOME]/libext/ directory; those libraries are available here (note they are available if you checkout the libraries from SVN like mentionned previously).

  • If your database is locked:

    File lock last access timestamp: 30 nov. 2007
        /path/to/RefleX/tutorial/xquery/server/data/dbx_dir.lck
    An exception occurred while launching the server:
        The database directory seems to be locked by another database instance.
    Found a valid lock file:
        /path/to/RefleX/tutorial/xquery/server/data/dbx_dir.lck
    org.exist.EXistException:
        The database directory seems to be locked by another database instance.
    Found a valid lock file:
        /path/to/RefleX/tutorial/xquery/server/data/dbx_dir.lck

    ...simply avoid running the server twice; to unlock and restart, simply delete the lock file before restarting.

  • If you get this error when running a query:

    java.lang.NoSuchMethodError: java.net.URI.compareTo(Ljava/net/URI;)I

    ...this is due to eXist that uses a higher version of Java that support this method. You just have to use the required JVM.

For other errors that obstruct the server starting, please refer to the eXist documentation and try to install it from the eXist distribution; you can use the settings in [doc/tutorial/xquery/server/server.xml] for configuring the server, or change the connexion URL in the active sheets to whatever you used in your settings.

Loading files in the XML database

In this tutorial, the XML documents are the current projects of a company. Loading this set of files in the database consist on copying them into the XML:DB repository:

[doc/tutorial/xquery/projects/load.xcl]

<?xml version="1.0" encoding="iso-8859-1"?>
<xcl:active-sheet xmlns:io="http://ns.inria.org/active-tags/io" xmlns:xcl="http://ns.inria.org/active-tags/xcl"> <xcl:set name="connexion" value="{ io:file('xmldb:exist://localhost:8181/xmlrpc/db/') }"/> <xcl:for-each name="source" select="{ io:file('files/')/* }"> <xcl:echo value="{ name( $source ) }"/> <!--we set the MIME type "application/xml" on the target file--> <io:file name="target" mime-type="application/xml"
uri="{ $connexion }/{ name( $source ) }"/> <io:copy source="{ $source }" target="{ $target }"/> </xcl:for-each> </xcl:active-sheet>

Running the batch script (linux) from the RefleX home directory :

 linux$ ./doc/tutorial/xquery/projects/run.sh load.xcl

The files loaded in the database should be displayed in the standard output :

increase-profits.xml
more-sales.xml
new-products.xml
new-projects.xml
reducing-costs.xml

Querying the database with XQuery

The active tag that allows to launch a query is <io:request>. It accept a parameter that indicates which XML:DB service to invoke, and the path to the query source. In the following active sheet, the XQuery source is supplied as a system property :

[doc/tutorial/xquery/projects/query.xcl]

<?xml version="1.0" encoding="iso-8859-1"?>
<xcl:active-sheet xmlns:io="http://ns.inria.org/active-tags/io" xmlns:sys="http://ns.inria.org/active-tags/sys" xmlns:xcl="http://ns.inria.org/active-tags/xcl"> <xcl:set name="connexion" value="{ io:file('xmldb:exist://localhost:8181/xmlrpc/db/') }"/> <io:request name="result" connect="{ $connexion }" style="stream" type="XQueryService"
source="{ string( $sys:env/query ) }"/> <xcl:transform source="{ $result }" output="{ $sys:out }" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <!--we don't want the serializer formatting an HTML output--> <xcl:param name="xsl:method" value="xml"/> </xcl:transform> <xcl:echo value=""/> </xcl:active-sheet>

The same shell script can be used for running the active sheet, but one more parameter must be used, the XQuery query, that will be set as a system environment variable :

 linux$ ./doc/tutorial/xquery/projects/run.sh query.xcl count-projects.xqy

The XQuery query :

[doc/tutorial/xquery/projects/count-projects.xqy]

<projects>
    { count( //project ) }
</projects>

The result of the query is displayed in the standard output :

<?xml version="1.0" encoding="UTF-8"?>
<projects>5</projects>

The next query lists the projects with their name and summary ; the result is an HTML document :

[doc/tutorial/xquery/projects/list-projects.xqy]

<html>
  <head>
    <title>Projects in our company</title>
  </head>
  <body>
    <h1>Current projects</h1>
    <p>{ count( //project ) } projects :</p>
    <table border="1">
      <tr><th>Name</th><th>Summary</th></tr>
      { for $project in //project
        return <tr><td>{ string( $project/name ) }</td><td>{ string( $project/summary ) }</td></tr>
      }
    </table>
  </body>
</html>
 linux$ ./doc/tutorial/xquery/projects/run.sh query.xcl list-projects.xqy
<?xml version="1.0" encoding="UTF-8"?>
<html>
    <head>
        <title>Projects in our company</title>
    </head>
    <body>
        <h1>Current projects</h1>
        <p>5 projects :</p>
        <table border="1">
            <tr>
                <th>Name</th>
                <th>Summary</th>
            </tr>
            <tr>
                <td>Increase profits</td>
                <td>We want to increase the profits in our company</td>
            </tr>
            <tr>
                <td>More sales</td>
                <td>We want our company selling more products</td>
            </tr>
            <tr>
                <td>New products</td>
                <td>We want new products in our company</td>
            </tr>
            <tr>
                <td>New projects</td>
                <td>We want new projects in our company</td>
            </tr>
            <tr>
                <td>Cost reduction</td>
                <td>We want to reduce the costs in our company</td>
            </tr>
        </table>
    </body>
</html>
[www]
Projects in our company - Web Navigator

Current projects

5 projects :

Name Summary
Increase profits We want to increase the profits in our company
More sales We want our company selling more products
New products We want new products in our company
New projects We want new projects in our company
Cost reduction We want to reduce the costs in our company

The last query accept a parameter to display the people involved in a project and the detail of the project. The project to display is supplied as a system environment variable. For this purpose, the shell script will set the last parameter as an environment variable, and another active sheet will be used for passing it to the XQuery query :

[doc/tutorial/xquery/projects/paraquery.xcl]

<?xml version="1.0" encoding="iso-8859-1"?>
<xcl:active-sheet xmlns:io="http://ns.inria.org/active-tags/io" xmlns:sys="http://ns.inria.org/active-tags/sys" xmlns:xcl="http://ns.inria.org/active-tags/xcl"> <xcl:set name="connexion" value="{ io:file('xmldb:exist://localhost:8181/xmlrpc/db/') }"/> <io:request name="result" connect="{ $connexion }" style="stream" type="XQueryService"
source="{ string( $sys:env/query ) }"> <!--forwarding the environment variable to the XQuery query--> <xcl:param name="proj" value="{ string( $sys:env/proj ) }"/> </io:request> <xcl:transform source="{ $result }" output="{ $sys:out }" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <!--we don't want the serializer formatting an HTML output--> <xcl:param name="xsl:method" value="xml"/> </xcl:transform> <xcl:echo value=""/> </xcl:active-sheet>

[tutorial/xquery/projects/project.xqy]

xquery version "1.0";
declare variable $proj as xs:string external;
<html>
  <head>
    <title>{ $proj }</title>
  </head>
  <body>
    <h1>Project "{ //project[@id=$proj]/name }"</h1>
    <p>Project team :</p>
    <ul>
      { for $person in //project[@id=$proj]/team/person
        return <li>{ string( $person ) }</li>
      }
    </ul>
    <p>Project's tasks :</p>
    <ol>
      { for $task in //project[@id=$proj]/tasks/task
        return <li>{ string( $task ) }</li>
      }
    </ol>
  </body>
</html>
 linux$ ./doc/tutorial/xquery/projects/run.sh paraquery.xcl project.xqy proj=more-sales
<?xml version="1.0" encoding="UTF-8"?>
<html>
    <head>
        <title>more-sales</title>
    </head>
    <body>
        <h1>Project "More sales"</h1>
        <p>Project team :</p>
        <ul>
            <li>Harvey</li>
            <li>Harold</li>
            <li>Henry</li>
            <li>Helen</li>
            <li>Hugo</li>
        </ul>
        <p>Project's tasks :</p>
        <ol>
            <li>We don't need more people in the production team ; just make them work faster.</li>
            <li>We could reduce the size of our products. The packaging would remain the same but we could sell more items.</li>
            <li>Our products looks like flat ; this is an opportunity to make working the idle guys from the design department.</li>
            <li>Make our products much more attractive. We could lie skillfully about their capabilities.</li>
        </ol>
    </body>
</html>
[www]
more-sales - Web Navigator

Project "More sales"

Project team :

  • Harvey
  • Harold
  • Henry
  • Helen
  • Hugo

Project's tasks :

  1. We don't need more people in the production team ; just make them work faster.
  2. We could reduce the size of our products. The packaging would remain the same but we could sell more items.
  3. Our products looks like flat ; this is an opportunity to make working the idle guys from the design department.
  4. Make our products much more attractive. We could lie skillfully about their capabilities.

Running XQuery within a Web application

[TODO[Use the same queries as above in a Web application...]]