Installation of the Joseki SPARQL Endpoint in front of Oracle on Ubuntu

Here’s one from my archives. It was a bitch to come by, so hopefully this will help some other poor soul tasked with setting up a HTTP interface to Oracle’s semantic store.

Preparation

First, install Oracle on Ubuntu. That’s a pain, and only partially covered elsewhere. My notes on that subject, however, are enough for an entire months worth of blog posts, so I’m saving them for next November.

Once you’re done with the first step, you’ll want to enable semantic support, install a semantic network, maybe create a model or two, populate them and then make them queryable with a shiny SPARQL web service endpoint.

To enable Oracle’s semantic support (from here):

sqlplus / as SYSDBA
@$ORACLE_HOME/md/admin/catsem.sql

To create a semantic network (from here):

CREATE TABLESPACE rdf_tblspace
  DATAFILE '/u01/app/oracle/oradata/tseval/rdf_tblspace.dat' SIZE 1024M REUSE
  AUTOEXTEND ON NEXT 256M MAXSIZE UNLIMITED
  SEGMENT SPACE MANAGEMENT AUTO;

EXEC SEM_APIS.CREATE_SEM_NETWORK('rdf_tblspace');

Create a user in Oracle and grant them privileges:

create user rdfuser
identified by rdfuser
default tablespace users
temporary tablespace temp;

grant connect to rdfuser;

SPARQL Endpoint

Install Tomcat 6:

sudo apt-get install tomcat6

Download Joseki 3.4.0 and the latest Jena adaptor for Oracle (from here and here):

unzip joseki-3.4.0.zip
mv Joseki-3.4.0 joseki  

mkdir jena_adaptor
cd jena_adaptor
unzip ../jena_adaptor_for_release11.2.zip

The Joseki web app must be carefully put together using files from each. Copy a directory from joseki as a starting directory to construct the web apps:

cp -R joseki/webapps/joseki joseki.war.d
cp -R joseki/lib joseki.war.d/WEB-INF

And copy away:

cp jena_adaptor/joseki/* joseki.war.d
cp jena_adaptor/web/* joseki.war.d/WEB-INF
cp jena_adaptor/jar/* joseki.war.d/WEB-INF/lib

The sdordfclient.jar in the jena_adaptor has since been superceded and doesn’t work with Tomcat anyway (insert grumblings about WebLogic here). I got my hands on a copy of the latest version thanks to having sweet contacts inside the Oracle mothership, if you do too, you’ll want to put that in there. If you don’t, you should give up now and make something of your life.

The webapp needs Oracle’s sdordf.jar from your installation:

cp $ORACLE_HOME/md/jlib/sdordf.jar joseki.war.d/WEB-INF/lib
cp $ORACLE_HOME/jdbc/lib/ojdbc6.jar joseki.war.d/WEB-INF/lib

And Joseki comes with a replacement for Tomcat’s default servlet-api jar:

rm joseki.war.d/WEB-INF/lib/servlet-api-2.5.jar
mv joseki.war.d/WEB-INF/lib/servlet-api-2.5-6.1.10.jar /usr/share/java
cd /usr/share/tomcat6/lib
rm servlet-api-2.5.jar
ln -s ../../java/servlet-api-2.5-6.1.10.jar servlet-api-2.5.jar

Next, add this file to make an Oracle datasource that Oracle’s Joseki service will look for:
joseki.war.d/META-INF/context.xml

<?xml version="1.0" encoding="UTF-8"?>
<Context>
  <Resource name="OracleSemDS"
            auth="Container"
            type="oracle.jdbc.pool.OracleDataSource"
            factory="oracle.jdbc.pool.OracleDataSourceFactory"
            user="" 
            password=""
            driverClassName="oracle.jdbc.driver.OracleDriver"
            url="jdbc:oracle:thin:@localhost:1521/orcl"
            validationQuery="select 1 from dual"
            maxActive="30"
            maxIdle="10"
            maxWait="-1"/>
</Context>

Also, I’d bump up the number of connections in Joseki’s connection pool
joseki.war.d/joseki-config.ttl:

<#oracle> rdf:type oracle:Dataset;
    joseki:poolSize     30 ;         ## Number of concurrent connections allowed to this dataset.
    oracle:connection
    [ a oracle:OracleConnection ;
    ];
    oracle:defaultModel [ oracle:firstModel "M1" ] .

Verify that your directory structure looks like so:

./joseki.war.d:
application.xml
crossdomain.xml
index.html
joseki-config.ttl
META-INF
query.html
robots.txt
sparqler-details.html
sparql.html
StyleSheets
update.html
validator.html
WEB-INF
xml-to-html.xsl

./joseki.war.d/META-INF:
context.xml

./joseki.war.d/StyleSheets:
joseki.css

./joseki.war.d/WEB-INF:
lib
web.xml

./joseki.war.d/WEB-INF/lib:
arq-2.8.0.jar
arq-2.8.0-tests.jar
icu4j-3.4.4.jar
iri-0.7.jar
jena-2.6.0.jar
jenatest-2.6.0.jar
jetty-6.1.10.jar
jetty-util-6.1.10.jar
joseki-3.4.0.jar
junit-4.5.jar
log4j-1.2.12.jar
lucene-core-2.3.1.jar
ojdbc5.jar
sdordfclient.jar
sdordf.jar
slf4j-api-1.5.6.jar
slf4j-log4j12-1.5.6.jar
stax-api-1.0.1.jar
wstx-asl-3.2.9.jar
xercesImpl-2.7.1.jar

Create the joseki war file:

cd joseki.war.d
jar cvf ../joseki.war .

Deploy it:

mv joseki.war /var/lib/tomcat6/webapps

Goodluck and Godspeed.

starting in the box

So how does this thing work? Well, it runs on a Dell Optiplex I keep under my desk.

I run Fedora 10 on my day-to-day laptop and absolutely love it. That’s why, after months of collecting dust (and a few very short weeks of running MINIX, but that’s another post) I popped the Fedora 10 LiveCD into my Optiplex’s drive and…waited. If I was dead-set on Fedora, I would still be waiting today because, for some reason, the LiveCD never finished loading.

My box being as old as it is, I took a stab in the dark and guessed that my hardware just wasn’t up to par with the latest Fedora drop. Shame.

This was around the turn of the year, and at that same time a group of very bright gentlemen that I work with had resolved to learn Haskell. Following along, I dove into the Haskell community a bit and became very impressed with the things that Haskellers had to say about Arch Linux. I burned a quick CD, installed it and played around with it a bit.

The Arch Linux LiveCD isn’t anything fancy. The more time I spend on my laptop, though, the more I find myself shutting things off or disabling notifications or swearing at software when I find I can’t do either. My tastes run towards the simple and substantive. Software is one thing I feel that, with increase in quality, should decrease in weight.

Arch Linux starts you off very low to the ground. The full install doesn’t even give you X11, much less a Desktop or windows or Minesweeper, which is disorienting. One thing that delighted me, however, was that it set me up with the filesystem of my choice, XFS. I remember very sharply that Fedora did not give me the option to install the XFS file system out of the box.

My daily programming touches maybe 10-20 userspace programs and I installed those about as quickly as I could think of them. With a package manager, though, it makes great sense to me to only install the programs I need when I need them. Anything else is just taking up space and processor that it doesn’t need to.

Beyond that, Arch Linux’s wiki was an invaluable and easily navigable resource. I spent about 12 hours over one weekend setting up my box. My final effort was installing stumpwm. In 12 hours of installing and configuring, it was the first program that gave me a wince of trouble. Turns out, though, that there was an active forum post on the Arch Linux forums that night discussing the very problems I was having. A few posts later and I not only had a sweet, minimal, emacs-like window manager installed, but I felt like I had left behind some advice useful to anyone who might follow in my footsteps.

After the Arch Linux installation, I had a CouchDB instance and sbcl to run my imagined lisp scraper. I just needed to (1) make my box visible to the internets and (2) fill it up with data.