« DataOne Workshop - Morning | Main | DataOne Workshop - Member Nodes »

September 30, 2011

DataOne Workshop - Installation Instructions

Installation notes
------------------

data - example data set
d1_common_python - types and service methods
d1_libclient_python - library of utility methods for calling d1 common
d1_client_cli - build a command line parse


Prerequisites
=============

0. Ubuntu 10.04 stock with patches (installed from the Ubuntu Server CD, with OpenSSH Server selected at the Software Selection screen)
1. Java
-- add deb "http://archive.canonical.com/ lucid partner" to
/etc/apt/sources.list
$ sudo aptitude update
$ sudo aptitude install sun-java6-jdk

2. Install certificates
-- Copy certificate files to /etc/ssl/certs
$ sudo cp dataone*.crt /etc/ssl/certs
$ sudo cp test_dataone_org.crt /etc/ssl/certs
$ sudo cp cilogon-*pem /etc/ssl/certs
$ sudo c_rehash /etc/ssl/certs
-- Copy private key to /etc/ssl/private
$ sudo cp test_dataone_org.nopassword.key /etc/ssl/private
-- Add certs to Java keystore
$ cd /usr/lib/jvm/java-6-sun/jre/lib/security
$ sudo keytool -import -alias DataOneCA -keystore ./cacerts -file
/etc/ssl/certs/dataone-ca.crt
$ sudo keytool -import -alias DataOneTestCA -keystore ./cacerts -file
/etc/ssl/certs/dataone-test-ca.crt
$ sudo keytool -import -alias CILogonSilver -keystore ./cacerts -file
/etc/ssl/certs/cilogon-silver.pem
$ sudo keytool -import -alias CILogonBasic -keystore ./cacerts -file
/etc/ssl/certs/cilogon-basic.pem
$ sudo keytool -import -alias CILogonOpenID -keystore ./cacerts -file
/etc/ssl/certs/cilogon-openid.pem

3. Tomcat 6
$ sudo aptitude install tomcat6
-- Edit /etc/tomcat6/server.xml to enable the AJP connector on 8009
$ sudo /etc/init.d/tomcat6 restart

4. Apache
$ sudo aptitude install apache2 libapache2-mod-jk
Modify metacat workers.properties to point at Java and Tomcat, then:
$ sudo cp -i debian/jk.conf /etc/apache2/mods-available/
$ sudo cp -i debian/workers.properties /etc/apache2/
$ sudo a2dismod jk
$ sudo a2enmod jk
$ sudo a2enmod rewrite
$ a2enmod ssl
$ sudo cp -i debian/knb-ssl /etc/apache2/sites-available/
$ sudo a2dissite 000-default
-- Modify knb and knb-ssl to fit the local host
$ sudo a2ensite knb
$ sudo a2ensite knb-ssl
$ sudo /etc/init.d/apache2 restart

5. Subversion
$ sudo apache2 libapache2-mod-jk

6. Set up user account
$ sudo adduser demo

7. Install ant
$ sudo apt-get install --no-install-recommends ant

8. Install maven2
$ sudo aptitude install maven2

9. Postgres
$ sudo aptitude install postgresql
Add "host metacat metacat 127.0.0.1/32 password" to pg_hba.conf

10. Create LDAP account
Via KNB web site, username = d1demo

11. Curl
$ sudo aptitude install curl

12. Python libraries
$ sudo aptitude install python-setuptools
$ sudo aptitude install python-dateutil
$ sudo aptitude install python-lxml
$ sudo easy_install PyXB
$ sudo easy_install minixsv
$ sudo aptitude install python-argparse python-argparse-doc
-- Also install the DataONE Python client libraries
$ cd d1_common_python
$ sudo python setup.py develop
$ cd ../d1_libclient_python
$ sudo python setup.py develop
$ echo "alias d1=~/d1_client_cli/src/d1_client_cli/dataone.py" >> ~/.bashrc

13. R system
$ sudo aptitude install r-base-core
$ sudo R CMD javareconf
$ R
> install.packages("rJava")
> q()
$


Metacat install
----------------
0. Set up postgres
$ sudo -s
# su - postgres ##switch to postgres user
$ createdb metacat ##empty postgres database
$ psql metacat ##login to metacat database
## create db user
> CREATE USER metacat WITH UNENCRYPTED PASSWORD 'metacat';
> \q
$ exit
# /etc/init.d/postgresql-8.4 restart
# exit

1. Create metacat storage directory
$ sudo mkdir -p /var/metacat/
$ sudo chown -R tomcat6 /var/metacat ##recursively give permissions to director for tomcat user

2. Servlet installation
$ cd metacat-1.10.0-snapshot10 ##dev snapshot of metacat
$ sudo cp knb.war /var/lib/tomcat6/webapps/ ##java web files to web server
$ sudo cp geoserver.war /var/lib/tomcat6/webapps/
$ sudo /etc/init.d/tomcat6 restart

3. Configure metacat

-- Open Metacat site in browser
-- https://demoX.test.dataone.org/knb/
-- admin is: uid=d1demo,o=unaffiliated,dc=ecoinformatics,dc=org
- "Metacat Administrator"
## To create own admin - set up own LDAP server or use KNB
-- Global properties
-- Set Database user/pw to metacat/metacat
-- set Context to knb
-- DataONE section
Node name: Demonstration Node 1
Node ID: DEMOX -- for example, 'DEMO1', ##unique and persistent identifier
Node Subject: CN=DEMOX, DC=dataone, DC=org
-- Note that this automatically registers as a MN
## Node account representing the MN during authentication actions
## True = CNs should approach MNs

-- Restart tomcat
$ sudo /etc/init.d/tomcat6 restart

####################################3
####################################
####################################

Run a few DataONE Web services

#refer to http://mule1.dataone.org/ArchitectureDocs-current/ for API
------------------------------
https://demoX.test.dataone.org/knb/d1/mn/v1/node ##our MN
https://cn-dev.dataone.org/cn/v1/node ##all nodes
https://demoX.test.dataone.org/knb/d1/mn/v1/object

Logon to CILogon
----------------
1. Visit: https://cilogon.org/?skin=DataONE
2. Choose your provider and log in
-- Likely: LTER, Google, or Protect Network
2a. If you don't have an account, create one
-- Either Google or Protect network
3. Note the name of the certificate file downloaded to your machine
4. Note: there is a preinstalled certificate on your demo machine

Insert data files and metadata files
------------------------------------
First set some defaults for client operation:

$ cd ~
$ d1 \
--mn-url https://demo2.test.dataone.org/knb/d1/mn/v1 \
--cn-url https://cn-dev.dataone.org/cn/v1 \
--dataone-url https://cn-dev.dataone.org/cn/v1 \
--sysmeta-submitter "CN=DEMO2,DC=dataone,DC=org" \
--sysmeta-rightsholder "CN=DEMO2,DC=dataone,DC=org" \
--sysmeta-origin-member-node DEMO2 \
--sysmeta-authoritative-member-node DEMO2 \
--sysmeta-access-policy-public \
--cert-path /etc/dataone/client/certs/DEMO2.pem \
--key-path /etc/dataone/client/certs/DEMO2.pem \
--fields "pid,origin_mn,datemodified,size,objectformat,title" \
--query "*:*" \
--store-config


Now add one data object:

$ d1 \
--sysmeta-object-format text/csv \
--sysmeta-access-policy-public \
create foo.1.1 data/data-sites.csv

List objects on the node now
----------------------------
https://demoX.test.dataone.org/knb/d1/mn/v1/object
$ d1 --mn-url https://demoX.test.dataone.org/knb/d1/mn/v1 list


View system metadata
--------------------
https://demoX.test.dataone.org/knb/d1/mn/v1/meta/foo.1.1
$ d1 \
--dataone-url https://demoX.test.dataone.org/knb/d1/mn/v1 \
meta foo.1.1

Get the object from the MN
--------------------------
https://demoX.test.dataone.org/knb/d1/mn/v1/object/foo.1.1
$ d1 \
--dataone-url https://demoX.test.dataone.org/knb/d1/mn/v1 \
get foo.1.1


Insert two more objects -- data and EML
---------------------------------------
$ d1 \
--sysmeta-object-format text/csv \
--sysmeta-access-policy-public \
create foo.2.1 data/data-samples.csv

$ d1 \
--sysmeta-object-format eml://ecoinformatics.org/eml-2.0.1 \
--sysmeta-access-policy-public \
create foo.3.1 data/eml-metadata.xml

Test that the node passes tests
--------------------------------
1. Visit: http://mncheck.test.dataone.org:8080/MNWebTester
2. Enter MN Base URL: https://demoX.test.dataone.org/knb/d1/mn

List the objects on the node
----------------------------
https://demoX.test.dataone.org/knb/d1/mn/v1/object
$ d1 --mn-url https://demoX.test.dataone.org/knb/d1/mn/v1 list

Show synchronization has happened
---------------------------------
https://cn-dev.dataone.org/cn/v1/object
https://cn-dev.dataone.org/cn/v1/resolve/foo.3.1
https://cn-dev.dataone.org/cn/v1/meta/foo.3.1


Search for data locally on Metacat
----------------------------------
Visit: https://demoX.test.dataone.org/knb/
Search for: %

Search for data on the D1 Index
-------------------------------
Using the CLI:

$ d1 --query "origin_mn:DEMOX" search

List fields available for searching:
## fields indexed out of any metadata standard
$ d1 fields

A couple more searches:

$ d1 --query "barnacle" search

$ d1 --query "origin_mn:DEMOX AND objectformat:text/csv" search

Through a web interface:
http://cn-dev.dataone.org/solr/search.html

Through the Mercury search interface (operating on older CN deployment):
http://cn.dataone.org/mercury3/

Posted by kkwaiser at September 30, 2011 02:33 PM

Trackback Pings

TrackBack URL for this entry:
http://mblog.lib.umich.edu/mt-bin/mt-tb.cgi/2247

Listed below are links to weblogs that reference DataOne Workshop - Installation Instructions:

» http://couponman.in/stores/babyoye-coupons/ from http://couponman.in/stores/babyoye-coupons/
Data Discussions: DataOne Workshop - Installation Instructions [Read More]

Tracked on February 13, 2014 05:22 AM

» Canada Goose Pas Cher from Canada Goose Pas Cher
Cette édition est à lire comme la suite [Read More]

Tracked on February 17, 2014 09:19 PM

» Michael Honari from Michael Honari
Data Discussions: DataOne Workshop - Installation Instructions [Read More]

Tracked on February 28, 2014 09:24 PM

» twitch views from twitch views
Data Discussions: DataOne Workshop - Installation Instructions [Read More]

Tracked on March 2, 2014 09:27 PM

» Mike Honari from Mike Honari
Data Discussions: DataOne Workshop - Installation Instructions [Read More]

Tracked on March 3, 2014 08:07 PM

» máy ép trái cây philips hr1810/70 from máy ép trái cây philips hr1810/70
Data Discussions: DataOne Workshop - Installation Instructions [Read More]

Tracked on March 5, 2014 02:27 PM

» セイコー アストロン 2014 from セイコー アストロン 2014
Its like men and women arent fascinated unless it is one thing to accomplish with Lady gaga! Your own stuffs outstanding. Always maintain it up!} [Read More]

Tracked on March 6, 2014 03:23 PM

» click here from click here
Data Discussions: DataOne Workshop - Installation Instructions [Read More]

Tracked on April 1, 2014 05:00 PM

» website from website
Data Discussions: DataOne Workshop - Installation Instructions [Read More]

Tracked on April 6, 2014 07:25 AM

» ray-ban.ellestique.com.au from ray-ban.ellestique.com.au
I and my colleagues watch the soccer game clips at YouTube for all time, for the reason that they have in nice quality. [Read More]

Tracked on April 13, 2014 01:46 PM

Comments

Login to leave a comment. Create a new account.