« October 2010 | Main | December 2010 »

November 30, 2010

Porting the REU Application form to Drupal

We host a Research Experience for Undergraduates program here at UMBS and I've been asked to build the registration form (process, actually) into our Drupal site. Here are the notes:

REU Components:

1) Content profile attached to User Registration page. This is our Person content type and collects basic, personal information. Also use, AutoAssignRole to associate the registration page (reu-registration) with the REU Role.

2) REU application (a Content Type with bunch of CCK fields):
- modules used: Private Download, CCK3, Markup, Multi-column checkboxes & radios, probably other "basic" modules

3) Use panels & views to customize /user page

4) Build custom Menu system for logged in User
5) Associate workflow states with the REU application
6) Build views for staff use of REU apps
7) Add a new role "REU Reviewers" for faculty who review incoming apps
- in setting up Role Permissions, had to go this route to view unpublished content.
8) Setup auto-email to fire whenever an REU application is moved to "Ready for Review"
9) Adjust confirmation of Account Creation email and redirect to new "CHECK YOUR EMAIL" page after hitting submit.

Upon confirmation of email account and first login, forward to
Staff Components:
not done yet

Posted by kkwaiser at 11:42 AM | Comments (0)

November 29, 2010

Overhaul the beta site


1) Install a local copy of the live site
2) Change database prefix and settings.php
3) Update current modules using drush
4) Run update.php, cron and check Status Report
- modded a few directory permissions
- cron giving me some crap about /tmp but I'm going to ignore is
5) Install new modules:

$ sudo drush dl content_profile modalframe email_registration rules token jquery_update noderelationships autoassignrole cck_required_by_role node_clone permission_select conditional_fields auto_nodetitle workflow

Following my post on d.o, I need the dev version of a few modules

Check for available versions of a module:
$ drush pm-releases tabs

$ sudo drush dl tabs-6.x-1.x-dev
$ sudo drush dl cck_fieldgroup_tabs-6.x-1.x-dev
$ sudo drush dl jquery_update-6.x-2.0-alpha1

jQueryUI notes:
$ cd sites/all/modules/jquery_ui/
$ more README.txt
$ sudo wget http://jquery-ui.googlecode.com/files/jquery-ui-1.7.3.zip
$ sudo unzip jquery-ui-1.7.3.zip
$ sudo mv jquery-ui-1.7.3 jquery.ui

jQueryUpdate notes:
$ cd ../jquery_update/
$ more README.txt
Oh, apparently you just enable the module...didn't expect that.

6) Enable said modules as well as Triggers and CCK Field Permissions. Run update.php and cron.php

7) Download, enable CCK3 (Multigroup) and Schema

8) Working on updating metadata structures...
Done: NodeRelationships (create node from within node); Re-establish datafile, check fields against DEIMS fields
- This still needs work

9) Setup permissions for Admin
- Give admin rights on Permission Select and then Master Permissions.

10) Create two new User Roles & Set permissionss:
- Colleague - for research, instructional and misc summer visitors
- REU Applicant- for incoming REU applications

11) Import housing Application
- adjust field permissions
- need to look at permissions for creating research project

12) Setup Content Profile and AutoAssignRole
- Person as profile component
- Upgrade to AAR-dev following errors reported here and here.
- Allow anonymous users to create account (w/email confirmation)
*Need to adjust welcome message
- Tweak field-level permissions
- adjust fields on Person content type to resemble housing application

13) Had to switch to dev version of CCK Required by Role and apply a patch. See my post here.

14) Make Person Content Profile available for REU registration and create Auto-Assign Role URL for registration.

15) Install CAPTCHA Module and enable on Housing Application. See related post here.
- I didn't add it to user registration because I plan to use email confirmation. Should I add it???

16) Use Panels to setup an advanced User Profile Page
- Bring in a View of content the user has authored
- Display User's Content Profile on Same page
- Need to build menu for users

17) Build REU application & setup workflows
- See this post for steps

18) Change permissions for Views (admin/build/permissions) so only admin can access all views

19) Change default of Person content type NOT to publish

20) Set defaults for Housing Application if the User is logged in. See this post for more.
- Defaults for Date not working See this post for more.

21) Alter Researcher back-end

See this post for more.

Change panel view of Person for people who are logged in
Remove images from Colleague Role
Remove Content Type Person from Your Content View (done)
Add First Time user rule
Modify permissions on new zipcode (done?)
- add private upload fields to data file

Adjust Research Project Fields and Permissions
--stopping for the day. more to come...

Posted by kkwaiser at 04:08 PM | Comments (0)

CCK Default Value Snippets in Drupal

Here are php snippets I used to set default values for logged-in users who are going to apply for housing.

Need to deal with
Date of Birth (See this post for ideas on how to deal with this)

PATTERN USED FOR CONTENT PROFILE FIELDS (role, first name, middle name, last name, gender, umid, address, city, state, country, perm_address, perm_city, perm_state, perm_country, perm_zip, phone, work_phone, emerg_fname, emerg_lname, emerg_phone1, zipcode)

global $user;
$default = content_profile_load('person', $user->uid);
if($user->uid!=1 && user_is_logged_in()){
if (!empty($default->field_person_emerg_email) ) {
$emerg_email = $default->field_person_emerg_email[0]['value'];
return array(
0 => array('value' => $emerg_email)
else return array();


global $user;
$default = content_profile_load('person', $user->uid);
if($user->uid!=1 && user_is_logged_in()){
if (!empty($default->field_person_institution_ref) ) {
$institution_ref = $default->field_person_institution_ref[0]['nid'];
return array(
0 => array('nid' => $institution_ref)
else return array();


global $user;
$default = $user->mail;
if($user->uid!=1 && user_is_logged_in()){
if (!empty($default) ) {
return array(
0 => array('email' => $default)
else return array();

Posted by kkwaiser at 03:58 PM | Comments (0)

November 18, 2010

How to redirect user upon first login in Drupal

Solution is in this comment. Will write a formal How To if I get the chance.

Posted by kkwaiser at 04:43 PM | Comments (0)

November 17, 2010

Advanced help patch for Views

This "how to" focuses on the bulk export and version control functionality of the Views module. The following steps document how to export multiple views at once and create a module from the exported code that will allow you to revert any subsequent changes to your views.

- You have views you want to bulk export; we'll call them myView1 and myView2
- You know how to install and enable a module

- Views (specifically 6.x-2.x)

  1. Modules you will need

    Install Views 2 and enable the following modules: Views, Views exporter, Views UI

  2. Evaluate the Current State of Things

    Navigate to the Views Administration page. You should have Edit | Export | Clone | Delete options next to myView1 and myView2. Once we're done, "Delete" will become "Revert"!

  3. Bulk Export Your Views

    1. Click Tools > Bulk Export to navigate to the Bulk Export page to export myView1 and myView2.
    2. Check the boxes to the left of myView1 and myView2
    3. Choose a module name: my_views
    4. Click Export
  4. Create a Module from the Exported Views

    The results of the export should now be in front of you. Specifically, the results page should be composed of three code-containing regions. This code and the following steps are all you need to create a module that will allow you to revert your views to the state they were in when you exported them.

    1. Create a directory called my_views within your sites/all/modules directory.
    2. Create the following files, paste in the appropriate code, and place them into the my_views directory:
      • my_views.info
      • my_views.module (add a <?php tag at the beginning of this file!)
      • my_views.views_default.inc (add a <?php tag at the beginning of this file!)
  5. Clear the cache

    Just to be safe. Navigate to the Performance Administration page and Clear cached data.

  6. Enable the Module Containing Exported Views

    Navigate to the Modules Administration page and enable my_views Export Module. Save.

  7. Verify Presence of Reversion Option for Exported Views

    Navigate to the Views Administration page. The right of myView1 and myView2 should now read Edit | Export | Clone | Revert. Congratulations, you now have version control in place for your exported views!

Posted by kkwaiser at 09:19 AM | Comments (0)

November 15, 2010

Bulk Modify Access Permissions of Views in Drupal

I'm an idiot and I'm not going to explain why.

Drupal's Views module allows you to limit access to views based upon user Role or by some permission setting. The catch is that the global permissions settings will override those of individual views. Unfortunately, I didn't know this when I started using Views and I have a lot of views I want accessible to Anonymous users and a few views I want accessible only to particular roles.

What are the implications of this:
1) I need to switch my Global permissions so that only the Admin can "access all views"
2) I then need to adjust the Access rights of every single view I've constructed.

There are two options to accomplish 2):
a) Click through every single view and modify the Access settings
b) Export all the views, run a script that modifies the code and then re-import the views

This post is a How To for number 2b):

1) Export all of the views you want to modify. In this case, I want identical permissions on the views I'm going to modify. Specifically, they will all go from no Access settings to access to Anonymous and Authenticated.

2) Use a script to cycle through the my_views.views_default.inc file and change this:

$handler->override_option('access', array(
'type' => 'role',

To this:

$handler->override_option('access', array(
'type' => 'role',
'role' => array(
'3' => 3,
'4' => 4,

I've used python to accomplish this:

#! /usr/bin/python/


marker = " $handler->override_option('access', array(\n"

f1 = open('/blah/sites/all/modules/my_views/my_views.views_default.inc').readlines()
# f1 = open("/blah/myview_test.txt").readlines()
fo = open("/blah/new_view.txt", 'w')
for index, line in enumerate(f1):
if line == marker:

del f1[ index+1]
f1.insert( index+1, " 'type' => 'role',\n")
f1.insert( index+2, " 'role' => array(\n")
f1.insert( index+3, " '3' => 3,\n")
f1.insert( index+4, " '4' => 4,\n")
f1.insert( index+5, " ),\n")
print index, line

for item in f1:

except IOError:

print 'Can\'t open file for reading.'

3) Creating a module of your exported views if you haven't already. Otherwise, place the new views_default.inc file in that module directory. Enable the module and revert all views.

4) These views are hard-coded, will need to fix manually:


Posted by kkwaiser at 09:54 AM | Comments (0)

November 12, 2010

How to use the Version Control and Bulk Export features of Views

This "how to" focuses on the bulk export and version control functionality of the Views module. The following steps document how to export multiple views at once and create a module from the exported code that will allow you to revert any subsequent changes to your views.

- You know how to install and enable a module
- You have views you want to bulk export; we'll call them myView1 and myView2

- Views (specifically 6.x-2.x)

1) Install Views 2 and enable (/admin/build/modules) the following modules: Views, Views exporter, Views UI

2) Navigate to the Views Administration page (admin/build/views)
- You should have "Edit | Export | Clone | Delete" options next to myView1 and myView2. Once we're done, "Delete" will become "Revert"!

3) Navigate to the Bulk Export page to export myView1 and myView2. Click Tools > Bulk Export or browse to admin/build/views/tools/export.

4) Export myView1 and myView2
- Check the boxes to the left of myView1 and myView2
- Choose a module name: my_views
- Click Export

5) Create a module from the exports
$ cd sites/all/modules
$ sudo mkdir my_views
$ cd my_views

- Create the following files, paste in the appropriate code, and place them into the my_views directory:
my_views.module (add a my_views.views_default.inc (add a

6) Clear the cache (just to be safe!)
- Navigate to admin/settings/performance > Clear cached data

6) Enable the my_views module (admin/build/modules)
- the module name should be "my_views Export Module"

7) Navigate to the Views Admin page (admin/build/views) Verify that you now have version control abilities for myView1 and myView2.
- The right of these should now read "Edit | Export | Clone | Revert"

Posted by kkwaiser at 12:40 PM | Comments (0)

November 11, 2010

Notes on NSF Programs related to bio-research collections

Advancing Digitization of Biological Collections (ADBC) - "This program seeks to create a national resource of digital data documenting existing biological collections...The national resource will be structured at three levels: a national hub, thematic networks based on collaborative groups of collections, and the physical collections."

Home Uniting Biocollections (HUB) - forming the coordinating scientific team...oversee implementation of standards and best practices for the collections, plan for the long-term sustainability of the national resource, facilitate communication and standards for training, and assure that results are disseminated to the scientific community utilizing collections, the collections community, and other similar efforts internationally.

Thematic Collections Networks (TCN) - "will conduct the digitization effort at a number of collections...justified by a research theme...integrate with other ongoing digitization activities, such as the new collaborative networks funded under the Improvements to Biological Research Collections Program (BRC)...BRC-funded collaborative projects will be expected to become part of this national resource"

Improvements to Biological Research Collections (BRC) - Next due date: July 22, 2011

Posted by kkwaiser at 01:07 PM | Comments (0)

November 10, 2010

Setting up Workflow and WF Required Fields in Drupal 6

Need to establish a workflow for our REU Application.

1) Install Workflow and Workflow Required Fields
$ sudo drush dl workflow wf_required_fields
$ sudo drush en workflow wf_required_fields

2) Patch WF Required Fields to associated required fields with workflow states. In this case, I will set fields to be required before workflow state can go to "Complete". Use this patch.

- Move patch to WFRF directory and apply
$ sudo mv wf_required_fields.patch sites/all/modules/wf_required_fields/
$ cd sites/all/modules/wf_required_fields/
$ sudo patch -p0 < wf_required_fields.patch

3) Administer module permissions (admin/user/permissions) for Workflow
- Full rights to Administrator Role, nothing else

4) Add a workflow (admin/build/workflow/add):
- Title: Status of REU Application
- State: In progress
- State: Ready for review
- Permissions for these states: Author, Administrator, REU Applicant, Staff
- No Comments
- Workflow Tab Permissions: Administrator

5) Associate workflow with content type
- Content type: REU Application
- Workflow: Status of REU Application
- Display Workflow Form for: Post

6) Setup Workflow Required Fields (admin/build/wf-required-fields)
- Select REU Application
- Upon Ready for review - selected a bunch of fields here

7) Test notes:
- Although the Workflow status does not change if required fields are empty the Node is still created and error messages show up at the top of the screen. This is actually good because it will allow people to save and return to their application.
- I had to change the settings of a CCK fields so that they were not required to be able to set up WF Required Fields.

Posted by kkwaiser at 02:59 PM | Comments (0)

November 09, 2010

Setup up Private Downloads module in Drupal

Goal: Your Drupal site has a Public file system but you have files you want to limit access to. This is the Public/Private File system conundrum. The following demonstrates how to solve your problem using the Private Download module.

- A Drupal site with a Public file system pointing to "sites/default/files".
- CCK and CCK FileField modules are installed.

1) Understand your system:

- There shouldn't be a private folder in sites/default/files/
$ ls -al sites/default/files
- Are you operating out of a sub-directory? If your Drupal code is found in www.example.com/drupal then you're in a sub-directory. For the record, I'm in a sub-directory and I've set the base_url in settings.php to represent this.

2) Install and enable the Private Download Module

3) Configure Private Download settings (admin/settings/private_download)

- if you're in a sub-directory check the RewriteBase rule and prepend your directory name to it if it isn't there already:
RewriteBase /[optional-subdirectory]/system/files/private
- Save configuration

4) Check to make sure you now have a "private" directory with an .htaccess file

$ ls -al sites/default/files/private
$ ls -al sites/default/files/private/.htaccess

5) Specify which Roles can access files within the "private" directory. Go to admin/user/permissions to configure permissions

- Under "private_download module" check the "authenticated users" box. This will allow anyone who logs into your system to "access private download directory." Of course, you can create your own roles (admin/user/roles).
- Save permissions

6) Add a CCK FileField to a content type, let's call them myFile and myContentType, respectively.

- When configuring myFile go to Path settings > File path and specify "private" (without quotes.) Any files uploaded through this field will end up in the "private" directory.

7) Upload a file
- Create Content > myContentType
- Upload a file into the myFile field. Let's call it test_download.txt.
- Save

8) Everything should be setup now but you can verify everything works:
- Right click on test_download.txt, it should download (or open in browser)
- Left click on test_download.txt > Copy Link Location
- Logout
- Open a new browser tab and try either of these URL's (both should produce an "Access denied" error:


Note: The second URL should be re-written to the first in the browser.

Posted by kkwaiser at 04:01 PM | Comments (0)

Private/Public File Storage in Drupal

Oh, LOOK! There are modules that do this in Drupal. On with the song & dance:

Private Upload

Private Download

The latter two modules are the best options. Great discussion on the two in this forum. This comment outlines integration of CCK FileFields with Private Download via another module, FileField Paths.

Protected Download (Drupal 5 only)

Overall File management module (not Private/Public from what I can tell):
Upload Path

This looks like a helpful post > click me, click me!!

Another Link:

Posted by kkwaiser at 02:02 PM | Comments (0)

November 08, 2010

Multistep Node Add Forms in Drupal

Will return when I figure this out. For now, it's external links:

MultiStep forms in Drupal

MultiStep forms in Drupal using Chaos and Domination

Posted by kkwaiser at 08:16 PM | Comments (0)

Setting up the CAPTCHA Module in Drupal

There are several CAPTCHA and CAPTCHA-like modules in Drupal. I'm using this one. The module currently needs a co-maintainer but with 80000+ users I'm fairly certain it won't fall a part.

Here are setup notes. Also posted on D.O.

Just a bit of documentation...

- You know how to install third-party modules for Drupal
- You have a content type and you want to add a CAPTCHA challenge

1) Install and enable the CAPTCHA Module

2) Configure permissions (admin/user/permissions)
- Probably want to allow Admin role to skip and administer captcha

3) Administer module (admin/user/captcha)
- Check "Add CAPTCHA administration links to forms"

4) Add CAPTCHA to a node-add form:
- Go to Create Content > [My Content Type] or (node/add/[my_content_type])
- Click on "CAPTCHA: no challenge enabled"
- On new page "CAPTCHA point administration" select a challenge type > Save

5) Return to CAPTCHA admin page (admin/user/captcha), refresh, and notice new entry in the Form Protection section [my_content_type_node_form]

Posted by kkwaiser at 11:17 AM | Comments (0)

Patching the CCK Required by Role module

There's a bug in this module that causes it to fail when working with a date field. Basically, the bug keeps you from exempting a Role from completing required fields.

To fix:

1) Update to the dev version:
$ sudo drush dl cck_required_by_role-6.x-1.x-dev

2) Place patch file into the module directory. Patch available here.

3) Apply patch:
$ cd sites/all/modules/cck_required_by_role/
$ sudo patch -p0 < cck_required_by_role.module.patch

4) Check the results

Posted by kkwaiser at 10:35 AM | Comments (0)

November 05, 2010

Incorporating the NBII Thesaurus into Drupal

Here's an email from a colleague (i.e., I deserve no credit for this but I want to post it due to relevancy.):

Hi folks,

"Integrate the NBII Biocomplexity Thesaurus" (integrate in our Drupal work, that is)

The task was stated clearly in both our NSF supplement, and also in our LTER / NBII cooperative agreement scope of work.

We have been exploring this an related ideas every now and then, but upon stumbling in blocks, we moved on to more pressing matters.

Yesterday I expend some time revisiting the taxonomy_xml module -- to the risk of experiencing again that insidious deja-vu feeling, kyle knows what i mean, i explain this a bit later.

Using the taxonomy_xml dev version, and being open minded, I found a winning strategy to at least transform and integrate NBII's thesaurus into the powerful Drupal taxonomy structure.

The workflow is actually simple - install the module, and work with a particular CSV format that the module is successful doing imports (See ISO 2788.. well, not really, see better the snippet below). NBIIs thesaurus is served also as an XML dump, following the Lib of Congress schema for thesaurus (I believe...) I wrote a small Perl script to transform it into the importable CSV format - and voila.

50,381 triplets. Here is a snippet:
Zoledronic acid, Related Terms, Risedronic acid
Zoledronic acid, Related Terms, Tiludronic acid
Zona pellucida, Broader Terms, Ova
Zonal distribution, Broader Terms, Geographical distribution
Zonation, Used for, Ecological zonation
Zonation, Related Terms, Benthos
Zonation, Related Terms, Biogeography

This promising taxonomy_xml module (here is why deja-vu) fails to work for most of the stated functionality. During your careful module hunt and exploration, you feel frustration when most of the "stuff" does not work as expected. Part of it is that web services change --the risk of third party dependencies is felt again-.

After finding the right formula, all I had to do is flatten the NBII thesaurus XML file that Lisa Zolly provides (Giri too) and then ingest it. In doing so, I delved some more in the little used Drupal taxonomies -- I had no clear idea that you can actually express a thesauri in a Drupal taxonomy losing little of its relational structure. I forgot about the "Related term". The "Synonym" feature took us closer, but it is the ability to encode tighter relations between terms that adds to the well know Drupal taxonomical (hierarchical) structure.

Finally, I had to break the triplets file in 5000 line chunks to avoid PHP script timeouts. The verbosity of the script is surprisingly enlightening, it questions terms, etc. fun.

I installed the NBII thesaurus in a Drupal vanilla instance ( http://inigo.lternet.edu/vanilla ) Before moving it into a closer to production, Im asking myself how to effectively use this monster vocabulary? is this is too large - 50,000 tangled net of terms? should we only ingest the (preferred terms?) to provide that functionality (through views/autocompletes, for example?)

Should we do a NBII-lite ? how do we pack this vocabulary so it actually provides functionality and not outlandish suggestions?

If any of you have some time to wrap your head around any of these questions, please, Im all ears. I will proceed to do some functional tweaks too, but the road for NBII knb Thes. is now open - we just need to figure out how to navigate it better.


I couldn't help but reply:

Great stuff, Inigo. After playing with a different type of hierarchical vocabulary (species taxa info, in this case) a few things became clear to me.

1) We're selling ourselves short if we deploy vocabularies that aren't hierarchically structured because a lot of functionality within Drupal is immediately lost to us.

2) There are 250+ modules that extend Drupal's taxonomy system meaning the potential is nearly limitless but an up-front investment is needed to locate the best options (and identify what it is we want to accomplish.) The good news is that there are several use-cases for Drupal's Taxonomy system and a well-thought out solution should work for most or all of them.

Another question is how to deal with the need for multiple keyword vocabularies (i.e., NBII, LTER Keyword Dictionary and site-specific terms)? It may be possible to keep them within one Vocabulary where the ultimate parent term is the keyword source (e.g. NBII). I think terms can have multiple parents, meaning a term could exist within the LTER Dictionary and the NBII simultaneously. Of course, if that is true, it compounds the problem of having an absolutely huge Vocabulary...

Either way. I noticed that taxonomy_xml throws an error on the Status Report page if the RDF Module isn't installed. Can't say it will help the imports but it probably won't hurt.



Posted by kkwaiser at 03:32 PM | Comments (0)

November 03, 2010

Delving further into the Data Module

Created two new issues on drupal.org with the hope that I can resolve lingering questions around the Data Module. As TableWizard is kaput, Data is the only game in town.

Do I need a view reference module if I am using the Data Module?

Clarifying the relationship/reference capabilities of the Data Module

Posted by kkwaiser at 12:27 PM | Comments (0)

November 02, 2010

Changing MySQL table prefixes

Pretty self-explanatory. Download php file from this site, place in /var/www and browse to the appropriate URL.

Note: This will cause problems if some tables are prefixed and others are not.

Posted by kkwaiser at 03:48 PM | Comments (0)

November 01, 2010

Assess disk space usage in Ubuntu

The dual-boot install I am working out of is the first one I'd set up and the partition is quite small (30 gb). Last week, I finally filled it up (took over a year) and here is what I learned from the process.

To check your disk space usage from command line:
$ sudo df -h

The following command will show disk usage per directory (from this forum):
$ sudo du -sh /*
$ sudo du -sh /home/[user]/*

Cleaning up installer files may also help:
$ sudo aptitude clean

Sort by size:
$ sudo du -s /var/* |sort -n -r

Show hidden directories (see above link):
$ du -s */ .[^.]*/ | sort -nr

Posted by kkwaiser at 04:31 PM | Comments (0)

Finding new content in Drupal 6

There are (at least) two ways to find content after it has been created in Drupal 6:

1) Search for it using the search bar. Note, this will not always work because the search function only hits indexed content and brand-new content is NOT indexed. I've set it up so that the site is indexed every 15 minutes.

2) Go to Administer > Content Management > Content (admin/content/node) where all content will be listed. Notice the filters on top to help narrow it down.

Posted by kkwaiser at 11:57 AM | Comments (0)