The data dictionary has now been integrated into version .5 ! You will now be presented with a button to show/hide a list of attributes that the data dictionary associates with that shape file. You may then check which attributes you want and click the "Get Selected in KML" to have the site generate a KML file for the given boundries and selected attributes.

We now have an upload page that allows users to submit their own files to the browser. A user must provide their name, team, a password ('password'), a file, and a title for the file. Additionally they may enter a latitude, longitude coordinate pair, or an address. If neither lat/lon nor address are provided we will automatically assign the file the polygon matching their team. For lat/lon, we make sure that the coordinates are atleast within a reasonable range of Detroit. The address is graciously geocoded for free by geocoder.us. These entries will be shown on the visual browser in a shapefile block called Usersub. If we add usersub to the data dictionary then a more appropriate title can be displayed. One problem is that for some unknown reason we can't upload files yet. Once we can do that we can also generate links to the files for download. Yeah!

The next step is to implement the web data dictionary so we can integrate it as well and then, when we get it, integrate GeoServer (which works on the slow machine).

Current plan of attack:

1. Integrate the 2kml into the visbrowser so users can download stuff with a button click
2. build web version of the data dictionary
3. integrate web data dictionary with visual browser so users can select desired fields for download into their kml file. Come to think of it, I could even let them name the kml file.

Let's all take a big, deep breath as we remind ourselves of our appraisal criteria.

We want things that can go into the visual browser. This means images, charette presentations or publications, and technical notes.

Which brings us to my:

Questionable Content List
.exe files
news articles
documents leading to research materials
external reports
camera manuals (wtf)
illustrator files
photoshop files
.apr (ArcView Project File) files
excel spreadsheets

I'm going to go think about these. I'll be back at five.

ETA: It's five and I'm still not sure about these. My instinct (and this is very un-archival.. no-no) is to convert all psd documents into jpgs, trash the psds and be done with it. But is there content in the layer data that would be useful? Even if there is, how could that possibly utilized in the visual browser?

I will create a *very* truncated finding aid that will list, on the folder level (with some description where necessary) what we have.

Whatever is discarded will be moved to a "discard" folder within the parent folder before being tossed.

Here are some resources about GIS and Geospatial Data Preservation, from the same blog.

Strictly speaking (and okay, I haven't had a chance to look at the data yet) we're not dealing with GIS databases so most of this information is less useful for our current task. We're taking the scribblings and half-baked documents from a few design workshops and trying to figure out what will be salvageable to put into our visual browser.

There may be so little data that we can feel free to hand-appraise this, but I'm not sure. We're talking about years of workshops, and I have absolutely no trust in the creators' record-keeping systems. Their conventions may not be useful and may not be followed thuroughly. We also need to think about how useful some of this will be. Do we want photos if they aren't geo-tagged? What do we do with previous design ideas and how do we categorize them?

::Timewarp:: It's now 4:16 and I'm looking at the Charette data. Jen and I have made a list of what we do and don't want, as we go through and appraise this data.

We DO want:
Site photos (even if not geotagged)
Technical notes
Anything produced by the charette (CAD, drawings, presentations, final productions [charette books], et al.)*
Administrative documents (Doug Kelbaugh or Eric Duweuke or past coordinators)

We DON'T want:
Census data, unless modified
Drafts of documents

This list is subject to change!

*a side note -- geotagging presentations would probably come in version 2.0, and would be represented by a polygon encompassing the scope of the presentation.

More to come this weekend as I flesh out the appraisal plan and start junking material.

The next step is to load the data from our shape files into a new table listing all of our geometries and their respective shape file. Once GeoServer is running we could also return the data as layers to display in the viewer and provide links to download the datasets as KMLs.

It appears that I am going to do some Perl script writting, however, to get all the data into the same place (actually three, but anyway...). As it is, our importer spits out way too much data. To compensate I need to write a Perl script (my kingdom for BASH) to extract the weat from the chaff. Because some of our shapefiles have point data (historic points), others line data (roads), and still others polygons (parcels) we need to store the data in three database tables and query them seperately. I can then take all the results and compile them together. I can make the import data script detect which one it is and select the appropriate table from there.

While you can currently see a version of the map explorer that shows the WKT polygon geometry to query the database against, the live machine has no database to query against. As concequence I set up a Dreamweaver site that uses the XAMPP tool to run a temporary webserver on the Novell machine. This way the php script has access to the database that stores our data for me to run tests against. - The small problem with this is that you need to be logged in as Jen to make it work.

All in all, progress is being made. My next task is working with the temp webserver to see if I can get the php script to effectively query the database.

I was thinking that a simple process for the app could be:

• Display default map view
• User pan/zoom to site of interest
• click "Get Datasets"
• query database: get all layers that contain points in this geometry
• display list of applicable layers (shp files) to user
• User selects layers & styles
• Presses "Display data"
• Add layers to existing map & display (OpenLayers pulls layers from Geoserver)
• Provide link to download map as ShapeFile, KMZ, pdf, etc.

So far I can:

• load a default map
• Allow user to pan and zoom
• Click “Get Datasetsâ€?
• Display the NE and SW (lat, lon) of the map’s current window.
(see http://www.lib.umich.edu/nsds/visbrow/v1/map_explorer2.php)

Next up:

• Get a running copy of PostgreSQL with PostGIS
• Play with querying the dataset for applicable shape files

Some of the things the website needs to do are:
show images of maps (samples) - to act as identification points, link to the report pages
deal with the interrelationships between variables
image (representing the mxd file)
attributes
layers
and subjects

I'm thinking that the two most important elements (the first to tackle) will be getting the image and the attributes together. The title of the map should also be there somewhere (since people tend to attach title to image if there is one?)

Adding pointers to layers are an added bonus

Both require the user to set touring options to certain settings such as "camera range" (the distance between the camera and the ground). In the case of Perim Path the camera range must be between 1.5 Km and 2 Km. Whereas the Core Area Cache camera range works best around 300m. The difference in the camera range makes the detail of the images grainier, but sufficient for a general idea of this extended area.

They are by no means perfect although they do the job perfectly well. Any suggestions for improvement are welcomed.

At Monday's meeting, we established a project timeline that leads us through the charette (now a week and a half away -- ahh!) and to our final deliverable, a visual browser/navigator.

This week, we plan to think/work through three subtasks:

1. describing and organizing existing charette data
2. interviewing represenatives from the charette
3. fashioning some form of data orientation for charette participants

For the first task, we will go through each of the existing charette files and note the following on an excel spreadsheet that currently exists in the charette folder:
- a brief description of what this file contains
- the scope of the file (is this a note or a report? is this a segment of a building or a map of the state?)
- if a shape file, it should be opened in ARC and its attributes should be noted
- [my notes fail me here] note in/completeness of dbf headers
- if the file is named poorly (e.g. duuuuuuude.pdf or FINISHED.jpg) suggest a more descriptive name

For the second task, we discussed interviewing Jennifer, the IT guy (Bill?), studio advisors and previous student participants. We wanted to remember to ask the IT guy (let's call him Bill) about what equiptment will be brought to the charette, whether participants will have a wireless connection, and what kind of software participants will have access to. We also had a question about CAD files (their existence?). We want to get our hands on the books from previous charettes, as well.

For the third task, the data orientation, we want to make sure that we have a google earth flyover ready for participants. We also need to think about how to present the resources that represent points 3.1-3.5 on our lists -- even if we just provide a list of links, we should think about how to explain each tool's features and weaknesses at a glance (perhaps a table?).

We also discussed dates for putting together our visual browser, which I will post on our shared google calendar.

See you Thursday!
Maureen