Surveying, Mapping and GIS

Exploring all aspects of mapping and geography, from field data collection, to mapping and analysis, to integration, applications development and enterprise architecture...

  • Geospatial Technology, End to End...

    Exploring all aspects of mapping and geography, from field data collection, to mapping and analysis, to integration, applications development, enterprise architecture and policy

OpenStreetMap

Posted by Dave Smith On 11/27/2009 10:20:00 PM 3 comments


Recently one of my little side pursuits has been playing with OpenStreetMap. I had tinkered with it a little bit some time ago, and decided to revisit it.


For those who have not yet looked at OpenStreetMap, it is an open mapping framework, which utilizes public domain and crowdsourced data: http://www.openstreetmap.org/

Currently, there are a number of excellent tools available for working with OSM, such as the Potlatch web editing environment (within OSM, click on the 'edit' tab and away you go - it provides basic tools for adding points of interest, moving and deleting items, and adding attributes to existing points...

For a quick start, help information is available via a wiki here: http://wiki.openstreetmap.org/wiki/Main_Page

With regard to other tools for editing OSM, I've been using Merkaartor in a Windows environment - http://www.merkaartor.org/ - several other users use a Java tool, JOSM - http://josm.openstreetmap.de/ - they both appear to have comparable feature sets.

Potlatch also provides some basic aerial photos, however Merkaartor and JOSM also allow users to work with WMS services, ESRI shapefiles and other sources of data toward fast digitization and capture of data.

On committing editing changes, new tiles are rendered rather quickly, allowing OSM to allow rapid development of base maps. Many people are also using GPS for track logs, which can then be uploaded to OSM as GPX to facilitate capture of trails and unmapped streets.

In many places, OSM is being used to create maps where there previously were none, such as in developing countries - this might serve efforts such as Engineers Without Borders - and I plan to revisit some of the GIS data I've collected up for places like Cameroon and Rwanda on behalf of EWB and look at getting it posted to OSM where appropriate.

Locally, I have been playing with the mapping for my own town - I found that it essentially just had some older TIGER data and Points of Interest (POIs) from USGS Geonames (GNIS). The road network came up jagged and inaccurate, many features missing, outdated and so on. Here's where local knowledge and feet on the ground comes into play in crowdsourcing.

I essentially started out with something that looks like this:
And have quickly been going to something that looks like this:
Here, I've been using orthophoto WMS services and other datasets for correcting streets and railroads, digitizing streams, putting in building footprints, parks, trails and amenities, and in just a short time am rapidly going to a useful and reasonably attractive map (note that this is still in progress). Further, the data can also be reused in a variety of ways, such as in Open Source routing services, using custom styling and symbology and so on.

I would highly encourage others to take a look and play with it: http://www.openstreetmap.org/ - however, with the caveat that it can be addicting...

Building a Headless Linux GeoServer Box

Posted by Dave Smith On 8/30/2009 08:40:00 AM 17 comments

I recently inherited some older machines and, to support some ongoing in-house experimentation I've been involved in, set them up as quick-and-dirty servers to help serve up geospatial data services - the approach I took was to build what are essentially minimal machines running linux in command-line mode, and then load GeoServer on them to serve the data - As I haven't blogged in a while, a friend suggested that posting a quick description of the mechanics of this might be a good thing to share for folks who haven't dipped their toes into Linux much.

As a disclaimer, I do not claim to profess guruhood when it comes to Linux or the other packages, this is not necessarily warranted to be a "hardened-and-tweaked" system for production, it's just some very quick and dirty steps toward standing up a headless Linux-based GeoServer instance. Note that this uses the default Jetty install - some folks prefer to run it under Tomcat, which is a different path.

So, I started out with the "minimal install CD" for Ubuntu 9.04, available here:

https://help.ubuntu.com/community/Installation/MinimalCD



Select a package appropriate for the CPU you are using - in my case, I chose Ubuntu 9.04 for 32-bit PC.

Burn the ISO and follow the prompts to install from the text-based installer as command-line interface (CLI). I essentially went with the defaults. You will want to have the machine connected to the internet so that it can identify and set up the network connection and grab any files needed during install.

Once you've installed a minimal version of Linux, you will be ready to configure and install the other goodies.


For remote administration, you may want to install OpenSSH- http://www.openssh.com/



The step for doing this is simple:

Log in to your Linux machine, and use the following command:

sudo apt-get install openssh-server

This will download and install the OpenSSH package. For folks new to Linux, sudo tells it to use superuser privileges and permissions, and will ask for the root password used when you installed Linux. apt-get install uses the Advanced Package Tool to search for, retrieve and install software packages for Linux - this makes installation of much standard software in Linux easy.

For remote administration, you'll want to know how to reach your machine on the network - you can get the IP address by using the ifconfig command, which will give results something like this:



If you use Windows as a primary OS for your other work, you can then access the box from a Windows machine using an SSH client. I usually use PuTTY: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html



From there, you can install PuTTY on your windows machine and then access the Linux box via command-line interface remotely for administration.

Plug in the IP address you got above:



and voila - you should be presented with a login screen for your linux box:



Tools like PuTTY are a great asset when it comes to administering boxes.

Side trip into remote administration aside, on to the REAL stuff: Installing GeoServer.

As a prerequisite, you will need to install the Java JDK - the GeoServer install page gives some recommendations, and here's how you would do it from the command line:

sudo apt-get install sun-java6-jdk

Next, you will need to do some configuration of the JDK

Define the default Java to use:
sudo update-java-alternatives -s java-6-sun

And set the JAVA_HOME directory - this is doable in a number of ways, you may or may not want to define it in /etc/environment. I really like 'nano' as an editor for command-line Linux environments and it comes pre-installed in the minimal Ubuntu 9.04 version.

sudo nano /etc/environment

Again 'sudo' makes sure you have an appropriate privilege level to write the changes.

In nano, you can navigate around in the file using your arrow keys. Insert the following:

JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.14/


nano is intuitive and easy to use, following the commands along the bottom of the screen, e.g. ctrl-O to write changes, ctrl-X to exit.

Now on to the fun stuff - installing GeoServer.

GeoServer isn't available via apt - so you will need to download and unzip it to install it.

To be able to use ZIP archives, sudo apt-get install unzip will provide that capability. Next, you can download GeoServer.

Decide where you want to put it - some folks put it in /usr/local, or /usr/share, or if you are just experimenting, you could even leave it in your home directory - if putting it /usr/share in you would cd /usr/share

To download it, the download location given on the GeoServer page is http://downloads.sourceforge.net/geoserver/geoserver-1.7.6-bin.zip

Thus, to download it, use wget -

sudo wget http://downloads.sourceforge.net/geoserver/geoserver-1.7.6-bin.zip

Then, unzip it

sudo unzip geoserver-1.7.6-bin.zip
and you should see the files extracting into a geoserver-1.7.6 folder.

Depending on where you put it and privileges held by the account you are using, you may also need to ensure you have ability to access and run GeoServer and that GeoServer can create any files it needs.

chown will change ownership, using -R makes it recursive through subfolders and files:

sudo chown -R geoserver_username geoserver-1.7.6 would change all files and directories to be owned by the user specified (geoserver_username as a placeholder).

You can list files using ls and navigate directories using cd.

You may or may not also then want to configure directories, such as defining the location of your GeoServer installation directory, e.g. GEOSERVER_HOME="/usr/share/geoserver-1.7.6" - again, you could do this using nano to edit /etc/environment - and there are also plenty of other ways to do this. You could also define other parts of GeoServer, such as GEOSERVER_DATA_DIR at this point as well - consult the GeoServer docs for details there... http://docs.geoserver.org/1.7.x/en/user/

Pretty much ready to run now... cd to the /bin directory under your geoserver install, e.g. cd /usr/share/geoserver-1.7.6/bin" and launch the startup script sh startup.sh and voila... You will see some program output scroll by,
ultimately resulting with an output line like

[main] INFO org.mortbay.log - Started SelectChannelConnector@0.0.0.0:8080 - this should tell you that the GeoServer Jetty container is up and listening for connections on 8080.

Now, open a browser, point it to your machine's IP address and enter it, pointing to port 8080 and the geoserver instance, e.g. http://192.168.2.125:8080/geoserver/ and after an initial "loading" screen you should get the GeoServer web interface:



And you are off to the races... Confirm that it works via the demos:

OpenLayers NYC Tiger map

Again, this is just meant to be a quick-and-dirty guide - enough to make even someone with minimal Linux experience armed and dangerous - and from here, there are many tweaks and customizations that can be made, such as optimizing performance, hardening and security and so on (there are plenty of discussions around the web and on listservs regarding this)- but I figured, I'd at least share this as a quick start for anyone looking to play with GeoServer in a minimal Linux environment...

National Environmental Information Exchange Network

Posted by Dave Smith On 4/11/2009 06:54:00 PM 2 comments

Here's another exiting bit of news - my firm is teamed with CGI Federal on USEPA's Software Engineering & Specialized Scientific Support (SES3) Contract, and we just got word that our team has won EPA's Central Data Exchange (CDX) task. This is very exciting news, CDX and the Exchange Network serve the community via facilitating exchange of a wide variety of environmental data between federal, state, tribal and other partnerships - it is a partnership that has proven itself to be tremendously effective and a great model for other types of data exchanges as well.

What I am particularly excited about is in leveraging the infrastructure that has already been built toward more robustly supporting geodata services, and ultimately toward enhanced reporting, metrics, analytical capabilities, and other capabilities to support feds, states, tribes and others in informed decisionmaking toward environmental policy and stewardship.

As such - we also anticipate we will be looking to grow as a company, and will be looking to hire additional technical gurus with capabilities in data exchange, data management and data flows, particularly if you have prior capabilities and knowledge of EPA's Exchange Network and CDX, and/or geospatial technology.

If you are interested, drop me a line at dsmith (at) synergist.tech.com

Microsoft Virtual Earth MVP

Posted by Dave Smith On 4/11/2009 05:58:00 PM 5 comments

The last several weeks have been quite hectic - busy on a number of fronts, which is a thankful thing, given the economy has slowed down a bit - but here is something quick that I wanted to share - I was awarded "Most Valuable Professional" (MVP) status by Microsoft for some of my ongoing work in Virtual Earth. Hierarchically, there isn't exactly a "VE MVP" program, but VE falls within Microsoft's broader Live Platform.


While I am quite thrilled and honored to be recognized by Microsoft, this is not to toot my own horn - but rather to get the word out to fellow geospatial developers that this is a great program, and to encourage them to look into it if they are doing integration work with VE.   Only a week into it, I am finding that the program provides a tremendous amount of outreach, technical resources and other things, such as teleconferences, message boards, newsletters and other great technical information (awardees are required to sign and comply with an NDA) and best of all, a complimentary subscription to Microsoft's Developer Network (MSDN).  There are also other great nontechnical benefits, such as $150 in credit at the Microsoft store and others which I am only just beginning to explore.

The way it works is that they receive nominations on a periodic basis - folks with talents in a given Microsoft technology can have friends nominate them - I would recommend putting together a portfolio of projects and related items, demonstrating innovation and engagement in the community, such as technical blog articles, engagement in online technical forums, any pro-bono work, and so on, and ask folks who are already MVPs to nominate you for your work.  It's a great program, and a great idea from Microsoft toward promoting and evangelizing their platform that they are supporting their developer community so well.  

With ESRI now partnering so tightly with Microsoft and supporting VE and Silverlight integration and other MS-oriented capabilities, this type of program is something that would be fabulous to see from them as well...  (hint to Jack Dangermond...)

A Blast from the Past

Posted by Dave Smith On 3/31/2009 07:38:00 PM 10 comments

Some late night discussion with @GeoBabbler (Bill Dollins) and others last night led to this little flight of fancy...  Bill was wondering if there were still any old copies of ArcView 1.0 around.  Being a GIS geezer and a bit of a technology packrat, it turns out I still had a copy.  Scary.  As @FantomPlanet suggested, I reckon I have a bit of a Museum of GIS Antiquity going on - I believe I actually still have ancient copies of MapInfo, Atlas, GeoMedia, AutoCAD Map 1.0 and others floating around, plus quite a bit of hand-coded stuff from the days when COTS GIS was not even widely available.


Scarier yet, I still have copies of MS-DOS and Windows 3.1 around.  

And...  to top it off, I was actually able to lay my hands on them and try things out again.

Actually, what I did was stand up a VirtualBox instance and load up Windows for Workgroups 3.11 (ArcView 1 does not run on Win95 and up) and lo and behold, I then got ArcView 1.0 loaded up.

Prepare to enter the time machine and go back over 15 years into the past:
Above:  Note the MDI interface
Above:  Supplied "neweng.av" Tutorial data with choropleth mapping of New England...

Above:  Version "1.0a"

Above:  "maplewd.av" Tutorial data...

Supported data types: Arc/Info coverages, workspaces images (note .bil image supplied as part of the AV1.0 tutorial data), address coverage...  Note also that shapefiles are NOT supported.


An interesting look at the past. A capable GIS viewing and querying tool, though no editing capability was supplied with 1.0. So... Why would one want to load up ArcView 1.0? You probably don't - probably best to live vicariously through my little adventure here. But... if you ever do, VirtualBox provides some dicey support for it (I did still get the divide By zero error initially, but eventually it started working) - and, you can actually still download a copy of 1.0 here: ftp://download1.geocomm.com/sd2/ARCVIEW10.ZIP and if you still have a copy of MS-DOS and Windows 3.1, you should be good to go.

Fun stuff!  Brought back some memories...

[edit - removed the hyperlinks from the images (and thereby the preview snapshots, per comment below)...  not sure why blogger wants to insert them by default anyways...]

Search