Surveying, Mapping and GIS

Exploring all aspects of mapping and geography, from field data collection, to mapping and analysis, to integration, applications development and enterprise architecture...

  • Geospatial Technology, End to End...

    Exploring all aspects of mapping and geography, from field data collection, to mapping and analysis, to integration, applications development, enterprise architecture and policy

ESRI and Volunteered Geographic Information

Posted by Dave Smith On 2/20/2010 10:36:00 AM 7 comments

Once again, I greatly enjoyed this year's ESRI Federal User Conference - I was able to make it to several sessions Thursday and Friday... Perhaps will post more on this, as time permits.

As he has done before, Jack Dangermond solicited feedback and questions in his FedUC wrapup following Recovery Accountability and Transparency Board chairman Earl E. Devaney's excellent talk - I was happy to pitch in and asked him my own question about ESRI's vision for volunteered geographic information / crowdsourcing / participatory GIS...

This got Jack Dangermond excited, it seems he has been thinking about the concept, though even with his response, we need to get beyond the initial technical hurdles - and in talking to some other ESRI folks during and after the conference, I am happy to hear that some other ESRI staffers are thinking about this as well.

The thing is this: It's not really about ESRI tools importing OpenStreetMap, GeoRSS or Twitter feeds, and so on. Mere import and merge with your own data is really just a tiny part of it.

The true power of VGI is in its dynamic nature - for example, in the case of Haiti response, there were dozens of volunteers all providing concurrent updates, imports and edits to OpenStreetMap, as well as hundreds consuming the map data. The state of the map changed, sometimes radically, from hour to hour, and often even from minute to minute. As one example, an individual on the ground in Haiti sent an SMS message to Ushahidi with GPS coordinates for two locales where supplies could be airdropped or landed via improvised helicopter landing zone. The maps were blank in those two areas. Yet, within minutes, I and other OSM mappers pulled up the declassified DMA maps, DigitalGlobe and other imagery that had been donated by various providers, and sketched in the roads, trails, streams, buildings and other culture and planimetrics for those communities in need.

Leveraging those dynamic updates is one key piece to making the most of VGI. That means, going beyond import, to being able to consume and integrate the data on the fly.

The second piece is that VGI is not a one-way street. To use Haiti again as an example, dozens of disparate agencies are all using OpenStreetMap - several are in turn actively contributing back as well. Each is then building up on the work of the others, and the efforts of each resource leverages successive investments of the prior effort. This is particularly useful for resource-constrained organizations and volunteer efforts. As an example of this, as a member of Engineers Without Borders, I have been trying to promote adoption of OpenStreetMap for mapping efforts - e.g. one effort providing potable water can then dovetail into another organization's efforts to do health assessments, and so on.

As to cultural and organizational resistance to crowdsourcing, accuracy and reliability - that can be handled via record-level metadata. The double-edged sword of OpenStreetMap is in its use of key/value pairs for attribute data, as opposed to rigidly structured tables and columns, which on one hand can lead to folksonomies with inconsistent tagging, but on the other hand can handle rapid, flexible, ad-hoc changes to accomodate new needs, as well as allow complex representations via a collection of tags - which in turn can then also be cross-walked to other agencies' data models for interaction and ETL. While OpenStreetMap, on its' face, leaves much up to the individual contributors, best practices can and should be implemented. All edits are tracked in OpenStreetMap, which provides some basic metadata as to who and when, however more robust means of reverting adverse changes would be useful. Similarly, best practices are generally communicated via wiki, such as adding tags for source (e.g. digitized from DigitalGlobe imagery, with date). One of my comments in my followup to Jack Dangermond was that some of the governance/user guidance can be put directly into the tools, such as via JOSM, Merkaartor or Potlatch presets and templates.

Some may push back and suggest that shared community platforms like OpenStreetMap lack accuracy or reliability. The beauty of it is that if you don't like it, you can fix it.

And another goal... Dynamic integration of community platforms like OpenStreetMap beyond just base mapping and visualization, to be incorporated into modeling and analysis, via crosswalks and semantic interoperability.

I am happy to see this is something of interest to Jack Dangermond and ESRI, and hope that the bidirectionality and dynamic nature of VGI are fully embraced down the road.

Live Blogging from FedUC, Day 1 (er... not really)

Posted by Dave Smith On 2/17/2010 06:51:00 PM 2 comments

This will be as close as I get to live blogging from the ESRI Federal User Conference, Day 1.

Problem is, I missed it.

Unfortunately, as my day went, I was tied up in a meeting in Baltimore and didn't get back to DC until the tail end of the reception.

So that should make this the shortest wrapup of FedUC Day 1.

I heard there was some tasty sushi at the reception, but by the time I arrived, it was already gone... However, having not had anything to eat since morning, I did inhale a handful of various hors d'ouvres and a Heineken...

To catch up, I did monitor a few assorted tidbits via Twitter throughout the day- some of these struck me:
  • Buzzword: VGI. Volunteered Geographic Information. a/k/a crowdsourcing. All pointing to the OpenStreetMap paradigm- Given the phenomenal success of OpenStreetMap in supporting base mapping for humanitarian efforts in Haiti, along with the fluid and adaptable nature of its key/value pair model, which was utilized for tagging crisis-related features, such as landslides, collapsed buildings, road obstacles, refugee camps and so on - how can one NOT talk about OpenStreetMap? Great to hear about OpenStreetMap being used in ArcGIS. But... the pundit in me asks, how transparent is this? Is it "importing OSM data"? That's great for many applications. But what of emergency response applications? Given the dynamic nature of OpenStreetMap, "import" might not cut it. How about direct, native OSM support? That I need to investigate further. But then, comes the other, far more important piece of VGI - the participatory piece? Can/will ArcGIS 10 support direct editing of OSM? And outside of OSM, how much robust ArcGIS 10 capability for participatory GIS exists right out of the box?

  • Offshoot: Citizen-centric science. I understand Audubon demonstrated eBird - interesting - a year ago, I contributed a proof-of-concept to the eBio conference in London, demonstrating harnessing social media such as Twitter to allow citizen science participation to allow folks to record observations of various species in the wild, toward such ends as assessing biodiversity, invasive species, and so on. I tied this in with web services such as GBIF's lookup services to, for example, translate between common names and scientific name, and so on. This, in turn, tied in to other sources such as Flickr, and combined, wherever possible, with available geographic information, for providing feeds and display in, for example Bing Maps or other platforms. Great to see this coming along... I'd be interested in looking at the Audubon effort more closely, along with further exploring the model for vetting and validating inputs.

  • The Cloud... Evidently a big focus on cloud hosting, the new partnership between ESRI and Amazon, and "rent ArcGIS Server by the hour". I think this is an interesting model and have been harping on this for years. Geodata services hosting at this point is nothing esoteric, and could/should essentially be commoditized *cough* GeoServer *cough*. But... where the less-well-charted and more-interesting territory still lies is not in just serving up data. ArcGIS Server is frankly often too much tool to waste on just serving up map layers and tiles. Where I see the opportunity for ArcGIS server is in true ANALYSIS, MODELING and so on. Web-based geoanalytical services and geoprocessing services. We need a good model, and some good strategic thinking in the community of how the long-range picture of all of that will look.

  • ArcGIS for iPhone... That was another feature of ArcGIS 10 noted by some who attended the Plenary... Sounds great, but... I want to know more about how it operates, and more importantly, how customizable, configurable, how many features and functionalities it supports. Hopefully someone can shed some light there...
Even for showing up at the last minute, I briefly ran into @jfrancica, @donatsafe, @MikeHardy and @sturich from Pen Bay Media and got to say hi to them, as well as various other friends - I know there are plenty more friends, tweeple and geobloggers in town this week - I also did see @pbissett and @cageyjames from afar - on locating the @weogeo booth, it still had a few people milling about, so I unfortunately didn't get to chat...

More to follow... Tomorrow, my luck should be better, and I plan to be able to stay for the whole day - and for #geoglobaldomination afterward. Hopefully others will be posting their recaps, observations and scuttlebutt as well... At this point, the handful of hors d'oevres and beer in my otherwise empty tummy are looking for company.

ESRI Federal User Conference 2010

Posted by Dave Smith On 2/15/2010 09:15:00 AM 0 comments

I am looking forward to spending this week in Washington DC, despite any further threats of snow (1"-3" expected tonight)... Living in Northeastern Pennsylvania, 1 to 3 inches of snow is no big thing. But the 30+ that hit the Washington area last week are astounding. Hopefully, much of the prior accumulation is now under control.

Despite the snow, I laugh in the face of the snow and am still anticipating a decent turnout at the 2010 ESRI Federal User Conference (FedUC). At present, it sounds like a lot of my own federal colleagues are still planning on attending, as are various friends, geobloggers, geotweeple and so on... It has rapidly become the east-coast version of the San Diego ESRI International User Conference, with solid attendance not just by feds, but by a wide variety of others as well. The agenda, again a mix of technical GIS topics and where GIS is being used in a wide variety of business domains, along with a collection of special interest group meetings. I will generally be following an environmental science track, along with a few excursions into other areas.

It is being held once again at the Walter E. Washington Convention Center in Washington, DC, February 17-19th, although I will be arriving today, for various meetings around Washington today and tomorrow, and a few throughout the FedUC as well. Free for feds to attend, relatively cheap for others...

I hope to be able to post here and there from the FedUC, depending on connectivity and power availability (my workhorse laptop no longer holds a charge as it used to) - as well as the periodic blips from Twitter...

And if you are in the DC area, but are NOT attending the ESRI Federal User Conference, then here's a definite to keep an eye out for: #geoglobaldomination - essentially, just an ad-hoc, twitter-organized, vendor-neutral, platform-agnostic gathering of geospatial folks getting together over a few beers to discuss esoterics and idiosyncrasies of the geospatial business...

It will be fun!