Surveying, Mapping and GIS

The last several weeks have been quite hectic - busy on a number of fronts, which is a thankful thing, given the economy has slowed down a bit - but here is something quick that I wanted to share - I was awarded "Most Valuable Professional" (MVP) status by Microsoft for some of my ongoing work in Virtual Earth. Hierarchically, there isn't exactly a "VE MVP" program, but VE falls within Microsoft's broader Live Platform.

Recently there has been a great deal of discussion about "NSDI 2.0" - and yet it seems there is much confusion about what it is or isn't - and what we should do, or whether we should bother discussing it at all.

History
To step backward in time, it primarily deals with National Spatial Data Infrastructure from a federal perspective, as enacted through the Office of Management and Budget Circular A-16.

This document was originally issued in 1990, followed by Presidential Executive Order 12906, and then subsequently updated in 2002 (which incorporated EO 12906).

OMB Circular A-16 (as revised 2002)

In its present form, the NSDI (if you were to consider it NSDI 1.0) consists of:

• Defined data themes (geodetic control, orthoimagery, elevation, transportation, hydrography, governmental units, and cadastral information)
  
• Metadata (FGDC Metadata Format)
  
• The National Spatial Data Clearinghouse (Geospatial One-Stop)
  
• Standards (developed only when no existing voluntary standards exist, in accordance with OMB Circular A-119)
  
• Partnerships
  

For developers using Virtual Earth, the Virtual Earth API version 6.1 was released a few days ago:

Official SDK Documentation: http://msdn2.microsoft.com/en-us/library/bb429619.aspx

Other discussion:

• Steve Lombardi's Blog (Microsoft) on 6.1 Enhancements: http://virtualearth.spaces.live.com/Blog/cns!2BBC66E99FDCDB98!14129.entry
• Chris Pendleton's Blog (Microsoft) on 6.1 Enhancements: http://blogs.msdn.com/virtualearth/archive/2008/04/11/new-virtual-earth-api-release-virtual-earth-6-1.aspx
• Johannes Kebeck's Blog on 6.1 Enhancements: http://johanneskebeck.spaces.live.com/Blog/cns!42E1F70205EC8A96!3858.entry

Overview of changes: http://msdn2.microsoft.com/en-us/library/bb412561.aspx

Virtual Earth Map Control SDK, version 6.1

What's New in the Map Control?

Welcome to version 6.1 of the Virtual Earth map control. This page explains the new features for version 6.0 and 6.1 on a high level. To see a detailed list of the objects, methods, and properties that have changed, see the Version Changelist.

Version 6.0 and 6.1 of the map control includes improvements in the following areas:

• Enhanced accuracy
• Additional functionality
• Enhanced performance
  

Enhanced Accuracy

The Virtual Earth team is committed to constantly improving the accuracy of the map control. This release of the map control includes improvements in the following areas.

• Enhanced Geocoding. The map control integrates multiple geocoders and datasets to provide the most relevant and accurate results. You can perform these searches using the VEMap.Find Method.



• Rooftop Geocoding. Rooftop locations are the most precise geocoding results available in the United States today. Rooftop geocoding is now available through the map control VEMap.Find Method.



• Consistent Pushpin Accuracy. When switching between aerial and birdseye views, pushpin accuracy is maintained, delivering a more seamless experience. Use the VEMap.SetShapesAccuracy Method to get the accuracy you desire.

Additional Functionality

This release of the map control includes additional functionality in the following areas.

• Printing support. Use the VEMap.SetPrintOptions Method to enable printing support.



• Reverse Geocoding. Reverse geocoding allows the user to find places based on a specific point on the map. Use the VEMap.FindLocations Method to accomplish this.



• Traffic-based Routes. A new option allows the use of available traffic information in route calculations, enabling quicker routes and more accurate route times during heavy traffic periods. Use the VERouteOptions.UseTraffic Property to turn on this feature.



• Birdseye Map Style with Labels. A road label overlay increases the usability of the Birdseye map style. This map style is called BirdseyeHybrid and is a new member of the VEMapStyle Enumeration.



• Walking Directions. Provide users the option to walk to their destination by returning walking directions instead of driving directions. To return walking directions, set the VERouteOptions.RouteMode Property to VERouteMode.Walking.



• Multipoint Routing. A new method takes multiple points for a route instead of just start and end points, allowing for more complex trip planning. Localized directions are also available in this version. Use the VEMap.GetDirections Method to get a multipoint route.



• Bulk Addition of Shapes. A new method to add multiple pushpins in one call while maintaining high performance and avoiding performance slowdowns. Use the VEMap.AddShape Method to add multiple pushpins.



• Improved Shape Control. You can now specify how a shape object appears relative to other shapes or tile objects, providing greater control in viewing data and objects. Use the VEShape Class to create shapes.



• MapCruncher (Beta). MapCruncher Beta for Microsoft Virtual Earth makes it easy to publish maps overlaid in an application using the Virtual Earth map control. See the MapCruncher Web page for further information.



• 3D Altitude Settings. Altitudes for three-dimensional objects can now be specified in meters.



• Updated interactive SDK. A new version of the Virtual Earth Interactive SDK is available, demonstrating the new functionality of the Virtual Earth Map Control. See the Virtual Earth Interactive SDK Web page for further information.

Enhanced Performance

Version Changelist is available here: http://msdn2.microsoft.com/en-us/library/bb412440.aspx

Though I wasn't able to attend ESRI DevSummit 07, I did get some good feedback - my own people report on the continued push toward web-based GIS and server-based GIS and SOA, with SOAP, REST, JavaScript and AJAX APIs being made available, along with tremendous demand from the .NET developer community.

I also scoured the blogosphere and scavenged together a set of posts and resources - dig through these, as there are copious notes, slides and podcasts to be had.

In no particular order:



Very Spatial 3/28: Dev Summit Overview "VSTV Episode 12": http://veryspatial.com/?p=1486

Very Spatial 3/28: Daily Podcasts: http://veryspatial.com/?p=1485

Very Spatial 3/26: Daily Podcast talking to DevSummit Attendees: http://veryspatial.com/?p=1478

Very Spatial 3/24: Standards and Interoperability Podcast: http://veryspatial.com/?p=1475

Very Spatial 3/23: ArcWeb Services Podcast: http://veryspatial.com/?p=1473

Very Spatial 3/22: ESRI Support Team Podcast: http://veryspatial.com/?p=1471

Very Spatial 3/22: Keynote Session Podcast: http://veryspatial.com/?p=1468

Very Spatial 3/21: Keynote Comments: http://veryspatial.com/?p=1466

Very Spatial 3/21: Gartner Keynote: http://veryspatial.com/?p=1463

Very Spatial 3/20: Plenary Session Podcast: http://veryspatial.com/?p=1462

Very Spatial 3/20: Plenary Part 2: http://veryspatial.com/?p=1458

Very Spatial 3/20: Plenary Session: http://veryspatial.com/?p=1457

Very Spatial 3/18: Preconference Podcast: http://veryspatial.com/?p=1453

I am actually not afraid to admit I am one who likes Microsoft products, a .NET developer, a Microsoft Registered Partner, a user of MS Office, Project, SQL Server, Exchange and so much more - and one who generally likes their vision and model... but sometimes it seems a juggernaut run amok.

BUT... while I'm on the topic of ranting, yesterday I went out to pick up another laptop, and was amazed to find that all of the local retailers only carry boxes with Vista (and generally some flavor of Vista Home to boot) - and no XP on any of the boxes in stock.

I was one of the Vista beta users, and readily admit I did not have any serious problems with it, but nonetheless am not yet ready to be flung headlong into adopting the new technology for any production machines. And further, I don't want to be bothered with half-crippled "Home" versions of much of anything.

I was happy with the model of shipping machines with XP along with a Vista upgrade certificate, but was surprised to see that this seems to have come to an abrupt end.

The other concurrent Microsoft marketing annoyance I have is that IE 7 is being pushed... hard. It comes up in my Updates list pretty regularly - and this is another key piece of infrastructure that I am just not ready to migrate on.

Don't get me wrong - I am generally on the leading side of the curve when it comes to adopting new technologies, but slooow it down, Microsoft. Given the track record of previous Windows and IE versions, we need time.

One of the stories circulating in the blogosphere is the Microsoft initiative to circulate some Vista-loaded machines to select bloggers - more here:

I Started Something Blog

It's one thing to give out copies of Vista for evaluation - after all I participated in the beta as well - but another to give out fancy $2000 Acer Ferrarri laptops with it. As one might expect, the response has been mixed, but generally not favorable.

Bribery? Unethical? I won't comment on that, as I think such a blanket condemnation would be premature. What might be more interesting is to look at who they selected, and why. I, in particular, would be more interested if it's pursuit of buy-in by outspoken critics, or reward to those troops who supported Microsoft in the past, neutral honest-broker bloggers or a mix of all of these.

The imponderable annoyance du jour: Verizon Online. So we moved across town in anticipation of our pending sale... everything has gone pretty smoothly so far, all things considered. On the 13th, we switched the phone line over to the other house. Same number, same town, just a couple of miles southeast.

Verizon says, 5-8 day wait time to switch the DSL over. What the.... I already have the DSL modem, the account is already set up, et cetera, et cetera, et cetera.... many reasons why this should be a very simple administrative thing for Verizon.

GRRR.... I tell them that the wait is unacceptable. "Um, sir, that's the ready date that comes up in the system." GRRR... "you can use Verizon Online via dial-up in the meantime for free".

Oh, gee.

Meanwhile, Verizon Online dial-up doesn't recognize my username and password.

Another call to Verizon tech support and they say "Mr. Smith, our records show you don't have dial-up". GRRRR. So they set me up with temporary access.

Temporary access doesn't work either.

Meanwhile, I hook the DSL modem up, and it detects DSL, 864kbps downstream, 150kbps upstream.

BUT, it's unable to logon.

Verizon Online dial-up folks tell me dial-up won't work if this is the case.

Well, anyways, in the meanwhile the Verizon Online DSL folks call and tell us that it won't be ready until the 27th. Yea, half a month. No explanation.... just going to take that long.

Add to this, all of the times I tried dialing up their customer service department and navigating their voice response system - to have it a.) not recognize what I was saying (particularly considering that with as much traveling as I have done, I have as close to a non-accent as is possible in American English), and b.) getting dropped after waiting 5 or more minutes on the call... AND of course, "can you give me the phone number you are calling about" AFTER their system has already confirmed this?

Well, the windows are fogged up from the steam escaping from my ears... Time to grab a cold beer and try to relax.

In a recent blog post, Matthew Perry (PerryGeo) lashed out, reaffirming his reluctance to rely on spatial web services such as WMS. Given that the NASA JPL WMS server was evidently down, many users and applications which consume the JPL WMS services also had issues and outages.

Yes, certainly any distributed system can face issues with downtime, or worse yet, with services going away altogether. How to remedy this?

As a service provider, one of the things that providers can work toward is providing the users with some information with which the users can make informed decisions. Essentially, manage their expectations. In some arenas, particularly where commercial services are offered, this should be in the form of a formalized Service Level Agreement (SLA), as part of the contract.

Here, the provider can outline the parameters of the content itself (such as FGDC-compliant metadata, including accuracy, source, et cetera), and beyond each iterative data snapshot, provide information on data latency (how current is the data?) and how often will it be refreshed? From an infrastructure standpoint, some hosting parameters should also be set forth, such as availability and reliability - to set the ground rules on what kind of downtime is acceptable or unacceptable, what kind of data throughput can be expected, and so on.

This becomes a bit more of a challenge in other arenas, such as agencies where external budgetary, legislative and other pressures may impact how well an organization can support published services.

But nonetheless, if an agency is forthcoming and up-front with these constraints at the outset, it can allow its' users and stakeholders to plan and prepare accordingly - by writing their code to handle outages appropriately, by utilizing alternative services, and so on. It also can lead toward consolidation, augmentation, replication of valued services as external users and stakeholders lobby for support (via legislative initiative and budget) for these services. An agency might not be able to provide formalized parameters as a commercial service would be expected to, however providing users with some basic insight will go a long way.

As organizations gradually move more and more toward Service Oriented Architecture, and as variegated Web Services appear right and left, and increasingly become available lights-out, via a simple URL, it becomes increasingly easy to connect to and consume these services - however that ease of access and use is ultimately bound to cause some disappointment, when things change down the road. I suggest and recommend SLAs and service metadata toward managing expectations.

Scott Guthrie discusses the planned Microsoft AJAX roadmap, for providing improved "ATLAS" support for developers who want to pursue robust Asynchronous JavaScript and XML in their ASP.NET applications - it comes with multiple pieces, which have now been officially named:

• Client-side support - Microsoft AJAX Library - consisting of JavaScript components to support any browser, and any backend system.
  
• Server-side support - ASP.NET 2.0 AJAX Extensions
  
• And a Controls toolbox - ASP.NET AJAX Control Toolkit - these are free, shared source controls and components.

The scrappad blog posts about a matrix of ArcWeb APIs that was recently published by ESRI - unfortunately the original link posted on scrappad is broken, but I was able to locate the document here:

http://support.esri.com/index.cfm?fa=knowledgebase.whitepapers.viewPaper&PID=20&MetaID=1183

The table lists functionalities, and if they are currently supported by SOAP, REST, OpenLS, J2ME, and JavaScript APIs. It is interesting to note that virtually everything is supported by SOAP, followed by JavaScript, with REST, OpenLS and J2ME bringing up the rear.

As things move forward, it will be interesting to see how these geospatial web services play with regard to BPEL, orchestration and other emerging SOA efforts.

The New York Times reports on the results of Strong Angel III, which was an interagency exercise for emergency responders that just took place in San Diego. This training effort involved a mock virus outbreak, coupled with cyberterror attacks, which cut off phones, internet and power.

Initially, the ad-hoc wireless network that was established failed, due to overwhelmed circuits and overtaxed bandwidth however, through a systematic restarting of the network, segment by segment, the network was re-established.

One everything was finally up, data including imagery and mapping was able to be shared seamlessly, with participants using Google, Microsoft, ESRI, and Intergraph technologies. Apparently, one of the technologies utilized was Simple Sharing Extensions, which is standards-based, and built on RSS.

I am certain there will be many more take-homes from this exercise, which ultimately proved to be a success - I will certainly be one to check through their site: Strong Angel III.

I have several interesting things cooking in the next few weeks –

One we just had a kickoff for - Agency Best Practices, which follows much of what I have already been doing in Geospatial Enterprise Architecture, but which is all the more serendipitous in light of the Geospatial Line of Business effort by OMB. I have already been working on a concept of operations for transitioning to and implementation of reusable, distributed service components for geospatial business (SOA and other enterprise-level solutions). My previous effort will be expanded and merged with efforts by other Domain Experts in other arenas, such as Portal/SSO, Service-Oriented Architecture, ETL, and other technologies. I’m putting a draft together for May 15th.

The other is to perform another MetaCarta integration, this one to geo-enable a repository containing 50 million documents, for MetaCarta-powered geocoding and mapping. I will likely follow a similar model as the previous integration project, using ArcIMS and existing enterprise mapping as the base with a query interface, perform the SOAP call to the appliance, parse the response and plot them on an acetate layer. This one will be a new J2EE implementation from scratch, as opposed to the previous one, which was an integration into a legacy ASP/IIS-based application.

As I have been playing around with Virtual Earth / Live Local and its APIs recently, I came across this interesting technology preview: http://preview.local.live.com/ - what this site presents is street-level photos corresponding with a given location in Live Local, taken from a roving camera van outfitted with an array of cameras and GPS.

The idea isn't entirely new- here in Pennsylvania our DOT has an online VideoLog that is linked to a web-based GIS application: http://www.dot7.state.pa.us/ividlog/video_locate.asp - it provides front, rightward and leftward photos. What amazes me with the Live Local implementation is that they have significantly more density with respect to the distance interval between photos.

Personally, I could do without the cutesey race car cockpit thing the Live Local team has going on... but the navigation and ability to turn and look to the side is interesting.

On another note, Rick Hallihan reveals a peek, caught in a picture window, of the roving van hard at work for Microsoft on his blog:

If you see an odd-looking van running around in your neighborhood, say CHEESE... You definitely won't miss PennDOT's van:

Some signs that Microsoft is trying to catch up and seeing the importance of geospatial support- Paul Flessner, Microsoft senior Vice President of Data and Storage Platforms touched on it in CRN recently - From the CRN article:

Flessner expanded on the company's data-and-storage vision, saying that the data store of the future must handle sound and graphical data types as well as the more standard relational and non-relational text information.

"We've got to go past words and numbers and get to sounds and sights," he told CRN in an interview Thursday.

One problem is that "pattern matching is not there yet," Flessner said.

Microsoft's plan for a unified store to handle all these data types is still on, but timing is unclear. He expects more and more data--including satellite information-- to flow into stores from sensors and that will enable creation of richer applications.

"You will see an investment in spatial indexing, geometry libraries. I want to do a good job supporting ESRI and other geo-spatial guys and make a good library available so if you can't afford those packages do good spatial analysis without them," Flessner said.

(ESRI is a leading maker of geographic information systems and mapping software.)

When will that happen? The next four to six years, although he'd like to get more done "sooner rather than later."

At PDC last year, Microsoft Chairman Bill Gates said that the "unified store" vision, already promised and late, would come with the next SQL Server or "Katmai" release wave.

I spent part of the weekend toying with Virtual Earth and the somewhat underdocumented VE API. I'm currently looking for a good way to develop a way to replace an existing map interface that, while it's slick-looking and features interactive Flash, the base map is basically a static image- no scale dependent renderers, fixed text height and symbology, and the like. Add to this the current map has been decoupled from any meaningful coordinate system. Having not found any solid Flash-based alternatives that I can put my hands on, and though I like SVG, it introduces the issue of plugin support. Finally, I had been looking at the big three AJAX-oriented offerings.

I see that Schuyler Erle has also been evaluating these as well- with his Google Maps Hacks background, it's interesting to see his perspective. The things that I am still grappling with most are basic - I'd like to overlay my own polygon layer, and have the polygon symbology linked to a dynamic web service that updates once a minute.

Now I have seen some interesting things on the Google side - VGMap is one attempt to accomplish it, but the map behavior and quirks are still kind of rough...

But by far the most interesting thing I've come across is the Virtual Earth Shapefile Reader - it appears to use a web service that converts the .shp and sends it out as XML. The application can then call the web service and bring in the ESRI shapefile on the fly. It's close, but not quite, to what I need. But the feel of the map ends up quite nice.

If I'm missing any other offerings out there, or if anyone has suggestions, definitely drop me a line.

The announcement went out yesterday morning - President Bush has nominated a new CIO for EPA, to replace Kim Nelson, who left late last year. The new CIO is Molly O'Neill, who has been heading EPA's Environmental Council of States (ECOS) efforts within the Exchange Network for bringing data in from the states. To date, Linda Travers had been filling the post as acting AA. Hopefully O'Neill will bring her experience in what's involved in integration of data from external sources toward further supporting ongoing efforts for internal integration and SOA.

I have been working extensively on the Geospatial side, toward mapping and modelling as-is and to-be architectures for the entire horizontal sector of Geobusiness across EPA. "Line of Business" is becoming a major issue within OMB these days, and Geospatial processes may be part of that equation. I also has been preparing for a presentation on MetaCarta to Linda Travers - this may now become my introduction to the new CIO instead.

Our EPA GIS Center of Excellence team did manage to get a couple more people aboard, they are starting in two weeks, but we still have room for one more GIS developer.

The main thrust of the work involves GIS web applications (such as ArcIMS-based apps), along with GIS web services (OGC WMS and WFS) and GIS database development (Oracle 9i/10g / Oracle Spatial and SDE 9.1 environment)...

A full description is available here.