Surveying, Mapping and GIS

Exploring all aspects of mapping and geography, from field data collection, to mapping and analysis, to integration, applications development and enterprise architecture...

  • Geospatial Technology, End to End...

    Exploring all aspects of mapping and geography, from field data collection, to mapping and analysis, to integration, applications development, enterprise architecture and policy
Showing posts with label web services. Show all posts
Showing posts with label web services. Show all posts

Think Globally, Code Locally

Posted by Dave Smith On 5/23/2010 10:35:00 AM 5 comments

One of those key philosophical notions that circulates among many stewardship and advocacy groups is to think globally, act locally - The same notion can be applied in best practices in solutions architecture, requirements development, and coding principles, where it applies to so many things - in terms of true enterprise architecture and integration approaches, in terms of data stewardship, and for public sector agencies, in terms of transparency and open government.

Each and every day, folks craft queries for accessing complex data, they write business logic for analysis and so on - and more often than not, they then wrap it up with a nice glossy user interface.  MVC and many other best practices applied, check...  But here is where another key opportunity to really open things up is lost.

Let's take the example of state or federal agencies - many of them are doing great things, in terms of publishing datasets (Open Government IntitiativeData.gov, Geospatial One-Stop, State Cartographers' Offices and Clearinghouses, and so on).  They are doing great things in terms of reporting tools, web queries and lookups, and mapping and visualization tools.  But...  As wonderful as all of these efforts are, imagine how powerful it would be if these could be linked and integrated, from agency to agency?  Imagine, for example, applications pulling in data from NOAA, NASA, EPA and other agencies?  Imagine state environmental agencies pulling in information from federal environmental agencies - the power of open government and transparency begins to multiply and become more and more powerful by orders of magnitude.

So what are we talking about?  Web services and APIs.  Not even anything complex, it could be as simple as REST and JSON, as in the space of public-facing applications, for a vast majority of data-centric or analytical applications, we are generally talking about read-only, non-transactional, nonsensitive data.  Why web services or APIs?   Isn't something like publishing datasets to Data.gov enough?  Publishing datasets to someplace like Data.gov is fine - but consider this - if someone downloads your data and integrates it into their own standalone application, what control do you then have over it?  Precious little.  When you refresh your database afterward and post a new dataset, is there any assurance that they will come back and get a fresh copy and keep up to date, or do you begin to incur a liability of your data being out of sync as it is represented in someone else's public-facing application?  Absolutely.

If, on the other hand, you post an easy-to-integrate RESTful service or API, you make it far easier to have your data current.

Criticisms?  Yes, there are many, but we can look at those as well - some might push back and say, "is there sufficient bandwidth and infrastructure for hosting those web services?"  The mere fact of web services in and of themselves generally incur no more new bandwidth or infrastructure demand than the public-facing web applications that you are already building.  So if that's really a concern, then maybe you shouldn't be building ANY applications.  The argument is generally a red herring.

Security concerns?  Same deal - you are already exposing the data and business logic via your web applications - if you think the web service poses new security concerns, then perhaps you haven't adequately thought through your EXISTING security.  So another red herring.

"They will come and take all our data!" - Again, why would they, if they know they can grab it dynamically and that you will be assured of their reliance on them by continually posting data refreshes?

What about poorly-behaved queries against your web service that might bring your database to its' knees?  Same thing - those are things that really should be handled in the logic for your existing application.  Better yet, think ahead a bit more and provide options - query filters, such as by a data attribute or domain, bounding boxes for geodata, and so on.

So - in conclusion, this type of thinking can up developers' approaches to how they deal with security, queries and filters, and so on, and generally tighten things up - but in the meantime, also provide a tremendous opportunity for data sharing and integration, whether within an agency, company or office, fostering collaboration across departments and programs, or outside of one's own organization, fostering collaboration across agencies and levels of government, across industry, academia and so on.

An individual organization can try all they like to anticipate all of the public's thoughts, needs, interests, they can try to walk in their stakeholders' footsteps - but ultimately, one can never fully anticipate all possibilities, and there are a lot of bright people out there with many untapped ideas - so why not open it up and facilitate that larger collective brainstorm of ideas by providing services ready to integrate?  Let's take things like the Open Government Initiative and other good things ongoing in government and elsewhere to the next level - dynamic, services and resources oriented architecture...

Seeking Web GIS Applications Developers...

Posted by Dave Smith On 8/08/2007 09:18:00 AM 1 comments

For the mashup artists, AJAX gurus, ArcIMS experts, .NET jockeys out there...
Synergist Technology Group, Inc. is currently looking for GIS web applications developers, for its Pennsylvania and Northern Virginia locations.

Requirements:

Strong background in web-based GIS applications development.


  • DHTML, CSS, and JavaScript;

  • ASP.NET, C#, VB.NET, or Java

  • ESRI ArcIMS

  • Web Services (XML, SOAP, JSON, REST, OGC)

  • Understanding of spatial databases
Desired Skills:

ArcGIS Server, Oracle Spatial query and analysis, mashups and exposure to Microsoft Virtual Earth, Google Earth, Google Maps, good documentation and communication skills.

Additionally, a background in emergency response, transportation, facilities management or environmental science may be very useful.

Applying:

Applicant must be a US Citizen or have US Permanent Resident Status

Interested applicants should respond with a current curriculum vitae and salary requirements to dsmith[@]synergist-tech.com.

Synergist Technology Group, Inc. - A Service-Disabled Veteran-Owned Small Business (SDVOSB) specializing in geospatial technology, environmental science, transportation and emergency response.

Jazz on a Sunday Afternoon

Posted by Dave Smith On 3/04/2007 10:00:00 AM 0 comments


I added a new thing to my blog sidebar - I downloaded a nifty gizmo from last.fm, called audioscrobbler. It lets me share what I'm listening to with others, by providing a web service that feeds current tracks being played in WinAmp, Windows Media Player or others.

Growing up, we always had eclectic tastes in music, and there was always some jazz in the house - Oscar Peterson, Dave Brubeck, and others... I have since acquired a taste for bop and funk-flavored jazz, Hammond B3 organ, sax, guitar, and other goodies - I enjoy Medeski, Martin and Wood, Hank Mobley, Cannonball Adderley, Wes Montgomery, Papa Grows Funk, Stanton Moore, Galactic, Art Blakey and the Jazz Messengers, John Scofield, Jimmy Smith, Jack McDuff, Joshua Redmon, Eddie Lockjaw Davis, and quite a few others...

I found that when I was studying, writing papers, and other kinds of intellectual pursuits, with many late nights listening to WRTI / Temple University Radio (especially B.P. with the G.M. and others...) and when trying to put myself in a groove to get things done, Jazz always fit the bill. Instrumental, my mind gets into the rhythm without being distracted by lyrics...

I have a collection of well over a thousand CDs, and also love being able to tune WRTI in when I am on the road, and another great thing is Pandora, which lets me create a customized radio station, designed after my own particular tastes... If you enjoy jazz, tune in...

MetaCarta JSON API

Posted by Dave Smith On 2/26/2007 08:41:00 PM 0 comments

After a great meeting this afternoon, I am looking to rehash some of my previous MetaCarta work. Looks like some of the previous kinks that I was running into with using a SOAP call to the HeadNode service originally developed on MetaCarta 3.0.1 have been worked out in newer versions (we are currently standing up two boxes with 3.7), and further, it may be a good opportunity to look at the JSON API that MetaCarta has developed.

I am also anxiously awaiting some of the document focus and other enhancements that MetaCarta has coming down the pike later this year. Other things I'd like to look at are streamlined, enhanced and more-focused document crawling.

W3C, REST vs. SOAP, and where are we headed?

Posted by Dave Smith On 2/22/2007 04:28:00 PM 2 comments

Looks like some major battles are heating up... There is an upcoming W3C workshop: Workshop on Web of Services for Enterprise - and it is stirring up quite a few emotions and arguments and throwing them out onto the table for debate.

Is it to be a battle or reconciliation between REST and SOAP/WS-*?

Here's one take on it, from Jérôme Louvel: Will we reconcile REST, WS-* and SOA?

Or is the bigger question, what W3C's role should even be? Nick Gall of Gartner put a few pointed, incendiary statements into his position statement, which are drawing quite a bit of attention:


It is my position that the W3C should extricate itself from further direct work on SOAP, WDSL, or any other WS-* specifications and redirect its resources into evangelizing and standardizing identifiers, formats, and protocols that exemplify Web architectural principles. This includes educating enterprise application architects how to design "applications" that are "native" web applications.

It appeared that there are also a few drawing a line between "World Wide" and "Enterprise"... Or suggesting that W3C should abandon aspects of their pursuits and leave it up to industry.

Certainly the geospatial community will need to sit up and take note of where this leads... Having written my own WFS and WMS services and clients from scratch, though I am still no Web Services guru, I still had to wonder about the wisdom of the approaches being used in the whole paradigm. Are the current OGC standards really in line technologically with the vision that was originally expressed by Tim Berners-Lee? I think not. Are they compatible with some of the security and other needs of the community? I think not. Not the end of the world, but certainly much iterative refinement to get us where we ultimately need to be.

Stefan Tilkov at InfoQ provides a good roundup of many of the position papers. Will we see some simplification and resolution to the ever-emergent convoluted forest of Web Services and Standards?



FedUC Thursday - Enterprise Service Bus?

Posted by Dave Smith On 1/11/2007 09:18:00 PM 0 comments

Just got home from FedUC and see snow on the ground here-

I actually ran out of room in my "little blue book" with all the thoughts and notes from the conference. Very productive, all in all.

I reckon I can probably share a few more of the thoughts going through my mind...

I followed some of the Enterprise Architecture track today... SAIC gave a presentation on DHS and their notional architecture, which was interesting, and applicable to where we are and where we want to be over at EPA. Their model consists of a foundational layer of geospatial data, harvested via ETL, consumed via web services, et cetera - essentially static data. Next, an OLAP layer, of analytical and modeling tools, and finally realtime, streamed and dynamic data. These are to then plug into an enterprise service bus, for consumption by clients which can make use of the BPEL, flows and integration platform provided by ESB.

We currently need an integration framework as well - we have been pursuing a few things in deconstructing and decomposing EnviroMapper into constituent parts, aligned with functional needs, to get them ready for this type of thing, but is ESB and BPEL really ready?

Now, here, Mark Eustis from SAIC is viewing OGC as the world's "virtual service bus". Is this really true? Are OGC services really up to the challenge - and further, ready to be plugged into ESB? Some say no. Time shall tell.

In another Enterprise GIS session, an application was demoed, using ArcGIS Server and IBM WebSphere Process Server as the ESB. ESRI does have ESB in mind for AGS, however here we are still ESRI-proprietary, which doesn't look good for mix-and-match map services in a dynamic application. What about WFS-T and transactional services?

I see I have much to learn about ESB. Seems exciting, but is it really ready for primetime? Our own pursuits of an integration platform are on hold in the meantime... but that doesn't mean we shouldn't still focus on build-out of services and look at the possibilities as things continue to mature...


Technorati tags:
, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Search