Surveying, Mapping and GIS

Exploring all aspects of mapping and geography, from field data collection, to mapping and analysis, to integration, applications development and enterprise architecture...

  • Geospatial Technology, End to End...

    Exploring all aspects of mapping and geography, from field data collection, to mapping and analysis, to integration, applications development, enterprise architecture and policy
Showing posts with label Geospatial. Show all posts
Showing posts with label Geospatial. Show all posts

Federal Geospatial Platform

Posted by Dave Smith On 6/25/2010 01:31:00 PM 0 comments




"In 2010 and 2011, Federal data managers for geospatial data will move to a portfolio management approach, creating a Geospatial Platform to support Geospatial One-Stop, place-based initiatives, and other potential future programs. This transformation will be facilitated by improving the governance framework to address the requirements of State, local and tribal agencies, Administration policy, and agency mission objectives. Investments will be prioritized based on business needs. The Geospatial Platform will explore opportunities for increased collaboration with Data.gov, with an emphasis on reuse of architectural standards and technology, ultimately increasing access to geospatial data."
- President’s Budget, Fiscal Year 2011
Earlier this week, I had the opportunity to attend one day of the multi-day National Geospatial Advisory Committee (NGAC) meeting in Shepherdstown, WV - Though I had attended a few of their meetings previously, the thing that I was most interested in for this particular meeting is discussion of the emergent Geospatial Platform.  As the NGAC is a FACA committee, non-member involvement and participation is limited, though was able to get to get a sense of things...   The intent has a lot of potentialities, such as being able to provide more robust visualization and geospatial tool capabilities such as geocoding services, to augment ongoing Data.Gov efforts, as well as support ongoing FGDC initiatives.

The sense I got is that thre are many pieces still undefined, many things not yet prioritized, and so on, and discussion of options and alternatives was a big focus of the discussion that I sat in on - and one big challenge yet ahead is refinement of governance structure - as to how its' strategic vision and board of directors would be structured and at what level of government, where its' managing partner and operational/tactical component would reside.  These things are being discussed in iterative fashion, as they develop their roadmap.

Some interesting thoughts and ideas, not new, but again being raised - of the need for a strategic ownership stake in geospatial data at the White House level, e.g. a Federal GIO to reside within OMB, who would have the authority and teeth to centrally direct and manage geospatial policy in executive agencies at the federal level.  Another question, at the operations level, of commoditized geospatial technology platforms which could be shared across all federal agencies - a federal geocoding service, for example, or a geospatial data services hosting platform where individual stewards in agencies which might not have their own robust geospatial infrastructure could upload data and register it, apply symbologies, et cetera to be served as fed-wide robust, shared OGC web services, and so on.

What shape this will take is yet to be determined, and it sounds like the plan is to firm up the roadmap internally within the NGAC and then circulate it more broadly when v3 of the Roadmap is completed.

More about the Geospatial Platform:


Other coverage, items and discussion:

U.S. Environmental Protection Agency...

Posted by Dave Smith On 4/05/2010 05:52:00 PM 4 comments


My big news to report:  After 6 years at Synergist Technology Group, Inc., I am changing directions.  I have accepted a position at the U.S. Environmental Protection Agency's Office of Environmental Information.  I will be starting at USEPA the week of April 26th, 2010.

I greatly enjoyed the work I was doing at Synergist, having been involved in everything from business development to writing code and everything in between, supporting geospatial, mapping and integration efforts to support diverse business areas from environmental protection to emergency response to transportation to military and intelligence applications.  I enjoyed the diversity.

I have worked in private sector, consulting for over 20 years, but all throughout this, have always been solutions-focused, and have always had a keen interest in supporting the core mission of the agencies that I worked with - though, the downside of private sector consulting is always in the vagaries of its' occasionally fragmentary nature, with competetive bidding and teaming arrangements often dynamic and changing.

So, coming into Federal service brings a whole gamut of new things.  In some ways, I will get to dig deep into some of my core passions of environmental protection, geospatial technology and integration - and I look forward to this. However, it will surely bring many other challenges along with it - but I am up for any challenge.

My new role will, among other things, involve support to EPA's Facility Registry System (FRS), which is a repository for facilities data for 2.5 million unique facilities regulated under a variety of environmental laws and statutes. Herein, I see opportunity for further geo-enablement of Agency business processes, along with continued integration potential, to put hooks and eyes into a variety of other processes for ever-improving analytical, reporting and querying capabilities, cross-media, cross-agency, cross-domain, cross-dimension, as well as facilitating further partnership-building, between local, state, and federal government, tribes, academia, industry and the public, and supporting data publishing, OGI and transparency initiatives.

I anticipate I will also be pulled into a variety of other geospatial efforts across EPA as well... In many ways, I will essentially continue to do many of the things I already did - only, from the other side of the fence. In terms of physical logistics, I have already essentially been commuting to Washington DC for 6 years - though, now having a permanent duty station there, I can now consider getting a place to stay, as opposed to living out of hotels and bouncing between various locales as I have been.

So - mixed emotions - I will certainly miss working with Synergist, but likewise, there is a great team at EPA, and I am sure I will have a great time there.

ESRI Federal User Conference 2010

Posted by Dave Smith On 2/15/2010 09:15:00 AM 0 comments

I am looking forward to spending this week in Washington DC, despite any further threats of snow (1"-3" expected tonight)... Living in Northeastern Pennsylvania, 1 to 3 inches of snow is no big thing. But the 30+ that hit the Washington area last week are astounding. Hopefully, much of the prior accumulation is now under control.


Despite the snow, I laugh in the face of the snow and am still anticipating a decent turnout at the 2010 ESRI Federal User Conference (FedUC). At present, it sounds like a lot of my own federal colleagues are still planning on attending, as are various friends, geobloggers, geotweeple and so on... It has rapidly become the east-coast version of the San Diego ESRI International User Conference, with solid attendance not just by feds, but by a wide variety of others as well. The agenda, again a mix of technical GIS topics and where GIS is being used in a wide variety of business domains, along with a collection of special interest group meetings. I will generally be following an environmental science track, along with a few excursions into other areas.

It is being held once again at the Walter E. Washington Convention Center in Washington, DC, February 17-19th, although I will be arriving today, for various meetings around Washington today and tomorrow, and a few throughout the FedUC as well. Free for feds to attend, relatively cheap for others...

I hope to be able to post here and there from the FedUC, depending on connectivity and power availability (my workhorse laptop no longer holds a charge as it used to) - as well as the periodic blips from Twitter...

And if you are in the DC area, but are NOT attending the ESRI Federal User Conference, then here's a definite to keep an eye out for: #geoglobaldomination - essentially, just an ad-hoc, twitter-organized, vendor-neutral, platform-agnostic gathering of geospatial folks getting together over a few beers to discuss esoterics and idiosyncrasies of the geospatial business...

It will be fun!

Pennsylvania Geospatial Coordination Council

Posted by Dave Smith On 1/18/2010 09:09:00 PM 1 comments

Last week, I attended a discussion session sponsored by the Pennsylvania Mapping and GIS Consortium (PaMAGIC) and facilitated by John Palatiello, regarding standing up a Geospatial Coordination Council as a formal governmental advisory body for Pennsylvania. This is not a new concept, it is something that PaMAGIC has been pursuing for several years, it has been discussed several times prior at the PA GIS Conference, and has been introduced in the legislature several times, in various incarnations. It was last introduced in 2007-2008 by Representative Russ Fairchild as House Bill 1304.

The discussion session was held at the Pennsylvania State Association of Township Supervisors' office just outside Harrisburg, and roughly 60 people attended in person and by phone - an excellent cross section of state government, county and local government, private sector, utilities and others. I did not, however, see any representation by the Pennsylvania Society of Land Surveyors or Pennsylvania Society of Professional Engineers. Similarly, I do not see a spot designated on the proposed Council for these organizations.

I have posted two documents - one is a summary of the meeting, the second is the newest version of the proposed bill - the GIS community is invited to provide comment, by the end of this week, to PaMAGIC President Glenn McNichol, gmcnichol -at- dvrpc.org.


What does give me more optimism in this iteration is that there appears to be additional championing of this cause coming from the Governor's office. It is anticipated that Rep. Fairchild will be introducing the bill again, and I am hoping that the geospatial community will be able to rally around the effort, hopefully PaMAGIC will spearhead a good communications campaign with John Palatiello's help.

Building a Headless Linux GeoServer Box

Posted by Dave Smith On 8/30/2009 08:40:00 AM 17 comments

I recently inherited some older machines and, to support some ongoing in-house experimentation I've been involved in, set them up as quick-and-dirty servers to help serve up geospatial data services - the approach I took was to build what are essentially minimal machines running linux in command-line mode, and then load GeoServer on them to serve the data - As I haven't blogged in a while, a friend suggested that posting a quick description of the mechanics of this might be a good thing to share for folks who haven't dipped their toes into Linux much.

As a disclaimer, I do not claim to profess guruhood when it comes to Linux or the other packages, this is not necessarily warranted to be a "hardened-and-tweaked" system for production, it's just some very quick and dirty steps toward standing up a headless Linux-based GeoServer instance. Note that this uses the default Jetty install - some folks prefer to run it under Tomcat, which is a different path.

So, I started out with the "minimal install CD" for Ubuntu 9.04, available here:

https://help.ubuntu.com/community/Installation/MinimalCD



Select a package appropriate for the CPU you are using - in my case, I chose Ubuntu 9.04 for 32-bit PC.

Burn the ISO and follow the prompts to install from the text-based installer as command-line interface (CLI). I essentially went with the defaults. You will want to have the machine connected to the internet so that it can identify and set up the network connection and grab any files needed during install.

Once you've installed a minimal version of Linux, you will be ready to configure and install the other goodies.


For remote administration, you may want to install OpenSSH- http://www.openssh.com/



The step for doing this is simple:

Log in to your Linux machine, and use the following command:

sudo apt-get install openssh-server

This will download and install the OpenSSH package. For folks new to Linux, sudo tells it to use superuser privileges and permissions, and will ask for the root password used when you installed Linux. apt-get install uses the Advanced Package Tool to search for, retrieve and install software packages for Linux - this makes installation of much standard software in Linux easy.

For remote administration, you'll want to know how to reach your machine on the network - you can get the IP address by using the ifconfig command, which will give results something like this:



If you use Windows as a primary OS for your other work, you can then access the box from a Windows machine using an SSH client. I usually use PuTTY: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html



From there, you can install PuTTY on your windows machine and then access the Linux box via command-line interface remotely for administration.

Plug in the IP address you got above:



and voila - you should be presented with a login screen for your linux box:



Tools like PuTTY are a great asset when it comes to administering boxes.

Side trip into remote administration aside, on to the REAL stuff: Installing GeoServer.

As a prerequisite, you will need to install the Java JDK - the GeoServer install page gives some recommendations, and here's how you would do it from the command line:

sudo apt-get install sun-java6-jdk

Next, you will need to do some configuration of the JDK

Define the default Java to use:
sudo update-java-alternatives -s java-6-sun

And set the JAVA_HOME directory - this is doable in a number of ways, you may or may not want to define it in /etc/environment. I really like 'nano' as an editor for command-line Linux environments and it comes pre-installed in the minimal Ubuntu 9.04 version.

sudo nano /etc/environment

Again 'sudo' makes sure you have an appropriate privilege level to write the changes.

In nano, you can navigate around in the file using your arrow keys. Insert the following:

JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.14/


nano is intuitive and easy to use, following the commands along the bottom of the screen, e.g. ctrl-O to write changes, ctrl-X to exit.

Now on to the fun stuff - installing GeoServer.

GeoServer isn't available via apt - so you will need to download and unzip it to install it.

To be able to use ZIP archives, sudo apt-get install unzip will provide that capability. Next, you can download GeoServer.

Decide where you want to put it - some folks put it in /usr/local, or /usr/share, or if you are just experimenting, you could even leave it in your home directory - if putting it /usr/share in you would cd /usr/share

To download it, the download location given on the GeoServer page is http://downloads.sourceforge.net/geoserver/geoserver-1.7.6-bin.zip

Thus, to download it, use wget -

sudo wget http://downloads.sourceforge.net/geoserver/geoserver-1.7.6-bin.zip

Then, unzip it

sudo unzip geoserver-1.7.6-bin.zip
and you should see the files extracting into a geoserver-1.7.6 folder.

Depending on where you put it and privileges held by the account you are using, you may also need to ensure you have ability to access and run GeoServer and that GeoServer can create any files it needs.

chown will change ownership, using -R makes it recursive through subfolders and files:

sudo chown -R geoserver_username geoserver-1.7.6 would change all files and directories to be owned by the user specified (geoserver_username as a placeholder).

You can list files using ls and navigate directories using cd.

You may or may not also then want to configure directories, such as defining the location of your GeoServer installation directory, e.g. GEOSERVER_HOME="/usr/share/geoserver-1.7.6" - again, you could do this using nano to edit /etc/environment - and there are also plenty of other ways to do this. You could also define other parts of GeoServer, such as GEOSERVER_DATA_DIR at this point as well - consult the GeoServer docs for details there... http://docs.geoserver.org/1.7.x/en/user/

Pretty much ready to run now... cd to the /bin directory under your geoserver install, e.g. cd /usr/share/geoserver-1.7.6/bin" and launch the startup script sh startup.sh and voila... You will see some program output scroll by,
ultimately resulting with an output line like

[main] INFO org.mortbay.log - Started SelectChannelConnector@0.0.0.0:8080 - this should tell you that the GeoServer Jetty container is up and listening for connections on 8080.

Now, open a browser, point it to your machine's IP address and enter it, pointing to port 8080 and the geoserver instance, e.g. http://192.168.2.125:8080/geoserver/ and after an initial "loading" screen you should get the GeoServer web interface:



And you are off to the races... Confirm that it works via the demos:

OpenLayers NYC Tiger map

Again, this is just meant to be a quick-and-dirty guide - enough to make even someone with minimal Linux experience armed and dangerous - and from here, there are many tweaks and customizations that can be made, such as optimizing performance, hardening and security and so on (there are plenty of discussions around the web and on listservs regarding this)- but I figured, I'd at least share this as a quick start for anyone looking to play with GeoServer in a minimal Linux environment...

National Environmental Information Exchange Network

Posted by Dave Smith On 4/11/2009 06:54:00 PM 2 comments

Here's another exiting bit of news - my firm is teamed with CGI Federal on USEPA's Software Engineering & Specialized Scientific Support (SES3) Contract, and we just got word that our team has won EPA's Central Data Exchange (CDX) task. This is very exciting news, CDX and the Exchange Network serve the community via facilitating exchange of a wide variety of environmental data between federal, state, tribal and other partnerships - it is a partnership that has proven itself to be tremendously effective and a great model for other types of data exchanges as well.

What I am particularly excited about is in leveraging the infrastructure that has already been built toward more robustly supporting geodata services, and ultimately toward enhanced reporting, metrics, analytical capabilities, and other capabilities to support feds, states, tribes and others in informed decisionmaking toward environmental policy and stewardship.

As such - we also anticipate we will be looking to grow as a company, and will be looking to hire additional technical gurus with capabilities in data exchange, data management and data flows, particularly if you have prior capabilities and knowledge of EPA's Exchange Network and CDX, and/or geospatial technology.

If you are interested, drop me a line at dsmith (at) synergist.tech.com

ESRI Federal User Conference Highlights

My recently-retired EPA friend and colleague Dave Wolf was the recipient of the "Making a Difference" award. Dave was involved in pioneering efforts in web mapping at EPA, with EnviroMapper and other efforts.

I unfortunately arrived a bit late, and missed the discussion of ArcGIS Explorer Build 900 - fortunately Jithen Singh has a good overview of it here: http://mandown.co.nz/esri/arcgis-explorer-build-900-showcased-at-the-esri-federal-user-conference-2009/

As I arrived, I noted that there was a big focus being put on integration between ArcGIS and remote sensing imagery capability, specifically with ENVI and IDL: http://www.ittvis.com/ProductServices/ENVI.aspx. Dan Zimble led into presentations showing some of this capability, particularly integration of IDL scripts with ModelBuilder

Other highlights and demos:

  • Time Layer Animation
  • Keyless License Manager capability
  • Microsoft Virtual Earth data as a subscription service
  • APIs: Demonstration of ArcGIS Server Flex API via Solar Boston map: http://gis.cityofboston.gov/solarboston/
In general, there seemed to be a big focus on themes mirroring very topical and current issues, particularly stimulus and infrastructure investment - in the case of Solar Boston, energy - another demonstrated "smart routing" for alternative-fuel vehicles, based on availability of CNG fuel.

Relating to water issues, another demonstration featuring the Flex API was for the Chesapeake Bay Program: http://wdcb10.esri.com/cbprl/

This showed use of federated assets, such as USGS stream gauges, USEPA STORET water quality data, and so on, and provides many tools for assessment, management and best practices for improving water quality for the Chesapeake Bay.

Colonel Alex Dornstauder of the US Army Corps of Engineers gave a good presentation on what USACE is doing relating to watersheds and water quality, using an approach of "3D decision space" and cross-agency "lenses". The approach utilized several different datasets and attributes with ModelBuilder to get a baseline assessment, which can then be utilized to in the target, with a collaborative vision to triage risk and prioritize investments to water quality.

(3d Decision Space image gleaned from a related presentation by Col. Dornstauder: http://gis.esri.com/library/userconf/ieug08/papers/watershed_dornstauder.pdf)

The Colonel closed with an excellent quote from George Washington, "Let us raise a standard to which the wise and honest can repair"

There was also discussion and demos of
  • Situational Awareness tools built on the Flex API in the code gallery (http://resources.esri.com)
  • ArcGIS Mobile SDK, with a demonstration, showing field data collection, domains, subtypes, dropdown list support.
  • 3D and network analysis - modeling of movement within a building in 3d (pedestrian egress via stairs, et cetera) - 3D proximity analysis
  • There was discussion of food safety, with a case study of the Hawaii Food Safety Center
  • Economic security and urban growth
  • Demonstration of ordinary least squares analysis vs. geographically weighted regression


Maryland Governor Martin O'Malley gave an excellent discussion, hitting many excellent notes - with many things that resonated: "can you show me my house" - which he tied in well in how "show me my house" repeatedly resonated in context of geography with the many case studies he presented. Other highlights "Maryland is ESRI Customer #008" -
He discussed trying to remedy many of the issues facing Baltimore, "hopeless & vacant hearts" and how CityStat http://www.baltimorecity.gov/government/citistat/ and iMap http://maps.baltimorecity.gov/imap/ have provided far more performance-oriented approaches - he touched on how previously, information had been collected in such a way as to not make it to management in any meaningful way, with emphasis on measurement of inputs, but not of outputs and outcomes - and the new paradigm of viewing outcomes against the map, tells where challenges lie. These types of geospatial approaches allow relentless followup and assessment, and drive the effort to move the graphs in right direction, with improvements in city services, reductions in shootings and homicides, improved response times - cleaning out and boarding of vacant houses, mapping service problems and opportunities for daily review. "The map does not care if neighborhood is white/black, rich/poor, republican/democrat"

As Governor now, he has taken this mapping-oriented and performance-oriented approach to the next level on a statewide level, with StateStat http://www.statestat.maryland.gov/ and BayStat http://www.baystat.maryland.gov/, and GreenPrint, which provides an ecological assessment of every single parcel in Maryland, along with ecological measures being put in force: http://www.greenprint.maryland.gov/ - and tie-in of stakeholders at all levels - "if it's not about the relationship, it's not about anything".

Governor O'Malley left the audience with a Native American proverb, "how we treat one another is reflected in how we treat the earth"

I did visit the EXPO floor and looked at some of the maps - the floor was definitely quite crowded. By one account, there were 2800 registrants this year for the ESRI Federal User Conference.

Some of the ESRI schwag:


(I don't think the DevSummit attendees will be getting ESRI umbrellas to go with their weather...)

As posted previously, I will try to live tweet more coverage tomorrow, using hashtag #feduc

If you are also attending and want to meet up, by all means, drop a line - dsmith (at) synergist-tech.com - while I have a few meetings, both inside, during FedUC, and a few outside meetings in the DC area next week, I will generally try to make FedUC my base of operations and will blog wherever conditions permit (if conference WiFi is available and/or my AT&T 3G service cooperates).

Also, I will try to post updates from my phone and laptop via Twitter - http://twitter.com/DruidSmith using the hashtag #feduc - if others are attending, I'd suggest using #feduc as well, tools such as TweetGrid will be helpful for tracking twitter traffic in realtime - here's a sample 1x1 TweetGrid already set up for tracking #feduc: http://tweetgrid.com/grid?l=0&q1=%23feduc

ESRI Federal User Conference

Posted by Dave Smith On 2/13/2009 08:09:00 PM 1 comments

I'm planning on attending at least part of the ESRI Federal User Conference next week - it's always good to get together with others working in the Federal community, to cross-pollinate ideas, talk, and see all the great things going on...

The details, from: http://www.esri.com/events/feduc/

GIS: The Geographic Approach for the Nation

Explore what geographic information system (GIS) technology can do for your agency at the largest geospatial conference dedicated to federal government. Whatever your GIS experience, the FedUC will give you the knowledge and resources you need to apply geography to problem solving, decision making, and accomplishing your missions.

Join other leaders, decision makers, and GIS professionals February 18–20, 2009, in Washington, D.C.

February 18–20, 2009
Walter E. Washington Convention Center
Washington, D.C.

Agenda-at-a-Glance

2009 Agenda [PDF]

Wednesday, February 18

9:30 a.m.–3:00 p.m. Plenary Session
2:30 p.m. Keynote Speaker
3:00 p.m.–6:00 p.m. GIS Solutions EXPO and Map Gallery Reception

Thursday, February 19

8:30 a.m.–5:30 p.m. Concepts of GIS Sessions

Paper Sessions

Technical Workshops
8:30 a.m.–5:30 p.m. Map Gallery
8:30 a.m.–6:00 p.m. Hands-On Learning Center
10:00 a.m.–4:00 p.m. GIS Solutions EXPO
4:00 p.m.–5:30 p.m. Industry Focus Session
5:30 p.m.–8:30 p.m. Thursday Night Social

Friday, February 20

8:30 a.m.–noon Paper Sessions

Technical Workshops

Industry Focus Sessions
9:30 a.m.–5:00 p.m. User Group Meetings
Noon–2:00 p.m. Closing Session

Who Should Attend and Why

Take the Geographic Approach

The FedUC is the ultimate resource when it comes to using geospatial technology in government. The conference offers presentations from technical and industry experts, valuable insight from your colleagues, and the latest solutions that fit your agency. Professionals across organizations are invited to discover effective and efficient ways to meet goals, overcome challenges, and address issues.

What You'll Experience

  • Hear from Jack
    Explore the future of GIS in government with ESRI president Jack Dangermond during the Plenary Session. Listen to him and a team of experts share what you can do with ArcGIS 9.3 in your organization. Plus, watch real-world demonstrations of ways government agencies are leveraging GIS.
  • Stay up-to-date
    Learn about the newest ArcGIS tools and capabilities. Hear firsthand from ESRI staff, your peers, solution providers, and consultants as you attend paper sessions, technical workshops, and the GIS Solutions EXPO.
  • Network with your colleagues and ESRI staff
    Build relationships with government and GIS professionals from both the public and private sectors as well as ESRI staff and business partners. Connect during sessions, exhibit times, and the evening reception.
  • Increase your knowledge of GIS
    Discover more about how GIS works and what it can do for your team and agency. See ways GIS is being used—from data sharing to security to budget control. Attend presentations given by professionals from across the nation and learn about successful implementations, best practices, tips, and tricks.
  • Learn how to meet your agency’s needs
    From compliance and accountability to visualizing patterns and trends, find out how taking the geographic approach improves your operations and decisions. Whether you work in federal, state, or local government, see how GIS can meet your organization’s unique needs.
  • Examine the most advanced technology
    Experiment with state-of-the-art tools you can use right away. Learn about recent developments for geotechnologies in your fields, from hardware, software, and data solutions to innovative applications and services.
  • Get your questions answered
    Pose your questions about GIS software, data, and implementation to ESRI staff members. Meet for a quick one-on-one discussion or set up a meeting to brainstorm and discuss project plans.

See geography in action.

If you are also attending and want to meet up, by all means, drop a line - dsmith (at) synergist-tech.com - while I have a few meetings, both inside, during FedUC, and a few outside meetings in the DC area next week, I will generally try to make FedUC my base of operations and will blog wherever conditions permit (if conference WiFi is available and/or my AT&T 3G service cooperates).

Also, I will try to post updates from my phone and laptop via Twitter - http://twitter.com/DruidSmith using the hashtag #feduc - if others are attending, I'd suggest using #feduc as well, tools such as TweetGrid will be helpful for tracking twitter traffic in realtime - here's a sample 1x1 TweetGrid already set up for tracking #feduc: http://tweetgrid.com/grid?l=0&q1=%23feduc

I am hoping to catch the Plenary, and am hoping to see what discussion ensues relating to the passage of the Stimulus bill, infrastructure investment and planning and National Spatial Data Infrastructure - I also hope to see old friends, and am looking forward to it...

NSDI for Democracy

Posted by Dave Smith On 2/07/2009 03:13:00 PM 0 comments

With news of Vivek Kundra joining the Obama administration to serve the Office of Management and Budget (OMB) as their top IT visionary, it brings me great encouragement. Vivek Kundra has been serving as the District of Columbia’s Chief Technology Officer, and he recently created some excitement through his Apps for Democracy initiative, where he pursued development of an “Open Data Catalog” containing over 250 data assets of various flavors (e.g. XML, Text/CSV, KML, ATOM/GeoRSS and ESRI Shapefile formats), and then promoted a contest wrapped around the Open Data Catalog, for development of innovative mashup-oriented applications. In just a short amount of time, 47 excellent applications were submitted, dealing with a broad range of topics and providing many innovative solutions, a great success.

So what is OMB all about – and what might Kundra’s joining OMB mean?
From Wikipedia, http://en.wikipedia.org/wiki/Office_of_management_and_budget

The Office of Management and Budget (OMB) is a Cabinet-level office, and is the largest office within the Executive Office of the President of the United States (EOP). It is an important conduit by which the White House oversees the activities of federal agencies. OMB is tasked with giving expert advice to senior White House officials on a range of topics relating to federal policy, management, legislative, regulatory, and budgetary issues. The bulk of OMB's 500 employees are charged with monitoring the adherence of their assigned federal programs to presidential policies. OMB performs its coordination role by gathering, filtering, and promulgating the President's annual budget request, by issuing bulletins, memoranda and circulars dictating agency management practices, by overseeing the "President's Management Agenda", and by reviewing agency regulations.

Executive oversight of federal agencies – via expert advice on federal policy, management, legislative, regulatory, and budgetary issues, to be implemented and monitored for adherence via the President’s Management Agenda. That’s quite powerful. And under the Bush administration, OMB has already begun engaging in some basic monitoring activities relating to geospatial technology and investments, under the Geospatial Line of Business (GeoLoB).

So where do we go from here? And what does Kundra’s selection mean in this mix? Only Kundra really has the answer to this, at present - however I do believe that we can make some informed guesses as to what may be on his mind, based on his past track record and accomplishments. District of Columbia’s Open Data Catalog? Think in terms of doing this across all of Federal government. Strengthen and bolster the existing OMB A-16 mandate, and drive publishing of open data. That certainly forms some excellent pieces of the National Spatial Data Infrastructure.

Foster partnership-building and collaboration, ala the Apps for Democracy effort. Perhaps, on a cross-government level, we should also be looking at approaches such as Forge.mil, where agencies can collaborate and share GOTS technology investments, and work together to enhance and expand technology and capability, as opposed to continually reinventing the wheel.

Other things for Kundra to look at? Alignment of efforts across government. Currently USGS and EPA collaborate on efforts to build and densify hydrology data, as the National Hydrology Dataset (NHD), and are working with states to get this to a 1:4,800 level. Meanwhile, FEMA is developing DFIRMs for flood mapping, based on county and other types of data for stream centerlines. How do we align such things as linear referencing between USGS stream gauges and FEMA for looking at flooding issues? Coordinate between NOAA and NWS for realtime storm tracking, and have models available, using all best-possible data, toward stream flood prediction? What if a truck tumbles off of a bridge and ends up in a river, releasing hazardous waste into the river – is the information flow adequate to deal with hazardous waste cleanup even where that river crosses the border into the next state downstream? Pieces and parts of these types of things are starting to happen, but where they do, it is typically only in an ad-hoc, reactive fashion, with very limited coordination or common framework. Where does one thing end and the next begin? What are the gaps? Overlaps?

Or, consider a military convoy, heading across multiple states to an exercise. Due to an emergency bridge closure, they are diverted off of the main highway and onto local roads. They may be carrying sensitive and/or high value goods, such as weapons systems. Who knows? Who SHOULD know? Perhaps local bridge weight restrictions restrict their travel even further. How do we handle this in any coordinated fashion? Who’s doing what, and who’s able to supply what data to smoothly deal with these types of situations? Federal government places some mandates on states to collect roadway data, but again, is there any mechanism for establishing data capture, transparent access and flow? What’s covered, and what isn’t? State-to-state, if there is a serious roadway closure issue just inside one state’s border, will the adjacent state know this and be able to notify motorists via VMS boards and other means?

Although there are a few exceptions here and there, more often than not, the answer to these types of questions and scenarios is “no/had no idea/what am I supposed to do about it” accompanied by shrugs. Streams do not care about political boundaries, they only understand watersheds. Roads are networks. Cars and trucks do not just reach the Edge of the Knowne Worlde and drift off into space when they cross the state line. Information access and flows must be able to bridge these gaps.

It’s pretty much a given that 90% of most business processes in Federal government touch on or deal with location in some form or fashion. Where are assets, where are people, who’s being served, and so on. Through implementation of best practices and through making data access and exchange more timely, transparent, and complete, through better alignment of technology investments and reduction of gaps and overlaps, these *shrug* moments start to vanish, and the *AHA* moments start to happen. I am hoping that Mr. Kundra is thinking the same way. If what lies ahead of us is anything like his efforts to date, we indeed have a bright future ahead of us.

Stimulus and Infrastructure Planning

Posted by Dave Smith On 1/31/2009 10:32:00 AM 1 comments

With considerable debate and controversy, HR 1, the Stimulus bill has passed in the House of Representatives and has moved on to the Senate for additional debate and deliberation.


Current Bill Status

The full HR 1 text and various summaries are posted below:
(note: these may change as HR 1 works its way through the Senate)

Additional Supporting Documentation:
There are also a few additional sites discussing the Stimulus, such as the GOP-driven http://readthestimulus.org/ which nonetheless provide useful resources.

Within the Stimulus bill, there are a number of investments proposed, e.g. transportation funding, mass transit, broadband infrastructure and much more.

Question is, how do we intend to properly assess, triage and plan how and where best, geographically, to make these investments to provide maximal benefit without spatial data on a national level? How can these investments be expended without an adequately informed decisionmaking process?

This need points toward NSDI, the National Map and the related pieces that serve it, and a core need for geospatial data and analysis, which should be an integral part of any of these planning and investment processes, as well as embedding geo-enabled technologies within the investments themselves.

Investment in infrastructure without also investing in the underlying planning process and supporting data and decisionmaking tools represents tremendous opportunity lost, in terms of making adequately informed decisions, leveraging efforts, and properly targeting infrastructure improvements to where they provide the greatest good to the American people as a whole.


National Spatial Data Infrastructure 2.0

Posted by Dave Smith On 1/28/2009 03:02:00 PM 2 comments

Recently there has been a great deal of discussion about "NSDI 2.0" - and yet it seems there is much confusion about what it is or isn't - and what we should do, or whether we should bother discussing it at all.

History
To step backward in time, it primarily deals with National Spatial Data Infrastructure from a federal perspective, as enacted through the Office of Management and Budget Circular A-16.

This document was originally issued in 1990, followed by Presidential Executive Order 12906, and then subsequently updated in 2002 (which incorporated EO 12906).

OMB Circular A-16 (as revised 2002)

In its present form, the NSDI (if you were to consider it NSDI 1.0) consists of:

  • Defined data themes (geodetic control, orthoimagery, elevation, transportation, hydrography, governmental units, and cadastral information)
  • Metadata (FGDC Metadata Format)
  • The National Spatial Data Clearinghouse (Geospatial One-Stop)
  • Standards (developed only when no existing voluntary standards exist, in accordance with OMB Circular A-119)
  • Partnerships
Within the NSDI, oversight is provided by the Federal Geographic Data Committee (FGDC)

Additionally, A-16 pursues the following:
  • Privacy and Security of raw and processed citizens' personal data and accuracy of statistical data
  • Access to these data, subject to OMB Circular A-130
  • Protection of proprietary interests to these data 
  • Interoperability between various federal agencies' information systems within these data
The NSDI supports the advancement for a Global Spatial Data Infrastructure that coincides with National Security interests. Any Federal system that develops international data in accordance with these systems must follow international voluntary standards as outlined by Circular A-119.

The Current Situation
With the FGDC and OMB mandate, this prior effort more than anything, establishes a framework.  This framework, in turn, is something that individual agencies and data stewards can build to.  Similarly, the recent Office of Management and Budget (OMB) Geospatial Line of Business (GeoLoB) and other initiatives, such as the Federal CIO Council's Federal Enterprise Architecture (FEA) Geospatial Profile 1.1 provide guidance toward harmonizing investment, technology and architecture.  Further, the recently-formed National Geospatial Advisory Committee (NGAC) has been formed under the Federal Advisory Committee Act to provide advice, from a community cross-section, to FGDC.

Here nonetheless remains the challenge of populating this framework.  Many diverse efforts are ongoing, which align with these efforts, such as Imagery For The Nation, such as EPA's Exchange Network, however some of these efforts still lack adequate resources for completion, may have issues with stovepipes, lack of interoperability, lack of access, and so on.

The drivers for completing this work are manifold - such as providing adequate tools for planning, to better allow informed decision-making for such things as roadways and transportation, for analysis of demographics toward broadband investments, for homeland security, for planning improvements to municipal sewers, for protecting natural and archaeological heritage and biodiversity, and so on.   As such, with discussion of massive stimulus and the H.R. 1 bill geared toward many of these things, it is imperative that decisions and investments be made in an informed fashion.

Currently multiple documents of relevance and proposals toward populating this framework and advancing the various initiatives are currently circulating within the GIS community:
All of these speak to the need for populating the NSDI framework.  The questions and differences only remain in approaches.    Many elements are crucial to success, such as:
  • Satisfying clear mandates, requirements and drivers for geospatial data
  • Delivering data in an accessible, vendor-neutral, platform-agnostic and interoperable fashion
  • Leveraging and dovetailing into existing initiatives and investments
  • Partnerships:  Federal/State/Local/Tribal/Academia/NGO/Industry
  • And many more...  These need to be considered carefully.
The hope is that the community can have an open, informed discussion of these elements and proposals, along with all of the key context, history and background.  And hopefully adequate open forums will become available for doing so.

Environmental Information Symposium 2008

Posted by Dave Smith On 12/01/2008 03:58:00 PM 2 comments


I will be flying out to Phoenix, Arizona December 9th-12th, to attend the US Environmental Protection Agency's annual Environmental Information Symposium... This year, I will be participating on a panel, to discuss collaborative geospatial tools, web services and data publishing, integration and visualization frameworks for environmental science, such as Microsoft Virtual Earth and Google Earth. I am definitely looking forward to sparking some discussion and engaging more people who collect, manage, consume or make decisions based on environmental data... My particular interest at this point is not just in publishing data via open, standards-based (not just OGC, but REST, JSON and others), accessible, dynamic data resources, but also in modeling and analysis, and beginning to look at workflows toward solving a wide variety of environmental problems.

Last year, I got to have a lot of fun developing a turbo Virtual Earth integration in 36 hours, for the Puget Sound Information Challenge. To blow my own horn, my application was recognized by EPA's CIO as one of the most interesting contributions made to the effort. Needless to say, I am really looking forward to attending again this year.

The event will be held at the JW Marriott Desert Ridge Resort in Phoenix,

Can't attend in person? Live streaming will be available as well.

(and thanks to Sean Gillies...)

Posted by Dave Smith On 11/03/2008 05:42:00 PM 1 comments

What's on my mind?



From Wordle, populated with my Blog's atom feed, capturing the most recent blog entries. Wordle analyzes the content and assembles a collage of words, with emphasis placed on words not in common usage, according to how often these words are used... Geospatial... Mapping... Integration... Technology... A few little trends that I didn't expect, but all in all, good stuff.

Mapping Integration with BusinessObjects

Posted by Dave Smith On 10/10/2008 11:08:00 AM 1 comments

I got to spend a little time in the last week or two examining integration of web mapping with BusinessObjects XI R2 Web Intelligence, for a little turbo proof-of-concept application - I hadn't seen many folks talking about integrating web mapping and BO, so I figured I would share my own experiences. Though I'm certainly no BusinessObjects guru, I was able to successfully do some basic integration, with the help of the BusinessObjects "Masher" application available on their "Labs" site: http://labs.businessobjects.com/mashup/default.asp.

What BusinessObjects presents via the Masher is a dynamically-named div tag, which is accessible via container in JavaScript, which is where your rendered DHTML content needs to be targeted. They also then present three basic functions for housing and executing all user-developed JavaScript, init(), mash() and dispose() - the intent of which is self-evident. They also provide some JavaScript objects, pivot and parameters - pivot contains the BO report table contents, and parameters contains values that can be preset and/or passed when initializing or formatting the template.

So, essentially just a matter of figuring out what's available to latch on to and use - and how to leverage what BO provides to permit building the application. For this proof-of-concept, there had been some discussion and interest in Google Maps from the customer, so - for the sake of simplicity, I chose to just stick with Google Maps. In reality, I'd be fairly confident in suggesting that virtually any web mapping framework that supports DHTML and JavaScript should be usable with this same approach, e.g. OpenLayers, Microsoft Virtual Earth, or others.

So, ultimately - the results:

"Turn table into..." (the intent being as simple as "click a button to turn this table or graph into a map")

Being a table, the application has to be able to understand and use designations in the underlying table, from it's "geography" dimension such as "AZ" to latch on to the appropriate mapping geographies. So... Turn map into...


Here, I have my newly published template showing up among the choices as "Google Maps Equal Interval Polygonal Choropleth Map", and click on "OK" - Et voila!


For this quick-and-dirty proof of concept, I just wrote some JavaScript that examines the array of pivot data and the range of values, and then just does some quick calculations to create a lookup table and legend. One trick was to dynamically rewrite BO's generic container div tag to embed two new div elements inside of it, one which I use to hold the map, and a second, which I use to hold my legend - other elements as well as a hidden control panel could be embedded here also.

For generating the classification and legend, a similar type of logic and approach could of course also be applied to a variety of classification schemes such as standard deviations, natural breaks, and so on. I then examine the pivot table to see what geographies are represented, display them on the map rendered with their appropriate hex color values matching their respective measures from the lookup table and add a Google Maps openWindowHtml popup for the click event handler for each location. Within the pivot object, labels are also provided - and given a table with multiple columns of measures, these could either be managed with code to run analysis on both, just thematically map the first, and display the remainder in the popup, and so on - a variety of possibilities for the adventurous coder.

Want to change the number of classification breaks? Didn't like the color scheme?

Within the template, there is opportunity to define parameters - here I set it up to allow me to change the number of classification breaks, starting color, ending color, and so on. And the BO folks nicely even include a color picker. So, for example, I can then retrieve parameters.startColor as a hex value, split it into RGB values and compute even jumps for my color ramp to parameters.endColor, based on the parameters.numBreaks value that the user specifies.

Like so... (deliberately odd colors selected to show the difference...)


Additionally, mash() fires every time the underlying table's filters or results are changed, which is nice, as the map updates accordingly.

Some additional development insights - I decided to keep a lot of my functions in an external .js library - one issue I encountered is that, given everything was encapsulated in the three BO Masher functions init(), mash() and dispose(), whenever I needed to create and use global variables, I needed to do so explicitly, e.g. window.myVarName. Additionally, I noted that BusinessObjects XI R2 uses the dojo JavaScript library... so for fans of dojo, that also provides a great deal of functionality and capability to leverage. As with anything, potential collision with other pieces of code or cross-browser compatibility issues are also a question, but through judicious naming conventions and error handling, these can generally be addressed.

So, while perhaps my turbo solution may not the most elegant, it's nonetheless illustrative that web mapping integration into a BO platform is definitely doable and definitely functional, in this case developed in just a few days' time with a little quick hackery, even for someone who may have web mapping expertise but little prior integration or development experience with a platform like BusinessObjects (such as myself in this instance) - hopefully this will be an encouragement to others who are looking to merge BI tools with geospatial technology, and likewise, hopefully the BI folks will continue to increasingly see the value of geospatial visualization in addition to their charts, graphs and dashboards. I have already seen some very interesting visual geographic trends popping out of the data, which is definitely not anywhere near as evident in some of the more standard BI views. Other possibilities include drilldown, however I found that the masher does not handle hyperlinks embedded in tables well - but within BO, URL-driven access to reports to allow drilldown is definitely a possibility as well.

What the future of the BO Masher is in the future is uncertain, but the potential use case and benefit of such integration is substantial.


In the long run, there will definitely be an ever-emergent intersect between these technologies and tools. All in all, another fun little proof-of-concept. Next, I may want to take on BusinessObjects Xcelsius 2008 and Flex-based integration with web mapping frameworks - will remain to be seen, depending on how various other projects go over the next few months...

MetaCarta Public Sector User Group Meeting

Posted by Dave Smith On 9/30/2008 08:59:00 PM 0 comments

MetaCarta is holding a users group meeting next week, October 8th - from their site:


MetaCarta Public Sector User Group Meeting 

How often can you hear key geospatial analysis findings from defense, intelligence, federal civilian, and industry analysts under one roof, in one day?

On Wednesday, October 8, 2008, MetaCarta invites you to join us for the 4th Annual MetaCarta Public Sector User Group Meeting as intelligence officers, industry analysts, and other senior executives share their geospatial technology experiences.

This is a perfect way to decompress from the Fiscal Year end, and come away with proven geospatial technology examples to employ within your own agency or organization.

Don't miss your chance to gain insight from industry experts including:

  • Jeff Vining, Research VP, Homeland Security and Law Enforcement, Gartner Research
  • Terry Busch, Senior Intelligence Officer, Defense Intelligence Agency
  • Dave Sonnen, Senior Consultant, Spatial Information Management, IDC Research
  • Dr. Jerry Johnston, Geospatial Information Officer, EPA / OEI

Collaborate with partners and colleagues to learn about best practices and take away critical knowledge about the latest uses and trends around geospatial intelligence, geosearch, and information retrieval. Hear about the “geoweb” and how MetaCarta exposes intelligence that would have been impossible to find any other way.


Highlights:


  • Share experiences and uses of MetaCarta technology
  • See the convergence of open source content and analysis
  • View in-depth customer presentations
  • See the updated 2008 / 2009 product roadmaps
  • Learn custom integration tips & tricks 
http://www.metacarta.com/geo-overview.htm



Farm Bill Prevents Sharing of Geospatial Data?

Posted by Dave Smith On 9/28/2008 12:16:00 PM 5 comments

I just came across this bit of disturbing information - a colleague was seeking historic aerial photographs from USDA, and was greeted with the following response:

"Please refer to Title 1, Subtitle F, Section 1619 titled "Information Gathering" on pages 256-259. This will give you the precise language of the 2008 Farm Bill which prevents FSA from providing geospatial information."

The bill is available here: http://www.usda.gov/documents/Bill_6124.pdf

The pertinent language is below:

SEC. 1619. INFORMATION GATHERING.

(a) GEOSPATIAL SYSTEMS.—The Secretary shall ensure that all the geospatial data of the agencies of the Department of Agriculture are portable and standardized.

(b) LIMITATION ON DISCLOSURES.—

(1) DEFINITION OF AGRICULTURAL OPERATION.—In this subsection, the term ‘‘agricultural operation’’ includes the production and marketing of agricultural commodities and livestock.

(2) PROHIBITION.—Except as provided in paragraphs (3) and (4), the Secretary, any officer or employee of the Department of Agriculture, or any contractor or cooperator of the Department, shall not disclose—
(A) information provided by an agricultural producer or owner of agricultural land concerning the agricultural operation, farming or conservation practices, or the land itself, in order to participate in programs of the Department; or
(B) geospatial information otherwise maintained by the Secretary about agricultural land or operations for which information described in subparagraph (A) is provided.

(3) AUTHORIZED DISCLOSURES.—
(A) LIMITED RELEASE OF INFORMATION.—If the Secretary determines that the information described in paragraph (2) will not be subsequently disclosed except in accordance with paragraph (4), the Secretary may release or disclose the information to a person or Federal, State, local, or tribal agency working in cooperation with the Secretary in any Department program—
(i) when providing technical or financial assistance with respect to the agricultural operation, agricultural land, or farming or conservation practices; or
(ii) when responding to a disease or pest threat to agricultural operations, if the Secretary determines that a threat to agricultural operations exists and the disclosure of information to a person or cooperating government entity is necessary to assist the Secretary in responding to the disease or pest threat as authorized by law.

(4) EXCEPTIONS.—Nothing in this subsection affects—
(A) the disclosure of payment information (including payment information and the names and addresses of recipients of payments) under any Department program that is otherwise authorized by law;
(B) the disclosure of information described in paragraph (2) if the information has been transformed into a statistical or aggregate form without naming any—
(i) individual owner, operator, or producer; or
(ii) specific data gathering site; or
(C) the disclosure of information described in paragraph (2) pursuant to the consent of the agricultural producer or owner of agricultural land.

(5) CONDITION OF OTHER PROGRAMS.—The participation of the agricultural producer or owner of agricultural land in, or receipt of any benefit under, any program administered by the Secretary may not be conditioned on the consent of the agricultural producer or owner of agricultural land under paragraph (4)(C).

(6) WAIVER OF PRIVILEGE OR PROTECTION.—
The disclosure of information under paragraph (2) shall not constitute a waiver of any applicable privilege or protection under Federal law, including trade secret protection.

Search