Archive for the ‘Silverlight’ Category

GPS Tracking with a Twitter Twist

Saturday, April 24th, 2010

dnr_roadtrip screen shot
Fig 1 – .NETRocks Road Trip Silverlight UI

It has been awhile since I had time to post here. Ive been busy on a whole raft of POCs using Silverlight Map Control. Silverlight seems to be a popular way to add map services to enterprise applications, at least it is generating a lot of POC traffic. Most are impressed with its flexibility. Today I wanted to take a couple of minutes and share a peek at a fun project I was working on in between real work. Its a tracking application with a Twitter twist. The vehicle tracking is tied into a Tweet community along a vehicle route.

DotNet Rocks is an online radio type show that does interviews with technical folks. They are primarily interested in .NET topics so you won’t find a lot of open source, or even GIS for that matter. Along with the recent release of Visual Studio 2010 and Silverlight 4, Richard Campbell and Carl Franklin, the hosts of the .NETRocks program, are taking a US tour in an RV. They will be hitting a dozen or so major cities around the country while doing their show on the road. They just had a stop in Phoenix last night and as I write this are heading out to Houston.


dnr_roadtrip screen shot
Fig 2 Event stop click events bring up a small Birdseye map using MapControl.ExtendedModes beta

The DNR RoadTrip project uses a SQL Server 2008 DB for maintaining the GPS history and caching Tweets. On the server we have a couple of one minute polling services that check for any new GPS positions or Tweets. There are some static FixedLocation Feeds that maintain venue locations. The GPS track feeds go into the DB directly. The Tweet feeds often have no geo tags so their datetime stamps are used to look up the GPS record closest in time. The location of that GPS position is then applied to the Tweet record location. Here is an edmx view of the DB, but notice geography data types are still not supported so the Location field is missing.

dnr_roadtrip screen shot
Fig 3 – .edmx view of the DB (does not show geography Location field)
dnr_roadtrip screen shot
Fig 4 – MSSMS Database Diagrams (shows Location geography field)

The Tweet feeds are queried by the polling service using a #hashtag Twitter search. The hashtag for this Road Trip project, #dnr_roadtrip, lets anyone interested conversevia twitter. The first Tweet at a track location is available as a rollover. If additional Tweets come in at the same location they are made available by an additional Feed Service call:

// Pull the ATOM feed from the Twitter API using the TweetSharp.com library.
var search = FluentTwitter.CreateRequest()
    .AuthenticateAs("Twitter account", "Twitter key")
    .Search().Query()
    .ContainingHashTag(TwitterHashTag)  // TwitterHashTag set in the config file.
    .Since(m_SinceId)
    .AsAtom();

The polling service makes use of the “since_id” as a place holder. This way only newer Tweets since the previous since_id need to be processed.

While these polling services are running on the server, the Silverlight Bing Maps UI is communicating with the DB using a straightforward WCF basicHttpBinding. The WCFRoadShow provides the DB queries for populating the UI. There four primary Feed Service WCF calls:

  • GetCurrentLocation() – renders the most recent vehicle location
  • GetFixedLocation() – renders venue stop icons from the DB
  • GetRoute(string updatetype) – renders the GPS tracking icons
  • GetTweet()- renders the Tweet icons related to GPS records

There is one additional Feed Service call which is used to return all Tweets at a given location. This last WCF Service call is triggered by a click event from one of the Tweet icons and is the most complex feed query. It makes use of a SQL Server 2008 geography data type proximity search. The returned Tweet Feed records are then used in a Tweet ListBox using Data Binding.

List tweets = new List();
XNamespace ns = "http://www.w3.org/2005/Atom";
foreach (FeedServiceReference.FeedItem item in e.Result.FeedItems)
{
  if (item.FeedContent != null)
  {
     // Note: namespace required
     // These are the entry xml elements from Twitter
     //   saved in RoadTrip2010 DB as a varbinary FeedItem
  MemoryStream m = new System.IO.MemoryStream(item.FeedContent);
  XDocument Xitem = XDocument.Load(m);
     foreach (var entry in Xitem.Elements(ns + "entry"))
     {
        string tweeturi = entry.Elements(ns + "link").ElementAt(0).Attribute("href").Value;
        string iconuri = entry.Elements(ns + "link").ElementAt(1).Attribute("href").Value;
        string title = entry.Element(ns + "title").Value;
        string author = entry.Element(ns + "author").Element(ns + "name").Value;
        tweets.Add(new TweetItem(tweeturi, iconuri, title, author));
      }
   }
}
((ListBox)_LayoutRoot.FindName("twitterItems")).ItemsSource = tweets;
((Border)_LayoutRoot.FindName("twitterPanelBorder")).Visibility = Visibility.Visible;

You can see from the above code snippet that the rss xml feed item for the tweets is returned and then using Binding is attached to a ListBox’s ItemTemplate DataTemplate. This is a handy way of rendering the indeterminate list length of Tweet records found at a given location. Also note from the code snippet that it was necessary to prefix the Atom namespace for accessing feed Elements and Attributes. This is because the returned Atom wrapper was discarded and only entry elements are stored as a varbinary FeedItems.

In addition to the DB Tweet feed cache, I decided to make use of Twitters newer geotagging capabilities. This is still relatively new and not used very frequently, but there is a simple Twitter location query that I hooked to a double click map event:

string twitterurl = "http://search.twitter.com/search.atom?geocode=";
twitterurl += string.Format(CultureInfo.InvariantCulture, "{0,10:0.000000},{1,11:0.000000},
 1mi", loc.Latitude, loc.Longitude).Replace(" ", "");

This takes the click map location in latitude longitude and searches for the nearest geotagged Tweets that fall within a 1mi radius.

Adding geotag Tweet queries injects some interest because unrelated geotagged Tweets can be viewed anywhere in the world. It is surprising to me how many people are unabashed about sharing their location with the world. I was also surprised how much street capitalism occupies the lowly Twitter world. Go to any urban area and a few clicks will reveal the whole spectrum of street talk, the good, the bad, and the ugly.

Since a similar Flickr query is available it was simple to add a geotagged Flickr photo query as well. Double clicking on the map will bring up ListBoxes for all Tweets and Flickr photos that are geotagged and fall within 1mile of the click spot.

string flickrurl = "http://api.flickr.com/services/rest/?
format=rest&method=flickr.photos.search&api_key={0}&safe_search=1
&radius_units=mi&radius=20&lat={3,10:0.000000}&lon={4,11:0.000000}";

This Flickr query returns a group of photo xml entries. Each of these have an id that can be use to obtain the actual photo using a second lookup.
string src = “http://farm{0}.static.flickr.com/{1}/{2}_{3}.jpg”;

In both cases the returned entries are used to populate ListBox’s ItemTemplate DataTemplate.


dnr_roadtrip screen shot
Fig 5 Radius search results for Flickr and Twitter near downtown LA.

Another interesting aspect of this project is the donation of a PreEmptive’s Dotfuscator. This is an injection type tool for monitoring traffic and obtaining detailed analytics about a UI. The Dotfuscator tool is easy to use, especially since PreEmptive was kind enough to provide a detailed config.xml.

Using the tool creates a new xap file of the UI which holds event triggers for different aspects of the code. These events trigger messages back to the PreEmptive’s Runtime Intelligence Service where they are aggregated and then made available for a nice display. You can click on the PreEmptive icon to take a look at the analytics generated on this project. I was impressed with how easy it was to use and how nice it was to get analytics at a highly granular level, down to clicks on individual elements in the xaml.

Since some interested viewers kept a viewer open I also added a timer refresh to update currentPosition, Tweets, and GPS locations:

  • refreshTimer = new DispatcherTimer();
  • refreshTimer.Interval = new TimeSpan(0, 0, 0, 0, 120000); // 120000 Milliseconds 2 min
  • refreshTimer.Tick += new EventHandler(Refresh_Tick);
  • refreshTimer.Start();

The key control for using any of the queries for GPS and Tweets is the DateTime Slider. This is a handy control found over in the DeepEarth codeplex project.
There are plenty of other useful controls available in the DeepEarth Bing Maps Toolkit. This DateSlider encapsulates selection of DateTime ranges across a min/max range:

dateRange.Minimum = DateTime.Parse("4/16/2010 00:00:00", CultureInfo.InvariantCulture);
dateRange.RangeStart = DateTime.Now.AddDays(-2);
dateRange.RangeEnd = DateTime.Now.AddHours(1);
dateRange.Maximum = DateTime.Parse("5/9/2010 00:00:00", CultureInfo.InvariantCulture);
startdatetext.Text = dateRange.RangeStart.ToString();
enddatetext.Text = dateRange.RangeEnd.ToString();

In the process of deploying this project I learned about CultureInfo. Surprisingly, for a RoadTrip in the USA there were a number of folks from other countries who wanted to watch Richard and Carl roam across North America. I had to go back through the application adding CultureInfo.InvariantCulture Globalization to get things to work for the non ‘en-US’ followers. Adjusting for at least working even without fully accommodating other languages turned out to be surprisingly simple. This is one of the pluses for working in the Microsoft world.

Another big plus is browser compatibility. A quick check with browsers I happen to have on hand verified that all of these work fine:

  • Firefox v3.0.17
  • Chrome 4.1.249.1069
  • Safari 4.0.4
  • Opera 10.51
  • IE of course

Don’t even try this with an SVG project.

Summary

Twitter is fun and Twitter with maps is a GIS social network. Some claimed to find it mildly addicting. The internet world is over flowing with connectable APIs, SDKs, etc and Bing Maps Silverlight makes hooking these all together relatively statraighforward. Remember it’s really just me and a few weeks working part time to put together a pretty interesting Twitter Map. I know there are lots of more sophisticated Twitter mapping applications already floating around, but of course the attraction to Silverlight is ease of ownership.

Phone Map Toys

Tuesday, March 16th, 2010


Fig 1 – Windows Phone 7 Series Emulator

New toys last night! Microsoft released the Windows Phone 7 Series Development CTP at MIX2010 yesterday. Tim Heuer has a helpful Getting Started page.

I’m curious to see the resulting acronymization for this, perhaps ‘WiPho7′, hopefully not ‘Wiph7′.

The great news is Silverlight support with XNA thrown in as well, with Silverlight life is easy, with XNA life is 3D.

Getting started was straightforward. I just followed Tim Heuer’s instructions and then went to Petzold’s blog.

and … “Houston, we have emulation.”


Fig 2 – Windows Phone 7 Series Emulator

After a couple of minutes I have the emulator running a simple Bing Maps image. It’s nothing more than an <image> with the new RESTful Beta as a source, but you get the idea.

Some of the nice hardware specifications:

  • 800×480
  • Wi-Fi (InternetExplorer)
  • Camera 5 megapixel
  • Accelerometer
  • Compass
  • Location (longitude, latitude, altitude, course, speed, reverse geocoded address)
  • Speech
  • Vibration
  • Push Notification

MIX2010 showed some devices, but real hardware should be available by this fall.

As far as maps, Charles Petzold has this to say:
“Programs are location-aware, have access to maps and
other data through Bing and Windows Live, and can interface with social networking sites.”

Summary:

Overall, this fills a big hole in the Silverlight framework. We will at least have a Silverlight target across the full range of platforms by 2011. Android has a big headstart, but I believe the base Silverlight/XNA CLR technology will provide a stronger development platform in the long run. Sorry Apple fans. With some real competitor technologies, iPhone will eventually recede due to Apple’s artsy, control freaky orientation.

Map Clipping with Silverlight

Sunday, January 24th, 2010



Fig 1 – Clip Map Demo

Bing Maps Silverlight Control has a lot of power, power that is a lot of fun to play with even when not especially practical. This weekend I was presented with a challenge to find a way to show web maps, but restricted to a subset of states, a sub-region. I think the person making the request had more in mind the ability to cut out an arbitrary region and print it for reporting. However, I began to think why be restricted to just the one level of the pyramid. With all of this map data available we should be able to provide something as primitive as coupon clipping, but with a dynamic twist.

Silverlight affords a Clip masking mechanism and it should be simple to use.

1. Region boundary:

The first need is to compile the arbitrary regional boundary. This would be a challenge to code from scratch, but SQL Server spatial already has a function called “STUnion.” PostGIS has had an even more powerful Union function for some time, and Paul Ramsey has pointed out the power of fast cascaded unions. Since I’m interested in seeing how I can use SQL Serrver, though, I reverted to the first pass SQL Server approach. But, as I was looking at STUnion it was quickly apparent that this is a simple geo1.STUnion(geo2) function and what is needed is an aggregate union. The goal is to union more than just two geography elements at a time, preferably the result of an arbitrary query.

Fortunately there is a codeplex project, SQL Server Tools, which includes the very thing needed, along with some other interesting functions. GeographyAggregateUnion is the function I need, Project/Unproject and AffineTransform:: will have to await another day. This spatial tool kit consists of a dll and a register.sql script that is used to import the functions to an existing DB. Once in place the functions can be used like this:

SELECT dbo.GeographyUnionAggregate(wkb_geometry)) as Boundary
FROM [Census2009].[dbo].[states]
WHERE NAME = ‘Colorado’ OR NAME = ‘Utah’ or NAME = ‘Wyoming’

Ignoring my confusing choice of geography column name, “wkb_geometry,” this function takes a “geography” column result and provides the spatial union:



Fig 2 – Helpful Union illustration from JTS

Or in my case:


Fig 3 – GeographyUnionAggregate result in SQL Server

Noting that CO, WY, and UT are fairly simple polygons but the result is 1092 nodes I tacked on a .Reduce() function.
dbo.GeographyUnionAggregate(wkb_geometry).Reduce(10) provides 538 points
dbo.GeographyUnionAggregate(wkb_geometry).Reduce(100) provides 94 points
dbo.GeographyUnionAggregate(wkb_geometry).Reduce(100) provides 19 points

Since I don’t need much resolution I went with the 19 points resulting from applying the Douglas-Peuker thinning with a tolerance factor of 1000.

2. Adding the boundary

The next step is adding this union boundary outlining my three states to my Silverlight Control. In Silverlight there are many ways to accomplish this, but by far the easiest is to leverage the builtin MapPolygon control and add it to a MapLayer inside the Map hierarchy:

<m:MapLayer>
  <m:MapPolygon x:Name=”region”
  Stroke=”Blue” StrokeThickness=”5″
    Locations=”37.0003960382868,-114.05060006067 37.000669,-112.540368
    36.997997,-110.47019 36.998906,-108.954404
         .
        .
        .
    41.996568,-112.173352 41.99372,-114.041723
     37.0003960382868,-114.05060006067 “/>
</m:MapLayer>


Now I have a map with a regional boundary for the combined states, CO, WY, and UT.

3. The third step is to do some clipping with the boundary:

UIElement.Clip is available for every UIElement, however, it is restricted to Geometry clipping elements. Since MapPolygon is not a geometry it must be converted to a geometry to be used as a clip element. Furthermore PathGeometry is very different from something as straight forward as MapPolygon, whose shape is defined by a simple LocationCollection of points.

PathGeometry in XAML:


<Canvas.Clip>
  <PathGeometry>
    <PathFigureCollection>
      <PathFigure StartPoint=”1,1″>
        <PathSegmentCollection>
          <LineSegment Point=”1,2″/>
          <LineSegment Point=”2,2″/>
          <LineSegment Point=”2,1″/>
          <LineSegment Point=”1,1″/>
        </PathSegmentCollection>
      </PathFigure>
    </PathFigureCollection>
  </PathGeometry>
</Canvas.Clip>


The easiest thing then is to take the region MapPolygon boundary and generate the necessary Clip PathGeometry in code behind:

  private void DrawClipFigure()
  {
    if (!(MainMap.Clip == null))
    {
     &nbspMainMap.ClearValue(Map.ClipProperty);
    }
    PathFigure clipPathFigure = new PathFigure();
    LocationCollection locs = region.Locations;
    PathSegmentCollection clipPathSegmentCollection = new PathSegmentCollection();
    bool start = true;
    foreach (Location loc in locs)
    {
      Point p = MainMap.LocationToViewportPoint(loc);
      if (start)
      {
       clipPathFigure.StartPoint = p;
       start = false;
     }
     else
     {
      LineSegment clipLineSegment = new LineSegment();
      clipLineSegment.Point = p;
      clipPathSegmentCollection.Add(clipLineSegment);
     }
    }
    clipPathFigure.Segments = clipPathSegmentCollection;
    PathFigureCollection clipPathFigureCollection = new PathFigureCollection();
    clipPathFigureCollection.Add(clipPathFigure);

    PathGeometry clipPathGeometry = new PathGeometry();
    clipPathGeometry.Figures = clipPathFigureCollection;
    MainMap.Clip = clipPathGeometry;
  }

This Clip PathGeometry can be applied to the m:Map named MainMap to mask the underlying Map. This is easily done with a Button Click event. But when navigating with pan and zoom, the clip PathGeometry is not automatically updated. It can be redrawn with each ViewChangeEnd:
private void MainMap_ViewChangeEnd(object sender, MapEventArgs e)
{
  if (MainMap != null)
  {
    if ((bool)ShowBoundary.IsChecked) DrawBoundary();
    if ((bool)ClipBoundary.IsChecked) DrawClipFigure();
  }
}


This will change the clip to match a new position, but only after the fact. The better way is to add the redraw clip to the ViewChangeOnFrame:

MainMap.ViewChangeOnFrame += new EventHandler<MapEventArgs>(MainMap_ViewChangeOnFrame);

private void MainMap_ViewChangeOnFrame(object sender, MapEventArgs e)
{
  if (MainMap != null)
  {
    if ((bool)ShowBoundary.IsChecked) DrawBoundary();
    if ((bool)ClipBoundary.IsChecked) DrawClipFigure();
  }
}


In spite of the constant clip redraw with each frame of the navigation animation, navigation is smooth and not appreciably degraded.

Summary:

Clipping a map is not terrifically useful, but it is supported with Silverlight Control and provides another tool in the webapp mapping arsenal. What is very useful, are the additional functions found in SQL Server Tools. Since SQL Server spatial is in the very first stage of its life, several useful functions are not found natively in this release. It is nice to have a superset of tools like GeographyAggregateUnion, Project/Unproject,
and AffineTransform::.

The more generalized approach would be to allow a user to click on the states he wishes to include in a region, and then have a SQL Server query produce the boundary for the clip action from the resulting state set. This wouldn’t be a difficult extension. If anyone thinks it would be useful, pass me an email and I’ll try a click select option.



Fig 4 – Clip Map Demo

Silverlight Video Pyramids

Thursday, December 24th, 2009


VideoTile Fig 1
Fig 1 – Silverlight Video Tile Pyramid

Microsoft’s DeepZoom technology capitalizes on tile pyramids for MultiScaleImage elements. It is an impressive technology and is the foundation of Bing Maps Silverlight Control Navigation. I have wondered for some time why the DeepZoom researchers haven’t extended this concept a little. One possible extension that has intriguing possibilities is a MultiScaleVideo element.

The idea seems feasible, breaking each frame into a DeepZoom type pyramid and then refashioning as a pyramid of video codecs. Being impatient, I decided to take an afternoon and try out some proof of concept experiments. Rather than do a frame by frame tiling, I thought I’d see how a pyramid of WMV files could be synchronized as a Grid of MediaElements:

<Grid x:Name="VideoTiles" Background="{StaticResource OnTerraBackgroundBrush}" 
 Width="426" Height="240"                             
 HorizontalAlignment="Center" VerticalAlignment="Center">                                
<Grid.ColumnDefinitions>
<ColumnDefinition/>
<ColumnDefinition/>
</Grid.ColumnDefinitions>
<Grid.RowDefinitions>
<RowDefinition/>
<RowDefinition/>
</Grid.RowDefinitions>
  <MediaElement x:Name="v00" Source="http://az1709.vo.msecnd.net/video/v00.wmv"
 Grid.Column="0" Grid.Row="0" />
  <MediaElement x:Name="v10" Source="http://az1709.vo.msecnd.net/video/v10.wmv"
 Grid.Column="1" Grid.Row="0" />
  <MediaElement x:Name="v11" Source="http://az1709.vo.msecnd.net/video/v11.wmv"
 Grid.Column="1" Grid.Row="1" />
  <MediaElement x:Name="v01" Source="http://az1709.vo.msecnd.net/video/v01.wmv"
 Grid.Column="0" Grid.Row="1" />
</Grid>

Ideally to try out a video tile pyramid I would want something like 4096×4096 since it divides nicely into 256 like the Bing Maps pyramid. However, Codecs are all over the place, and tend to cluster on 4:3 or 16:9 aspect ratios. Red 4K at 4520×2540 is the highest resolution out there, but I didn’t see any way to work with that format in Silverlight. The best resolution sample clip I could find that would work in Silverlight was the WMV HD 1440×1080 Content Showcase. Since I like the Sting background music, I decided on “The Living Sea” IMAX sample.

Not enough resolution to get too far, but I am just looking at multi tile synching for now and two levels will do. I ended up using Expression Encoder 3 to take the original resolution and clip to smaller subsets.

Zoom Level 1:

00 10
01 11

Zoom Level 2:

0000 0010 1000 1010
0001 0011 1001 1011
0100 0110 1100 1110
0101 0111 1101 1111

I encoded ZoomLevel 1 as 4 tiles 640×480 and Zoom Level 2 as 16 tiles at 320×240. I then took all these tiles and dropped them into my Azure CDN video container. Again this is not a streaming server, but I hoped it would be adequate to at least try this in a limited time frame. Now that I have the video pyramid with two zoom levels I can start trying out some ideas.


VideoTile Fig 2
Fig 2 – Silverlight Video Tile Pyramid Zoom Level 1


VideoTile Fig 3
Fig 3 – Silverlight Video Tile Pyramid ZoomLevel 2

First, it is fairly difficult to keep the Grid from showing in the layout. Moving around with different sizes can change the border but there is generally a slight line visible, which can be seen in Fig 2. Even though you don’t see the lines in Fig 3, it also is made up of four tiles. This is setup just like a normal tile pyramid with four tiles under each upper tile in a kind of quad tree arrangement. In this case very simple with just the 2 levels.

I tied some events to the MediaElements. The main pyramid events are tied to MouseWheel events:

void Video_MouseWheel(object sender, MouseWheelEventArgs e)
{
    int delta = e.Delta;
    if (delta < 0)
    {
      //zoom out
      VideoZoomLevel--;
      if (e.OriginalSource.GetType() == typeof(MediaElement))
      {
        VideoCnt = 0;
        MediaElement me = e.OriginalSource as MediaElement;
        currentPostion = me.Position;
        v00.Source = new Uri("http://az1709.vo.msecnd.net/video/v00.wmv");
        v10.Source = new Uri("http://az1709.vo.msecnd.net/video/v10.wmv");
        v11.Source = new Uri("http://az1709.vo.msecnd.net/video/v11.wmv");
        v01.Source = new Uri("http://az1709.vo.msecnd.net/video/v01.wmv");
      }
    }
    else if (delta > 0)
    {
      //zoom in
      if (e.OriginalSource.GetType() == typeof(MediaElement))
      {
        VideoZoomLevel++;
        if (VideoZoomLevel <= maxVideoZoom)
        {
            VideoCnt = 0;
            MediaElement me = e.OriginalSource as MediaElement;
            currentPostion = me.Position;
            string quad = me.Source.LocalPath.Substring(0, me.Source.LocalPath.IndexOf(".wmv"));

            v00.Source = new Uri("http://az1709.vo.msecnd.net" + quad + "00.wmv");
            v10.Source = new Uri("http://az1709.vo.msecnd.net" + quad + "10.wmv");
            v11.Source = new Uri("http://az1709.vo.msecnd.net" + quad + "11.wmv");
            v01.Source = new Uri("http://az1709.vo.msecnd.net" + quad + "01.wmv");
        }
        else
        {
            VideoZoomLevel = maxVideoZoom;
        }
      }
    }
}

I'm just checking a MouseWheel delta to determine whether to go in or out. Then looking at the original source I determine which quad the mouse is over and then create the new URIs for the new Zoom Level. This is not terribly sophisticated. Not surprisingly the buffering is what is the killer. There are some MediaOpen and Load events which I attempted to use, however, there were quite a few problems with synchronizing the four tiles.

If you can patiently wait for the buffering it does work after a fashion. Eventually the wmv are in local cache which helps. However, the whole affair is fragile and erratic.

I didn't attempt to go any further with panning across the Zoom Level 2. I guess buffering was the biggest problem. I'm not sure how much further I could get trying to move to a Streaming Media server or monitoring BufferProgress with a timer thread.

The experiment may have been a failure, but the concept is none the less interesting. Perhaps some day a sophisticated codec will have such things built in.

The high altitude perspective

One aspect which makes MultiScaleVideo interesting is just its additional dimension of interactivity. As film moves inexorably to streaming internet, there is more opportunity for viewer participation. In a pyramid approach focus is in the viewer's hand. The remote becomes a focus tool that moves in and out of magnification levels as well as panning across the video 2D surface.

In the business world this makes interfaces to continuous data collections even more useful. As in the video feed experiment of the previous post, interfaces can scan at low zoom levels and then zoom in for detailed examination of items of interest. Streetside photos in the example Streetside path navigation already hint at this, using the run navigation to animate a short photo stream while also providing zoom and pan DeepZoom capability.

One of the potential pluses for this, from a distributor point of view, is repeat viewer engagement. Since the viewer is in control, any viewing is potentially unique, which discourages the typical view and discard common with film videos today. This adds value to potential advertisement revenue.

The film producer also has some incentive with a whole new viewer axis to play with. Now focus and peripheral vision take on another dimension, and focal point clues can create more interaction or in some cases deceptive side trails in a plot. Easter eggs in films provide an avid fan base with even more reason to view a film repeatedly.

Finally, small form factor hand held viewers such as iPhone and Android enabled cell phones can benefit from some form of streaming that allows user focus navigation. The screen in these cases is small enough to warrant some navigation facility. Perhaps IMAX or even Red 4K on handhelds is unreasonable, but certainly allowing navigation makes even the more common HD codecs more useable. A video pyramid of streaming sources could make a compelling difference in the handheld video market.

Summary

MultiScaleVideo is a way to enhance user interaction in a 2D video. It doesn't approach the game level interaction of true 3D scene graphs, but it does add another axis of interest. My primitive exercise was not successful. I am hoping that Microsoft Labs will someday make this feasible and add another type of Navigation to the arsenal. Of course, you can imagine the ensuing remote controller wars if DeepZoom Video ever becomes common place.

One more thing, check out the cool scaling animation attached to the expander button, courtesy of Silverlight Toolkit Nov drop.

Azure Video and the Silverlight Path

Tuesday, December 22nd, 2009


Fig 1 – Video Synched to Map Route

.
My last project experimented with synching Streetside with a Route Path. There are many other continuous asset collections that can benefit from this approach. Tom Churchill develops very sophisticated software for video camera augmentation, Churchill Navigation. He managed to take some time out of a busy schedule to do a simple drive video for me to experiment with.

In this scenario a mobile video camera is used along with a GPS to produce both a video stream and a simultaneous stream of GPS NMEA records. NMEA GPRMC records include a timestamp and latitude, longitude along with a lot of other information, which I simply discarded in this project.

First the GPS data file was converted into an xml file. I could then use some existing xml deserializer code to pull the positions into a LocationCollection. These were then used in Bing Maps Silverlight Control to produce a route path MapPolyline. In this case I didn’t get fancy and just put the xml in the project as an embedded resource. Certainly it would be easy enough to use a GPS track table from SQL Server, but I kept it simple.

NMEA GPRMC Record Detail:

$GPRMC,050756,A,4000.8812,N,10516.7323,W,20.2,344.8,211109,10.2,E,D*0E

0	$GPRMC,	http://www.gpsinformation.org/dale/nmea.htm#RMC
1	050756,	time 05:07:56
2	A,		Active A | V
3	4000.8812,	Latitude
4	N,		North
5	10516.7323,	Longitude
6	W,		West
7	20.2,		ground speed in knots
8	344.8,	track angle degrees true
9	211109,	date 11/21/2009
10	10.2,		magnetic variation
11	E,		East
12	D*0E		Checksum

XML resulting from above NMEA record

<?xml version="1.0" encoding="utf-16"?>
<ArrayOfLocationData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <LocationData>
    <ID>050754</ID>
    <Description>Boulder GPS</Description>
    <Location>
      <Latitude>40.0145083333</Latitude>
      <Longitude>-105.278808333</Longitude>
      <Altitude>0</Altitude>
      <AltitudeReference>Ground</AltitudeReference>
    </Location>
  </LocationData>
  <LocationData>
    <ID>050756</ID>
    <Description>Boulder GPS</Description>
    <Location>
      <Latitude>40.0146866667</Latitude>
      <Longitude>-105.278871667</Longitude>
      <Altitude>0</Altitude>
      <AltitudeReference>Ground</AltitudeReference>
    </Location>
  </LocationData>
      .
      .

Once the route path MapPolyline is available I can add a vehicle icon similar to the last streetside project. The icon events are used in the same way to start an icon drag. Mouse moves are handled in the Map to calculate a nearest point on the path and move the icon constrained to the route. The Mouse Button Up event is handled to synch with the video stream. Basically the user drags a vehicle along the route and when the icon is dropped the video moves to that point in the video timeline.

Video is a major focus of Silverlight. Microsoft Expression Encoder 3 has a whole raft of codecs specific to Silverlight. It also includes a dozen or so templates for Silverlight players. These players are all ready to snap in to a project and include all the audio volume, video timeline, play-stop-pause, and other controls found in any media player. The styling however, is different with each template, which makes life endurable for the aesthetically minded. I am not, so the generic gray works fine for my purposes. When faced with fashion or style issues my motto has always been “Nobody will notice,” much to the chagrin of my kids.

Expression Encoder 3 Video Player Templates

  • Archetype
  • BlackGlass
  • Chrome
  • Clean
  • CorporateSilver
  • Expression
  • FrostedGallery
  • GoldenAudio
  • Graphing
  • Jukebox
  • Popup
  • QuikSilver
  • Reflection
  • SL3AudioOnly
  • SL3Gallery
  • SL3Standard

At the source end I needed a reliable video to plug into the player template. I had really wanted to try out the Silverlight Streaming Service, which was offered free for prototype testing. However, this service is being closed down and I unfortunately missed out on that chance.

Tim Heuer’s prolific blog has a nice introduction to an alternative.

As it turns out I was under the mistaken impression that “Silverlight Streaming” was “streaming” video. I guess there was an unfortunate naming choice which you can read about in Tim’s blog post.

As Tim explains, Azure is providing a new Content Delivery Network CTP. This is not streaming, but it is optimized for rapid delivery. CDN is akin to Amazon’s Cloud Front. Both are edge cache services that boast low latency and high data transfer speeds. Amazon’s Cloud Front is still Beta, and Microsoft Azure CDN is the equivalent in Microsoft terminology, CTP or Community Technical Preview. I would not be much surprised to see a streaming media service as part of Azure in the future.

Like Cloud Front, Azure CDN is a promoted service from existing Blob storage. This means that using an Azure storage account I can create a Blob storage container, then enable CDN, and upload data just like any other blob storage. Enabling CDN can take awhile. The notice indicated 60 minutes, which I assume is spent allocating resources and getting the edge caching scripts in place.

I now needed to add Tom’s video encoded as 640×480 WMV up to the blob storage account with CDN enabled. The last time I tried this there wasn’t a lot of Azure upload software available. However, now there are lots of options:

Cloud Berry Explorer and Cloud Storage Studio were my favorites but there are lots of codeplex open source projects as well.

Azure Storage Explorer

Factonomy Azure Utility(Azure Storage Utility)

SpaceBlock

Azure Blob Storage Client

I encountered one problem, however, in all of the above. Once my media file exceeded 64Mb, which is just about 5min of video for the encoding I chose, my file uploads consistently failed. It is unclear whether the problem is at my end or the upload software. I know there is a 64Mb limit for simple blob uploads but most uploads would use a block mode not simple mode. Block mode goes all the way up to 50GB in current CTP which is a very long video. (roughly 60 hours at this encoding)

When I get time I’ll return to the PowerShell approach and manually upload my 70Mb video sample as a block. In the meantime I used Expression Encoder to clip the video down a minute to a 4:30 min clip for testing purposes.

Here are the published Azure Storage limits once it is released:

•Blobs:
200GB for block blobs (64KB min, 4MB max block size)
64MB is the limit for a single blob before you need to use blocks
1TB for page blobs


Fig 2 – Video Synched to Map Route

Now there’s a video in the player. The next task is to make a two way connect between the route path from the GPS records and the video timeline. I used a 1 sec timer tick to check for changes in the video timeline. This time is then used to loop through the route nodes defined by gps positions until reaching a time delta larger or equal to the current video time. At that point the car icon position is updated. This positions within a 1-3second radius of accuracy. It would be possible to refine this using a segment percentage approach and get down to the 1sec timer tick radius of accuracy, but I’m not sure increased accuracy is helpful here.

The reverse direction uses the current icon position to keep track of the current segment. Since the currentSegment also keeps track of endpoint delta times, it is used to set the video player position with the MouseUp event. Now route path connects to video position and as video is changed the icon location of the route path is also updated. We have a two way synch mode between the map route and the video timeline.

  private void MainMap_MouseMove(object sender, MouseEventArgs e)
  {
    Point p = e.GetPosition(MainMap);
    Location LL = MainMap.ViewportPointToLocation(p);
    LLText.Text = String.Format("{0,10:0.000000},{1,11:0.000000}", LL.Latitude, LL.Longitude);
    if (cardown)
    {
    currentSegment = FindNearestSeg(LL, gpsLocations);
    MapLayer.SetPosition(car, currentSegment.nearestPt);
    }
  }

FindNearestSeg is the same as the previous blog post except I’ve added time properties at each endpoint. These can be used to calculate video time position when needed in the Mouse Up Event.

  private void car_MouseLeftButtonUp(object sender, MouseButtonEventArgs e)
  {
    if (cardown)
    {
    cardown = false;
    VideoBoulder1.Position = currentSegment.t1;
    }
  }

Silverlight UIs for continuous data

This is the second example of using Silverlight Control for route path synching to a continuous data collection stream. In the first case it was synched with Streetside and in this case to a gps video. This type of UI could be useful for a number of scenarios.

There is currently some interest in various combinations of mobile Video and LiDAR collections. Here are some examples

  • Obviously both Google and Microsoft are busy collecting streetside views for ever expanding coverage.
  • Utility corridors, transmission, fiber, and pipelines, are interested in mobile and flight collections for construction, as built management, impingement detection, as well as regulatory compliance.
    Here are a couple representative examples:
      Baker Mobile
      Mobile Asset Collection MAC Vehicle
  • Railroads have a similar interest
      Lynx Mobile
  • DOTs are investigating mobile video LiDAR collection as well
      Iowa DOT Research

Summary

Mobile asset collection is a growing industry. Traditional imagery, video, and now LiDAR components collected in stream mode are becoming more common. Silverlight’s dual media and mapping controls make UIs for managing and interfacing with these type of continuous assets not only possible in a web environment, but actually fun to create.

Streetside in Silverlight

Wednesday, December 16th, 2009


Fig 1 – Streetside in Silverlight

A flood of announcements have been coming up over in Bing Land. You can try some of these out here: Bing Maps Explore

Significant is the “Bing Maps Silverlight Control Extended Modes Beta and Control 1.0.1.0″ coming just a week or two after initial release 1.0.0.0 of Bing Maps Silverlight Control. I have to get a handle on these names, so regretfully, let’s assume the military style – “BMSC.” These are separate from the the larger stable of Silverlight 4 Beta releases, recently announced.

Chris Pendleton’s Bing Blog (Why the urge to add “Bop” after this?) has some early detail on how to install, what is included, and how to make use of it.

What is included of note are two new map modes:

    BirdseyeMode.AddModeToNavigationBar(myMap);
    StreetsideMode.AddModeToNavigationBar(myMap);

Birdseye adds a pseudo 3D imagery mode that lets us move around in oblique space with realistic helicopter view angles. We are given tantalizing heading rotation capabilities, but limited to the compass quadrants, north, south, east, and west.

Streetside is the additional deep zoom street side photo views which afford zoom pan and a navigation run figure for hustling along the street. Streetside navigation includes circular heading, vertical pitch, and magnified zoom capabilities. Anyone used to the DeepZoom affects will find this familiar, but the running figure adds some travel legs to the process.

Neither of these modes are universal at this point. Here is the coverage extent shown in cyan for Streetside photos:


Fig 2 – Streetside current extent



Fig 3 – Streetside current extent at more detail

How to make use of these riches?

After downloading and replacing BMSC 1.0.0.0 with the new BMSC 1.0.1.0, I then installed the BMSCEM Beta and I’m ready to do some experiments. One thought that occurred to me was to add some streetside to a rout demonstration. The idea is to have an additional panel in a more prosaic geocode/route demo that will show streetside photos along the route path returned by the Bing Maps Web Services Route Service i.e. BMWSRS hmmmm? even the military might balk at this? Fig1 above shows how this might look in Road mode. This proceeds in several steps:

  1. Geocode with BMWSGS the start and end addresses
  2. Route between the two locations with BMWSRS
  3. Turn the resulting RoutePath into a MapPolyline with Pushpin images at start and end
  4. Add vehicle icon at the start
  5. Add some event handlers to allow users to drag the car icon constrained to the route path
  6. And finally connect the car icon position to streetside photos shown in the panel


Fig 4 – Streetside in Silverlight Aerial mode

In the sample you can see the navigation figure inside the streetside panel that lets users move inside the streetside photo space.

It is interesting to see how route path overlays are affected by the several map modes including Birdseye and Streetside. As you can see from the next couple of figures the Route MapPolyline does not pass through a transform to match either Birdseye or Streetside view.


Fig 5 – Streetside in Silverlight Birdseye mode
(note route path is shifted off to south by about 9 blocks)


Fig 6 – Streetside mode with heading rotated to see a route MapPolyline that is totally unrelated to the photo view

In Fig 5 and Fig 6 above, you can see that the streetside photos and Birdseye perspective are taking advantage of 3D perspective transforms, but the custom layer is aligned with the tilted road mode hidden behind the photo/image. It appears that visibility of other Map Layers will need to be part of a mode switch between Road or Aerial and Birdseye and Streetside.

Aside from that another interesting aspect of this example is the route constraint algorithm. The user fires a
MouseLeftButtonDown += car_MouseLeftButtonDown
event attached to the car icon to start the process which flags cardown mode true. However, the corresponding button up event is added to the MainMap rather than the icon.
MainMap.MouseLeftButtonUp += car_MouseLeftButtonUp;

This insures that the user can continue moving his mouse with the left down even while no longer over the car icon and that we can continue to lookup nearest point on route path for constraining our icon to the path.

private void car_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{
e.Handled = true;
cardown = true;
}

private void car_MouseLeftButtonUp(object sender, MouseButtonEventArgs e)
{
if (cardown) cardown = false;
}

The real action, though, takes place in the MainMap MouseMove event, where we find the nearest point on the route path, set the car map position, set the streetside view location, and finally calculate the streetside view heading from the segment vector:

Point p = e.GetPosition(MainMap);
Microsoft.Maps.MapControl.Location LL = MainMap.ViewportPointToLocation(p);

 if(cardown) {
  Segment s = FindNearestSeg(LL, routePts);

  MapLayer.SetPosition(car, s.nearestPt);
  StreetviewMap.Mode.Center = s.nearestPt;
  double dx = (s.linePt2.Longitude - s.linePt1.Longitude);
  double dy = (s.linePt2.Latitude - s.linePt1.Latitude);
  double angle = 0;

  if (dx > 0) angle = (Math.PI * 0.5) - Math.Atan(dy / dx);
  else if (dx < 0) angle = (Math.PI * 1.5) - Math.Atan(dy / dx);
  else if (dy > 0) angle = 0;
  else if (dy < 0) angle = Math.PI;
  else angle = 0.0;// the 2 points are equal  
  angle *= (180 / Math.PI);
  HeadingText.Text = String.Format("{0,10:0.000000}", angle);
  StreetviewMap.Heading = angle;

}

Here is some code adapted from my venerable Bowyer and Woodwark “Programmer’s Geometry” book. Yes, they used to have books for this type of algorithm. Bascially the code loops through each segment and finds the nearest perpindicular point of intersection using parametric lines. Since I’m only comparing for the least distance, I don’t need to use the Math.Sqrt function to get actual distances.

private Segment FindNearestSeg(Location LL, LocationCollection routePts){

  Segment s = new Segment();
  double dist = Double.MaxValue;
  double d = 0;
  double t = 0;

  int pos = 0;
  Location pt0 = new Location();
  foreach (Location pt1 in routePts)
  {
    if (pos++ > 0)
    {
      double XKJ = pt0.Longitude - LL.Longitude;
      double YKJ = pt0.Latitude - LL.Latitude;
      double XLK = pt1.Longitude - pt0.Longitude;
      double YLK = pt1.Latitude - pt0.Latitude;
      double denom = XLK * XLK + YLK * YLK;
      if (denom == 0)
      {
        d = (XKJ * XKJ + YKJ * YKJ);
      }
      else
      {
        t = -(XKJ * XLK + YKJ * YLK) / denom;
        t = Math.Min(Math.Max(t, 0.0), 1.0);
         double xf = XKJ + t * XLK;
         double yf = YKJ + t * YLK;
        d = xf * xf + yf * yf;
      }
      if (d < dist)
      {
        dist = d;
        s.nearestPt = new Location(pt0.Latitude + t * YLK, pt0.Longitude + t * XLK);
        s.linePt1 = new Location(pt0.Latitude, pt0.Longitude);
        s.linePt2 = new Location(pt1.Latitude, pt1.Longitude);
      }
    }
    pt0 = pt1;
  }

  return s;

}

Here is a small class to hold the results of the minimal distance segment and intersect point:

public class Segment    
{
 public Location linePt1 { get; set; }
 public Location linePt2 { get; set; }
 public Location nearestPt { get; set; }
}

All of this is fired with each mouse move event to scan through the route path nodes and get the segment nearest to the current user position. That this all works smoothly is another testimony to the advantage of CLR over Javascript on the client.

The resulting Segment then holds the segment end points and the intersection point. The intersection point is available for placing the car icon position as well as the StreetviewMap.Mode.Center in our Streetside panel. The additional segment endpoints are handy for calculating the heading of the streetside view direction. This is used to point along the vector direction of the path segment by calculating the compass direction and applying it to the StreetviewMap.Heading = angle;

The result slides the car icon along our route path while updating the Streetside View panel with a StreetviewMap heading in the same direction as the route.

Unfortunately, the Streetview loading is currently too slow to do a film strip down the route, which is too bad as my next idea was an animated route with concurrent streetside views from whatever direction (angular delta) set in Streetside View by the user. Of course the run navigator does a good job of moving in heading direction, it’s just not in my control. If I can get control of this navigator, I would try a run navigate along a route leg until reaching a waypt, and then turn to a new heading before starting navigation run again for the next route leg.

However, I just discovered that by adding a ViewChangeEnd event to my streetview panel,
StreetviewMap.ViewChangeEnd += car_Update;,
I can use the Center location to update my car location.
MapLayer.SetPosition(car, StreetviewMap.Mode.Center);
Now I can hustle down the street with the Streetside Navigation runner, and at least have my car location shown on the map. Cool!


Summary

Streetview and Birdseye are nice additions to the list of map modes. They have some restrictions in availability, but where they are available, add some useful features. My experiment just scratches the surface available with Silverlight Control Extension Modes.

Here are a few problems that surfaced with my example:

  1. The routepath returned by RouteService can be just slightly off the route traveled and consequently the Streetside location will be incorrect. Also setting StreetviewMap.Mode.Center to a location will snap to the closest street photo even if it is off route at an intersection. This appears as a series of streetviews for streets crossed by a route instead of on the route desired. At least that’s my theory for the occasional odd view of lawns instead of the street ahead.
  2. Map children layers are not automatically transformed to these new views and mode change events will need to manipulate visibility of custom layers.
  3. I had to turn off drag on the Streetview panel so that users can do normal streetview pan and zoom.
  4. It is necessary to check for a valid streetside photo. If the route runs off streetside coverage, the last valid streetside view location remains in the panel regardless of the new icon positions.

Obviously “alternate reality” continues to converge with “ordinary reality” at a steady pace, at least in web mapping worlds.

Silverlight Map Pixel Shader Effects

Thursday, December 3rd, 2009


Fig 1 – Pixel Shader Effects

Sometimes it would be nice to change the standard color scheme of well known map sources. I recently saw a blog post by Ricky Brundritt that shows how to make color map changes to Bing Maps tiles. By applying a new skin with a pixel by pixel color mapping, there are some interesting effects possible. At least it can be a little out of the ordinary.

Silverlight and WPF XAML include an Effect property for all UIElements. There are currently a couple of built in Silverlight Effects, BlurEffect and DropShadowEffect, which can be added to Silverlight UIElements including the Silverlight Map control. However, these are not especially interesting, and I assume Microsoft will grow this list over time.

This video post by Mike Taulty goes further by showing how to create your own Pixel Effects:
http://www.silverlight.net/learn/videos/all/pixel-effects/

Creating your own Pixel Effects involves the DirectX SDK and some DirectX code that is a little involved, but fortunately there is a codeplex project which has a set of WPF Pixel Shader Effects.

Here is a video showing the 23 different Pixel Effects currently available: Video WPF Pixel Shader

Here is an example of a DirectX fx file for EmbossedEffect that looks like this:

//--------------------------------------------------------------------------------------
//
// WPF ShaderEffect HLSL -- EmbossedEffect
//
//--------------------------------------------------------------------------------------

//-----------------------------------------------------------------------------------------
// Shader constant register mappings (scalars - float, double, Point, Color, Point3D, etc.)
//-----------------------------------------------------------------------------------------

float Amount : register(C0);
float Width : register(C1);

//--------------------------------------------------------------------------------------
// Sampler Inputs (Brushes, including ImplicitInput)
//--------------------------------------------------------------------------------------

sampler2D implicitInputSampler : register(S0);

//--------------------------------------------------------------------------------------
// Pixel Shader
//--------------------------------------------------------------------------------------

float4 main(float2 uv : TEXCOORD) : COLOR
{
   float4 outC = {0.5, 0.5, 0.5, 1.0};

   outC -= tex2D(implicitInputSampler, uv - Width) * Amount;
   outC += tex2D(implicitInputSampler, uv + Width) * Amount;
   outC.rgb = (outC.r + outC.g + outC.b) / 3.0f;

   return outC;
}

This method, float4 main(float2 uv : TEXCOORD) : COLOR takes the pixel position (u,v) and modifies the color. Once this has been compiled into a .ps file it can be used as a UIElement Effect property in Silverlight.

This blog post by Pravinkumar Dabade explains the details of using the Silverlight version of this codeplex Pixel Effect library.
http://pkrd.blogspot.com/2009/07/23-pixel-shader-effects-in-silverlight.html

After compiling the WPFSLFx project, it is easy to add a reference to SLShaderEffectLibrary.dll in a Bing Maps Silverlight Control project. Now you can start adding Pixel Effects to the map controls as in this shader:EmbossedEffect:

 <m:Map
        Name="EmbossedMap"
        CredentialsProvider=""
        NavigationVisibility="Visible"
        Grid.Column="0" Grid.Row="1" Grid.RowSpan="1" Padding="5"
        Mode="Aerial">
     <m:Map.Effect>
         <shader:EmbossedEffect x:Name="embossedEffect" Amount="1" Width="0.001" />
     </m:Map.Effect>
 </m:Map>

Note each type of effect has different sets of input parameters. In the case of the EmbossedEffect an amount and width are required, which I’ve bound to sliders. The simpler InvertEffect or MonochromeEffect require no parameters.


Fig 2 – Pixel Shader EmbossedEffect



Fig 3 – Pixel Shader EmbossedEffect

Multiple Effects can be applied by wrapping a Map inside a Border. Here is a shader:ContrastAdjustEffect + shader:InvertColorEffect:

<Border Visibility="Visible" Grid.Column="0" Grid.Row="1" Grid.RowSpan="1" Padding="5" >
    <Border.Effect>
        <shader:ContrastAdjustEffect
	x:Name="contrastEffect"
        	Contrast="1"
        	Brightness="0"
         />
    </Border.Effect>

    <m:Map  Visibility="Collapsed"
       Name="InvertMap"
       CredentialsProvider=""
       NavigationVisibility="Visible"
       Grid.Column="0" Grid.Row="1" Grid.RowSpan="1" Padding="5"
       Mode="Road">
        <m:Map.Effect>
            <shader:InvertColorEffect  x:Name="invertEffect" />
        </m:Map.Effect>
    </m:Map>
</Border>



Fig 4 – ContrastAdjustEffect + InvertColorEffect



Fig 5 – ContrastAdjustEffect + InvertColorEffect

Effects are applied progressivly down the rendering tree, which means additive effects of arbitrary length are possible. You can use any enclosing UIElement to stack your Effects tree such as MapLayer:

<m:Map  Visibility="Visible"
        Name="MainMap"
        CredentialsProvider=""
        NavigationVisibility="Visible"
        Grid.Column="0" Grid.Row="1" Grid.RowSpan="1" Padding="5"
        Mode="Road">
     <m:MapLayer x:Name="effectsLayer1">
         <m:MapLayer x:Name="effectsLayer2">
             <m:MapLayer x:Name="effectsLayer3">

             </m:MapLayer>
         </m:MapLayer>
     </m:MapLayer>
 </m:Map>

Because these UIElement Pixel Effects are compiled to DirectX and are run in the client GPU they are extremely effficient (as long as the client has a GPU). You can even use them on video media.

public void addVideoToMap()
{
    MediaElement video = new MediaElement();
    video.Source = new Uri(
@"http://mschnlnine.vo.llnwd.net/d1/ch9/9/2/9/9/1/4/TCS2NBCOlympics_ch9.wmv",
UriKind.RelativeOrAbsolute);
    video.Opacity = 0.8;
    video.Width = 250;
    video.Height = 200;
    video.Effect = new ShaderEffectLibrary.InvertColorEffect();
    Location location = new Location(39,-105);
    PositionOrigin position = PositionOrigin.Center;
    effectsLayer3.AddChild(video, location, position);
    effectsLayer2.Effect = new ShaderEffectLibrary.MonochromeEffect();
    effectsLayer1.Effect = new ShaderEffectLibrary.SharpenEffect();
}



Fig 6 – Effects stacked on Video Element

Summary

Pixel Effects can add some interest to an ordinary map view. With the WPF Pixel Shader Effects library, effects are easy to add to Silverlight Bing Maps Control Layers and even media elements. In viewing imagery it is sometimes nice to have Effects like ShaderEffectLibrary.SharpenEffect() to enhance capture or imagery interpretation so this is not just about aesthetics.

Big Bing Maps Silverlight Control 1.0 Release Today

Tuesday, November 10th, 2009

Lots of interesting new stuff to explore in this 1.0 release of the Bing Maps Silverlight Control. That seems like a mouthful, and I’m sure it will be acronymed to something like BMS Control, but the one thing that immediately stands out from the announcement is this:

“Sessions will be used with the Bing Maps AJAX Map Control and the Bing Maps Silverlight Control. A session is basically defined as loading the map control and exploring at will, no tile limitations.”

  • “Bing Maps AJAX Control all maps rendered onto the client upon the initial request is considered 1 session. Session includes any requests for geocoding, routing or search.”
  • “Bing Maps Silverlight Control – all maps rendered onto the client upon the initial request is considered 1 session. There are no services built into the Bing Maps Silverlight Control, so you would use the Bing Maps Web Service for geocoding, routing and search, but will include those too.”
  • “Bing Maps Web Service all maps, geocodes, routes and searches will each invoke 1 transaction.”

“With the new terms of use for the Bing Maps Platform you get 125,000 sessions per year for FREE. You also get 500,000 transactions a year for FREE. “

Educators – free unlimited use of the Bing Maps platform

Not-for-Profits – free unlimited use of the Bing Maps platform

Commercial, non-commercial and government – proof of concept development free

More details here:
Bing Maps terms of use changes benefit educators, not-for-profits, and developers
Bing Maps Silverlight Control 1.0 released
Terms of Use

This alleviates a big concern I had originally with use of the Silverlight Map Control CTP. Transaction based licensing did not align with Google pricing and was nearly impossible to predict for tile navigation, which is the engaging part of Silverlight Control. This announcement wipes out these problems and makes my job as a developer a whole lot easier.

Thanks Microsoft!

Azure and GeoWebCache tile pyramids

Friday, October 23rd, 2009


Azure Blob storage tile pyramid
Fig 1 – Azure Blob Storage tile pyramid for citylimits

Azure Overview

Shared resources continue to grow as essential building blocks of modern life, key to connecting communities and businesses of all types and sizes. As a result a product like SharePoint is a very hot item in the enterprise world. You can possibly view Azure as a very big, very public, SharePoint platform that is still being constructed. Microsoft and 3rd party services will eventually populate the service bus of this Cloud version with lots and lots of service hooks. In the meantime, even early stage Azure with Web Hosting, Blob storage, and Azure SQL Server makes for some interesting experimental R&D.

Azure is similar to Amazon’s AWS cloud services, and Azure’s pricing follows Amazon’s lead with the familiar “pay as you go, buy what you use” model. Azure offers web services, storage, and queues, but instead of giving access to an actual virtual instance, Azure provides services maintained in the Microsoft Cloud infrastructure. Blob storage, Azure SQL Server, and IIS allow developers to host web applications and data in the Azure Cloud, but only with the provided services. The virtual machine is entirely hidden inside Microsoft’s Cloud.

The folks at Microsoft are probably well aware that most development scenarios have some basic web application and storage component, but don’t really need all the capabilities, and headaches, offered by controlling their own server. In return for giving up some freedom you get the security of automatic replication, scalability, and maintenance along with the API tools to connect into the services. In essence this is a Microsoft only Cloud since no other services can be installed. Unfortunately, as a GIS developer this makes Azure a bit less useful. After all, Microsoft doesn’t yet offer GIS APIs, OGC compliant service platforms, or translation tools. On the other hand, high availability with automatic replication and scalability for little effort are nice features for lots of GIS scenarios.

The current Azure CTP lets developers experiment for free with these minor restrictions:

  • Total compute usage: 2000 VM hours
  • Cloud storage capacity: 50GB
  • Total storage bandwidth: 20GB/day


To keep things simple, since this is my first introduction to Azure, I looked at just using Blob Storage to host a tile pyramid. The Silverlight MapControl CTP makes it very easy to add tile sources as layers so my project is simply to create a tile pyramid and store this in Azure Blob storage where I can access it from a Silverlight MapControl.

In order to create a tile pyramid, I also decided to dig into the GeoWebCache standalone beta 1.2. This is beta and offers some new undocumented features. It also is my first attempt at using geowebcache as standalone. Generally I just use the version conveniently built into Geoserver. However, since I was only building a tile pyramid rather than serving it, the standalone version made more sense. Geowebcache also provides caching for public WMS services. In cases where a useful WMS is available, but not very efficient, it would be nice to cache tiles for at least subsets useful to my applications.

Azure Blob Storage

Azure CTP has three main components:

  1. Windows Azure – includes the storage services for blobs, queues, and cloud tables as well as hosting web applications
  2. SQL Azure – SQL Server in the Cloud
  3. .NET Services – Service Bus, Access Control Service, Work Flow …

There are lots of walk throughs for getting started in Azure. It all boils down to getting the credentials to use the service.

Once a CTP project is available the next step is to create a “Storage Account” which will be used to store the tile pyramid directory. From your account page you can also create a “Hosted Service” within your Windows Azure project. This is where web applications are deployed. If you want to use “SQL Azure” you must request a second SQL Azure token and create a SQL Service. The .NET Service doesn’t require a token for a subscription as long as you have a Windows Live account.

After creating a Windows Azure storage account you will get three endpoints and a couple of keys.

Endpoints:
http://sampleaccount.blob.core.windows.net/
http://sampleaccount.queue.core.windows.net/
http://sampleaccount.table.core.windows.net/

Primary Access Key: ************************************
Secondary Access Key: *********************************

Now we can start using our brand new Azure storage account. But to make life much simpler first download the following:

Azure SDK
Azure Tools for Visual Studio

Azure SDK includes some sample code . . . HelloWorld, HelloFabric, etc to get started using the Rest interface. I reviewed some of the samples and started down the path of creating the necessary Rest calls for recursively loading a tile pyramid from my local system into an Azure blob storage nomenclature. I was just getting started when I happened to take a look at the CloudDrive sample. This saved me a lot of time and trouble.

CloudDrive lets you treat the Azure service as a drive inside PowerShell. The venerable MSDOS cd, dir, mkdir, copy, del etc commands are all ready to go. Wince, I know, I know, MSDOS? I’m sure, if not now, then soon there will be dozens of tools to do the same thing with nice drag and drop UIs. But this works and I’m old enough to actually remember DOS commands.

First, using the elevated Windows Azure SDK command prompt you can compile and run the CloudDrive with a couple of commands:

C:\AzureTools\samples\CloudDrive\buildme.cmd
C:\AzureTools\samples\CloudDrive\runme.cmd

Now open Windows PowerShell and execute the MounteDrive.ps1 script. This allows you to treat the local Azure service as a drive mount and start copying files into storage blobs.


Azure sample CloudDrive PowerShell
Fig 1 – Azure sample CloudDrive PowerShell

Creating a connection to the real production Azure service simply means making a copy of MountDrive.ps1 and changing credentials and endpoint to the ones obtained previously.

function MountDrive {
Param (
 $Account = "sampleaccount",
 $Key = "***************************************",
 $ServiceUrl="http://sampleaccount.blob.core.windows.net/",
 $DriveName="Blob",
 $ProviderName="BlobDrive")

# Power Shell Snapin setup
 add-pssnapin CloudDriveSnapin -ErrorAction SilentlyContinue

# Create the credentials
 $password = ConvertTo-SecureString -AsPlainText -Force $Key
 $cred = New-Object -TypeName Management.Automation.PSCredential -ArgumentList $Account, $password

# Mount storage service as a drive
 new-psdrive -psprovider $ProviderName -root $ServiceUrl -name $DriveName -cred $cred -scope global
}

MountDrive -ServiceUrl "http://sampleaccount.blob.core.windows.net/" -DriveName "Blob" -ProviderName "BlobDrive"

The new-item command lets you create a new container with -Public flag ensuring that files will be accessible publicly. Then the Blog: drive copy-cd command will copy files and subdirectories from the local file system to the Azure Blob storage. For example:

PS Blob:\> new-item imagecontainer -Public
Parent: CloudDriveSnapin\BlobDrive::http:\\127.0.0.1:10000\devstoreaccount1

Type Size LastWriteTimeUtc Name
---- ---- ---------------- ----
Container 10/16/2009 9:02:22 PM imagecontainer

PS Blob:\> dir

Parent: CloudDriveSnapin\BlobDrive::http:\\127.0.0.1:10000\

Type Size LastWriteTimeUtc Name
---- ---- ---------------- ----
Container 10/16/2009 9:02:22 PM imagecontainer
Container 10/8/2009 9:22:22 PM northmetro
Container 10/8/2009 5:54:16 PM storagesamplecontainer
Container 10/8/2009 7:32:16 PM testcontainer

PS Blob:\> copy-cd c:\temp\image001.png imagecontainer\test.png
PS Blob:\> dir imagecontainer

Parent: CloudDriveSnapin\BlobDrive::http:\\127.0.0.1:10000\imagecontainer

Type Size LastWriteTimeUtc Name
---- ---- ---------------- ----
Blob 1674374 10/16/2009 9:02:57 PM test.png

Because imagecontainer is public the test.png image can be accessed in the browser from the local development storage with:
http://127.0.0.1:10000/devstoreaccount1/imagecontainer/test.png
or if the image was similarly loaded in a production Azure storage account:
http://sampleaccount.blob.core.windows.net/imagecontainer/test.png

It is worth noting that Azure storage consists of endpoints, containers, and blobs. There are some further subtleties for large blobs such as blocks and blocklists as well as metadata, but there is not really anything like a subdirectory. Subdirectories are emulated using slashes in the blob name.
i.e. northmetro/citylimits/BingMercator_12/006_019/000851_002543.png is a container, “northmetro“, followed by a blob name,
/citylimits/BingMercator_12/006_019/000851_002543.png.”

The browser can show this image using the local development storage:
http://127.0.0.1:10000/devstoreaccount1/northmetro/citylimits/BingMercator_12
/006_019/000851_002543.png

Changing to producton Azure means substituting a valid endpoint for “127.0.0.1:10000/devstoreaccount1″ like this:
http://sampleaccount.blob.core.windows.net/northmetro/citylimits/BingMercator_12
/006_019/000851_002543.png

With CloudDrive getting my tile pyramid into the cloud is straightforward and it saved writing custom code.

The tile pyramid – Geowebcache 1.2 beta

Geowebcache is written in Java and synchronizes very well with the GeoServer OGC service engine. The new 1.2 beta version is available as a .war that is loaded into the webapp directory of Tomcat. It is a fairly simple matter to configure geowebcache to create a tile pyramid of a particular Geoserver WMS layer. (Unfortunately it took me almost 2 days to work out a conflict with an existing Geoserver gwc) The two main files for configuration are:


C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps\
                     geowebcache1.2\WEB-INF\geowebcache-servlet.xml
C:\Program Files\Apache Software Foundation\Tomcat 6.0\webapps\
                    geowebcache1.2\WEB-INF\classes\geowebcache.xml

geowebcache-servlet.xml customizes the service bean parameters and geowebcache.xml provides setup parameters for tile pyramids of layers. Leaving the geowebcache-servlet.xml at default will work fine when no other Geoserver or geowebcache is around. It can get more complicated if you have several that need to be kept separate. More configuration info.

Here is an example geowebcache.xml that uses some of the newer gridSet definition capabilities. It took me a long while to find the schema for geowebcache.xml:
http://geowebcache.org/schema/docs/1.2.0/
The documentation is still thin for this beta release project.

<?xml version="1.0" encoding="utf-8"?>
<gwcConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:noNamespaceSchemaLocation="http://geowebcache.org/schema/1.2.0/geowebcache.xsd"
  xmlns="http://geowebcache.org/schema/1.2.0">
<version>1.2.0</version>
<backendTimeout>120</backendTimeout>
<gridSets>
  <gridSet>
  <name>BingMercator</name>
  <srs><number>900913</number></srs>
  <extent>
  <coords>
  <double>-11706995</double>
  <double>4839671</double>
  <double>-11687135</double>
  <double>4861458</double>
  </coords>
  </extent>
  <alignTopLeft>true</alignTopLeft>
  <levels>15</levels>
  </gridSet>
</gridSets>
<layers>
  <wmsLayer>
  <name>citylimits</name>
  <gridSubsets>
  <gridSubset>
  <gridSetName>BingMercator</gridSetName>
  <zoomStart>0</zoomStart>
  <zoomStop>10</zoomStop>
  </gridSubset>
  <gridSubset>
  <gridSetName>GoogleMapsCompatible</gridSetName>
  </gridSubset>
  </gridSubsets>
  <wmsUrl><string>http://localhost:80/geoserver/wms</string></wmsUrl>
  <wmsLayers>northmetro:citylimits</wmsLayers>
  <wmsStyles>citylimits</wmsStyles>
  </wmsLayer>
</layers>
</gwcConfiguration>

After editing the configuration files, building the pyramid is a matter of pointing your browser at the local webapp and seeding the tiles down to the level you choose with the gridSet you want. The GoogleMapsCompatible gridSet is built into geowebcache and the BingMercator is a custom gridSet that I’ve added with extent limits defined.
http://localhost/geowebcache1.2/rest/seed/citylimits

This can take a few hours/days depending on the extent and zoom level you need. Once completed I use the CloudDrive PowerShell to copy all of the tiles into Azure blob storage:

PS Blob:\> copy-cd C:\Program Files\Apache Software Foundation\Tomcat 6.0\temp\geowebcache\citylimits

This also takes some time for the resulting 243,648 files of about 1Gb.

Silverlight MapControl

The final piece in the project is adding the MapControl viewer layer. First I add a new tile source layer in the Map Control of the MainPage.xaml

  <m:Map
      Name="MainMap"
      NavigationVisibility="Visible"
      Grid.Column="0" Grid.Row="1" Grid.RowSpan="1" Padding="5"
      Mode="Road">
    <m:Map.Children>
       <!-- Azure tile source -->
       <m:MapTileLayer x:Name="citylimitsAzureLayer" Opacity="0.5" Visibility="Collapsed">
         <m:MapTileLayer.TileSources>
             <local:CityLimitsAzureTileSource></local:CityLimitsAzureTileSource>
         </m:MapTileLayer.TileSources>
      </m:MapTileLayer>
             .
             .

The tile naming scheme is described here:
http://geowebcache.org/trac/wiki/filestorage2
The important point is:

“Most filesystems use btree’s to store the files in directories, so layername/projection_z/[x/(2(z/2))]_[y/(2(z/2))]/x_y.extension seems reasonable, since it works sort of like a quadtree. The idea is that half the precision is in the directory name, the full precision in the filename to make it easy to locate problematic tiles. This will also make cache purges a lot faster for specific regions, since fewer directories have to be traversed and unlinked. “

An ordinary tile source class looks just like this:

  public class CityLimitsTileSource : Microsoft.VirtualEarth.MapControl.TileSource
  {
        public CityLimitsTileSource() : base(App.Current.Host.InitParams["src"] +
          "/geoserver/gwc/service/gmaps?layers=northmetro:citylimits&zoom={2}&x={0}&y={1}")
        {
        }

        public override Uri GetUri(int x, int y, int zoomLevel)
        {
           return new Uri(String.Format(this.UriFormat, x, y, zoomLevel));
        }
  }

However, now I need to reproduce the tile name as it is in the Azure storage container rather than letting gwc/service/gmaps mediate the nomenclature for me. This took a little digging. The two files I needed to look at turned out to be:

GMapsConverter works because Bing Maps follows the same upper left origin convention and spherical mercator projection as Google Maps. Here is the final approach using the naming system in Geowebcache1.2.

public class CityLimitsAzureTileSource : Microsoft.VirtualEarth.MapControl.TileSource
{
  public CityLimitsAzureTileSource()
  : base(App.Current.Host.InitParams["azure"] + "citylimits/GoogleMapsCompatible_{0}/{1}/{2}.png")
  {
  }

  public override Uri GetUri(int x, int y, int zoomLevel)
  {
   /*
   * From geowebcache
   * http://geowebcache.org/trac/browser/trunk/geowebcache/src/main/java/org/geowebcache/storage/blobstore/file/FilePathGenerator.java
   * http://geowebcache.org/trac/browser/trunk/geowebcache/src/main/java/org/geowebcache/service/gmaps/GMapsConverter.java
   * must convert zoom, x, y, and z into tilepyramid subdirectory structure used by geowebcache
  */
  int extent = (int)Math.Pow(2, zoomLevel);
  if (x < 0 || x > extent - 1)
  {
     MessageBox.Show("The X coordinate is not sane: " + x);
  }

  if (y < 0 || y > extent - 1)
  {
     MessageBox.Show("The Y coordinate is not sane: " + y);
  }
  // xPos and yPos correspond to the top left hand corner
  y = extent - y - 1;
  long shift = zoomLevel / 2;
  long half = 2 << (int)shift;
  int digits = 1;
  if (half > 10)
  {
     digits = (int)(Math.Log10(half)) + 1;
  }
  long halfx = x / half;
  long halfy = y / half;
  string halfsubdir = zeroPadder(halfx, digits) + "_" + zeroPadder(halfy, digits);
  string img = zeroPadder(x, 2 * digits) + "_" + zeroPadder(y, 2 * digits);
  string zoom = zeroPadder(zoomLevel, 2);
  string test = String.Format(this.UriFormat, zoom, halfsubdir, img );

  return new Uri(String.Format(this.UriFormat, zoom, halfsubdir, img));
  }

/**
  * From geowebcache
  * http://geowebcache.org/trac/browser/trunk/geowebcache/src/main/java/org/geowebcache/storage/blobstore/file/FilePathGenerator.java
  * a way to pad numbers with leading zeros, since I don't know a fast
  * way of doing this in Java.
  *
  * @param number
  * @param order
  * @return
  */
  public static String zeroPadder(long number, int order) {
  int numberOrder = 1;

  if (number > 9) {
    if(number > 11) {
      numberOrder = (int) Math.Ceiling(Math.Log10(number) - 0.001);
    } else {
      numberOrder = 2;
    }
  }

  int diffOrder = order - numberOrder;

    if(diffOrder > 0) {
      //System.out.println("number: " + number + " order: " + order + " diff: " + diffOrder);
      StringBuilder padding = new StringBuilder(diffOrder);

      while (diffOrder > 0) {
        padding.Append("0");
        diffOrder--;
       }
       return padding.ToString() + string.Format("{0}", number);
    } else {
      return string.Format("{0}", number);
    }
  }
}

I didn’t attempt to change the zeroPadder. Doubtless there is a simple C# String.Format that would replace the zeroPadder from Geowebcache.

This works and provides access to tile png images stored in Azure blob storage, as you can see from the sample demo.

Summary

Tile pyramids enhance user experience, matching the performance users have come to expect in Bing, Google, Yahoo, and OSM. It is resource intensive to make tile pyramids of large world wide extent and deep zoom levels. In fact it is not something most services can or need provide except for limited areas. Tile pyramids in the Cloud require relatively static layers with infrequent updates.

Although using Azure this way is possible and provides performance, scalability, and reliability, I’m not sure it always makes sense. The costs are difficult to predict for a high volume site as they are based on bandwidth usage as well as storage. Also you may be paying storage fees for many tiles seldom or never needed. Tile pyramid performance is a wonderful thing, but it chews up a ton of storage, much of which is seldom if ever used.

For a stable low to medium volume application it makes more sense to host a tile pyramid on your own server. Possibly with high volume sites where reliability is the deciding factor moving to Cloud storage services is the right thing. This is especially true where traffic patterns swing wildly or grow rapidly and robust scaling is an ongoing battle.

Azure CTP is of course not as mature as AWS, but obviously it has the edge in the developer community and like many Microsoft technologies it has staying power to spare. Leveraging its developer community makes sense for Microsoft and with easy to use tools built into Visual Studio I can see Azure growing quickly. In time it will just be part of the development fabric with most Visual Studio deployment choices seamlessly migrating out to the Azure Cloud.

Azure release is slated for Nov 2009.

Storm Runoff modelling and MRLC

Monday, October 12th, 2009

MRLC ImperviousSurface Screenshot
Fig 1 – Screen View showing ImperviousSurface layer from MRLC

The Multi-Resolution Land Characteristics Consortium, MRLC, is a consortium of agencies at the federal level that produces the National Land Cover Database, NLCD 2001. The dataset was developed using a national coverage set of 8 band Landsat-7 Imagery along with 30m DEM. The imagery is processed from three separate dates to give a seasonal average land cover classification. The resolution is a bit coarse at 30m square, but it is a valuable resource because of its consistent national coverage.

More detailed information on MRLC:

In addition to the NLCD coverage there are two derivative layers:

  • NLCD 2001 impervious surface: The impervious surface data classifies each pixel into 101 possible values (0% – 100%). The data show the detailed urban fabric in which many of us reside. Low percentage impervious is in light gray with increasing values depicted in darker gray and the highest value in pink and red. White areas have no impervious surface.
  • NLCD 2001 canopy density: Like the impervious surface data, the canopy density database element classifies each pixel into 101 possible values (0% – 100%). The canopy density estimate apply to the forest only. These data can be combined with the land cover to estimate canopy density by forest type (deciduous, evergreen, mixed, woody wetland)

The data is available for public download. There is also one of those vintage ESRI viewers that qualifies for James Fee’s “Cash for Geo Clunkers” proposal. These Ancien Régime viewers are littered all over the federal landscape. It will take years for newer technology to replace this legacy of ArcIMS. Fortunately there is an exposed WMS service ( See GetCapabilities ), which permits access to MRLC layers without going through the “Viewer” nonsense. This WMS service proved very useful on a recent web project for Storm Runoff Mitigation.

I am no hydrologist, but once I was provided with the appropriate calculation approach the Silverlight UI was fairly straightforward. Basically there are a number of potential mitigation practices that play into a runoff calculation. Two fairly significant factors are Impervious Surface Area and Canopy Area, which are both available through MRLC’s WMS service. One simplified calculation model in use is called the TR-55 method.

Runoff calculations TR-55
Fig 2 – TR-55 method of calculating runoff depth

By making use of the MRLC derived layers for Impervious Surface and Canopy at least approximations for these two factors can be derived for any given area of the Continental US. The method I used was to provide a GetMap request to the WMS service which then returned a pixel shaded image of the impervious surface density. Most of the hard work has already been done by MRLC. All I need to do is extract the value density for the returned png image.

Impervious Surface layer
Fig 3 – Impervious Surface shaded density red
Impervious Surface layer
Fig 4 – Canopy shaded density green

The density values are relative to gray. I at first tried a simple density calculation from the color encoded pixels by subtracting the base gray from the variable green: Green – Red = factor. The sum of these factors divided by the total pixel area of the image times the max 255 byte value is a rough calculation of the percentage canopy over the viewport. However, after pursuing the USGS for a few days I managed to get the actual percentile RGB tables and improve the density calculation accuracy. This average density percentile is then used in TR-55 as An*CNn with the Canopy CN value of 70.

The process of extracting density from pixels looks like this:

  HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(new Uri(getlayerurl));
  using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
  {
    if (response.StatusDescription.Equals("OK"))
    {
      using (Stream stream = response.GetResponseStream())
      {
        byte[] data = ReadFully(stream, response.ContentLength);
        Bitmap bmp = (Bitmap)Bitmap.FromStream(new MemoryStream(data), false);
        stream.Close();

        UnsafeBitmap fast_bitmap = new UnsafeBitmap(bmp);
        fast_bitmap.LockBitmap();
        PixelData pixel;
        string key = "";
        double value = 0;
        for (int x = 0; x < bmp.Width; x++)
        {
          for (int y = 0; y < bmp.Height; y++)
          {
            pixel = fast_bitmap.GetPixel(x, y);
            key = pixel.red + " " + pixel.green + " " + pixel.blue;
            if (imperviousRGB.Contains(key))
            {
              value += Array.IndexOf(imperviousSurfaceRGB, key) * 0.01;
            }
          }

        }
        fast_bitmap.UnlockBitmap();
        double total = (bmp.Height * bmp.Width);
        double ratio = value / total;
        return ratio.ToString();
                   .
                   .
                   .

C#, unlike Java, allows pointer arithmetic in compilation marked unsafe. The advantage of using this approach here is a tremendous speed increase. The array of imperviousRGB strings to percentiles was supplied by the USGS. This process is applied in a WCF service to both the Canopy and the Impervious Surface layers and the result passed back to the TR-55 calculations.

Possible Extensions:

There are several extensions beyond the scope of this project that could prove interesting.

  1. First the NLCD uses a color classifications scheme. A similar color processing algorithm could be used to provide rough percentages of each of these classificcations for a viewport area. These could be helpful for various research and reporting requirements.
  2. However, beyond simple rectangular viewports, a nice extension would be the ability to supply arbitrary polygonal area of interests. This is fairly easy to do in Silverlight. The draw command is just a series of point clicks that are added to a Path geometry as line segments. The resulting polygon is then used as a clip mask when passing through the GetMap image. Probably a very simple point in polygon check either coded manually or using one of the C# ports of JTS would provide reasonable performance.
MRLC NLCD 2001 Colour Classification
Fig 3 - MRLC NLCD 2001 Colour Classification

What about resolution?

It is tempting to think a little bit about resolution. Looking at the MRLC image results, especially over a map base, it is obvious that at 100 ft resolution even the best of calculations are far from the fine grained detail necessary for accurate neighborhood calculations.

It is also obvious that Impervious Surface can be enhanced directly by applying some additional lookup from a road database. Using pavement estimates from a road network could improve resolution quality quite a bit in urbanized areas. But even that can be improved if there is some way to collect other common urban impervious surfaces such as rooftops, walkways, driveways, and parking areas.

NAIP 1m GSD 4 band imagery has fewer bands but higher resolution. NAIP is a resource that has been applied to unsupervised impervious surface extraction. However, the 4 band aquisition is still not consistently available for the entire US.

Now that more LiDAR data is coming on line at higher resolutions, why not use LiDAR classifications to enhance impervious surface?

LidarClassification1
Lidar All Elevation
ILidarClassification2
LidarAll Classification
LidarClassification3
Lidar All Intensity
ILidarClassification4
Lidar All Return

Just looking at the different style choices on the LidarServer WMS for instance, it appears that there are ways to get roof top and canopy out of LiDAR data. LiDAR at 1m resolution for metro areas could increase resolution for Canopy as well as rooftop contribution to Impervious Surface estimates.

In fact the folks at QCoherent have developed tools for classification and extraction of features like roof polygons. This extraction tool appled over a metro area could result in a useful rooftop polygon set. Once available in a spatial database these polygons can be furnished as an additional color filled tile pyramid layer. Availability of this layer would also let the Runoff calculation apply rooftop area estimates to roof drain disconnect factors.

Additionally improved accuracy of impervious surface calculations can be achieved by using a merging version of the simple color scan process. In a merging process the scan loop over the MRLC image does a lookup in the corresponding rooftop image. Any pixel positive for rooftop is promoted to the highest impervious surface classification. This estimate only applies so long as roof top green gardens remain insignificant.

Ultimately the MRLC will be looking at 1m GSD collection for NLCD with some combination of imagery like NAIP and DEM from LiDAR. However, it could be a few years before these high resolution resources are available consistently across the US.

Summary

The utility of WMS resources continues to grow as services become better known and tools for web applications improve. Other OWS services like WFS and WCS are following along behind, but show significant promise as well. The exposure of public data resource in some kind of OGC service should be mandatory at all government levels. The cost is not that significant compared to the cost effectiveness of cross agency, even cross domain, access to valuable data resources.

By using appropriate WMS tools like Geoserver and Geowebcache, vastly more efficient tile pyramids can become a part of any published WMS service layer. It takes a lot more storage so the improved performance may not be feasible for larger national and worldwide extents. However, in this Runoff Mitigation project, where view performance is less important, the OGC standard WMS GetMap requests proved to be quite useful for the TR-55 calculations and performance adequate.