Mar 12 2010

Sharing Terrain With the World – Google Earth style

It’s not fully 3D immersive, but hey, 2-1/2D ain’t half bad. The “dsm40cm” model of Marin County has been published as the county’s default terrain on Google Earth. It’s a great pleasure to work with folks who are not troubled by a county representing its surface on a 40cm single-precision float grid that weighs in at 77 GB. In terms of data bulk, that is about the same as the entire 30-meter version of the US National Elevation Dataset.

What one gets when piling that much detail into a single county of around 520 square miles of land area is every building pad, driveway, and crown of road paving that were resolved. The dsm40cm model was derived from an ESRI Terrain Dataset that incorporates our best available topographic contours (1:4800 scale 10-foot; 1:2400 scale 2-foot,) photogrammetric break and water lines, FEMA LiDAR and NCALM (GeoEarthScope) LiDAR data sets. The Terrain Dataset currently comprises 40 GB of vector GIS data.

When the finely detailed surface grids were first developed, we broke the county up into 20 work areas to maintain ArcGIS 9.3.1 in a stable and productive state, and 30cm posting interval grids were generated that covered the entire county–at least during development. When necessary, these grid tiles were mosaicked with ERDAS Imagine into a single seamless grid. The 40cm version was produced directly as a single seamless grid using ArcGIS 9.4 beta 1, on a workstation imaged with Windows Server 2003. The WGS84 UTM, NAVD88-Geoid 2003 result was provided to the Google Earth team earlier this year.

As with all GIS data sets, it seems, the more detailed it is, the more rapidly it may need updating. In the works for the next year or so are several improvements to the dsm40cm model. First: the photogrammetric break lines will be segregated into steeper sets that tend to run along ridges, and shallower slopes that tend to delineate road cuts and building pads. The ridge set will be used as soft constraints to resolve some artifacts where they rise above some contours.
Second: incorporate new LiDAR data as it becomes available. Some data has already been provided for the lowest part of Lagunitas creek, and it appears that Prof. Ellen Hines of San Francisco State University’s Department of Geography and Human Environmental Studies has been funded by USGS to gather LiDAR county-wide this year.

So there will be revisions, but an exciting aspect is to see data flows being brought into existence that support different levels of mirror world development.
Publishing the dsm40cm model in Google Earth is an important (and beautiful) threshold to cross. Making use of the dsm40cm model in county operations such as creek and watershed delineation will be the practical benefit that drives the work in the first place. And before too many more weeks, there may be entirely new approaches to publishing the data in an immersive environment (neither Second Life nor Opensim) to share.

Building pad in Kent Woodlands shows driveway-level detail

Kent Woodlands building pad and driveway, in the shadow of Mt. Tam

No responses yet

Jul 06 2009

OpenSim Terrain notes, and Darb has Process Credit history!

I’d read about this, but never before experienced the agony first-hand.  Extracting funds from SL, the wait for funds to arrive at PayPal was a bit slow.  In fact, in the time it took funds to go from Linden to PayPal, a bamboo shoot in my back yard could have grown taller than me (that’s my RL not SL height!), and would have been over 2 meters tall.  Anyway, Process Credits are quite lacking in symmetry with how quickly credit charges can flow into the Linden realm.

During this week of waiting my random prims have been cleared out from Amida and nary a trace of Berkurodam BART Station remains besides a video in Gualala.  The video screen was actually entombed by a neighbor, who may not like it but did not send any message.

Anyway–for me this week is all about generating maps and graphics while keeping up with work.  I’ve generated a 50cm terrain grid for parts of my county where perhaps 150,000 people live.  With computational process improvements I should be able to make production stable enough to generate a 25cm grid.  The point is to model terrain slope and aspect within urban parcels.  OpenSim can pack 64 terrain megaprim sculpties over each region to refine terrain more than the built-in 1-meter postings, and display 10cm orthoimagery at full resolution.

Last year, I used first-return LiDAR data of the UC Berkeley campus to generate a 25cm grid for 10cm imagery.  Now, I’m working with bare-earth LiDAR data from FEMA, topographic contours (densified to 1.5m vertex spacing), and most importantly, photogrammetric terrain and water break lines.

Throwing all those data into the mix, the data are built into an ESRI Terrain Dataset, from which I generate TIN and GRID models at various reolution and extent.  The ESRI ArcGIS 3D Analyst Terrain-to-TIN generator breaks down after about 10 mega-faces (so would I…)  And the ArcGIS Terrain-to-GRID generator seems to drift into Windows-unconsciousness after about 1.0 giga-cells.  So for the grid, I break it down and do the pieces, then merge the tiles using ERDAS Imagine, because the ESRI ArcGIS raster mosaic function does not produce output grids much over 10 GB.  As annoying as learning these ArcGIS limits can be, it is very satisfying (and instructive) to see huge swaths of seamless terrain with great detail once it all comes together.  Thanks to the break lines, many driveways and most home building site cuts and fills are resolved.  And it will be a lot of terrain by OpenSim standards–enough to calibrate terrain for over 20,000 contiguous regions–not that I ever expect to build it all at 1:1 scale!

No responses yet

Jul 31 2008

OpenSim LiDAR sculpties – too much of a good thing?

Wow it has been a lot of effort to get the 40 regions built out with their LiDAR surface sculpties. Someday when the process gets automated I’ll look back and likely feel like a fool for not loading the BLOBs in MySQL directly, and coding the XML to load these directly. But I built them by hand, using 64-prim linked sets that covered each region. Region-wide linking works fine when the Admin settings are used to “go to God”. Also there’s a new SL client that works OK and helped a wee bit with the build.

My priority is to create good graphics for large-format display, but I’ve posted a rush of the first end-to end plod by the Ruth named “UC08 Visitor2”. At the moment, I’m still a bit shaky about just how solid the sim will be for demonstration purposes. None of he LiDAR surface sculpties are physical; I’ve been able to turn on ODE. I can’t bear the time to let the sculpties get meshed so there’s none of the cool walking on them, but I have tried boosting the detail settings to 64, in hopes that perhaps the sim will put out 4096 points per sculptie [my bump-maps to define each one are 16K so there’s plenty of info behind the detail boost.

As usual, if YouTube is blocked or you don’t have flash in the browser that you read this with, the video is here:

2 responses so far

Jul 22 2008

OpenSim – First LiDAR reflective DEM sculpties placed

OK, not everyone in the immersive modeling universe has been holding their breath on this one, but hey, I’m happy to say that the production line is fired up and creating 2560 sculptie bumpmaps to inflate the Open Berkurodam sim using LiDAR reflective digital elevation model (DEM). The registration with orthoimagery is not perfect, and small offsets are very distracting, but the first two have been placed, and should illustrate the concept. (Two rezzed, 2558 to go…)

Sather Tower as two reflective digital elevation model sculpties

The reflective DEM sculpties have 16 times greater resolution (that’s resolution as NURB point density) than the underlying terrain megaprims. This means that for the 40-region Open Berkurodam sim, there are 160 terrain megaprims, and 160 tiles of 10-cm orthoimagery. The reflective DEM sculpties number 2560 and will be textured using the same orthoimagery.

Reflective DEM surfaces ride over the tops of trees, rooftops, or any structure. They are defined by the first return of the LiDAR reflected signal. By contrast, the terrain megaprims are based on a model of the last return (in these data up to the seventh return signal) that represents the ground under and around all structures and trees.

No responses yet

Jun 17 2008

Reflective DEM has been gridded

Published by under OpenSim

After mucking around a bit, I was able to use free tools to browse the contents of the classified LiDAR, then used ArcGIS 9.2 tools from 3D Analyst and ERDAS Imagine to get where I wanted to go with this surface. First, I needed to know how many returns there were, and what each of the classes meant. The LAStools info function helped there. Then I used ArcGIS 9.2 3D Analyst “LAS to multipoint” conversion tool, but selected only the first return. Multipoint was an annoying format because it did not seem to fit anywhere in the cool new ESRI “terrain” feature data type. In the end, I gave up on ESRI terrain and went straight to the classic TIN. For maximum overlap, I did not filter out any specific angle from nadir, taking whatever was sent along from the contractor to Alameda County.

Of course, I had to negotiate the treacherous 3D Analyst menu items that were necessary. Getting multipoint into a TIN required creation of a TIN (obvious, but with blank result) and then the non-obvious choice of “Edit TIN” which effectively accepted the multipoint data that were imported from LAS and allowed me to specify the delunay method of choice. Once canned as a TIN, it was a familiar step to specify a raster gridding. I haven’t found a way to reproject the TIN, so I was still in NAD83 California coordinate US Survey feet, and an assumed NAVD88-Geoid 2003 CONTUS-feet vertical while I tried several grid resolutions. In the end, I was happy with 1 foot gridding.

Then, raster on disk, I was able to reproject to WGS84 UTM zone 10 north meters, and chose bilinear resampling on a 25 cm grid posting interval. Once in my favored projection, I rescaled the Z values to NAVD88-Geoid 2003 CONTUS-meters, and began to examine the need for a bit of grayscale morphological processing. I’ve been a great fan of mathematical morphology for over 20 years, so it was a pleasure to craft a kernel or 3 to compensate for some artifacts. Because the TIN-to-grid was so highly oversampled, I was able to use a combination of a tall, narrow 7×3 kernel for morphological CLOSE, followed by a 3×3 DILATE, and a diamond-shaped 5×5 ERODE to finish off the task. In case this morphological stuff sounds like odd stuff to do, these operators are variations on focal max and focal min convolutions. The results are rather important for my application, as shown in the following images.

First is the reflective DEM surface, and the same with the Open Berkurodam 40-region overlay.

This is the gridded reflective DEM here it is with the OB40 image overlay

Next are more detailed images, near the Greek Theater, showing why I ran the morphological filtering and also how I was able to mostly conserve building footprint areas while inflating trees. The main artifact attenuated was interlace-type effects at the end of overlapping LiDAR scans. The long axis of the morphologcial CLOSE kernel was perpendicular to these artifacts.

Here is the reprojected and rescaled reflective DEM, unfiltered Same area, but with morphological filtering as sketched above

Here is the morpho-filtered reflective DEM, with the 10 cm natural color imagery overlaid.

overlay of natural color image on morpho-filtered reflective DEM

Next up, I’ll need to figure out how to best use this 25 cm surface. It really seems a shame to use it in the way that I have thus far with terrain megaprims–where using four megaprims per region I have effectively downsampled the terrain to 4.26-meter grid postings. That wasn’t so bad for the bare earth model. Here I’ve got something over 290 times denser with 0.25-meter grid surface samples.

But to use many more than 160 megaprims for the entire 40-region model, I really must automate the placement of the (auto-generated) sculpties. For that, I’ll need to ask around the OpenSim community for advice!

No responses yet

Jun 13 2008

Classified LiDAR data have been viewed

Published by under SL In General

The classified LiDAR data that I hope will provide some inflated structure and tree surfaces for draping the orthophoto have been reviewed. I find the data beautifully detailed, and fascinating to see with GeoCUE Point View LE. I’m working with the UC Berkeley Geospatial Imaging and Informatics Facility UCB GIIF, also known as the Maggi “Kelly Lab” when proximal to Mulford Hall.

Right now my goal is to interpolate the first return surface in a way that I can grid and filter most appropriately to inflate buildings and trees. In principle, I should be able to use the first return LiDAR point cloud to create a NURB surface that would be expressible as an OpenSim/SL sculptie. But I’m going to take a more cautious approach and try to get the whole thing gridded in a consistent way so that I can reasonably expect to cover the entire 40-region sim with good inflated surfaces rather than the bare earth that has been a fine demonstration, but a bit flat for draping the orthophoto.

I’m going to throw out a lot of images and let them speak somewhat for how the classified (into ground, structure, low veg, med. veg, tall veg) LiDAR point clouds look.

Here’s the plain elevation image and the classified view of same

elevation view of classified LiDAR Classified LiDAR of UC Berkelye vicinity
this is how the classified image looks with intensity shading. It gives a first impression like a photo
classified LiDAR near UC Berkeley more detailed view of UC Berkeley area

Some perspective views also help to show what information will be available for gridding. For these I’ve displayed with vertical exaggeration of 1.5X
Northeasterly perspective view of LiDAR Easterly view of UCB campus in classified LiDAR

No responses yet