Jul 06 2009

OpenSim Terrain notes, and Darb has Process Credit history!

I’d read about this, but never before experienced the agony first-hand.  Extracting funds from SL, the wait for funds to arrive at PayPal was a bit slow.  In fact, in the time it took funds to go from Linden to PayPal, a bamboo shoot in my back yard could have grown taller than me (that’s my RL not SL height!), and would have been over 2 meters tall.  Anyway, Process Credits are quite lacking in symmetry with how quickly credit charges can flow into the Linden realm.

During this week of waiting my random prims have been cleared out from Amida and nary a trace of Berkurodam BART Station remains besides a video in Gualala.  The video screen was actually entombed by a neighbor, who may not like it but did not send any message.

Anyway–for me this week is all about generating maps and graphics while keeping up with work.  I’ve generated a 50cm terrain grid for parts of my county where perhaps 150,000 people live.  With computational process improvements I should be able to make production stable enough to generate a 25cm grid.  The point is to model terrain slope and aspect within urban parcels.  OpenSim can pack 64 terrain megaprim sculpties over each region to refine terrain more than the built-in 1-meter postings, and display 10cm orthoimagery at full resolution.

Last year, I used first-return LiDAR data of the UC Berkeley campus to generate a 25cm grid for 10cm imagery.  Now, I’m working with bare-earth LiDAR data from FEMA, topographic contours (densified to 1.5m vertex spacing), and most importantly, photogrammetric terrain and water break lines.

Throwing all those data into the mix, the data are built into an ESRI Terrain Dataset, from which I generate TIN and GRID models at various reolution and extent.  The ESRI ArcGIS 3D Analyst Terrain-to-TIN generator breaks down after about 10 mega-faces (so would I…)  And the ArcGIS Terrain-to-GRID generator seems to drift into Windows-unconsciousness after about 1.0 giga-cells.  So for the grid, I break it down and do the pieces, then merge the tiles using ERDAS Imagine, because the ESRI ArcGIS raster mosaic function does not produce output grids much over 10 GB.  As annoying as learning these ArcGIS limits can be, it is very satisfying (and instructive) to see huge swaths of seamless terrain with great detail once it all comes together.  Thanks to the break lines, many driveways and most home building site cuts and fills are resolved.  And it will be a lot of terrain by OpenSim standards–enough to calibrate terrain for over 20,000 contiguous regions–not that I ever expect to build it all at 1:1 scale!

No responses yet

Mar 25 2009

Terrain Tenacity, fresh ortho pixels

Terrain has been in the mix for me quite a bit these past four weeks.  I’ve worked on pushing ESRI ArcGIS 3D Analyst to its limits of masspoint digestibility, trying hard to bring everything into focus at the same time that everything is sinking down to NAVD88 datum.  An abundant set of waterlines and terrain breaklines have helped to make possible some terrain models that appear to be as good as any one is likely to get from photogrammetric data.  As with LiDAR source, I’m working toward a 30cm gridding interval to sample any reasonable-looking TIN models.

One fascinating aspect of the terrain model is where it ends.  There appears to be a new 1:1200 or 1:4800 shoreline that can be sussed out of some combination of 2.5-foot elevation waterlines, 2-foot elevation contours, and related artifacts.  In fact, it’s a great patchwork of artifacts that must be stringed together.  In the tidal flat areas, there is also plenty of need for validation with multiple photos (hopefully shot at times of lower tides).

Adding to the data bulk there’s a new ortho in town, 30cm natural color flown just about two years ago.  There’s hope of extracting it from the grip of California HARN coordinates after it is all mosaicked.

No responses yet

Sep 11 2008

Case Study: USGS terrain in OpenSim, a GIS approach

ABSTRACT: When creating sims based on real-world terrain, there’s usually a lot more extent and detail available than is typical when one must hand craft every region landscape. When compared with commercial multi-user virtual environment (MUVE) hosting, OpenSim has extremely low marginal cost for adding more simulator regions. Whenever a real-world region of interest has free 1/3 arc-second DEM data available for download from the USGS seamless server, then with a bit of GIS help from someone at a local university or interested government office, the free DEM data can be converted to a fine 1/10 (1:10) scale sim of terrain in OpenSim. This case study takes an idea posted to a blog, and turns it into a sim of 54-regions in OpenSim based on real-world terrain available at a 10-meter posting interval. To run all the regions, one first obtains OpenSim, then downloads the 54 terrain tiles, the 54 region definition XML files, and a loader script available from this post. The region definition files will likely need to be regenerated with the supplied PHP script to point to your machine’s IP address.

The Case Study tour video is nearly 10 minutes, and if it doesn’t embed in your browser is here at http://www.youtube.com/v/EfBvEtTvLYU

Two days ago I was intrigued by Brian White’s posting of a Tutorial for real-world terrain in OpenSim and I followed several of his well-documented steps. Then I read all the way to the end of his posting and realized that he wasn’t fully satisfied with the results. I’ve helped a couple of folks with generating specific terrains before, and Orcas looked like an intriguing bit of topography–sort of a miniature version of how I imagine Greenland might look in, say, a couple decades…

So without an intent to cross-talk Brian’s posting, I’ll offer my own parallel perspective on how I’d approach making a recognizable, maybe even fun-to-have-around model of the Orcas Island in OpenSim. Starting from where he did, I’d like to emphasize the profound opportunities for following this approach, as it would work virtually anywhere in the continental USA (CONUS) where the 1/3 arc second seamless DEM is pretty much complete. In other words, what I’ll show here could be applied in an hour or so’s effort to produce an OpenSim version of Yosemite Valley, for example.

First, choose thy locale. The USGS seamless server is a worldwide resource, but the greatest detail available seems to be in the “lower 48” states. A digital elevation model (DEM) with postings on 1/3 arc second interval has samples about every 10 meters in Y (north-south) and more closely spaced than that in X (east-west) everywhere off the equator. The 1/9 arc second data that Brian mentioned was actually not available for Orcas, although some urban areas may have it at this time. In this case, Orcas Island offers a nominal 10-meter DEM downloadable from seamless.usgs.gov server.

Second, define thy scale for the sim. If one will be hosting on a commercial sim, even like Linden’s Open Space sims in Second Life’s Agni grid, then there are very strong reasons to scale things down and perhaps pack everything into a single region or four. With OpenSim, unless you’re going to have tons of prims and game-grade dynamics running, it may be possible to balance the number of regions you set up with the terrain quality that you desire. For the Orcas, I downloaded the islands (using the default format which seemed to be an ESRI GRID) and explored them in ESRI ArcGIS. With that I had the sense that I wanted more or less 24 km in X, and 16 km in Y, to cover the Orcas. Since OpenSim allows me to load floating-point terrain samples on a 1 meter posting interval, I knew that my available (1/3 arc second) DEM would support a 1/10-scale sim, analogous to a 1:10 scale map. That way, I would not discard too much of the terrain information on its way into OpenSim.

Third, do the math to tile the regions. It’s not too hard, but it’s fairly unforgiving. After a bit more ArcGIS exploration, I found a way to crop the DEM to exactly 23.04 km in X, and 15.36 km in Y. Sounds wierd, or compulsively precise? Not really. Since each of my terrain samples was planned to be on a 10-meter grid interval in real-world, and my sim scale is 1/10, that means that every region would cover a 2560-meter square, or 2.56 km on a side. The dimensions that I chose were simply six regions north-south and nine regions east-west, for a total of 54 regions of 1:10 scale sim.

Fourth, get your DEM gridded. Sure, it comes from USGS that way, but the seamless server works world-wide, and that requires a geographic (Longitude / Latitude) way of defining its coordinates. To do this rigorously, I used Geographic Information Systems (GIS) software to re-project the DEM from geographic, the way it downloaded, into a projected grid [World Geodetic System 1984 (WGS84) datum, Universal Transverse Mercator (UTM) zone 10 north projection, with distances measured in meters]. The UTM grid system is defined world-wide, with the proviso that at any given spot, only one of the sixty (60) grids in the system would be the best choice. For the Orcas, it’s zone 10. For the reprojection process, I used a remote sensing package, Leica ERDAS Imagine that provides precise control over both the reprojection and the resulting data’s sampling interval. I used ERDAS Imagine to grid the UTM-projected DEM into a Cartesian grid of 10-meter interval in both X and Y, and cropped the projected data to 2304 by 1536 samples.

Fifth, dice the DEM. This is necessary for multi-region sims where your grid has more than (and exact multiples of) 256 samples in X or Y. ERDAS Imagine dices, so i used it. If you’re trying this in Photoshop, be sure to keep the pixels in 32-bit (single precision) floating point values, and don’t go to integer grayscale or the flatter terrain will get stair-step artifacts. The naming convention used in ERDAS Imagine starts at the top row and increases downward for Y, and also puts Y first then X in the tile name so that the south-westernmost of the 54 regions is called “orca_6_1”

Sixth, reflect each diced DEM tile, this reverses the direction of samples in Y dimension. What starts out as the top, northernmost row is moved into the bottom-most position, and vice-versa. This does not change the size of the tile, nor any of the gridded elevation values themselves. Another way to describe this is to flip the image around the X axis. ERDAS Imagine has a geometric correction function that allows one to specify this as an affine mapping, which I did. I’d expect that most of the world uses GIMP or Photoshop to do this, too. Although it would appear that this will scramble the terrain, it is necessary to do this to adapt the data to the sequence in which OpenSim reads the terrain file when loading it into the region.

Seventh, export the reflected DEM tiles to f32 raw binary. After trial and error, I found that using IEEE 32-bit single precision float with Motorola “swapped” byte order works for OpenSim. This is true even though my machines (both Windows and Ubuntu) are running Intel processors. Go figure.

Eighth, place the f32 terrain tiles in a directory within OpenSim’s ./bin for easiest referencing in the region definitions. For this case, I have prepared 54 tiles for Orcas that can be downloaded, unzipped, and placed in ./bin

Ninth, create the appropriate region definition XML files in OpenSim’s ./bin/Regions directory. Long before one gets up to 54 terrains, it makes sense to have a bit of code to help this process. I started modifying a sample provided with OpenSim a while ago, and include it, along with 54 region files for Orcas that can be downloaded, unzipped and used to replace your Regions directory.

Tenth, create a command script to load the terrain once your 54 regions are running. I include such a script that can be downloaded and invoked from the OpenSim console with “command-script terrain54.txt”.

Et Voila: A 10-minute tour of most of the sim has been recorded and is being uploaded to YouTube

No responses yet