This is the first of what should be a set of posts that detail a server build process for the San Francisco Enterprise Geographic Information Systems Program (SFGIS) Standard Geospatial Server (SGeoS). In fact, the build work has been ongoing for several weeks and is concluding here, with OpenSim.
The motivation for including OpenSim in the platform was a desire to provide support for legacy .NET applications that may exist in various departments. In the interest of creating a Microsoft-neutral build that is framed with Open Source components, it was natural to bundle the Mono framework into the SGeoS design. And while individual department applications are their own business and not part of the standard build, OpenSim serves as an excellent demonstration of the utility of the Mono framework as included on the server. That , together with my perspective that immersive 3D clearly should be associated with geospatial servers, is why OpenSim is included in the Standard Geospatial Server.
OpenSim is not trivial by any means, and yet it is not such a resource hog that it would be infeasible to bundle it. What’s more, it is an opportunity to distribute immersive 3D technology packaged with other geospatial capabilities.
Since the build descriptions are being transcribed from a build document that is approaching 80 pages on Google Docs, it seems prudent to break it up into individual modules. And since WordPress here is configured to show older posts below newer ones—I’ll start down at the end modules and post new build descriptions for earlier modules in later days.
The original notion for SGeoS was to have modular build chapters that could provide a unit of capability. That way, only selected modules need be configured. After discussions with VMware engineers, I became intrigued by the notion of making a single server image that could run everything, all at once, and then disable unneeded featured in an actual deployment. So the build document was initially structured with module-like chapters, but in fact the server builds them all—so it’s worth viewing the build document in sequence.
The modules will probably end up numbering about 10, including packaging for production and possibly default-disabling of most items. If one watches too closely, it might seem like I’m making a countdown to completion. But this will end with a stub for deployment packaging, work back through an OpenSim build, and end up with imaging an install of CentOS 6.5 onto a new VM guest system.
I don’t have much to say about these regions that hasn’t been written already, and my views have been less aesthetic than Shenlei’s.
But in the interest of boosting the bandwidth by which I can share OpenSim, I’ve invested in a much newer Adobe Premiere Elements than I’d been using for the past five or so years. It’s a gas to have it multi-thread while rendering, and I have direct-to-FLV write. Trying to share as much of the motion and fidelity via YouTube as I possibly can, I’ve crafted a video resolution that is a multiple of my Hippo / SL viewer screen. The FRAPS video direct to AVI (sorry, it’s Win XP) is 1600 x 1140 @ 10 fps. Yup, those are video frames. In the interest of surviving an upload, I’ve rendered them highly time-compressed, with output at 1515 x 1080 @ 15 fps. As of tonight there’s no sound, no intertitles, just the rushes. oops, if I read the YouTube Instructions for best formats, I should have trimmed the width to 1440, which is a multiple of 16.
Also, I have more direct upload options now with Premiere 8 than I had with my (recently demised) copy Premiere 1.0. Go Figure ;^)
While the Windows box grinds out the video print, I’m over here on Ubuntu blogging in a tab of 64-bit Chrome 188.8.131.52 and it is fine & fast.
For these videos, I visited ScienceSim Geography22_44 region and set the view to wide angle, then sat up at about 500 meters and watched the regions rez their terrain. For some folks, it will rank right up there with watching lead-based paint dry. For geography folks I’m hoping that these few minutes of sped-up video will convey, by dogged repetition, the primacy of regions in the provision of virtual environment simulators.
By the way, I’ve got a task: I need to find a better buzz word for the GIS community. I’ve been advised by some serious and well-intentioned (not to mention well-informed) folk that terms like “virtual” and “immersive” are actually boring to GIS’ers. So I’ll need to think about how to convey the concepts of “Mirror World”, “Multiuser Virtual Environment”, “Immersive Connected Experience”, “Third-Person Virtual World”, and related concepts into a catchy moniker. Hopefully, one that is not presently trademarked, either!
I’m trying to remain serious about this, but some of the options are treacherous. Geography in Social Media has a possibly awkward acronym; maybe it can be saved in recognizable form as “GIS for Social Media” or “Geography for Social Media”: GFSM
The term “3D Map with Me” is terse, slightly ambiguous
Here is the video chopped as it was when uploaded with 1515 x 1080 resolution. Problem with that is that by not preserving dimensions at a multiple of 16, and saving my viewer’s aspect ratio rather than the (standard since 16mm film) 4:3 aspect, my upload is clobbered into something perhaps suitable for a smartphone. So please consider this the Smartphone Version of last night’s rushes:
Then, once again with feeling, or at least with a little more rest, there is what I hope to be an HD-friendly moving vision of OpenSim, as it appears on the ScienceSim Geography regions. Yes! After it ripened on the YouTube servers for a few hours, I now see all the higher-res versions available. At 1440 x 1080, this is pretty close to what I see on my screen with a live Hippo viewer.
And after a day’s cogitation: anyone care to comment on the term: “Social Immersive Media GIS” as a moniker? Oops — I used “immersive” 8^(
There’s some progress on a couple of project fronts. I’ve started assembling some USGS terrain for a 1:10 scale Level 1 build that could involve more OpenSim regions than I’ve ever stood up on one machine before. Snapshot of progress is here, with a goal of 304 OpenSim regions for the model. I expect that the OpenSim server will be re-imaged and a new build attempted in the next couple of weeks.
Vicinity of Colorado Springs, CO - with a full mile of terrain elevation subtracted
The site design for the Marin Civic Center build in Second Life Stanford region is also moving along with its target 1/8 region (two SL acres) based on RL terrain and building at 1:1 scale Level 3 build. Progress sketch below:
Context model data of terrain for Civic Center Administration Building
A bit more can be reviewed by looking at the PDF of the same map here.cc_topo_20090211
Finally, a sky tag has been added to the space above the build. It is visible as a streak to anyone who visits SLURL.com by a mouse roll. The Stanford region now has a large “MARIN” visible in its upper reach, squarely in the middle of the ancient Second Life Outlands.
Down in the “basement” a server awaits a bit of configuration to become an OpenSim lab to support the Marin Civic Center development. How nice it would be to load terrain into a portion of a Second Life region, but alas–it seems unavailable to those of us without Estate controls. The 1:1.00 scale terrain will need to be calculated to back the base ortho-image that has already been tiled over the site as shown in the last blog posting. I have verified that one of the 40-meter sphere oversize prims in my inventory can be rebuilt into a massive terrain sculptie. Placing a 32×32 sampling (30×30 at most that are usable for terrain) of 1-meter terrain samples will give me a guide, and then I’ll just need to diddle with the SL client bulldozer tool to approximate the terrain prim, before disposing of it.
Thanks to the frequent updates of the region map at SLURL.com, I can already see some of the build taking shape. On the second image below I sketched my virtual moving van’s path from Gualala to Stanford. To aid overview map navigation and VFR-flying avatars, I have constructed a large readable sky label in crude imitation of the long-standing skywriting by SL resident Web Page in region Da_Boom.
Da Boom — probably named after De Boom
From that origin, the old Gualala locale was at region grid (1008,998), and the new Stanford site is at region grid (1006,1000).
The heart of the old SL Mainland
Oddly enough, Stanford appears to be the fourth-oldest region, according to this relatively ancient map.
Second Life regions 2002 11 21
On the ground, I’m still in process on some lot line adjustment I’d like to make before breaking ground. The terrain sculptie method has been proven, although I have yet to grid actual terrain values for the project site. Also, I’m trying to minimize RL dimension measurements if possible, by using best available historical information on the RL site.
Some things have changed in SL viewers in the last few months. About a year ago, it was possible to take an oversize prim and modify a single dimension, having that snap to 10 meters. Currently, any change in a dimension of an oversize prim results in all three dimensions snapping to values less than or equal to 10 meters. It’s a new challenge, but can be managed. Still, once a builder has tasted the freedom of OpenSim, it is awfully hard not to chafe at those sorts of restrictions.
I’m giving thought to a copy of Second Inventory to facilitate the use of OpenSim for dev and SL for production, but the issue of prim size shrinking will be a big issue for me.
Curiously, I have found that physical prims to not drop to the ground in Stanford. This has never been the case in Gualala, so I’m intrigued and opened a ticket with Linden Lab. I’ll see what they say. Meanwhile, I’ll keep my eye on the sky for the project’s mark (in the old ceneter of OUTLANDS)
Only the four-story Administration building (wing), not the two-story Hall of Justice. I’m tired so I’ll let the shot speak for me.
photo from 2009 01 30
To me, it’s mildly amazing to realize that F.Ll.Wright’s design fits so snugly in 1/8 of a Second Life region at 1:1.00 scale. The Civic Center Administration building is a Real-Life building that can be visited, providing an easy way to get a true RL immersive sense of its scale. Building at 1:1 scale in Second Life for the first time, this has been my first experience of transferring that awareness into the multi-region contigous space of the very beautiful Second Life. Sure, I’ve built large areas at 1:1 using draped LiDAR data, but to have a rather large single building (or at least its footprint for now) in context with existing builds that I’ve seen for months, well, at the moment SL seems larger than I’d thought. That shift in my perception of SL scale may be the contrast between flying (quite fast as it turns out) around 40 to 100 OpenSim regions versus walking around the site and knowing how long it takes to traverse the RL building.
Anyway, check out the build’s progress at secondlife://Stanford/100/235/30
Today the (San Francisco) Bay Area Automated Mapping Association hosted a wonderful URISA Certified Workshop given by Tim Case, describing Best Practices and Project Implementation Methods for 3D Geospatial work. The all-day event provided a very broad and even-handed overview of many 3D technologies that hold promise for the near future.
With this presentation as an extra boost for my focus on a new build, I’m gearing up with even more enthusiasm for a new build in the Agni grid Mainland. I’ve also tuned the Berkeley parcel for sale. Its price amounts to about US$382.00, and that price is set to help cover purchase costs for the next build’s likely parcel. The tuning involved reducing the parcel size by 64 square meters, so that the three Gualala parcels total 4608 square meters, or exactly the maximum amount allowed for Linden Lab’s US$25/month tier rate. With that size, it would be possible for an interested party to purcahse the Berkeley BART station and maintain it for $300/year in tier (the Linden land property tax).
Also, based on today’s Geospatial tag, I’ve noted just this morning two mentions of the Berkeley BART build. The New World Notes item by Wagner James Au 2009 01 19 was wonderful to find after our in-world messages last month. For clarification, while true at the time of that conversation, no longer do I work for City of Berkeley. The TidalBlog item by Peter Miller mentions interesting developments in the overlap between simulators and geospatial models, as well as some shots from his visit to the Berkeley BART model. Thanks to both authors for their posts!
I’m finding little time for keeping the OpenSim instance current. For me, there’s been a lot of problems with the more recent (last six weeks) versions simply working on first try. Also, since I have so many hours invested in the content that was created at rev 5411 that I’m a bit skittish with the bleeding edge updates.
Most recently, I’ve had the experience at osim.bargc.org of having only a single region be accessible at a time with the latest 1.21.6 SL viewer. So I vacillate between thinking “how convenient and attractive to use hosted Second Life Grid servers” and the hot-rodder thoughts “My 40-region sim is worth $7525 up front and $1610/month in tier for a nonprofit, so I can be tough.” I do tire of keeping the OpenSim server up and running with its load of content, yet with this real-world economy even avatars need to be frugal, no?
Thanks to Gualala neighbor Misty Rhodes for background
From here on the west coast of the USA, the world has seemed a bit tumultuous in the past four weeks. In a more compact and local way, this has been a time of review and reflection for me. This week I drafted an article for a local GIS Journal to review some of the explorations I’ve made since 2006 of how to create a civic-scale Mirror World. The deadline for the article has motivated me toward a bit more recapitulation on this topic than I’d expected.
My interest in the topic was first kicked off by media attention given to Linden Lab around the registration of the one-millionth resident for Second Life. I checked it out, made a rapid getaway from Orientation Island in around 30 minutes, and within a week or so the broad outlines of some intersections between GIS data and Multi-User Virtual Environments (MUVE) were taking shape in my thoughts. By Summer 2007, I’d made a fairly complex build on the mainland of the Berkeley BART station, and realized how hard it would be to justify the tier for 500 regions to host, much less build out, the entire city. But by October, I’d learned how OpenSim would solve the issue of tier, if only it were possible to make a build efficiently.
Then meshes arrived in the form of sculptie prims, and when an opportunity arose to collaborate with IBM on a multi-region terrain, I devised a way to drape orthoimagery over the region terrain. This Summer 2008 I was able to do that with LiDAR data that draped orthoimagery over terrain, buildings, and trees. The past year has been very much focused on OpenSim for me with this activity.
But behind the week-to-week excitement of OpenSim growth, and even before that, there has been a steady stream of good new stuff from Linden Lab–a stream that I haven’t reflected on so much. First off, the confluence of LibSL’s stabilization by the end of 2006, the open sourcing of the Second Life client in early 2007, and the initiation of OpenSim shortly afterward, together made possible the environment that I’m, if not taking for granted, really expecting to be there for awhile. Meanwhile, back in Second Life, there’s been integration of VoIP, new HAVOK physics, way cool Windlight enhancements to the SL client, a growth in land area that just keeps on going, plus new Openspace regions.
For reasons unrelated to my journal-article recapitulation, today I enjoyed a pleasant visit with a Linden person. It was more time to chat about civic Mirror World applications with a Linden than I’d ever had before, either in-world or real-world. In the course of our conversation, seeing the eyes of someone who is among those directly and personally impacted by OpenSim in the sense of unrealized revenue growth for Linden Lab, I gained an awareness of what is perhaps the largest contribution of Linden Lab to the OpenSim community. That would be Linden forbearance.
It’s growing late this evening for me to write much more on this right now. And as I noted, this feels like a tumultuous time for the world as viewed from west-coast USA, so I need my rest for what might be a tough week ahead. But I’ve felt a shift in perspective today, and wonder anew what the future of an operational civic Mirror World will really look like.
ABSTRACT: When creating sims based on real-world terrain, there’s usually a lot more extent and detail available than is typical when one must hand craft every region landscape. When compared with commercial multi-user virtual environment (MUVE) hosting, OpenSim has extremely low marginal cost for adding more simulator regions. Whenever a real-world region of interest has free 1/3 arc-second DEM data available for download from the USGS seamless server, then with a bit of GIS help from someone at a local university or interested government office, the free DEM data can be converted to a fine 1/10 (1:10) scale sim of terrain in OpenSim. This case study takes an idea posted to a blog, and turns it into a sim of 54-regions in OpenSim based on real-world terrain available at a 10-meter posting interval. To run all the regions, one first obtains OpenSim, then downloads the 54 terrain tiles, the 54 region definition XML files, and a loader script available from this post. The region definition files will likely need to be regenerated with the supplied PHP script to point to your machine’s IP address.
The Case Study tour video is nearly 10 minutes, and if it doesn’t embed in your browser is here at http://www.youtube.com/v/EfBvEtTvLYU
Two days ago I was intrigued by Brian White’s posting of a Tutorial for real-world terrain in OpenSim and I followed several of his well-documented steps. Then I read all the way to the end of his posting and realized that he wasn’t fully satisfied with the results. I’ve helped a couple of folks with generating specific terrains before, and Orcas looked like an intriguing bit of topography–sort of a miniature version of how I imagine Greenland might look in, say, a couple decades…
So without an intent to cross-talk Brian’s posting, I’ll offer my own parallel perspective on how I’d approach making a recognizable, maybe even fun-to-have-around model of the Orcas Island in OpenSim. Starting from where he did, I’d like to emphasize the profound opportunities for following this approach, as it would work virtually anywhere in the continental USA (CONUS) where the 1/3 arc second seamless DEM is pretty much complete. In other words, what I’ll show here could be applied in an hour or so’s effort to produce an OpenSim version of Yosemite Valley, for example.
First, choose thy locale. The USGS seamless server is a worldwide resource, but the greatest detail available seems to be in the “lower 48” states. A digital elevation model (DEM) with postings on 1/3 arc second interval has samples about every 10 meters in Y (north-south) and more closely spaced than that in X (east-west) everywhere off the equator. The 1/9 arc second data that Brian mentioned was actually not available for Orcas, although some urban areas may have it at this time. In this case, Orcas Island offers a nominal 10-meter DEM downloadable from seamless.usgs.gov server.
Second, define thy scale for the sim. If one will be hosting on a commercial sim, even like Linden’s Open Space sims in Second Life’s Agni grid, then there are very strong reasons to scale things down and perhaps pack everything into a single region or four. With OpenSim, unless you’re going to have tons of prims and game-grade dynamics running, it may be possible to balance the number of regions you set up with the terrain quality that you desire. For the Orcas, I downloaded the islands (using the default format which seemed to be an ESRI GRID) and explored them in ESRI ArcGIS. With that I had the sense that I wanted more or less 24 km in X, and 16 km in Y, to cover the Orcas. Since OpenSim allows me to load floating-point terrain samples on a 1 meter posting interval, I knew that my available (1/3 arc second) DEM would support a 1/10-scale sim, analogous to a 1:10 scale map. That way, I would not discard too much of the terrain information on its way into OpenSim.
Third, do the math to tile the regions. It’s not too hard, but it’s fairly unforgiving. After a bit more ArcGIS exploration, I found a way to crop the DEM to exactly 23.04 km in X, and 15.36 km in Y. Sounds wierd, or compulsively precise? Not really. Since each of my terrain samples was planned to be on a 10-meter grid interval in real-world, and my sim scale is 1/10, that means that every region would cover a 2560-meter square, or 2.56 km on a side. The dimensions that I chose were simply six regions north-south and nine regions east-west, for a total of 54 regions of 1:10 scale sim.
Fourth, get your DEM gridded. Sure, it comes from USGS that way, but the seamless server works world-wide, and that requires a geographic (Longitude / Latitude) way of defining its coordinates. To do this rigorously, I used Geographic Information Systems (GIS) software to re-project the DEM from geographic, the way it downloaded, into a projected grid [World Geodetic System 1984 (WGS84) datum, Universal Transverse Mercator (UTM) zone 10 north projection, with distances measured in meters]. The UTM grid system is defined world-wide, with the proviso that at any given spot, only one of the sixty (60) grids in the system would be the best choice. For the Orcas, it’s zone 10. For the reprojection process, I used a remote sensing package, Leica ERDAS Imagine that provides precise control over both the reprojection and the resulting data’s sampling interval. I used ERDAS Imagine to grid the UTM-projected DEM into a Cartesian grid of 10-meter interval in both X and Y, and cropped the projected data to 2304 by 1536 samples.
Fifth, dice the DEM. This is necessary for multi-region sims where your grid has more than (and exact multiples of) 256 samples in X or Y. ERDAS Imagine dices, so i used it. If you’re trying this in Photoshop, be sure to keep the pixels in 32-bit (single precision) floating point values, and don’t go to integer grayscale or the flatter terrain will get stair-step artifacts. The naming convention used in ERDAS Imagine starts at the top row and increases downward for Y, and also puts Y first then X in the tile name so that the south-westernmost of the 54 regions is called “orca_6_1”
Sixth, reflect each diced DEM tile, this reverses the direction of samples in Y dimension. What starts out as the top, northernmost row is moved into the bottom-most position, and vice-versa. This does not change the size of the tile, nor any of the gridded elevation values themselves. Another way to describe this is to flip the image around the X axis. ERDAS Imagine has a geometric correction function that allows one to specify this as an affine mapping, which I did. I’d expect that most of the world uses GIMP or Photoshop to do this, too. Although it would appear that this will scramble the terrain, it is necessary to do this to adapt the data to the sequence in which OpenSim reads the terrain file when loading it into the region.