Jun 30 2014

SGeoS Add EAS Enterprise Address System Geoserver components – Module 7 of 9

Published by under SL In General

While this module 7 of 9 may not have the same appeal as other similarly-named characters, assimilate the following steps to enable the Open Source Geospatial Foundation’s GeoServer as used by EAS.

Add EAS Geoserver components

Build steps for configuration Module-stage-7

Just a few more dependencies to deal with here.

1) Start from completed system Module-stage-6

2) Branded Java Install  The OSGeo GeoServer crew is partial to Oracle-branded Java, so download that from someplace near http://www.oracle.com/technetwork/java/javase/downloads/  then click through to Server JRE, and finally Linux x64.  Note that the last link will still require one to click to initiate the download—it doesn’t link directly to the desired file.  So, download to some Windows path since your server system won’t have a desktop.   Consider using the SMB share in /ags1022 a.k.a. O:\ on Windows to get the download easily in reach, and copy it over to /opt/installs along with the other downloads.

cp /ags1022/server-jre-8u5-linux-x64.gz /opt/installs/jdk-server-jre-8u5-linux-x64.gz
tar xvf jdk-server-jre-8u5-linux-x64.gz
cd jdk1.8.0_05
sudo alternatives --install /usr/bin/java java /opt/installs/jdk1.8.0_05/bin/java 3
sudo alternatives --config java

 

3) Microsoft TrueType core fonts install  Some fonts are desired, and for some reason EAS is partial to Microsoft fonts.  So install them already.

sudo yum localinstall msttcorefonts-2.5-1.noarch.rpm

4) Legacy Speed Tweak for Java-based Imaging   Next is Java Advanced Imaging, which helps speed up Geoserver.  This is a pretty old and stale-looking project, with no changes in 8 years, but the Geoserver wiki says they depend on it for better speed.  Its installation instructions may be hard to find.  If you’ve use the alternatives approach to access JDK 1.8.0 above, then the top part of the destination is shown below.  Otherwise, place the resources in the active/chosen JDK-directory/ jar/lib/ext

cd /opt/installs/jdk1.8.0_05/jre
wget http://download.java.net\
/media/jai/builds/release/1_1_3/jai-1_1_3-lib-linux-amd64.tar.gz
tar xvf jai-1_1_3-lib-linux-amd64.tar.gz
cd jai-1_1_3/lib
cp *.jar /opt/installs/jdk1.8.0_05/jre/lib/ext
cp *.so /opt/installs/jdk1.8.0_05/jre/lib/amd64

 

5) Prerequisites for a second Tomcat install  Once Java is emplaced, it’s time for another Tomcat; the first one was added as part of the ArcGIS 10.2.2 for Server silent install.  That first tomcat has already snagged a listen on :6080 and provides the pathway to ArcGIS Server resources.  This being a second one, and given the very recent version of Java just installed, let’s push forward with a late compatible version of Tomcat, where the installation version choices are discussed here.The ArcGIS for Server 10.2.2 has installed a reasonably current Tomcat, yet for the sake of isolation between Geoserver and Esri ArcGIS for Server and to pursue maximum Geoserver performance, we’ll try one major version step forward on both Java and its associated (second) Tomcat instance.

# cd /ags1022/arcgis/server/framework/runtime/tomcat/bin
# sh version.sh

Second_Tomcat

For this example, Tomcat 8.0.8 will be built in the /opt/installs location, after creating a user specific to the Tomcat major version being installed.

useradd tom8
passwd tom8

 

Now is the time to record this assignment in the run book

usermod -G tom8,installer tom8

 

If the entire /opt directory has group ownership of installer and 775 permissions, then tom8 should be able to create the /opt/tomcat/base directory and build Tomcat there.

# chgrp -R installer /opt
# chmod 775 /opt
ls -ld /op*

Tomcat_check

6) Build Tomcat 8 from source  Once ready, it’s time to set up for multiple Tomcats.   In this example, the Tomcat is built alongside other work in /opt/installs.  The deployments will be given named instances in numbered directories for ease of management.  As the first Tomcat (instance 0) listens to port :8080, the second (instance 1) will be configured to port :8180, (instance 2) to :8280, and so forth.  There are a few other control and redirect ports that will also scale by :+100.
But first, build a most-recent-stable Tomcat.  Get a source tarball (or is it a Tomcat hairball?) here.  The link to your chosen version number e.g. 8.0.8 is what to click through.  Build instructions based on this page, and multiple Tomcat instance deployments modeled from this page.

tomcat_download1
use path from the tar.gz link
tomcat_download2

su - tom8

cd /opt/installs
export ANT_HOME=/opt/installs/apache-ant-1.9.4
PATH=$PATH:/opt/installs/apache-ant-1.9.4/bin
export PATH

wget http://<some_apache.org_mirror>\
/apache/tomcat/tomcat-8/v8.0.8/src/apache-tomcat-8.0.8-src.tar.gz
tar xvf apache-tomcat-8.0.-src.tar.gz
cd apache-tomcat-8.0.8-src

Since this build is being done by the tom8 user, it is not permitted to build at the default system location /usr/share/java.  Avoid this faux pas by editing a proper, build.properties file.

cp  build.properties.default  build.properties

edit the value defined for base.path to something like /opt/tomcat/base
tomcat_path

ant

Ant will be downloading many jar files from Apache.org and  Eclipse.org over about a minute.
The dependencies will reside in /opt/tomcat/base, and the actual built Tomcat ends up down in /opt/installs/apache-tomcat-8.0.8-src/output/build   as a reference copy.
tomcat_check2
7) Boost performance with Apache Portable Runtime (APR) and Tomcat Native Library  As observed by the Tomcat configuration test, the best performance in production may be achieved by adding a native library.  This should provide some functions natively compiled faster than interpreted Java code, in particular SSL encryption.

wget http://<some apache mirror site>\
/apache//apr/apr-1.5.1.tar.gz
tar xvf apr-1.5.1.tar.gz
cd apr-1.5.1
./configure
make
sudo make install

This should place the APR configuration file at /usr/local/apr/bin/apr-1-config

wget http://<some apache mirror site>\
/apache/tomcat/tomcat-connectors/native/1.1.30/source/tomcat-native-1.1.30-src.tar.gz
tar xvf tomcat-native-1.1.30-src.tar.gz
cd /tomcat-native-1.1.30-src
cd jni
ant
cd native

8) Install two Tomcat instances in multi-instance configuration  With a clean deployment directory structure in mind, deployment is a matter of copying the reference directories from  build to a destination, and changing a few items in configuration files.  First copy into two instance directories; Instance 1 will be given to GeoServer and named geosrvr while Instance 2 will be kept available for the next Java servlet application and named tomtwo.

cp -pr  /opt/installs/apache-tomcat-8.0.8-src/output/build  /opt/tomcat/1
cp -pr  /opt/installs/apache-tomcat-8.0.8-src/output/build  /opt/tomcat/2

Then, make a (very) temporary install of a legacy Tomcat from distribution to grab its RHEL configs

yum -y install tomcat6
cp /etc/sysconfig/tomcat6  /opt/tomcat/base/tomcat6_sysconfig
cp /etc/init.d/tomcat6  /opt/tomcat/base/tomcat6_init_d
cp /etc/tomcat6/tomcat6.conf  /opt/tomcat/base/tomcat6_orig.conf
cp /usr/sbin/tomcat6  /opt/tomcat/base/tomcat6_sbin
yum remove tomcat6
rm -rf /usr/share/java/tomcat6*
rm -rf /usr/share/tomcat6
mkdir -p /var/cache/tomcat8/temp
chown -R tom8 /var/cache/tomcat8
chgrp -R tom8 /var/cache/tomcat8

Modify the  the  tomcat6_orig.conf  script with values like these

CATALINA_BASE="/opt/tomcat/1"
CATALINA_HOME="/opt/tomcat/1"
JASPER_HOME="/opt/tomcat/1"
CATALINA_TMPDIR="/var/cache/tomcat8/temp"
CATALINA_PID=”/var/run/tomcat8.pid”

Modify the  the  tomcat6_init_d  script with values like these

TOMCAT_CFG=”/etc/tomcat8/tomcat8.conf”
TOMCAT_SCRIPT=”/usr/sbin/tomcat8”

 

Then place these scripts where they need to go:

mkdir /etc/tomcat8
cp -p /opt/tomcat/base/tomcat6_orig.conf  /etc/tomcat8/tomcat8.conf
cp /opt/tomcat/base/tomcat6_sbin  /usr/sbin/tomcat8
cp /opt/tomcat/base/tomcat6_init_d  /etc/init.d/tomcat8

These config files are the key to making named Tomcat instances out of the numeric Tomcat instance directories.  By installing these copies, each can be modified to launch independent instances.  Here is an example for the first two.

ln -s /etc/init.d/tomcat8  /etc/init.d/geosrvr
ln -s /etc/init.d/tomcat8  /etc/init.d/tomtwo
cp  /opt/tomcat/base/tomcat6_sysconfig  /etc/sysconfig/geosrvr
cp  /opt/tomcat/base/tomcat6_sysconfig  /etc/sysconfig/tomtwo

Edit  /etc/sysconfig/geosrvr to point to the first Tomcat instance path by setting these lines. Uncomment them, or copy and uncomment the copied line as you choose.

JAVA_HOME=”/opt/installs/jdk1.8.0_05/bin/java”
JRE_HOME=”/opt/installs/jdk1.8.0_05/jre/bin/java”
CATALINA_BASE=”/opt/tomcat/1”
TOMCAT_USER=”tom8”
CATALINA_PID=”/var/run/geosrvr.pid”
CONNECTOR_PORT=”8180”
TOMCAT_LOG=”/opt/tomcat/1/logs/catalina.out”

Edit  /etc/sysconfig/tomtwo  to point to the first Tomcat instance path by setting these lines. Uncomment them, or copy and uncomment the copied line as you choose.

JAVA_HOME=”/opt/installs/jdk1.8.0_05/bin/java”
JRE_HOME=”/opt/installs/jdk1.8.0_05/jre/bin/java”
CATALINA_BASE=”/opt/tomcat/2”
TOMCAT_USER=”tom8”
CATALINA_PID=”/var/run/tomtwo.pid”
CONNECTOR_PORT=”8280”
TOMCAT_LOG=”/opt/tomcat/2/logs/catalina.out”

Secure the installation with a Java key store as along the lines of this discussion

$JAVA_HOME/bin/keytool -genkey -alias tomcat -keyalg RSA \
-keystore /opt/tomcat/1/conf/.keystore

cp  /opt/tomcat/1/conf/.keystore  /opt/tomcat/2/conf

Tune definitions in each instance’s  server.xml to be distinct.  There is virtue in continuing the pattern for each instance, should it ever be necessary to add yet other instances.  Recall that the Esri installation uses Tomcat ‘0’, and the new numbered instances configured here start with Tomcat ‘1’.  The pattern is to add (Instance*100) to the port numbers for each successive Tomcat instance.  The first two configurations can go like this:

Edit  /opt/tomcat/1/conf/server.xml  to have these (the Server tag is closed at end of file)

<Server port=”8105” shutdown=”SHUTDOWN”>
<Service name="Catalina1">

<Connector port=”8180” protocol=”HTTP/1.1”connectionTimeout=”20000”
redirectPort=”8543” />

<Engine name="Catalina1" defaultHost="localhost">
<Connector port=”8109” protocol=”AJP/1.3” redirectPort=”8543” />

Edit  /opt/tomcat/2/conf/server.xml  to have these (the Server tag is closed at end of file)

<Server port=”8205” shutdown=”SHUTDOWN”>
<Service name="Catalina2">

<Connector port=”8280” protocol=”HTTP/1.1”
connectionTimeout=”20000”
redirectPort=”8643” />

<Engine name="Catalina2" defaultHost="localhost">
<Connector port=”8209” protocol=”AJP/1.3” redirectPort=”8643” />

Test the installation with its bin/  configtest.sh  script to

su - tom8
cd /opt/tomcat/1/bin
./configtest.sh

Manage the installation as tom8 through their scripts if the standard “service” install is tedious.
NOTE:  using this method, be certain to start up this service as user tom8, not as root

/opt/tomcat/1/bin/startup.sh
/opt/tomcat/1/bin/shutdown.sh

/opt/tomcat/2/bin/startup.sh
/opt/tomcat/2/bin/shutdown.sh

Edit  /opt/tomcat/1/conf/tomcat-users.xml  and  /opt/tomcat/1/conf/tomcat-users.xmlto add an administrative user who can access the manager GUI interface for each of the Tomcat instances

<role rolename="manager-gui"/>
<role rolename="admin-gui"/>
<user username="tom_admin" password="secret" roles="manager-gui,admin-gui,tomcat"/>

Edit  /opt/tomcat/1/conf/server.xml  to comment out the direct http Connector and create an SSL connector to use as the new default

<!--
<Connector port="8180" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8543" />
-->

<Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
port="8543"
maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
keystoreFile="/opt/tomcat/1/conf/.keystore" keystorePass="secreto"
clientAuth="false" sslProtocol="TLS" />

Edit  /opt/tomcat/2/conf/server.xml  to comment out the direct http Connector and create an SSL connector to use as the new default

<!--
<Connector port="8280" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8643" />
-->

<Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
port="8643"
maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
keystoreFile="/opt/tomcat/2/conf/.keystore" keystorePass="secreto"
clientAuth="false" sslProtocol="TLS" />

 

9) Install Apache Maven from source [Only required if building Geoserver from source] Building GeoServer from source depends on Apache Maven, a Java project management and build tool.  It’s possible to build Maven like this; uses Ant, which crawls slowly for 9 minutes.  Don’t set the build directory to be within the extracted source.

cd /opt/installs
wget http://<<some apache.org mirror site>>\
/apache/maven/maven-3/3.2.1/source/apache-maven-3.2.1-src.tar.gz
tar xvf apache-maven-3.2.1-src.tar.gz
cp -rp apache-maven-3.2.1 apache-maven-3.2.1-src
rm -rf apache-maven-3.2.1
cd apache-maven-3.2.1-src
export M2_HOME=/opt/installs/apache-maven-3.2.1
ant

maven1o2

export M2=$M2_HOME/bin
export PATH=$M2:$PATH

mvn --version

maven2o2

10) Build Geoserver from Git Clone  (or…)  Source is only for a development build.
Greater deployment efficiency will be found with a stable deployment from WAR (below).

mkdir /opt/installs/geoserver
cd /opt/installs/geoserver
git clone https://github.com/geoserver/geoserver.git
cd /opt/installs/geoserver/src
mvn clean install

 

Maven does a remarkably good job of building GeoServer and then running a great many tests on the resulting application.  After all, this is the developer’s version so you’d hope it’s well tested.
If there’s no patience for fixing Java version dependencies in some of the tests, just use guidance from http://docs.geoserver.org/latest/en/developer/maven-guide/index.html

mvn -DskipTests clean install
mvn clean install -P restconfig

 

11) Install Geoserver from WAR  The source web app archive (WAR) is available in .zip format.

cd /opt/tomcat/1/webapps
wget http://sourceforge.net/projects/geoserver/files/GeoServer/2.5.1/geoserver-2.5.1-war.zip
unzip  geoserver-2.5.1-war.zip
cd /opt/tomcat/1/bin
./shutdown.sh
./startup.sh

 

12) Observe Geoserver in Tomcat Admin GUI  Tomcat has a built-in web administration app, but one must first configure a Tomcat admin user who can access it.  This can be done by editing the user file  /opt/tomcat/1/conf/tomcat-users.xml and adding sections like

<role rolename="manager-gui"/>
<user username="tom_admin" password="*******" roles="manager-gui,tomcat"/>

Then navigate a browser to the Tomcat instance, such as https://sg11:8543 and click the button in the upper right labeled Manager App
tomcat_admin1

Once logged in, the screen should show that Tomcat container instance’s running applications
tomcat_admin2
13) Localize the Geoserver Install  Clicking on the path link to /geoserver in the Tomcat Web Application Manager, one should get to the running Geoserver Welcome page.  If you are provoked by the defaults to localize, the Welcome page can be tuned by editing     /opt/tomcat/1/webapps/geoserver/data/global.xml

More importantly, prepare production use by moving data directory to a separate location per http://docs.geoserver.org/stable/en/user/production/data.html by first creating a new top-level data direcotry for Geoserver 2.5.1

cd /opt/tomcat/1/webapps/geoserver
sudo mkdir /gdata
sudo chown tom8 /gdata
sudo chgrp tom8 /gdata
cp -r data /gdata

then involves editing /opt/tomcat/1/webapps/geoserver/WEB-INF/web.xml
To include an uncommented section like this

<context-param>
<param-name>GEOSERVER_DATA_DIR</param-name>
<param-value>/gdata/data</param-value>
</context-param>

Save the edits, and then either Reload, or Stop then Start the geoserver application
tomcat_geoserver_app

It’s then possible to delete the old data directory  /opt/tomcat/1/webapps/geoserver/data

14) Verify Geoserver runs properly  Just browse to the geoserver app’s home page and explore some of Data > Layer Preview items

https://sg11:8543/geoserver/

No responses yet

Jun 27 2014

SGeoS Add EAS Enterprise Address System Web App Server components – Module 8 of 9

Published by under SL In General

Add EAS Web Application Server components

Build steps for configuration Module-stage-8

These final sections are adapted from deployment notes here
https://sfgovdt.jira.com/wiki/display/MAD/linux+deploy
https://sfgovdt.jira.com/wiki/display/MAD/web+server+-+setup+and+admin+-+linux
At this late point in the module assembly, there should already be a new Python 2.7.6 environment available

1) Start from completed system Module-stage-7

2) Verify access to updated Python  Since the Centos 6.5 system default python 2.6.6 remains intact, activate the updated by invoking the virtual environment created earlier

source /opt/installs/venv/bin/activate
python --version
   Python 2.7.6

3) Enable httpd server-info for EAS debugging   By default this will (rightly) be disabled. These directions are adapted from http://httpd.apache.org/docs/current/mod/mod_info.html
Edit /usr/local/httpd/conf/httpd.conf to uncomment these lines

LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authz_host_module modules/mod_authz_host.so
#   disable ASAP after EAS debugging
LoadModule info_module modules/mod_info.so

and also add a section like this at the bottom of /usr/local/httpd/conf/httpd.conf

<Location /server-info>
   SetHandler server-info
   Allow from 10.x.xx.108
</Location>

then navigate a browser to the new location https://sg11/server-info

4) Build mod_wsgi from source   This is the mighty Web Server Gateway Interface module for Apache httpd that lets Python in; steps below adapted from the project documentation at https://code.google.com/p/modwsgi/wiki/QuickInstallationGuide
When a migration was forced from code.google.com, Graham Dumpleton appears to have created the new distribution point on GitHub.

cd /opt/installs
wget -O mod_wsgi-4.2.3.tar.gz \
https://github.com/GrahamDumpleton/mod_wsgi/archive/4.2.3.tar.gz
tar xvf mod_wsgi-4.2.3.tar.gz
cd mod_wsgi-4.2.3
./configure --with-apxs=/usr/local/httpd/bin/apxs --with-python=/usr/local/bin/python2.7
make

5) Install mod_wsgi as httpd module   Done as root.

cd /usr/local/httpd/modules
cp /opt/installs/mod_wsgi-4.2.3/src/server/.libs/mod_wsgi.so  .

Edit  /usr/local/httpd/conf/httpd.conf to include the following line in the general vicinity of line 153  below similar statements, and load  wsgi_mod

LoadModule wsgi_module modules/mod_wsgi.so

Restart the newly modded web server

apachectl2  restart

verify the richness of the new wsgi_mod by browsing to the location like
https://sg11/server-info#mod_wsgi.c

6) Build Psycopg from source and install   Done as system user  postgres after making the upgrade to Python 2.7.6 a default choice

cp  /usr/bin/python  /usr/bin/python2.6
rm  /usr/bin/python
ln -s /usr/local/bin/python2.7  /usr/bin/python
su - postgres
cd /opt/installs
wget  https://pypi.python.org/packages/source/p/psycopg2/psycopg2-2.5.3.tar.gz
cd psycopg2-2.5.3

In the psycopg directory, edit setup.cfg to point to PG configuration script pg_config
by uncommenting and setting this line to point at the postgresql built from source NOT using the –with-gssapi  configure option.

pg_config=/usr/local/pgsql/bin/pg_config
static_libpq=1

then build the module, and install

python2 setup.py build
sudo python setup.py install

then verify the psycopg2 version using the newer Python.
psycopg_version

 
7) Confirm PROJ.4 and GEOS installs    Already done as Module-Stage-3, Step 4 and Step 5 for PROJ.4 and GEOS, respectively.

8) Drop in Django    Done as root to load as a resource in system Python.  Uses pip installer that was installed during  Module-Stage-1, Step 9, which makes the process exceedingly easy.

pip install Django==1.6.5

Then run python, perhaps as system user postgres to confirm the install
django_version

9) Get Jogging  The python logging wrapper–pronounced “yogging”

cd /opt/installs
wget -O zain-jogging-0.2.2.tar.gz https://github.com/zain/jogging/tarball/v0.2.2
tar xvf zain-jogging-0.2.2.tar.gz
cd zain-jogging-976ff35
python2 setup.py install

10) Install EAS source  Clone the EAS source from Atlassian Bitbucket repository. The following example describes the URL for bitbucket user “RacerX”, then saves a copy of the clone directory with a date stamp.  The cloning time might be over 10 minutes, so

cd /opt/installs
hg clone https://RacerX@bitbucket.org/sfgovdt/eas
cp -r eas eaas_clone_yyyymmdd_hhmm
cd /opt/installs/eas

Initial deployment involves copying the eas folder to root of httpd web content.  Here the copy is named with trailing ‘0’ for testing purposes.

cd /opt/installs
cp -r eas /htdocs/eas0

11) Grab latest stable OpenLayers  The Open Source Geospatial Foundation’s OpenLayers Project provides a JavaScript API for open maps that is used by EAS.

cd /opt/installs
wget http://openlayers.org/download/OpenLayers-2.13.1.tar.gz
tar xvf OpenLayers-2.13.1.tar.gz

12) Obtain  ExtJS  The JavaScript object framework produced by Sencha Inc. for efficiently crafting very high-quality interfaces.  This is a commercial product that can be obtained by agreeing to terms of the Gnu Public License.

cd /opt/installs
wget http://cdn.sencha.com/ext/gpl/ext-4.2.1-gpl.zip
unzip ext-4.2.1-gpl.zip

13) Deploy EAS web application code  The deployment of EAS web app involves copying code to a chosen directory in Tomcat / webapps.  Part of the process has been automated, but it is not detailed here in the system build procedure.

No responses yet

Jun 26 2014

SGeoS OpenSim 0.8 on Mono 3.6.1 in CentOS 6.5 for SGeoS (Standard Geospatial Server) – Module 9 of 9

Published by under SL In General

As written yesterday, I’m working my way back through build notes.  This draft shows how I reached a couple of dead ends, backed up and took a small turn.  Here’s the end point where new Open Simulator 0.8, not quite two weeks old, is on a git clone of yesterday’s Mono.
The preceding steps have been written as they were developed–and will be posted subsequently.

Install OpenSim to Demonstrate Mono Apps

Build steps for configuration Module-stage-9

 

This installs the immersive 3D virtual world Open Simulator, developed as a C# application, to demonstrate the use of mono on the SGeoS testbed.  It also provides an update to the latest possible mono version, an ASP.NET module for Apache, and the curious little web server XSP as a more direct way of deploying .NET applications.

1)   Start from completed system Module-stage-8


2)  Verify prerequisites and build libGDI+  These are adapted from a build description page http://stackoverflow.com/questions/13184384/mono-3-0-0-build-on-centos-6

 yum -y  update
 yum -y install libpng-devel libjpeg-devel giflib-devel libtiff-devel libexif-devel libX11-devel \
     fontconfig-devel gettext httpd-devel

Confirm that the system standard pkg-config is used from /usr/bin/pkg-config or if any updates have been made, copy them elsewhere and link to /usr/bin/pkg-config; the make can take 12 minutes to build on a single thread, and using the -j4 thread saves nine minutes on an Atom host.

 which pkg-config
 /usr/local/bin/pkg-config
 cd /opt/installs
 wget http://download.mono-project.com/sources/libgdiplus/libgdiplus-2.10.9.tar.bz2
 tar xvf libgdiplus-2.10.9.tar.bz2
 cd libgdiplus-2.10.9
 ./configure --prefix=/usr/local
 export echo=echo
 make -j 4
 make install
 ldconfig

3)  Build mod_mono   This is an Apache connector for .NET requests.   It does not appear to be tuned for use with Apache 2.4, but can get along fine with CentOS 6.5 default Apache 2.2.16

Build instructions adapted from
https://www.bluewhaleseo.com/blog/asp-netc-linux-centos6-apache2-ispconfig3-mono3/
and requires axps which appears from httpd-devel group.

 cd /opt/installs
 wget http://download.mono-project.com/sources/mod_mono/mod_mono-2.10.tar.bz2
 tar xvf mod_mono-2.10.tar.bz2
 cd mod_mono-2.10
 ./configure prefix=/usr/local
 make
 make install
 ldconfig

To use the module, edit the relevant httpd.conf (likely either /etc/httpd/conf/httpd.conf or /usr/local/httpd/conf/httpd.conf)   to add this line to load mod_mono module and associate all the typical Microsoft web serving stuff, like index.aspx, Default.aspx, default.aspx, and associate ASP.NET file extensions with the ASP.NET MIME type.

 Include /etc/httpd/conf/mod_mono.conf

If, instead of all that, one simply want the mod_mono module alone to be loaded, then use this
LoadModule mono_module /usr/lib64/httpd/modules/mod_mono.so

4)  Clone Mono from git   There appeared to be issues with the very recent release tarballs, and for deployment of department apps, Mono could be an important part of the system.  These steps have been blended from several build descriptions
http://stackoverflow.com/questions/13184384/mono-3-0-0-build-on-centos-6
https://www.bluewhaleseo.com/blog/asp-netc-linux-centos6-apache2-ispconfig3-mono3/
http://stackoverflow.com/questions/22844569/build-error-mono-3-4-0-centos
http://stackoverflow.com/questions/11410020/compile-install-mono-on-centos-cant-get-past-make
while it’s compiling, mono looks like a mess of warnings for 20 minutes.

Cloning a git repository is at the edge; if it does not work one day, try pulling again in a day or two when the developers have patched it up.

While building works in the end, it does evoke a certain Microsoft-like cloud of doubt while watching mono build.  When it runs well, mono can be a very cool capability in the eyes of those invested in .NET apps and a lever for mitigating anti-Linux attitude among Windows developers.

 cd /opt/installs
 git clone  https://github.com/mono/mono.git
 cd mono
 ./autogen.sh --prefix=/usr/local

mono_autogen0

 

The next step is the  intriguing “Use mono to make mono” step.  Monolite is the spartan build that only has enough to run the old gmcs.exe compiler, from which the real mono can be bootstrapped.

 make get-monolite-latest
 make EXTERNAL_MCS=${PWD}/mcs/class/lib/monolite/gmcs.exe -j 4
 make check -j 4

Mono should pass all checks

 make install
 mono -V

mono_version_20140626

 

For future updates of Mono, pull from git if desired

 cd /opt/installs/mono
 git pull
 ./autogen.sh --prefix-/usr/local
 make
 make install

5)  Install Mono devleopment packages  these appear to be required by nant to build OpenSim.
These notes were made with reference to
http://stackoverflow.com/questions/16900575/install-mono-and-monodevelop-on-centos-5-x-6-x
It’s important that these Windows things know their way around so set the environmental variable such that PKG_CONFIG_PATH gets to the directory that’s got your latest mono.pc in it

 yum install gtk2-devel libglade2-devel
 export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:\
 /usr/share/pkgconfig:/usr/lib/pkgconfig:/usr/lib64/pkgconfig

Go for gtk-sharp, a dependency of mono-develop

 cd /opt/installs
 wget http://download.mono-project.com/sources/gtk-sharp212/gtk-sharp-2.12.8.tar.bz2
 tar xvf gtk-sharp-2.12.8.tar.bz2
 cd gtk-sharp-2.12.8
 ./configure --prefix=/usr/local

Patch three sources in glib/glue/*.c  that try to include below <glib.h/> to only include <glib.h>
This was necessary on 2014.06.25 and may not remain so.
patch_1o2

change them to look like the following, then make; expect to see warnings and some sorts of errors.

patch_2o2

 make
 make install

Now go for gnome-sharp, another dependency of mono-develop

 yum install pango-devel atk-devel libgnome-devel libgnomecanvas-devel libgnomeui-devel
 cd /opt/installs
 git clone git://github.com/mono/gnome-sharp
 cd gnome-sharp
 ./bootstrap-2.24 --prefix=/usr/local
 make
 make install

Now go for mono-addins, yet another dependency of mono-develop

cd /opt/installs
 git clone git://github.com/mono/mono-addins
 cd mono-addins
 ./autogen.sh --prefix=/usr/local
 make
 make install

Finally go for mono-develop itself

cd /opt/installs
 wget http://download.mono-project.com/sources/monodevelop/monodevelop-3.1.1.tar.bz2
 tar xvf monodevelop-3.1.1.tar.bz2
 cd monodevelop-3.1.1
 ./configure --prefix=/usr/local
 make -j 4
 <!-- fail 2014.06.25>
cd /opt/installs
 git clone git://github.com/mono/monodevelop
 cd monodevelop
 ./configure --prefix=/usr/local --select

<< maybe just choose main >>

make
 <!-- fail 2014.06.25>

Don’t be too disheartened if the full monodevelop binary doesn’t build, as having the dependencies is a big part of the purpose here.

6)  Install Nant from binary  a popular MS adaptation of Apache Ant build tool.  Used for building Open Simulator or other complex .NET assemblies from source, if one wants to try that later.

cd /opt/installs
 wget http://sourceforge.net/projects/nant/files/nant/0.92/nant-0.92-bin.tar.gz
 tar xvf nant-0.92-bin.tar.gz

that extracts a working binary version of NAnt.exe, so make a script to invoke it

touch /usr/local/bin/nant

then edit that file to include the invocation against mono

mono /opt/installs/nant-0.92/bin/NAnt.exe

finally set the launch script to be executable and perhaps owned by installer group.
The test is ironic, yet informative.

chmod 754 /usr/local/bin/nant
 chown root:installer /usr/local/bin/nant
 nant

NAnt_version_20140626

 

7)  Install MySQL from repository  Just the basics; MySQL might even already be installed.  This is a preferred means of asset storage for Open Simulator, although not configured there by default.

yum install mysql

 

8)  Install Open Simulator binary  A build from source might be attempted, as CentOS doesn’t seem to have been the most popular choice among Opensim adherents.  Requires Mono >= 2.4.3 and NAnt >= 0.85 which both should be satisfied at this point.  A better choice: the compiled binary will very likely just run with Mono (!)

For a source build attempt, satisfy nant with a symlink at a deprecated location.

ln -s /usr/local/lib/mono/4.5/mcs.exe /usr/local/lib/mono/2.0/gmcs.exe

Then just get the OpenSim 0.8 binary; you’ve already got a very current Mono available

cd /opt/installs
 wget http://opensimulator.org/dist/opensim-0.8.tar.gz
 tar xvf opensim-0.8.tar.gz
 cd opensim-0.8
 cd bin

Back up default OpenSim.ini  to  OpenSim_orig.ini — and other files as appropriate from bin/config-include plus configurations described at http://opensimulator.org/wiki/Build_Instructions
To launch a configured OpenSim, it’s like this:

mono OpenSim.exe

Edit  /etc/sysconfig/iptables to have a line like this to allow OpenSim access on :9000

-A INPUT -m state --state NEW -m tcp -p tcp --dport 9000 -j ACCEPT

Then restart iptables

service iptables restart

Testing was conducted with the Singularity viewer for 64-bit Windows, with connections to the simulator running on SGeoS  http://www.singularityviewer.org/

SGeoS_OpenSim_first_light_20140626

In-world saved image of initial simulator region, demonstrating terrain edits, vegetation, object construction, and time-of-day adjustments.  Viewer and interactive editing from Windows 7 workstation, using Singularity Viewer (64-bit) 1.8.5 (5617) to  Open Simulator 0.8 server as compiled C# project running on SGeoS under Mono 3.6.1.

 

That’s it for this module’s draft.  The server, it’s working.  If one were to only care about getting OpenSim 0.8 running on Mono 3.6.1, then perhaps this is everything.  For the SGeoS, these build notes are only the ninth of nine modules.

No responses yet

Jun 25 2014

Open Simulator joins the SGeoS build — a strategy for blogging the builds

This is the first of what should be  a set of posts that detail a server build process for the San Francisco Enterprise Geographic Information Systems Program (SFGIS) Standard Geospatial Server (SGeoS).  In fact, the build work has been ongoing for several weeks and is concluding here, with OpenSim.

The motivation for including OpenSim in the platform was a desire to provide support for legacy .NET applications that may exist in various departments. In the interest of creating a Microsoft-neutral build that is framed with Open Source components, it was natural to bundle the Mono framework into the SGeoS design.  And while individual department applications are their own business and not part of the standard build, OpenSim serves as an excellent demonstration of the utility of the Mono framework as included on the server.  That , together with my perspective that immersive 3D clearly should be associated with geospatial servers, is why OpenSim is included in the Standard Geospatial Server.

OpenSim is not trivial by any means, and yet it is not such a resource hog that it would be infeasible to bundle it.  What’s more, it is an opportunity to distribute immersive 3D technology packaged with other geospatial capabilities.

Since the build descriptions are being transcribed from a build document that is approaching 80 pages on Google Docs, it seems prudent to break it up into individual modules.   And since WordPress here is configured to show older posts below newer ones—I’ll start down at the end modules and post new build descriptions for earlier modules in later days.

The original notion for SGeoS was to have modular build chapters that could provide a unit of capability.  That way, only selected modules need be configured.  After discussions with VMware engineers, I became intrigued by the notion of making a single server image that could run everything, all at once, and then disable unneeded featured in an actual deployment.  So the build document was initially structured with module-like chapters, but in fact the server builds them all—so it’s worth viewing the build document in sequence.

The modules will probably end up  numbering about 10, including packaging for production and possibly default-disabling of most items.   If one watches too closely, it might seem like I’m making a countdown to completion.  But this will end with a stub for deployment packaging, work back through an OpenSim build, and end up with imaging an install of CentOS 6.5 onto a new VM guest system.

No responses yet

Jun 12 2014

Real-life geospatial in virtual machines – an adapted focus

Published by under SL In General

In the past 31 months of silence, there’s been both great focus on one topic and a transition to something new.

For 27 of those months, creative energy was given over largely to modeling of surface water flow on a seamless topographic-bathymetric surface for Marin County, California.  Creek mapping has been either pure-2D work, or 2.5-D work with storm water pipe diversions beneath the bare earth surface.  The model is produced to standards of the US Geological Survey’s National Hydrography Dataset and associated Watershed Boundary Dataset.  Collaborative discussion about workflow and product review may be forming at http://nhd.3dg.is over the next few months.

In the past three months, a new full-time day job has provided huge boosts of energy to some more urban applications.  A discussion site may be visible at https://www.yammer.com/standardgeospatialserversgeos .
The big new thing here is documented build instructions for an Open Source-maximized, Licensing cost-minimized modular standard design for geospatial servers.

The prime use case for the design is a local government agency that may have several department-level sites running Esri ArcGIS for Server, each configured in its own way and with different server operating systems and products from multiple database vendors.  There may be an existing applications over an Open Source GIS stack, or there may be interest in implementing such a stack for testing.  There may be a desire to connect SQL-skilled staff with geospatial data and analysis capabilities.  As new 3D geospatial data and field devices arrive, there might even be teams that want a virtual world simulator—maybe even OpenSim.

For that environment or one similar to it, the San Francisco Enterprise Geographic Information System Program (SFGIS) has devised a unified geospatial server platform to maximize performance while minimizing marginal licensing costs: the Standard Geospatial Server (SGeoS).

The SGeoS design packs many enterprise geospatial capabilities into a single configurable server module, including

  • Esri ArcGIS 10.2.2 for Server Standard
  • Esri ArcGIS 10.2.2 Web Adaptor
  • Esri ArcGIS 10.2.2 Enterprise Geodatabase
  • PostgreSQL 9.2.8 Enterprise Database
  • PostGIS 2.1.3 with SFCGAL database geoprocessing
  • Apache httpd 2.4.9
  • OpenSSL 1.0.1h
  • Apache Tomcat 8.0.8 Java 1.8 Servlet container
  • Django 1.6.5 / GeoDjango
  • OpenLayers 2.13.1 / Ext JS 4.2.1
  • Mono 3.2.8  for .NET 4.5 compatability
  • CentOS 6.5 (unbranded RHEL 6.5) operating system

Testbed deployment of one SGeoS device requires a single-core VM guest with 4GB memory and 40GB of storage.  Production deployment is anticipated to involve multiple SGeoS machines for each site; unused capabilities will be disabled.  This means that a web-tier module will have geoserver and database functions disabled, a geoserver-tier module can have httpd and database disabled, and a database-tier module can have httpd and geoservers disabled.

In testbed form, capabilities are configured to run all at once without conflicts, absent significant user load.

No responses yet

Nov 22 2011

Visiting an old haunt – Darb in Gualala

Published by under SL In General

Long has it been, but there’s still a bit of energy in the little guy Darb.
For visitors to the old Berkurodam site in Second Life’s Gualala region, a text link to the celebratory YouTube video now glows.

New Video billboard in Gualala

No responses yet

Nov 15 2011

3D Geospatial For Real—not a simulation and Kitely, on-demand Opensim

Thanks to the astronauts aboard the International Space Station, their time-lapse photography at very high ISO that helps to share some of what their eyes may well see, and of course Michael Koenig for his care and smoothing of the HD video, with some loungy score, too.
Take five (minutes) and watch it on HD in a darkened room. You might find yourself pausing, reviewing, and spending 20 minutes enjoying.

Earth | Time Lapse View from Space, Fly Over | NASA, ISS from Michael König on Vimeo.

I was fascinated by an orange wiggle, that turned out to be the astoundingly well-lit India-Pakistan border, around 1000 km long.

Meanwhile, I’m forming some plans for next semester’s course, and have realized that it may well be possible to offer students training in multi-user virtual environments without hacking one of the lab workstations to image it as an Opensim server. Thanks to the incessant business analytics of Maria Korolov over the past few years, it was possible for me to quickly get caught up in the new and improved options for cloud hosting of Opensim regions.

Right away it became clear that the business model of Kitely was quite compatible with my modest but area-expansive needs for real-life terrain simulations.  I’ve found it quite easy to get set up with a single region, and that’s a really big start.  I was able to use the latest beta Second Life 3.2 viewer to connect to the latest Opensim 0.7.2 stable release, tweak terrain and set up a few flexi-prims to test the weather.  Nice work technically, and a very nice pricing scheme for my sort of use.  I’m also very sympathetic to Ilan Tochner’s philosophy of “just keep building new regions”—it’s a consistent theme with cloud solutions, and refreshing to see it in connection with Opensim.

No responses yet

Nov 01 2011

Google Maps new MapsGL engine – heavy use and lots of love

Published by under Google Maps MapsGL

I’ve got a version of streams set up, and we’re poised to update our terrain before running through some cycles of review and revision.

But first, I need to get our community base map updated, and that’s been one detour after another it seems.  An innocent-sounding reviewer request to straighten out our parks has led to a months-long campaign of updating shoreline, lower-low water inundation, the judicious trimming of private parcels at the high water line, the representation of park lands both named and parceled as well as public access spaces in the intertidal reaches, and more.  That more has involved a serious effort to accurately represent our marsh lands in terms of tidal channels, mud flats, and vascular marsh vegetation—all of it based on aerial photography and much of it derived from our latest National Agricultural Imagery Program (NAIP) 1-meter 4-band imagery, which has proven very high quality both in spatial edge content as well as dynamic range of lighting.  Marsh features both current and fading in less-than-vigorously reclaimed land have been given much attention.  The details are covering the Petaluma River marsh (downstream of the outflow of San Antonio Creek waters into the Petaluma River) and the Petaluma River banks.

But that led to the latest  detour: those massive steel lattice towers that support electrical transmission lines.  They’re actually very significant landmarks, as well as eminently mappable features.  They were easy, but led to harder stuff: the sub-transmission network.  It’s like you start out with the easy 240 kV lines, and then come back for more.  One day it’s just a few big towers, and then the next thing you know, one’s back for 120 kV, and maybe 64 kV and 32 kV rural lines too.  You know it’s bad when you can’t stop and you just want to locate a few more poles to make it to the next county line…  And then one day you wake up and it’s been 10,000 poles.  ;^)

Anyhow, as a result of what began as a mappable affront to wetland areas, tidal marshes in particular, has now turned into a draft electrical transmission and sub-transmission (not single-house distribution) network feature class.  Turns out that the NAIP 2010 imagery, together with 10cm imagery from 2004 in the urban areas and various 30cm sources in other areas and years has been quite enough to dial in  public utility assets, frequently constructed in public rights-of-way, extracted using basic geospatial intellgence techniques applied to publicly available imagery resources.  The catalyst has been the evolved Google Maps MapsGL viewer engine.

I’ve only started to use MapsGL intensively in the past 10 days or so, but it is astoundingly well integrated.  Right now, I have the sense that there is nothing else quite like it out there for public use.  The interface experience was very different at once, and the viewer actually suggested that I try it when I was very actively moving between 2D map view and Street View, using the mouse wheel.  When I switched over to the new viewer, I was very pleasantly shocked.  The 2D “satellite” view, the “45-degree” views, and Street View were all smoothly mediated by a 3D model textured with imagery from the 45-degree views—using features from the Google Earth plugin.

It was shocking, and something that got me to jump out of my chair to share with a colleague, when I realized that the Earth view was being used to generate a transition between different rotations of the 45-degree view.  Sound obscure?  Consider an oblique 45-degree view looking default north, where you want to look toward the west instead.  Click the compass ring and it will turn, as expected.  What’s not expected (at first) is that the oblique view transition, rather than blanking out and plopping the next view onto the screen, instead puffs out to become a textured 3D sculpty model.  Yes, that means that the buildings, terrain, and trees are shown as they might be when very far zoomed in on Google Earth, and then that view rotates just like it would in a well-handled Second Life viewer, until it settles into the new oblique direction, after which the 3D effect fades and the oblique is presented.

But in one’s mind, crucially, the 3D impression remains and informs the interpretability of the the oblique.  The tree that covers the back part of the house now has been ascribed a 3D volume in the user/analyst’s mind, and suddenly makes more sense than the flat 45-degree view would on its own.

Even without the obliques as an intermediary, popping from 2D map view to Street View is mediated by the virtual reality of textured 3D sculpty objects, and this helps make the Street View perspective far more readable in an instant after arrival.

From my perspective, the MapsGL interface engine is a major evolution of 3D GIS–because it uses a simulated 3D textured surface space to mediate among 2D vertical orthoimagery, 2D oblique imagery, and panoramic ground-level imagery.  That it’s public and cost-free makes it compelling to use for meaningful applications.  In the past few days, I’ve been able to follow sub-transmission pole sequences through fairly rugged forested suburban settings, because my GIS-based moderately detailed imagery allows me to digitize whatever I can see or estimate, while on a full adjacent screen, MapsGL provides sharper orthoimagery, frequent oblique views in urban and adjacent areas, and Street View to tenaciously follow lines as they pass under tree canopy along roadways.

No responses yet

Aug 12 2011

New Flow Lines, and Marin Community Map progress

Published by under SL In General

I’m ashamed to see that posts have been blank since May. I have been busy on another site related to Cr-48 Chromebook usage—but that’s not about this stuff.

In the past months, I’ve been grinding on the Marin Community Map, in particular working out the details of how park lands interact with the tidal reaches. This as graded into a representation of tidal lands, a pulling back of water polygons to lower-low water, and the start of harmonization with the San Francisco Estuary Institute’s Bay Area Aquatic Resource Inventory (BAARI).

I’ve spent hours dealing with topographic (elevation)-based definitions of shorelines such as were used in our model of San Francisco Bay Conservation and Development Commission (SF BCDC) jurisdiction. But as it turned out, all of our interesting marshes and tidal lands are tilted down toward the bay—go figure! So using guidelines for delineation that were very aptly documented by SFEI for BAARI, I started returning to the National Agricultural Imagery Program (NAIP) 1-meter, 4-band imagery of tidal lands for photointerpretation. In many cases, the wetlands were more appropriately mapped using the NAIP imagery than they were using terrain-derived contours. Features like tidal channels creep up to higher elevations while maintaining their widths, while contours tend to pinch out at some point and start going back down the other side of the tidal channel.

One of the by-products of all this attention is that I’ve split out the tidal lands around Marin in that span between lower-low water—which will be cartographically filled in with a blue polygon and bathymetry contours—and high water, where the public easement for beaches stops. For consistency, I’ve detailed out every little patch of space between these tidal ranges, all around the county, and only left out places that were plainly in private ownership, like a back yard with a dock. In keeping with BAARI criteria, I’ve used NAIP color infrared imagery to detail out polygon areas for vascular life forms (marshes) and the tidal channels and outboard mud flats around them.

Particularly good views of lower-low water were captured in NAIP 2005 imagery. Fair views of medium tide were found in NAIP 2009 imagery. A nice mix of low tide and improved quality near-infrared band data are in the NAIP 2010 images. In the end, I’m using NAIP 2005 to trace the outer limits of mud flats at lower-low water, and using NAIP 2010 to detail out the extent of marshes, because excellent red contrast makes it easy in that year’s data.

Also, we’ve had significant progress / closure on the ArcHydro generation of flow lines countywide for Marin and associated watersheds. As of now, we have got flow models for drainage networks below 1-hectare catchment in all of Lagunitas creek, and below 1-acre catchments elsewhere. These flow lines have been attributed with catchment area every 10 meters along their length, which has allowed us to provisionally classify them for perennial, intermittent, ephemeral, tidal, or impoounded flow. Also, we have attributed USGS NHD FCode feature codes for every segment as either a flow-specific creek, various storm drain pipes and ditches, or artificial paths through standing water. All of this stuff is being run on our 45cm topographic-bathymetric surface model, so all of the ArcHydro flow lines are running seamlessly through the tidal reaches and out into deep water. It’s been particularly interesting to see where soft sediments meet granite and other outcrops offshore, as flow lines go from largely parallel sheets to dendritic patterns even when they are underwater, using this technique.

The modeled flow lines can be found at this link.

No responses yet

May 12 2011

My Goodness — it’s full of *stars*… WebGL fun and games

Published by under SL In General

I’ve been catching up on this week’s Google I/O 2011 via some videos.  Much of my interest has been on how Chrome is presented, and the video does not disappoint.

 

 

The most fascinating insight, in terms of 3D GIS, and shared (not yet multi-user) virtual environments, appears to be Web GL.   For the benchmark of performance that leads to hours of time wasting entertainment, would you care to see Angry Birds?  I’ve only tested this with Chrome 12 on the Cr-48 and Chromium 13 on Ubuntu, but here’s the site

And what’s (much) more, a vision of seamless integration of 2D animation, video, and interactive immersive 3D environments at 25+ FPS — the project at ro.me

If you’re like me, and can’t view it in full WebGL glory because you’re on a Cr-48 or some other earlier browser, here’s the trailer to help give you motivation to try out something new in the browser world.

 

 

And what really caught my ear, and hasn’t been a top note in yesterday’s blogs was this announcement: the Chrome Book subscription pricing of $20/user/month for education institutions is also for government institutions.  If this should include the same centralized web-based management of user cadre—it would seem a very attractive price point.  Right now, a typical well-endowed elementary school might have a cart with 28 MacBooks that gets wheeled around between classrooms.    If 30 Chrome Books weigh in at $600/month, they’ll get replaced under the upgrade program before the cost approaches that of the initial MacBook acquisition, won’t they?    Government offices might stand to get 60%–80% of their users off of Windows desktops and onto something less costly.

The initial pricing of the retail Chrome Books at $500 seems a bit steep, although the Samsung unit will probably have an Atom processor with four threads and some better GPU capability.  It’s the subscription pricing that really seems to be the main kick—because it sounds far more attractive than the retail option.

No responses yet

« Prev - Next »