Open source operating system quest

While waiting to see if my X300 sells on Craig’s list and before I eBay it, I figured I would go back to my roots and install a bunch of operating systems on the machine. It is very new hardware and I wanted to see what else is out there. For those who don’t know or haven’t ever read my white-paper, I wrote a fairly lengthy paper around 2000 that outlined how to build an open source operating system that was geared 100% towards desktop and laptop users. It covered mainly changes to Linux, but could be easily abstracted out to any operating system. It had things like:

  • Drop LHS (linux file system standard) in favor of a more friendly naming convention like /config, /system (or operating-system), /users, /applications, /libraries, etc.
  • Drop X-Windows in favor of OpenGL foundation with no networking and everything vector based where possible
  • Standard system APIs for everything, including graphics. No more Gnome vs. KDE vs. whatever
  • Better file permissions
  • Better login
  • Remove TTYs
  • Remove termcap and all that jazz
  • Assume latest modern hardware everywhere, even shell/terminals
  • Fix run-levels and services
  • Standard hardware abstraction
  • Better packaging (no more littering files everywhere)

It had a bunch of other stuff, but you get the drift. Some of this stuff has actually happened in the last decade or so. However, a lot of it hasn’t quite gotten there and Linux has suffered from more and more server syndrome that it probably will always be rough around the edges for desktops and laptops.

Anyways, back to the main point… I downloaded and installed a bunch of different operating systems and here’s what I found.

Linux (Ubuntu, Fedora, OpenSuse, etc)

Same old story. ACPI is rough, the CPU is a hog, the battery life is short, 3D desktop doesn’t quite work, missing drivers, bloated, etc, etc. Definitely a thumbs down on my scale, even though I am currently running Ubuntu on all my machines and love Linux to death, it still sucks for desktop and laptop compared to other OS’s.

OpenSolaris

Sorry to say that this is just like Linux with a different kernel and some minor tweaks here and there. It still runs X-Windows, Gnome and all the other Linux upper layers. Plus, it has worse driver support and very little modern desktop and laptop necessary support such as ACPI, CPU scaling, low voltage, etc. Definitely a thumbs down for now. We’ll see what Sun does. If they are smart, which they don’t appear to be yet, they would drop all the upper layers and build something better and new. They would also drop all the file system standard crap and just start fresh. They have the man power and the money to do it, just not the vision or the drive it seems.

Syllable

This is a very promising OS. They have a lot of the key components, but they have also been using Linux and Unix too long to deviate drastically enough to make it truly usable for the average Laptop and Desktop user. However, it still isn’t 1.0 and they might update some of these things. I never actually got it running, but from what I’ve seen on their website, they have the right idea. If I had some advice for all the developers and users of this OS it would be: hang a sign that says, “can my grandmother use this OS?” on your wall next to your computer and if that answer is ever “no”, fix things until it is “yes”. If I had a million bucks or so laying around, I’d definitely put my money on this project.

Hackitosh

Yeah, just wanted to see if I could get it running and the answer is no. Apple definitely has a great OS. Although some things are still lacking, it is probably the best out there right now. However, it runs best on Apple hardware and I’m not about to fight that. I’m planning on a 100% switch to Mac OSX here soon.

There are a few others that I didn’t get to like Haiku, but they seemed quite new as well and probably wouldn’t have worked all that well. Another thing that was lacking from most of the newest OS variants out there was 64bit support.

Multiple Tomcat instances Debian scripts

I opened a new Google Code project to manage the scripts I wrote that allow multiple instances of Tomcat to be run on Ubuntu. These scripts are now fully open source (more so than before I guess) and available to everyone. They are also more up-to-date than they were in my previous blog post about them.

Anyways, here’s the project link:

http://code.google.com/p/debian-tomcat-scripts/

You can check them out from SubVersion or browse them online. Enjoy!

Ubuntu Gusty and Tomcat 5.5

Looks like Gusty (or something) changes some of the installation for Tomcat 5.5 and it requires some work to get back to normal. Here’s my changes thus far:

sudo rm /usr/share/tomcat5.5/common/endorsed/*
cd /usr/share/tomcat5.5/common/lib
sudo apt-get install libmysql-java
sudo ln -s /usr/share/java/mysql-connector-java.jar

This fist off removes the bad XML jar files that Tomcat is setup with in its endorsed directory. These assume that you only need a subset of JAXP and don’t provide everything (i.e. XSLT). You could also symlink to the rest of the XML jars in /usr/share/java such as xalan2.jar, but I find that the bundled JAXP in the JDK (sun-java6-*) work much better.

The second part symlinks in MySQL drivers into the main installation. This is required if you are going to be setting up JDBC connection pools inside the Tomcat contexts to a MySQL database.

These are it thus far, but if I find more I’ll update this.

Multiple Tomcat instances on Ubuntu

I’m spent about 6 hours total working on getting multiple instance of Tomcat to run on Ubuntu and finally wrote some scripts that I’d like to share with folks. These scripts do the following:

1. Setup the box to host multiple instances (JVMs) running tomcat

This process cleans the old layout and sets up a new layout that allows for multiple instances. This includes a number of “instances” directories in /var/lib/tomcat5.5, /etc/defaults/tomat5.5 (a new directory) and /var/log/tomcat5.5.

2. Create new instances

This allows you to setup a new instance quickly. It creates all the necessary directories and also installs the new instance into the init scripts an run levels.

Things the scripts don’t do:

– Setup multiple mod_jk configurations and apache virtual hosts
– Any SSL or security management

Here’s how to use the scripts:

1. First make sure you have Java and Tomcat installed!

bash$ sudo apt-get install sun-java6-jdk tomcat5.5

2. Grab ALL of the scripts and place them in some directory (anywhere is fine)

3. Run the multiple-instances.sh script as root from the directory you put it in

bash$ sudo ./multiple-instances.sh

4. Run the new-instance.sh script to create a new instance

bash$ new-instance.sh <instance-name>

5. Configure the new server by editing the files in /var/lib/tomcat5.5/instances/<instance-name> (mainly conf/server.xml)

6. Make sure each instance has a unique port for everything including the AJP, all listeners and the container

7. Fire it up!

bash$ sudo /etc/init.d/tomcat5.5_<instance-name> start

8. Check the logs in /var/log/tomcat5.5/instances/<instance-name>/catalina.out

Due to security considerations, I’ve included the scripts inline. Any comments are welcome:

multiple-instances.sh

#!/bin/bash

# Verify we are root
if [ "$USER" != "root" ]; then
  echo "You must run this script as root"
  exit 1
fi

# Shutdown any running instace
if ! /etc/init.d/tomcat5.5 stop; then
  echo "Unable to stop the running tomcat instance. Please stop it before running this script."
  exit 1
fi

# Copy over the tomcat.sh file before we change directories
if [ ! -f tomcat.sh ]; then
  echo "The tomcat.sh file is missing. It must be in the current directory so it can be placed"
  echo "into the new layout"
  exit 1
fi
cp tomcat.sh /etc/tomcat5.5
chmod go-wx /etc/tomcat5.5/tomcat.sh

# Now, head out and clean up
if ! cd /var/lib/tomcat5.5; then
  echo "You must first install tomcat from the apt repository"
  exit 1
fi

# Make the instances layout
mkdir instances
chown -R tomcat55:nogroup instances
chmod -R o-rwx instances
chmod -R g+w instances

# Clean up old layout
rm work
rm -rf webapps
rmdir temp
rmdir shared/classes
rmdir shared/lib
rmdir shared
rm logs
rm -rf conf

cd /usr/share/tomcat5.5
rm conf
rm logs
rm shared
rm temp
rm work

# Clean up defaults for the template script
cd /etc/default/
mv tomcat5.5 tomcat-bak
mkdir -p tomcat5.5/instances tomcat5.5/template
mv tomcat-bak tomcat5.5/template/tomcat5.5
chown -R root:root tomcat5.5
chmod -R go-w tomcat5.5
chmod -R g+r tomcat5.5

# Clean up the logs
cd /var/log/tomcat5.5
mkdir old
mv * old
mkdir instances
chown -R tomcat55:nogroup *
chmod -R o-rwx old instances
chmod -R g+w old instances

# Remove the old init script and turn off all script links for the run levels
rm /etc/init.d/tomcat5.5
update-rc.d -f tomcat5.5 remove

echo "Successfully setup the machine for multiple tomcat instances and cleaned up the single instance layout"

new-instance.sh

#!/bin/bash

if [ "$USER" != "root" ]; then
  echo "You must run this script as root"
  exit 1
fi

if [ $# != 1 ]; then
  echo "Usage: new-instance.sh "
  exit 1
fi

# Setup the instance
cd /var/lib/tomcat5.5/instances
if [ -d $1 ]; then
  echo "Instance $1 already created."
  exit 1
fi
mkdir $1

cd $1
mkdir conf webapps bin temp
if ! cp /etc/tomcat5.5/web.xml conf; then
  echo "Unable to create new instance $1 because /etc/tomcat5.5/web.xml doesn't appear to exist"
  exit 1
fi

if ! cp /etc/tomcat5.5/server.xml conf; then
  echo "Unable to create new instance $1 because /etc/tomcat5.5/server.xml doesn't appear to exist"
  exit 1
fi

if ! sed "s/@INSTANCE_NAME@/$1/g" /etc/tomcat5.5/tomcat.sh > bin/tomcat.sh; then
  echo "Unable to find the custom tomcat.sh file in /etc/tomcat5.5. This file must exist."
  exit 1
fi
chown -R tomcat55:nogroup /var/lib/tomcat5.5/instances/$1
chmod -R o-rwx /var/lib/tomcat5.5/instances/$1
chmod -R g+w /var/lib/tomcat5.5/instances/$1
chmod ug+rx bin/tomcat.sh

# Setup the logs
mkdir /var/log/tomcat5.5/instances/$1
chown -R tomcat55:nogroup /var/log/tomcat5.5/instances/$1
chmod -R o-rwx /var/log/tomcat5.5/instances/$1
chmod -R g+w /var/log/tomcat5.5/instances/$1
ln -s /var/log/tomcat5.5/instances/$1 logs

# Create catalina.policy (for the security manager)
echo "// This file is an example of a policy file. You can edit this file for the specific instance" > conf/catalina.policy.example
echo ""  >> conf/catalina.policy.example
cat /etc/tomcat5.5/policy.d/*.policy >> conf/catalina.policy.example
chown tomcat55:nogroup conf/catalina.policy.example
chmod o-rwx conf/catalina.policy.example
chmod g+w conf/catalina.policy.example

# Setup auto start
ln -s /var/lib/tomcat5.5/instances/$1/bin/tomcat.sh /etc/init.d/tomcat5.5_$1
update-rc.d tomcat5.5_$1 defaults 90

tomcat.sh

#!/bin/sh
#
# /etc/init.d/tomcat5.5 -- startup script for the Tomcat 5 servlet engine
#
# Written by Miquel van Smoorenburg .
# Modified for Debian GNU/Linux	by Ian Murdock .
# Modified for Tomcat by Stefan Gybas .
#
### BEGIN INIT INFO
# Provides:          tomcat
# Required-Start:    $local_fs $remote_fs $network
# Required-Stop:     $local_fs $remote_fs $network
# Should-Start:      $named
# Should-Stop:       $named
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start Tomcat.
# Description:       Start the Tomcat servlet engine.
### END INIT INFO

set -e

PATH=/bin:/usr/bin:/sbin:/usr/sbin
NAME=tomcat5.5
DESC="Tomcat servlet engine"
DAEMON=/usr/bin/jsvc
CATALINA_HOME=/usr/share/$NAME
INSTANCE_NAME=@INSTANCE_NAME@
DEFAULT=/etc/default/$NAME/instances/$INSTANCE_NAME

. /lib/lsb/init-functions
. /etc/default/rcS

# The following variables can be overwritten in $DEFAULT

# Run Tomcat 5 as this user ID
TOMCAT5_USER=tomcat55

# The first existing directory is used for JAVA_HOME (if JAVA_HOME is not
# defined in $DEFAULT)
JDK_DIRS="/usr/lib/jvm/java-6-sun /usr/lib/jvm/java-1.5.0-sun /usr/lib/j2sdk1.4-sun /usr/lib/j2sdk1.4-blackdown /usr/lib/j2se/1.4 /usr/lib/j2sdk1.5-sun /usr/lib/j2sdk1.3-sun /usr/lib/j2sdk1.3-blackdown /usr/lib/j2sdk1.5-ibm /usr/lib/j2sdk1.4-ibm /usr/lib/jvm/java-gcj /usr/lib/kaffe"

# Directory for per-instance configuration files and webapps
CATALINA_BASE=/var/lib/tomcat5.5/instances/$INSTANCE_NAME

# Use the Java security manager? (yes/no)
TOMCAT5_SECURITY=no

# Timeout in seconds for the shutdown of all webapps
TOMCAT5_SHUTDOWN=30

# End of variables that can be overwritten in $DEFAULT

# overwrite settings from default file
if [ -f "$DEFAULT" ]; then
	. "$DEFAULT"
fi

test -f $DAEMON || exit 0

[ -z "$TOMCAT5_USER" ] && TOMCAT5_USER=tomcat55

# Look for the right JVM to use
for jdir in $JDK_DIRS; do
	if [ -r "$jdir/bin/java" -a -z "${JAVA_HOME}" ]; then
		JAVA_HOME_TMP="$jdir"
		# checks for a real JDK like environment, needed to check if
		# really the java-gcj-compat-dev package is installed
		if [ -r "$jdir/bin/jdb" ]; then
			JAVA_HOME="$JAVA_HOME_TMP"
		fi
	fi
done
export JAVA_HOME

# Set java.awt.headless=true if JAVA_OPTS is not set so the
# Xalan XSL transformer can work without X11 display on JDK 1.4+
# It also looks like the default heap size of 64M is not enough for most cases
# se the maximum heap size is set to 128M
if [ -z "$JAVA_OPTS" ]; then
	JAVA_OPTS="-Djava.awt.headless=true -Xmx128M"
fi

JAVA_OPTS="$JAVA_OPTS -Djava.endorsed.dirs=$CATALINA_HOME/common/endorsed -Dcatalina.base=$CATALINA_BASE -Dcatalina.home=$CATALINA_HOME -Djava.io.tmpdir=$CATALINA_BASE/temp"

# Set the JSP compiler if set in the tomcat5.5.default file
if [ -n "$JSP_COMPILER" ]; then
	JAVA_OPTS="$JAVA_OPTS -Dbuild.compiler=$JSP_COMPILER"
fi

if [ "$TOMCAT5_SECURITY" = "yes" ]; then
	JAVA_OPTS="$JAVA_OPTS -Djava.security.manager -Djava.security.policy=$CATALINA_BASE/conf/catalina.policy"
fi

# juli LogManager disabled if running under libgcj (see bug #395167)
gcj=no
"$JAVA_HOME/bin/java" -version 2>&1 | grep -q "^gij (GNU libgcj)" && gcj=yes
if [ "$gcj" != "yes" ]; then
  JAVA_OPTS="$JAVA_OPTS -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=$CATALINA_BASE/conf/logging.properties"
fi

# Define other required variables
CATALINA_PID="/var/run/$NAME-$INSTANCE_NAME.pid"
LOGFILE="$CATALINA_BASE/logs/catalina.out"
BOOTSTRAP_CLASS=org.apache.catalina.startup.Bootstrap
JSVC_CLASSPATH="/usr/share/java/commons-daemon.jar:$CATALINA_HOME/bin/bootstrap.jar"

# Look for Java Secure Sockets Extension (JSSE) JARs
if [ -z "${JSSE_HOME}" -a -r "${JAVA_HOME}/jre/lib/jsse.jar" ]; then
    JSSE_HOME="${JAVA_HOME}/jre/"
fi
export JSSE_HOME

case "$1" in
  start)
	if [ -z "$JAVA_HOME" ]; then
		log_failure_msg "no JDK found - please set JAVA_HOME"
		exit 1
	fi

	if [ ! -d "$CATALINA_BASE/conf" ]; then
		log_failure_msg "invalid CATALINA_BASE specified"
		exit 1
	fi

	log_daemon_msg "Starting $DESC" "$NAME"
	if start-stop-daemon --test --start --pidfile "$CATALINA_PID" 
		--user $TOMCAT5_USER --startas "$JAVA_HOME/bin/java" 
		>/dev/null; then

		# Clean up and set permissions on required files
		rm -rf "$CATALINA_BASE"/temp/*
		chown --dereference "$TOMCAT5_USER" "$CATALINA_BASE/conf" 
			"$CATALINA_BASE/conf/tomcat-users.xml" 
			"$CATALINA_BASE/logs" "$CATALINA_BASE/temp" 
			"$CATALINA_BASE/webapps" "$CATALINA_BASE/work" 
			"$CATALINA_BASE/logs/catalina.out" || true

		$DAEMON -user "$TOMCAT5_USER" -cp "$JSVC_CLASSPATH" 
		    -outfile "$LOGFILE"  -errfile '&1' 
		    -pidfile "$CATALINA_PID" $JAVA_OPTS "$BOOTSTRAP_CLASS"
	else
	        log_progress_msg "(already running)"
	fi
	log_end_msg 0
	;;
  stop)
	log_daemon_msg "Stopping $DESC" "$NAME"
        if start-stop-daemon --test --start --pidfile "$CATALINA_PID" 
		--user "$TOMCAT5_USER" --startas "$JAVA_HOME/bin/java" 
		>/dev/null; then
		log_progress_msg "(not running)"
	else
		$DAEMON -cp "$JSVC_CLASSPATH" -pidfile "$CATALINA_PID" 
		     -stop "$BOOTSTRAP_CLASS"
	fi
	log_end_msg 0
	;;
   status)
        if start-stop-daemon --test --start --pidfile "$CATALINA_PID" 
		--user $TOMCAT5_USER --startas "$JAVA_HOME/bin/java" 
		>/dev/null; then

		if [ -f "$CATALINA_PID" ]; then
		    log_success_msg "$DESC is not running, but pid file exists."
		    exit 1
		else
		    log_success_msg "$DESC is not running."
		    exit 3
		fi
	else
		log_success_msg "$DESC is running with pid `cat $CATALINA_PID`"
		exit 0
	fi
        ;;
  restart|force-reload)
        if start-stop-daemon --test --stop --pidfile "$CATALINA_PID" 
		--user $TOMCAT5_USER --startas "$JAVA_HOME/bin/java" 
		>/dev/null; then
		$0 stop
		sleep 1
	fi
	$0 start
	;;
  try-restart)
        if start-stop-daemon --test --start --pidfile "$CATALINA_PID" 
		--user $TOMCAT5_USER --startas "$JAVA_HOME/bin/java" 
		>/dev/null; then
		$0 start
	fi
        ;;
  *)
	log_success_msg "Usage: $0 {start|stop|restart|try-restart|force-reload|status}"
	exit 1
	;;
esac

exit 0

My latest Ubuntu setup

I’ll keep editing as I go:

Install LaunchBox

bash$ sudo apt-get install gnome-launch-box
bash$ gconf-editor

Open apps->gnome-launch-box->bindings and set activate to

<Super>l

This allows gnome-launch-box to be activated using the Windows-L shortcut.

Next setup gnome-launch-box to start automatically. Open System->Preferences->Sessions and create a new Startup Program. I use this command line:

/usr/bin/gnome-launch-box -t -n

Install Compiz/Beryl merge latest

In progress.

Ubuntu VPN issues

I figured out how to get Ubuntu to correctly connect to a Windows style PPTP VPN. Apparently there are a number of large issues with NetworkManager that make it all but unusable for VPN. The fix is enormously simple, so I’m extremely confused as to why on earth they haven’t patched it yet. Okay, so here’s what I have done:

1. Create a shell script that you will run after you connect to the VPN via NetworkManager applet. The contents are like this:

#!/bin/bash

echo "search your-domain-name.com" > /etc/resolv.conf
echo "nameserver ip-address-of-your-companies-dns-server" >> /etc/resolv.conf
sudo route add default gw 10.10.30.1

2. Connect to the VPN using NetworkManager

3. Run this script as root:

bash$ sudo ./vpn

What this does is first it updates my resolv.conf so that I can find the DNS server that my company provides and also setups up my search domain for easy access to stuff like http://wiki. Next it adds a new route to the kernel IP routing table. The issue with NetworkManager is that it not only completely clears out resolv.conf (eeck!) but it also doesn’t always setup a route to get to the VPN network. In my script I just setup a simple default route that will direct all traffic to my VPNs gateway. This procedure assumes that NetworkManager is setting up a host route to the VPN gateway on the ppp network device. If this doesn’t happen, well your routing table could get all screwed up, but I think for now NetworkManager isn’t that stupid (however it might be).

Good luck and happy VPNing.

Vista morning ponderings

I’ve tried to post this message 3 times this morning and three times I’ve lost the post. First it was IE7 sucking and then it was my hosting company sucking worse and finally I’m starting over again. This time I’m keeping it short. Here’s my morning thoughts.

  • VIsta looks freakin’ amazing!
  • Looks aren’t everything, but help make life nicer
    • I can’t run Beryl or Compiz on my desktop because of ATI support – sucks – so linux still looks horrible even with the best theme available and hours of tweaking.
    • Even with Beryl or Compiz, X windows sucks because of the networking code at the bottom. OpenGL, FB, DRM, DRI, whatever should really be at the bottom of X and the rest of it can go. It sucks a lot anyways.
  • Vista sucks for developing. No shell with completion, no virtual desktops, you have to install everything by hand and it sucks to find good free software.
  • Linux has so much good support now, that even though it looks like butt, it is still better for developers. Any developer that say they are more productive on Windows has been smokin’ some serious crack.

External monitor (VGA output) on my Dell 700m with Ubuntu

I can’t recall if I already posted this, but I did manage to get the VGA output working in order to use a external monitor or projector for my Dell 700m on Ubuntu. For this laptop the Fn-F8 key doesn’t work for whatever reason. Probably some BIOS or motherboard firmware problem that requires a driver in windows. Anyways, trying to use i810switch or i855crt both produce horrible results. Therefore, the only way to get this to work is to enable a dual head configuration.

In order to do this, just add these two Option lines to your xorg.conf file:

Section "Monitor"
        Identifier      "Generic Monitor"
        Option          "DPMS"
        HorizSync       28-64
        VertRefresh     43-60
        Option          "MonitorLayout" "CRT,LFP"
        Option          "Clone" "true"
EndSection

Update
Found this great page after much searching and most of the configurations work well.
http://www.celifornia.com/documents/dell700m.html#Video-out

Setting a root password in Ubuntu

This is a response to a comment about Ubuntu that is on this blog entry. The person was having trouble setting a root password. This is really simple, but not really needed for almost everything since Ubuntu gives your user sudo privileges, which allow you to do anything root can. You simple prefix the command with ‘sudo’ and type in YOUR password. However, if you absolutely MUST log in as root, just do this to set a root password:

brian@brian-lt:~$ sudo passwd root
Password:
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

This sets the root password and you can now log in as root like this:

su -