What is Linux?

For those of you who don't know already, Linux is an operating system (like Windows or Mac OS-X), that is built by the public and it's free.* If you know how to use Windows or OS-X, You can install Linux and be using it with no training in about 30 minutes. It's made entirely of programs donated to the public good and many distributions come with office applications, games, and web tools already installed.

There are hundreds of companies, causes, and individuals who package Linux into a "distribution" consisting of the Linux kernel (the guts of the operating system) and a set of software designed for a specific purpose. For example:

- Easy-to use desktop: Ubuntu
- Corporate web-server: RedHat
- Free (GNU) software purity: Debian or gNewSense
- Minimal hardware: Puppy
- Run from a CD: PCLinuxOS
- Repair your computer after a crash: SystemRescueCD
- To be just like another distribution: CentOS (like RedHat)
- To protect networks: IPCop
- Breaking into other computers: PHLAK (Professional Hackers Linux Assault Kit)
- Christianity, bible study, and parental control: Ubuntu Christian
- Online learning environment, suitable for classroom use: Edubuntu

That's a random sampling taken mostly from Distrowatch. There is a lot of crossover between the goals and included packages of various distributions, but this gives you a sense of what's out there. Because the source code for the entire operating system and most of the software generally distributed with it is free and/or open-source, a programmer can compile their own version and alter it to work with custom hardware such as a new phone, or a robot.

Interestingly, no-one told anyone to create Linux. In 1989, Richard Stallman, a visionary programmer who could be called the world's first software activist, created a new kind of software license - the GNU Public License, GPL, or "copyleft. The GPL guarantees everyone's right to copy, share, modify, and (re-)distribute software. In 1991, A programmer named Linus Torvalds started working on a version of Unix that he could use on his own computer. He published it under the GPL and released it on the web and soon many other programmers (or "hackers") contributed changes and improvements. The work of untold numbers of programmers, writers, graphic artists, testers, and other largely self-selected volunteers, under the GPL and other "open source" licenses, has created Linux in all its flavors - and it continues to evolve.

Who uses Linux? We all do. At least half of all web sites run on Linux servers. Mobile phones such as the Android run Linux. 443 of the top 500 supercomputers run Linux. Increasingly, personal desktop computers run Linux. If you aren't familiar with Linux, I encourage you to get yourself a LiveCD of any distribution and give it a try. Ubuntu is currently the most popular for non-techies.

For your entertainment, here's a brief history of Linux through advertising (and one spoof):

Mac Spoof: Upgrading

I'm a Linux

The Heist

* Thanks to DigitalMan for helping me come up with a one-line explanation of Linux.

Settling in to Ubuntu Linux

I'm surprised at how much I like Ubuntu Linux better than Windows! The UI is basically the same, only everything is faster. Especially when you do anything to a file (write to it, read from it, move it, or delete it), Linux turns hours into Minutes. When you consider that doing anything useful on a computer generally involves changing a file, that says a lot!

Printing to a network printer was actually easier on Linux. Our vendor-supplied proprietary Windows driver hangs the printer every second or third time you try to print a PDF! I've got my complaints (a few sound limitations and a few software packages that I still have to run in a Windows VM). But whenever I have to fix someone's Windows computer I pity them. Even Windows runs better on Linux (in a VM). That way you can keep working while you reboot, reboot, reboot...

As I'm writing this, I can hear my wife downstairs using Windows... "Network Problems!?! I can't have network problems right now! Operation failed???" I feel sorry for her, I've been there myself, but I have to laugh at the background to this blog post.

I encouraged some people I know who are absolute beginners with computers to buy Macs. But now I'm not so sure. Ubuntu Linux is actually easier to navigate in some ways than a Mac.

Update 2010-04-01: I spent 2 hours trying to copy some pictures from a camera to the hard drive on a Mac and it was awful. In Ubuntu, your camera gets mounted as a drive and you can drag and drop the pictures wherever you want them.

The only drawback to running Linux is that some programs only run on Windows. Screen sharing programs (except for TeamViewer) all require Windows or Mac. The latest version of Microsoft Office only runs on Windows, Photoshop only runs on Windows/Mac, and Proprietary VPNs only run on Windows. Most people don't need those things, but I do, so I have to run Windows in a Virtual Machine. But I only have my VM running about 25% of the time.

Windows in a Virtual Machine

My Windows XP SP3 CD came in the mail yesterday. This time around I did a more-or-less default install in VirtualBox with 20GB expanding drive, 1.2GB memory, 24M video memory, and audio disabled. Instead of Symantec Antivirus, I tried ESET NOD32. I also disabled Adobe Drive this time because I got some weird messages. The install and all setup took about 8 hours, much of which was waiting, which is much more reasonable than the 3 days my previous attempt took.

The new image uses only about 6GB of disk space instead of 15, presumably because it doesn't contain any junk from my old hard drive. It is immeasurably faster than my 140GB image of my previous hard drive, half-of which was unreadable. Not sure how much was VM settings, Symantec, or other junk, but I think everything helped.

I deleted the old virtual hard drive image. It took hours because I had a dozen huge "snapshots" I had taken in VirtualBox, each of which took 1-20 minutes to delete. Before I started, my new 1TB drive was 60% full. Now it's only 7% full! What a disaster that was.

I currently use Windows in VirtualBox a few times a week for the following, but I work mostly in Linux:

Photoshop CS4

I use this a lot and it requires 1024x768 resolution for some of the dialogs to work properly (to show the Save button). Takes a lot of resources, but as much as I love GIMP, it's not a substitute for Photoshop for professional work.

A couple of proprietary VPNs

The nice thing about running these in a virtual machine is that it lets me check email and access the web from my primary OS while the VPN is connected.

Citrix GoToMeeting

Works great

Remote Desktop Connection

Necessary for work.

Internet Explorer

Necessary for work (for testing)

Microsoft Office

Open Office is a fantastic substitute. It changes the layout only slightly I think because the available fonts are different on Linux. I still need the real thing for presentations and testing for work.

Firefox HTML Validator plug-in

I really wish there was a Linux version of this!

Hopefully, over time, things like GoToMeeting and the Firefox HTML Validator will work on Linux. I'm still looking for a secure backup strategy for my entire drive.

Alternative for spacer.gif in cross-browser standards-compliant html

Instead of spacer.gif I use:

CSS:
.spacer2px{font-size:2px;line-height:2px;}

HTML:
<div class="spacer2px">&#160;</div>

Works with <td> as well. We have spacer1px, spacer4px, spacer6px, etc. as needed. Works the same on all browsers, so long as each HTML page starts with the following, which keeps IE from entering quirks mode and makes it as standards compliant as it gets (requires no blank lines until after the opening HTML tag):

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">

    ... headers ...

<body>
    <div id="page">

    ... content goes here ...

    </div>
</body>

We currently have two lines of browser-specific code in the entire application, and they are both in the CSS file:

/* This sets defaults for all elements and helps overcome browser differences */
* {
 margin: 0px;
 padding: 0px;
 box-sizing: border-box;
 /* -moz-box-sizing: border-box; Firefox used to need this */
}

/* Add space for scrollbar on IE 6 so it doesn't obscure the page.
This uses the Commented Backslash Hack so only IE can see this: \*/
#page { margin-right: 16px; }
/* End Hack */

Well, that's my recipe that I've been using for 5-6 years now. Tested on IE 6, 7, 8, 9, Firefox 2, 3, Safari 3, 4, Chrome, Opera 9, Epiphany, Konqueror and on Windows XP, Windows 7, Mac OS-X, the iPhone, and Linux (Ubuntu/Xubuntu).

Special thanks to quirksmode.org for probably being the best resource for these kind of details. The O'Reilly CSS: The Definitive Guide and HTML & XHTML: The Definitive Guide books are great for the basic rules.

Moving from Windows XP to Ubuntu Linux this "Weekend"

Sat, Sep 19, 2009

6 PM

Bought a Western Digital 1TB Caviar (Green) drive and 4GB memory. I wanted to keep the old drive untouched in case there were issues with the install.

10:02 PM

new hard drive installed, Ubuntu install begun. I almost went with Xubuntu, but the display got messed up in VirtualBox when I applied patches. I think Ubuntu is going to be more reliable/compatible and that's even more important to me than performance.

11:53 PM

sent first email from new system using Google Chrome! I spent one of those 2 hours googling and thinking about partitioning before deciding to let the installer do what it wanted. Finally I clicked "OK" about 5 times and in 20 minutes it was done. Ran the updater, rebooted, added and removed some programs; another 20 minutes total. Really, it couldn't have been easier.

Sun, Sep 20, 2009

10:13 AM

Massive frustration. So far, it looks like installing Windows is going to be the hardest part of installing Linux. All I can find is an XP Home edition upgrade CD from 2004.

Microsoft doesn't sell XP any more. The Windows 7 beta download program has ended and you can't buy it until October 22nd (or thereabouts). What you can buy (for $300) is Windows Vista Professional Non-Upgrade with a "free" download of Windows 7 when it's released so you don't have to miss out on upgrading, patching, and being the first to find bugs. Even if that sounded good, I can't buy it now because the Windows store doesn't open until noon. Tech support isn't open either.

I decided to follow the instructions here:
http://www.virtualbox.org/wiki/Migrate_Windows

It looked good, but it's not the easiest, or necessarily the best way. Read on...

4:01 PM

Booting a 150GB image of my old Windows hard drive. Woohoo!

4:06 PM

Windows asks for a key code. I copy it off the sticker on the side of my computer. It doesn't take it. I call in and punch the numbers over the pone. No luck.

It turns out that it wouldn't let me re-register because I have an OEM version that was pre-installed on the drive and the virtual machine had different "hardware" then it was expecting. Maybe I could have faked the bios, the MAC address, the hardware, and I'm not sure what else in order to satisfy XP that I wasn't trying to install it on a different PC. I'm installing it on the same PC, so it's legal, but it's in a virtual machine, so I couldn't be sure the hardware would ever match. At this point I gave up and installed my XP Home Upgrade from 2004 over the XP Pro image because the upgrade has to be installed over an existing version of Windows (apparently it doesn't have to be a validated version).

The install kept the files on the C:\ drive, but it overwrote the My Documents folder and left my old home directory inaccessible. Using CACLs showed it to be owned by WINDOWS NT/my-old-id. I was able to make it readible, but couldn't delete anything. I began deleting my now useless applications but it was taking hours. The solution? Boot the VM into Ubuntu and delete them from Nautilus. It took minutes. How can Microsoft charge $300 for an operating system when a free one has a 100 times faster file system?

8:01 PM

Windows XP home is installed and works. Beginning "Windows Update."

8:29 PM

Installed vpnc, which connects to a Cisco VPN from the Linux command line. Nice! XP is still applying updates.

9:10PM

Windows updates, updates, updates. 50 at once this time.

Mon, Sep 21, 2009

8:24 AM

Windows XP home was finally upgraded to SP3 around midnight last night after many reboots, and lots of clicking and waiting. Installed VirtualBox Guest Additions this morning which was quite easy.

Noonish

Had a work call that wanted to use Citrix GoToMeeting. I accidentally laughed out loud at the suggestion (it was a little akward). But within 5 minutes I had the Citrix client installed on Windows and thanks to VirtualBox GuestAdditions I was looking at a full-screen screen-share without any noticeable lag or any issue whatsoever. Nice!

At some point I called Microsoft to find out my options. This took about 4 calls. One of which was a voice recognition system that couldn't recognize a thing I said. later I played "20 questions" with an automated voice response system that put me on hold for 10 minutes. When I finally got a human, the response was, "If you got your operating system with a computer, call that computer's manufacturer, not us." and "We don't sell Windows XP. Want to buy Vista?"

I called Lenovo and they could only offer a CD of the original drive image for $45, but I would have had the same hardware vs. validation issues in VirtualBox.

Tue, Sep 22, 2009

4:35 PM

I'm basically Done. XP upgraded so SP3, Office 2003 installed and upgraded to SP3, Norton Internet Security installed and upgraded, 2 VPNs installed on Windows. I tried to compress the virtual drive image to no avail. They aren't really designed to have their size changed. There's inaccessible or otherwise useless little bits of junk spread out on the drive image (some are unmovable by a defragment) which probably kept it from compressing. At one point, I ran some program on my XP image that was supposed to fill the drive with zeros (to make the image compress better) by making a big enough text file with each character being the null character. It was supposed to delete the file when finished, but presumably your computer would crash about the same time once the drive was full, but it was running so slowly that I killed it and never got to see.

I thought I'd just write a little script that piped the small file of zeros many times into a new file until the machine crashed. But you can't pipe like that in DOS. I downloaded sdelete instead. It took about 5 hours for it to write 140GB of zeros.

While I was waiting for Windows/Office to install, I installed my development environment and got all my Linux apps up and running.

Mon, Oct 5, 2009

11:27 AM

I've installed Photoshop and Cygwin on Windows in the last week. They work fine, but Photoshop really needs the full-screen 1024x768. I might have to get a real graphics card and bigger monitor.

Had to share encrypted WinZip archives with Windows users. Installed PeaZip and it encrypted/decrypted zipped/unzipped WinZip files perfectly. Everything works so much faster than on Windows. There was definitely pain involved in making the switch, but it's also definitely paying off.

Lessons Learned:

1. A new hard drive was a great idea. I could pop the old one out of the machine and be sure I wasn't deleting anything important by accident. I wouldn't try an upgrade like this without a blank drive.


2. A Windows XP Pro SP3 CD would have been a godsend. I still might get one and reinstall to bring the disk image size down to about 10GB instead of almost 150 and to be rid of any lingering junk on the drive.


3. Dropbox and KeePass make managing online accounts across machines really, really easy.


4. I did the right thing. I was really unsure about it. But consider:
   
   1. I was able to be up and running on Linux FOR FREE in 2 hours (would have been 1 if I didn't second-guess the installer for an hour). The XP install took 2 days (would have been 1 if I paid $130 for a disk which Amazon, not Microsoft sells).
   
   
   2. Deleting big files on Windows was incredibly slow. Rebooting into Linux running off a CD (inside the same VirtualBox VM) turned hours into minutes or less.
   
   
   3. Support: If I actually had a support question (not a pre-sales question), I would have had to pay Microsoft $45 to ask it. The Mac Ads are very funny, but they are accurate in this case. I can get on Freenode IRC for free and have virtually any (polite, pre-researched, well worded) question answered in about 10 minutes, and probably learn something else interesting while I'm there.
   
   
   4. My whole system is much, much faster. Startup is in about 20 seconds as opposed to 3 minutes. Suspend (sleep) takes about 2 seconds, waking up takes 5-10. The wake-up on Windows often took 5 minutes!
   
   
   5. In this process I expected to discover that Linux wasn't ready for prime time. Instead I discovered a lot of negative things about Windows and a lot of positive things about Linux!
   
   

Conclusion

I'm installing Photoshop now on Windows so a Windows VM is probably here to stay, but I would be very surprised if I ever switched back.

If Google comes out with a killer Netbook operating system this fall, more people will use Linux. If Windows 7 forces developers to rewrite their apps, and if people have to run XP in a virtual machine to use older applications, they can do that just as well on Linux (or Mac) and have a better, faster, more secure host operating system. All this adds up to Windows having a smaller market share and developers having more reason to come out with a OS-X and/or Linux port of their software. If they use a toolkit like GTK+ (free) instead of Microsoft Visual Studio (very expensive), it shouldn't be such a big deal to run on all three operating systems. At least for me, the future looks a lot more like Linux and less like Microsoft.

Free (GPL) Software Restricts Your Rights!

When I say Free Software in this article, I am referring specifically to software released under the GNU Public License (GPL), a license written by Richard Stallman that has revolutionized the software world.  Revolutionized in the sense of completely and indelibly changing it, but also in the sense of starting a movement of people.  I owe great thanks to RMS, his copyleft, and the ideas he set in motion.  I use emacs and cygwin so much every day that I don't even notice when I dream in emacs.  I have created or contributed to a small number of Free or Open-Source Software (F/OSS) projects, but like most professional programmers, I have mostly just helped myself of the bounty of fabulous F/OSS software that exists.

But for all it's talk of freedom, free as in "Free Speech" and free as in "Free Beer", Free (GPL) Software still restricts your rights as a programmer in one very important way that some other Open Source software does not; it cannot be bundled with "non-free" or proprietary software for distribution because the act of doing so turns the proprietary software into free software.  Most professionals get around this either by violating the terms of the GPL (intentionally or through ignorance) or by adopting the Software as a Service (SaaS) model of hosting web applications themselves so that the software does not need to be distributed in order to be used.  Other strategies for making money with Free Software include selling support and packaging (as RedHat does with Linux) or dual-licensing (as Oracle does with MySQL).

At my current job we pursued the SaaS model as far as it would take us, but in the end we found we had to distribute our product.  We were unwilling to license our company's only asset (its software) under the GPL, so we removed all GPL code instead.  That was a sad day indeed and it has since been a struggle to keep up with the sheer number of Licenses that F/OSS developers use for their products.  For some minor components we have had to choose tools based more on the license than the quality.

The saving grace for us has been the Apache Software Foundation and their license.  For F/OSS developers on the fence, software released under the Apache License v2.0 can be converted or combined with GPL 3.0 software at any time, but the reverse is not true.

The trouble with the GPL is a very specific, technical detail which requires some background in order to explain.  Modern computer programs tend to be composed of a number of proprietary and/or open-source components "linked" together to form a whole.  The definition of "linking" varies somewhat between programming languages and platforms and is somewhat disputed, but it basically means what it sounds like - components are developed separately then combined into a single whole.  The trick the GPL plays on programmers is that merely linking to GPL code (without changing it in any way) allows the GPL to extend to all linked code and effectively puts all of it in the public domain.  The Free Software Foundation will tell you that it's fine to link GPL code to proprietary code and it is, so long as you have the right and the desire to change that code into GPL code.

I have made my living writing software and charging for it since 1992.  I am happy to contribute bug fixes and non-core components of software I have written to the Open Source community, but I want the freedom to distribute critical components under the license of my choosing (generally a proprietary license, granting ownership of the code to my employer).  The Apache License (and other Open Source licenses) allow me to do that.  The GPL does not.

RMS wrote an article in the latest CACM entitled, "Why 'Open Source' Misses the Point of Free Software."  With all due respect to RMS, I have to say I'm glad that it does and I'd like to encourage anyone familiarizing themselves with F/OSS to read several licenses very carefully before choosing one.  The GPL is a classic, but I'm afraid history will show its application is limited to two scenarios:

1. People who believe that proprietary software is fundamentally wrong and should be eliminated.
   
2. Companies who write software which is much more useful when linked as a component of other software than it is as a stand-alone product.  These companies will pursue a strategy like MySQL where they dual-license their products under the GPL so that the world is free to try, examine, change, and distribute their software for non-commercial purposes.  But those who wish to use the software commercially will have to pay for another version of what might be the same code released under a different license.
   

I predict that the GPL will see the most usage by the second camp.  For the rest of us, there are a few hundred other F/OSS licenses to choose from.  A great source for learning about these licenses is the Open Source Initiative.  Their approval process is very instructive for anyone thinking they want to write their own license.  The proliferation of essentially duplicate licenses has run increasingly rampant as more programmers have gotten excited about F/OSS, mostly through the writer's ignorance of an existing license that already does exactly what they want.

Using @Deprecated more liberally

I recently created a class and an interface for managing dates as just a month and year (ignoring the day) and called them YearMonth and YearMonthInterface. I then added comparison methods to YearMonth: greaterThan(YearMonthInterface x), lessThan(YearMonthInterface x), and equals(YearMonthInterface x). Do you see the problem?

equals() overrides Object.equals() and has a special significance for the Java Collections framework. According to the Java SE 6 spec, equals() must be reflexive, symmetric, transitive, and consistent, and x.equals(null) should return false. I had no way of knowing that any class that implemented the YearMonthInterface could satisfy those conditions. In fact, I already had one implementing class that could never satisfy symmetry: myYearMonth.equals(myOtherClass) would often not equal myOtherClass.equals(myYearMonth) because my other class was dependent on another object and could only be equal when its associated objects were also equal.

I decided to keep the YearMonth.equals(Object o) method as overriding Object.equals() so that YearMonth would work correctly with collections, but create a new method, YearMonth.equalTo(YearMonthInterface x), for manual comparisons. My concern was that I never wanted to mistakenly use equals() instead of equalTo(). The solution? Deprecate YearMonth.equals() and provide a comment suggesting the use of equalTo() instead. The compiler will now catch any attempt to use the wrong method and the collections framework can still use the equals() method without problem.

/**

This is deprecated because it does not compare YearMonthInterface objects,

rather it overrides Object's equals method.  It still works in

collections, but there should be no reason to ever call it directly.

Use equalTo() instead which really compares two YearMonthInterfaces as

you would expect, but doesn't need to be symmetric.



@param other another YearMonth (or it will return false).

@return true of the YearMonths are equal.

*/

@Deprecated

@Override

public boolean equals(Object other) {

   // Check address and null first for speedy return

   if (this == other) {

       return true;

   }

   if (other == null) {

        return false;

   }

   if ( !(other instanceof YearMonth) ) {

       return false;

   }

   final YearMonth that = (YearMonth) other;

   return (this.year == that.year) && (this.month == that.month);

}



public boolean equalTo(YearMonthInterface that) {

   // Check address and null first for speedy return

   if (this == that) {

       return true;

   }

   if (that == null) {

        return false;

   }

   return (this.year == that.getYear()) && (this.month == that.getMonth());

}

This technique would not work if I were to release this class to the public, but for a utility class with limited application, it can prevent me and my coworkers from making a simple but costly mistake.

For a full explanation of how to write a good equals() method, see Effective Java Second Edition by Joshua Bloch Chapter 3 (pp 35-36 is specifically about symmetry). I'm about half way through this book and really enjoying it.

New Favorite Find/Replace (bash) shell commands

These commands took a while for me to find and I now use them very, very often. One is to find files in all subdirectories excluding certain directories (in this case, the subversion ".svn" directories). The second expression replaces regular expressions in all the files that match the find in the first expression:

# To grep for things excluding subversion .svn directories

find . -path '*/.svn' -prune -o -type f -print | xargs -e fgrep -I PATTERN



# To replace the same (using sed), only in files that grep matched, only in non .svn files.

find . -path '*/.svn' -prune -o -type f -print | xargs -e fgrep -l -I 'OLD' | xargs sed -i -e 's/OLD/NEW/g'

UPDATE 2009-02-09
I was browsing at Barnes and Noble over the weekend and the O'Reilly Pocket Guide to Grep mentioned an --exclude-dir option. I looked at the man page again and it's there too. I even googled again and found it right away. How did I miss it? That makes things much easier.

# To grep for things excluding subversion .svn directories

fgrep --exclude-dir='.svn' -rI PATTERN *



# To replace the same (using sed), only in files that grep matched, only in non .svn files.

fgrep --exclude-dir='.svn' -rIl 'OLD' * | xargs sed -i -e 's/OLD/NEW/g'

UPDATE 2010-03-31

# To find whole words, using an even briefer syntax for

# chaining the output of grep into the input for sed.

# Taken from a shell script that sets oldWord and

# newWord, but you get the idea:

sed -i -e "s/\<$oldWord\>/$newWord/g" $(fgrep --exclude-dir='.svn' -wrIl "$oldWord" *)