It’s the end of the year, and my office is empty, so seems like as good a time as any to reflect on the past year of FOSS developments.

Noteworthy Happenings

Ubuntu 7.04 and 7.10 were released: While perhaps not as revolutionary as the original Ubuntu releases, these releases represented incremental improvements, and solid steps forwards for Ubuntu as a viable desktop alternative. Dell also started selling Ubuntu pre-installed on a limited line of laptops, though the uptake, as far as I understand it, has been pretty small. The newest Ubuntu release also brough 3D desktop effects to the masses, though not without its problems.
Several media-covered Linux-based devices released: This includes the Asus eee PC as well as the OLPC, and the Wallmart gPC. All these are instances of Linux based systems finding niche markets to fill.
ATI’s major shift in attitude: After AMD bought them, they became much more agreeable to opening specs for their drivers.
That’s far from a complete list, just what comes to mind.

Commentary

Money still makes the world go ’round
As far as I can tell, nothing happened this year to show otherwise. The most successful projects were those with significant commercial interest and financial backing supporting them. The canonical example is Firefox. The Google-Mozilla partnership is a key example of how development of open source technologies can be developed for money but be distributed for free. Another example in the same domain is WebKit. With Apple, Nokia, and now Adobe and Trolltech all riding no the same code base, there is a lot of money interest in advancing this piece of code for the benefit of all.
As far as a desktop operating system, there are few instances where desktop Linux distributions have been connected to strong revenue streams, at least not directly (I don’t quite count Ubuntu, even though it’s backed by Shuttleworth’s money. I’ll count it when it’s financially self-sustaining). There is a lot of peripheral work happening that makes one hopeful for the future however. Companies like Intel have realized that being tied to Microsoft is not necessarily a good thing. Support for the latest CPU features seems to always happen in Linux first, and I’m sure Intel’s contributions have a big part to do with that. I’ve been told also that many Intel engineers are working on core ACPI support. Advancing the Linux platform features for desktop and laptop scenarios can only benefit Intel, providing that they can do it cost-effectively.
On the other hand, devices like the gPC, eee PC, and OLPC may not have been possible without Linux. Sure, there are devices like the UMPC’s, but seriously, who’s going to pay a $1000+ for these devices? Linux on the small form factor allowed super-cheap devices with lots of functionality for the price. Better yet, these devices actually sell, which provides financial incentive for developers of the software running on them to advance the platform.
If you look at it in the context of the lessons described in the Innovator’s Dilemma, small form factor machines might just be the “new use case” that Linux needs to really go main stream.
FOSS does provide value and facilitates innovation
Just not in the way that people traditionally think. FOSS software ‘products’ usually don’t succeed in the traditional sense of shrink-wrapped or consumer-purchased software. By themselves, they don’t provide turn-key solutions for a user’s problems.
Instead, FOSS is really good at producing good quality commodity software components. These are reasonably solid implementations of commodity software ideas. For example, a good XML library, or a good mail sending mechanism, etc. While these components aren’t useful to end users directly, their free (as in speech) availability to commercial companies greatly improves the companies’ ability to produce new value by re-using them.
WebKit is a great example.. it allows Apple to develop a slick browser, while all of the core HTML code is essentially commodity (though they have made their share of improvements). Adobe is able to develop a radical idea like the Adobe Integrated Runtime because it can just re-use webkit, and not have to develop a HTML rendering library all on their own.
VMware is another great example of this. We use open source software components all over the place, whether it be GTK, libxml, or a bunch of random Linux/Unix tools. A huge part of our internal developer environment is based on free tools, and we have a fairly high number of Linux fans as developers. We also try to work with the community where it makes sense, whether it be submitting patches to GTKmm, or trying to get a fair paravirtualization framework into the kernel. If you ask many of our developers, they will say without a doubt that our ESX product (the main money maker) would not have been possible if Linux didn’t exist.
A great number of web companies also work the same way, whether it be Facebook, or Google, or Yet Another Web 2.0 startup. The free availability of Linux, MySQL, Apache, and various server-side programming environments really lowers the barrier for new companies who want to build upon existing technologies.
You could even look at distributions as an example of this. They provide a lot of value (an entire OS package with a lot of built in features) for free! Surely this would not be possible were it not for the myriad of open source projects that are out there producing good code.
So while it will probably remain for at least the medium-term that “products” that land in the user’s hands will be produced by “companies”, FOSS has already provided immense value to companies who know how to leverage it to further their own products.
FOSS keeps companies (and prices) honest
That’s all great for companies, but what about Joe consumer?
I just talked about how FOSS technologies allows companies to create high-value products at less cost by re-using commodity technologies. There’s a corollary effect that benefits the consumer as well.
Lower production cost for a product should result in lower prices. But even then, a company may choose to try to sell a product for a price higher than what you’re willing to pay for. Especially if the market is willing to pay it (perhaps it’s also ironic that I mention this since VMware is accused of doing the same thing). But here’s the deal.. if the product is partially based on freely available commodity technologies, then a company can only really charge money for the parts of the product which are not free. If a commercial product is mostly built with freely available parts, then it should be close to free.
With time, as more and more consumers and corporations use open source products, the feature set available in FOSS products tends to increase. This also means that the functionality using just baseline open source software tends to increase. Assuming the cost of this baseline functionality is virtually free, companies that provide non-free solutions will be immediately measured against the free solution. This essentially limits companies from setting prices on their products too high. If they do, it creates a gap for other companies to come in, leverage more free software, and produce a “good enough” product for a lower price.
There are so many examples of this effect that I’ll just talk about a few that I know about.
Home NAS server based on Linux: For the power user, these storage devices (available from numerous vendors) provide a huge amount of functionality. The DLink DNS-323 not only does basic file service, but has an FTP server, an ITunes server, and a UPnP server, all based on open source code. Further more, with a little work, you can install all types of other servers. All for less than $200. Windows Home Server debuted at less than $200, and it’s still unclear that they will even succeed at this price given all the alternatives.
Linksys WRT54G router: This is another famous one. Firmware’s based on Open source code now give this less than $50 router the features of a high-end enterprise product.
$3 Windows XP in China: Microsoft has to sell XP in china for $3 a piece to get anyone to buy it. This isn’t entirely due to Linux, since the main problem is the pirated copies of Windows all over the place, but the availability of the Linux alternative clearly has an effect here.
Vista Ultimate for $400: If you look at XP, and see it was several hundred dollars for a full version, and then look at Vista, look at the new features, and do the math, you might be able to somehow justify that Vista Ultimate can be worth $400 dollars. But these days, instead, you look at Ubuntu. You say, what do I get in Vista that I can’t get in Ubuntu. Is that really worth $400?
Apple computers: I claim that Apple machines would be way more expensive, and way less featureful if it weren’t for open source software. They use OSS in tons of places to add features to their platform.. whether it be gcc, or WebKit for Safari, sqlite for spotlight, or the BSD stack. It’s clear that Apple is winning a lot of converts from the Unix/Linux world because of their inclusion of all the basic Unix features. And at what cost? It’s probably far cheaper for them to maintain their Unix tools than it is to develop all their core closed-source code. And yet, their small investment has won them a lot of new users and a lot more hardware sales.
Windows is here to stay (especially on the desktop)
For a while at least. I know I just wrote all about how FOSS is great and its changing the world. But here’s the thing, Windows has lots of aspects that aren’t so easily commoditized. Therein lies it’s real strength. Some of these aspects include:
User Community: Open source projects like to talk about their big communities. They should, as community is fundamental to their success. However, look a the size of the community around Windows. It is astronomically large compared to Linux or even the Mac.
A larger community leads to more diversity. The diversity of software on windows (at least in areas where MS allows it) is amazing. The stuff you can find for free is also amazing. Have some problem with XP? Google is much more likely to find the answer for you than some random problem with some random Linux distro.
Great Commercial Applications: Nobody is going to contest the fact that there are still great commercial apps available on Windows for which no FOSS equivalent exists. Photoshop (and the entire Adobe/Macromedia suite), Office, Games, Sound Editing and production apps.
Porting these applications to Linux or even OSX is non-trivial. Companies are unlikely to invest in these efforts without a crystal-clear business case.
Also, since the Windows platform represents so many users, many of the best open source applications are available on Windows. Firefox, Thunderbird, Vim, Emacs, Pidgin, MySql, Apache, Gimp, Cygwin, the list goes on. The more such programs that are available on Windows, the less the advantage of running an open source OS to run these apps. (Note however, that if these apps become the best-of-breed apps on the platform, then running Windows becomes less important)
Infrastructure and Eco-system: This is related to community, but slightly different. There’s a machine like aspect to the Windows software and hardware world. And many a company’s financial livelihood depends on this machine, for better or for worse. There are many aspects to this machine, but some of the more important ones are:

• The WHQL process
• Stable driver frameworks and application API’s
• Strong documentation
• Backwards Compatibility
• Best-of-breed Application development environments (VS, IIS, .NET)

To put it another way, all these “features” allow other companies to make money off of Windows. If a software company wants to produce a new desktop software product, not only will it choose to support Windows first because it has the biggest market share, it also has the best environment for deploying a new application. And this is no coincidence. MS has spent tons of money to make this work well (which also leads to the mystery of why they broke it so much with Vista), and Linux (especially on the desktop) has got a long way to go to figure to match Windows in these aspects.
That’s not to say that distributing third party apps on Linux is impossible. Google, VMware, and a handful of other companies have shown that it can be done. The problem really is that it’s still really hard. Hard enough to make it unattractive. Hard enough to make Linux not the first choice.
What’s even more unclear is whether the FOSS world will be able to solve this problem. FOSS projects tend to have a “you become OSS too, or screw off” attitude. Such is the case with kernel drivers, and in practice, with other components as well. Purely FOSS projects are not good at maintaining backwards compatibility, stable API’s (both source-level and binary), and documentation. This is not usually because the project members are incapable, rather, it is a dynamic of the development model. These problems are really, hard, require lots of work, and the community hasn’t figured out how to solve them without paying people to do the work.
There are exceptions, of course, like the Linux syscall interface. Or the basic X protocol. Even glibc, or the python language. But these things are all really basic. Microsoft extends this stability, compatibility, and documentation much more broadly across all their products and technologies.
This also applies to hardware itself. If you’re a hardware company, and you want to get your new device out there as quickly as possible, which platform are you going to choose? If you go with Windows, you get a stable driver interface that you know will work for at least a few years, as well as lots of potential users. You can also distribute the driver with the hardware, and update it on your own schedule. If you go with Mac, you can still write a driver, but who knows if Apple will break the driver interface. If you go with Linux, you can write the code, even submit it to the kernel, but who knows when the next distro release will pick it up? What if you want to keep some of the code closed? play nasty binary blob tricks like the intel driver or the nvidia driver?
Anyways, the point is there are still lots of aspects of Linux that make it less suitable for the desktop. It’s not surprising really. It’s Microsoft’s turf, and they’ve mostly succeeded at it for the last 15 years.
Evolving the Open Source development model
If open source is to succeed more and more, the development model needs to evolve. Companies continually evolve their development processes as the competition environment changes.
One of FOSS fundamental strengths is it’s ability to connect developers to their users in a way that cuts through a bunch of bullshit. It’s not just about the source code. It’s about the open development model — which means open discussion forums, open bug tracking systems, open decision making systems. This strength also lets hobby developers, user-developers contribute back to the project relatively easily. The easier this is, the better for everyone.
Here are a few areas where I think there can be improvement:
Reporting of crashes: Right now, on most Linux distros, users have to install weird debug packages to get any kind of useful backtrace info on a crash. That’s a big burden. A windows-like PDB (separate debugging info file) could be a huge benefit here.
Improve the communication chain from the user to the developer: I think Launchpad.net is fantastic in this regard. There is not really that big of a reason why every OSS project should have it’s on bugtracking system. There is, on the other hand, a huge benefit of consolidating various bug tracking system into one that is cross-linked and comprehensive. The easier it is for a motivated user to contribute problem reports back to a developer, the better.
Make it easier to hack on things: It’s surprisingly easy to start hacking on windows, even for a component that is in the middle of a complex dependency chain ( a dll for example). Download the source code, build it, replace the dll, and go. I don’t think it’s so easy on Linux. Complex package managers are partially to blame. How do I hack on just cairo and pango, for example? I have to check out their trees, then know enough about my distro’s layout to rename files and make symlinks and do all kinds of hacky stuff just to make it use my local versions.
Another way to put this is that for Linux development, the developer’s environment can look very different from the user’s. A different kernel, different libraries, different compiler. All are possibilities. The more the development environment can be standardized, the more efficient the development environment can be.
A wild and crazy idea maybe is a more unified distro and upstream development system. The distro side doesn’t take tarball drops from upstream, but rather just integrates their source control directly. The distro builds its packages by synching to known revisions in the upstream tree (maybe also maintaining patches locally), but then also gives the power for the end user to do the same. So as a user, I could check out the same sources that the distro is using, rebuild the same piece of my installed system, and also get all the benefits of a SCM system for my local changes (which also makes it easy for me to send patches).
I still use Windows
Lastly, I’ll just note my personal reasons for still using Windows (XP 32), so that I can re-evaluate next year.
Applications: Some apps I want are only on Windows; some commercial (Lightroom), some free (FABDVD Decryptor). The OSS apps I want are mostly on Windows too (Firefox, Gimp, Python). Notable exception is Rhythmbox, but I can live with iTunes. I also like gnome-terminal over PuTTy, but I can also live with the latter.
Other features: Font rendering is still better on Windows. Though I don’t mind the Mac’s rendering so much either. Color management. Solid support for input methods. Webcam/skype. Blackberry. Occasional games.
I could see moving back to a desktop form factor Mac in the next year though, especially now that VMware Fusion is available.
Finally
Longest blog post evar. I wrote it mostly for myself to solidify a lot of the stuff I’ve been thinking about recently. Hopefully someone else will at least find part of it interesting.