dd if=/dev/random of=/dev/null bs=always

This morning, I came across two Phoronix articles (here and here)  generating some sort of hype with regards to a ZFS port for the Linux kernel, with article headlines as Native ZFS Is Coming To Linux Next Month. My response to this, as I will explain below is: Why?

First of all (as the first article cites) there has always been the ZFS over FUSE project. There was a reason as to why this project was initiated and that is primarily a result of a conflict in licensing. ZFS is CDDL’d while Linux is GPL’d. The one is incompatible with the other. I will not spend any time in the details of this but you can read some of my opinions with the licensing here. Although using any file system over FUSE has its drawbacks; one of which is performance. But you still had ZFS functionality. In a worst case scenario, you can always deploy Solaris, (at one time) OpenSolaris, Nexenta, FreeBSD, among a few other lesser known operating systems using ZFS.

Despite the reasons given in the articles for porting ZFS to the Linux kernel, it becomes almost irrelevant that (a) it will never be integrated into the mainline kernel (unless Oracle were to re-license it, but then it would become incompatible with Solaris 11+ and Solaris 11+ Express), (b) will never obtain any real commercial support, because if it does, then most likely Oracle will step in to destroy everything in its path for infringement of patents and other intellectual property. Also, (c) you will never be able to commercially redistribute the code, binaries and modules in a custom Linux enabled appliance because if Oracle does not get you for IP infringement, other companies such as NetApp will. Who will protect you then? Finally, (d) it will be a long while before this ZFS port is considered stable for Linux. Especially when it will not be in the mainline kernel to receive additional exposure. These reasons alone are enough for a storage company to not seek interest in this port.

At the end of the day, all of this excitement means nothing. Who cares if this ZFS port for the Linux kernel comes out next month? I don’t. Don’t misunderstand me. I love ZFS as I use it all the time on Solaris and OpenSolaris. It is a wonderful file system. This Linux port is also extremely limited in functionality. Today, the GPL’d Btrfs, a competitor to ZFS, is considered “generally stable“. Why not use that instead without all the concerns listed above?

Well, it has been quite a week, last week. Not too long after the OpenSolaris Governing Board issued an ultimatum to Oracle and closely following the unveiling of the Illumos project, Oracle officially announced (in a memo to internal employees) that they will be leaving OpenSolaris behind. To most (including myself) this is not at all surprising. The writing had been on the walls, yet many still felt hopeful. It is disappointing news for more than one reason. OpenSolaris was a solid and great operating system. Not many of my readers may really know too much about the SunOS family but when Sun Microsystems developed a software product, it was always a quality product; that is, stable and extremely well polished.

When I think of Sun Microsystems, I think of a true engineering company. Although being all engineering and little to no marketing may have been one of many paths to its decline and eventual downfall. Either way, Sun gave us NFS, Java, ZFS, DTrace and plenty more. And whenever I stepped behind a Solaris or OpenSolaris system, I always felt comfortable. I would never use a GUI as most seasoned UNIX and Linux administrators don’t. The command line was my main access to the system’s facilities and this is where Sun had it right. Managing volumes and exported shares, boot environments, services and more. They had simplified and polished the ability to manage the system and all of its capabilities. Unfortunately Linux lacks in some of these but who is to say that someone will not take ownership and develop similar tools in the near future.

It is my opinion that this focus to detail and a stable and simplified environment is going to slowly disappear under the direction of Oracle. Again, Sun was an engineering company and nothing more. Oracle has more of a concrete vision and focus; that is, making money. How much will quality of products such as Solaris 11, VirtualBox, OpenOffice.org and especially MySQL (among others) suffer when deadlines are trying to be met and corners must be cut, because a late product won’t bring in revenue. I will admit that this is most likely where I will end my use of Solaris, unless Oracle surprises the world and continues to maintain the type of quality that Sun brought to the operating system.

In what is supposedly a leaked internal memo to Oracle staff, the adoptive parent company of Sun’s OpenSolaris had announced the fate the project’s (and binary release’s) future. That is, they will support it no more outside of CDDL’d package updates for future Solaris (i.e. Solaris 11) releases.

“We will distribute updates to approved CDDL or other open source-licensed code following full releases of our enterprise Solaris operating system. In this manner, new technology innovations will show up in our releases before anywhere else. We will no longer distribute source code for the entirety of the Solaris operating system in real-time while it is developed, on a nightly basis.Anyone who is consuming Solaris code using the CDDL, whether in pieces or as a part of the OpenSolaris source distribution or a derivative thereof, would therefore be able to consume any updates we release at that time, under the terms of the CDDL, LGPL, or whatever license applies.”

Also:

“All of Oracle’s efforts on binary distributions of Solaris technology will be focused on Solaris 11. We will not release any other binary distributions, such as nightly or bi-weekly builds of Solaris binaries, or an OpenSolaris 2010.05 or later distribution. We will determine a simple, cost-effective means of getting enterprise users of prior OpenSolaris binary releases to migrate to S11 Express.”

Reading this is somewhat disappointing as OpenSolaris was truly an excellent operating system.

After spending a week on the forums at opensolaris.org, it would seem that a great number of individuals are waiting for Oracle to make it official: “that OpenSolaris will be no more.” If this ends up being the case then it would be very upsetting as OpenSolaris is a very stable and robust UNIX operating system. During this time I came across this blog posting which may shed some light. It discusses the Oracle directed Solaris Next label svn_140. Solaris Next is the development name given to Solaris 11.

Initially Oracle had publicly announced that it will stay involved in the OpenSolaris community and continue to support it. This was more than a few months ago, and they have been silent since. If they do decide to kill the OpenSolaris project to shift all focus on Solaris Next alone, then I am curious as to if they will kill the OpenSolaris binary distribution project (opensolaris.com) only or the binary plus the community (opensolaris.org). The community can still live on without the binary. They will instead focus on the Solaris Next builds as opposed to OpenSolaris.

I am still unsure about Oracle’s promise on the future of OpenSolaris. Phoronix has just posted an article showing that what was once supposed to be 2010.02 and then 2010.03 to eventually 2010.1H has not been released at the end of the first half of the year. It would be a shame if nothing came out of this. OpenSolaris is such a great platform and ZFS brought some much needed features which helped increase my development and productivity. Should we cross our fingers for a 2010.2H release?

I guess what it boils down to for Oracle is, do they see a profit from this open source project? What does OpenSolaris bring to Oracle and how does it help them solidify their newly acquired Solaris platform? If we ask for my personal opinion, I feel the advantages outweigh the disadvantages. A working example can be seen with Red Hat and the Fedora Project. The more open platform (in this case OpenSolaris) defines the bleeding edge technologies, stabilizes them and is a testing ground for all that will be ported to the official stable release (Solaris). In fact that is how it was under Sun Microsystems.

Although as the months pass by, the more dim OpenSolaris’ future gets. Fortunately enough it is an open source project and can easily be forked into a new community driven distribution. So even in the darkest of hours not all is lost.

Earlier this week I came across this blog posting about data integrity testing on ZFS title: ZFS data integrity tested. It was a few months old from Robin Harris’ blog Storage Bits. I guess the most exciting part was validating Sun Microsystem’s claims to ZFS having the ability to correct data corruption even with error injection to both the disk and memory. ZFS continues to prove its worth on enterprise class systems and applications.

My only frustatrions with ZFS are that cluster support is currently not available, at least until Lustre 3.0 is out, whenever that will be. Another frustration is trying to write an application that will work directly with a zpool. For instance, there is no simple method to send a zpool a generic ioctl() such as DKIOCGGEOM to obtain the size of the volume. In most cases I don’t care about the number of cylinders, heads and sectors. In the end I calculate the total volume block and/or byte count. So those values could be generic and made up.

In the early stages of my discovering this, I posted a simple question on the OpenSolaris Forums:

“As I was navigating through the source code for the ZFS file system I saw that in zvol.c where the ioctls are defined, if a program sends a DKIOCGGEOM or DKIOCDVTOV, an ENOTSUP (Error Not Supported) is returned.

You can view this here: http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/zvol.c

My question is, what is an appropriate method to obtain the zpool’s volume size from a C coded application?”

After posting my question, I immediately went to view the open source to the general zpool/zfs binaries and observe how zpool reported the drive pool’s capacity back into user space. Unfortunately it utilized some cryptic method not as straight forward as sending a simple ioctl() to the desired volume. This was a bit frustrating as it was such an ugly approach to only receive the size of the volume.

I was grateful to have a response confirming my fear of choosing the ugly route; but it also made me realize the true value of open source. What if I simply patched a supported ioctl() definition to return the total accessible “block” count of a zpool? It would be similar to the Linux BLKGETSZ/BLKGETSZ64. This would be the most realistic and proper method; to add a new ioctl() and then modify all storage modules to accommodate it. For instance in the usr/src/uts/common/sys/dkio.h file we would need to define:

And then go back to the zvol.c file and add the extra ioctl() to handle this:

To give a level of consistency across all storage devices, we will need to add the ioctl() definition to the following modules:

Although we do not necessarily have to support it and can instead interpret it as such:

Who knows, one of these days I may get around to patching this myself and if the OpenSolaris community doesn’t accept it I can always make it available on any one of my website. I will most definitely post about it.

As long as I can remember I had been using the Netscape web browser which evolved to Mozilla and now Firefox. I still use Firefox and have grown so comfortable with it that I don’t really desire to move onto anything else. Needless to say, Mozilla’s products are not perfect and there is always room for additional features and what I believe to be necessities in order to function in today’s world of computing.

For instance, I wish there was more of a concentrated effort to bring additional usability and manageability of these same Mozilla products (i.e. Firefox, Thunderbird, to even the SeaMonkey suite) where it can truly compete with Microsoft to even IBM’s Lotus in the professional world.

In this initial example I will choose to focus Lotus Notes and SameTime. This application was built around enterprise productivity. Everything is integrated in such a way where I do not only have the ability to work from my e-mail but through SameTime I can easily connect to the same directory of individuals for instant IM messaging. Google saw an advantage to such an approach and had it integrated into their GMAIL web client. So how difficult would it be to have Thunderbird do the same thing? Now, SeaMonkey may be a better candidate for this integration (as it resembles more of what Netscape used to be) but nobody really knows of its existence.

Another example is the way Microsoft integrates all of its products together to provide a complete solution. If I am using Internet Explorer and I click on something that requires Powerpoint, Excel or something else Microsoft developed, there are no problems in opening up those files and working with them in a new tab of my web browser. It could be beneficial for Mozilla to partner up with Oracle and provide similar transparent integration into their products with Star Office and OpenOffice.org. In fact, what is stopping Red Hat, Canonical, Novell or even Oracle from developing such integration modules in their workstation solutions? I always found it annoying that when I click on a PDF file to open it up in a new tab, it runs outside of the browser instead (unless I were to install a third party developed plug-in most of which are written for Microsoft Windows anyways).

Other things that I would like to see Mozilla work on is better management for corporate environments. Internet Explorer has a centralized managed infrastructure in that it can be controlled using Group Policies across an entire network emphasizing consistency. If you need to change the settings across all web browsers in your network, IE makes that easy and realistic. It is not realistic to e-mail co-worker HOWTOs (especially if they do not have the required permissions) nor is it realistic to visit or remote into every node to address those same changes. This form of management also includes patching/upgrading the browser/e-mail clients. Sometimes this needs to be controlled on a corporate basis as opposed to an individual user.

Mozilla has been doing an excellent job in conquering their fair percentage of market share. All without the billions of dollars dumped into marketing (as seen by their competitors). They are a known household name. A lot of end users know what Firefox is. To move on to the next best thing, I feel they need to start concentrating more on the corporate world.

It would appear that the OpenSolaris project is back on track. This is a bit long overdue. The acquisition of Sun by Oracle left a few projects in questionable states. It was unknown as to whether Oracle would continue supporting these open source projects. OpenSolaris was included in that list.

The great news is, Oracle locked down b134 (which can be obtained at Genunix). It is going to be build for internal testing and we are possibly looking at a May release. I sure do hope they clarify the release name and rename it from 2010.03 to 2010.05.

All of this can be read here. It was Albert Lee who posted the following:

“Those of us feeling left in the dark might be pleased to know that build
134a, the first candidate for the next stable release of OpenSolaris, has
been tagged in Oracle’s release branch (in project jargon: “snv_134a, the
first respin of 134, closed earlier this week”). A packaged build should be
available for internal QA soon, but even if it passes, it will be while
some time before the release can be published to the external repo.“

Two days ago, I ran through a long needed image update to the OpenSolaris 2010.03 preview. I was updating through the pkg update manager from build 129 to build 134. So when I say, it was much needed, I wasn’t kidding. Anyway, after over 1 GBytes of updates was completed, a new boot environment (BE) was created with the native ZFS snapshot feature and I shut down the PC for the night.

The next day I turned the PC on into the latest boot environment to find that my gnome-terminal was giving me problems. The obvious symptom was that certain characters were not being echoed and their was misalignment with every entry and output displayed within the terminal.

After some research I came across OpenSolaris bug 12380: image-update loses /dev/ptmx from /etc/minor_perm. The fix (workaround) was simple: boot into the previous boot environment, mount the newest boot environment and clone the /etc/minor_perm from the one to the other. The steps are as follows:

And the problem was fixed. It was quick and easy thanks to ZFS.

This entry is a continuation of one published in May of 2009. In fact it is relating to a comment made earlier today which I responded to in brief words. I am now taking the time to offer my viewpoint on the whole ZFS licensing under the CDDL and the reasoning for it.

It wasn’t until I started working with the OpenSolaris kernel and by working I mean, modifying code and going through the build process that I finally realized why OpenSolaris was licensed under the Common Development and Distribution License (CDDL). A lot of other people and companies have claim to code used within Solaris. That includes copyrighted code to which Sun does not have the authority to publish in an open source license. This is why they needed to work with a weak copyleft license such as the Mozilla Public License and modify it to their expectations. The CDDL was eventually approved by the Open Source Initiative (OSI) as a valid open source license and Sun Microsystems was then able to release code under its limitations.

Now before I continue I wish to describe 3 different open source licensing models: (1) the strong copyleft license, (2) the weak copyleft license and (3) the non-copyleft license.

The strong copyleft license is a project based license in which it requires that any derived code from the original project must remain under the original license. This method of licensing makes it nearly impossible to link with code under a non-strong copyleft license. As a result of this approach, strong copyleft licenses are often referred to as viral licenses. The most popular of these licenses is the General Public License (GPL) with 3 available versions. The Linux kernel is licensed under this and its success and growth can be attributed to it.

The weak copyleft license is similar to the strong copyleft license except that it is file-based instead of project based. This means that if there are any modifications to a file, the original license must apply; but that file can be combined in a project with code under a different license. This method makes the type of licensing non-viral. The CDDL and the MPL are categorized as weak copyleft licenses.

The third type is the non-copyleft license which offers no requirement for derived works to stay under the original license. In fact, there is also no requirement for derived code to be released under any open source license. This makes it simple for someone to take an open source project and use it as a basis for a proprietary product. A best known example is the BSD license; and Apple’s adoption of FreeBSD kernel code in their XNU kernel or NetApp and their use of FreeBSD in their customized storage appliances.

Continuing where I left off, it would not have been possible to open source the Solaris kernel for the OpenSolaris project if it weren’t for the CDDL license. In turn, ZFS would have been incompatible with the CDDL license if it were licensed under the GPL; although it has no conflict with non-copyleft licenses such as the BSD license. Because of this and now because of Oracle’s admitted support and commitment to Solaris, I doubt this licensing will change; especially to merge it into the Linux kernel. That is why we should be grateful that: (a) ZFS is available under an open source license making it impossible for it to disappear and (b) that Oracle has been committed to Btrfs and bringing an enterprise class solution into the Linux kernel.

This is why we have choices. If you want ZFS functionality, use OpenSolaris or Solaris. If you don’t necessarily need ZFS and are more comfortable with Linux, you have a lot more distributions to choose from. Or if you want ZFS and a familiar Linux environment, there is also Nexenta.