Why Pick Just One Operating System?

Thomas Lord

Sat May 10 19:57:41 -0700 2008

manage

Virtualization is a neat trick and if you have a rack of mostly-idle servers you can use virtualization to save a lot of power and, for new projects, hardware cost.

A trivial example from real life: a certain org I worked did IT for a large group of professionals in domain X (not a software or IT domain). It so happened that they wanted to give each professional a "personal wiki," perhaps with a little sandbox for installing this or that other personal web service.

Virtualization penciled out nicely there. The limited IT staff could herd complete OS images for each professional, host them all under virtualization, thereby save HW costs, boil down and regularize the administrative stuff, and not have to worry so much about professional A doing something dumb that horks the host for professionals B, C, and D.

Another trivial example (this time client-side) is the case of office workers on the periphery of the open source industrial complex who must, on the one hand, use MSFT apps to work with certain trading partners, but Open Office apps to work with open source types. A desktop that can toggle back and forth is handy there.

---

It starts to break down server-side when what you want to put under the virtual image is going to be busy. Virtualization doubles down on the dubious bets of virtual memory hardware and hardware-based process protection. You can take your super-fast machine, try to split between two 50% loads, but you'll be delivering well south of 50% to each load by default. I'm sure you can get savings anyway, especially compared to the piggish practices that were common until energy prices started steepening and dollars falling - but you don't get a lot of savings.

A bigger source of virtualization savings from what I can tell (I'm drawing some inferences here) is exactly in IT department streamlining. You can cut down on the number of HW reqs that have to get worked through. IT staff can homogenize their management policies. So, there are potential labor cost streamlining possibilities and HW purchasing efficiencies -- but fairly huge and exceptional is the IT dept. that will get much from that, even.

---

Client-side -- you ponder "browser with its own OS?". To some degree we already have that, mostly in mobile and mostly sans virtualization. Low-power, simpler CPUs, short OS stacks, and purpose-built browsers. If (and it's a big "if") hosted applications get, let's say vaguely, "thrice as good as they are now" -- then maybe we'll finally see thin (desktop) clients based on browsers really take off in which case the stacks developed for mobile will probably get first-mover.

---

The alternatives really involve smarter stacks that can share resources at a higher level. And that quickly gets into higher level languages. And once you're going to a greater emphasis on higher level languages throughout the stack, well, you may as well go for language-based process protection and language based memory hierarchy management. In other words, for starters, you can toss out virtual memory hierarchy (and, in my view, shrink the direct address space of the CPU down to about the size of an L2 cache -- the old quip "640K is enough for anybody" is only, at most, a slight underestimate).

Once you're going down that path, for a lot of apps, the language designers are going to want programmable hardware. FPGA's etc. That'll be hecka fun when we get there.

---

And this is not a story people tell.
It is something I know myself.
And when I do my job, I am thinking about these things.
Because when I do my job, that is what I think about.

-t

Why Pick Just One Operating System?

duhavid

Sat May 10 21:38:50 -0700 2008

manage

"A bigger source of virtualization savings from what I can tell (I'm drawing some inferences here) is exactly in IT department streamlining. You can cut down on the number of HW reqs that have to get worked through. IT staff can homogenize their management policies. So, there are potential labor cost streamlining possibilities and HW purchasing efficiencies -- but fairly huge and exceptional is the IT dept. that will get much from that, even"

 

The number of machines goes down, but the cost of that smaller number of machines goes up, as you need higher quality equipment, with much more memory capacity.

The labor savings I dont see as much, as the virtual OS requires all the maintainance as the real ( there being little if any difference ), and there is still hardware.

I think the savings, if any, come from all the small "department" servers, where the vendor wants only that app running on the server, or the IT dept wants only that app running, for many ( good, I think ) reasons.  So, all those can be consolidated on one peice of hardware, but running isolated at the OS level ( assuming the virtualizing software is not buggy :-) )

 

"The alternatives really involve smarter stacks that can share resources at a higher level. And that quickly gets into higher level languages"

I agree with you, but I dont think it will happen fast or soon.  As I see it, that would mean that you would have to knowledgable programmers.  The chaff will fall away from the wheat, and the wheat will start commanding more money.  Which is the exact opposite of where the big players want things to go.

Why Pick Just One Operating System?

Thomas Lord

Sun May 11 08:23:53 -0700 2008

manage

To clarify one point, you write:

The number of machines goes down, but the cost of that smaller number of machines goes up, as you need higher quality equipment, with much more memory capacity.

Yes, but you also write:

I think the savings, if any, come from all the small "department" servers, where the vendor wants only that app running on the server, or the IT dept wants only that app running, for many ( good, I think ) reasons. So, all those can be consolidated on one peice of hardware, but running isolated at the OS level [....]

That's the kind of thing I was thinking of both for labor savings and purchasing efficiency:

From what I read in the trade press, the trend has sloshed back and forth over the years between "centralized" IT departments and "federated" or even "anarchic" departments. In centralized, you have one IT org chart, say, under the CIO. In the ideal, they control all of the firm's hardware and all software installs; they can give you a complete list of all the installed systems and, where necessary, show you the corresponding license purchases. They set company-wide policies and then control exceptions ("everyone will upgrade to Vista, except finance until the accounting app is available"). In anarchic arrangements, a large firm has lots of little IT departments, each under some division manager. They can improvise and go fast for their division, but the firm as a whole has no "big picture" of its IT set-up. Federated is in between with a central authority but some divisions able to have their own IT departments that cooperate with, report back to, and may be subject to the authority of a centralized ubrella group.

Federated seems to be very common, de facto. Executives become uncomfortable with anarchy so they create a central ubrella group. Or, division managers get frustrated with pure centralization, the CIO has only so much power, and some divisions start getting their own IT groups so as to "get stuff done".

When a lot of the action in a firm is done by the non-centralized IT staff, you get purchasing inefficiency just as you suggest. A typical case: The firm undertakes Project X in some division. The needs of Project X are so very special that the division running it is authorized to go outside of centralilzed IT and do their own thing. The IT guys there want to be smart, smart, smart and think ahead and lay a good foundation for the new fiefdom so they make a bunch of purchases anticipating the growth of Project X. They'll overbuy for immediate needs in anticipation of growing into what they buy. Then Project X is canceled or cut way back. If canceled, it may be discovered that the purchases are totally useless to the rest of the company -- a big white elephant. If cut way back, then you have a dinky project running on way-to-beefy infrastructure, jealously guarded by the Project X manager's and private IT department. Multiply that across the firm and across several years.

Virtualization helps central IT avoid that problem. They can say "Ok, Project X, perhaps you grow tomorrow and need your own hardware but today you certainly do not. We'll give your private IT staff a bunch of virtual hosts and they can prototype there. The Project X managers are happy enough because they can still have their project-specific IT staff responding quickly to Project X needs. The firm as a whole is happy because they're paying at most an incremental and predictable cost for new blades / storage and those new purchases can be yanked at any time from Project X and reassigned to the new Project Y, no questions asked. Purchasing efficiency.

One step further with that: The firm can even, as you say, wind up with larger invoices for HW purchases this year than if they stayed with a non-virtualized approach. The difference is in cost recovery because there's much less risk that large percentages of the HW will, in future years, sit mostly idle or have to be sold off. Utilization of the HW is likely to be higher. Down the road, relatively less "equivalent resources" (one virtual host = one physical) will be needed.

Labor efficiency follows from that. Project X doesn't need staff setting up machine rooms. Project X doesn't generate as many purchase orders to keep track of. Shipping and Receiving has less to do. A little more subtly: when Project Y comes along, if appropriate, cloning the IT work done by Project X in a virtualized world can be cheap and easy compared to cloning it in a real world. Finally, virtualization suppliers work hard to take advantage of the homogeneity of virtualized set-ups to deliver tools for herding the virtual images that virtual hosts run. You wind up getting back one of the labor-savings of centralized services: if central IT decides "all systems must run this new version of our back-up software" then, using the new tools, someone can install and test that on all virtual images from her desk rather than running around the company, finding all the machines, and installing it by hand again and again.

I suspect that this also saves executive and upper-level management bandwidth because it reduces the amount of budget haggling people need to do to "get stuff done."

---

Higher level languages. You express a popular thought, with a neat twist. The popular thought:

I agree with you, but I dont think it will happen fast or soon. As I see it, that would mean that you would have to [have] knowledgable programmers.

People often say that. The neat twist is drawing out the cynical implication:

The chaff will fall away from the wheat, and the wheat will start commanding more money. Which is the exact opposite of where the big players want things to go.

I'll be radical here. I'll go out on a limb. You probably expect me to drag out old refrains from the common lispers and high power haskell types and say something like "Yeah, but the best programmers are demonstrated to be 100 times more productive than average line coders." That's not what I'll say because I think it is only of indirect importance and that it underestimates the problem.

Two facts emerge when we look at the constellations of labor and of business needs:

In labor, there are now and long will be a large supply of those "average line coders." Famously bad ones are, well, famously bad but many are just "ok" -- not disastrous; actually get some useful things done; just not your "goto guys" for really hard problems.

In business needs, there are lots of details to be addressed. Almost every commercial web site needs someone doing styles. Many commercial back-ends, even if they use generic tools, need site-specific business logic. There are even only barely (and often "not right for our needs") turnkey solutions for basics like feedback forms, e-commerce, content management work-flow, etc.

Well, "Hello and greetings, you fine line coders!" Just because you don't want some of the less experienced and under-trained folks writing your frameworks doesn't mean they aren't good for all "that stuff". By analogy, telephony folks who get very good and smart about maintaining junction boxes in the field may not be the guys that you want planning out next generation switching technology but, conversely, you won't be pleased if you expect the ones strategizing switching to go hook up new lines, repair breaks, audit closets, etc.

We already have a situation in which the Super Geniuses in computing systems work indirectly: applying high-level, highly-experienced / trained knowledge to create frameworks and environments designed to be operated by front-line guys. That will not change anytime soon.

What I say we are missing is investment in practical R&D on the high-level side, where that research work is aimed at constant reconsideration and improvement of the system stacks and frameworks.

Do you want to make those front-line line coders more useful? The more effective, efficient, well organized, and lucidly presented are the frameworks they train on, the better they get. The past decade of web services has demonstrated that quite effectively -- my point is that with better Practical Research we could be taking quantum Giant Steps rather than watching people riff on the Ruby Rag or the PHP Polka.

And, meanwhile, systems research has another timely job to have really started in earnest 14 years or so ago: simpler, more energy efficient HW is a big need; better software strategies for resource sharing is a big (energy) need. Meanwhile, fabrication for simpler HW is falling in price -- there are bargains to be had there if you can find a use for them. Practical research with an emphasis on full-stack application of higher-level languages is the most promising potential enabler of radical disruption of today's most deeply rooted marketplace hegemonies in IT.

---

Few things tick me off quite as much as when self-appointed spokespeople for the Open Source Industrial Complex start spouting about how some degree of software freedom inspires people all over the world to start contributing, creating a fountain of innovation that can't be topped. This is almost, but not quite, the "million monkeys" theory.

Regarded as a strategy for research, the "open source community" almost entirely explores the question: "What can the brightest line coders make in their basement from their received system stacks and frameworks?" Rather perniciously, it also explores the question: "What innovations in marketing can the line coders come up with that we may have missed or can't do ourselves (e.g., signing up a critical mass of Harvard students to a social networking site)?"

That amounts to race to find the point of diminishing returns on the received systems stacks and frameworks. And that's fine. But it's not fine if it involves pimping out volunteer communities. And it's not fine if it isn't accompanied by a proper research program to keep expanding the "basis set" of improvisation with disruptively new technology.

-t

Why Pick Just One Operating System?

duhavid

Sun May 11 13:42:22 -0700 2008

manage

"In labor, there are now and long will be a large supply of those "average line coders." Famously bad ones are, well, famously bad but many are just "ok" -- not disastrous; actually get some useful things done; just not your "goto guys" for really hard problems."

 

Quite, and since there are problably thousands or 10s of thinusands for every superstar, they are probalby important.  What I find, thought, is that I am often constrained in what tools *I* am allowed to use, because the average coder on my staff is not up to using them.

The more I get into leading teams, the more I find that what Free Brooks said about team composition is true.  You have a great, talented person as the center of the team, directing the others ( potentially specialized ) persons in their work to achieve the tasks set to the team.

Why Pick Just One Operating System?

Thomas Lord

Sun May 11 14:55:22 -0700 2008

manage

Neat.

Also, "Free Brooks" is a funny typo. :-)

If you don't mind the question, can you say a word or two about how you got into a leadership role?

On the tool constraints: yes, exactly. Which I mean in this sense and as another point in the argument for a return to "practical research" and Great Labs:

One of distinctions (as I get it, at least) of "tools you can use" in that circumstance vs. those you can't is really the learning curve for the tools. People will often say something related but different -- that the skills available in the labor market are the limiting factor. I'm suggesting you have to look one-up from that, in two ways.

People self-train, take courses, get the latest books from O'Reilly, read all the instructional web sites: that's where the bulk of line programmer skill sets seems to come from. (It's where they start, anyway -- of course these guys get better and gain more skills with on-the-job experience.)

That sets a higher bar for systems research that really wants to radically disrupt lower levels of the stacks or put forward new application frameworks. You don't get so far saying "Here's Ruby, a cool language," but Rails makes all the difference. You don't get so far saying "Subversion fixes a lot of problems CVS has" but you get farther adding a book, Eclipse hooks, web GUIs, etc.

And those examples are a bit like the Bell Labs guys writing typesetting tools for chemical diagrams: the power of the typesetting pipeline becomes infinitely more meaningful when it's presented in ways that the domain experts you want using it can speak something closer to their familiar language.

Research has to generate (not necessarily itself produce) those tools but, that's just half. Producing those tools by brute force can be done but the cost is high. (Thus, for example, it's currently "all in" on Eclipse, for many gamblers.) Another trick is needed, and that's to make those nice domain-assisting tools cheaper and easier to develop -- you need research aimed at "basis set thinking". You need design elegance -- an elusive goal but one you can hunt down in somewhat systematic ways.

I don't have a perfect or commonly recognized vocabulary for it but in systems design there are phenomena that could reasonably be dubbed "harmonies and dissonances," "gross structure and fine detailing", stuff like that.

A fine example might be what is commonly dubbed "impedance mismatches" as in data representations in web services. If you've got XML on the wire, Ruby objects in RAM, and SQL tables on disk you've got three layers of conversion and, worse, conversion that generally can't be done losslessly and reversibly. You have huge amounts of code doing naught but tedious bookkeeping. You have redundant code at each level. So you have a framework that at its most important levels has "all the maneuverability of a brick". That's "dissonance."

There is an art and a systematic craft to building worlds in which "line coders" will come along and do lasting, beautiful, valuable work. It's an art that, in my experience, gets beat back wherever it is detected, these days.

-t

p.s.: I should fold in this thread. I've been quite long-winded in this thread. It's hard to condense this stuff into bite-sized, easily digestible pieces. It's nice to have an occasional forum to try out some more refined attempts but.... I should probably back off here and wait for something else to come around. I've plenty to digest, myself, now. :-)

Why Pick Just One Operating System?

duhavid

Sun May 11 15:50:30 -0700 2008

manage

"Also, "Free Brooks" is a funny typo. :-)"

  I have a song to go witth that.  :-)

"If you don't mind the question, can you say a word or two about how you got into a leadership role?"

  Sure.  Peter Princpal.  I was competent at my job, so my bosses eventually handed me additional responsbility leading efforts under the main effort, or whole projects that were not the highest priority for them, and I got trained up in how to take a team and get something done with it.  To date, I have not been hired expressly as a team lead or manager, but I keep getting promoted to it.  And that is not a a problem, I like that role, as long as it does not move me out ( for long, at least, at Wireless Knowledge, I didnt touch code for a long while, but didnt mind much ) out of a direct coding position.  On Monday, I interview with someone ( submitted my resume for a part time, telecommute position, as my main client wants me to ratchet down the hours, and the company I consult thru wants to change me from W2/hourly to salaried ), for something where they are considering me for a team lead position from the outset.  Should be interesting, I am not sure if I will take it or not, they are talking about having deliverables in 6 months, but I dont have a clear picture of what they expect built yet.

Why Pick Just One Operating System?

cowardly dragon

Mon May 12 12:41:10 -0700 2008

manage

Here is one thing to keep in mind. As a developer of a typical enterprise framework, it would be nice to have a large array of WinXP-scale machines to run servers from. Usually, processor speed isn't the problem. It's memory. But Memory is dirt dirt cheap these days. 32bit windoze can't seem to use more than 3-3.5GB easily, but really that's all I need.

If you had a rack of blades each with 32-64GB RAM and several quad-cores, that may actually be cheaper to manage, and each could hold 16-32 server machines at probably near-100% availability. For dev+test+training, you could probably double that.

I think that would be cheaper in the log run, and make virtualization a smart plan. Plus, virtualization gives you snapshots, backups, quick redeployment/moves between machines, rapid reimaging and repurposing.

The key is to stay close to commodity hardware and not encroach into superexpensive big iron. Stay to the sweet spot.

Once you get to production, well, that is a different story.

what do you mean by "long run"

Thomas Lord

Mon May 12 12:56:51 -0700 2008

manage

The subject line has the question.

In your profession, when you think about "in the long run," what amount of time do you mean?

In fairness I should explain that I ask because of my view that there is an artificial scarcity (anti-competitive collusion and suppression) of practical research.

-t

Why Pick Just One Operating System?

Alcibiades

Sat May 10 23:42:05 -0700 2008

manage

You notice that it does not do OSX on Solaris or anything else.  The virtualization companies must know how to do this (Parallels just about said they did) but do not release it.  Interesting.

Why Pick Just One Operating System?

Mathinker

Sun May 11 00:43:47 -0700 2008

manage

Isn't that because the EULA of OS X forbids its use on anything but Apple hardware? Or am I remembering incorrectly?

Why Pick Just One Operating System?

Kai Howells

Sun May 11 00:59:23 -0700 2008

manage

OS X requires, among other things, a TPM chip to decrypt some stuff in the kernel. Due to the design of the TPM chips, they can't (can't - there's a strong word. really hard to) be virtualised. OS X boots, doesn't see the TPM chip and you've got your Darwin kernel running, but none of the funky OS X goodness.

There are hackintosh projects out there that delve deep into the source of OS X (all the stuff at this level is Open Source) and remove some of these restrictions - with the result that I've seen VMWare hosting a Mac OS X 10.4 image, in fact hosting it very well.

All this aside, as mentioned in the parent post, it's against the EULA for Mac OS X Client. Mac OS X Server, however is a different story. Apple specifically changed the EULA for OS X Server in 10.5 in order to allow multiple instances of the OS to legally run on Apple server hardware. VMWare are (and I'm sure Parallels are) working on a virtualisation solution for OS X Server. VMWare reportedly have an unmodified OS X Server image booting in a VM on Apple hardware.

Why Pick Just One Operating System?

Bhima

Sun May 11 01:19:42 -0700 2008

manage

According the book "Mac OS X Internals" by Amit Singh, Apple does not actually use the TPM module. And as far as I can tell Apple quit including the hardware in late 2006... certainly it is not on my early 2008 Mac Pro. (I looked).

Singh provided the details online... and I submitted this to technocrat when he did.  Personally I'd welcome a TPM module on any computer I maintained, as long as I fully controlled the module.

(but I'm not really an IT guy).

Why Pick Just One Operating System?

Kai Howells

Mon May 12 00:02:06 -0700 2008

manage

That's very interesting information. I've read some of Amit Singh's other work, and he really knows what he's talking about.

I wonder what methods Apple use to "lock" Mac OS X to Apple hardware then? I'm now making the educated guess that it's something to do with the EFI layer - not many other vendors ship PCs with EFI.

Why Pick Just One Operating System?

David Leppik

Mon May 12 09:06:37 -0700 2008

manage

EFI instead of BIOS is the main thing, but in general if Apple doesn't need it, they don't supply a driver. Graphics cards and sound chips, for example.  It's just hard enough to keep honest users (and vendors) honest.  And for everyone else, there's the possibility of lawsuits.

Why Pick Just One Operating System?

Richard Young

Mon May 12 04:50:52 -0700 2008

manage

I tried Virtual Box a while back at work.  My network connection is plugged in via my Cisco IP phone.

Everytime I started Virtual Box up, the Cisco phone went into a hard reset!

A shame really seeing as the feature set is almost as good as VMWare workstation 6.

 

Richard