A little about Woodwind Timbre and Oboe Timbre specifically

I've always been fascinated by timbre: the quality of a sound, or tone, that distinguishes it from other sounds of the same pitch and volume.

I have (or had) perfect pitch, the ability to identify or reproduce a musical note without a reference point.

I've written quite a bit about this. Here, what I want to say (or repeat) is that I came to this by a fascination with the timbre of a particular note, the lowest note on a cheap Gemieinhardt. The lowest note on woodwind instruments is the most funky, and while the timbre variation on the flute is not as pronounced as on other instruments like the oboe (which this will largely be about), still there is some variation.

Woodwind instruments as a class of instruments are unusual in this respect: the variation in their timbre across individual notes, from one instrument to another, and between different instruments of the family are great. Much more than say string instruments, a piano, or electronically synthesized instruments, even electronically synthesizing woodwind instruments.

There is a reason for this, but I won't go into the technical reasons here.

But to try to get this idea of inhomogeneity between different woodwind instruments, consider opening from a woodwind quintet by Jacques Ibert, Trois Piece Breves:

Notice how the instruments don't blend together. In this clip, to me it feels like there are at least 3 separate color palettes for the 3 sections of this little clip. In the opening, there is the sound of all of the instruments together. And there is a section with the horn solo which has the lower instruments. In the last section of the clip we this particular clip, there is the oboe solo with its predominantly bassoon accompaniment which gives it a more reedy palette.

Note: not all of the instruments in a woodwind quintet are strictly woodwinds; there is one brass instrument, the French Horn.

To give a more extreme sense of not blending in, here is a more extreme example that I like. It adds more brass and some percussion too:

My heartbeat and blood pressure rise every time I hear the above clip.

Now compare this to the string family in the opening passage from Elgar's Enigma Variations, Nimrod Variation:

These examples were selected and contrived to show my point in the extreme. In particular, the Elgar violin, viola, and cello parts are divided so that the first viola parts match the lower violin parts, the lower viola parts match the upper cello part, and the lower cello part matches the bass. So there is deliberate extra blending going on here.

But the point still remains. To me, this is like the differences in paintings I mention in Key and Psychology

Ok. So let me now tie this non-homogenity into the oboe in particular. But in a later post, I may show how the oboe also blends and how that had a historical importance.

Above I gave an example of nonhomgeniety between different members of the woodwind family: flute, oboe, clarinet, bassoon.

For the oboe, the timbres of the instrument vary greatly as well and that is a whole topic on its own.

However, within a single instrument played by a single person, there is variation between adjacent notes in a scale across the three or so octaves that the oboe spans.

To ease into demonstrating this, consider this pretty well-known example from Prokofiev's Peter and the Wolf:

This passage is in the lower part of the oboe's range. As I mentioned briefly for the flute, the lowest notes of a woodwind instrument are the "rawest", not blending, or are richest in overtones.

I suspect Prokofiev picked this low region for its honkiness and duck-like quality. As a friend sarcastically remarked: The versatility of the oboe (which to him always sounds like a duck) and the genius of Prokofiev combine to produce startling barn-yard effects!

Timbre changes in wind instruments are related in a way similar to the elements in the Chemical Periodic table; think of each octave of a flute or oboe as being a row in the periodic table. It has a row-like character or "heaviness". But pitches within an octave (or periodic table columns) also have a timbre similarity as well. (For the clarinet the term "octave" is changed to "register", because the relation at the lowest level is an octave and a fifth

Here is a guide to oboe timbres heard in this passage. The non-natural notes Eb and Db are more muted than their neighboring notes E natural and D. C, since it was the original lowest note of the instrument is also very honky, like a duck. In the last two notes of measure 4 where we jump up to D and B, some of the rawness goes away but you can hear traces of it.

If this is too involved to follow, just focus on the first four distinct pitches, E, Eb, D, and Db; see if you notice how the timber quality changes. In particular open E, muted Eb, Open D, muted Db.

The above was about the lower range of the oboe near Middle C. As for timbres in the last part of the solo: 

The sound of the upper C above middle C has less of that lower rawness and is more pure. The neighboring Eb while having the second octave characteristic still has some of the timber of its lower Eb counterpart, just less pronounced.  

 The Eb is a little more muted than say its surrounding notes E or D. Similarly Db which appears in the second measure is muted too (a little bit more so than the Eb). Bb while not as muted as the lower Eb is still muted compared to C. F is a little muted compared to G that follows. And notice the change in timbre between the C above Middle C and the Eb below it.

Here is the same passage that is heard an octave higher that appears later in this piece. Notice how the register timbre is different and a lot of the coarseness has been removed.

Towards the end of the passage, the oboe goes into the lower part of the 3rd octave above middle C and the upper part of the second octave of middle C. And here the coarseness is removed even more. (But again, I can hear the timbre changes between individual notes.)

I realize all of this may be hard to hear, and may seem like a wine connoisseur describing different kinds of wine profiles.

When I hear an orchestra, as a result of these slight timbre changes, I can pick out the specific notes that are being played, and thus calibrate my perfect pitch sense. I can do the same on hearing a flute as well. I should say that since I played both of these instruments in the past, these kinds of timbre changes have been ingrained.

Oboe Reed Enginering

On a Tuesday night hike, I started ranting about how the Oboe is the most engineering-like in the construction of its reed and how the reed can drastically change the timbre.

In this part, I'll mention a little bit about oboe reed construction. The timbre aspect is equally fascinating but that will be done separately.

Here is how an oboe reed is scraped in the style developed by Marcel Tabuteau and John Delancie around  1948-1951. 

Here is a back-lit image of this kind of reed.  Oboists who use this style of scrape, do hold the reed up to the light like this to see what's up.

In Tabeteau's reeds, the spine down the middle did not exist. However it is now generally added since it makes the read more sturdy.

This style is now popular; it has changed the oboe sound in the U.S. This style and sound then spread to Australia, Japan and Asia, Latin America, England (after a longer delay), and many countries. France and Germany are more steeped in their own traditions and styles and of sound,  so the spread there is less than in Europe.

Here is an image of the "French" style which was almost universally used before 1948:

The image is from Regency Reeds, a seller of oboe reeds from England. There generally is not a backlit view of these kinds of reeds because there is no need to. However, here is a diagram comparing the two

from https://oboewan.com/index.php/about-oboe-reeds/. 

Now here is a back-lit image of a slightly larger oboe used mostly in the Baroque period, called the Oboe d'amore: 

It uses the American scrape, so it is probably used on a modern instrument. The Oboe d'amore is slightly larger and is in A rather than C.  It is a transposing instrument. The current modern Oboe's lowest note goes down to Bb so you might wonder what's the big deal. But in the Baroque period, the Oboes went down only to middle C. And since they had only three keys (where one key was a symmetric duplicate of the other so one could play the instrument with the left hand or the right hand in the upper position), playing in a key like A major (and closely related keys) sounded very different when played on an Oboe in C instead of an Oboe tuned in A.

Now compare this reed with the reed for the smaller Baroque Oboe of the kind used back in the 18th century when the oboe was first developed from an older instrument called the "Shawm": 

The Baroque Oboe is made of a softer wood -  boxwood instead of the harder grenadilla wood or plastic. And the bore of a Baroque Oboe is larger than its modern counterpart, so there is less air pressure on the player and there is more airflow that passes through the instrument.

Notice that as is the custom with the Oboe d'amore, English Horn (in F), or Bassson, there is no cork on the metal part or "staple". This reed is a little bit larger and wider than an oboe. So it is more forgiving than a modern oboe d'amore reed, which is more forgiving than a modern oboe reed.

You'll see that the baroque version in C is about the same size or larger. It is wider or more V-shaped at the tip that goes into the mouth. Also, no cork is on the staple.

Compare the oboe reed with a bassoon reed:

See how the bassoon reed is much wider and the cane portion down to the string portion is about twice as long. The bassoon plays about an octave lower than the oboe.

This means that tolerances are larger and can be more forgiving. It is influenced less by lip mass. The scrape is the French style. Typically there is a wire that can control the aperture of the reed opening as well as help to keep air from coming out the sides. Since there is less pressure, it is less likely to occur on the bassoon. One of the French-scrape Oboe reeds above also has a wire. Wires are typically frowned upon because, at the scale of oboe size, it can interfere with the tone. Moreso for American scrapes than the French scrape where there is more cane bark near the wire. But for a commercial maker such as this English firm, it is more practical to ensure there is no air leaking from the side.

About the cork on the staple. In modern oboes, there is a cylindrical reed well: 

from Yamaha's the Oboe: double reed mechanism. 

In the modern Oboe, not inserting the reed cork completely to the bottom of the well will cause a break in the conical bore wind flow at a place of higher pressure where it is more critical to change sound production. So, in contrast to most other instruments, you can't or don't "tune" the modern instrument other than what you can do by "lipping" the note up or down. (Lipping down is easier than up.) The oboe's inflexibility in tuning is one of the reasons the orchestra tunes to the oboe. (The oboist tunes the instrument by making the reed length and scrape so the pitch comes out correct.)

Other instruments of the orchestra are inflexible too, like keyboard instruments and tuned percussion instruments (xylophone and chimes). When the keyboard instrument is a soloist like in a concerto, the orchestra tunes to that instrument. However since xylophone and chimes are rare minor instruments, those are not used for tuning even though they are inflexible as well. (The oboe's clear or perhaps "piercing" sound also makes it good to tune to).

Python in 2016 feels like C of 1996

(This originally appeared on Quora)

In 2016 I use Python a lot. I've been using it over 10 years or so. But even with that much use it still feels clunky. It reminds me of my experiences with C which go back even longer.

Let me explain.

There are people who feel that Python has revolutionized the ease with which they work. xkcd had an comic about it: xkcd: Python and I guess the pythonistas liked it so much that since Python 2.7 (and pypy 2.6) it has been in the standard library distribution that you can get to it via the antigravity module that was added since the comic came out.

And yes, many find Python an improvement over say earlier languages, much in the same way was felt about C. I can see how they feel that way. It is true that there is such a great body of libraries in the language, like C. Python has about everything.

Except elegance, and orthogonality.

But before diving into that, let me come back to C circa 1996. As I said that's exactly how people felt and I suppose still feel about C. For example, what is the most-used version of Python written in? C, of course. In fact, that version of Python is often called CPython.

Back in 1996, people were ranting about C, I guess rightfully, was because it was like the Fortran for systems programming. So I guess in a sense C around 1989 or so was the Fortran of 1959. And I'll go into the Fortran comparison just a little bit as well.

When I first started working at IBM research I worked with a guy, Dick Goldberg, who worked on the original Fortran project. Back then I felt Fortran was a little clunky too: there was no BNF language definition (but note that the B in BNF stands Backus, the guy that also was behind Fortran. Fortran came before this and this aspect was addressed in general, if not in Fortran).

And it had these weird rules like you couldn't index arbitrary expressions like x[i*j] but you could do some limited forms of that like x[2*i] or is it x[i*2]? And the reason for that was that back then they translated this kind of thing into IBM 360 assembler which allowed a memory index (a multiplication) along with a memory offset adjustment (an add) as a single instruction and the Fortran compiler could turn some of these into that form although it couldn't even re-associate 2*i to i*2 if that was needed.

But the big deal, as Dick explained to me, is that scientific programmers and numerical analysts didn't have to learn assembly language, because Fortran did a good enough job, on average, to obviate that need for the most part.

And then that brings us back to C. Fortran is not suitable for writing operating systems, or systems code and so around 1990 many Operating Systems were written in assembly language. Microsoft's OS's were. IBM's were as well.

But now for Unix, then minix and then Linux, you could write in the higher level language, the same as you could in Fortran for numerical or scientific software.

And libraries were written and to this day the bulk of complex libraries are still written in C: for regular expressions, for XML parsing, Database systems like mysql and postgres, and a lot of systems code. Even the guts of numpy, the numerical package for Python, has a lot of C code at its core and that is very common. You'll find other popular programming languages like Ruby, or Perl are written in C and their numerical packages fall back to C as well. Also the bindings that interface to those C systems.

Inelegance

Ok. So now let’s get back to inelegance in Python.

In Python, some functions are top-level functions like len(), hasattr(), type(), getattr() str(), repr() and so on, and some functions are methods off of an object instance, like the .join() or .append() methods of string and list objects, even though lists and strings are built-in types. So instead of writing in a uniform o.join('a').split('x').len().str() or str(len(split('x', join(o)))) you have to go back and forth depending on which arbitrary way it was decided for the function, e.g. str(len(‘o’.join(‘a’).split('x'))).

This is like learning in arithmetic expression which operators are infix, postfix, prefix and what the precedence is when there aren't parenthesis. And yes, Python follows in that tradition like many programming languages including C nowadays as well. This is in contrast to Polish prefix of Lisp, or the old HP calculators, Forth, or Adobe's Postfix.

I'm not suggesting languages change it. But I am saying it is more cumbersome. And when you extend that notion, it hurts my little brain.

C was notorious for extending that into "address of" (&) and "indirection of" (*), pre and post increment (e.g. ++) bitwise operators (|), logical operators (&&) and indexing ( [] ), and selection/indexing ( . , ->). Keeping in mind the precedence was so difficult that most people just used parenthesis even when not strictly needed.

I do need to back off the comment about top-level versus method functions in Python. But only a little...

"len" for example is a method off of some type like string or list. But it has these ugly double underscores around it, I think that was added to deter people from writing it in the more natural or uniform way like you would in say javascript.

Python does stuff like this: take something that would be usable but muck it up more to make it less usable. And there is a community of Python Nazis out there who will scream at you if you use x.__len__() however ugly it is instead of len(x).

These are the same kinds of people who insist that you should write:

import os

import sys

rather than the shorter:

import os, sys

When you ask why? They say just because. Well, actually they don't say it that way, they refer you to an acronym and number like PEP8, which is taken like legal law. Everyone has to do it that way. Fascism.

The Fascists say that doing things this way makes it easier to understand if people do things the same way. Really? Mankind has long been able to deal with variation in expression without any effort whatsoever. I can write "It is one thing" or "It is a thing" and most people don't obsess over whether it has to be "one" or "a". Actually, in English there is a certain style difference if I want to emphasise the singleness I might choose "one" over "a".

And so I'd argue that kind of nice subtlety is missing by the Python fascists. For my part, I'd prefer stipulating it would be okay to stipulate that every program imports "os" and "sys" and be done with that and get on with the rest of my life and the more important task of programming. The next best thing is just to string along that preamble crap in one line the way Jewish blessings always start "Baruch atoy adenoy eluhanu" (Blessed is God, King of the universe").

Oh, and by the way the "import" statements have to be at the top, (except that sometimes there are situations where they won’t work if they are at the top). That seems so 1980's Pascal like to me which required all constants, types and variables to be put at the top of the program. But in Pascal it was not for the same kind of fascism, but just that the Pascal grammar was limited in that way.

And while on the topic of Pascal, let me mention another aspect related to Pascal. Originally and to the end of Python 2, "print" was a reserved word. Very similar to Pascal's "println". And because it was a reserved word, you didn't put parenthesis around the arguments to print. I'm sure this was thought clever in the same way it was clever in Pascal. When Python 3 came about that changed so that print is now the more regular and uniform function call. Oh, but by then a decade-or-so-old code base then had to change.

Now you might think, "who knew"? Well, although Pascal had that feature (and most other languages including C just didn't), by the time the successor to Pascal, Modula, was developed the mistake was corrected. And here's the point: this was all done before or contemporaneous with Python's start.

It's one thing to make a mistake and correct it. But another thing to make the same mistake as was just fixed in another language, and then spend several years before you fix it the same way everyone else does. Pascal and Modula were developed in Europe, same as Python, so it's really no excuse not to have known about it.

Stubbornness and Doing things differently

So why is it that such things take so long to address? The Python language has been stubborn and unrelenting in its wrongness. I think Guido now agrees that indentation thing was a mistake. However there was a long period of time when its superiority was asserted. To me, the mistake is not so much about using indentation for a block begin, but the lack of adding even an optional "end" terminator.

But because of Python's indentation, printing python programs in print is subject to error since there isn’t the needed redundancy check. And more seriously, you can't embed that in another a templating language because the templating can mess up the fragile indenting. So if you are doing say web development in Django, you need to learn another language which of course doesn't have that indentation rule. Ruby programmers don't suffer that limitation and their templating systems use Ruby.

It also feels to me like stubbornness and arrogance that Python has long resisted using common idioms of other languages. (By the way, I sense that Dennis Ritchie and Guido Rossum are a little similar here). So the backtick notation of shell, Perl, and Ruby is not in Python. Nor for a long time was there simple variable interpolation. (In Ruby that's done via # rather than $ but the change was for technical reasons, not arrogance or a desire to do things differently).

In Python, how to do subprocesses has changed over the years, and still isn't as simple as backtick. But I suppose that has some cover because there are security considerations. Variable interpolation inside a string was also long resisted, although there was something like the C format specifiers inside a string with the '%'. But once Python added its own flavor of variable interpolation using the .format() method C style variable format specifiers are eschewed.

This kind of being the last on the block to give in to something that is common among other languages and then add it in a different way seems to be the Python way. A ternary if/else operator that C introduced and adopted in other languages is another example.

For those that want to work across multiple languages this kind of awkward originality just adds more warts.

Like Python, when C first came out, it had a number of unfamiliar things like all those post and pre operators, I guess the format specifier thing, variable arguments, the C preprocessor with its macro expansion. (Many of these weren’t strictly unique: you could find examples of these things elsewhere, such as in assemblers; but compared to say the high-level ALGOL-like languages and things like Cobol or Fortran, this was different). In C's case though they largely stuck to this and didn't change course. When developing Go, the mistakes of C were just fixed and so there I feel that Go is a very welcome upgrade to C.

…

Misc other things: comprehensions (not needed in better-designed languages), language drift, import statements.

Another example of why NYC is a great place to live but I wouldn't want to visit.

Today is Santacon. 

It is the day when people young and old, big and small, dress up as Santa Claus. Or if that's not your thing, just wear felty or furry brown and put on a pair of felt antlers and that works too. 

I see such diversity in costume today, maybe just the hat, maybe just the pants, as I walk the 1/2 mile to the Saturday Farmer's Market. I look for basically two items.

The first item is brussel sprouts on the stalk. If the brussel sprouts are very fresh, you can eat the middle of the stalks. They have a carrot-like texture and the taste is a mild kind of horseradish with a slightly skunky smell that you find in cabbage or broccoli which brussel sprouts are related to. For the last month or so, they have been appearing at the Farmer's market.

The second item is concord grapes. I like them for their pissy taste. That taste comes from a native grape called the "fox grape" that is native to the northeastern coastal part of the U.S. 

I have never had a fox grape and would like to try it. Like raspberries or  low-bush blueberries, they are a bit fragile and do not transport very well. So people make jams with these. 

Historically, I imagine they are interesting for this reason. Since they are the native grape, I imagine that some ancient Viking real-estate developer hit on the idea of calling this area "Vinland" after discovering one if these. I imagine also that this was the same guy who previously had a successful real estate run after naming this other piece of land: "Greenland". There is a spot not far from NYC across the great psychological divide, the Hudson River, which one of his descendants named "Meadowlands". This is a double lie: it is a swamp. 

Anway, today I could not find either at the Farmer's market. I imagine it is this kind of specificness that people from NYC contributes to why others, not from the region,  may view us as "snobby". 

On the way back, I stopped into a large bookstore on Broadway. Yes, that Broadway; the bookstore is just not in the theater or show-business district. 

I have this weird idea that one day I will find a script to a Radio show I once heard by David Mamet and someone else that has music to the lyrics:

Hail to thee, George Topax

'ner against the foe lax,

Tibia quo pax,

Ee-i-ee-i-o.

          As we move along,

          through the goolagong, 

          we know where you belong,

          on your big fat throne, so .. 

          here's to thee, George Topax, ...

I don't find it. I check also the science and math section. And being a geek, the computer section as well. 

I see a book called "The Computer and the Brain" by John von Neumann. 

From the photo attached you can see a 1950's style computer, complete with wires on the front. Those large cylindrical things are some sort of memory.

I open it and read the table of contents. I am interested: PART I: THE COMPUTER. ... PART II: THE BRAIN

Von Neumann is, of course, responsible for the architecture of the modern-day computer with its CPU and memory.  It is not surprising to me that he was also interested in the brain. He was that kind of guy. And I find it cool and reassuring that right from the beginning, to his way of thinking he separates and distinguishes how a computer of his design works from how a brain works. As I look to see the price, and I come across this: 

$4.00 - I can afford that. And then I notice the writing. M. L M-something-or-other. Cambridge, Mass Nov. 1960. 

Then it hits me. ...  Could this be Marvin L. Minsky? The beginning matches, but I don't see anything that looks like the end: "sky".  I look on my phone to see if I can find a signature for Minsky. This is in the dingy and slightly damp basement of the Strand. No cell phone connection is available. I pass it by, and move on looking at other sections. 

But I figure: for $4, why not? 

So I go back and pick it up.  At the cash register I see that I should have looked at the back of the book, it was $20 not $4 which was the original price of the book. The $20 figure is because his book is "out of print". Yeah, I can understand why that might be.

Coming home, I look to find Minsky's signature. Signatures he writes in "signed copy" books do have a visible "sky". However I asked a friend who know more about handwriting analysis and says that the "M" and the "k" do look like they were formed the same way. 

According to Wikipedia, Minsky joined MIT (in Cambridge MA)  in 1958, the same year as this 3rd printing. The book's topic is clearly a topic he was interested in. 

What do you think? 

Anyway, it was his. That's my story -- and I am sticking to it!

To come back to the title, here is an example of what is cool about NYC.  I am just kind of lumbering around on a routine walk and stumble across something like this. On another occasion, pretty much that same route, I was walking home and passed a theater that said: "World Premiere". It was Samuel Beckett's penultimate play, Worstward Ho I thought now there's something you don't see every day (a phrase Edgar often says to Chauncy in the Rocky and Bullwinkle cartoon) . But then I thought. Well, I am tired tonight. I'll go tomorrow night, and did. 

Another time I was walking around, about 3 blocks from my apartment on an Easter Sunday, and there is no one out on the street except this guy, Philip Glass who I later learned lives here.

As for the "but I wouldn't want to visit" part, the touristy kinds of things tend not to thrill me. 

I did however like going up the top of the  World Trade Center because of inside the view was very myopic and all four compass sections are very very different. And then you could go to the top and see how it all pieces together. But to really appreciate something like this you have to know something about the area.

The two times I have been misled by Compiler Canon (and the Dragon book)

Twice in my life there have these weird moments in compilers where I realize something I have been taught, something that has been considered canon turns out to be misleading.

The time this happened was when I started at IBM Research in a compiler group. Previously my formal knowledge of compilers was from the first edition of the dragon book.

Register allocation focus was on reducing the stack depth in evaluating expressions. There was something called Sethi-Ullman numbers which computed the maximum stack depth.

Since one of the authors of the Dragon Book was Jeff Ullman, it is not a surprise would be mentioned.

And for the DEC PDP-11 machines this made sense, because there were stack-like increment/decrement instructions on a special "stack-pointer"register. It never made sense for IBM's computers which had general-purpose registers without such increment/decrement instructions.

When I got to IBM around 1982, there was this register allocation implementation by Greg Chaitin based on an idea by Russian Computer Scientist A. P. Ershov via IBM Fellow John Cocke. While the idea had been floating around since 1971 or so by John Cocke, Greg's simple elegant implementation it happen and put it into the PL.8 compiler.

Subsequent editions of the Dragon book ditched Sethi Ullman numbers and described this instead. 

The second time though is recent in relation to how compilers work and how decompilation works.

In around 2006, I was interested in decompilation as a way to get precise location information about where a program is when it is running code, such as inside a debugger, or just before producing a stack dump. There was some code that was started around 2000 and largely abandoned a year or two later after going through two people. Subsequent maintainers I don't think understood it as well since it has a compiler-centric orientation.

There were a couple of people who wrote thesis on this topic, and their work seems to be canon for decompilation. It assumes a general-purpose decompiler -- one that starts with machine code and produces source code. It has no understanding of the source code used, translator, or compile system used. Therefore its source code has to be some sort of lower-level language bridgings the semantics of machine-code-level instructions and control flow. C is a good language for this kind of thing.

The decompiler-as-a-kind-compiler approach however is at odds with this. It does assume a specific compilation system and its transformations into a high-level bytecode. And by doing this the source code is the same as the source code we started out with. Furthermore, by doing this we don't need really to understand the semantics of the bytecode instructions!

At first I thought this was kind of a one-off. Part if this was thinking that bytecode as a general principle is a one off. I now realize, that this indeed is not the case. As the Wikipedia article on p-code-machine explains, the idea goes back as early as 1966 and traces through Euler and Pascal via Niklaus Wirth. Rather than bytecode being a one-off. It is used a a number of systems like Smalltalk, and Java, probably gaining popularity from the Pascal P-code interpreter, UCSD Pascal and Turbo Pascal.

There was this school of thought at IBM is that there was a holy grail of a universal source-code and machine independent register-based intermediate language. Something akin to LLVM. To some extent there  still is a school of though that believes this. And there is possibly some truth to this. However there are a lot of systems that use a bytecode designed for a particular language for a particular family of languages.  Microsoft CIL and JVM are examples of a family of languages camp.

See https://www.youtube.com/watch?v=IT__Nrr3PNI&t=5311s for the history of JVM and how it was influenced by bytecode.

Sysadmin Horror Story with a Moral

Of all of my horror stories working for a NYC financial organization, the worst one was also the most memorable, not because of its horribleness, but rather because there was something that I discovered which has served me well in the decades since.

Back in the days before server clouds and virtual machines, we had real computing hardware.

Sometimes, as in this financial institution, the servers, all 1,000 or so of them, were housed right inside the business. This firm owned an entire 24-floor building not far from Rockefeller Center.

We were informed by NYC's power company, Con Edison, that in order to beef up our growing power demand they needed to turn off the power to the building for a little bit. It might have just been to the two or three floors where we housed all of the computers. This would be done on a Saturday night when all of the financial markets were closed.

The plan was that Saturday evening we'd power down all of the servers and Sunday morning at 6AM the servers would be powered back up. Because every server had to be checked, we divided the power-on phase into two groups: one starting at 6AM, and another starting at noon. We had the more experienced people doing the noon shift, because we expected that some servers might be tricky, so the early group could focus on quantity and leave the more difficult machines for the more advanced group.

This seemed pretty reasonable and clever. I was on the second shift.

Things went pretty well and we got most of the servers up and running.

Except there was one. And it was one of the more important real-time trading servers.

So now we flash back six weeks earlier...

Back in the days before the widespread use of server configuration management and orchestration tools like kubernetes, ansible, chef, or puppet, we had to write our own scripts to do mundane things.

One time I was assigned the task of writing a script to update root passwords everywhere.

Actually the assignment wasn't really that. A batch of sysadmins had burned out and quit and upper management wanted to just make sure ASAP that they didn't inflect damage on us. Financial intuitions weren't friendly places to work.

Taking the longer view of things, I wrote a general program to change root passwords. You weren't going to change the attitude of the financial institution. They treated everyone like a disposable wipe. So you adjusted; this was an activity was going clearly going to happen over and over again.

Part of the task of was, of course, to write the program to update the root password. The other part of the task, do it everywhere, was actually more difficult: get and maintain an inventory of servers. A number of the servers you could get from a service developed at Sun Microsystems called NIS.

However not all servers used this, so NIS information had to be melded in with DNS. The two lists were largely distinct because servers were set up two different and somewhat competing groups. However the groups got merged sometime after I was hired.

When a server has more than one network interface, it was important to catch that. I had written blast script that would take a list of servers, copy some code to that server and run it. The code here was the thing that changed the root passwords.

One of these servers had more than one interface, was in listed both in DNS and NIS and so it was listed twice. My blast script updated servers independently rather than serially.
When you have 1,000 or so servers to update, you need to update servers in parallel if you want to get any task done in a single work shift. (More about that aspect perhaps some other time; that story has a moral, too.)

There is a funny thing about the Unix file system (and by extension Linux as well): consistency is not guaranteed over independent file writes. In other words, if you have two clients each of which opens the same file and writes to it in parallel with the other client, the result will probably be garbage. And that's what happened when I ran my password update in parallel through two different network interfaces.

The result was that we had a shadow password file which was garbage.

Much later when I learned about "idempotency" in configuration management, a light bulb went off in my head. See definition 3 in https://foldoc.org/idempotent .

Now, pretty soon we discovered the mistake. I went to upper management to report the problem. I figured, since we were low on sysadmins which is why I was doing this in the first place, my job was safe for a little while, at least until I got the program working reliably by adding a couple of file locks into the program, and updating the inventory for the mistake.

Then I was asked what the current situation of the server was. Was the important trading system still trading?

Interestingly the answer was, well, actually yes. It was. Any program that started out with root privileges and that was running was still functioning normally. It was only new requests that couldn't log in as root.

So upper management said, don't worry about it then: just don't reboot the box.

This is exactly the highly skilled 2-steps-ahead kind of answer typical of those minds that work at places making decisions about your financial future.

So now we come back to the problem that Con Edison had decided that we were going to reboot. Most of the important servers were hooked to battery backup with specially monitored power regulators. However this one server was in the trading area on some guy's desk hooked into the wall outlet.

And as we learned that day, it also had disk clustering of the root file system which was also slightly dysfunctional. The process of rebooting involved using a magical command as root which we were only to find out about later. I hope you see where we are going with this...

If you are still with me, now I get to the main part of the story.

We spent hours trying all sorts of things and the usual tricks to bring up a server in repair mode. But because of the root partition being part of disk clustering which was messed up, we couldn't. All the other folks were told to go home because it was just this one server and it was just me and the sysadmin team lead.

We had a contract with the manufacturer/vendor for support of the important servers, but not for this particular server which wasn't racked with other servers. Finally, the sysadmin team lead, Sam, decides to put on his own personal credit card 1/2 hour live support for about $800. (This was in 1999 dollars). I was impressed: it was resourceful and took guts.

We were told that we would deal with the expert in Veritas Cluster Management, our clustering software, at Sun.

So we were hopeful.

He had us try a few thing things, but none of them worked. And then after 3 minutes of elapsed time, he said, "I hope you have good backups to restore from." Well, predictably, this important but rogue trading server that happened to be sitting on some dude's desk and had neither fault-tolerant power nor server support maintenance wasn't in the list of servers to be backed up either.

But Sam just said: I want a second opinion.

The guy said: Okay, I hope your résumés are up to date.

I was very agitated. Using a low-level disk reading command, dd, I was able to see that the all the clustering info was there. It was just a matter of reconstructing whatever information is needed at boot so we could then get to a single-user root shell with /etc/ mounted. To spend $800 to be told in 3 minutes to basically blow it all away and restore everything from backup (which would have taken several hours and lost some amount of trading data) really irked me.

But Sam kept cool. We talked to the manager who explained that this guy really was the top-of-the-line expert in Veritas Clustering that they have.

Sam patiently said this wasn't acceptable and that we still had 15 minutes of paid time to figure out something. The manager said well, in 5 minutes, at midnight, the shift changes and we have another guy on duty. He's just the guy we use to fill in off hours when there is low activity. And he doesn't know that much about Veritas Clustering. Sam said, okay, let's try him.

We wait the 5 minutes and talk to the other guy. He affirms that the person we talked to really was the expert on Veritas Clustering. He didn't know much about it, but he will look up whatever resources he has available and call us back. Okay. And in another 5 minutes he says well, by googling he sees that you can get direct support for Veritas Clustering from the Veritas company itself. Sun Microsystems was just an OEM provider.

So we called up Veritas, and gave them the dd information I had gotten earlier. With that, someone in Veritas was able to write a custom shell script which we were able to recreate the volume manager information with all of the current data in tact, so that we could boot the server to at least a root shell. YAY!

Moral: It is sometimes more important to find someone willing to help and work with you than someone who has the most knowledge about something. With someone trying to be helpful, you might be able to figure the things you don't know, things that an expert might overlook.
 

Aftermath:

While we now had root access we still had a lot of work to recreate shadow passwords and go over the /etc/ filesystem. We had both been there over 12 hours. Sam had to get some rest before doing this important, unforgiving and custom bit of work. And this was wise, too, because when you are tired it is too easy to make a mistake. So Sam was going to go home and sleep to 5AM.

For my part, I just slept on the wall-to-wall carpeted floor in the sysadmin room. The carpeting was there because this was the floor with the trading room, not because the company cared about the comfort of sysadmins sleeping on the floor.

At about 3AM I hear the locked door open. It was the night guard who said he was suspicious because he heard some heavy snoring. I assured him I heard it, too, and was certain it was coming from the next room over.

Greg Chatin, Computer Programmer

As a friend of Greg Chaitin, and an avid follower and reader of his writings, one thing that strikes me is that there is very little mentioned about the 25 years or so that he spent at IBM, most of it at IBM Research. Perhaps the situation is analogous to that of Archimedes. His mathematical works are well preserved, but we don't really know the extent of his considerable practical engineering and invention.

Greg started out as a IBM Service Engineer in Argentina and I came to know him when he worked at IBM Research.

We worked on a compiler for IBM's first RISC, then called 801. It later became known as the POWER architecture.

The work he did on that project was first rate and I learned a lot from him about writing code. Some of this is described below. In addition, he used insightful algorithms. You can find a little bit about his work on the register allocator in a couple of papers listed in the references below. These papers have become the standard references whenever something describes register allocation by graph coloring, which is called Chaitin-style graph coloring.

The papers convey the elegance of the idea but, having studied the working code as it appeared inside the compiler, I don't think they do justice to the underlying engineering or volume of output Greg produced when I had the pleasure of working with him on that project.

It is surprising there is not more about this period in biographical or autobiographical writings, and I would like to address that here.

A little background.

I started working at IBM Research right after getting a masters' degree. Although I had worked as a programmer and written code in school and on part-time jobs, this was my first full-time job at any large company. The project I worked on was an optimizing compiler. The code was hundreds of thousands of lines long, written by maybe a dozen people over several years. This kind of scale is still true for many compilers and interpreters. The "optimizing" part of "optimizing compiler" means that clever techniques are applied to make the code run fast.

Needless to say in any such endeavor of this size there is bound to be a lot of chaos. Here is an example that I encountered in watching a bug get fixed.

Before Greg's involvement, after the input source program was parsed, information was mostly managed in expression trees. The structure for an expression node could have a number of fields.

The compiler also consisted of a number of "phases," as found in many compilers. In particular, the phases were:

1. A parse phase
2. A code-improvement phase
3. A register-allocation phase
4. A final assembly to machine-code phase

One day, a bug turned up because a field that was needed in the final assembly phase got mangled. That field was called "ppb" for "phil's private bits." Phil worked on the parse phase.

Peter, who worked on the code-improvement phase, occasionally needed to store a bit of information for his purposes. He figured that since his phase came after Phil's phase, he could do a little code optimization of his own and reuse "phil's private bits" since, as he put it, Phil wasn't looking. That way he wouldn't have to add another temporary field which would make the expression-tree structure larger and more unwieldy. So "phils private bits" became "peter's private bits" when Phil wasn't looking. 

Unfortunately Hank, who worked on the final assembly phase, needed the information stored in phil's private bits. But this had been destroyed in the code-improvement phase by Peter.

Greg's phase, affectionately known as Phase III, was the register-allocation phase. When I studied Greg's code in order to take over the phase, I was amazed at how clean, efficient and elegant it was. Not at all like phil's/peter's private bits. At one point, I thought I could come up with an improvement, only to find that, even though it hadn't been mentioned in the papers describing the code, it was already implemented.

The first thing that struck me about Greg's code was that it followed the kind of formalism found in physics equations. Physics often uses short names in a particular domain. For example E, m and c in E = mc^2; this code was like that. There was a short and consistent notation for program variables that represented all registers, another for a single register, an idiom for looping over a register, and so on. This was very different from the rest of the code base.

In contrast to the looseness described in the Phil/Peter bug, there was strict consistency, clarity, succinctness, and focus on purpose which was applicable generally. This was the first time I had encountered code that exuded a sense of art rather than the typical robotic engineering that I was familiar with and pervaded the rest of the code.

As I said, the algorithm behind the register allocation can be found in a couple of papers. However what you won't find printed anywhere is a description of the excellent engineering that made this work. So I'd like to describe this next.

To support working on the register-coloring algorithm, Greg invented a register-transfer language and its display format. It has a couple of features not found in many register-transfer languages, such as a way to indicate that an instruction has more than one output. For example on the IBM System/370 the a 32 bit multiply is done, the output is stored in a pair of registers when the multiplication completes. So in the register-transfer language, both of these registers were on the left-hand side of an assignment statement. If an instruction, like 'add', set a condition bit such as 'carry', that bit was indicated as an output as well. When a register went

"dead"—the last time a value from that register was used—it had a tick mark after it.

All of this was needed in the register-allocation process, and the concise and precise language Greg invented for this purpose made it very easy to understand what was going on and how registers got allocated or "colored."

Nowadays, it is not uncommon in compilers to transform the tree representation that comes out of a parser (Phase I) into a register-based representation. But circa 1982 it was not a widespread practice. Greg's work predates the invention of Single Static Assignment, or SSA, which followed shortly afterward at IBM, presumably under Greg's influence. Both promoted the proliferation of register-based intermediate languages.

Greg's work was contemporaneous with the first edition of the Dragon Book which popularized this idea. (The first edition described a register allocation scheme that was suited for stack-architecture CPUs. Later editions of this book switched to describing the register allocation scheme that Greg developed.)

Compare this with the earlier books by David Gries or McKeeman et al. In the 1970s, there were several compilers that worked off of a tree representation. Some interpreters, such as those for Lisp, Perl5, or Korn shell, still work off of expression trees.

When Greg joined the project, the final assembly phase, Phase IV, used the expression trees built in Phase I. There was quite a bit of engineering and coding that Greg had to do to convert the expression trees into his register-transfer language. But he also had to do the work to convert it back again into the expression trees after coloring so that Phase IV could do its work! That idea that you could wall yourself off from the surrounding chaos blew my mind, but by doing so Greg had created a little corner of the world with order in the middle of a big mess.

Eventually, Greg convinced Hank, the person who worked on the phase after Greg's phase, to use his more elegant register-transfer language to drive final assembly. Hank told me that in doing so, his code got a lot better, simpler, shorter, and easier to maintain.

I have used the technique of coming up with an elegant or more appropriate representation and language in which to think and describe things in, even if it means writing additional and messy engineering code to do transformations both into and out of your language. Each time this comes up there is great benefit. I am grateful to Greg for adding this technique to my programming arsenal. In algorithms you sometimes find this idea in conjunction with, say, the Discrete Fast Fourier Transform. However it is not something usually mentioned in conjunction with programming.

This kind of elegance and simplification was not an isolated incident, but part of the fiber of Greg's programming. Here is another example.

After this work was completed, Greg embarked on a totally different aspect of the compiler, the binder, sometimes called a linker. This is a piece of code that is needed after the source code is compiled into machine code and just before running or loading the code. It adjusts the code so that it can work in an arbitrary section of computer virtual memory.

To do this, the compiler needed to be modified so that it used only those instructions that were position-independent. Nowadays this is called PIC Position-Independent Code) In other words, certain instructions or instruction formats had to be avoided. These typically include the "absolute address" forms of  "load from memory" and "store from memory" instructions.

The first versions of Greg's binder started out by more or less following what some other clever IBM Researchers, notably Chris Stephenson, did to perform this process on conventional IBM System/370 hardware. In the beginning Greg probably knew no more about this process than any casual programmer would. He finished a preliminary version of the code which worked and reflected his understanding of Chris's code for the 370. It was probably more or less a translation of that code adapted to the new hardware instructions and architecture.

However after this was completed and working, he had an epiphany—the process of binding together a set of machine code fragments could be thought of as a data-dependency driven process which uses a topological sort to pull everything together followed by a garbage-collection process that removes those parts of the code fragments that are bundled in "object decks" but aren't needed for the particular program that is being bound together.

At this point he discarded his thousands of lines of code and rewrote everything from scratch in a couple of days! I had never heard of anyone doing such a thing.

The end result was, of course a very concise and clean piece of code. Getting this code adapted into IBM's product engineering went pretty quickly. Most of the things that came out of IBM Research needed significant rewriting before they would be considered by the Product team, but this was not the case here due to its usefulness, simplicity and elegance. It became known as IBM's TOC binder.

One other incident in conjunction with the TOC binder attests to its unity of purpose and simplicity.

One day I happened to be in Greg's office. Usually, everyone worked on a bullpen that had 6--8 terminals in it. (Unix was developed in such an environment, too.) Before the days of IRC and Slack, a bullpen like this had the same effect. If you had a problem, you'd just blurt it out. Someone might hear you and quickly tell you what was going on.

However we also had private offices. Greg's office, like his code, was free from clutter. It consisted of a single terminal on a desk otherwise empty except for a phone. A single framed picture hung on the wall. Nothing more.

I had dropped by there and Dick, who was working on a database for our operating system, came in to describe a problem he had encountered. Just as Sherlock Holmes would do, Greg faced Dick the entire time, listening intently and quietly. After that, there was a pause of about 30 seconds as he reflected. Then after asking a few questions, he offered a theory of what might be wrong. With a couple of clicks on the terminal he brought up a file, just one small part of the rather substantial code. In a couple of clicks more, he found the problem spot. The whole process took less than 5 minutes.

It was amazing to me that he figured out the bug completely without having run the code even once. In order to verify that his theory and fix were correct, the code had to be run, but that came later. This kind of bug fixing is something I have rarely encountered. The fact that something like this can be done at all I attribute to the simpleness, cleanness and overall good organization of the code. He could reason about its behavior without having to run it.

My last example of clean concise code is published.

Greg wrote computer simulations for five well-known physical models:

• a satellite going around the Earth according to Newton,
• propagation of an electromagnetic wave according to Maxwell,
• the same satellite going around the Earth according to Einstein,
• an electron moving in a one-dimensional potential according to Schrödinger, and
• sums over all histories according to Feynman.

Each of these simulations fits on a single page of APL2 code. This includes setting up the data and drawing plots of the output. In fact this code was so short and elegant that for a while one of the 30-line simulations replaced the picture that had been hanging in his office.

References

• An Overview of the PL.8 Compiler
• Register allocation via coloring Computer Languages Volume 6, number 1, 1981
• Register Allocation and Spilling via Graph Coloring
• A Computer Gallery of Mathematical Physics--A Course Outline
• Live variable analysis
• Position-independent code
• Principles of Compiler Design, 1st Edition
• Compiler construction for Digital Computers
• A Compiler Generator
• Overview TOC AIX
• Part of a small discussion where I mention this part of this story