It seems to me…
Most people consider product development, especially in the areas of science and technology, to be relatively logical. They have read or heard some products can be attributed to accidental or synergistic circumstances but tend to discount the overall extent from which such development actually has resulted. It is true that product development occurs after a period of preparation, research, and related effort permitting recognition of some achievement which otherwise would remain rarely recognized without such sufficient preparatory effort, but a significant proportion of all development, while perhaps related, was still unanticipated in the initial planning process.
Having been either directly responsible or involved in the development of quite a few products, there are several examples which might provide some insight into the somewhat circuitous process in which they came about. While these might not be representative of the majority of available products, they do illustrate what is not untypical. These examples all occurred a number of years ago so do not apply to any currently on the market. I’ll only cover the first example here; other examples will be in another in other blogs…
Around 1969, I was one of three lower-level managers responsible for the design and development of a complete revision of an operating system for what at the time was a leading computer manufacturer. The operating system development fell far behind schedule, eventually was cancelled, and most of the developers working on it were transferred from Sunnyvale, CA, to Minneapolis, MN. I had moved from Madison, WI, about a year prior and for personal reasons did not wish to move back to that area of the country.
The concept of computer databases was relatively undeveloped at the time and I was asked if I would be interested in working on a research project to determine the feasibility of producing a marketable product in this area – YES! The initial design proposal was in BNF (Backus-Naur Form) and dependent upon the CODASYL database recommendations in that it used a separate DDL (Data Description Language) for data design. It almost was rejected as being too academic.
When asked how soon I could have something to demonstrate, I said if I did it by myself – six months. Any additional people assigned to work on it would lengthen that by an additional six months – I had read Fred Brook’s laws of product development in his book The Mythical Man-Month. Three of us were assigned to work on it and it took almost two years.
Rather than being a general product, it was intended to be a standalone product for non-programmers; a concept somewhat unique at that time. I would ask secretaries, salespeople and anyone else how in their language they would try to enter or retrieve data. Bits and pieces of the program would be torn out and replaced in what now would be considered rapid prototyping. It quickly turned into what can only be described as an ugly “hack”.
One day a person from the sales department came into my office along with three or four other people I did not know and said he heard I was working on a kind of database thing. Would I mind showing it to them?
After seeing it, they thanked me for my time and left.
About an hour later he returned and asked how quickly I could load it on a magnetic tape (remember this is 1970…).
“We sold it.”
“You’re crazy… It is a hack. It would take at least a month to even begin to clean it up.”
“It has to ship this afternoon. And, by the way, what is it called?”
“It doesn’t have a name. It’s only a feasibility project.”
“What does it do?”
“You can query and update data stored in a file.”
“We’ll call it Query Update. No – QU has a better sound.”
He left my office but ran back in about ten minutes later: “Query Update. Otherwise the second version would be called “Q You too”.
It shipped and became the basis for an entire successful database product line. It also had the worst code I ever worked on and always was embarrassed to admit my involvement with it.
I was teaching an evening graduate-level Information Theory class at San Jose State University at the time and included a number of concepts in the course material from E.F. Codd’s seminal 1970 article titled “A Relational Model of Data for Large Shared Data Banks” published in Communications of the ACM. Quite frankly, at the time I considered the entire concept impractical. While it constituted great theory and provided us with a way to look at data from a mathematically logical perspective, the cost of ever implementing it would be impossible. (Which just goes to show how limited my imagination was…)
I’m not aware of any commercially available product ever having implemented all of Codd’s ideas for a relational algebra or calculus. Some operations, such as a logical divide, apparently are not even necessary. Still, we realized his concept of a logical union could easily be incorporated in our product so it probably was the first database product marketed incorporating any relational features – at least any product of which I am aware.
The second version of the product seemed like an excellent opportunity to cleanup the code as well as to incorporate some features customers had requested. While I did not like the way the first version was implemented, I took my time and carefully started writing some of the basic modules. One of the first modules was a relatively complete stand-alone math package. It was product independent and used stacks (unusual for that time on our computer) for token analysis and calculation. It also was faster than any comparable math package in any of our other products and I considered it to be about the best software I ever had written.
Unfortunately, I was asked to work on a critical distributed operating system implementation for the United Bank of Switzerland. It was a great project but I normally worked from around 7PM in the evening until 7AM the next morning so had very little contact with any of the database developers. The next project I was assigned to work on was an implementation of the PL-1 programming language. PL-1 was a procedural imperative computer programming language designed for scientific, engineering, business and systems programming applications that was considered by many at the time to be the next big programming language (but I have not heard anything about it for quite a few years). That math package I had written seemed like a logical piece of code to utilize but when I returned to the database group to get a copy of it, found it had been discarded and deleted. Yes, it was faster and required less code than what they were using and, no, they had not found any errors in it – but no one understood how it worked and they did not know how to get hold of me. So much for some of the best and worse code I think I ever have written.
That’s what I think, what about you?