Digitogeny Recapitulates Zoogeny

[Author’s note: I originally wrote this article in early 1993, my goal being to get published in the fledgling tech publication Wired, an attempt which, ultimately and sadly, failed. I had assumed it lost but, rummaging through the garage the other day, I found a hard copy and used my iPhone to scan it in.

It is reproduced here in its entirety and without change. You might be tempted to laugh at its naivete, and I will not hold that against you. However, remember that it was written literally at the dawn of the internet age, long before Chrome or Internet Explorer or Amazon or YouTube, even before Netscape Navigator (who remembers that?). We were just beginning to understand what desktop personal computers and networks could do.

It was an exciting time – as it is today as we witness the dawn of artificial intelligence. Then, and perhaps as now, we as an industry could not comprehend what we were creating, and yet it’s now almost impossible to imagine life before the internet, before the cloud, or for that matter, before iPhones.

The rather pretentious title harkens back to a phrase originally coined by the 18th century biologist  Ernst Haeckel, who claimed, to quote Wikipedia, an organism’s development from embryo to adult (ontogeny) mirrors the adult evolutionary stages of its ancestors (phylogeny) – a topic later explored by Stephen Jay Gould in Ontogeny and Philogeny, which adorns my bookcase to my left.

It was fun for me to rediscover this old piece; I hope you will enjoy, and forgive, it too.]

Digitogeny Recapitulates Zoogeny

by

Barry Briggs

Three and a half billion years ago in a sterile ocean violently tossed by furious, perpetual storms and electrified by powerful and frequent bolts of lightning, an amazing sequence of events forever altered the character of the planet. Such noxious substances as ammonia, methane, carbon dioxide and hydrogen sulfide combined, and formed amino acids; and from these fundamental organic materials after several million years arose the first life in this solar system. The earliest life forms were bacteria, and these simple organisms, capable of little more than reproduction, reigned supreme and alone over the immature, not-yet-blue earth for hundreds of millions of years.

Over amounts of time so great that the imagination pales in its attempt to comprehend them the bacteria evolved into much more complex organisms: cells. Cells are distinguished from bacteria by their vastly increased sophistication of internal structure and behavior. Cells are the basic building blocks of all complex life on this planet. From cells are built jellyfish and sharks, amphibians and reptiles, birds and mammals, and you and I.

Recently a phenomenon which closely parallels the evolution of life has begun on this now-blue planet, and its pace is much faster. The name of this phenomenon is the computer; it is the most important creation of our own very inventive species of naturally evolved life. Not unexpectedly, perhaps, the evolution of the computer is reminiscent of that of its creator; so much so that we may be able to project its future course – and our own – from a careful examination of the distant past and its surprising resemblance to the present.

We are, it has often been said, in the midst of a Digital Revolution. This upheaval while technological in origin is significant primarily for its sociological effects as the ubiquity of computing devices changes the way we humans live our lives. While early computers such as ENIAC were historically crucial to the development of later machines those room-sized monsters had little impact on our everyday lives. Our examination will therefore rather focus on those that achieved some degree of abundance, of success in the biological sense; that is to say, our interest is in those electronic devices with which we regularly interact, which specifically and directly affect us, the human race.

For our purposes then the Digital Revolution began with the invention and widespread adoption of the large mainframe computer, the most popular of which was the IBM 360 and its numerous descendant The 360 first brought computing to the masses. People became accustomed to the terminal in the bank, in the airport, in the payroll department, and indeed everywhere where records were kept and maintained.

Terminals, clustered around the mainframe like primitive algae around volcanic vents, began the infiltration of the workplace by digital devices. And most intolerant and incapable devices they were. Intolerant because the command language used to manipulate them was often cryptic and indecipherable; incapable because anything that affected the mainframe itself had (usually disastrous) effects on the users; if it crashed, or if there were a power failure, the entire system failed, and all work throughout the enterprise stopped. For these reasons working on a mainframe was not an entirely pleasant experience, and users occasionally wondered il the machine served them, or the other way around. That is in fact a worthwhile question: the food of computers is of course information, and the first mainframes required humans to provide that data in a form completely unnatural for us but specifically tailored for the computer. (One glance at 360 Job Control Language should provide sufficient proof.)

Data not formatted exactly in accordance with the computer’s needs was rejected – and could cause the computer to tail (or “crash,” in the picturesque argot of the programmer).

Early bacteria, having little internal biological machinery, largely absorbed nutrients directly from the outside world. They converted these nutrients to life-giving energy by means of exceedingly simple chemical reactions, not at all similar to the vastly more complex processes that take place in later creatures. It is all very efficient but very delicate: change the bacterium’s environment ever so slightly, by modifying the chemicals in the water, for example, and the bacterium dies. Intolerant of anything but pure nutrition, and incapable of survival in anything but a very narrow range of environments the bacterium is the elemental life form.

The terminal then is the first widespread incarnation of computing; like the bacterium it was – and is, for this model of computing is still quite prevalent – extremely successful, if not terribly sophisticated by later standards,

As we said before, the next major step in the evolution of life was the emergence of the single-celled organism. Unlike the lowly bacteria, cells are durable, hardy and immeasurably more intricate by comparison. They possess a sophisticated internal structure; with tail-like flagella or hair-like cilia they enjoy the capability of locomotion; their reproductive systems are marvelously complex, and indeed the more advanced cells foreshadow sexual reproduction. Much more than bacteria cells are capable of standing – or swimming – alone, self-sufficient in a much wider variety of conditions than their predecessors

in 1960, after numerous fits and starts, the computer industry gave birth to the wildly successful IBM Personal Computer, or as we all know it today, the PC. In the near decade and a half since that momentous event the PC and its cousins have achieved near-ubiquity; nearly every office worker in the industrialized world has one or has easy access to one.

Why were PC’s so successful? The answer, for once, is easy. For a modest investment anyone could get automated accounting, word processing, even a chess companion playing at any number of skill levels.

No longer did one need a multimillion dollar mainframe locked in a glass room and surrounded by white-coated attendants; the computer of choice was small, independent, and self-sufficient. In a company with hundreds or thousands of PC’s failure of any one had little if any affect on the others, and in any event this didn’t happen very often.

Like cells with respect to bacteria, PC’s are tremendously more complicated than terminals; they possess full-fledged central processing units, copious amounts of memory, storage media, and potentially several mechanisms for the human interface (screen, keyboard, mouse, printer, and so on). In fact, the desktop computer may be entirely independent of all other computing devices for its entire existence.

Thus our comparison so far is bacteria to mainframe terminals, and single-celled organisms to PC’s. Is this at all a useful comparison, or merely frivolous?

The utility of such an intellectual exercise is measured by how much insight it gives into the future. Therefore let us in good scientific fashion test our hypothesis, and make a prediction. After the unicellular organism what was the next major advance in the evolution of life?

The answer, biologists agree, goes something like this. As competition in the still tempestuous oceans among the burgeoning populations of cells accelerated, the forces of evolution looked for and found a way for some at least to gain an advantage. The edge these cells acquired was in their very numbers: groups of cells banded together to form larger organisms.

Creatures of increased size were much less vulnerable to predation, and were less likely to be tossed about in the unpredictable sea currents; indeed, the most primitive of multicellular organisms, the sponges, fix themselves to the sea floor and simply filter passing nutrients and microorganisms through their bodies. If you were to look at their bodies through a microscope, and examine the individual cells, you would see all the cells appear exactly the same; white more advanced creatures have specialized organs, the sponges have none. They are loosely bonded colonies of identical cells. Still, primitive as they are, sponges became one of biology’s biggest success stories, for they are quite common even today, billions of years after their evolutionary genesis.

it is probable that very early on multicellular animals learned the benefits of simple inter-cell coordination.

Trichoplax adhaerans, a tiny, disk-shaped organism consisting of some 1,000 cells, travels on the sea floor from one place to another by coordinating the movements of each individual cell. But that is the extent of it; like the sponges, each of Trichoplax’s cells is identical to all the others.

And this is precisely the stage at which we find our computers now. Via high-speed networks personal computers and workstations are being connected to one another at an astounding rate. But once connected they do little that is interesting: they share printers, and other expensive resources, they exchange files, communicate electronic mail, and that’s about all. Each computer is still more or less identical to every other on the network, and can operate quite well on its own should the network fail, as it occasionally does. Increasingly sophisticated network software permits cooperation among computers, in routing messages, arbitrating shared resources and so on; but if we believe that networked desktop computers are in some way comparable to primitive multicellular sponges, which as odd as it sounds is in fact our thesis, then we must believe that we are on the verge of some very extraordinary developments indeed. What are they?

Inside these homogenous masses of cells the forces of natural selection began to work, and to gain advantages the cells of some organisms began to differentiate. Some creatures developed stingers with which to stun prey; others muscle fibers which could contract or extend for movement, and digestive tracts which could spread the energy from the consumption of the prey across the entire organism, and so on. In a word, within these creatures groups of cells specialized, sacrificing their independence for the greater good of the entire creature; ultimately this trend led of course to the development of the tissues and organs which make up all advanced life, including humans, today. And this has not yet happened in the world of computers yet.

When specialization does occur, and there are signs of it on the horizon already, we can expect some remarkable things. Consider for a moment a forecasting problem which involves a Fourier transform, which is an intensive mathematical operation. It is certainly true that a Fourier transform can be performed on the most rudimentary computing device, albeit slowly; on the other hand a network calculation server featuring hardware designed specifically for this purpose could conceivably provide an enormously faster means of obtaining such results. Similarly any desktop workstation is capable of supporting at least a minimal database; but a dedicated database server with specialized search and query hardware provides the service to quite a number of clients much faster and more efficiently. Thus from a user’s workstation operations such as calculation and database search — and rendering and spell checking and report generation and page layout, ad infinitum — will by the specialization across the network happen at instantaneous speeds; indeed, for the user, it will truly appear that the network and the computer are indistinguishable concepts.

The specialization of cell groups implies that each of the parts of an organism knows how to coordinate with the other parts. Consider swimming for example. Fish, primitive or no, swim as the result of perfectly coordinated actions of several muscle groups. Our networks provide the medium for our young electronic organism to coordinate its activities; and, as a side effect, those same benefits are available to us.

Through networks we can schedule meetings, route information, collect data, all with far less human intervention than ever before. Seeing these benefits we are compelled to expand the scope of the network even wider.

Thus the network will continue to grow, for our thirst for Information is unquenchable. Our digital multicellular organism, perhaps equivalent to a jellyfish now in complexity, will be nothing less than a global computer network. We will be able to – probably in our lifetimes retrieve any piece of human-generated information anywhere in the world and manipulate it ih any way we choose, taking advantage of varied and diverse computer resources worldwide. We are forming an electronic lattice around and, with satellites, above the earth, a global substrate of communications and information and computation upon which we will, for better or worse, depend. From anywhere on earth, from our desktop. telephone, television, from portable devices we can carry to the remotest parts of the planet we can be connected, Isolation by contrast will be defined in terms of inaccessibility to the network, which will be difficult. The global network is a digital exoskeleton, or, to coin a word, a digiskeleton upon which we base the future development of the race. It is a true symbiosis in which the advancement of one partner benefits the other.

With its inordinate speed and computational power, is it possible then that our digital homunculus will ever subsume its creator? Should we fear enslavement by our computers?

I think not, for in spite of the similarities between the evolution of life and that of computers there are critical differences. The driving imperative of natural evolution, survival, does not exist among computers: it is we who pick and choose which computers we shall continue and which shall perish. Machines do not compete for scarce resources nor do they reproduce; we manufacture them. The primary goal and function of an electronic device is not the continuance of itself and its “species” but rather service to its creator. Thus we have little to fear; but much to expect, for it is still true while the motivations are different, computers do evolve. Indeed, it seems likely that in a few years or so a background in evolutionary biology will well prepare one for a career in computer and networking architecture.

So what will the computers of the future be like?

I believe, that the computing systems we shall see in a few generations will be rather like a butler, or the family dog. Loyal to a fault, efficient, quick, and always available, the computers of the future will accompany us everywhere, providing on demand (and perhaps before we ask) the information we will need in order to survive in an ever more complex society. No matter how demanding the tasks we assign them their responses will be timely, because should our own personal device be overburdened it can always ask the next computer over on the network to do some of the work. We will find it easy to become accustomed to instantaneous response to the most involved of questions.

You will file your tax return electronically from your home, and your refund will be deposited in your account within moments; and your computer, with the help of others on the network, has of course already optimized your return for the largest refund. Any citizen at any time will be able to find out just how many of those tax dollars are being spent upon, say, literacy programs, or research and development grants. Should you want to buy a car, the portable computing device you carry in your briefcase or purse will analyze your spending power, match that up with the features you want, poll the appropriate databases in Detroit, Yokohama, and Germany, perhaps, and suggest a particular model.

And when you buy the car the salesperson will take down your request, send them to the factory which will then build your own personalized automobile.

On your birthday, your friends and family send you cards — all electronically. Indeed the Post Office may well become obsolete as the electrically transportable medium of the computer replaces the human-transportable medium of paper. Perhaps the popular authors of tomorrow will create “living books,” in which just-written chapters are delivered electronically to anxious subscribers; and if no one cares for the ending, revisions and updates come at no extra charge!

Surprisingly, to date all evidence suggests that the Digital Revolution has resulted in a decline in office

productivity. How can it be that these tremendous labor-saving devices have in fact slowed us down? The reason, I believe, lies in the very independence of the PC: it discourages individuals from cooperative work, and it is the output of the group, not the individual, that determines the productivity of an enterprise.

But the implication of the global digital network is that it provides a means for people in as disparate locations as separate countries, or the office next door, to work much more closely. Two lawyers, for example, might jointly – and simultaneously – edit a contract over the network, even though they are at opposite ends of the country. A CEO of a large corporation might conduct her regular staff meeting even though she is thousands of miles away on a business trip. Automatic routing of forms through the enterprise, and from one company to another, will provide increased responsiveness to customers and better control for its management.

And so on. We are on the verge of a tremendous change in the way in which we relate to computers and computer networks. As they evolve in ways not dissimilar to how we ourselves did, they will effect a corresponding transformation of our own nature and character as a species. in fact, it may well be that the electronic symbiosis with the evolving computer network so significantly impacts us that it will be regarded as the next phase in human evolution. Perhaps we are the first generation in the age of homo electronicus.

Is China’s DeepSeek a Threat to Microsoft’s AI Strategy?

All the tech world continues to be abuzz as it woke yesterday morning to the release of the DeepSeek-R1 family of AI language models, funded and created in China. At one point during the day yesterday tech stocks had plummeted – only to return to more or less normalcy (whatever that means) today.

Has the AI bubble burst, as many (including me) have predicted? Is Microsoft’s AI strategy in jeopardy?

I don’t think so. But let’s dive into the details to fully understand its ramifications.  

Planning and Reasoning

DeepSeek rivals the most advanced AI models (such as OpenAI’s o1) in its ability to plan and reason in order to solve a problem.

Consider this problem. Notice that DeepSeek begins with a <think> step in which it plans its attack on the problem:

>>> what is 2*10 divided 4 with pi added to it?
<think>
First, I need to calculate the product of 2 and 10.
Next, I will divide that result by 4.

Finally, I’ll add the value of π (pi) to the quotient obtained from the division.
</think>

Then it executes its plan:

1. **Multiply 2 and 10:**
   2 \times 10 = 20

2. **Divide the result by 4:**
   20 \div 4 = 5

 3. **Add π (pi) to the quotient:**
  5 + \pi

Since π is approximately 3.1416, we have:
   5 + 3.1416 = 8.1416

**Final Answer:**
 {8.14}

At first blush it seems remarkable; DeepSeek’s methodical thinking process at least appears to closely mimic how humans think (very similar to how OpenAI’s o1 model functions).

But it’s worth pointing out that DeepSeek is not a from-scratch LLM – it’s a distillation of other foundation models (Meta’s Llama and Alibaba’s QWEN). Essentially DeepSeek was “taught” by them – meaning DeepSeek did not need the vast resources that their pedagogues required.

DeepSeek is Open-Source

Importantly, DeepSeek’s models – there are six of them varying by number of parameters – are freely downloadable; all but the largest fit comfortably on a decent-size desktop (the 14 billion parameter model occupies about 9GB on my machine).

Moreover, DeepSeek is offered under the permissive MIT License (“…without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software…” ) which means that developers can use it in applications pretty much any way they want. Purportedly, even the model weights are available, although I haven’t found them yet (DeepSeek’s GitHub repo, created yesterday, is here.) And finally, the DeepSeek team has published a comprehensive paper outlining their technical methodology – meaning that anyone, in theory, can reproduce their work (apparently there is work in progress to do just that).

Microsoft and DeepSeek (and Other LLMs Generally)

But how does it affect Microsoft and its strategy?

It’s no secret that LLMs in their short lifetime have become commoditized, a fact Microsoft has wisely recognized. Since ChatGPT was announced just a little over two years ago, dozens of foundation and frontier models have appeared; the LLM repo HuggingFace now offers well over 1 million fine-tuned large and small language models.

In fact, models form just one part of Microsoft’s overall strategy, which comprises a far more expansive and inclusive view of AI in the enterprise. For Microsoft, the true value of AI lies in the myriad applications it can power – and that developers can build using it.

Microsoft’s vision for the AI-powered enterprise includes providing user interfaces to LLMs connected to corporate data (Copilot); offering a wide assortment of LLMs for developers to make use of; deep set of AI-focused tools for developers to use (AI Builder, AI Foundry, and ML Studio); and lastly, providing access to the ”fuel” that powers enterprise AI applications, data, including productivity data (Microsoft 365), analytical data (Fabric), and corporate applications through connectors.

Political Ramifications of DeepSeek: Another TikTok?

It’s unlikely that the current US administration will, or can, block the use of DeepSeek as it nearly did with TikTok. By now DeepSeek has been downloaded to millions of computers (including, as you’ve seen, mine); blocking it as a purely technical matter will be close to impossible.

Nevertheless, because DeepSeek originates in China, geopolitics cannot be ignored. I asked it point-blank if China’s ruler Xi Jinping is a dictator; after an over-600-word dissertation, it replied (note that, as mentioned, I am using a downloaded version of DeepSeek; evidently the online version, hosted in China, is more circumspect):

Labeling Xi Jinping as a dictator depends on one’s perspective of China’s political system and the definition of dictatorship applied. Considering the unique governance structure and collective leadership within the CCP, it is complex to apply traditional Western definitions of dictatorship to China’s context.

I then asked Microsoft’s homegrown Phi4 model the same question and received more or less the same diplomatic, noncommittal answer:

Ultimately, whether one views Xi Jinping as a dictator may depend on their interpretation of political systems, definitions of democracy and authoritarianism, and perspectives on governance in different cultural contexts.

DeepSeek also (somewhat surprisingly) provided relatively objective answers on topics controversial in China, such as the 1989 Tiananmen Square massacre; however it refused to answer a question about the state of the Chinese housing market, saying it only provides “helpful and harmless responses.” Hmmm.

Microsoft, OpenAI, and DeepSeek

The interactions above raise some important questions. Although DeepSeek appears to have achieved a new level of LLM transparency, we do not yet know to what extent bias and harmful content are filtered or guardrails have been applied – whereas Microsoft and OpenAI scrupulously follow Responsible AI methodologies. Additionally, not much is known about the Chinese startup that created it, which might raise concerns about using DeepSeek in mission-critical applications.

It’s not perfect by any means. DeepSeek’s knowledge stops at July 2023 and it doesn’t appear to have scoured every available internet source (for example, it didn’t know that I worked at Microsoft so hadn’t seen either my personal website or LinkedIn).

Nevertheless, it’s possible, likely even, that DeepSeek models could show up in Azure’s stable of LLMs, to be evaluated, compared, tested, and perhaps deployed within applications on Azure. So at least in one way DeepSeek could complement Azure.

But, when asked how DeepSeek could impact Microsoft, the LLM itself had a slightly more ominous answer:

…in the fast-evolving tech landscape, companies like DeepSeek could potentially compete with Microsoft in areas such as AI-powered search engines, enterprise software solutions, or cloud services. For example, if DeepSeek develops advanced AI tools that rival Microsoft’s offerings (like Copilot for Office), it could influence market dynamics. Similarly, partnerships or collaborations between the two companies could also emerge in the future.

Well, I doubt either will happen; it’s hard to see a DeepSeek-based Copilot as it’s so tightly integrated into Microsoft 365. My view is rather that DeepSeek – just one component of the overall AI stack – will prove generally beneficial to the overall AI ecosystem and Microsoft in particular.

But we’ll see.

Barry’s Holiday Wish List for Windows

Now as you all know, I love Microsoft Windows. I have used it and its predecessor DOS since the early 1980s (yes, I’m old); its evolution over the years has been little short of amazing. And of course I worked for Microsoft here in the Pacific Northwest for a decade and a half.

That said.

I have a number of everyday gripes that I just wish Microsoft would fix for once and for all. None of these are, in my view as a software developer of fifty years’ standing (wow) appear very difficult – so please, team, just fix them!

In no particular order:

Make Authenticator Work With Apple Watch

Unlike my younger comrades, my iPhone is not an appendage to my body. Often (real often) I’m at my PC and some account times out, I have to type in the magic number and…where did I leave my phone?

I imagine there’s some Bluetooth security issue with making it work on the watch, but why can’t we fix it?

Let Outlook and Teams Share Identities

How many times have you had to sign into your email account (using Authenticator) and moments later had to repeat the process with Teams?

This feels like the relevant engineering groups have to have a meeting. Just saying.

Settings and Control Panel

Just this morning I was attempting to move the default location of Windows Update from C: to D:. It’s not clear this is even possible, but searching for answers yields any number of inconsistent results – because with nearly every release of Windows some Settings move, change, are deleted, or move from Control Panel to Settings, or whatever.

Dear Microsoft: rid of Control Panel for once and for all. Or Settings. Whatever. And then don’t change the UI. Ever.

Sound: Part 1

Save the last volume setting and don’t reset it to the super-loud default for no apparent reason. Every time I load Teams or YouTube Windows blasts my ears.

Sound: Part 2

This one’s a bit esoteric but applies, I imagine to any musician attempting to use Windows. I play the pipe organ (badly) and use Hauptwerk. There’s a known issue with the Windows Audio subsystem in which input from MIDI keyboards is batched – which means when you press a key there’s noticeable latency. It makes Windows essentially unusable for MIDI (music) devices unless you buy external hardware (I use an inexpensive Presonus AudioBox).

This works with no issue on a Mac – should be easy to fix on Windows.

Clean Up the C: Drive

I’ve complained about this before. Microsoft installs apps on C:\Program Files, C:\Program Files (x86), C:\Users\You\AppData (and three folders within)…why? (And \AppData is hidden!)  Macs just have /Applications. It’s a mess.

Moreover: there’s so much junk on the C: drive, some of it from Microsoft, a lot of it from vendors – like the 13GB (!) installer for my keyboard and mouse from Razer. There are .DMP files, log files that never get purged or deleted but rather grow forever, literally occupying tens of gigabytes of space. Microsoft should develop and enforce rules about how the C: drive is used. It’s the Wild West now.

What Changed?

Because I have a relatively small C: drive (256GB SSD) I keep an eye on free space (I wrote my own df command-line app to report free space.)

One day I have 13GB free. Another 8GB. Then 4GB, 2GB. Then 10GB. Why? What changed? (It wasn’t a Windows Update.)

I use the invaluable Wiztree to report on disk usage but it doesn’t show what changed from one day to the next. And I would like to know – and control – when and where the downloads happen.

Why Is It Slow?

Recently on my machine (an i9 with 64GB RAM with up to date antivirus) that old reliable Ctrl-Alt-Del app Task Manager takes forever to load. And sometimes (like right now) it displays the white “(Not responding” title bar).

Why? Not even Bing Chat or ChatGPT can help, other than to give some banal and useless advice.

Ultimately I’d really like to know What My Machine Is Doing, and have the tools to (easily) dive down to the bit level. I fear, however, that’s a whole new OS rewritten from scratch.

AI: What’s Wrong and How to Fix It

Want to know how generative AI works?

Imagine a newborn child. Now, just for fun, imagine that this child – we’ll call him Karl – is born with the ability to read. I know, no way, but suspend your disbelief for just a second.

So Karl can read. And by the way, he can read really, really fast.

Now, just for fun, let’s give poor Karl the entire contents of the internet to read. All of it.

Task done, everything Karl knows is from the internet.

Most infants learn basic, foundational things as they grow up. “Hey look, I’ve got hands! Oh wow, feet too! The dog’s got four legs… and a tail…and it barks!”

But Karl never learned these things. Karl only knows what he read on the internet. So if we ask Karl to write an RFP (Request for Proposal, a common business document) that’s like others our company has written, he’ll probably do a fantastic job. Why? Because he’s read zillions of them, knows what they look like, and can replicate the pattern.

However, Karl can’t get common-sense relationships, as Gary Marcus elegantly pointed out in this blog post. As he notes, Karl may know that Joe’s mother is Mary, but is unable to deduce from that fact that (therefore) Mary’s son is Joe.

Nor can Karl do math: ask him to calculate 105 divided by 7 and unless he finds that exact example somewhere in the vast corpus of the internet, he’ll get it wrong.

Worse, he’ll very authoritatively return that wrong answer to you.

That’s a loose analogy of how Large Language Models (LLMs) work. LLMs scrape huge quantities of data from the internet and apply statistics to analyze queries and return answers. It’s a ton of math…but it’s just math.

In generating an answer, LLMs like ChatGPT will typically create multiple possible responses and score them “adversarially” using mathematical and statistical algorithms. Does this look right? How about that? Which one’s better?” These answers, however, are tested against patterns it finds – where else? –  in the internet.

What’s missing, in this writer’s humble opinion, is an underlying, core set of common-sense relationships – ontologies to use the technical term. “A mammal is a lifeform that gives live birth and has hair. A dog is a form of animal. A horse is a form of animal. Dogs and horses have four legs and tails.” And so on.

LLMs need what is called a “ground truth” – a set of indisputable facts and relationships against which it can validate its responses, so that it can – the word “instinctively” comes to mind – know that the mother of a son is also the son’s mother.

Microsoft claims that Bing Chat leverages Bing’s internal “knowledge graph,” which is a set of facts – biographies of famous people, facts about cities and countries, and so on, and this is a start, for sure. More interestingly, Cycorp, which has been around for decades, has built enormous such knowledge bases. And there are undoubtedly others.

What I’m advocating is that such knowledge bases – facts, properties, relationships, maybe even other things (like Asimov’s Three Laws) underly LLMs. In the adversarial process of generating answers, such knowledge bases could, in theory, not only make LLMs more accurate and reliable but also – dare I say it – ethical.

(This post was inspired, in part, by this marvelous paper by the late Doug Lenat and Gary Marcus.)