I Learned About * from That: A Meditation on Bugs

By Barry Briggs
Second in a series: previous

[Programming note: I have changed the title of this series from “I Learned About Programming from That” to “I Learned About * from That,” the idea being I’ll probably talk about lessons learned not only regarding programming but also lots of other areas – in computing – as well. * (star, or wild-card) as I’m sure most of you know means “anything and everything” in computer-talk. Everybody got that? Now back to your regular show.]

So, with just a smattering of theory and a tiny bit of FORTRAN training under my belt, I somehow got hired to work at Goddard Space Flight Center.

Yep.

[What’s FORTRAN, you ask? It’s one of the earliest high-level languages, targeted primarily at scientific and engineering applications. It stands for FORmula TRANSlation and it’s always capitalized. Back then it was one of the two more popular languages, the other being COBOL, a business-focused language.]

And with my extensive background in computer science, where did I get assigned? To write operating system-level code in assembly language for the giant Univac 1100 mainframe that controlled communications for the first Tracking and Data Relay Satellite (TDRS, pronounced tee-dress; see the image below).

Fortunately, one of the great things about youth is you often don’t know what you’re not supposed to be able to do, and so I dove in. (And there’s a lesson right here: be FEARLESS!)

But…talk about being at the wrong end of a firehose! Our little team was charged with writing an entire transaction processing monitor (a pretty new concept at the time) for the applications group whose apps sent and received commands and data to the satellite (which controlled communications to the Space Shuttle). Their apps involved positioning it correctly, connecting to ground stations, correcting errors, and all kinds of stuff I don’t remember.

Moreover, the topology of the computing environment was nothing if not complex: ; it connected to a bunch of Varian minicomputers which did preprocessing and a bunch of PDP-11s which did reporting (I think). The mainframe itself was a multiprocessor, so we had to use semaphores, test-and-sets, and locks; daunting stuff for a very junior programmer! And all our code ran at the highest privilege levels which means it was pretty easy for us to bring the whole kit and kaboodle down if we had bugs.

Which I did a few times, early on.

On the other hand, there were some cool aspects. Everything was open-source, so if you wanted to know how the operating system’s task scheduler worked, you could just go look – in the multi-thousand-page printout.

Because millions of dollars and for that matter, lives (of astronauts), depended directly or indirectly on our code, we were very careful. Every line of code that was written was reviewed. Let me tell you, those first few code reviews were not a lot of fun for that young newbie that I was.

Eventually, however, I got pretty disciplined in writing and testing my own code, and eventually got to the point where I flew through the code reviews.

The thing was, no one complained about them, really, because we all knew that so much was riding on what we did.

Commercial Computing is, or was, Very Different

It was a big surprise to me, then, when I entered the world of commercial computing, that programmers had rather a more relaxed view toward bugs. In fact, I was quite shocked that we’d have serious discussions (and am to this day) about how many “severity-1” bugs we could ship with!

Working on one major commercial project – a release of Lotus 1-2-3 – we were actually awarded tchotchkes like the one shown for the number of bugs we programmers fixed. You can imagine that this was a pretty easy metric to track!

However, somewhere along the line someone realized that there were fundamental things wrong with the development process:

  1. In the early phases of the project, developers were rewarded for a feature working – even if just barely – in other words, in prototype quality.

  2. Later in the project, developers were rewarded for patching the prototypes they’d created early on.

  3. And (therefore) just maybe it would be better to reward developers for checking in clean code in the first place. Not as easy a metric to track, but nevertheless rewarding the right behavior.

Automated Testing: Present at the Creation

How did our QA teams find all these bugs? Of course, back in the Jurassic we had armies of hardworking folks doing feature testing – taking a build from us developers each week and trying out each feature to see what worked, what still didn’t, what worked last week but didn’t this week, and so on.

Well, the product of course had a macro facility. Somebody realized along the way that we could use the product to test itself! And so our test teams moved from being feature testers to macro experts, and perhaps this was the birth of the profession we now commonly call SDET – Software Development Engineer in Test, for the uninitiated. (For historical purposes, the person who came up with the original idea was the late Mike Kleinert; the person who refined, drove, and scaled it was the young woman who later became my wife.)

The Purpose of Testing

Over time I realized something which I think is commonly understood now, which is that the purpose of testing is not, in the final analysis, to find bugs, although that is certainly a core part of the job.

In fact – and incidentally I use this as an interview question – the function of test is to determine the product’s readiness to ship. When leading software development projects, therefore, I never asked the architect or the developers if the product was ready (you’ll either get “I’m sick of this damn thing, ship it” or the perfectionist “I have to refactor everything first”).

Instead it’s the test team that is, or should be, closest to the customer/user and their experiences. So at the end of the day they have the final say. And in my view the head of test has the single most important decision-making power in the entire team: when we’re done!

That’s it for this installment. I have lots more in the queue, so watch this space. And I look forward to your comments!

I Learned About Programming from That: First in a Series

Barry Briggs

I recently experienced the completely horrifying realization that I’ve been writing code for half a century. It occurred to me that over the years I’ve actually learned some stuff possibly worth passing on, so this article is first in an occasional series.

I’ve had the privilege (and it has been a true privilege!) to meet and work with many of the amazing individuals who shaped this remarkable industry, and to work on some of the software applications that have transformed our world.

It’s been quite a journey, and over the years I’ve come to realize that I started it in a way that can only be described as unique; indeed, I doubt that anyone began a career in the software industry in the same way I did.

So the first edition of “I Learned About Computing from That” is about how I actually first learned about computers.

Early Days

I wrote my first program in BASIC in high school on a teletype (an ASR-33, for old-timers) connected via a phone modem to a time-sharing CDC mainframe somewhere in Denver. As I recall it printed my name out on the attached line printer in big block letters.

I had no idea what I was doing.

In college, I majored in English (really) but maintained a strong interest in science, with several years of physics, astronomy, math and chemistry, even a course in the geology of the moon. Yes, it was a very strange Bachelor of Arts I received.

In short, I graduated with virtually no marketable skills.

Luckily, I found a job with the federal government just outside Washington, D.C. where I was almost immediately Bored Out Of My Mind.

However, I soon discovered a benefit of working for Uncle Sam: a ton of free educational opportunities. A catalog listed page after page of in-person and self-paced courses, and thank God, because I desperately needed something to stimulate my mind.

Digital Design

Having no idea what the subject actually was, other than it sounded Really Cool, I chose a self-paced course called “Design of Digital Systems,” by PC Pittman (original publish date: 1974).

This utterly random decision changed the course of my life forever.

These six paperback booklets (I still have a few of them) began with different number bases (binary, octal, digital) then introduced the concept of Boolean logic – AND and OR gates, and so on. It was hard!

After that it covered half-adders, more complicated gates (XOR) and eventually got to memory and – gasp! – the idea of registers.

Finally, it described the notion of CPUs and instructions, how you could actually have a number stored in memory when loaded by the CPU would cause another number to be fetched from memory and added to the contents of a register, leaving the sum in the register: the essence of a program! (To this very day!)

O the light bulbs that flashed on for me!

I suddenly got it, how – at a foundational level – computers work, and how they could be made to do useful things. For me at that point, everything else – assemblers, high-level languages (I took a FORTRAN programming class next), operating systems – fell out from this very profound gestalt of computing.

And I realized I loved this stuff.

They Teach Programming All Wrong Today

These days in high school students can take “computer science” classes which, at least in my son’s case, turned out to be nothing more than a Java programming class. Imagine being thrown into the morass of Java – classes, namespaces, libraries, variables, debugging, IDEs – with no understanding of what the heck is actually going on in the guts of the computer itself!

Completely the wrong approach! Guaranteed to confuse the hell out of a young mind!

As accidental as it was, I highly recommend my approach to learning the art of programming. Teach kids the logic underpinning all of computing first. By developing a deep understanding of what’s actually going on at the digital logic level you develop an intuition of what’s going on, which makes so much easier to write code and to figure out what’s going wrong when it breaks.

When I talk about how computers work, I start with a faucet (transistor) and tell people if you get how it works, you understand how computers work.

String a couple of faucets together and you get an AND gate. You get the idea. And so on and so on.

Anyway …

That’s how I got started. Shortly after my introduction to Digital Design I wound up taking (more formal) courses at George Washington University and Johns Hopkins (again courtesy of the US government) and not long thereafter this English major (!) found himself programming mainframe OS code for NASA (believe it or not).

Where I learned some very important lessons about coding. That’s for next time. Stay tuned!

A Little Family History

This story is about William Charles Newstead who was born around 1834 in the Norwich area of England. Sometime prior to 1840 his father, also named William, brought his family to the United States. They settled in a small town called Burke, New York in Franklin County, in the very far north of New York State on the Canadian border. The industries of the time, besides farming,  included “only a grist mill, saw mills, tanneries, asheries, starch factories, brick yards and stone quarries.” We know the rough time of the family’s arrival as the Newsteads are listed on the 1840 Federal Census.

I cannot imagine what brought them to the small village whose population at the time was about 2000 people. (Today it’s less than 300.) Perhaps the Newsteads had friends or relatives already there; or perhaps there were the promises of land grants for new settlers. In any event the Newstead family set up as farmers in the late 1830s.  William’s parents no doubt expected to raise their family in this quiet, peaceful town, watch them grow up and then enjoy their grandchildren.

But events were about to take a turn.

On April 12, 1861, South Carolina militia bombarded Fort Sumter, near Charleston, and the island fort  surrendered the next day. Thereafter newly elected President Lincoln called for 75,000 volunteers to help put down the incipient rebellion. These recruits were only expected to serve for 90 days, the thinking being that the war against the “bumpkins” in the South would be over very quickly.

Battle of Bull Run

A few months later, on July 21st 1861, when William would have been about 26 or 27, Confederate and Union armies clashed at the Battle of Bull Run, at Manassas Junction in Virginia, heralding the start of the Civil War (or as the South calls it, the War between the States).

At Bull Run, the Union forces were, to the shock of the spectators who had come to witness the event, decisively defeated; both sides recognized that the conflict henceforth would be a protracted affair. Calls for volunteers went up on both sides, and in the North recruitment periods were adjusted to three-year terms.

On September 28th, 1861, young William enlisted in the Union Army and was assigned to the 16th Regiment in one of the three-year recruitments. In those days regiments were defined by the states that they were formed in so William’s regiment was known as the 16th New York. (There were 16th regiments from other states as well.) The 16th New York was composed of volunteers from the most northern parts of New York State, including Franklin County, where Burke is located.  Regiments of the time comprised around 1000 men in 10 companies.

William probably underwent training, drilling primarily, for a few short weeks before being sent off to active duty (perhaps according to Hardee’s Rifle and Light Infantry Tactics, the standard work of the time, ironically commissioned by Jefferson Davis in 1853 and published in 1855; Davis, who was Secretary of War at the time, later went on to be President of the Confederate States of America. Hardee himself also joined the Confederate army as a lieutenant general; nevertheless the book was used by the Union as well).

Why did he join? Was it a sense of duty? Was it for the “bounty” (signing bonus) offered? Were all the boys of Franklin County joining up? Or was he an idealist perhaps desiring to help rid the country of the scourge of slavery? We’ll never know.

And the war was over slavery, much as the South for years afterward tried to claim otherwise. Even in the late 1960s when I lived in the Deep South it was still taught in school that the war was about socioeconomic issues or “states’ rights” and not about slavery (a false Yankee claim, they said); such were the textbooks of the time and the prejudices of the place.

Private Newstead joined his unit on October 5, 1861 a member of I Company, commanded by Captain J.J. Seaver, part of the famous Army of the Potomac. (Specifically: “it was assigned to the Second brigade (Gen. H. W. Slocum) of Gen. Franklin’s division. This brigade was composed of the Sixteenth and Twenty-seventh New York, the Fifth Maine, and the Ninety-sixth Pennsylvania, and was not subsequently changed during the period of service of the Sixteenth, except by the addition of the One hundred and twenty-first New York early in September, 1862.”)

Soldier of the 16th; note the straw hat

The 16th was known as the “straw hat men” because alone among Union units its troops wore straw hats – a gift apparently from a friend of the regiment.

The regiment overwintered just south of Alexandria, Virginia and did not see action until the spring of 1862. In April 1862 Slocum’s brigade, including the 16th, boarded a ship and sailed to Yorktown to take part in the Peninsula Campaign, an abortive attempt by the Union army to capture the rebel capital at Richmond. On Wednesday, May 7th, 1862, the 16th New York participated in its first battle at Eltham’s Landing (also known as the Battle of West Point) — really more of a skirmish.

The 16th retreats after the Battle of Gaines’ Mill

However, on the 27th of June 1862 the 16th New York was heavily engaged at the Battle of Gaines’ Mill — one of the bloodiest battles of the war yet not one of the better known. The Army of the Potomac went up against the bulk of the Army of Northern Virginia under the command of Robert E Lee and the result was a disaster for the Union forces. Private Newstead’s regiment lost about 230 killed, wounded, or missing in that engagement. A few days later, the 16th fought an inconclusive battle at Frayser’s Farm.

What must it have been like, leaving this idyllic village in northern New York State and thrust into the meat grinder where thousands of furious men in one uniform focused on nothing else but the wholesale slaughter of men in the other? Where the wounded lay screaming in the fields,  where doctors routinely amputated arms and legs in the most primitive of conditions, and where, after dark, crows, coyotes and wolves feasted upon the human carrion?

Field Hospital at Savage Station, June 30, 1862, following the battle
(Look closely to see the straw hats)

The 16th New York was next engaged at the Battle of South Mountain (Maryland), also known as the Battle of Cramptons Gap, on September 16, 1862. The 16th among other units was charged with dislodging a sizable Confederate force from one of three passes on the mountain. The 16th led the advance – a “brilliant dash” it was called –  and suffered 63 killed and wounded.

Battle of Cramptons Gap

History records it thus: “Though the Federals ultimately gained control of all three passes, stubborn resistance on the part of the Southerners bought Lee precious time to begin the process of reuniting his army, and set the stage for the Battle of Antietam three days later,” although the 16th did not participate in that bloody but indecisive battle. Lincoln, claiming a “victory,” announced the Emancipation Proclamation a few days afterwards.

During the mild winter of 1862 to 1863 the division participated in the famous “mud march” in which Union troops attempted to surprise the South by crossing the Rappahannock River; however because of bad weather and disagreements between the Union generals this attack failed.

Following the conclusion of the winter the 16th moved out with the rest of the division and joined General Hooker at Chancellorsville, one of the most important battles of the war and a decisive victory for the Confederates. At that battle, beginning on April 30th  — a Thursday –  and lasting through May 6th of 1863,  the 16th was positioned on the frontline on the right flank. At Chancellorsville the regiment lost 20 killed, 49 missing and 87 wounded.

Perhaps because of the heavy losses it had sustained, the 16th regiment was disbanded later in May of 1863. The so-called “3-year-men,” including William, were transferred to the 121st New York.

The 121st Infantry, under the command of Colonel Emory Upton, was thus known as Upton’s Men. Upton, who subsequently had a distinguished career, rising to the rank of Major General, would later write a book entitled The Military Policy of the United States which shaped American Army policy for decades: arguably, to this day. The 121st was part of the second brigade under Brigadier General Joseph Bartlett, of the First Division led by Brigadier General Horatio Wright who in turn reported to VI Corps, commanded by Major General John Sedgwick

Monument to 121st at Gettysburg

From June until July of 1863 the 121st was part of the Gettysburg campaign . On the evening of July 2nd 1863, following the heroic defense of Little Round Top by Colonel Joshua Chamberlain and the 20th Maine — perhaps the most decisive engagement of the entire war – the 121st was assigned to reinforce and relieve Chamberlain. It occupied the north end of Little Round Top and held this position until the end of that battle. Two enlisted men were wounded. Thereafter, from July 5th until July 24th to 121st pursued Lee to Manassas Gap Virginia.

Gettysburg was the turning point of the war. Lee, forced to halt his advance into Pennsylvania, began a retreat with his armies to Virginia. Today, on Little Round Top there stands a monument to the 121st .

The 121st, pursuing Lee, subsequently took part in the Bristoe Campaign, a series of bloody battles fought in Virginia during October and November of 1863. 

Somewhere in all of these fights young William was wounded, losing a finger. Perhaps because he was no longer able to shoot — it was his right index finger — he was discharged on October 5th, 1863, just short of the full three years. From the gore and agony and horror of the battlefields he returned to quiet Burke, where some time later he married Amanda Esterbrook, an orphan 10 years his junior from the next town over. It’s hard not to wonder how all his wartime experiences colored the rest of his life.

William’s Record in the US Civil War Pension Index

William’s parents were lucky enough to see him return; indeed, they did not pass away until the 1890s. William and Amanda had six children including a daughter, Eva Melinda, who moved to Boston to marry one Patrick Hansbury, in 1893.

Main Street, Burke, New York (date unknown; probably 1920’s)

William’s obituary

William died in 1896 at age 62 of “epilepsy,” according to his obituary, just a few years after his long-lived parents. After his death Amanda moved to New Hampshire to live with another of her daughters, Emma. Amanda collected William’s veteran’s benefits until she passed away at age 79 in 1922, in Litchfield. She was survived by 18 grandchildren and two great grandchildren.

Eva and Patrick lived in Newton, Massachusetts and had a large family. The eldest, a daughter named Estella, said in later years that her father “treated his daughters like sons, taught them how to do everything.” A stableman, he died in Plymouth in 1924 during a ferocious storm trying to save the horses in his charge (eerily reminiscent of an incident a year prior, when the Newton Journal chronicled the actions of another of Patrick’s daughters, Delia: “Leading ten terror stricken horses through the smoke of a burning stable, Miss Delia Hansbury early Saturday morning saved the lives of every one of the horses belonging to the riding school of her father, Patrick J Hansbury, …”) Eva died in 1939, aged 63, from complications from an appendectomy.

Estella would grow up and marry  (on June 10th, 1922, in St Paul’s Cathedral in Boston) Lester Briggs, himself a Purple Heart veteran of World War I and a photo lithographer. They had two children, Wilbur and Kenneth. Lester died of colon cancer in 1946; Estella lived a long and independent life — for years she ran a cosmetics shop in Milton, Massachusetts — passing in 1987 at the age of 92.

Their oldest son, Wilbur, fought as a fighter pilot in World War Two. Upon his return he attended college at Boston University where he met his wife, Elizabeth Bowen. Wilbur and Elizabeth (“Bill” and “Betty”) had two children themselves, Barry and Geoff.

To all the Briggs kids reading: you know the rest; and now you know where you come from –  at least one small part of it.  Know, as well, that every one of these people I’ve written about is proud of you!

Postscript:

There’s more to our story:

On October 13, 1862, a nineteen-year-old Albany man and immigrant from the Hesse-Darmstadt region of Germany named Valentin Ahlheim enlisted in the 177th New York Infantry “to serve nine months.” The 177th fought in the Western theater of the war, at McGill’s Ferry, Pontchatoula, Civiques Ferry, and “fought gallantly” in the final assault at Port Hudson in Louisiana, the longest siege (48 days, days, called at the time “forty days and nights in the wilderness of death” ) in US military history to that point.

The 177th was mustered out in September 1863; somewhere in this time Valentin was transferred to the 21st Independent Battery NY, which was involved in the same battles; Valentin remained with the unit until the end of the war.

Valentin’s cousin Elizabeth, a few years younger, married an Albany farmer named August Meyer, also a German immigrant. They had seven children; their youngest daughter, Augusta Henrietta, married Albert Edward Bowen on July 10, 1909. Their youngest daughter, Elizabeth, married Estella’s son Wilbur in 1950 – and again, you know the rest.

Images:

Fort Sumter: Currier & Ives, https://www.loc.gov/pictures/resource/cph.3b49873/  public domain, via Wikipedia

First Battle of Bull Run: chromolithograph by Kurz & Allison, 1880 public domain, via Wikipedia

Straw Hat NY 16th Infantry http://www.framingfox.com/16newyoincp1.html

Retreat from Gaines’ Mill https://en.wikipedia.org/wiki/16th_New_York_Volunteer_Infantry_Regiment#/media/File:Retreat_from_Gaines’s_Mill.jpg

Field Hospital at Savage Station VA, June 30, 1862 https://en.wikipedia.org/wiki/16th_New_York_Volunteer_Infantry_Regiment#/media/File:After_Battle_of_Savage’s_Station.png

Cramptons Gap https://dnr.maryland.gov/publiclands/PublishingImages/SouthMountain_Battle-of-Cramptons-Gap-large.jpg

121st Monument at Little Round Top https://en.wikipedia.org/wiki/121st_New_York_Volunteer_Infantry#/media/File:121st_regiment_1.jpg

Burke, NY http://freepages.rootsweb.com/~tollandct01/genealogy/gfentonancestry.html (links on this page are broken but here is the direct link to the photo: http://freepages.rootsweb.com/~tollandct01/genealogy/burkeny.jpg )

William’s Pension Record https://www.ancestry.com/imageviewer/collections/4654/images/32959_033013-00751?usePUB=true&_phsrc=hwQ403&usePUBJs=true&pId=11890614

William’s obituary, courtesy of Tammy Traster on Ancestry.com https://www.ancestry.com/mediaui-viewer/tree/5601189/person/316626918/media/d168aaef-778e-4307-9a00-b1306c320175

Muster record for 177th NY: https://dmna.ny.gov/historic/reghist/civil/MusterRolls/Infantry/177thInf_NYSV_MusterRoll.pdf Valentin listed on page 82 (about 2/3 down the page). Also see https://dmna.ny.gov/historic/reghist/civil/rosters/Infantry/177th_Infantry_CW_Roster.pdf page 1081.

History of the 177th:
https://dmna.ny.gov/historic/reghist/civil/infantry/177thInf/177thInfMain.htm

RACI, DADS and WIIFM

Barry’s Guide To Accountability and Initiatives in Big Organizations

You’ve just been given a new task: drive a significant change across your entire organization. Maybe that means driving all teams to a common reporting system or getting all the teams to use the new CRM system or even just getting everybody to use the same logo on their communications.

I feel your pain.

Sometimes, the organization readily adopts change, can’t wait to change, actually. Team leaders welcome you at their door.

But more often, it’s harder. Conflicting priorities, deadlines, even personalities can throw sand in the gears of success.

The RACI Chart

One of the most common tools that the initiative manager has at their disposal is the ubiquitous RACI chart, which lays out which people and which roles are Responsible, Accountable, Consulted and Informed.

Imagine you’re driving a new program. Who has decision-making responsibilities, and who can only speak out (but has little or no authority)?

The RACI chart can help:

(There are many ways to draw a RACI. This is a very simple one to make the point.)

In this case, the GM, at the end of the day, responsible for the product. It fails, and it’s the GM’s head. The CxO and SVP, as important as they are, are consulted for resources (etc.) and informed of progress, but do not have day-to-day decision-making responsibility as the GM does. The next level of management – development manager and so on – are accountable for getting their pieces of the project done, and various other folks may be kept in the loop and/or asked for their opinions.

It’s usually bad if more than one person or role is responsible, for obvious reasons.

Now this is just a review. If you’ve worked in large organizations, you know all about RACI’s – probably more than me.

But the RACI is just a part of the picture.

The DADS Chart

A while back I was managing a function that required the active participation and support of a large number of teams. (Yes, you’re right: lots of responsibility and not a lot of, if any, authority.)

It wasn’t going all that well, and I realized that certain roles and people had the authority to place roadblocks in the way of my initiative.

Enter the DADS chart:

Think of it as the “dark side” of RACI: it tells who’s empowered to slow you down, and how much. Some people feel like they can disagree with you – but you can still get your way. Others will argue – at great length – slowing you, and your project, down and wearing you out.

More seriously, some will deny you the resources to do your job. Maybe you need somebody’s time for one day a week. Maybe you need to borrow some compute resources. What if they don’t want to give them to you?

And then there are those that can just say no and shut the damn thing down.

That’s the DADS chart. Obviously, the more X’s you see on the chart – the worse position you’re in.

Don’t think your org “has” a DADS?

Wake up and smell the coffee.

Why Your Project Will Never Succeed

Over the years, incidentally, I’ve cataloged many of the reasons why those people will tell you your project is hopeless and you’re wasting your time and theirs.

Here are some. How many have you heard?

• This will never work in this organization.
• We tried that before, and it didn’t work.
• I don’t understand it.
• I can’t see the benefit.
• We already have enough change going on.
• It’s too expensive.
• I don’t have the resources.
• It requires retraining the teams.
• It would disrupt current work streams, which are critical.
• [Some very influential person] hates this idea.
• [Some very influential person] hates the person whose idea this was.
• [Some very influential person] hates you.

WIIFM

Let’s say you have a lot of X’s on your DADS chart. How can you possibly make progress?

Recognize that it’s going to take a lot of work, and by work, I mean meetings, and by meetings, I mean politics. Cajoling, arm-twisting, threatening, begging – whatever tack you take, it all comes down to one simple phrase:

WIIFM.

WIIFM: What’s In It For Me?

You can (almost) always change somebody’s mind if you give them a stake in your goals. Some of these are easy: they help you, you’ll help them get that promotion, or that raise, or that bonus. Or maybe you’ll say nice things about them to the CxO in a big, visible meeting.

More often, it turns into horse-trading. They help you, you loan them one of your best players for a week or a month. Or give their group priority on PC upgrades instead of your team.

Failing all of those, there’s always the Wall of Shame: that very public list – in a newsletter or on an intranet site somewhere – of Who Didn’t Help. You have to be careful with this one because (obviously) you can piss people off – and the resulting grudges can last a long time.

You May Want a New Job Now

All that having been said, the satisfaction of driving a successful change can be exhilarating. Some people live for it. Others hate all drudgery of it all (I fall somewhere in the middle). Regardless, knowing what you’re in for – and not being surprised by all the DADS’ers – can open your eyes and, if you play your cards right, accelerate your project.

You’re welcome.

How to Disrupt an Election and Destroy a Democracy (and How to Stop It)

Several people have asked me, being in the software business, how it is possible to use technology to disrupt an election and destroy a democracy.

So, I don’t know for sure. I’ve never done it, never plan to do it (obviously).

But thinking about it, it’s probably not that hard. And as I’ll show later, it’s already happening.

Pretty scary, actually.

What follows is what Einstein called a “thought experiment.” If you (not me) were to do such a heinous thing … well … how would you do it?

And how would you stop it?

First things first, understand the culture

Culture, to paraphrase various great thinkers, trumps everything. So let’s start there.

Let’s talk about “red” and “blue” for a moment. Let’s imagine for a moment it isn’t just one or the other, but that there are scales for each. A “red-1” is a moderate that leans Republican, a “blue-2” is generally pretty progressive (say, like Bernie), and “red-5’s” and “blue-5’s” are batshit crazy.

Studies have shown that the overwhelming majority of the American population have until recently fit between the 2’s – more or less in a bell curve, like this:

Which is why we can elect a centrist Democrat president one election, followed by a centrist Republican the following. Clinton was probably somewhere around a “blue-1”, Bush 43 probably a “red-1”, Obama perhaps slightly more liberal than Clinton at a “blue-2” (a totally qualitative assessment by yours truly).

Next: destroy the culture

Now, the objective here —  and this is important, it’s the core of our strategy –  is to take all those 1’s and 2’s and move them further out on the spectrum. Make all the “red-1’s” “red-3’s” or 4’s or 5’s. Ditto with the middle-of-the-road blue-1’s.

In effect, polarize the electorate.

Why? Well, of course “red-1’s” can talk to “blue-1’s”. We’re not that far apart, after all! In fact, you might switch from one election to the next! That’s what swing voters are, after all.

If you’re successful, there will be no more swing voters.

If you’re really successful, America will find itself split down the middle between crazy-ass “red-5’s” and equally nut-job “blue-5’s”.

So what’s the plan? It’s this simple: for every district – and for every precinct, and, yes, it’s possible – for every voter, if they’re red, you want to make them more red – move all the 1’s to 3’s, or even 4’s, or – home run! – 5’s. Do the same for blues.

In short: turn the bell curve into a “well curve”:

This, friends, is the definition of a polarized electorate.

5’s on one side don’t talk to those on the other side like 1’s and 2’s do. 5’s want to beat the crap out of each other. 5’s might compromise with 4’s … but no way with those bastards of the other color!

How do you use technology to radicalize people?

It’s really not terribly difficult, and most of the technology is based on long-proven methods – there’s nothing all that new in the basics. Large companies have used techniques like this for years; every company these days has to have a “Customer Relationship Management” (CRM) system, running on a bunch of servers with customer segmentation, analytics, and predictive modeling (what are you most likely to buy from us next?).

Screwing up a democracy is just an exercise in CRM at scale.

Here’s a rough outline:

    • Gather your voter data. This is easy, since voter registration data is all in the public domain. Just download it. To your computer wherever you are, which, since it’s the internet after all, could be anywhere on the planet. China. North Korea.

 

  • Now, learn as much as you can about your voters. Again, the basics are pretty easy. You’ll want:
    • Historical voting records for each district
    • Economic data by district (easy to find on data.gov)
    • Crime and other demographic data

This will give you a pretty detailed view district by district. From this, you can start to build a probability for each voter on how they’ll vote. If a district always votes blue, and your voter is in (say) a fairly affluent neighborhood, then there’s a reasonable chance that voter is blue.

But there’s lots of room for error with just this coarse-grained level of information. Your next goal in this exercise is to get very granular with the data: to not just understand area trends, but where a particular individual is on the spectrum. In other words, make that probability a certainty.

Fortunately, there’s lots more data to be gathered!

    • If you can get credit information, and with the Experian hack, it should all be out there, you can get a pretty good economic view of every single American citizen.

 

    • Ditto for military records, union membership records. Hack into church membership lists. You’ll want to target evangelicals in particular with your “make ‘em redder” strategy.

 

    • Ditto with political contribution records.

 

    • Get college records. Which colleges are liberal and which conservative – this information is well known. If you went to Oral Roberts University, it’s not likely you’re going to vote for Hillary. If you went to UC Berkeley, it is pretty likely. Philosophy majors are more liberal than engineering majors.

 

    • Find out people’s habits. Hack into the cable companies and find out what they’re watching. Hannity or John Oliver? Then there’s the ISP’s. What are they searching on?

 

    • You can formulate rules, and apply them. Whites in rural areas with low credit scores will vote Republican. Urban women with children are more likely to vote Democratic. In both cases, who knows why, but who cares? Build these rules – hundreds or thousands of them – and test them.

 

And finally, social media. Deep breath. Yes, Facebook, Twitter, and for that matter, Match.com, GoCupid and the rest.

Let’s take Facebook as an example. Here’s what you (by “you”, I mean either you or your software) do:

    • For every voter name you have, see if you can find their Facebook page. This shouldn’t be too hard, given you have their name, address, and other information.

 

    • See if you can read their posts. Lots of people on Facebook make all their posts public – in which case, you’re done. If not, that is, if they’re actually paying attention to privacy, then friend them.

      How do you friend somebody you don’t know?

      Make up a name (there are lots of software programs that allow you to synthesize a name; here’s one); find a face somewhere (if trying to friend a male, pick the face of an attractive young woman, and so on; again, there are databases of these things, here’s one); boom, you’re done.

      Create a FaceBook (or Twitter, or whatever) profile for your “person.” Using that profile, send friend requests.

    • Then write a program to automate this, run it on a few hundred servers, and send friend requests by the thousands in seconds.

 

    • Once you’re in, scan the posts for political content. Use sentiment analysis (here’s how, from Google’s AI team) to figure out where your person is on the political spectrum.

 

Easy.

And no Cambridge Analytica.

Now there’s lots more you could do (and I mean LOTS), but I’m thinking by now you get the idea. At the end of all this you now have a pretty good analysis of every single voter in the United States.

Loose the Kraken

Now it’s time to start the offensive work. In CRM we call these “campaigns,” and we distinguish campaigns as being primarily digital or direct. “Direct” means we send you a physical mail – a brochure, or whatever. Because of printing and postage costs, direct-mail campaigns are falling out of favor.

Digital is cheap, easy, and scales.

First step: hire a bunch of people with basic Photoshop skills. They’re easy to find, cheap, and plentiful. They’ll be your meme-makers.

Second step: provide your meme-makers with design guidelines. They’ll be creating internet memes customized to their recipients. Here are some examples:

Audience Sample Meme
Red-1 It’s an evil world: prioritize defense spending over social programs
Red-2 Affordable Care Act is socialism and therefore bad
Red-3 Immigrants are criminals and Democrats want open borders
Red-4 Government will imminently impose martial law
Red-5 “Deep State” is controlled by some mysterious, secretive group
Blue-1 Spend more on social programs like schools and health care
Blue-2 Republicans want to backtrack on civil rights advances
Blue-3 Republicans want to eliminate Social Security and Medicare
Blue-4 Republicans support KKK and other radical/racist organizations
Blue-5 Government is controlled by a secretive group of CEO’s and bankers

 

Third step: go through your voter database, and where you see a “Blue-1,” post on their FaceBook page or on their Twitter feed a “Blue-2” meme. Push all the groups over one or two notches.

Fourth step: watch the movement, and repeat the campaigns. Move ‘em!

A Note on Hacking Voting Machines

Well, I mean, you could do it. Apparently, even five-years-olds can.

But that’s not really going to work. It would certainly cause confusion but undoubtedly a technical hack like that would eventually be discovered.

If the overwhelming majority of voters are 1’s or 2’s, on either side, and miraculously a 4 or 5 candidate is elected, somebody’s going to suspect something.

More importantly, you wouldn’t really have effected a cultural change that will take decades to repair.

And then there are hard – and expensive – technical questions, like how many different kinds of voting machines are there, how many versions of the software, how do you clandestinely procure all of them so you can debug and test, how do you test all the variations so that you know they work, how do you deliver the virus if the machines are not connected to the internet, and so on.

If it were me, I wouldn’t bother.

Now, evidently, the Russians did attack voting systems in 2016,  My guess however is that this attack was done less to actually change the results and more with the expectation that they would get caught, thus throwing the integrity of the election in doubt.

How Much Would All This Cost?

Like, next to nothing. There are about 235 million people in the US of voting age (according to the University of California, Santa Barbara). So, 235 million records in your database.

So 235 million records, let’s say a megabyte of data per person, that’s like maybe a hundred or so of these puppies.

Maybe – oh, let’s go wild – a thousand servers to hold all the data, for your database, for your Photoshoppers, for development and testing, trial runs, etc. Maybe 500 people working on it. A thousand servers adds up to less than $5 million (you could do it in the cloud for less, and wouldn’t that be grimly ironic, using an American cloud provider to bring down America). 500 folks maybe another $25 million (that’s a very generous $50k/year).  So for under $30 million you’ve brought down the greatest democracy in the history of the world.

Sound like a lot?

It isn’t. Netflix (Netflix!) rents tens of thousands of servers every night – tens of thousands — in the Amazon cloud, just to run analytics to predict what you might want to watch tomorrow night.

This is nothing compared to that.

And Russia (just to pick a country at random, okay, not at random) has a total military budget (according to the Stockholm International Peace Research Institute) in 2016 of around $69 billion. So our little project would be one tenth of one percent.

Basically, that’s discretionary spending.

With a pretty amazing return on investment.

And by the way, make no mistake: this is war, 21st century style. No country will ever challenge the US militarily – that would be suicide – but hostile governments are learning that they don’t have to in order to achieve their objectives vis-à-vis the US.

Von Clausewitz said that warfare is “the continuation of politics by other means.”

What we have here is “warfare by other means.”

Oh, and it’s all 100% totally legal. Chew on that for a second.

Think It’s Nuts?

Here’s a study done by the non-partisan Pew Research Center about shifting political attitudes in the US (cited in The Economist, July 14, 2018 issue).

Look familiar?

Look at it: in 1994 and 2004 the majority of Republicans weren’t all that ideologically different from Democrats. But then in 2017: the majority of voters in each party have nothing in common.

Actually it looks exactly like the bell and well curves I showed up at the top of the article!

It’s happening.

Right now.

Scared? You should be.

What’s to be Done?

The situation is not hopeless. There’s lots we can do.

Most immediately, the Good Guys (that’s us) can take steps to counter the Bad Guys’ CRM. We can do the same analysis they’re doing and anticipate the attack.

Both the major parties have large technology organizations that do just this sort of thing: NGP Van for the Democrats, Deep Root Analytics for the GOP. The latter, by the way, just happened to leave all its voter data – on 200 million voters – exposed on the open internet in 2017, in case you thought it would be hard to get this information (and yes, that link goes to Fox News, in case you were worried about liberal media bias).

Both sides are well skilled in CRM skills. I would suggest that the intelligence services be the ones to watch for the onslaught of social media cyberattacks, and then share the data with the parties – both of them, so that both can take whatever action they feel appropriate.

That’s tactical. In the longer term we need to inoculate ourselves against this sort of disinformation, and that can only be done through education. Our kids have to know how to find Korea on a map and how to do long division without a calculator or computer, and they should know in what year the War of 1812 was fought without going to Wikipedia. And they really ought to know that cavemen did not have pet velociraptors. They need to be able to look critically at a meme on Facebook and say, “that’s bullshit.” (The esteemed University of Washington in Seattle actually taught a course on this in 2017. Really: it was called “Calling Bullshit: Data Reasoning in a Digital World,” and it was oversubscribed).

And we need to demand more of the so-called “mainstream media,” on both sides. It’s fine that there’s a left-leaning MSNBC and a right-leaning Fox (actually in my view they’re not leaning, they fell over, but whatever). But too often they either don’t ask the hard questions – of the politicians or of themselves – or they report exaggerations and outright lies in order to get better ratings.

The Great Leveler

The proposed US defense budget for 2019 is $681.1 billion. But using the techniques I’ve outlined, you can cause chaos in any open society for practically nothing. Any country no matter what its GDP can do it. Russia, North Korea, Somalia – as long as they have a pool of reasonably competent technologists (and not that many) they can draw upon.

In a world where it would be the height of insanity for any country to challenge the US military, there are other means.

Winning this information war – and that’s what it is – isn’t going to be easy. But we must: our survival as a democracy depends on it.