Unsupervised Algorithms are like Unsupervised Kids: Be Careful, or They will Drag You Into the Headlines!

Victorian drawing of several unsupervised children starting a bonfire

Someone should have told that to this Chinese dad before his toddler jumped in their SUV and drove around looking for him until he had an accident! And precocious kid and dad are not the only unsupervised things in the news today. Another thing hit all over the morning news drive today in the US: an unsupervised news aggregating algorithm gone bad.

Algorithms Gone Bad

So here’s what happened. In the US, our federal agency that runs all of our court system is called the Department of Justice (DoJ). Here’s apparently what happened, according to Buzzfeed who broke the story:

“An email sent from the Justice Department to all immigration court employees this week included a link to an article posted on a white nationalist website that “directly attacks sitting immigration judges with racial and ethnically tinged slurs,” according to a letter sent by an immigration judges union and obtained by BuzzFeed News.

According to the National Association of Immigration Judges, the Justice Department’s Executive Office for Immigration Review (EOIR) sent court employees a link to a blog post from VDare, a white nationalist website, in its morning news briefing earlier this week that included anti-Semitic attacks on judges.”

You are probably having the same reaction that I am.

OMG – How Did That Happen???

So DoJ said that the weekly news e-mails are sent by a contractor. So, what’s up with this contractor? Here is what Buzzfeed reported:

A former senior DOJ official said that the email in question was “generated by a third-party vendor that utilizes keyword searches to produce news clippings for staff. It is not reviewed or approved by staff before it is transmitted.” (my emphasis)

Oopsie!

I don’t have any kids, but when I’m in charge of any – especially teenagers – they are on a way short leash. And that’s the way we handled our news feed on our web page back when I worked a short time for the city I will call North Overshoe (with a shout out to Jim and Margery).

Bad News in North Overshoe

So the good news was that the city of North Overshoe’s council member, Councilor Turncoat, had matched zillions of dollars of his own millionaire money to pay for a city program for elders that everyone agreed was needed. Everyone loved the program.

The problem – everyone hated Councilor Turncoat because he was, well, a turncoat. He always made deals behind the scenes and then surprised people in the council meeting by saying something the opposite. And because we were entirely funded by this program, we had a whole internal lobbying arm that practically lived at City Hall to keep our funding coming.

Relegated to the IT department, I helped the team put up our web page. Our lobbying arm was really successful at getting local media to cover our program, so we wanted these hot news stories to be on our front page.

But the problem we found in testing was that if we just set up a news feed algorithm and let it run, most of the time the top story would be something about our successful elder program. But in that rare, horrible once-in-a-while, it was about the city council members calling us dirty names and saying we should be fired.

Our developer at the time was slowly working through a master’s degree in software engineering, and happened to be taking an algorithm class. He tried and tried to set up an algorithm that would serve us news on our program at the top, and elder affairs in the city also grouped in at a lower rank, but somehow avoided these city council fights. No matter how he tried, the sensitivity and specificity were not there – if we cut out the fights, we missed some of the really good news, and if we promoted all the great news, we picked up some of the fights.

No one knew what to do, but I knew what to do, and we all sighed and agreed – it had to be done.

We Hired a Babysitter

I’ll call the 19-year-old kid we hired Sonic because I remember he really liked Sonic the Hedgehog. He was a young gamer type – very sweet and quiet – who had been swept in as a political hire with Turncoat’s entourage. Beyond holding open doors, building IKEA furniture, and hand-eye coordination, he really had no skills.

But you should hire for character anyway, he in that sense, he was golden. We sat down with him and showed him how the algorithm worked, then gave him an interface kind of like the one below:

Notice that our algorithm assigned exponential values (for both ranking of display and aging out), so 1 was hot and 16 was not. Now, see where our algorithm picked up the “false positive” of a fight in the city council with Councilor Turncoat? That’s where Sonic could edit the manual ranking – I put an arrow where he did that. By setting it to 0, he kicks it out of the algorithm. As he used to say, “Man versus machine!” while flexing his wiry muscles. I miss that kid!

Hiring a Babysitter Is Cheaper and Much Less Dangerous than Letting Algorithms (and Kids) Go off on Their Own

And it’s a good way that contractor could have stayed out of the headlines!

Monika Wahi is a LinkedIn Learning author who isn’t very good at building algorithms, but does a good job of supervising them. Want a good online course to learn the basics of programming algorithms? Try Joe Marini’s course on LinkedIn Learning – he works at Google, and will give you a good foundation in programming algorithms.

Victorian painting by Ida Waugh.

Post last updated August 23, 2019.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.