‘AI’ Meets the Law of Unintended Consequences

Since I started this blog I've naturally been listening out for topics of conversation, trends etc. With the tailoring of content within social media, blogs, news it's very easy to find yourself in an echo chamber.

I was reflecting on this over the last week or so after noticing waves of articles in my feeds:

• AI investment boom => Death of AI investment boom
• AI code assistant releases => AI coding isn't up to scratch
• Agentic AI is the future => AI Agents are dangerous

It’s hard to keep up, let alone know what to believe. It was only once I started paying more attention to the dates that it stopped feeling like I was constantly being pulled between different co-existing realities. The Unintended Consequence of "the algorithm" being the more we look at certain content, the more we see on related subjects. So because I read Steve's article on Vibe Coding my feed gets flooded, I like some articles, write my own, then read about Coding Agents and we start again.

This got me thinking about the Unintended Consequences of "AI"!

What is the Law of Unintended Consequences?

It’s the idea that actions have effects that were not intended or even considered, both good and bad, Artificial Intelligence is no different to any other innovation in this regard.

We covered a few of these in previous articles:

• The birth of Vibe Coding — a whole new way to build software using natural language prompts.
• The rise in cyber risk — more tools, more endpoints, more exposure.
• The rise of "AI" Washing - referring to everything as "AI", regardless of whether it is or not.

What others might exist? It's hard to know since we're still so early in the technology cycle.

I've often had conversations with people around how bugs or problems could get into production or that certain software or features weren't intended to be used the way they are. At it's most professional, the software development "industry" is incredibly sophisticated, structured and process driven, so how can this keep happening?

I've always replied with broadly the same response; no matter what you do, until you put it in the hand of a user you cannot be 100% sure what's going to happen. If the likes of Microsoft, Apple, Google etc with effectively unlimited resources are unable to release perfect software or services, then everyone else should give themselves a break, provided of course they're still doing it "properly".

Another example closer to home, I have two kids under 10 and set up a private messaging app on their tablets with them and some family members so they could start learning how to communicate safely. It worked brilliantly and has been great to be able to interact with them directly... less great was my eldest video-calling her Grandma (who was driving at the time) 10 times in five minutes just to say hi. Or my youngest sending 50 GIFs in 30 seconds because she realised it popped notifications on her sister's tablet while she was playing a game!

But, what about "AI"? What Unintended Consequences have we've seen and what others might be coming?

Already observed consequences:

Increase in Power Consumption:

Training and running models takes power and lots of it! As models and technology become more efficient we increase the limits of what we're asking, there is no end in sight as it stands.

Employment Re-structuring:

This is often the most hotly debated consequence.

"AI" is going to replace us all! Personally, I'm looking forward to it putting the bins out and sweeping up the leaves, but maybe that's just me.

Without trivialising it, much like previous innovations we're likely to mostly see a re-alignment of the work force, but predicting whether that will result in a net increase or decrease is very difficult.

Long Live Markdown:

Might be exaggerating a little but since it's creation Markdown has largely been the preserve of the more technically inclined, used in readme files, forums, wikis etc.

The simple structured formatting makes it ideal for use in chat windows which have come to dominate the AI space and create semi-structured input that LLMs and users can both understand more clearly.

Emerging Consequences

The Death of Search / SEO:

This is something we're now seeing quite clearly and has several other knock-ons.

The integration of search and LLMs has had an enormous impact on search already. No longer do we search, click a link and go to the site. We search within our favourite LLM or use the "AI Overview".

The figures are out there to see but the drop in pure search traffic and referrals over the last 6 months has sent shockwaves through more than one industry.

This may not herald the end of search engines, but we're likely to see a rapid evolution over the coming months / years around their use.

Stifled Creativity:

Ever noticed how a lot of sites are starting to look suspiciously familiar? It turns out when everyone uses the same models and data, novelty becomes harder to find.

The flip-side is also true of course, real creativity and innovation stands out that much more.

Growth of One-Person Businesses:

With the birth of so many "AI" tools in almost all verticals we're starting to see real force multiplier effects to the point where an individual can build and operate a much more substantial business than they might have been able to previously.

This is greatly aided by other innovations and services, but nonetheless, individuals are empowered in a way like never before.

Re-emergence of "the forum":

With the growth of social media, forums have largely disappeared or moved towards specific niches in corners of the internet, many of which are best left alone.

The drive for real data and more "human" models has seen organisations scouring for more data sources, especially where those organisations don't have their own social media platforms. Go Reddit, GO!

What's next?

New Revenue Models:

The shift of search and browsing traffic fundamentally breaks the advertising model for on-site ads, outside of the LLMs and search operators.

Sites that rely on that revenue will no doubt be looking to how they can replace it; subscription services? Payment for AI bot access? What else?

What about brand owners? Will we see more focus on influencers and organic marketing through forums / reviews, rather than direct advertising?

The Evolution of Websites:

What purpose will they serve if all the information on them is being retrieved via LLMs?

This will have an enormously long tail to it, but we could see the biggest shift in the make-up of the internet since it's inception.

Some will embrace and make data easy to access, some will retreat behind bot shields / paywalls to preserve the control of the audience

Might the next generation of websites be interactive? Perhaps being crafted to the viewer on the fly? Or by solely engaging through chatbot or AI video interaction?

Educational Shake-Up:

This is obviously a huge target area for the use of AI, so why the Unintended Consequence?

Aside from the obvious use by students to complete work, I've heard some fascinating stories from friends in the education sector.

The British gap year student who found himself teaching in a school in Thailand while travelling and used ChatGPT to create lessons for the students. In turn he ended up sharing that with other teachers there who used it to freshen up stale lessons and content.

New Jobs:

Five years ago a "Prompt Engineer" was a description of a person, not a job title, now we have "Context Engineers", "Chief AI Officers", but what else might we see?

Years ago I worked in a product design firm who employed a "Chief Business Imagineer", to this day I have no idea what he did, but in an AI-aided world maybe he would "imagine" a business idea and have his/her AI agents build and run it? Maybe not... who knows?

The case against Public Cloud?

In recent years the use cases for Private Hosting have been dropping away, most businesses are now comfortable to host in one of the major Public Clouds, could we see that change?

There is a growing understanding around data, training rights, access to models and I think this could lead to a growth in the Private Hosting space, or at the very least, a new use case. Privately Hosted, Managed, Secure Data Lakes and "AI" instances / services, certainly for those businesses and individuals with the means and concerns over data privacy and ownership, makes a lot of sense.

This won't be universal for DC operators though, providers that have failed to innovate and invest in their infrastructure will be left behind given the power and processing demands.

So... what's the take-away?

I've mostly come down on the optimistic side here, you could however very easily make plenty of the counter-arguments.

What's apparent already is that with the speed the technology is advancing and our desperation to not be left behind, it's leaving very little time to fully assess any consequences that comes from it's adoption. Many of those are net positive, some won't be.

Many of the big societal issues are less technical and largely out of our individual control. We absolutely should embrace the technology and what it could enable but could all do with pausing every now and again to ensure we've properly considered the potential impacts and not just chased the next shiny thing because Steve on Linked In wrote an article about it and everyone agreed.

What are your thoughts? What are the unintended consequences you've come across? Let me know!!