Return of the Worms

Watching this pivot-to-ai video I was struck by a thought (as it sometimes happens)… I think “we” may be setting ourselves up for the return of the internet worms, 1999-style, but with a vengeance. Let me explain.

Software security in the ’90s

In the early days of the public internet, especially as e-mail became widely adopted (mid/late ’90s), a very peculiar thing happened on the client (end user) software side: In the name of progress, some productivity software like the Outlook e-mail client, the Word word-processor and other applications would begin supporting “macros”; little snippets of software written in a high level language such as (at the time) VBScript, embedded inside the data the application would be processing (so embedded inside an e-mail or a document for example).

The problem with this was not the macros themselves, the problem was, that the productivity software would execute the macros automatically, no questions asked. For a word processor, that meant, if someone sent you a document and you opened it, upon opening the document macros would potentially execute without you knowing or having any chance of reviewing or refusing.

Now that in itself would not be terrible if the macros had sufficiently limited capability and access. After all, when you open a document in a word processor “much software runs” based on the contents of the document and this can be both safe and useful. But the macros were neither safe nor limited, they had basically full access to everything on your computer. That certainly was a problem.

In effect, what this meant was, that anyone could send you an e-mail that would contain a macro instructing your mail client to do nasty things to your computer (like install bad software or delete files or add 5% to all numbers in all spreadsheets on your company file server or whatever else you can dream of). This macro would execute simply if the recipient viewed the e-mail - and in some cases, merely receiving the e-mail would be sufficient to launch the macro, a true 0-click attack.

This type of gullible client software directly enabled the ILOVEYOU worm that caused about 10% of all computers connected to the internet at the time, to become infected. Crucially, this was not exploiting a previously unknown flaw, it was not a security bug, it was merely exploiting the deliberate and intentional design of productivity software at the time.

Of course, anyone running software that was not designed with this mind-set, were completely safe. Again, it was not an unintended flaw that was being exploited, it was a deliberate design choice that simply opened up the users to compromise with no mitigating mechanisms.

One might think that this would cause industry to change their thinking and introduce basic security measures. That would not happen for another decade however…

The anti-virus era

If you have a piece of software that functions in a way that is not desired, when given certain input, and you understand precisely the reasons for that, well then a typical engineering approach would be to change the software so the problem ceases to exist. That was not what happened.

Instead, other businesses made it their business, to build “veils” of software that would sit in between the internet and the known-flawed productivity software, and try to detect “patterns that are bad” in information coming from the internet, before passing on the information to the known-flawed software.

This was the age of anti-virus software. Great business, for all the wrong reasons.

Aside from this approach being, let’s say, sub-optimal (papering over a simple but serious problem is not smart when you could just fix the problem), an equally large problem was the way that anti-virus software was built:

Anti-virus software at the time would hold a large database of hash-values of known malicious macros, internet worms, viruses and so forth. You can probably already now see that this approach has a couple of fundamental flaws:

• It can only recognise a worm that is already known (so at best this is an after-the-fact remedy, not an actual solution) - so someone must be hit before others can gain protection
• Any change to the worm changes the hash so future variants avoid detection - making it trivial to generate new effective variants of any previous attack

The second problem was the most mind-boggling to me. Since these worms were often written in VBScript, a scripting language, it would be trivial to insert a single additional space in the code somewhere (which would have no effect on the functionality of the macro) to avoid detection by even the most up-to-date AV software.

In fact, this happened when people would e-mail versions of the VBScript to each-other on mailing lists and the like. I remember an employee of a well known AV company at the time, scolding someone for sending such a VBScript macro out to the list because the sender’s e-mail client had introduced line-breaks in a couple of the longer lines. The AV employee directly said that this macro would now avoid detection. Rather than looking inwards he was scolding someone for accidentally inserting a newline in a piece of VBScript since that would circumvent detection by a market leading AV system at the time, and at the same time admitting that AV software was - at best - a thin veil over a deep and fundamental lack of basic application security.

This really struck me. Not only was I baffled at the naïveté of the productivity software executing any code received from the internet without question, I was also struck by the market deciding that rather than fixing the problem, it should be papered over. Finally I was absolutely dumbfounded by the lack of actual sophistication of the leading AV solutions of the time.

It would not take a genius to break “all” AV security at the time it seemed.

Genetic optimisation

As with genetic optimisation, one could imagine that an internet worm written in VBScript would not only do its deeds and as a last step propagate itself to every contact in your contacts list, as was common at the time. You could imagine one additional step: Introduce variety in each copy of the code passed on to the contacts.

At the time, simply inserting one additional white-space where white-space already existed, would have beaten every single AV solution out there. Of course, just inserting white-space could be mitigated against from the AV side, so the bar would have had to be raised a little bit.

A more interesting approach, in my mind, at the time, would be to do more sophisticated modifications of the code itself. It could be altering constants in the code, renaming things, it could be regrouping code blocks, it could be inserting snippets from other code found in the users mailbox. A set of individually simple re-writing rules contained as part of the malicious macro that could be applied at random to the entirety (including the rules themselves) of the macro, thereby generating a new “generation” of the macro.

Upon propagation, the macro would then transmit a new generation to every possible recipient, rather than transmitting a verbatim copy of itself. This would, effectively, be a distributed genetic optimisation on a block of code, to transform it to the most effective copy that it can be. And everyone’s computers would be participating in the “solution”.

Thinking about the very simple transformations described by Conways' game of life and the astonishingly non-trivial patterns this simple system generates, I would imagine that a fairly simple code transformation system could be devised, that would generate interesting and non-trivial variants of the original worm.

But even more importantly; the variants do not all need to be viable. If each successfully infected system generates on average 100 new generations in e-mails to contacts, you need very few of those to “survive” (successfully infect the next system) for this to still be an effective worm. The more effective ones will propagate more, the less effective branches will die off. This is, really, natural selection - or in computer terms - a genetic optimisation algorithm.

Also this would not have to be limited to VBScript. For example, the SQL Slammer worm in 2003 could have employed this technique on its machine code too.

Luckily, it seems at the time none of the bad people doing these things thought about this. I did, but it was purely an academic exercise in my mind, and one that I chose to not publish, simply because at the time nothing good would have come from that. I feel that here 20-some years later we are beyond naïve AV and gullible e-mail clients and this information should both be safe to publish and also really this should not be news.

We may not have blind macro execution any more in the form we used to have, but there is a new play in town…

Enter AI

These days I see companies rushing to re-invent the productivity software security mistakes of the ’90s, and while I am far from the only person writing about this, I do feel I have an angle that I have not seen covered elsewhere.

What limited the spread of the ILOVEYOU worm was that it affected Outlook clients only. It infected the client machine and that was it. It spread to 10% of all client computers world wide, but it did not spread to the company database servers, the CRM systems, the industrial robots and all the other infrastructure.

Same story with SQL Slammer; it only hit MS SQL databases. It also temporarily overloaded company network infrastructure in some cases, but once the SQL Servers were patched all returned to normal.

Imagine, if these worms had been written in a language so universal that it could execute not just on Outlook, not just in SQL server memory, but on most of the client and infrastructure systems running most businesses? Imagine if the e-mail client worm could “jump” to your internal documentation system, from there to your BI system, from there to your finance system, from there to your CRM, from there to your customers, and so on…

Limitations of technology used to save us here: Since these are all different systems from different vendors using different technology, there is no such common language. Your mail client VBScript does not execute on your PLSQL-running database, PLSQL won’t work on your AutoLISP CAD suite and so on.

These are limitations that have existed until now…

English as the universal machine language

With everyone bolting Large Language Models (LLMs) into every piece of software from client software to your business critical back-office systems (and the infrastructures they run on!), there now is a universal machine language: It is called “English”.

As David Gerard pointed out in the video I linked to in the beginning of this post, history is now repeating itself:

• All software, not just client productivity software, is now getting the capability to execute instructions given in plain English by means of LLMs.
• For LLMs to be useful, they are given access to a wide range of tools - while they may not have total system level access like VBScript of the ’90s, they now have networked access to a wide range of services
• Any information passed to the LLM may be interpreted as instructions not just data - this is known as prompt injection. In effect, any information received from the internet (an e-mail, a web page, …) may therefore be executed as instructions on behalf of the user
• AI vendors are attempting paper over the problem of prompt injection by recognising patterns of known prompt injection methods - does this ring a bell to anyone? Remember AV? Remember hashes? Remember white-space and newlines?

Therefore, since it seems industry is tripping over its own feet in a rush to re-implement the failure that was internet software security in the ’90s, but on a much grander scale with much further reaching implications, I felt that I would share this fairly obvious (when you think about it) problem so that people at least have a chance of pumping the brakes before this goes too far. This is not serious yet - but it will be, if the right people don’t put their thinking hats on.

The return of the worm

As demonstrated by Håkon Måløy in this post, it is possible to embed malicious instructions to Copilot in a document which not only causes Copilot to act on those instructions, but also to propagate the instructions to more documents.

He is not the first to discuss this either; the Morris II worm appears to be the first take on this LLM-powered modernisation of internet worms.

However, and this is where I think it really falls apart; it’s like a perfect storm:

• The mitigations implemented by the AI vendors are pattern based and therefore will be inherently vulnerable to variation in the malicious instructions.
• While a particular phrasing of malicious instructions may only work on a limited number of systems, re-phrasing has the potential to allow malicious instructions to work on previously immune target systems.
• The “execution engine” of the malicious instructions is no longer a primitive VBScript interpreter - the mechanism that acts on and propagates the instructions is itself an LLM, a purpose-built text generator - the LLM is the pinnacle of mechanised English text generation.

In other words, LLMs are not only inherently vulnerable to malicious instructions with no “hard” security solution in sight (unlike the ’90s era software there is no simple fix), they are also the ideal attack propagators for this style of internet-wide “genetically optimised worm” attack.

Nothing will re-phrase a malicious instruction written in the English language more elegantly than an LLM. All it takes for the malicious code to accomplish this, is that it asks the LLM kindly to do so when propagating the attack.

LLMs are powerhouses when it comes to re-phrasing English text. And when English becomes the lingua franca for computer system instructions (including malicious instructions), that means LLMs are powerhouses for generating “genetic” variability in a worm that would spread not just across client devices, but across every LLM they are connected to either directly or indirectly.

And with the only current defence employed by vendors being “signature based” basically (or “pattern based” as they say), variation is likely an effective bypass for any such mitigations. I feel this is something that deserves attention, and I hope this article is going to aid in that regard.

Mitigating factors

Worms have not been very popular for a long time. Part of that is of course that systems have, for a long time, been engineered so that they would not just execute arbitrary instructions received from the internet. But another important factor is that computer crime has moved into the big leagues - it is no longer the pimple-faced teenager in a basement trying to “break stuff for fun”, it is instead about making money. Ruining systems at random with a worm is not making anyone money.

Criminal organisations have moved to targeted attacks, generally ransomware - where a single target is hit, ransom is demanded, and upon payment maybe the target will get their data back. That is a money-maker - simply setting the world on fire is not.

Worms of course can be enablers of malicious targeted access (by installing back-doors or leaking information). And even though maybe people don’t want to just break systems at random much anymore, I do feel just from an engineering standpoint that it is unreasonable to ignore this issue.

Closing

I hope this write-up will inspire people to think twice about what they connect their LLMs to and where their LLMs get their information.

If this could move people to think twice about whether that system really needs an LLM in the first place, I think that would really be something.

More often than not, a couple of hours of some consultant’s time can automate (with a few lines of reliable code) whatever it is you’re about to employ an LLM to do. Or you could do it yourself (either manually or again by automating with a few lines of reliable code) and by doing so remain capable, engaged, inspired and relevant.

It’s up to you…