Generative AI Risks
There are realistic short term risks which are facing with the currently available Large Language Models. This article tries to separate the hype from the actual risks.
These last few weeks we’ve all read posts and articles about the risks of Generative AI.
Some have expressed fear and concern that these technologies could bring about ‘Black Mirror’ type existential risks and scenarios where AI replicates itself and takes over.
I believe it is more important to ignore the hype and look at the realistic risks and missteps which we will have to fix and manage in the short to medium term. These are risks which have already been introduced by other technologies over the years but will be amplified greatly by Generative AI.
Misinformation & Hallucination Risk
Large Language Models can and will output misinformation at times. This happens due to two main reasons:
The first is that they are mostly trained on public data on the internet; therefore some of their training data is already biased and contains misinformation.
The second is the ‘hallucination’ effect where LLMs will sometimes output false and misleading information if they predict that the false information is the best possible output to the provided prompt.
This is not a new risk, in fact misinformation is also an issue with Social Media and current internet search. The main concern is that since users have less understanding on how these LLMs work and tend to anthropomorphise these models, we are more prone to accepting false outputs as fact.
As these models become more accurate, hallucinate less and are trusted more blindly then the risk will actually increase since it will be more difficult to discern when the output is factual or not.
Closed Model & Access Risks
Up to a couple of years ago a lot of AI research was open and each new model was accompanied by a paper documenting the training datasets, architecture and size.
This trend has changed quickly due to the competitive landscape. In fact OpenAI withheld all architectural details related to their latest LLM model (GPT-4). Therefore companies building their software upon the OpenAI APIs are doing so without understanding the underlying technology’s full capabilities, risks, limitations and biases.
The closed AI model also creates additional risks:
Dependency on one provider allowing the provider to increase API fees and block access.
Ability for the provider to choose who gets to access these incredibly powerful models. For example OpenAI are selecting which customers get early access to GPT-4 at the moment.
New editorial role of AI companies (See below).
Due to the risks of of generative AI such as misinformation, dangerous output and misuse, we see that AI research companies are building guardrails around their models. They do this in four main ways:
Filtering and selection of training data that is fed into their systems.
Fine tuning the model using reinforcement learning by giving feedback to the model and rating its output as desirable or undesirable. By doing so they influence the model based on their own biases and objectives.
Filtering both user input and model output to actively block and hide certain undesirable actions.
Through selectively accepting users and binding them to terms and conditions of use.
The above points are important and help to address some of the risks of this emerging technology, but they also give the AI companies an editorial role. Such a role allows them to control the information generated by the models and control who has access to such information. As we have seen with Social Media companies this is a very difficult balance to achieve.
Erosion of Trust Risk
It is not far fetched to imagine that the internet will soon be flooded by huge amounts of AI generated photos, websites, apps, videos and audio in the coming months and years.
As the technology improves and we start seeing more realistic AI generated photos, videos, synthetic voices, web publications and news sites — it will become much more difficult to discern what is AI generated vs Human generated.
If we can no longer watch a video / receive a phone call / read an email and be sure of it’s authenticity and origin - then we can no longer trust anything we consume online.
Combine this with misinformation and closed models, and this erosion of trust is probably the greatest potential risk to society at the moment.
Emergent Capabilities
Traditionally software algorithms were deterministic. The developers understood the outputs that the algorithm would generate based upon the inputs. Tests could easily be developed to ensure that software worked in the way it was designed to.
Emergent capabilities are new features and capabilities that appear after changing the training dataset, increasing the model size or tweaking the architecture. Once Large Language Models (LLMs) reached certain sizes, new capabilities such as translating between languages or generating source code started to appear. The latest models are also showing stronger reasoning and mathematical capabilities.
There were also some more surprising emergent features such as model ‘jailbreaking’ (where the built in safety features are bypassed by the model itself due to certain user prompts) which have caught AI developers off guard.
Clearly these emergent capabilities are a important feature and make these LLMs powerful and the transformative. They also pose a new type of risk which prevents the developer of the technology from being able to fully test all outcomes of the system. The developer is also unable to give assurances on the behaviour and outputs of the system because they cannot be fully understood.
In conclusion
The catastrophic warnings such as AI taking over humanity are at this stage hyperbole and our current models are not close to displaying such capabilities — despite this it is the mundane risks described above which can be dangerous.
These risks (and also the benefits) are not new to technology. As technology advanced over the years we have had to tackle misinformation, closed and opaque systems, trust and privacy issues in multiple forms. Perhaps what is different this time is that the risks are amplified because of AI and the pace at which they are being amplified is faster than ever before.
A term which you will read about more and more in the future is ‘AI Alignment’. This refers to ensuring that AI Models are aligned with human requirements and are beneficial to society. I assume that we will soon start to see regulation introduced to ensure such alignment. Unfortunately this will not protect us from bad actors with the resources and competence to deploy and develop their own ‘mis-aligned’ models.
It is clear that we now have access to a technology that will transform the way we live, work, learn and manage our health. It also seems that this transformation will take place faster than we imagined possible and it will also be more disruptive than we thought. Whenever we have huge technological advancements we have the opportunity to create amazing benefits for humanity. Therefore it is important to keep on researching AI and building bigger and better models despite the risks. It must be done in a responsible and open manner which is aligned with humans so that we can all benefit.