GenAI: Is transparency and traceability possible with 405 billion parameters and counting?

If you’re following the news on AI then you’re probably aware of Llama 3.1 from Meta and the 405 billion parameters it contains.  In consideration, how much computational power is needed to do real-time analysis on 405 Billion parameters on 130,000+ users per second?  If you know the answer, please let me know.  In the mean time, let’s take a look at Chaid analysis and how it might apply.

Chaid stands for Chi Squared Automatic Interaction Detection and has been very useful to me over the years in identifying significant relationships in large data sets when doing campaign analysis and Customer Performance analysis.  Understanding which variables (e.g. gender, title, industry, year established, recency, frequency, monetary value, product category, product sub-category, sku, area code, post code, etc) demonstrate the strongest correlations and are most predictive can be a challenge and Chaid has proven to be reasonably adept in presenting complex relationships.  I particularly like Chaid because it presents the data in simple numbers and in decision-tree format.  It can also take advantage of regression analysis and machine learning methods.

Setting aside the possibilities of Chaid for a minute, let’s get to the question of transparency and traceability, especially as it relates to Llama 3.1.

Returning to the question: how much computational power is needed to do real-time analysis on 405 Billion parameters on 130,000+ users per second?  I ask this question because Google receives roughly  3.5B searches daily.  Facebook gets only 2.1B daily and they both spread that volume across the same 26,400 seconds per day.  Sure, we can argue the exact numbers but the volume of users is staggering and most of us don’t have to solve that problem.  However, the 405 Billion parameters is a pretty interesting benchmark from an open source AI model. 

Whether you have 10,000 daily customers or 3B+ daily searches, if you adopt Llama 3.1 what are you going do with 405 billion parameters when leading AI authorities and academics are calling for transparency and traceability in GenAi decision making?  I’d like to put more context on the question with following example:

XYZ ecommerce company offers one million products to the public and their product catalog has 25 product categories, 10 sub-categories and a total of 236 discreet categories of information. They have two million active buying customers and by this I mean they buy at least one time or more each year; and they have 800 inside sales account managers on the phone.  The company decides to deploy Llama 3.1 AI from Meta (Facebook) inside their sales centers / call centers and they have previously deployed a “human in the middle” two-way text messaging platform.  In addition, they have 100+ algorithms and  business rules that determine dynamically assembled emails that are event triggered based on daily updates of click-stream data from the company web site and transaction data from the customer.  Every email is unique and personalized to each user.  Llama 3.1 open source AI will be deployed on the web site to serve up appropriate content, including the creation of new content, in real-time, using the product catalog as one source, the transactional records from the customer master and the conversational histories from their CRM and two-way SMS platforms.  Because they use “human in the middle” processes to manage text messaging conversations with customers, the actual text message responses sent to customers are a blended response of AI and human input.  GenAI creates the initial draft which is then edited by a live human (one of the 800 account managers) before being sent to the customer.  And this blended conversation history inside the SMS platform has three years of history.

The Generative AI Governance Framework says we should have transparency and accountability, including traceability of decisions made by GenAI.

Using the scenario above, what kind of meta data and traceability records will the company need to create, due to their use of Llama 3.1, in order to track and satisfy the requirement of traceability of GenAI decisions?

This week I watched a news story about AI being used to detect glaucoma.  The system yielded a 97% accuracy rating in diagnosing glaucoma – and this was back in 2021.  This is a very specific solution and the equipment is used by one person at a time to evaluate one patient at a time.  Relatively easy to be transparent in telling the patient the system uses AI and in tracing the diagnoses that yes, you have been diagnosed with glaucoma.  This is essentially one-to-one communication and relatively straightforward governance, transparency, accountability and traceability.

Not so much in the digital world where content and interactions are created and delivered to customers in real-time.  The digital world is fluid.  It is not static.  We may to go a static URL, but the experience and delivery of content can be specific and personalized for every single person.  The offers presented can change.  The prices calculated can change.  An enormous amount of variables can change – and they do so in real-time. 

When we consider the introduction of Llama 3.1 and its capacity of 405 billion parameters, we might need to take a step back and reconsider the recommended governance requirements.  In a vein similar to the human body and the human brain, we can easily record and capture the words that are spoken.  We can measure the distance we walk each day or how fast we run 1500 meters on the track.  However, we don’t really look for transparency and traceability on every decision that is made by the person.  As a comparison, here are a few numbers about the human body:

• 30 trillion cells

• 171 billion cells in the average mail human brain

• 86 billion neurons in the human brain

• 150 million synapses in the human brain

• 206 ones in the human body

• 78 organs in the human body

With 405 billion, there are more parameters in Llama 3.1 than the total number of cells, neurons and synapses combined in the human brain and I would argue we have limited understanding of how every decision is made by a single person, during every second of every day.  Multiply this by 100,000 daily customers and we might just want to focus on outputs instead of focusing on transparency and traceability when it comes to GenAI decision making.

We are early in the development of AI and when we embrace “human in the middle” practices like we have with LeadSticker, a two-way text messaging platform, little companies especially will have a difficult time creating transparency and traceability when GenAI is deployed as a service.

Software-as-a-Service is deeply entrenched in our economy and our companies.  GenAI solutions, like Llama 3.1 are commonly deployed as a service through API protocols, apps and web browsers.  Given this environment, setting an expectation for transparency and traceability of GenAI decision making, sets an awfully high bar.  I suspect 405 billion parameters will be a small number when we look back on AI in the future. 

Returning to my earlier reference with Chaid; when using Chaid, the user can “prune the trees” as it were.  As a result, we may only want to look at 100 levels within the tree or perhaps only 10 levels after the pruning.  We might even want to put the dominant correlations in rank order and save the “top 50” or the “top 500”.  Maybe, just maybe this becomes the start of our transparency and tracking mechanism to see what decisions GenAI is making.  However, the real question for me:  can Chaid or any other stat package analyze 405 billion parameters and present the results in a user friendly representation like a Chaid tree?  And if yes, can it be done in real time?  And can we take a snapshot of the analytics engine results page (AERP) every second and save it to storage so we have tracking and traceability of the decisions GenAI is making?

Or is it simply easier and better to ask GenAI to create a record of every decision it makes and to provide traceability with a visual trail through the data of how it arrived at that decision.

And perhaps that sounds just a bit crazy, and maybe a little too easy.  However, solving this problem is necessary if we want transparent and trackable GenAI decision making.

Your thoughts? Text me now: 206-312-2129