Generative AI FAQ

Here are some common questions I see / have to field every time I have to talk about Generative AI in the current hype cycle environment, so I want to type some answers to save everyone some time.

Something missing you’d like to talk about? My consulting rate is $250/hr. If you’re interested, hop over to the Contact page and introduce yourself.

I’ve talked about this at length, but right now it’s a marketing term used to describe Large Language Models (LLMs) or Diffusion Models (DALL-E, Midjourney, Sora, etc).

In the case of LLMs they use an attention training mechanism to produce text that looks like it was generated by a human, using statistics and randomness.

Diffusion models do something similar but with images and an interesting technique using random noise. Both of them use Neural Networks under the hood.

I don’t see how. It’s using math and statistics to produce the next tokens in a sequence. The idea that just feeding more data than humans have ever created into an LLM will somehow give it the ability to learn on its own is magical thinking and OpenAI and the like are banking on you believing it to keep giving them money until they pull the rug out from under you.

I suspect the true believers think that human cognition is the same kind of math, but even if you believe that, our brains work on far less electricity with far less information in order to do way more than just regurgitate text.

Relatedly:

No. A “neuron” in a neural network is based on a thing called a perceptron, which itself is based on the observation on how biological neurons fire. You can read more about how artificial neurons differ from biological neurons here. Artificial Neurons have been iterated upon and now no longer look like a perceptron, but they also look very different from a biological neuron.

Does that actually matter? We’re not sure. We still don’t fully understand how the brain works.

Regardless, biological systems can do way more than neural networks.

Uhh, well, that makes a lot of assumptions:

1. We have a definition for AGI that everyone agrees on (we don’t).
2. We have a way to accurately measure LLM performance (we don’t).
3. LLM performance will increase exponentially (it’s probably not).
4. The Energy Grid can keep up with demand or energy usage will decrease drastically (it’s not, and big tech isn’t incentivized to do so).
5. Synthetic training data will actually function. (We don’t know!)
6. LLMs will be integrated into a product that makes decisions on its own
   1. IFTTT is making a comeback! Wooo.

Long story short, there are a lot of reasons to believe AGI is not a few years away.

The “AI Industry” (another marketing term) as such has done these bust and boom cycles many times before and they look…pretty similar to this. The term for this is AI Winter. Likewise, certain billionaires has been promising universal self driving cars is just a few years away (sometimes it’s 2, sometimes it’s 5). Heck there’s even a paper on the subject.

Oh goodness, no. They’re statistical models. The bias in the training data is reproduced in the output of the model.

There are so so many articles about this but this Bloomberg Article does a much better job than I ever could.

RLHF (Reinforcement Learning through Human Feedback) is a technique by which you nudge a model to give different output by making a reinforcement algorithm that makes it produce output that you prefer.

It has a couple of fun problems, namely you have to have a human do the reinforcement for every thing that you want it to be better at. You also have to ensure the humans themselves are not introducing more bias (unconsciously, purposely, or because their employers want them to).

You can have it run through multiple rounds with different humans as a mechanism to reduce the bias, but as far as I can tell a lot of this work is being done by low-wage workers in the global south working on platforms like Mechanical Turk for a few dollars an hour because the AI companies want to keep costs down.

Regardless, biased data goes in, biased data comes out and the internet is biased.

I don’t think RLHF is just going to magically solve bias, I think you need to carefully curate the inputs and use RLHF and carefully add safety checks.

Which, btw, hackers can easily bypass. We just keep finding new and exciting jailbreaks. It’s great.

Yes! Remember that time that Facebook’s AI identified black men as primates? Well I do!

Bias in training data!

Classifier models are not generative AI, btw, and they can be useful for a wide variety of things, but you have to be careful how you deploy them. For example, Apple offers a nice tutorial on how to train a model on pictures of hands making rock-paper-scissors gestures which will teach you about things like confidence scores and the like - but that’s very low stakes.

Now imagine you train a model that identifies which mushrooms are safe to eat? What happens when the algorithm gets it wrong and identifies a poisonous mushroom as safe? That could kill someone. Adding a warning label probably isn’t going to cut it. You have to really critically examine the good and harm on these things.

Several someones have also used LLMs to generate foraging books which will likely have the same effect were someone to take the advice in them.

But let’s talk about making books and art!

What does that actually mean?

I suspect it means that you don’t need to actually be involved in the production of the output and you can just shout an idea at the LLM or Diffusion model and it makes something you might enjoy or think is useful for copywriting, or slap on a LinkedIn post… but you’re not doing art.

You’re essentially “commissioning” something that “approximates” art, but you’re not doing the art yourself.

“But I spent hours crafting the perfect prompt!”

Okay, cool, you nudged the statistical probability of the model to give you something you thought was cool by random chance. Nice.

But look, that’s not the art. The art is in the act of creation. You can already commission an artist to do this for you, you just don’t get the instant gratification. What you gain instead is a human who has ideas and can work with you to get something great out the other end.

What it is doing is commoditizing the production of some things that execs think will be cheaper in the long run.

Meanwhile, Mira Mirati is sitting in front of a camera just showing her disdain for artists:

"Some creative jobs maybe will go away," Murati said in an interview at The Thayer School of Engineering at Dartmouth University last month, as quoted by Fortune. "But maybe they shouldn't have been there in the first place."

I don’t want her view of the future.

I talked about this a lot over in the rant but there are already tools for these things that work better, use way less energy, and weren’t built off of the back of unlicensed content.

For example, even though Grammarly has added some more Generative AI stuff as part of the hype cycle, they've had grammar checking ML models for a long time that work better than the current iteration of chat bots at spotting and surfacing grammar, syntax, and other errors you might make in writing. It doesn't rewrite the whole thing for you, it takes the words you've already written and gives you some pointers that you can take or leave. Those are small, specialized models, rather than a "magical text regurgitation machine".

For the "blank page problem" there are a bunch of techniques that you can use to get started that don't involve asking the chat bot for ideas.

Seriously, the rubber duck uses zero energy (after it was produced) and works just as well for me as an LLM would. Asking the LLM to get you started will bias your writing towards the absolute average for a given topic, because the LLM is going to give you ideas that are (probably) statistically significant in its dataset. Sometimes those will include giant falsehoods.

A traditional random sentence generator may even spark ideas just as well as the LLM while using 1/1000th of the power.

Moore’s law is Dead.

That’s it. Maybe we’ll get more powerful chips that use less energy if we specialize those to the kind of Matrix Math these algorithms use, or maybe we’ll come up with a new architecture, but that’s not going to come from current LLM / Diffusion architectures.

Of course, people are going to try and reduce the amount of power these things take, like this Ars Technica Article reports. It remains to be seen if new architectures are going to actually going to pan out. But if you change the entire architecture of the thing, is it really the same thing?

Meanwhile, the big three are blowing through their emissions targets while Texas cannot keep its power grid functional during record heat.

So, that’s super cool. We’re running low on time to deal with the climate crisis, but Sam Altman thinks that LLMs can solve climate change somehow. Neat.

Also, apparently we're going all in on nuclear power while also not shutting down older power plants. We're going to blow past our carbon goals for fancy autocomplete.

I disagree. I wrote a whole post about this. The long story short, it’ll generate code it’s seen before, sometimes that code will be right, but I very much expect the humans to just trust the generated code, not vet it, and everything is going to be a security nightmare for years.

This is going to especially be a problem for the Junior developers, as I mention in this blog post. Those folks who haven't learned enough yet to critically examine the LLM's output. That's going to be a compounding problem. Related problem, the more these things proliferate, the worse that problem is going to be. You should go read this article by Baldur Bjarnason related to the topic wherein he points out that LLMs don't produce anything new, they simply entrench the old (poorly).

I think I’m in the minority opinion here, partly because there’s extreme pressure from the management class to do more with less. So I get the sentiment, but I don’t think it’s going to drive quality.

Jensen Huang? Is that you?

No, I don’t think this is going to displace software developer jobs any time soon. Or maybe ever. First off, who’s making the AI models? Secondly, Jensen Huang (and others like him) are not developers, and they’re incentivized to say things like this to sell their chips and rocket their stock price to the moon.

It’s not true and it’s not being said from a place of honesty.

Nevermind that writing code is only part of the job, these things just don't actually work that well. They still hallucinate, they don't understand context, and their overall language support is totally biased towards their dataset (I guess if you're a React dev you might be in trouble).

Oh, not at all. I think the Cybersecurity field is going to have a lot of work over the next decade. If you’re in that field, hat tip to you, friends. It’s gonna be fun.

Here's a pile of links:

• AI hallucinates software packages
• OpenAI Faces 2 Massive Security Problems This Week
• Gemini Scanning Google Drive without permission
• Romantic AI Chatbots Don't have your Privacy at Heart - this one is privacy not security but it does mention security in the article.
• The Unseen Threat: Data Leakage in Generative AI

There are a few categories here, but the major ones are that the LLMs themselves can create insecure code (because their training data includes a lot of insecure code), it can hallucinate fake packages which will enable supply chain attacks, and the companies rushing to roll all of this out are firing the normal people involved in securing their products to hire more AI people or buy more servers.

Well, I want to say "helping them do dull administrative tasks", but as near as I can tell it's mostly "intimate chat bots".

Seriously, Character.ai and Replika are full of them. OpenAI had to ban a whole bunch of them.

Not to mention that there's a huge problem with the diffusion models creating deepfake porn of regular people and creating abuse material. I'm not going to link to that, you can find it easily via a search.

So, sort of. There's a temperature setting you can use on most of the models that influences how often the model picks the most likely word vs one of the less likely words. If you set that to 0, it will only pick the most likely words in a chain, making it...mostly deterministic for that specific prompt.

If you change the prompt at all, or include something else in the context window, it will change.

That's part of the "magic", the randomness is a feature. It's also what causes "hallucinations".

The thing you need to understand is that everything in an LLM is a hallucination. That's how it works. It has no concept of "truth", it's just generating the statistically most likely (sometimes, see the temperature setting) token in the sequence. If it goes on a wild tangent about pringles in the middle of a book report about Moby Dick, that's a feature of the system.

I don't see how. You could use a ton of RLHF for everything someone could possibly ask, but given how the response changes (sometimes drastically) when you change the prompt I'm not sure how you can solve the "problem" with that. You could keep using things like RAG (retrieval augmented generation) to skew the results to the things inside of the context window, but it will still go off the rails sometimes.

Which makes it pretty useless for things that require factual accuracy - because if you have to check the whole output, are you saving any time or effort?

Cool, how? Like… how are you going to wire up the LLM to something? I talked a lot about GPT’s code execution capabilities, but it can still hallucinate and get stuff wrong.

The onus is on these companies to tell us what they’re actually going to do.

Not at all. It's very useful to understand how they work so you don't fall into a magical thinking / anthropomorphism of these things but there are folks out there which will say that "you just don't understand" when you see massive energy usage from these things, or that "it'll take everyone's jobs".

Educate yourself, but don't let the likes of AI company CEOs (most of whom aren't data scientists themselves) make bold claims without evidence.

A lot, but the AI companies don't actually publish reports with numbers that make them look bad so we've got to go to independent reporting.

Relatedly, I've seen the argument that "Oh the water is dumped back into the river just a few degrees hotter". Which isn't true, a lot of it is being turned into steam, where it'll enter the water cycle and not be available to that region for however long it takes to rain in that particular location.

Not to mention, they apparently often treat the water with chemicals which makes the water unusable as drinking water if it doesn't get evaporated off.

Sure, here's a non-exhaustive list.

• The Intelligence Illusion

• On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
• On the Limitations of Large Language Models (LLMs): False Attribution

• Challenging The Myths of Generative AI
• The LLM Honeymoon Phase
• What do Google's AI Answers cost the Environment

• Internet of Bugs Coding LLM Showdown