If you’ve been closely following the open AI advances led by Sam Altman, whose neural nets can now write original text and create original images with astonishing ease and speed, you can skip this part.
On the other hand, if you’ve only been vaguely interested in the company’s progress and the increasing traction that other so-called “product” AI companies are suddenly gaining, and want to better understand why, you might benefit from this interview with James Currier, five-time entrepreneur and now venture investor, who five years ago started his Co-founded the NFX company with several serial entrepreneur friends.
Currier falls into the camp of people who follow progress closely — so closely that NFX has made several related investments in “product technology,” as he describes it, and it’s gaining more and more of the team’s attention each month. Of course, Currier doesn’t think the buzz around this new wrinkle in AI is too much hype, as the wider startup world is suddenly facing a much bigger opportunity for the first time in a long time. “Every 14 years, we get one of these Cambrian explosions,” Currier said. We had one on the Internet in 94. We have a mobile phone in 2008. Now we have another one in 2022.
In retrospect, this editor wishes she had asked better questions, but I’m also learning here. Excerpts from our chat follow, edited for length and clarity. You can listen to our long conversation here.
TC: There’s a lot of confusion about generative AI, including how new it is or how it’s become the latest buzzword.
JC: What’s happened to the AI world in general is that we think we can have deterministic AI that can help us determine the truth of something. For example, was it a broken piece on the manufacturing line? Is this the right meeting? You decide anything using AI just like a human would. AI has been on the rise for the past 10 to 15 years.
Other algorithms in AI are these diffusion algorithms, which are meant to look at huge corpora of content and generate something new from them, like, “Here’s 10,000 examples. Can we create a 10,001st example like this?”
Until about a year and a half ago they were very fragile, pretty fragile. [Now] Algorithms have improved. But more importantly, the corpus of content we’re looking at has gotten bigger because we have more processing power. So what happens is that these algorithms run on Moore’s Law — [with vastly improved] Storage, bandwidth, computational speed – and suddenly being able to produce something that looks like something a human would produce. That means the face value of the text it writes and the face value of the drawing it draws are very similar to what a human does. And all this happened in the last two years. So it’s not a new idea, but it’s new at that threshold. That’s why everyone looks at it and says, “Wow, magic.”
So it’s the compute power that suddenly changed the game, not the technological infrastructure that was missing before?
It didn’t change suddenly, it changed gradually until it got to where the quality of its generation made sense to us. So the answer is generally no, the algorithms are very similar. Among these diffusion algorithms, they are somewhat better. But really, it’s all about processing power. Then, about two years ago, the [powerful language model] GPT came out, which was an on-premise type of computation, and then GPT3 came out [the AI company Open AI] will do [the calculation] For you in the cloud; Because the data samples are so large, they have to do it on their own servers. You can’t afford it [on your own]. And at that point, things really escalated.
We know that because we invest in a company that makes AI-based productive games, including “AI Dungeon”, and I think a large part of GPT-3’s calculation is coming from “AI Dungeon” at one point.
Does “AI Dungeon” require a smaller team than another game-maker?
That’s one of the big advantages, for sure. They don’t have to spend all that money to keep all that data, and they can put together tens of thousands of gaming experiences with a small group that everyone can take advantage of. [In fact] The idea is that you’re going to add generative AI to older games, so your non-player characters can actually be more interesting than they are today, but you’re going to have fundamentally different gaming experiences coming from AI into gaming. Adding AI into existing games.
So is there a big change in quality? Will this technology plateau at some point?
No, it’s always better. Differences in increments will be small over time because they are already coming along so well.
But the other big change is that Open AI isn’t really open. They created this wonderful thing, but it was unopened and very expensive. So groups like Stability AI and other people got together and they said, “Let’s make open source versions of this.” And in that time, the cost has come down 100x in the last two or three months.
These are not branches of Open AI.
Not all of this generative technology is simply built on the Open AI GPT-3 model; That’s just the first. The open source community has now replicated a lot of their work, and in terms of quality they’re probably eight months behind, six months behind. But it will get there. And because the open source versions are a third or a fifth or a twentieth of the price of open AI, you’re going to see a lot of price competition and you’re going to see a proliferation of these models that compete with open AI. . And you’ll probably end up with five, or six, or eight, or 100 of them.
Then unique AI models are built on top of them. So you might have an AI model that actually composes poetry, or you might have AI models that actually see how you make visual images of dogs and dog hair, or you might have one that really specializes in writing sales emails. You’re going to have a whole layer of these unique AI models that are purpose-built. Then above That, you have all the manufacturing technology, it’s like: How do you get people to use the product? How do you get people to pay for the product? How do you get people to sign up? How do you get people to share it? How do you create network effects?
Who makes money here?
The application layer and network effects where people go after distribution is where you’re going to make money.
What about larger companies that can incorporate this technology into their networks? Isn’t it very difficult for a company without that purpose to come out of nowhere and make money?
I think what you’re looking for is something like Twitch, where YouTube could have integrated that into its model, but they didn’t. And Twitch has created a new platform and a valuable new piece of culture and value for investors and entrepreneurs, albeit a difficult one. So you’re going to have great founders who are going to use this technology to give them an advantage. And that creates a seam in the market. While the big guys are doing other things, they are able to build billion dollar companies.
The New York Times recently ran a piece featuring some creatives who said the productive AI apps they’re using in their respective fields are tools in a broader toolbox. Are people here naive? Are they at risk of being replaced by this technology? As you say, the team working on the “AI Dungeon” is small. This is good for the company but bad for the developers who worked on the game.
I think with most technologies, there’s a discomfort that people have [for example] Robots replacing jobs in auto factories. When the Internet became available, many people doing direct mail threatened to sell directly instead of using their paper-based advertising services. But [after] They have embraced digital marketing or digital communication through email, they have probably had tremendous bumps in their careers, their productivity has increased, speed and efficiency has increased. The same is true of online credit cards. We probably didn’t feel comfortable putting credit cards online until 2002. But those who received it [this wave in] 2000 to 2003 was better.
I think that’s what happens now. Forward-thinking writers and designers and architects who embrace these tools to deliver a 2x or 3x or 5x productivity lift are going to do very well. I think the whole world will increase productivity in the next 10 years. It’s a great opportunity for 90% of people to simply do more, be more, do more, connect more.
Are you wrong not to do Open AI? [open source] What was born around it?
A leader behaves differently than followers. I don’t know, I’m not inside the company, I can’t really say. I know there’s going to be a big ecosystem of AI models, and I’m not clear how an AI model will be different, because they’re all the same quality, and it just becomes a price game. It seems to me that the winners are Google Cloud and AWS, because we all produce stuff like crazy.
Open AI ends up moving up or down. Maybe they’ll become like AWS themselves, or maybe they’ll start making specialized AIs that they sell to specific verticals. I think everyone in this space has the potential to do well if they navigate it properly; They have to be smart about it.
NFX has a lot more about generative AI on its site that’s worth a read; You can find it here.