GPT-3 Just Changed What I Think Is Possible

I got access to GPT-3 in early June, after watching the initial demos circulate on Twitter. The reactions ranged from "this is the singularity" to "it's just a statistical pattern matcher." Both reactions made sense. Neither captured what actually happened when I started playing with it.

What struck me wasn't that it worked. It's that it worked in ways I didn't expect, and failed in ways that taught me something different about what language models can do.

The Prompt Engineering Surprise

The first thing that surprised me wasn't how smart GPT-3 was. It was how much the way you asked mattered.

Ask it "What is the capital of France?" and you get a boring answer. Ask it "Complete this table of countries and capitals: France | Paris, Germany | Berlin, Italy |" and something different happens. Same model. Radically different output quality depending on how you frame the problem.

This immediately clicked as important. It meant you weren't just calling an API. You were having a conversation with the model's training data. You had to know how to speak to it. Ask for a Shakespearean sonnet about a technical topic and it would genuinely produce one. Not perfect, but genuinely trying.

The implication: this changes how we build software. Instead of writing explicit logic for every edge case, you might be able to describe what you want and let the model figure out the pattern. But you have to describe it right.

The Reliability Question

Here's what nobody on Twitter was talking about: when I tried to use GPT-3 for something I actually cared about, it failed unexpectedly often.

I built a simple customer support ticket classifier. Train a model on your own data, you'd get consistent results. GPT-3 gives you different results depending on how you phrase the prompt, what examples you give it, and what phase the moon is in (okay, not that last one, but the randomness is real).

Sometimes it's brilliant and intuitive. Sometimes it's confidently wrong about something a fifth grader would get right. There's no clear pattern to when each happens.

This is frustrating if you're trying to build production software, but it's valuable if you understand what it means: GPT-3 is brilliant at interpolation—predicting what comes next in a pattern it's seen—but terrible at reasoning about things it hasn't seen clearly enough to internalize.

The Copyright Question

As people started building with GPT-3, everyone simultaneously realized the same thing: we don't really know where this training data came from or what rights we have to use patterns extracted from it.

If GPT-3 generates code by interpolating billions of examples from GitHub, what copyright issues does that create? If it reproduces entire paragraphs from its training data (which sometimes happens), whose content is it reproducing?

This isn't a technological problem. It's a legal problem that nobody has solved. And it matters because if GPT-3 becomes critical infrastructure, the legal uncertainty becomes everyone's problem.

What This Changes About Building AI

Watching people build GPT-3 experiments in June, I started seeing three different categories emerge:

The unambiguous wins: Tasks where the model was clearly better than any existing approach. Generating reasonable-sounding product descriptions from features. Classifying text into predefined categories. Tasks that were previously expensive to automate became cheap.

The uncertain middle ground: Things where GPT-3 was surprisingly good but not reliably good. Writing code snippets. Translating between formats. You'd get great results 85% of the time and completely broken results 15% of the time. These tasks stayed experimental.

The failures: Things that required actual reasoning or specialized domain knowledge. GPT-3 would output confident nonsense. Financial analysis. Medical advice. Code review. Areas where you need reliability more than capability.

The practical implication for anyone building AI products: this dramatically lowers the barrier to entry for tasks in the first category. Your competitive advantage isn't "can we make an AI?" anymore. It's "can we find edge cases, handle failures gracefully, and build something useful around this capability?"

The Economic Shift

Here's what I was thinking about by the end of June: what jobs become different when a single API call can generate reasonable customer service responses or write functional code snippets?

Not "what jobs disappear," because it's not that binary. But entire categories of jobs become leverage-based. A customer support agent with GPT-3 handling the first-pass response becomes dramatically more productive. A developer with GPT-3 writing boilerplate becomes faster at the tedious parts and can focus on architecture.

The companies that win aren't the ones that replace humans with AI. They're the ones that use AI to amplify human capability. Give people superhuman tools and see what becomes possible.

The Hype Reality Check

By July, I was seeing two different reactions coalescing. The "this is everything" crowd was talking about AGI and the singularity. The "it's just statistics" crowd was pointing out limitations.

Both were right, but they were measuring different things. Yes, it's just pattern matching. And yes, pattern matching applied to 500 billion examples of human text produces something that sometimes feels like intelligence.

What I kept coming back to: this is a tool that changed what's possible, but not in the revolutionary way. It's more like the internet—extremely significant, completely changes what you can do, but you still need actual engineering and design to build anything worthwhile.

What I'm Actually Betting On

By June 2020, I was already convinced that language models were going to be infrastructure, not products. The companies that win won't be the ones that sell "AI-powered X." They'll be the ones that integrate this capability deeply into existing tools in ways that make people more capable.

The demos going viral on Twitter were cool. But the actual value was going to accrue to people who understood both the capability and the limitations and could thread that needle into real products.

I didn't know exactly when that future arrived or what it would look like. But I knew the trajectory had changed.

The singularity? That still seems unlikely. But making software that reasons about language in interesting new ways? That felt suddenly inevitable.