Cracks in Fortress Tech - AI is getting interesting

A long post, but it explains how the major tech firms huffed the paint and believed their own bullshit to block competition but were blindsided by the Chinese. Oops.

fortress built around the tech companies Google, Microsoft, Oracle, Apple, OpenAI and Nvidia

I am going to go loooong today. Sorry in advance, but I here’s a TL;DR

The first wave of tech created a class of obscenely wealth dweebs who then thought they were super smart. But they were mostly lucky, and serially scored. They spent the 2010’s on the stupidest of ideas, as companies like Microsoft, Apple, Meta and Google were entrenching themselves in defensible markets.

These majors tried to do the same thing with the rise of Generative AI (aka ChatGPT), locking out small competitors by funneling lots of money into ever larger systems.

This led Nvidia to become the first $3T company.

In China, a spinout of a tech trading firm built on a (apparent) shoestring a GPT like solution that out reasons OpenAI’s flagship offer.

Shit hits the fan, and a lot fo US based tech companies are scrambling.

Oh, and the Deepseek splash happened LITERALLY days before the $500B “Stargate” announcement that the government was going to build a massive series of data centers in Texas to support future AI innovations (doubly sweet is that Musk’s AI company, xAI, was omitted - sick burn - from this investment) on the assumption that even more scaling is needed.

Now they are flooding the market with FUD to downplay them getting pantsed. But all the major tech leaders are secretly shitting their drawers and trying to figure out how to blunt this attack.

Alas, upon re-reading this, it is superficial, and I could go A LOT deeper and with more meat. If you would like that, please let me know in the comments

Part 1: The Rise of Tech

While I am not going to chronicle from the rise of IBM, Burroughs and Tandem, it is worth a brief digression to the mid 1990’s to anchor the story.

What we know today as the “World Wide Web” really was built from the original DarpaNET that was spec’d designed, and built starting in the 1970’s. It was a text based system¹ where the academic and government research groups could geek out. If you’ve ever used Usenet, you may look back at this era fondly.

In the early 1990’s, a researcher at CERN, Tim Berners-Lee created the HTML or Hypertext Mark-Up Language, a way to reduce the need to drop into various text based tools to access data and services.

This led to what we would call the internet today.

But that original HTML driven “world wide web” used apps like Lynx to access the documents that were beginning to be published. There was no Google, no indexes, you often got the addresses to use from other places like Usenet and peers. Or if you were fancy, you belonged to services like Prodigy or Compuserve and they had indexes of hand compiled links.

Then in early 1993, the NCSA Mosaic² graphical browser was released, and the world was instantly changed.

We’ve come a long way baby!

Immediately, the world wide web as we would recognize it was born.

And almost immediately, the graphical web was born.

Wave 1: Netscape Navigator

As an early adopter, I used the original Mosaic browser, but it was not a great experience. Soon, my circles of online nerds were whispering about this new version called “Netscape”.

The original Netscape was a “fork” of Mosaic with a lot of usability enhancements. One of the original software programmers who worked on this was Marc Andreesen. Netscape was a commercial success, and by selling it, Marc Andreesen became super wealthy. Like serious F-you money.

And like all these dweebs from that era that became wealthy in Tech, he set out to become a Venture Capitalist (VC). That is where you invest your cash in promising, plucky start-ups with the hope that they go big, and then your pot of money gets ever larger.

Lather, rinse, repeat ad nauseum.

The Dot-Com bubble

If you were alive in the late 1990’s, and aware of the tech scene, you probably remember the bat-shit insane number of new companies that appeared, and had these VC’s toss ridiculous amounts of money at them. All to grow to scale. Names like WebVan, Pets.com, and others were instantly huge and appeared in the zeitgeist and all were well funded by these VC’s chasing the next hundo mil.

It was the wild - wild west, until it wasn’t. In late 2000 the party was fading, and in early 2001 (before 9/11) the bubble burst. You could browse this new site eBay and find hundreds of pages of the liquidators of bankrupt start-ups selling new in box hardware. Sun Microsystem servers, Cisco switches and routers, as well as top tier office swag like Hermann Miller chairs and the like.

The VC’s did take a bath, but they came out ok in general.

Web 2.0

There is some uncertainty as to when this second “generation” of the web began, but I broadly tie it to two events that sorta happened about the same time.

First was the rise of the “cloud”. This was kicked off by the launch of Amazon Web Services (AWS). There is a lot of lore about how this happened, but those romanticized tales are largely not true.

What it did was shift the risk of starting a company. In the DotCom era, to start a tech company, you needed to buy hardware (servers and networking gear) and then negotiate space at a data center to install it, and every company had to build out their tech stack. So it literally cost more than a million dollars to even test your hypotheses and business models.

AWS swooped in, and with a credit card, and an Amazon account you could “buy” compute, storage and network functions.

Instantly, the cost to start a start-up became almost negligible. Any Tom, Dick, or Harry could self-fund a start-up.

Two years later, another tectonic shift happened. Apple released the original iPhone, sparking the ‘computer in your pocket’ revolution. Suddenly, you had the world at your literal fingertips.

The pair of these two technologies together drove a revolution in how people interacted with the internet, and sparked a space-race of app developers, and a gamut of services that led to the 2010’s hypergrowth.

But there was a dark swan that hit almost as soon as when this revolution kicked off: The Global Financial Crisis that really hit with the collapse and bankruptcy of the investment bank, Bear-Stearns.

The antidote to the massive world-wide recession that set in when the housing bubble finally and violently burst was a little bit of stimulus, a lot of austerity policies, and central banks basically driving the interest rate to 0%, thus initiating the era of Zero Interest Rate Policy (ZIRP).

A side tour: capital (aka “money”) always chases returns. That is, people with lots of money want to leverage that money to make more money. This is the “Market.” If putting your money in banks basically gives you no return, and austerity measures reduces the return on bonds, that money goes to more speculative assets. Stocks and other vehicles will see a massive inflow. And looking at the 2010’s that is exactly what happened.

So, we are starting the decade of the 2010’s, interest is pegged at about 0%, VC’s who think they are gods on earth, looking for huge successes, watching the high-flying companies like Facebook, Apple, Amazon, and Google going bonkers, and they begin to get antsy to join in on the fun.

That is where this story starts, and it only took about 1,100 words to get here.

---

---

The rise of Crypto and Uber clones

In 2009, as the financial markets began to buckle as the housing bubble unwound, a paper was dropped that described a new, decentralized currency that uses an online, cryptographically signed ledger, and the resulting release of Bitcoin began³.

Bitcoin was thought to be anonymous and untraceable⁴, and if you have more than 2 brain-cells to rub together, you probably can guess what happened next.

If your guess was that Bitcoin, this emergent “crypto-currency” became a vehicle for the trade of illegal items (drugs, weapons, terrorism) and for criminal syndicates to launder dirty money for untraceable clean hard currency, give yourself a gold star.

Read up on the Silk Road Market that Ross Ulbricht ran from 2011 through 2013 for more fun.

But criming⁵ isn’t the only use that was found for this nascent technology. Soon there were imitators like Solana, and they brought additional functionality to the block chain, and enabled people to fork and build “smart contract” businesses on the Solana chain.

And these businesses became very good vehicles for VC’s to pump in some seed funds, lock up large swaths of these Initial Coin Offers (ICO) and then to use the burgeoning ranks of the influencer ecosystem on Twitter, Facebook, and Youtube to pump up the value, before selling the coins and pocketing the money. A pretty good con if you can manage it!

From around 2017 through 2022 this worked a treat, and VC firms like A16Z (Andreesen Horowitz) used this on repeat to not fund innovation — the purported societal value these VC’s espouse — but to do these seedy pump-n-dump scams, separating the unsophisticated from their money, and growing their war chests.

Marc Andreesen rubbing his hands over a treasure chest of gold and jewels

But, in fall of 2022, something happened that really shut this shitty practice down. First the ‘stable⁶’ coins began to wobble, and that caused a lot of semi serious investors to start panicking.

And then it turns out that the exchanges and “investment funds” such as Alameda Research and FTX were really not separate entities, but had mixed flows of cash, doing a lot of inappropriate and uncontrolled activities. Thus, the house of cards crashed back to earth.

FTX is gone, Sam Bankman-Fried is in prison, and Bitcoin is now valueless (oops, it is sitting at about $105,000 per token now, thanks Trump).

But VC’s still had fucktons of money, and nowhere obvious to invest it.

Enter ChatGPT

ChatGPT enters the chat. November 30, 2022, this plucky startup, OpenAI, originally formed as a public good organization to responsibly deploy AI, stuns the world with ChatGPT, a “miraculous” tech that you can have a decent conversation with.

I will not go too far into my issues with the whole generative AI thing, but if you want to get a feel for a skeptic’s views, check out Ed Zitron at Where’s your Ed at? newsletter (not on SS) (a sample: OpenAI is a bad business)

ChatGPT captured the attention of the masses. Like drug dealers, they gave you teasers, and the public lapped it up. The technology is really an extension of the prior generation of tools that are called “Machine Learning”, but this extension into natural language processing amazed people with its conversational nature.

And I will admit that it is impressive.

To do its magic, OpenAI had to scour the internet, basically anything that is publicly available (and some that wasn’t) to create a training set. This is literally petabytes of data.

Then the training happens, and that requires a modest amount of compute, but a lot of GPU power. GPU’s are Graphics Processing Units, and they were developed for highly computationally expensive processing for 3D applications (read: video games).

This drove an exponential increase of spending on these GPU’s, and special data centers to facilitate the training and then public access of these models.

This takes a lot of money, and OpenAI quickly allied itself with Microsoft who committed funding the massive hardware required. Soon Google, Meta and startups began entering the fray. Not just text like ChatGPT, but Dall-E and other image generators happened (note: I used the Substack provided image generator for the graphics in this newsletter), and now video is at your fingertips.

To build this out has required literally C5-A loads of $100 bills

a c5a transport plane being loaded with $100 bills

How much money? It is estimated that the total spend on hardware and data centers to support the rise of generative AI in calendar year 2024 was about $294 BILLION.

And all of this is hugely unprofitable. Right now, OpenAI does sell subscriptions, and they do have paid access to the APIs and that has driven a firestorm of integrations of ChatGPT technology into just about every nook and cranny of apps you use. If you use Microsoft Word, the predictive type-ahead feature is this (and it fucking sucks).

But OpenAI is really just a pit that burns money. They spend about $2.35 to generate $1 in revenue. I do not have a high-falutin MBA, but that doesn’t seem like a good business.

Further, each generation of release (3.5→4→4o) has exponentially increased the cost to train. Already it takes several hundred million dollars of compute, and that is predicted to go to one billion dollars.

The majors (Amazon, Google, Microsoft, and Oracle) are all investing heavily to build out optimized data centers, stuffed with hundreds of thousands of these Nvidia H100 chips (street price about $30K each).

This demand propelled Nvidia to be the most valuable company on the planet, valued at over $3T⁷.

That value was predicated and justified by the exponential demand to build bigger and better training and inference solutions. It became a Tech Bro bragging ritual on how many of these Nvidia GPUs they had been able to hoard and deploy.

Holy shit, 2,100 words to get to the good stuff. What the fuck.

The Deepseek drop

As part of the policies of the government, the US placed export restrictions on the best of the GPU’s going into China. The ostensible goal being to prevent them from competing in the Generative AI bonanza.

The older tech chips were fine, but the tech majors were little piggies and they were able to convince the US to impose these restrictions.

Turns out that caused the Chinese researchers to become very creative in their work.

Deepseek, a startup that was founded by a Hedge fund in China who used Nvidia GPU’s to build a machine learning system to facilitate real time market analysis and trading. This led to them attracting programming and systems talent, home grown, and they decided to try their hands at building a reasoning engine version of the AI Chat technology.

Their lack of access to the H100’s made them look really hard at optimizing, essentially going to near bare metal to optimize the training process.

The net result is that they claim they spent about $5.4M dollars to train their reasoning model that performs better than the latest o1 offer from OpenAI, and they do it with so little processing power that it is between 1/100th and 1/1000th of the cost of the OpenAI offer⁸.

All using far less data center, power and cooling resources.

And, it is open source. That is really freaking out the US AI business⁹. All of them are either dismissive of the technology, doubtful of the claims (and the $5.4M training cost is hard to believe), and spreading tons of FUD.

But, a colleague, my man who is Mr. AI, has installed it and put it through its paces, and on a modest Macbook Air with Apple Silicon, it performs amazingly well. It gets better answers, and unlike the OpenAI, their reasoning steps are exposed, so you can see what it uses to validate, test, and build upon prior results to deliver an answer. OpenAI hides that, as they considered the steps proprietary, and they didn’t want to tip their hand to their competitors.

When Deepseek R1 landed, it caused a tsunami in the market, and in a single day, Nvidia’s market cap lost $600B dollars. Oopsies.

The lessons (and finally the fortress)

We had a confluence of a few factors.

first - the VC’s lost their cash engine of crypto when FTX crashed. Suddenly there was tons of cash with no obvious places to put it to grow.

second - tech majors had milked their platforms. The smartphones are now a saturated market, and the enshittification of their platforms is late stage. No obvious growth.

third - ChatGPT and its splashy intro was the shiny bauble that companies like Google, Meta, and Microsoft could dominate.

And that third item is what happened. Microsoft first sunk $10B into OpenAI, Google and Meta started their own efforts. The costs are huge, so large that a small, scrappy start up can’t afford a seat at the table. If you need hundreds of millions of dollars just to train a new model, instead, you leverage the majors via APIs and the majors become like a new AWS.

Then along comes Deepseek with their siege engine that breaks these fortifications, doing it with a fraction of the investment, on prior generation hardware.

The $600B rout is just the start.

As Ed Zitron is fond of saying, the people running Google, Meta, Apple, and Microsoft are incapable of innovating, and they have adopted the Michael Porter method of erecting barriers.

But barriers don’t work if there are real talented people working to innovate.

Your mileage may vary.

(whew, more than three thousand words, and I could write five thousand more easily)

1. I am simplifying here, humor me

2. Not long after this, I was able to get on. Using Windows 3.1, I had to download and install a TCP/IP stack - a program called WinTrumpet, and then FTP the Mosaic files to get online. It was definitely an experience to be that geeky in the day

3. Being a geek, I was aware of it not long after it became a thing, but upon reading about it, I thought it was a solution to a problem that really didn’t need to be solved, so I ignored it. Now I am kicking myself for not turning my beast of a computer at the time into mining and minting a few hundred coins when they were worth about a few cents each. Sigh.

4. Except that it isn’t really untraceable. As soon as you want to convert your ill-gotten Bitcoins into cash, that transaction is recorded, and your personal information can be extracted. Oops.

5. Fuck you spell check, criming is a word, fight me for it.

6. Stable coins are crypto currencies that are tied to a base currency, typically the US Dollar at a 1:1 ratio, that is one Tether (a common coin) = $1

7. Nvidia designs the chips, but pays semiconductor foundries like TSMC to build the chips. They are mostly an IP based company, even though they make fucking BANK on selling the chips.

8. Full access to the latest ChatGPT offer, marketed to uber users is $200 a month. And OpenAI is still far from profitable.

9. Sure, Meta’s Llama is open source and I have run it on my computer, but Deepseek R1’s small footprint model runs on a Raspberry Pi, a hobbyist platform that is like $150.