On August 20, 2011, Marc Andreessen of a16z, published a pivotal story in the Wall Street Journal: Why Software is Eating the World. Today, September 26, 2022, I’m publishing Why AI Is Eating The Web3 Creator Economy.
You see, it all stems from the advances in machine and deep learning that has exploded on the scene with DALL-E, MidJourney, Stable Diffusion and now more recently with NVIDIA’s announcement about Get3D.
NVIDIA’s new GET3D AI-powered tool will mess with many recent startups who have developed tools and apps that scan objects to populate metaverse worlds.
“Trained using only 2D images, NVIDIA GET3D generates 3D shapes with high-fidelity textures and complex geometric details. These 3D objects are created in the same format used by popular graphics software applications, allowing users to immediately import their shapes into 3D renderers and game engines for further editing.”
NVIDIA says it took just two days to feed around 1 million images into GET3D using A100 Tensor Core GPUs — this should give you a sense of the speed and scale that this will disrupt other tools designed to scan everyday objects manually.
Its ability to instantly generate textured 3D shapes could be a game-changer for developers, helping them rapidly populate virtual worlds with varied and exciting objects.
With the help of another NVIDIA AI tool, StyleGAN-NADA, it’s also possible to apply various styles to an object with text-based prompts — so roughing up a building with decay or creating a 4x4 covered with mud would be entirely possible and easily done.
What’s more, is that these images will no doubt end up as USD (Universal Scene Description) file formats which NVIDIA and others are pushing to become one of the interoperable standards for objects.
This means that promise of a democratized creator economy where people can make money from uploading their own images to sell on something like Sketchfab will actually become a thing of the past already.
And if you take at Quixel as an example, their megascan library could be ingested and instantly become redundant as a business.
Between MidJourney, DALL-E, and now Stable Diffusion it won’t be very long before we’ll be able to enter a text prompt and AI will generate a metaverse from it.
Fully realized 3D and immersive environments aren’t that far off, I’d hazard a guess at less than 2 years before we could have a plug-in for Unity or Unreal Engine to be able to procedurally generate virtual worlds that we could move through and interact with purely with a phrase or few words.
There are already tools out there that shorten the design process but the final step in this journey is just telling the AI the result you want with little else involved.
It’ll then be down to the user to tweak into shape or leave it as is. We’re already seeing signs that “prompt engineers”, people who understand how to manipulate the phrases needed to get the best initial results, will spring up and command a decent price — for a short period of time at least.
But they’re not true creators, and AI will be leaving those people in its wake as it dominates the industry.
Fabian Stelzer has created a set of sci-fi short videos 100% generated using these tools together with community-driven narratives and this is extremely exciting to see just how AI can democratize creativity but in the same breath it’s also removing huge swathes of traditional creators who would normally get involved in these projects.
“The experience of being able to “prompt a trailer into existence” on the fly while discovering this world myself was incredibly thrilling, a glimpse of the future.
As the tools are fed more real-world data to train on it only increases the likelihood of this happening faster.
But there’s a backlash already.
In a polarizing move, Getty — the visual media conglomerate and popular supplier of stock imagery and editorial photography — announced this week it would be banning artwork with artificial intelligence, citing copyright issues.
“There are real concerns with respect to the copyright of outputs from these models and unaddressed rights issues with respect to the imagery, the image metadata, and those individuals contained within the imagery,
We are being proactive to the benefit of our customers.”
Getty is a massive image repository that is used by media companies all over the world. It contains images and original photographs that subscribers can use to illustrate their articles, for a licensing fee.
Getty announced that it would be stepping up the policing of images made using AI on its platform saying that any images made using programs such as Stable Diffusion, DALL-E, and Midjourney would be banned.
So now there’s a divide happening — a war between creators and those who use AI to create. And this will have casualties that ripple across the industry.
Software may be eating the world, but AI is fast eating the creator economy before it’s even had a chance to start.