Generative AI Models – What Are They?

Introduction

The conversations around artificial intelligence (AI) in media and entertainment can be conflicting because there are varying perspectives on how it fits into the narrative of creating and consuming content. Depending on who you talk to, their experiences or personal philosophies will shape how they interpret AI’s role in content creation pipelines.

At the heart of these discussions are generative AI models, trained on large datasets to generate specific types of content, such as images, videos, music, voice, and 3D assets. These models are improving rapidly, with new versions and updates released frequently, which can make it difficult to keep pace with the latest advancements.

However, no single model truly ‘works’ for all aspects of the content pipeline. Different applications, hardware, interfaces, and workflows can affect inference, which is the operation of a trained model using inputs such as prompts, images, or other control variables to predict and generate content as an output. In turn, inference impacts processing time, output quality, and computational requirements, meaning the most suitable model and platform may vary depending on your needs.

While there are many avenues for discussion around generative AI, it’s worth mentioning that this post is not an attempt to identify the ‘best’ model or tool for content creation. That question is inherently subjective, and I believe your own perspectives will help you determine whether a given model or tool’s output meets your standards. What this post will cover, at a high level, is the different types of generative AI models and where you can access them.

Generative AI Explained

Artificial intelligence is a broad term that refers to many different technologies. In content creation, it can describe various AI models, software, tools, and features – making it difficult to distinguish what each actually does or how they differ from one another.

To make sense of these differences, it can be helpful to categorize AI based on how they are offered and used:

• AI as Products – Artificial intelligence solutions offered commercially for access or deployment.
• AI as Tools – Products functioning within a creative pipeline or workflow to generate content.
• AI as Features – Capabilities embedded in software that support or enhance parts of the creative process.

As a product, generative AI can be commercially distributed through web-based platforms, APIs, downloadable models, or managed hosting solutions. For organizations deploying generative AI solutions, your infrastructure influences how effectively systems scale to meet demand, maintain responsiveness under variable workloads, and integrate with existing applications and workflows.

Within the content creation pipeline, generative AI tools enable creators to produce various types of content. These tools are powered by AI models, each trained to generate specific types of outputs, such as images, video, music, sound effects, or 3D assets. As a result, content creators have the opportunity to experiment, iterate, and explore alternative workflows that might have been difficult, time-consuming, or costly using traditional production. However, not all models within a given category produce the same type of content. Developers train models on datasets that shape the content they generate, and these datasets can influence stylistic outcomes, consistency, quality, and performance. There is no single model that works ‘best’ for every task, so it’s important to research and test models to find those that most align with your needs.

Aside from generative AI tools, some mainstream software applications include features that leverage other types of AI or machine learning (ML) to automate tedious or repetitive tasks that support the creative process. Classification AI labels or categorizes content and encompasses activities like analyzing and tagging media files, object detection, and masking. Other ML-assisted functions include enhancing audio, speech-to-text transcriptions, stabilizing footage, and motion tracking.

Different Types of Generative AI Models

As mentioned in the intro, the generative AI landscape is evolving rapidly, with new models and updates constantly emerging. To provide a clearer view of the current tools shaping the field, I have included four tables below that break down many of the popular generative AI base models available for different types of content.

It’s worth noting that some models may be missing, as my research was limited to what could be reasonably reviewed at the time of writing, and the total number of models available is too large to comprehensively cover for this type of post. The tables include only base models and exclude LoRAs or fine-tuned variants. Covering topics such as data security, copyright, and licensing for commercial or personal use is also outside the scope of this overview. Please note that these tables have multiple pages of entries and can be sorted or filtered if you want to narrow the list down.

Image Models

The table below highlights generative AI models for creating 2D images.

Video Models

The table below highlights generative AI models designed for video generation.

Audio Models

The table below highlights generative AI models pertaining to music, voice, lip-sync, and sound effects.

3D Models

The table below highlights generative AI models for various types of 3D content, including 2D→3D outputs, environments, meshes, textures, and point cloud generation.

How to Access and Use Generative AI Models

Generative AI models can be accessed through software applications hosted locally on your system or through cloud and web-based platforms. This includes new tools explicitly built for generating content, as well as traditional applications integrating AI features into their software. That said, not all platforms offer the same models or features. For example, Adobe Photoshop runs inferences in their cloud servers from prompts and tools such as generative fill – but it only offers a limited set of models, whereas other applications or platforms may provide similar features with access to a larger number of models.

Generative Fill tool in Adobe Photoshop with generative AI model options

To perform inference on your own hardware, you can download AI models from repositories such as Huggingface or GitHub and use tools like Comfy UI or TouchDesigner for generating content locally. Running models on your own system allows you to work offline and maintain full control over your data, creating a self-contained workflow. However, not all models can run on local hardware, and larger or more complex models may only be accessible through cloud-based platforms or other hosted services.

Cloud- and web-based platforms such as Adobe Firefly, Artlist, Freepik, Higgsfield, Krea, and OpenArt provide access to multiple generative AI models. All processing is done on the platform’s servers, so content creators don’t need to invest in high-performance systems; they only need a web browser and an internet connection to run inferences. These platforms also differentiate themselves by offering tools and features that are exclusive to their subscribers. For example, MidJourney makes its models available only through its website and requires a subscription. Other web-based platforms like ComfyUI Cloud, Flora, SOTA, and Weavy offer a different type of interface, such as node-based workflows that allow users to combine models, adjust parameters, and manage outputs in a structured way, which can lead to more predictable and consistent results when generating content.

Node-based workflow in ComfyUI using Flux.2 Klein 4B model for image generation

Where Can Generative AI Models Be Useful in Content Creation?

There isn’t a single “right” way to use generative AI in content creation. Some models run locally on consumer or workstation-class hardware, giving more control over your workflow. Others are accessed through cloud- or web-based platforms, where computations are handled remotely, so creators don’t need a high-end workstation to generate content.

From my perspective, regardless of the models or applications used, generative AI should be considered as an alternative method or supplementary tool that bridges gaps across different stages of the content creation pipeline. Rather than replacing people, I hope that AI models can let artists and creators do more and avoid tedious, repetitive tasks. For example, within pre-production, they can help you generate mood boards, refine storyboards, or mock up placeholders for conceptual ideas. In production, they can be used as an alternative method to traditional practices for music, graphics, photography, or video productions. In post-production, generative AI tools can assist with tasks such as refining visual effects, completing or reconstructing missing frames, enhancing audio clarity, adjusting colors or lighting, removing unwanted elements, and preparing assets for final assembly.

For content creators, it’s important to understand both the capabilities of generative AI models and the resources required to select the right hardware and platform for your needs. It is equally important to understand how models are trained and the terms under which outputs can be used. Not all models produce content that is safe for commercial use, so reviewing licensing and copyright terms before generating or distributing outputs, and understanding which models are trained on “clean” data, will help protect your projects from legal risks and support ethical and responsible use of generative AI.

Nonetheless, whether you are looking to deploy, host, or generate content using AI models, hardware will be the core component that enables you to offer or produce great content. If you are a creator looking to experiment with AI or incorporate it into your workflow, we also have recommended systems for Generative AI to help you get started. For those looking to develop or deploy AI for a small team, you can check out this list of recommended systems as a starting point. Organizations and teams looking to host, train, and scale the development of their models will eventually need server-class solutions.

---

Whatever your workflow, if you are looking for a new computer, the Puget Systems workstations on our solutions page are tailored to excel in various software packages from content creation to engineering and scientific computing. If you prefer to take a more hands-on approach, our custom configuration page can help you to configure a system that matches your exact needs. Or, if you would like more guidance in configuring a workstation that aligns with your unique workflow, our knowledgeable technology consultants are here to lend their expertise.

---