Stable Diffusion integration with Replicate and Laravel

Laravel wrapper around the Replicate API to use Stable Diffusion to generate text2img.

• 🎨 Built-in prompt helper to create better images
• 🚀 Store the results in your database
• 🎇 Generate multiple images in the same API call

Installation

You can install the package via composer:

composer require rulilg/laravel-stable-diffusion

You can publish and run the migrations with:

php artisan vendor:publish --tag="stable-diffusion-migrations"
php artisan migrate

You can publish the config file with:

php artisan vendor:publish --tag="stable-diffusion-config"

This is the contents of the published config file:

return [
    'url' => env('REPLICATE_URL', 'https://api.replicate.com/v1/predictions'),
    'token' => env('REPLICATE_TOKEN'),
    'version' => env('REPLICATE_STABLEDIFFUSION_VERSION', 'a9758cbfbd5f3c2094457d996681af52552901775aa2d6dd0b17fd15df959bef'),
];

Register in Replicate and store your token in the REPLICATE_TOKEN .env variable.

Optionally, you can publish the views using

php artisan vendor:publish --tag="laravel-stable-diffusion-views"

Usage

It's important to understand that the image generation process is async. We need first to send a request to Replicate and, after a few seconds, we can fetch the results back.

This is the code we need to generate an image. This will return a StableDiffusionResult model.

use RuliLG\StableDiffusion\StableDiffusion;
use RuliLG\StableDiffusion\Prompt;

StableDiffusion::make()
    ->withPrompt(
        Prompt::make()
            ->with('a panda sitting on the streets of New York after a long day of walking')
            ->photograph()
            ->resolution8k()
            ->trendingOnArtStation()
            ->highlyDetailed()
            ->dramaticLighting()
            ->octaneRender()
    )
    ->generate(4);

After the generate() method is called, an API request to Replicate is sent so they can start processing the prompt. This will generate a record in our database with a status="starting" value, and the ID returned by Replicate.

To retrieve the results back from Replicate, we need to run the following code:

use RuliLG\StableDiffusion\StableDiffusion;

// we get the Replicate ID and fetch the results back.
// This method will automatically update the record with the new information
$freshResults = StableDiffusion::get($result->replicate_id);
if ($freshResults->is_successful) {
    dd($freshResults->output); // List of URLs with the images
}

If the results were already fetched, then no API request will be made, so we can safely call this method every time.

The StableDiffusionResult model

After a request to Replicate is made, a new record is inserted in your database with the following information:

Additionally, the model has the following attributes to know the status of the prediction:

• $result->is_successful
• $result->is_failed
• $result->is_starting
• $result->is_processing

Also, as results have to be manually updated through the StableDiffusion::get($id) method, we can also fetch the results that are not in a finished status:

use RuliLG\StableDiffusion\Models\StableDiffusionResult;
use RuliLG\StableDiffusion\StableDiffusion;

$results = StableDiffusionResult::unfinished()->get();
foreach ($results as $result) {
    StableDiffusion::get($result->replicate_id);
}

Generating prompts

There are several styles already built-in:

Additionally, you can add custom styles with the following methods:

• as(string $canvas): to add a string at the beginning (i.e. "a photograph of")
• paintingStyle(string $style): to add a painting style (i.e. realistic, hiperrealistic, etc.)
• by(string $author): to instruct the system to paint it with the style of a certain author
• effect(string $effect): to add a finishing touch to the prompt. You can add as many as you want.

To learn more on how to build prompts for Stable Diffusion, please enter this link.

Credits

• Raúl López
• Calima Solutions

License

The MIT License (MIT). Please see License File for more information.