Fix Flux LoRA Training Out of Memory Errors on RTX 4090

The RTX 4090's 24GB of VRAM represents the current consumer maximum, and marketing materials suggest it's sufficient for local AI training. Yet when you attempt Flux LoRA training with default configurations, CUDA out of memory errors crash your process within minutes. Flux LoRA training OOM errors are a common frustration - the training might start successfully, run for a few steps, then die with a Flux LoRA training OOM crash. You reduce batch size and it still crashes with Flux LoRA training OOM. The memory consumption seems to spiral beyond your 24GB no matter what you adjust. This frustrating Flux LoRA training OOM experience results from Flux's architecture demanding more VRAM at default settings than 24GB provides, but the good news is that proper optimization makes stable training on 4090 not just possible but comfortable.

Understanding what consumes memory during training is essential for fixing Flux LoRA training OOM errors. Flux's large model size is only part of the story. During training, you simultaneously need memory for the model weights, gradients for trainable parameters, optimizer states that track momentum and variance, and activation tensors stored during the forward pass for use in backpropagation. At training resolution of 1024x1024 with default settings, these combined requirements exceed 40GB, causing Flux LoRA training OOM crashes. The solutions in this guide reduce this to under 20GB while maintaining training quality, completely eliminating Flux LoRA training OOM errors and leaving headroom for stable operation on your 24GB card.

Memory Consumption During Flux Training

Each component of memory consumption has specific characteristics that determine which optimizations are effective. Understanding this helps you target optimizations to the biggest consumers rather than making changes that have minimal impact.

Model Weights

Flux's base model consumes approximately 23GB at full FP32 precision just for the weights. This alone nearly fills a 4090's memory before training even begins. At FP16 or BF16 precision, weights drop to roughly 12GB, which is why mixed precision training is essential rather than optional.

During LoRA training, you freeze these base weights (they don't get updated) but they must remain in memory for the forward pass. The LoRA weights you're actually training are much smaller, typically 50-500MB depending on rank, but the base model remains present.

Gradient Memory

Gradients store the computed partial derivatives used to update trainable parameters. Each trainable parameter needs a gradient of the same size and precision.

For LoRA training with typical rank, gradient memory is modest because you're only training the small adapter layers, not the full model. But gradients still add 50-500MB depending on LoRA configuration.

Optimizer States

Adam and AdamW optimizers store two values per trainable parameter: the first moment (momentum) and second moment (variance) estimates. This doubles the memory needed for trainable parameters.

For LoRA training, optimizer states add 100MB-1GB depending on rank. Using 8-bit optimizers quantizes these states to INT8, halving their memory footprint while maintaining similar training dynamics.

Activation Memory

Activations are intermediate values computed during the forward pass and stored for use during backpropagation. This is typically the largest memory consumer during training and scales dramatically with resolution and batch size.

For a single 1024x1024 image through Flux, activation memory can reach 15-25GB depending on model variant and precision. This is where most of your VRAM goes during training, and this is why gradient checkpointing is so impactful.

The memory scales with resolution squared. A 512x512 image uses roughly 25% of the activation memory of 1024x1024. Batch size multiplies this linearly. Batch size 2 uses twice the activation memory of batch size 1.

The Total Picture

Summing typical components for Flux LoRA training at 1024x1024 with batch size 1:

• Model weights (BF16): ~12GB
• LoRA weights: ~0.1GB
• LoRA gradients: ~0.1GB
• Optimizer states (8-bit): ~0.1GB
• Activations: ~20GB

Total: ~32GB

This clearly exceeds 24GB. Reducing activation memory through resolution and gradient checkpointing is essential.

Essential Memory Optimizations to Fix Flux LoRA Training OOM

These optimizations are not optional for fixing Flux LoRA training OOM on 24GB cards. Apply all of them as a baseline configuration to prevent Flux LoRA training OOM errors.

Enable Gradient Checkpointing

Gradient checkpointing provides the most significant memory reduction for fixing Flux LoRA training OOM, cutting activation memory by 60-70% at the cost of 20-30% more computation time. This is the most critical fix for Flux LoRA training OOM errors.

Instead of storing all intermediate activations during the forward pass, checkpointing stores only selected checkpoints and recomputes intermediate values during backpropagation as needed. This trades compute for memory, using essentially constant activation memory regardless of model depth.

In Kohya SS, enable gradient checkpointing with a simple configuration flag:

gradient_checkpointing: true

In custom training scripts using diffusers:

from diffusers import FluxPipeline

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev")
pipe.enable_gradient_checkpointing()

This single optimization reduces activation memory from 20GB+ to 6-8GB, making 24GB training feasible and fixing Flux LoRA training OOM. The 20-30% time penalty is absolutely worth it for stable training that actually completes without Flux LoRA training OOM crashes.

Reduce Training Resolution

Resolution has squared impact on memory. Halving resolution quarters memory consumption.

Train at 512x512 instead of 1024x1024:

resolution: "512,512"

This reduces activation memory from ~20GB to ~5GB. Combined with gradient checkpointing, activation memory drops to ~2GB.

You might worry that 512x512 training produces inferior LoRAs. In practice, the concepts and styles you're training transfer well to higher inference resolution. You're teaching the model features, patterns, and associations that aren't resolution-specific. Generate at 1024x1024 after training at 512x512.

Use Batch Size 1 with Gradient Accumulation

Batch size directly multiplies activation memory. Batch size 4 uses 4x the activation memory of batch size 1.

Set batch size to 1 for minimum memory:

train_batch_size: 1

Simulate larger effective batch sizes through gradient accumulation:

gradient_accumulation_steps: 4

This accumulates gradients over 4 forward passes before updating weights, giving effective batch size 4 while only ever holding 1 sample's activations in memory. Training dynamics approximate larger batches without the memory cost.

Enable Mixed Precision Training

Mixed precision uses FP16 or BF16 for most operations while keeping critical values in FP32 for numerical stability.

For Ampere and newer GPUs (RTX 30xx, 40xx), use BF16:

mixed_precision: "bf16"

BF16 has the same exponent range as FP32 but reduced mantissa precision, making it better for training than FP16 which can underflow with small gradients.

Enable Memory-Efficient Attention

Standard attention implementations allocate large intermediate tensors for attention scores. Memory-efficient implementations like xFormers or Flash Attention process attention in chunks, dramatically reducing peak memory.

For xFormers:

xformers: true

First install xFormers:

pip install xformers

Memory-efficient attention can reduce attention operation memory by 80% or more. For large models like Flux with many attention layers, this translates to significant total savings.

Use 8-bit Optimizer

Standard Adam stores two FP32 values per parameter. 8-bit Adam from bitsandbytes quantizes these to INT8, halving optimizer memory.

optimizer_type: "AdamW8bit"

First install bitsandbytes:

pip install bitsandbytes

Quality impact is minimal for LoRA training. The optimizer states don't need high precision since they're tracking momentum over many steps, smoothing out quantization noise.

Complete Kohya SS Configuration

Here's a complete working configuration for Flux LoRA training on RTX 4090:

# Model
pretrained_model_name_or_path: "black-forest-labs/FLUX.1-dev"

# Precision
mixed_precision: "bf16"

# Memory optimizations
gradient_checkpointing: true
xformers: true

# Resolution and batch
resolution: "512,512"
train_batch_size: 1
gradient_accumulation_steps: 4

# Network (LoRA)
network_module: "networks.lora"
network_dim: 16
network_alpha: 16

# Optimizer
optimizer_type: "AdamW8bit"
learning_rate: 1e-4
lr_scheduler: "constant"

# Training
max_train_steps: 1500
save_every_n_steps: 500

# Sampling during training
sample_every_n_steps: 200
sample_prompts: "sample_prompts.txt"
sample_sampler: "euler"

This configuration uses approximately 16-18GB VRAM, leaving comfortable headroom on a 24GB card.

Adjusting for Different Training Goals

Character LoRAs: Use rank 16, 1000-1500 steps. Character features are relatively simple to capture.

Style LoRAs: Increase rank to 24-32 and steps to 2000-3000. Styles have more variance requiring more capacity.

Complex concepts: Higher rank (32-64) and more steps. Monitor memory usage since higher ranks increase LoRA parameter count.

Caption Configuration for Best Results

Good captions significantly improve training quality. Flux responds well to natural language descriptions.

Create caption .txt files for each training image:

A portrait of ohwxperson, a young woman with short black hair and brown eyes, wearing a white t-shirt, studio lighting, neutral background

Include your trigger word (ohwxperson) in every caption. Vary the descriptions to reflect what's actually in each image.

Troubleshooting Persistent Flux LoRA Training OOM Errors

If Flux LoRA training OOM crashes continue after applying all optimizations, investigate these additional factors to resolve persistent Flux LoRA training OOM issues.

VRAM Fragmentation

PyTorch's memory allocator can fragment VRAM over time. Configure PyTorch to reduce fragmentation:

# Linux/Mac
export PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128

# Windows
set PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128

Also restart your system between training runs. Fresh starts defragment memory.

Monitor Memory During Training

Watch VRAM usage in real time:

watch -n 1 nvidia-smi

Note peak usage, not average. OOM occurs when peak exceeds capacity, even if average is well below.

Dataset Issues

Preprocess your dataset to match training resolution:

from PIL import Image
import os

def resize_dataset(input_dir, output_dir, max_size=768):
    os.makedirs(output_dir, exist_ok=True)
    for filename in os.listdir(input_dir):
        if filename.lower().endswith(('.png', '.jpg', '.jpeg')):
            img = Image.open(os.path.join(input_dir, filename))
            if max(img.size) > max_size:
                ratio = max_size / max(img.size)
                new_size = (int(img.size[0] * ratio), int(img.size[1] * ratio))
                img = img.resize(new_size, Image.LANCZOS)
            img.save(os.path.join(output_dir, filename))

resize_dataset("raw_images", "training_images", max_size=768)

Other GPU Memory Consumers

Check for other applications using GPU memory before training:

nvidia-smi

Close Chrome, Discord, game launchers, and monitoring tools. Multiple Python processes can hold VRAM from previous crashed runs.

Quality Verification During Training

Enable sample generation during training:

sample_every_n_steps: 200
sample_prompts: "prompts.txt"

Watch samples for these phases:

• Steps 0-300: Trigger word has no effect, generic output
• Steps 300-800: Subject begins appearing, features emerging
• Steps 800-1500: Subject clear and consistent
• Steps 1500+: Watch for overfitting, loss of variation

Stop training when samples show your subject clearly but still vary naturally.

Dataset Preparation Best Practices

The quality of your training dataset directly determines the quality of your LoRA. Investing time in proper dataset preparation yields significantly better results than any hyperparameter tuning.

Image Quality Requirements

Every image in your dataset teaches the model something. Poor quality images teach poor quality representations. Ensure your training images meet these standards:

Resolution: Images should be at least training resolution (512x512). Higher resolution is fine since they'll be resized, but lower resolution images appear blurry when scaled up for training.

Focus: Your subject must be in sharp focus. Blurry, motion-blurred, or poorly focused images create blurry concept representations.

Lighting: Good lighting that reveals your subject's features. Avoid harsh shadows that obscure important details unless you specifically want that style.

No artifacts: Remove images with compression artifacts, watermarks, text overlays, or noise. These artifacts can be learned as part of your concept.

Variety and Diversity

Your LoRA generalizes from variation in training data. If all training images show your character in the same pose, the LoRA associates that pose with the character. Vary these elements:

Poses: Different body positions, hand placements, head angles

Expressions: Various facial expressions if training a character

Lighting: Different lighting setups (natural, studio, dramatic)

Backgrounds: Various backgrounds to prevent background association

Angles: Different camera angles (front, side, three-quarter)

10-20 diverse images train better than 50 similar images because the model learns what's consistent (your concept) versus what varies (pose, lighting, etc.).

Captioning Strategy

Captions teach the model what each image represents and what should be associated with your trigger word.

Use natural language descriptions rather than tag lists:

ohwxperson, a young woman with auburn hair wearing a white dress, standing in a garden with flowers, soft natural lighting, portrait photography

Describe what varies between images in the captions. If one image has studio lighting and another has natural light, say so. This teaches the model that lighting is separate from your concept.

Keep your trigger word consistent. Use exactly the same trigger word in every caption. Variations like "ohwxperson" and "ohwx_person" are treated as different tokens.

Dataset Organization

Kohya SS expects a specific folder structure:

dataset/
└── 10_ohwxperson/
    ├── image1.png
    ├── image1.txt
    ├── image2.png
    ├── image2.txt
    └── ...

The "10" prefix indicates repetition count. Smaller datasets need more repetitions to present each image enough times during training. 10-15 repetitions for small datasets (under 20 images), 5-8 for medium datasets (20-50 images), 3-5 for larger datasets.

Alternative Training Approaches

When 4090 training remains problematic despite optimization, alternatives exist.

Cloud Training Services

Cloud GPU instances eliminate memory constraints entirely. Services with good AI training support include:

RunPod: Popular for AI workloads with A100 40GB/80GB instances available. Community cloud offers affordable pricing around $1-2/hour for capable instances.

Lambda Labs: Reliable A100 and H100 availability. Slightly higher prices but consistent quality and availability.

Vast.ai: Marketplace model with lowest prices but variable reliability. Good for non-critical training where you can retry if instances fail.

Cost for typical LoRA training:

• A100 40GB: $1-2/hour
• Training time: 30-60 minutes
• Total: $1-3 per LoRA

This is economical for occasional training. Upload your dataset, run training, download results. No local hardware constraints.

Experimental Optimizations

Cutting-edge optimizations push memory limits further but require testing:

FP8 training: Reduces precision beyond BF16 for additional memory savings. Quality impact is being evaluated by the community.

LoKr and LoHa: Alternative low-rank adaptation methods that can use less memory than standard LoRA with different capacity characteristics.

Model sharding: Distributes model across GPU and CPU memory. Slower but enables larger models on smaller GPUs.

These approaches are less proven than the standard optimizations in this guide. Test results carefully before using for important projects.

Frequently Asked Questions

Why does Flux LoRA training OOM crash after exactly 1 step every time?

The forward pass fits in memory but adding gradients during backward pass exceeds capacity, causing Flux LoRA training OOM. This precise failure point indicates you're slightly over the limit. Enable gradient checkpointing and reduce resolution together rather than one at a time to fix this Flux LoRA training OOM pattern.

Can I train at 1024x1024 on RTX 4090?

Technically possible with extreme optimization (rank 8, aggressive checkpointing, 8-bit everything) but impractical. Quality at 512x512 is good enough and the training experience is dramatically better. Train at 512, generate at 1024.

Does batch size 1 produce worse LoRAs than larger batches?

Not significantly when using gradient accumulation. Accumulated gradients smooth noise from small batches. Quality difference is minimal compared to other factors like dataset quality and learning rate.

How do I know if my LoRA is training correctly?

Sample images during training show progress. Your trigger word should affect outputs after 200-400 steps, with clear subject by 800-1000 steps. If samples don't change, increase learning rate. If they immediately degrade, decrease it.

Why does VRAM usage creep up during training causing Flux LoRA training OOM?

Memory fragmentation causes gradual increase leading to eventual Flux LoRA training OOM. The allocator creates fragments that can't be reused. Set PYTORCH_CUDA_ALLOC_CONF and restart between runs for clean memory state to prevent late-training Flux LoRA training OOM crashes.

What network rank should I use?

Start with 16 for characters, 24-32 for styles. Higher ranks capture more detail but need more memory and data. Test lower ranks first since they often work well and leave more headroom.

How many images do I need for good results?

10-20 for characters with variety. 50-100 for styles with range. Quality and variety matter more than quantity. Well-captioned diverse images train better than many similar ones.

Can I resume training after an OOM crash?

Yes, if you enabled checkpointing. Kohya SS saves progress periodically. Fix memory settings, then resume from the last checkpoint.

Should I use xFormers or native PyTorch attention?

xFormers provides better memory efficiency in most cases. Try xFormers first; only switch to native if you encounter specific issues. Flash Attention is another option with good performance on recent GPUs.

Is there any way to predict if my config will OOM?

Rough estimation: add up model size (12GB BF16), activation estimate based on resolution (5GB at 512x512 with checkpointing), optimizer and gradients (~0.5GB). Leave 4GB headroom. Tools like accelerate estimate-memory provide estimates but actual usage varies.

Integration with Inference Workflows

After successfully training your LoRA, integrate it into ComfyUI workflows for image generation.

LoRA Loading Configuration

Properly load and configure your trained LoRA in ComfyUI.

LoRA Loader node placement should be between your checkpoint loader and the rest of the workflow. The LoRA modifies model weights, so it must load before those weights are used by KSampler.

Strength tuning adjusts how strongly the LoRA affects output. Start at 0.7-0.8 and adjust based on results. Too high causes oversaturation of your concept; too low produces weak effect.

CLIP strength independently adjusts text encoder influence. For character LoRAs, matching model and CLIP strength usually works well. For styles, you might reduce CLIP strength to maintain better prompt adherence.

Testing and Quality Verification

Systematically verify your LoRA works correctly.

Trigger word testing confirms the LoRA activates properly. Generate identical prompts with and without your trigger word to see the difference.

Prompt adherence testing verifies the LoRA doesn't override prompts. Generate your character in various settings, poses, and clothing. Good LoRAs maintain prompt flexibility; overfitted LoRAs ignore prompts.

Resolution testing at inference resolution (1024x1024) confirms your 512x512 training transfers effectively. Quality should be good despite resolution mismatch.

For comprehensive workflow understanding, see our ComfyUI essential nodes guide.

Combining with Other Techniques

LoRAs work alongside other ComfyUI capabilities.

Multiple LoRAs stack together for combined effects. Chain LoRA Loader nodes, keeping combined strength under 1.5 to avoid artifacts. Character plus style LoRA is a common combination.

ControlNet integration guides composition while LoRA provides character. Use pose references, depth maps, or other ControlNet inputs alongside your LoRA.

Upscaling workflows improve output resolution beyond native generation. Generate at 1024x1024, then apply ESRGAN or similar upscalers for print-quality output.

Advanced Training Techniques

Beyond basic optimization, advanced techniques improve training results or enable more complex training scenarios.

Learning Rate Scheduling

Dynamic learning rates can improve convergence and final quality.

Cosine annealing gradually reduces learning rate following a cosine curve. This allows aggressive early learning while fine-tuning toward the end.

Warmup periods start with low learning rate and ramp up. This prevents early instability when gradients are noisy.

Restarts periodically reset learning rate to escape local minima. Useful for complex concepts that benefit from fresh optimization attempts.

Experiment with schedulers if you find training quality plateaus before reaching optimal results.

Network Architecture Variations

Different LoRA architectures offer different characteristics.

Standard LoRA uses low-rank decomposition with consistent rank across layers. This is the default and works well for most use cases.

LoCon/LoHA extends adaptation to convolutional layers and uses different decomposition methods. Can capture more detail but uses more memory and training time.

DoRA (Decomposed Rank Adaptation) separates magnitude and direction updates. May improve training dynamics for some subjects.

Try standard LoRA first. Only explore alternatives if results are unsatisfactory despite good training practices.

Regularization Techniques

Prevent overfitting through regularization approaches.

Dataset augmentation varies training images through cropping, flipping, and color adjustments. Creates more apparent variety from limited images.

Dropout randomly zeros some network outputs during training. Prevents co-adaptation and improves generalization.

Weight decay penalizes large weights, encouraging simpler representations. May help with overfitting on small datasets.

Apply regularization if you see overfitting despite reducing training steps. Small datasets benefit most from regularization.

Troubleshooting Training Quality

Beyond memory issues, training quality problems require diagnosis and correction.

Underfitting

If your LoRA doesn't capture the concept adequately:

Insufficient training shows little effect from trigger word. Increase steps or epochs until samples show your subject clearly.

Learning rate too low produces very slow learning. Increase learning rate if progress is minimal after hundreds of steps.

Network rank too low limits capacity. Increase rank if concept is complex (styles need higher rank than simple characters).

Overfitting

If your LoRA reproduces training images too exactly:

Too many steps memorizes training data. Reduce steps when samples stop improving or start losing variety.

Learning rate too high causes rapid memorization. Reduce if concept appears but loses flexibility immediately.

Dataset too small or similar provides insufficient variety. Add more images or increase augmentation.

Style Leakage

If your LoRA affects all generations, not just those with trigger word:

Caption issues associate style elements with common words rather than trigger. Ensure consistent trigger word placement and varied descriptions.

Overtraining burns the concept too deeply into the model. Reduce training steps or learning rate.

For memory-efficient training solutions, see our VRAM optimization guide.

Performance Monitoring and Optimization

Monitor training performance to identify optimization opportunities.

GPU use

Maximize GPU efficiency during training.

Low GPU use indicates bottlenecks elsewhere. Check data loading speed, disk I/O, and CPU preprocessing.

Memory spikes beyond steady-state usage suggest fragmentation or allocation patterns. Set PYTORCH_CUDA_ALLOC_CONF for better allocation behavior.

Temperature throttling reduces performance as GPU overheats. Ensure adequate cooling for sustained training.

Training Speed Benchmarks

Establish baselines for your configuration.

Steps per second indicates overall training throughput. Note this for your configuration to identify regressions.

Time per epoch helps estimate total training time. Important for planning and scheduling training runs.

Memory high water mark shows peak usage. Keep this well under 24GB for stability.

Iteration Time Analysis

Identify where time goes during training.

Forward pass time through the model. Scales with resolution and model size.

Backward pass time computing gradients. Gradient checkpointing increases this for memory savings.

Data loading time reading and preprocessing images. Should be small compared to compute time; if large, optimize storage and preprocessing.

Ecosystem and Community Resources

use community resources for improved training.

Model and Configuration Sharing

Learn from others' successful configurations.

CivitAI hosts trained LoRAs with descriptions of training parameters. Study high-quality LoRAs to learn effective configurations.

HuggingFace provides model weights and training notebooks. Good source for Flux-specific training approaches.

GitHub repositories contain training scripts and configurations. Kohya SS and similar repos include example configs to start from.

Community Support

Get help with training issues.

Discord servers for Kohya SS, ComfyUI, and Flux provide real-time assistance from experienced trainers.

Reddit communities like r/StableDiffusion discuss training techniques and troubleshooting.

YouTube tutorials demonstrate training workflows visually, helpful for understanding complex processes.

Staying Current

Training techniques evolve rapidly.

Follow releases from Kohya SS, diffusers, and other training frameworks. New versions often include optimizations and features.

Monitor research in efficient fine-tuning. Techniques like LoRA continue improving through research advances.

Test new approaches as they appear. The community rapidly adopts and validates new methods.

Conclusion

Flux LoRA training OOM on RTX 4090 can be completely eliminated with comprehensive memory optimization that produces excellent results when properly configured. The essential optimizations for fixing Flux LoRA training OOM are gradient checkpointing, 512x512 resolution, batch size 1 with gradient accumulation, BF16 mixed precision, memory-efficient attention, and 8-bit optimizer.

Together, these optimizations fix Flux LoRA training OOM by bringing memory requirements from 35GB+ down to 16-18GB, leaving comfortable headroom on a 24GB card.

Start with the complete configuration provided in this guide. Run a short test of 100-200 steps to verify stability before committing to full training. Monitor samples and stop training before overfitting.

If you consistently struggle with memory limits, cloud training offers a practical alternative. An A100 instance costs $1-3 per LoRA and eliminates memory constraints entirely. Services like Apatero.com make this accessible without managing cloud infrastructure.

Your RTX 4090 is fully capable of local Flux LoRA training with proper configuration. Apply these optimizations and you'll have a reliable training setup for creating custom characters, styles, and concepts.

For those new to AI image generation, our complete beginner guide provides foundational knowledge that helps contextualize these LoRA training techniques within your overall AI image generation journey.