How to Train a LoRA Locally for SDXL Models with AMD GPU 2025

You have an AMD GPU and want to train SDXL LoRAs locally, but every guide assumes NVIDIA hardware and SDXL's higher VRAM requirements make you wonder if your Radeon card can handle it. SDXL LoRA training AMD is absolutely possible in 2025 with ROCm improvements, but requires more VRAM than SD 1.5 and specific configuration adjustments for the dual text encoder architecture. This SDXL LoRA training AMD guide will walk you through everything you need to know.

Quick Answer: SDXL LoRA training AMD requires 16GB+ VRAM minimum (24GB recommended), ROCm 6.2+, Python 3.10, and Kohya's sd-scripts with AMD-specific configuration. Key differences from SD 1.5 for SDXL LoRA training AMD include 1024x1024 training resolution, tokenizer path fix for dual CLIP encoders, aggressive caching to manage memory, and longer training times. RX 7900 XTX (24GB) handles SDXL LoRA training AMD comfortably, RX 6800 XT/6900 XT (16GB) requires careful optimization, and cards under 16GB cannot train SDXL LoRAs reliably.

What Makes SDXL LoRA Training Different on AMD GPUs?

SDXL LoRA training AMD introduces architectural complexity beyond SD 1.5 that affects AMD GPU training. Understanding these differences in SDXL LoRA training AMD helps you configure appropriately and avoid common failures. If you're new to LoRA training concepts, our troubleshooting guide covers common issues.

The dual text encoder architecture uses both OpenAI's CLIP-ViT-L and OpenCLIP's bigG encoder. This dual encoding provides richer text understanding but doubles the text processing memory footprint. SD 1.5 uses a single CLIP encoder, making SDXL inherently more memory-intensive before considering the larger UNet.

Resolution requirements increase from 512x512 to 1024x1024 as SDXL's native training resolution. This quadruples the pixel count, dramatically increasing latent generation and VAE encoding costs. While you can train at lower resolutions, SDXL LoRAs work best when trained near the model's native resolution.

Model size grows substantially with SDXL's UNet containing approximately 2.6 billion parameters versus SD 1.5's 860 million. This 3x parameter increase translates directly to higher VRAM requirements for model weights, activations, and gradients during SDXL LoRA training AMD.

The tokenizer configuration issue specifically affects SDXL LoRA training AMD with Kohya. The default tokenizer path for the second encoder points to a model that often fails to download reliably. You must manually edit sdxl_train_util.py to change both TOKENIZER1_PATH and TOKENIZER2_PATH to "openai/clip-vit-large-patch14" before SDXL LoRA training AMD works.

VRAM consumption during SDXL LoRA training AMD typically reaches 18-22GB with standard settings. This puts 16GB cards at the absolute edge, requiring every optimization technique. 20-24GB cards provide comfortable headroom for reasonable batch sizes and less aggressive caching. For detailed VRAM management techniques, see our VRAM optimization flags guide.

Training time for SDXL LoRA training AMD increases proportionally to the added complexity. Where SD 1.5 LoRA training might take 1-2 hours on an RX 7900 XTX, equivalent SDXL LoRA training AMD takes 2-4 hours. The larger model, higher resolution, and dual text encoders all contribute to slower iteration.

For users wanting SDXL image generation without training custom LoRAs, platforms like Apatero.com provide access to professionally trained SDXL models through optimized interfaces.

How Do You Set Up Your Environment for SDXL Training?

SDXL LoRA training AMD setup mirrors SD 1.5 setup with identical software requirements. If you already configured your environment for SD 1.5 LoRA training on AMD, you can use the same setup for SDXL LoRA training AMD.

ROCm 6.2 or newer remains the requirement with ROCm 6.3 providing best performance and compatibility. Verify your installation with rocm-smi showing your GPU correctly. Set HSA_OVERRIDE_GFX_VERSION appropriately for your architecture (11.0.0 for RDNA 3, 10.3.0 for RDNA 2).

Python 3.10 provides optimal compatibility with current training scripts and ROCm-enabled PyTorch. Create a virtual environment specifically for SDXL training or reuse your SD 1.5 training environment if you have one configured.

PyTorch installation uses the ROCm 6.3 build. After activating your venv, install with pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.3. This PyTorch build properly interfaces with AMD GPUs through ROCm.

Kohya sd-scripts installation follows the standard process. Clone from GitHub, install requirements, configure Accelerate for single-machine training with fp16 precision, and install additional dependencies like tensorflow-rocm and onnxruntime-rocm as detailed in the SD 1.5 guide.

The critical SDXL LoRA training AMD-specific fix involves editing ./sd-scripts/library/sdxl_train_util.py. Open this file and locate the TOKENIZER1_PATH and TOKENIZER2_PATH variables near the top. Change both to "openai/clip-vit-large-patch14". The default path for TOKENIZER2_PATH points to "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k" which cannot be reliably located, causing SDXL LoRA training AMD to fail with tokenizer errors.

Storage requirements increase for SDXL. The base SDXL model weighs 6-7GB. Training datasets at 1024x1024 consume more space than 512x512 datasets. Cache files grow proportionally to resolution. Budget 150-200GB minimum for comfortable SDXL training workflows with multiple experiments.

Verification after setup involves testing PyTorch GPU detection as with SD 1.5. Import torch, check torch.cuda.is_available() returns True, and verify your AMD GPU appears with torch.cuda.get_device_name(0).

What Training Parameters Work Best for SDXL on AMD?

SDXL LoRA training AMD parameters require adjustment from SD 1.5 defaults to account for higher resolution and memory requirements. These configurations optimize SDXL LoRA training AMD for AMD GPU characteristics.

Batch size must be 1 for most 16GB cards even with aggressive optimization. Cards with 20-24GB can experiment with batch size 2 but may still need batch size 1 depending on network dimensions and caching strategies. Start with 1 and only increase if VRAM monitoring shows substantial headroom.

Mixed precision with fp16 or bf16 is mandatory for SDXL training on AMD. Full precision fp32 is impractical due to memory requirements. Use --mixed_precision="fp16" as standard. Some users report slightly better quality with bf16 if your GPU supports it well.

Learning rate for SDXL typically ranges from 1e-4 to 5e-5. The larger model sometimes benefits from slightly lower learning rates than SD 1.5. Start with 1e-4 and reduce to 5e-5 or 8e-5 if you observe artifacts or instability. Use cosine scheduler with warmup for smooth training dynamics.

Network dimension (LoRA rank) for SDXL LoRA training AMD often runs lower than SD 1.5 due to memory constraints. Where SD 1.5 commonly uses 64-128, SDXL LoRA training AMD works better with 32-64. For 16GB cards, dimension 32 or 48 may be necessary. For 24GB cards, dimension 64 works comfortably. Higher dimensions risk OOM errors or excessively slow training during SDXL LoRA training AMD.

Network alpha follows the half-dimension rule. For dimension 32, use alpha 16. For dimension 64, use alpha 32. This ratio provides good learning dynamics for most subjects.

Resolution for SDXL centers on 1024x1024 as the model's native training size. You can train at 896x896 or 960x960 to save memory, but quality may suffer slightly. Going below 896x896 significantly degrades LoRA quality. For 16GB cards, 896x896 or 960x960 represents a reasonable compromise. For 20GB+ cards, use full 1024x1024.

Max epochs decrease compared to SD 1.5 because SDXL's larger capacity learns faster and training takes longer per epoch. Where SD 1.5 might train for 20-30 epochs, SDXL typically needs 10-20 epochs for similar dataset sizes. Monitor sample images to catch optimal stopping point before overfitting.

Caching becomes absolutely critical for SDXL memory management. Enable all caching options. Use --cache_latents --cache_latents_to_disk --cache_text_encoder_outputs --cache_text_encoder_outputs_to_disk. For dual text encoders, caching text outputs saves substantial VRAM. Disk caching trades storage and some speed for dramatically lower memory usage.

The no_half_vae flag remains important with --no_half_vae preventing numerical instabilities in VAE operations. SDXL's VAE has similar precision sensitivities as SD 1.5.

Optimizer choice affects memory significantly. AdamW8bit uses less memory than standard AdamW with minimal quality impact. For tight VRAM situations, specify --optimizer_type="AdamW8bit" to save memory. Adafactor saves even more memory but with adaptive learning rate side effects.

Gradient checkpointing with --gradient_checkpointing trades compute for memory by recomputing activations during backpropagation rather than storing them. This technique saves substantial VRAM on SDXL training, enabling training on 16GB cards. The speed penalty is acceptable given the memory savings.

How Do You Prepare Datasets for SDXL Training?

Dataset preparation for SDXL follows similar principles to SD 1.5 but with resolution considerations. Proper preparation significantly impacts training outcomes.

Image collection should target SDXL's higher resolution capabilities. Gather high-quality images ideally at 1024x1024 or higher resolution. Lower resolution source images upscaled to 1024x1024 work but don't fully use SDXL's capabilities. For best results, use source images of 1024x1024 or larger.

Image count remains similar to SD 1.5 guidelines. Collect 15-40 high-quality images for most subjects. SDXL's larger capacity learns from slightly smaller datasets than SD 1.5 in some cases, but quality still matters more than quantity.

Folder structure uses the same {repeat_count}_{trigger_word} convention. For example, 10_charactername trains on images in that folder 10 times per epoch. Adjust repeat counts to balance training emphasis across different image categories.

Captioning gains importance with SDXL's dual text encoders. The model understands more subtle descriptions and benefits from detailed, accurate captions. Spend time writing quality captions that describe subjects, actions, settings, styles, and relevant details. The improved text understanding makes good captions more impactful.

Regularization images remain important for SDXL training. The model's power makes overfitting a real risk without regularization. Use 30-100 regularization images of your subject's class (people, objects, styles) at 1024x1024 resolution in a folder like 1_person or 1_art.

Bucketing handles various aspect ratios automatically, but SDXL benefits from training data that matches your intended use cases. If you'll generate 9:16 vertical images, include vertical training images. If you focus on space 16:9, prioritize space training images.

Preprocessing considerations include ensuring adequate lighting and detail in source images. SDXL can generate fine details, so training images should show the details you want the LoRA to learn. Blurry or low-quality training images waste SDXL's capabilities.

What Is the Complete Training Command for SDXL?

Launching SDXL training involves a long command with numerous parameters. Understanding each component helps you adjust for your specific needs.

A typical SDXL training command for AMD GPUs looks like: accelerate launch --mixed_precision="fp16" sdxl_train_network.py --pretrained_model_name_or_path="/path/to/sdxl_base_1.0.safetensors" --train_data_dir="./train" --reg_data_dir="./reg" --output_dir="./output" --output_name="mySDXL_LoRA" --network_module="networks.lora" --network_dim=48 --network_alpha=24 --learning_rate=8e-5 --max_train_epochs=15 --save_every_n_epochs=3 --train_batch_size=1 --max_token_length=225 --xformers=False --cache_latents --cache_latents_to_disk --cache_text_encoder_outputs --cache_text_encoder_outputs_to_disk --no_half_vae --mixed_precision="fp16" --optimizer_type="AdamW8bit" --gradient_checkpointing --persistent_data_loader_workers --resolution="1024,1024".

Key parameter explanations include max_token_length set to 225 instead of SD 1.5's 75. SDXL supports longer text encoding, and 225 tokens accommodates more detailed captions. The resolution parameter explicitly sets 1024x1024 training size.

The sdxl_train_network.py script name differs from SD 1.5's train_network.py. Ensure you use the SDXL-specific script which handles the dual text encoder architecture properly.

All caching options are enabled with disk storage to minimize VRAM usage. This configuration is critical for 16GB cards and beneficial even for larger cards.

Gradient checkpointing trades speed for memory, essential for most AMD GPU training scenarios with SDXL. The speed penalty is acceptable given the ability to train on available hardware.

Monitor VRAM usage during initial training with rocm-smi in another terminal. If usage approaches your card's limit, reduce network dimension, enable more aggressive optimizations, or lower resolution to 960x960 or 896x896.

Training time expectations for this configuration on an RX 7900 XTX with a 25-image dataset at 15 epochs range from 3-5 hours. RX 6800 XT with 16GB takes 5-8 hours for similar training due to tighter memory requiring more conservative settings.

Sample generation during training uses --sample_every_n_epochs=3 --sample_prompts="./sdxl_sample_prompts.txt" to generate test images periodically. Create sample_prompts.txt with prompts testing your LoRA. SDXL sample generation itself consumes significant VRAM, so generate samples less frequently than SD 1.5 (every 3 epochs instead of every 2).

Frequently Asked Questions

What's the minimum VRAM for SDXL LoRA training on AMD?

16GB represents the absolute minimum with aggressive optimization including fp16 precision, batch size 1, network dimension 32-48, all caching to disk, gradient checkpointing, and possibly reducing resolution to 896x896. The RX 6800 XT and 6900 XT with 16GB can train SDXL LoRAs but require careful configuration. For comfortable training, 20GB (RX 7900 XT) or 24GB (RX 7900 XTX) is recommended. Cards under 16GB cannot reliably train SDXL LoRAs.

Why does my SDXL training fail with tokenizer errors?

This is the most common SDXL training failure on AMD. Edit ./sd-scripts/library/sdxl_train_util.py and change both TOKENIZER1_PATH and TOKENIZER2_PATH to "openai/clip-vit-large-patch14". The default path for TOKENIZER2_PATH points to a model that fails to download reliably. This fix is mandatory for SDXL training on AMD with Kohya's scripts. After editing, save the file and restart training.

Can I train at 512x512 to save VRAM?

You can technically train SDXL at 512x512, but quality suffers significantly. SDXL's architecture trained at 1024x1024, and LoRAs trained at much lower resolutions don't transfer well. Minimum practical resolution is 896x896 for acceptable quality. If your card can't handle 896x896, it's too small for effective SDXL LoRA training. Consider cloud GPU rental or focusing on SD 1.5 instead.

How much longer does SDXL training take versus SD 1.5 on AMD?

SDXL training typically takes 2-3x longer than SD 1.5 for equivalent epochs and dataset sizes. The larger model, higher resolution (4x pixels), and dual text encoders all contribute. On an RX 7900 XTX, SD 1.5 LoRA training might take 1.5-2 hours while SDXL takes 3-5 hours for comparable configurations. On 16GB cards, the gap widens further due to more aggressive memory optimizations slowing SDXL training.

Should I use AdamW or AdamW8bit for SDXL?

AdamW8bit is recommended for most SDXL training on AMD GPUs. It uses less memory than standard AdamW with minimal quality impact, enabling higher network dimensions or less aggressive caching. Only use standard AdamW if you have 24GB VRAM and want to experiment with potential marginal quality gains. For 16GB cards, AdamW8bit is essentially mandatory.

Can I use the same LoRA rank as SD 1.5?

SDXL typically uses lower LoRA ranks than SD 1.5 due to VRAM constraints. Where SD 1.5 commonly uses rank 64-128, SDXL on AMD works better with rank 32-64. Start with rank 48 for balanced results. Only increase to 64 if you have 24GB VRAM and want maximum LoRA capacity. For 16GB cards, rank 32 may be necessary. SDXL's larger base model means lower-rank LoRAs still provide significant adaptation capability.

What if I get OOM errors during SDXL training?

Reduce network dimension to 32, enable all caching with disk storage, use AdamW8bit optimizer, enable gradient checkpointing if not already active, reduce resolution to 960x960 or 896x896, ensure batch size is 1, close other GPU applications, and verify mixed precision is set to fp16. If OOM errors persist after all optimizations, your GPU likely lacks sufficient VRAM for SDXL training.

Do SDXL LoRAs work with SD 1.5 models?

No, SDXL LoRAs are not compatible with SD 1.5 models due to completely different architectures. SDXL's dual text encoders, larger UNet, and different VAE make the models incompatible. You must train separate LoRAs for SD 1.5 and SDXL. However, once trained, SDXL LoRAs work with all SDXL-based models and checkpoints (SDXL base, refiner, and community fine-tunes).

Can I train SDXL LoRAs faster by reducing epochs?

Reducing epochs lowers training time but risks undertrained LoRAs that don't capture your subject well. SDXL typically needs 10-20 epochs for good results with standard dataset sizes. Going below 10 epochs often produces weak LoRAs. Instead of reducing epochs, optimize per-epoch speed with proper caching and settings, or accept the longer training time as necessary for quality results. SDXL's complexity requires sufficient training.

What batch size can I use with 24GB VRAM?

With 24GB VRAM on cards like RX 7900 XTX, batch size 2 is possible for SDXL training with moderate network dimensions (48-64) and standard caching. Some configurations allow batch size 4 with very aggressive caching and lower network dimensions, but gains are minimal and instability risks increase. Batch size 1 works reliably and produces good results, so many users stick with it even on 24GB cards. Experiment carefully if trying larger batches.

Advanced Training Optimization Techniques for AMD GPUs

Beyond basic parameter configuration, several advanced techniques can significantly improve your SDXL LoRA training results on AMD hardware. Understanding these optimizations helps you extract maximum quality from your training runs while working within hardware constraints.

Caption Quality and Trigger Word Strategy

The quality of your captions directly impacts how well your LoRA learns the target concept. SDXL's dual text encoder architecture makes this even more critical than with SD 1.5. Each caption should include your trigger word consistently in the same position, detailed descriptions of the subject's key features, context about the environment, lighting, and style, and any relevant technical details like camera angle or framing.

For character LoRAs, describe distinguishing features explicitly in every caption. If your character has blue eyes, mention "blue eyes" even when they're not prominently visible. This reinforcement helps the model learn the association between your trigger word and specific visual features. For style LoRAs, focus on artistic elements like brushwork, color palette choices, composition patterns, and mood.

Consider using automatic captioning tools like BLIP or WD-Tagger as a starting point, then manually refine the results. Automatic captions often miss subtle details that make your subject unique. The time investment in quality captions pays dividends in LoRA performance. If you're new to LoRA training concepts, our Flux LoRA training guide covers foundational techniques that transfer to SDXL training as well.

Training Monitoring and Early Stopping

SDXL's longer training times make monitoring particularly important. You don't want to discover after 8 hours that your LoRA overfitted at epoch 10. Set up sample generation every 3-5 epochs using diverse prompts that test different aspects of your LoRA: various poses, lighting conditions, artistic styles, and contexts.

Watch for signs of overfitting in your sample images. These include increasing difficulty blending with other styles, output looking increasingly similar regardless of prompt variation, loss of flexibility in poses or expressions, and the LoRA dominating even at low strength values. When you spot these signs, your optimal checkpoint was likely 2-3 epochs earlier.

Underfitting manifests as weak trigger word response, inconsistent subject features, and the LoRA having minimal effect even at strength 1.0. If you see underfitting throughout training, consider increasing epochs, adjusting learning rate upward slightly, or reviewing dataset quality.

Multi-Concept Training Considerations

Training LoRAs that include multiple related concepts requires careful dataset organization on SDXL. The model's larger capacity can handle complex multi-concept training, but poor organization leads to concept bleeding where features from one subject appear on another.

Organize datasets with clear folder separation for each concept. Use distinct trigger words that are unlikely to appear naturally in prompts. Consider training separate LoRAs for each concept and combining them during inference rather than training a single multi-concept LoRA, especially when starting out. Multi-concept training adds complexity that's easier to manage once you have experience with single-concept SDXL LoRAs.

Regularization Image Strategy

Regularization images prevent your LoRA from overfitting to specific features of your training images that you don't want to learn. For character LoRAs, regularization images should be other people in similar contexts but with different features. For style LoRAs, regularization images should be art in other styles.

Generate regularization images using the base SDXL model with prompts similar to your training image captions but without your trigger word. This teaches the model what "normal" looks like in the relevant context, so it only attributes the differences to your trigger word. Use 30-100 regularization images depending on dataset size, typically 2-3x the number of training images.

Place regularization images in a folder with repeat count 1, formatted like 1_person for a character LoRA. This lower repeat count compared to training images balances their influence appropriately.

Troubleshooting Common AMD SDXL Training Issues

Even with careful setup, you'll encounter issues during SDXL training on AMD GPUs. These solutions address the most common problems.

Training Hangs or Freezes

If training stops progressing without error messages, ROCm's watchdog timer may be triggering. Increase the timeout value with export GPU_FORCE_64BIT_PTR=1 before training. Also try export HSA_FORCE_FINE_GRAIN_PCIE=1 for improved memory handling on some systems.

Very low learning rates combined with small network dimensions can cause apparent freezes where progress is simply too slow to notice. Monitor loss values to confirm training is actually progressing.

Tokenizer Download Failures

Besides the mandatory tokenizer path fix in sdxl_train_util.py, you may encounter failures downloading models from Hugging Face due to rate limiting or network issues. Pre-download required models using huggingface-cli download openai/clip-vit-large-patch14 to cache them locally before training. This eliminates network-dependent failure points during training.

VRAM Spikes Causing Crashes

SDXL training has higher peak VRAM usage than sustained usage. If you're crashing despite apparently having headroom, enable gradient accumulation to smooth out memory usage. Set --gradient_accumulation_steps=2 to halve peak memory at the cost of slower iteration.

Also verify no other applications are consuming VRAM. Even small allocations from browsers or desktop effects can push you over the edge during peak usage moments.

Poor Quality Despite Correct Settings

If your LoRA trains without errors but produces poor results, examine your dataset first. Common issues include images that are too similar (causing overfitting), resolution too low for SDXL's detail level, inconsistent subject presentation across images, and captions that don't match the actual image content.

Try training with a smaller subset of your best images. More data isn't always better if quality varies. A LoRA trained on 15 excellent images often outperforms one trained on 40 mediocre images. Consider working with an online LoRA training platform like RunPod if you need to iterate quickly on training parameters without hardware limitations.

Succeeding with SDXL LoRA Training on AMD Hardware

SDXL LoRA training AMD requires more resources and careful optimization compared to SD 1.5, but remains practical for users with adequate hardware. The 16GB minimum VRAM threshold limits accessibility compared to SD 1.5's 12GB minimum, but RX 7900 series cards handle SDXL LoRA training AMD capably. For ComfyUI users, our essential nodes guide covers loading trained LoRAs.

The mandatory tokenizer fix addresses a specific compatibility issue that would otherwise prevent SDXL LoRA training AMD entirely. This one-line edit in sdxl_train_util.py is non-obvious but critical, illustrating the AMD-specific considerations that differ from NVIDIA workflows.

Aggressive caching strategies, gradient checkpointing, 8-bit optimizers, and batch size 1 form the optimization toolkit that makes SDXL LoRA training AMD work on AMD hardware with limited VRAM. Understanding which optimizations provide the best memory-speed tradeoffs helps you configure appropriately for SDXL LoRA training AMD. If you're new to AI image generation, our beginner guide provides foundational knowledge.

Training times of 3-8 hours depending on hardware represent substantial investments compared to SD 1.5's 1-3 hours, but remain practical for users training occasionally. The quality improvements SDXL provides over SD 1.5 justify the additional time and resource requirements for many use cases.

For users wanting SDXL image generation without training custom LoRAs, platforms like Apatero.com provide access to professionally trained SDXL models through optimized interfaces, eliminating setup and training complexity entirely.

As ROCm continues maturing and AMD's AI compute presence grows, expect SDXL LoRA training AMD workflows to become smoother with better performance parity to NVIDIA solutions. The foundations established in 2025 position AMD GPUs as viable platforms for SDXL LoRA training AMD in the Stable Diffusion ecosystem.