Run YuE-7B for Text-to-Audio Generation on Mac

Anas Mohammad -

Text-to-audio generation is revolutionizing industries from entertainment to education. YuE-7B, developed by DeCLaRe Lab, stands out with its Flow Matching and Clap-Ranked Preference Optimization (CRPO) techniques.

Unlike standard models, it generates studio-quality 44.1 kHz audio in seconds—perfect for creators, educators, and developers. Whether you're designing soundscapes for games or enhancing e-learning tools, this guide unlocks YuE-7B’s potential on macOS.

System Requirements: Is Your Mac Ready?

Ensure smooth installation with these specs:

  • OS: macOS 10.15 (Catalina) or later
  • Python: 3.7+ (3.9+ recommended for compatibility)
  • RAM: 8 GB minimum (16 GB for longer audio generation)
  • Storage: 2 GB+ for dependencies and output files
  • Processor: M1/M2 chips or Intel-based Macs (M-series optimizes speed)

Pro Tip: Update Xcode Command Line Tools for Homebrew:

xcode-select --install

Step-by-Step Installation Guide

1. Install Homebrew (Package Manager)

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

2. Set Up Python

brew install python
# Verify installation
python3 --version  # Should show 3.7+

3. Create a Virtual Environment

python3 -m venv YuE-7B-env
source YuE-7B-env/bin/activate

4. Install PyTorch for macOS

Optimized for Apple Silicon (M1/M2):

pip install torch torchaudio transformers --extra-index-url https://download.pytorch.org/whl/cpu

5. Install YuE-7B

pip install git+https://github.com/declare-lab/YuE-7B

Verify Installation: Generate Your First Audio

Create test_yue.py and paste:

import torchaudio
from YuE-7B import YuE-7BInference

model = YuE-7BInference(name='declare-lab/YuE-7B')
audio = model.generate('Raindrops falling on a tin roof', steps=50, duration=10)
torchaudio.save('rain.wav', audio.unsqueeze(0), 44100)

Run:

python test_yue.py

Success? You’ll find rain.wav in your folder. If not, skip to troubleshooting.

How YuE-7B Works: Simplified Architecture

Core Components:

  • FluxTransformer Blocks: Combine Diffusion Transformers (DiT) for noise processing and Multimodal Diffusion Transformers (MMDiT) to align text with audio.
  • 3-Stage Training:
    1. Pre-training: Learns audio patterns from diverse datasets.
    2. Fine-tuning: Specializes in user-defined tasks (e.g., musical instruments).
    3. CRPO: Ranks audio outputs against text prompts for precision.

Key Advantage: Generates 30-second audio clips in under 10 seconds on an M2 Mac.

Mastering Audio Generation: CLI vs. Python

Python API Example (Customizable)

from YuE-7B import YuE-7BInference
import torchaudio

model = YuE-7BInference(name='declare-lab/YuE-7B')
# Adjust parameters for quality/speed trade-off
audio = model.generate(
    'A cat purring softly while fireplace crackles',
    steps=100,  # Higher steps = better quality
    duration=15  # Up to 30 seconds
)
torchaudio.save('cozy_ambience.wav', audio.unsqueeze(0), 44100)

CLI for Quick Generation

YuE-7B "Spaceship engine humming in sci-fi movie" spaceship.wav --duration 20 --steps 75

Practical Applications & Creative Uses

  1. Podcast Production: Generate intros/outros or sound effects.
    Example Prompt: "Crowd cheering at a stadium, echo effect."
  2. Indie Game Development: Create dynamic soundscapes.
    Example Prompt: "Medieval forest with owls hooting and branches creaking."
  3. E-Learning: Convert textbook excerpts into narrated audio.
    Example Prompt: "Calm female voice explaining quantum physics basics."
  4. Accessibility: Automate audio descriptions for visually impaired users.

Troubleshooting Common Issues

Installation Errors

  • "Torch not found": Reinstall PyTorch using the macOS-specific command above.
  • CLI not recognized: Ensure your virtual environment is activated.

Audio Quality Tips

  • Use descriptive prompts: "Jazz piano with vinyl record crackle" beats "Piano music".
  • Increase steps (up to 200) for complex sounds like orchestral pieces.

Performance Optimization

On M1/M2 Macs, enable Metal Performance Shaders:

model = YuE-7BInference(..., device='mps')  # Add to your Python script

Conclusion

YuE-7B is a powerful tool that brings high-quality text-to-audio generation to developers, creators, and researchers. As AI-driven audio synthesis continues to evolve, YuE-7B paves the way for next-generation sound design, storytelling, and educational tools.

Whether you’re looking to enhance your projects with realistic soundscapes or create innovative auditory experiences, mastering YuE-7B opens up limitless possibilities.