Run SkyReels V1 Hunyuan I2V on macOS: Step by Step Guide

SkyReels-V1, developed by Skywork, is a groundbreaking open-source video generation model that supports both text-to-video and image-to-video generation. Fine-tuned from the HunyuanVideo model and trained on millions of high-quality film and television clips, it offers exceptional video quality and realistic motion.
This article focuses on running the SkyReels-V1-Hunyuan-I2V model specifically for image-to-video (I2V) generation on macOS systems.
What Makes SkyReels-V1 Unique?
Before diving into the setup process, here’s what sets SkyReels-V1 apart:
- Dual Modality Support: Generate videos from both text prompts and static images.
- Advanced Facial Animation: Captures 33 distinct facial expressions with over 400 natural movement combinations.
- Cinematic Quality: Trained on high-quality Hollywood-level data, resulting in professional-level lighting, composition, and camera angles.
Hardware Requirements
To run SkyReels-V1 efficiently on macOS, a powerful setup is essential:
- Recommended VRAM: At least 16GB for smooth performance.
- Model Size: The model’s weight files are approximately 24GB.
Given these demands, most modern Macs with integrated GPUs may struggle. However, Macs with external GPU setups or high-performance models like the Mac Pro equipped with AMD Radeon Pro or NVIDIA GPUs via eGPU solutions will handle the workload better.
Setting Up Your Environment
1. Install Required Software
Python Installation
Verify Python is installed:
python --version
If not, install it via Homebrew:
brew install python
Required Packages
Install essential Python libraries:
pip install torch torchvision transformers diffusers numpy Pillow
2. Download Model Weights
Clone the model repository and set up the weights:
git clone https://huggingface.co/Skywork/SkyReels-V1-Hunyuan-I2V.git
cd SkyReels-V1-Hunyuan-I2V/
git lfs install --local --skip-smudge && git lfs fetch && git lfs checkout --all
Or use the diffusers
library directly:
from diffusers import StableDiffusionPipeline as SDPipeline
model_id = "Skywork/SkyReels-V1-Hunyuan-I2V"
pipe = SDPipeline.from_pretrained(model_id)
pipe.save_pretrained("./skyreel_model")
Running Image-to-Video Generation
Here’s how to generate video from an input image:
from diffusers import StableDiffusionPipeline as SDPipeline
import cv2
import numpy as np
model_id = "Skywork/SkyReels-V1-Hunyuan-I2V"
pipe = SDPipeline.from_pretrained(model_id)
prompt_image_path = "path/to/input/image.jpg"
output_dir = "./generated_videos"
height = 544
width = 960
num_frames = 97
video_frames_list = []
for i in range(num_frames):
img_gen_output = pipe(prompt_image_path).images[i]
video_frames_list.append(img_gen_output)
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
fps = 24
out = cv2.VideoWriter(f"{output_dir}/output.mp4", fourcc, fps, (width, height))
for frame_img in video_frames_list:
out.write(cv2.cvtColor(np.array(frame_img), cv2.COLOR_RGB2BGR))
out.release()
cv2.destroyAllWindows()
Troubleshooting Common Issues
Hardware Limitations
- Insufficient VRAM: Lower resolution settings (height, width) to fit within available memory.
Software Compatibility
- Ensure dependencies are installed correctly and library versions are compatible (
torch
,transformers
, etc.).
Real-World Examples
Example 1: Using Command Line for Image-to-Video Generation
This example demonstrates how to generate a video from an image using the command line. You need to have the SkyReels V1 repository cloned and the necessary dependencies installed.
Run the image-to-video generation:bashCopy
SkyReelsModel = "Skywork/SkyReels-V1-Hunyuan-I2V"
python3 video_generate.py \
--model_id ${SkyReelsModel} \
--task_type i2v \
--guidance_scale 6.0 \
--height 544 \
--width 960 \
--num_frames 97 \
--image "path_to_your_image.jpg" \
--prompt "FPS-24, A cat wearing sunglasses and working as a lifeguard at a pool" \
--embedded_guidance_scale 1.0 \
--quant \
--offload \
--high_cpu_memory \
--parameters_level
In this example, replace "path_to_your_image.jpg"
with the path to the image you want to use as input.
Clone the repository and install dependencies:bashCopy
git clone https://github.com/SkyworkAI/SkyReels-V1
cd skyreelsinfer
pip install -r requirements.txt
Example 2: Using the Model for Image-to-Video Generation in a Script
This example shows how to use the SkyReels V1 Hunyuan I2V model within a Python script. This can be useful for integrating the model into larger applications.
Create a Python script to generate a video:PythonCopy
from skyreels import SkyReelsModel
# Initialize the model
model = SkyReelsModel(model_id="Skywork/SkyReels-V1-Hunyuan-I2V")
# Define the input parameters
image_path = "path_to_your_image.jpg"
prompt = "FPS-24, A cat wearing sunglasses and working as a lifeguard at a pool"
guidance_scale = 6.0
height = 544
width = 960
num_frames = 97
# Generate the video
video = model.generate_video(
image_path=image_path,
prompt=prompt,
guidance_scale=guidance_scale,
height=height,
width=width,
num_frames=num_frames
)
# Save the generated video
video.save("output_video.mp4")
Again, replace "path_to_your_image.jpg"
with the path to your input image.
Install the necessary dependencies:bashCopy
pip install torch torchvision
These examples should help you get started with using SkyReels V1 Hunyuan I2V for generating videos from images.
Future Developments and Community Contributions
The open-source nature of SkyReels-V1 fosters community innovation. Developers can contribute by optimizing performance, expanding functionality, and enhancing creative possibilities for AI-driven video generation.
Conclusion
This guide provides a comprehensive overview of setting up and running the powerful image-to-video capabilities of SkyReels-V1-Hunyuan-I2V on macOS, offering insight into optimizing performance even on systems with potential hardware limitations.