Tencent's Hy-Nomas is an interactive video world model released on Hugging Face under Apache-2.0, and the README says it runs on a single RTX 4090. It does, on paper: INT8 weights, an FP8 decoder, CPU offload, and about 22.5 GB of downloads instead of the roughly 196 GB full repo. What the README doesn't spell out is that a manual install also needs a CUTLASS checkout, the decoder kernel is compiled for Ada only, and the web demo still needs a separate prompt-enhancer model with 55.6 GB of weights that defaults to 8 GPUs.
I read through the Hy-Nomas model card, the GitHub repo and the code itself to work out what running Hy-Nomas locally on one 4090 actually takes, including the steps the docs skip.
What Hy-Nomas is
Hy-Nomas generates video one segment at a time. After each segment you can change the text instruction or the camera, and the next segment continues from what came before. It's the same idea as DeepMind's Genie demos, a world you steer through segment by segment, except you can download this one.
Specs from the model card:
- Segment-autoregressive diffusion transformer (MMDiT), 54 double-stream blocks, hidden size 2,048, 16 heads of 128 dims, FFN size 8,192
- Output at 720×1280, 24 FPS. The distilled checkpoint uses 8 denoising steps with an Euler ODE sampler
- Text encoder: the language-model part of Qwen2.5-VL-7B-Instruct
- Camera control via 4×4 extrinsic matrices, grouped over four frames and fed through cross-attention in every block
- Latents from DCAE-V (32× spatial, 4× temporal compression), decoded by a fine-tuned PixelUMM decoder in a single step
- A persistent KV cache carries context between segments, which is how it "remembers" the world
That KV cache is where the memory bill comes from. The standard setup keeps the full history on the GPU. The single-GPU setup caps it at 18 latent frames on the GPU and archives everything older on the CPU.
The repo also ships two LoRAs (cinematic and dual-arm robotics) and the non-distilled base autoregressive DiT for fine-tuning.
Hardware tiers: 1 GPU, 8 GPUs, or 8× B200
| Setup | GPUs | Weights | What you get |
|---|---|---|---|
| Standard (inference.yaml) | 8, sp_size=8 | FP32 distilled DiT, PixelUMM | Full history on GPU, distributed decoding |
| Low-memory (single_gpu_low_mem_inference.yaml) | 1× RTX 4090, sp_size=1 | INT8 DiT, INT8 KV cache, FP8 PixelUMM | CPU offload, 18-frame GPU history window |
| Web demo (Reactor runtime) | Tested on 8× B200 | Distilled DiT | Live prompts, WASD camera control |
| Nomas-PE prompt enhancer | Its own node, 8 GPUs by default | PE/ | Required for the apps/ web demo; Reactor and offline runs don't load it |
Here are the memory-relevant lines of the low-memory config (the full file also sets the attention backend, sparse attention and torch_compile):
model:
vae:
cutlass_fp8: true
cpu_offload: true
parallel_mode: none
parallelism:
sp_size: 1
shard_text_encoder: false
text_encoder_cpu_offload: true
dit_cpu_offload: true
dit_cpu_offload_for_decode: false
inference:
dit_weight_quantization: int8
kv_cache_quantization: int8
sliding_window:
enabled: true
past_frames: 18
sampling: temporal_context
output_chunk_frames: 8The text encoder, the DiT and the decoder each move onto the GPU for their own stage and go back to the CPU afterward. That's how it fits in 24 GB, but your system RAM has to hold all of those models plus a history archive that grows the longer your session runs. The README only says offload "requires enough system RAM for the models." Neither the model card nor the README gives a VRAM number, a RAM number or any latency figure, for any tier.
Why "one 24 GB card" means a 4090, not a 3090
The demo docs say CUTLASS FP8 requires SM89, which is Ada. The code is stricter than the docs: hymm/models/vae/pixelumm_decoder/cutlass_fp8.py JIT-compiles its kernel with -gencode=arch=compute_89,code=sm_89. The 4090 is SM89. The 3090 is SM86. The DiT side would be fine (the config comment says the flex attention backend is for SM89 or below), but the FP8 decoder path as shipped is Ada-only. Getting a 3090 working would mean swapping the decoder path yourself, and nothing in the repo says that's supported.
Setup option 1: Docker
You need Docker and the NVIDIA Container Toolkit. Straight from the model card:
git clone https://github.com/Tencent-Hunyuan/Hy-Nomas.git
cd Hy-Nomas
export HY_NOMAS_IMAGE="hynomasoss/hynomasoss:cu130"
docker pull "$HY_NOMAS_IMAGE"
docker run --rm -it --gpus all --ipc=host --network=host \
-v "$PWD:/workspace/Hy-Nomas" \
-w /workspace/Hy-Nomas \
"$HY_NOMAS_IMAGE" bashThe single-GPU path wants exactly one GPU visible to nvidia-smi. On a multi-GPU box, swap --gpus all for --gpus '"device=0"' (or set CUDA_VISIBLE_DEVICES=0 inside the container). Your model directory also has to be visible inside the container, so either download into the mounted repo or add a second -v for it.
I'd take this route. The image is built against CUDA 13.0, it already has CUTLASS 4.8.0 headers in /opt/cutlass with CUTLASS_PATH set, and the manual install compiles kernels.
Setup option 2: uv
You need Python 3.12+, uv, a CUDA 13.0 Toolkit with nvcc, and a C++ compiler:
uv sync --extra all
source .venv/bin/activateThe all extra pulls PyTorch 2.10.0 built for CUDA 13.0, FlashAttention 2 and 3, and MagiAttention. The README warns that compiling the kernels "may take some time." If you've built FlashAttention from source before, you know how much that sentence is hiding. If your distro still ships CUDA 12.x, use Docker.
The step the README skips: CUTLASS_PATH
The FP8 decoder kernel is built on first use from a CUTLASS checkout, and the loader refuses to start without one:
RuntimeError: Set CUTLASS_PATH to a CUTLASS checkout containing Ada GemmBlockwise (example 94)The main README's single-GPU section doesn't mention this; it's buried in apps/README.md. The published Docker image handles it for you (its environment sets CUTLASS_PATH=/opt/cutlass), so this only bites on the uv install. The loader checks for include/cutlass/gemm/device/gemm_blockwise.h, which is in NVIDIA's current main branch, so a shallow clone is enough:
git clone --depth 1 https://github.com/NVIDIA/cutlass.git /opt/cutlass
export CUTLASS_PATH=/opt/cutlassDownload only what the 4090 path needs (22.5 GB, not 196)
The full repo is about 196 GB going by the Hugging Face file listing, mostly two 34.2 GB FP32 DiTs, a 31.6 GB FP32 PixelUMM decoder and the 55.6 GB PE/ folder. The README's hf download command grabs all of it. The low-memory path needs none of those. Keep the DiT/, VAE/, PE/ layout and grab only the INT8 DiT (9.1 GB), the FP8 decoder (9.1 GB), the DCAE-V autoencoder (4.3 GB) and the example prompts:
export HY_NOMAS_MODEL_DIR="$HOME/models/Hy-Nomas"
hf download tencent/Hy-Nomas --local-dir "$HY_NOMAS_MODEL_DIR" \
--include "DiT/distilled_int8/*" --include "VAE/PixelUMM_FT_FP8/*" \
--include "VAE/DCAE-V/*" --include "examples/*"Repeat --include for each pattern. On the current hf CLI (huggingface_hub 2.3.0), passing several patterns after one --include makes it treat the extras as filenames, and the download fails with "File not found". Add --dry-run first if you want to see the list: it should report 4 files totalling 22.5G.
Qwen2.5-VL-7B-Instruct downloads automatically on first run, on top of that. If you already have it, point to it with model.text_encoder.model_name=/path/to/Qwen2.5-VL-7B-Instruct.
One limitation: the INT8 loader can't load a separate LoRA at inference time, so the cinematic and robotics LoRAs are off the table on the 4090 path.
Run Hy-Nomas on one RTX 4090
bash scripts/inference/inference_from_config.sh configs/experiment/single_gpu_low_mem_inference.yaml \
dit_weight="$HY_NOMAS_MODEL_DIR/DiT/distilled_int8/diffusion_pytorch_model_ema_0_995_int8.safetensors" \
model.vae.checkpoint="$HY_NOMAS_MODEL_DIR/VAE/PixelUMM_FT_FP8/diffusion_pytorch_model_fp8.safetensors" \
inference.prompt="$HY_NOMAS_MODEL_DIR/examples/prompts.json" \
inference.save_path=./outputs/single_gpuFor a first run, append overrides that make it cheaper:
inference.video_size=[480,832] \
inference.seed=42The README notes that peak GPU memory depends on resolution and segment length, so 480×832 is the sensible starting point. It's also what the demo's own preview example uses, at 6 steps instead of 8. Steps are set with inference.infer_steps_video. To drive the camera, pass inference.camera_trajectory=/path/to/camera.npz with c2ws shaped [N, 4, 4] and intrinsics shaped [N, 3, 3].
Prompts are a JSON list. Each entry has a test_id, Style and perspective_view fields, a subject_list, and segments with start_time/end_time in MM:SS.mmm. The first segment carries a visual_description_long, later segments carry a description_change, and each segment lists its contained_actions with their own timestamps. Segment lengths come from the timestamps. Copy examples/prompts.json (a two-segment fox-in-a-forest example) and edit it rather than writing these from scratch.
The catch: the web demo needs Nomas-PE, and Nomas-PE needs 8 GPUs
I'd want to know this before downloading anything. Nomas-PE, the prompt enhancer that turns "walk toward the lighthouse" into the structured segment format above, is a separate online service. The demo deployment guide runs it with SGLang (an image "that supports Qwen3.8 inference"), and apps/serve_nomas_pe.sh passes --tp-size "${PE_TP_SIZE:-8}", so one node with 8 GPUs by default. The docs don't give a parameter count, but the PE/ weights are 55.6 GB on Hugging Face. That doesn't fit on a 24 GB card next to the video model, or on its own.
The demo does have a --single-gpu-low-mem-mode flag for the video workers, and the very next line in the guide says: "PE remains required."
The other live option is the Reactor integration. It doesn't load Nomas-PE at all, so there's no automatic conversion of plain text into structured JSON. It also ships an optional one-GPU hy_nomas_low_mem.yaml, but its tested profile is eight B200s, it asks for at least 128 GB of host RAM, and the guide admits that "RTX 4090 GPU execution has not been validated." So unless you have a Nomas-PE endpoint somewhere, the only setup Tencent actually says works on one 4090 is offline batch generation from hand-written segment files. That's still useful for testing camera trajectories or building a dataset, but you lose the live, Genie-style steering from the demo video.
No benchmarks, no 4090 numbers yet
There are no published benchmark scores. Tencent released Hy-Nomas-Bench, with 280 test cases across multi-turn text interaction, camera navigation, interleaved text and camera control, and spatial memory, but the model card reports no results on it. It says outright that the architecture and sampling defaults are configuration details, not benchmark results. The tech report is marked "Coming Soon", with a PDF link that has no report behind it yet.
At the time of writing the repo has 195 stars and no open issues, and the only thread on the Hugging Face model page is a merged pull request that added the LICENSE file. Nobody has posted 4090 timings or RAM usage yet, so whoever runs it first gets to report the real numbers.
My take: the Apache-2.0 license, the LoRA training recipe and the INT8 path make this a generous release. But "runs on one RTX 4090" is doing a lot of work in that README when there's no latency figure, no RAM figure for that card, a CUTLASS_PATH step the main README leaves out, and both interactive setups are documented on 8-GPU boxes.
Would you run a world model on a single 4090 with CPU offload and an 18-frame memory window, or is this still cloud-only for anything practical? If you've got it generating, post your system RAM, peak VRAM and seconds per segment in the comments.