Turning a Photo into a Rough Unreal Engine Asset

A good time after first learning Unreal Engine, I started using UE to PoC small game ideas and the assets were one problem that showed up fast. Marketplace assets help but they can get expensive and they often bring a different theme or art style with them than I would like to use for the PoC I am thinking.

For prototypes I wanted a faster way to make rough custom props that actually fit the scene. Not final production assets or complete production models. Just quick objects for greyboxing, background props, scene dressing and early experiments. Then I learned about all of the current AI 3D model generator websites that essentially have their own finetuned model or just use an already trained model, like Trellis.2 or Hunyuan3D. I started looking at the existing local solutions and didn’t really find anything that did more than just creating the model.

That is what this quick project is for, it takes a simple screenshot or text and turns it into an reference image and runs it through a local asset pipeline that creates the model.

The useful part is not only the generated mesh. It is the workflow around it, since it not only creates the model, but it prepares the image, keeps the raw output, inspects it, cleans it and then exports Unreal-friendly files.

The high level flow ended up something like this:

input image
-> prepare image
-> run 3D backend
-> inspect raw mesh
-> clean through Blender
-> inspect cleaned mesh
-> write GLB, FBX, preview and asset_report.json

That is the part I wanted from this project, not just “generate a chair”, but a repeatable local workflow around the generated chair.

The CLI shape

Polywright is a Python CLI tool. The installed command comes from pyproject.toml:

[project.scripts]
polywright = "polywright.cli:app"

The main commands stay small from the outside:

.\.venv\Scripts\polywright.exe prepare-image --input .\examples\test_asset1\chair.png --asset-name chair --out .\out\chair
.\.venv\Scripts\polywright.exe inspect --input .\test_assets\sample_cube.obj --out .\out\sample_cube
.\.venv\Scripts\polywright.exe clean --input .\test_assets\sample_cube.obj --asset-name sample_cube --out .\out\sample_cube_clean
.\.venv\Scripts\polywright.exe generate --input .\examples\test_asset1\chair.png --backend trellis2 --asset-name chair_80k --out .\out\chair_80k --target-faces 80000 --texture-size 1024

generate is the main command because it connects the steps that already work:

result = run_generate_asset(
    input_path=input,
    backend_name=backend,
    out_dir=out,
    config=config,
    asset_name=asset_name,
    max_size=max_size,
    clean=not skip_clean,
    target_height=target_height,
    decimate_ratio=decimate_ratio,
    render_preview=preview,
    quality=quality,
    target_faces=target_faces,
    texture_size=texture_size,
)

That function is the real pipeline entry point right now. run_pipeline.py exists, but it is only a placeholder.

The output folder matters

The first useful thing to standardize was the output folder:

input/
raw/
cleaned/
textures/
preview/
unreal/
logs/

This keeps each stage separate: input files, raw model output, cleaned exports, previews, logs and later Unreal files.

The folder layout is just a small helper:

WORKSPACE_DIRS = (
    "input",
    "raw",
    "cleaned",
    "textures",
    "preview",
    "unreal",
    "logs",
)

def create_asset_workspace(asset_name: str, out_dir: Path) -> AssetWorkspace:
    safe_name = slugify_asset_name(asset_name)
    root = out_dir
    root.mkdir(parents=True, exist_ok=True)

    paths = {name: root / name for name in WORKSPACE_DIRS}
    for path in paths.values():
        path.mkdir(parents=True, exist_ok=True)

    (root / ".polywright").write_text(f"asset_name={safe_name}\n", encoding="utf-8")

    return AssetWorkspace(root=root, **paths)

The .polywright file is mostly a marker for now. Later it can support package, compare or resume-style commands.

Preparing the image

Single-image 3D models already have to guess a lot, so the input should at least be predictable.

The preparation step supports .jpg, .jpeg, .png and .webp. It copies the original image, applies EXIF orientation, flattens transparency, resizes if needed and writes the PNG used by the backend:

with Image.open(input_path) as source_image:
    original_size = source_image.size
    mode_before = source_image.mode
    exif_orientation = source_image.getexif().get(274)
    image = ImageOps.exif_transpose(source_image)
    exif_applied = exif_orientation not in {None, 1}
    image = flatten_to_background(image, config.prepared_background)
    image, resized = resize_to_max_size(image, max_size or config.prepared_max_size)
    image.save(prepared_path, format="PNG", optimize=True)

It also writes input_metadata.json:

source input
original image path
prepared image path
original size
prepared size
mode before and after
whether EXIF orientation was applied
whether resize happened
input warnings

The warning rules are simple:

def build_input_warnings(*, original_size: tuple[int, int], prepared_size: tuple[int, int]) -> list[str]:
    warnings: list[str] = []
    if min(original_size) < 512:
        warnings.append("image_too_low_resolution")
    if min(prepared_size) < 512:
        warnings.append("prepared_image_too_low_resolution")
    if max(original_size) / max(1, min(original_size)) > 3:
        warnings.append("extreme_image_aspect_ratio")
    return warnings

There is no segmentation or object masking yet. For now, the tool expects a clean, centered object image.

Backend choice

The official backend path right now is TRELLIS/TRELLIS.2. Polywright keeps it behind a backend interface so the rest of the tool only needs a raw mesh path and metadata:

class GenerationBackend(ABC):
    name: str

    @abstractmethod
    def is_available(self) -> bool:
        """Return whether this backend can run on the current machine."""

    @abstractmethod
    def generate(
        self,
        prepared_input: Path,
        output_dir: Path,
        asset_name: str,
        options: GenerationOptions | None = None,
    ) -> GeneratedMesh:
        """Generate a raw mesh from a prepared input image."""

Right now the registry is small:

def create_backend(name: str, config: dict[str, Any] | None = None) -> GenerationBackend:
    normalized = name.lower().strip()
    if normalized in {"trellis", "trellis2", "trellis.2"}:
        return Trellis2Backend(config)
    if normalized == "triposr":
        return TripoSRBackend()
    raise ValueError(f"unknown backend '{name}'")

TripoSRBackend is still a placeholder, so the real generated artifacts use TRELLIS.2.

I also kept Hunyuan3D out of the official path. It may be interesting, but the licensing and territory constraints are not a clean default for an EU-maintained public repo.

For this version, the backend shape is:

official backend: TRELLIS/TRELLIS.2
future baseline: TripoSR
excluded official backend: Hunyuan3D, unless the license situation changes

Running TRELLIS from the repo

The TRELLIS adapter reads its paths from polywright.yaml:

backends:
  trellis2:
    enabled: true
    repo_path: "D:/ai/TRELLIS.2"
    python_executable: "C:/Users/Ietu/miniconda3/envs/trellis2win/python.exe"
    model: "D:/ai/models/trellis.2-4B"
    attention_backend: "xformers"
    sparse_attention_backend: "xformers"
    sparse_conv_backend: "flex_gemm"
    xformers_attention_op: "cutlass"
    seed: 1
    decimation_target: 1000000
    texture_size: 4096
    remesh: true

The backend does not import TRELLIS inside the normal Polywright process. It runs a small adapter script in a subprocess:

command = [
    str(self.python_executable),
    str(runner),
    "--repo-path",
    str(self.repo_path),
    "--input",
    str(prepared_input.resolve()),
    "--output",
    str(mesh_path.resolve()),
    "--model",
    str(self.model),
    "--seed",
    str(self.seed),
    "--decimation-target",
    str(decimation_target),
    "--texture-size",
    str(texture_size),
    "--remesh" if remesh else "--no-remesh",
    "--remesh-band",
    str(self.remesh_band),
    "--remesh-project",
    str(self.remesh_project),
]

The adapter inserts the TRELLIS repo into sys.path, loads the pipeline, runs image-to-3D and exports a GLB:

pipeline = Trellis2ImageTo3DPipeline.from_pretrained(args.model)
pipeline.cuda()

image = Image.open(args.input)
mesh = pipeline.run(image, seed=args.seed)[0]
mesh.simplify(16_777_216)

glb = o_voxel.postprocess.to_glb(
    vertices=mesh.vertices,
    faces=mesh.faces,
    attr_volume=mesh.attrs,
    coords=mesh.coords,
    attr_layout=mesh.layout,
    voxel_size=mesh.voxel_size,
    aabb=[[-0.5, -0.5, -0.5], [0.5, 0.5, 0.5]],
    decimation_target=args.decimation_target,
    texture_size=args.texture_size,
    remesh=args.remesh,
    remesh_band=args.remesh_band,
    remesh_project=args.remesh_project,
    verbose=True,
)
glb.export(output, extension_webp=True)

This keeps the TRELLIS environment separate, which is useful on Windows. The main CLI can still prepare images, inspect meshes and run Blender cleanup without importing the model stack.

The backend also writes raw metadata:

{
  "asset_name": "chair_80k",
  "backend": "trellis2",
  "prepared_input": "out\\chair_80k\\input\\prepared.png",
  "mesh_path": "out\\chair_80k\\raw\\trellis2_raw.glb",
  "model": "D:/ai/models/trellis.2-4B",
  "seed": 1,
  "decimation_target": 80000,
  "texture_size": 1024,
  "remesh": true
}

That metadata matters because the same input can change when the model, seed, face target, texture size or remesh settings change.

Inspecting the mesh

Inspection checks whether the generated file is actually usable.

The current implementation uses trimesh to load .obj, .glb and .gltf. For GLB/GLTF it also reads glTF metadata so material and texture counts are not missed:

loaded = trimesh.load(path, process=False)
meshes = _extract_meshes(loaded)

vertices = sum(int(len(mesh.vertices)) for mesh in meshes)
faces = sum(int(len(getattr(mesh, "faces", []))) for mesh in meshes)
bounds = _combined_bounds(meshes)
materials = count_materials(meshes)
textures = count_textures(meshes)
gltf_metadata = read_gltf_metadata(path)
if gltf_metadata is not None:
    materials = max(materials, gltf_metadata.materials)
    textures = max(textures, gltf_metadata.textures)
has_uvs = any(mesh_has_uvs(mesh) for mesh in meshes)

The report answers the basic questions:

what format is this?
how many geometry parts are there?
how many vertices and faces?
does it have materials?
does it have textures?
does it have UVs?
what are the bounds?
what warnings should I care about?

The current warning rules are also simple:

if vertices <= 0 or faces <= 0:
    warnings.append("empty_mesh")
if faces > 250_000:
    warnings.append("high_poly_for_small_prop")
if not has_uvs:
    warnings.append("missing_uvs")
if materials <= 0:
    warnings.append("missing_materials")
if textures <= 0:
    warnings.append("missing_textures")
if any(axis <= 0 for axis in bounds.size):
    warnings.append("invalid_bounds")

This is not a full game-asset validator. It just catches obvious problems: empty meshes, missing UVs, missing materials, missing textures, invalid bounds and very dense props.

In one local chair run, the raw TRELLIS output for chair_preview had 607,194 vertices and 976,429 faces, so it was flagged as high_poly_for_small_prop. After cleanup, the GLB had 271,315 vertices and 146,463 faces, and that warning went away.

That is more useful than saving a GLB and hoping it imports cleanly.

Blender as the cleanup engine

I use Blender for cleanup instead of writing mesh cleanup from scratch. Blender already handles imports, exports, transforms and preview renders.

The cleanup command runs Blender headlessly:

command = [
    str(blender_executable),
    "--background",
    "--python",
    str(script_path),
    "--",
    "--input",
    str(raw_input.resolve()),
    "--output-glb",
    str(output_glb.resolve()),
    "--output-fbx",
    str(output_fbx.resolve()),
    "--asset-name",
    safe_name,
    "--target-height",
    str(target_height if target_height is not None else blender.target_height_meters),
    "--decimate-ratio",
    str(decimate_ratio if decimate_ratio is not None else blender.decimate_ratio),
]

Then polywright/blender/cleanup_asset.py owns the cleanup steps:

clear_scene()
import_mesh(input_path)
obj = join_meshes(args.asset_name)
clean_mesh(obj, target_height=args.target_height, decimate_ratio=args.decimate_ratio)

export_glb(output_glb)
export_fbx(output_fbx)
if preview is not None:
    render_preview(obj, preview)

The cleanup itself stays plain:

def clean_mesh(obj: bpy.types.Object, *, target_height: float, decimate_ratio: float) -> None:
    bpy.context.view_layer.objects.active = obj
    obj.select_set(True)
    bpy.ops.object.transform_apply(location=False, rotation=True, scale=True)

    normalize_origin_to_bottom_center(obj)
    normalize_height(obj, target_height)
    remove_loose_and_merge(obj)
    recalculate_normals(obj)
    apply_decimate(obj, decimate_ratio)
    normalize_origin_to_bottom_center(obj)

That cleanup pass does the practical fixes:

clear the default Blender scene
import the raw mesh
join multiple mesh objects if needed
apply rotation and scale
move the origin to bottom center
normalize height to a target meter value
delete loose geometry
merge very close vertices
recalculate normals outside
optionally apply a decimate modifier
export GLB
export FBX
render a static preview

The origin fix matters in Unreal. For props, bottom-center is usually a better default than whatever the generated file came with.

def normalize_origin_to_bottom_center(obj: bpy.types.Object) -> None:
    local_points = [Vector(corner) for corner in obj.bound_box]
    min_x = min(point.x for point in local_points)
    max_x = max(point.x for point in local_points)
    min_y = min(point.y for point in local_points)
    max_y = max(point.y for point in local_points)
    min_z = min(point.z for point in local_points)
    offset = Vector(((min_x + max_x) / 2.0, (min_y + max_y) / 2.0, min_z))

    for vertex in obj.data.vertices:
        vertex.co -= offset

    obj.location = (0.0, 0.0, 0.0)
    bpy.context.view_layer.update()

After cleanup, the chair_80k report showed centered bounds and a height of exactly 1.0:

"bounds": {
  "min": [-0.2666746973991394, 0.0, -0.3115478456020355],
  "max": [0.2666746973991394, 1.0, 0.3115478456020355],
  "size": [0.5333493947982788, 1.0, 0.623095691204071],
  "center": [0.0, 0.5, 0.0]
}

That small fix makes generated props easier to use later.

Quality presets and face counts

I added --quality, --target-faces and --texture-size because generated meshes can get dense quickly.

The code has three presets:

QUALITY_PRESETS: dict[str, dict[str, int | float]] = {
    "preview": {
        "decimation_target": 80_000,
        "texture_size": 1024,
        "decimate_ratio": 1.0,
    },
    "standard": {
        "decimation_target": 300_000,
        "texture_size": 2048,
        "decimate_ratio": 1.0,
    },
    "high": {
        "decimation_target": 1_000_000,
        "texture_size": 4096,
        "decimate_ratio": 1.0,
    },
}

Or the values can be passed directly:

.\.venv\Scripts\polywright.exe generate `
  --input .\examples\test_asset1\chair.png `
  --backend trellis2 `
  --asset-name chair_80k `
  --out .\out\chair_80k `
  --target-faces 80000 `
  --texture-size 1024

For the chair_80k run, the raw GLB had 45,910 vertices, 77,026 faces, one material, two textures and UVs. The cleaned GLB had 76,395 vertices and the same 77,026 faces, with origin and scale normalized.

A higher-density preview run was close to a million faces, and the report flagged it. A preview image alone would not show that.

This also showed that heavy Blender decimation can make hard-surface outputs jagged. For now, it is better to control size at the backend export stage, then use Blender mostly for transform, origin, scale, export and preview.

The asset report

The report is simple JSON, but it is one of the most useful outputs:

{
  "raw_mesh": {
    "file_path": "out\\chair_80k\\raw\\trellis2_raw.glb",
    "file_format": "glb",
    "geometry_count": 1,
    "vertices": 45910,
    "faces": 77026,
    "materials": 1,
    "textures": 2,
    "has_uvs": true,
    "warnings": []
  },
  "cleaned_mesh": {
    "file_path": "out\\chair_80k\\cleaned\\chair_80k_unreal.glb",
    "file_format": "glb",
    "geometry_count": 1,
    "vertices": 76395,
    "faces": 77026,
    "materials": 1,
    "textures": 2,
    "has_uvs": true,
    "warnings": []
  },
  "warnings": []
}

It does not judge the art. It tells me whether the file has textures, UVs, sane bounds and a face count worth checking.

The final output folder for the local chair run looked like this:

out/chair_80k/
  input/
    original.png
    prepared.png
    input_metadata.json
  raw/
    trellis2_raw.glb
    trellis2_raw_metadata.json
  cleaned/
    chair_80k_unreal.glb
    chair_80k_unreal.fbx
  preview/
    cleaned_front.png
  asset_report.json

That is the artifact folder I want from this tool. It shows what input was used, what settings were used, what cleanup changed and whether the result is obviously broken.

What is not done yet

This is not a finished AI-to-Unreal pipeline.

The package-unreal command exists, but it deliberately exits as not implemented:

@app.command("package-unreal")
def package_unreal(
    input: Annotated[Path, typer.Option("--input", exists=True, file_okay=True, dir_okay=False)],
    out: Annotated[Path, typer.Option("--out")],
) -> None:
    """Package an existing cleaned mesh for Unreal. Implementation starts in Milestone 6."""
    console.print(f"[yellow]Unreal packaging is not implemented yet.[/yellow] Input: {input}, out: {out}")
    raise typer.Exit(code=2)

The compare command is also a placeholder. TripoSR is not wired up yet. There is no Unreal import script, collision generation, LOD generation or backend comparison report.

There are also model limits that cleanup cannot fix. A single-image 3D model has to guess hidden geometry, and thin parts, backsides, legs, holes and hard-surface edges can come out wrong.

So I treat the output as a rough asset candidate, not a final asset.

What the current version proved

The current repo proves the useful middle part:

single image
-> prepared PNG
-> TRELLIS.2 raw GLB
-> raw mesh report
-> Blender cleaned GLB and FBX
-> static preview render
-> cleaned mesh report

That is already better than a one-off model script because every run leaves behind inspectable artifacts.

The local chair example shows the current quality level. The result is recognizable as the same kind of chair, and the cleanup path gives it a usable pivot, scale, GLB, FBX, preview and report.

And with this, it makes creating a bunch of assets for a scene a 30 minute job instead of a 30 hour job, for me.

What I would improve next

The next useful step is finishing the missing asset-pipeline pieces:

wire TripoSR as the baseline backend
add compare mode for TRELLIS vs TripoSR
generate Unreal import helper files
add collision recommendations
add stronger input checks
add before and after preview images
add optional scan import mode

And compare Trellis.2 with Hunyuan3D 3.0 model.