manga-image-translator: Image Text Translation Project

A project that translates images such as comics, with support for image repair (text removal) and typesetting.

Verified

Core facts

Read by a script from the official API, not rewritten by a model.

License
GPL-3.0
Languages
Python
Maintainer
zyddnys
Verified

What it is

This project aims to translate images that are unlikely to be professionally translated, such as comics or images on group chats and image boards. It mainly supports Japanese, and also supports Simplified and Traditional Chinese, English and 20 other minor languages. The project supports image repair (text removal) and typesetting.

Main features

  1. The config-help mode shows the documentation for the JSON schema of the configuration.

  2. The pre-dict option can correct common OCR errors or irrelevant special effect text before translation.

  3. The post-dict option can modify common mistranslations or unnatural phrasing after translation to conform to the habits of the target language.

  4. When the output format is set to xcf, psd or pdf, Gimp is used to generate the files.

  5. On Windows, this assumes Gimp 2.x is installed in the AppData\Local\Programs\Gimp 2 folder under the profile directory of the current user.

  6. The resulting xcf file contains the original image as the lowest layer and the inpainting as a separate layer.

Getting started

  1. In the local setup, dependencies are pinned to current releases.

  2. The local setup requires installing Git and a C++ compiler before running pip.

  3. Pip installs its Cython build dependency automatically.

  4. Models are downloaded automatically at runtime.

  5. The project supports Docker.

  6. The README text says the Web Server starts on port 8000 by default; however, the Docker command included in this section sets a different port explicitly with -p and --port, so when running that command use the port given in the command, not 8000.

From the README · bash
# First, ensure you have Python 3.12 or later installed on your machine
# The very latest version of Python might not be compatible with some PyTorch libraries yet
$ python --version
Python 3.13.16

# Clone this repository
$ git clone https://github.com/zyddnys/manga-image-translator.git

# Create a venv (optional, but recommended)
$ python -m venv venv

# Activate the venv
$ source venv/bin/activate

# If you want to use the --use-gpu option, please visit https://pytorch.org/get-started/locally/ to install PyTorch, which needs to correspond to your CUDA version.
# If you did not use venv to create a virtual environment, you need to add --upgrade --force-reinstall to the pip command to overwrite the currently installed PyTorch version.

# Install dependencies
$ pip install -r requirements.txt
From the README · bash
docker run \
  --name manga_image_translator_cpu \
  -p 5003:5003 \
  --ipc=host \
  --entrypoint python \
  --rm \
  -v /demo/doc/../../result:/app/result \
  -v /demo/doc/../../server/main.py:/app/server/main.py \
  -v /demo/doc/../../server/instance.py:/app/server/instance.py \
  -e OPENAI_API_KEY='' \
  -e OPENAI_API_BASE='' \
  zyddnys/manga-image-translator:main \
  server/main.py --verbose --start-instance --host=0.0.0.0 --port=5003
From the README · bash
# Replace <path> with the path to your image folder or file.
$ python -m manga_translator local -v -i <path>
# The results can be found in `<path_to_image_folder>-translated`.

Who it is for

The project is still in the early stages of development and has many shortcomings. The online version uses the latest version from the main branch. In the Rust version, stable diffusion is not rewritten yet and only a CLI version is available.

The Docker image contains the dependencies and models required for the project. The Docker image is about 15GB in size. For CUDA or GPU acceleration, Docker version 19.03 or later is required.

Nvidia Container Runtime is optional and is needed only if you want to use CUDA. Building the Docker image locally requires the make tool installed on the machine. When using Docker through the command line, referencing files on the host requires mounting the relevant files as volumes inside the container.

Some translation services require API keys, which are passed to the docker container as environment variables. In the translator reference, the API Key column indicates whether the translator requires API keys to be set as environment variables. In the translator reference, the Offline column indicates whether the translator can be used offline.

The env file contains sensitive information. Increasing box_threshold can help filter out gibberish from OCR error detection to some extent. The sakura_dict glossary is effective only for sakuratranslator.

The custom_openai translator cannot load the glossary file, which OpenaiTranslator loads. Gimp converts text layers to regular images when saving psd files. Gimp does not handle rotated text well.

Sources

This page is written from the official sources below. Open them to check the original text.

Verified ·

This page is written by AI from the official sources above and checked automatically. This site has not tested it. Found a mistake? Tell us.