2022-03-26 07:36:57 +08:00
# The Torch-MLIR Project
2021-09-24 02:27:03 +08:00
2021-09-29 04:53:33 +08:00
The Torch-MLIR project aims to provide first class compiler support from the [PyTorch ](https://pytorch.org ) ecosystem to the MLIR ecosystem.
2020-08-01 11:53:09 +08:00
> This project is participating in the LLVM Incubator process: as such, it is
not part of any official LLVM release. While incubation status is not
necessarily a reflection of the completeness or stability of the code, it
does indicate that the project is not yet endorsed as a component of LLVM.
2021-09-29 04:53:33 +08:00
[PyTorch ](https://pytorch.org )
2021-09-24 02:27:03 +08:00
An open source machine learning framework that accelerates the path from research prototyping to production deployment.
[MLIR ](https://mlir.llvm.org )
The MLIR project is a novel approach to building reusable and extensible compiler infrastructure. MLIR aims to address software fragmentation, improve compilation for heterogeneous hardware, significantly reduce the cost of building domain specific compilers, and aid in connecting existing compilers together.
2021-09-24 03:29:35 +08:00
[Torch-MLIR ](https://github.com/llvm/torch-mlir )
2022-03-26 07:36:57 +08:00
Multiple Vendors use MLIR as the middle layer, mapping from platform frameworks like PyTorch, JAX, and TensorFlow into MLIR and then progressively lowering down to their target hardware. We have seen half a dozen custom lowerings from PyTorch to MLIR. Having canonical lowerings from the PyTorch ecosystem to the MLIR ecosystem would provide much needed relief to hardware vendors to focus on their unique value rather than implementing yet another PyTorch frontend for MLIR. The goal is to be similar to current hardware vendors adding LLVM target support instead of each one also implementing Clang / a C++ frontend.
2021-09-24 02:27:03 +08:00
## All the roads from PyTorch to Torch MLIR Dialect
We have few paths to lower down to the Torch MLIR Dialect.
![Torch Lowering Architectures ](Torch-MLIR.png )
2022-03-26 07:36:57 +08:00
- TorchScript
This is the most tested path down to Torch MLIR Dialect, and the PyTorch ecosystem is converging on using TorchScript IR as a lingua franca.
- LazyTensorCore (Based on the PyTorch [`lazy_tensor_staging` branch ](https://github.com/pytorch/pytorch/tree/lazy_tensor_staging/lazy_tensor_core ))
This path provides the upcoming LTC path of capture. It is based of an unstable devel branch but is the closest way for you to adapt any existing `torch/xla` derivatives.
2021-09-24 02:27:03 +08:00
2021-11-09 03:18:00 +08:00
## Project Communication
- `#torch-mlir` channel on the LLVM [Discord ](https://discord.gg/xS7Z362 ) - this is the most active communication channel
- Github issues [here ](https://github.com/llvm/torch-mlir/issues )
- [`torch-mlir` section ](https://llvm.discourse.group/c/projects-that-want-to-become-official-llvm-projects/torch-mlir/41 ) of LLVM Discourse
2021-10-07 09:20:15 +08:00
2021-09-29 04:50:25 +08:00
## Check out the code
2020-09-17 12:57:46 +08:00
```shell
2021-09-29 04:50:25 +08:00
git clone https://github.com/llvm/torch-mlir
cd torch-mlir
git submodule update --init
```
2020-11-04 05:46:46 +08:00
2021-09-30 00:59:39 +08:00
## Setup your Python VirtualEnvironment and Dependencies
2020-09-17 12:57:46 +08:00
2021-10-05 02:56:11 +08:00
```shell
2021-09-29 04:50:25 +08:00
python -m venv mlir_venv
source mlir_venv/bin/activate
2021-10-07 09:20:15 +08:00
# Some older pip installs may not be able to handle the recent PyTorch deps
python -m pip install --upgrade pip
2021-10-22 12:18:08 +08:00
# Install latest PyTorch nightlies and build requirements.
python -m pip install -r requirements.txt
2020-10-08 09:31:24 +08:00
```
2021-10-22 12:09:00 +08:00
## Build
2021-10-05 02:56:11 +08:00
```shell
2021-09-29 04:50:25 +08:00
cmake -GNinja -Bbuild \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
2021-10-13 23:47:42 +08:00
-DPython3_FIND_VIRTUALENV=ONLY \
2021-09-29 04:50:25 +08:00
-DLLVM_ENABLE_PROJECTS=mlir \
2022-02-16 15:02:28 +08:00
-DLLVM_EXTERNAL_PROJECTS="torch-mlir;torch-mlir-dialects" \
2021-09-29 04:50:25 +08:00
-DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR=`pwd` \
2022-03-27 00:12:27 +08:00
-DLLVM_EXTERNAL_TORCH_MLIR_DIALECTS_SOURCE_DIR=`pwd`/externals/llvm-external-projects/torch-mlir-dialects \
2021-09-29 04:50:25 +08:00
-DMLIR_ENABLE_BINDINGS_PYTHON=ON \
-DLLVM_TARGETS_TO_BUILD=host \
2022-03-27 00:12:27 +08:00
externals/llvm-project/llvm
2021-09-29 04:50:25 +08:00
# Additional quality of life CMake flags:
# Enable ccache:
2022-01-20 05:19:46 +08:00
# -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache
# Enable LLD (links in seconds compared to minutes)
2021-09-29 04:50:25 +08:00
# -DCMAKE_EXE_LINKER_FLAGS_INIT="-fuse-ld=lld" -DCMAKE_MODULE_LINKER_FLAGS_INIT="-fuse-ld=lld" -DCMAKE_SHARED_LINKER_FLAGS_INIT="-fuse-ld=lld"
# Use --ld-path= instead of -fuse-ld=lld for clang > 13
2021-10-22 12:09:00 +08:00
# Build just torch-mlir (not all of LLVM)
cmake --build build --target tools/torch-mlir/all
# Run unit tests.
cmake --build build --target check-torch-mlir
# Build everything (including LLVM)
2022-01-20 05:19:46 +08:00
cmake --build build
2021-09-29 04:50:25 +08:00
```
2021-09-28 08:10:47 +08:00
## Demos
2020-09-17 12:57:46 +08:00
2021-09-30 00:59:39 +08:00
## Setup Python Environment
2021-10-05 02:56:11 +08:00
```shell
2021-09-30 00:59:39 +08:00
export PYTHONPATH=`pwd`/build/tools/torch-mlir/python_packages/torch_mlir:`pwd`/examples
2021-09-29 04:50:25 +08:00
```
2021-09-28 08:10:47 +08:00
2021-09-29 04:50:25 +08:00
### TorchScript
2021-09-30 00:20:04 +08:00
2021-09-28 08:10:47 +08:00
Running execution (end-to-end) tests:
2020-10-08 09:31:24 +08:00
2021-10-05 02:56:11 +08:00
```shell
2021-09-28 08:10:47 +08:00
# Run E2E TorchScript tests. These compile and run the TorchScript program
2021-09-29 12:43:38 +08:00
# through torch-mlir with a simplified MLIR CPU backend we call RefBackend
python -m e2e_testing.torchscript.main --filter Conv2d --verbose
2020-09-17 12:57:46 +08:00
```
2021-09-30 00:20:04 +08:00
[Example IR ](https://gist.github.com/silvasean/e74780f8a8a449339aac05c51e8b0caa ) for a simple 1 layer MLP to show the compilation steps from TorchScript.
2021-09-29 12:43:38 +08:00
Standalone script to Convert a PyTorch ResNet18 model to MLIR and run it on the CPU Backend:
2021-07-01 05:13:21 +08:00
2021-10-05 02:56:11 +08:00
```shell
2021-10-01 03:48:35 +08:00
# The example uses PIL and requests to get the image.
pip install requests pillow
2021-09-28 08:10:47 +08:00
# Run ResNet18 as a standalone script.
python examples/torchscript_resnet18_e2e.py
2021-09-29 12:43:38 +08:00
load image from https://upload.wikimedia.org/wikipedia/commons/2/26/YellowLabradorLooking_new.jpg
2021-09-30 00:59:39 +08:00
Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /home/mlir/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth
2021-09-29 12:43:38 +08:00
100.0%
PyTorch prediction
[('Labrador retriever', 70.66319274902344), ('golden retriever', 4.956596374511719), ('Chesapeake Bay retriever', 4.195662975311279)]
torch-mlir prediction
[('Labrador retriever', 70.66320037841797), ('golden retriever', 4.956601619720459), ('Chesapeake Bay retriever', 4.195651531219482)]
2021-09-28 08:10:47 +08:00
```
2021-07-01 05:13:21 +08:00
2021-09-28 08:10:47 +08:00
Jupyter notebook:
2021-10-05 02:56:11 +08:00
```shell
2021-09-28 08:10:47 +08:00
python -m ipykernel install --user --name=torch-mlir --env PYTHONPATH "$PYTHONPATH"
# Open in jupyter, and then navigate to
# `examples/resnet_inference.ipynb` and use the `torch-mlir` kernel to run.
jupyter notebook
2021-07-01 05:13:21 +08:00
```
2022-03-26 07:36:57 +08:00
### LazyTensorCore
2021-07-01 05:13:21 +08:00
2022-03-26 07:36:57 +08:00
The LazyTensorCore integration is still in progress, and is being built on the
[`torch_mlir_ltc_backend` branch ](https://github.com/llvm/torch-mlir/tree/torch_mlir_ltc_backend ).
2021-07-10 03:22:45 +08:00
2021-09-29 04:50:25 +08:00
## Repository Layout
2021-07-10 03:22:45 +08:00
2021-09-29 04:50:25 +08:00
The project follows the conventions of typical MLIR-based projects:
2021-07-10 03:22:45 +08:00
2021-09-29 04:50:25 +08:00
* `include/torch-mlir` , `lib` structure for C++ MLIR compiler dialects/passes.
* `test` for holding test code.
* `tools` for `torch-mlir-opt` and such.
* `python` top level directory for Python code
2021-07-10 03:22:45 +08:00
2021-09-29 04:50:25 +08:00
## Interactive Use
2021-07-10 03:22:45 +08:00
2021-09-29 04:50:25 +08:00
The `build_tools/write_env_file.sh` script will output a `.env`
file in the workspace folder with the correct PYTHONPATH set. This allows
tools like VSCode to work by default for debugging. This file can also be
manually `source` 'd in a shell.
2021-10-22 12:18:08 +08:00
## Build Python Packages
We have preliminary support for building Python packages. This can be done
with the following commands:
```
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
CMAKE_GENERATOR=Ninja python setup.py bdist_wheel
```