2020-04-27 06:50:23 +08:00
|
|
|
# npcomp - An aspirational MLIR based numpy compiler
|
|
|
|
|
|
2020-04-27 11:14:20 +08:00
|
|
|
This is a research prototype of MLIR dialects for representing
|
|
|
|
|
numpy programs, and a set of reference tracing/compiler tools.
|
2020-04-30 07:11:18 +08:00
|
|
|
The primary purpose at this point is to establish a solid modeling
|
|
|
|
|
of restricted Python programs and Numpy based computations in MLIR.
|
|
|
|
|
While this project will provide some reference implementations to prove
|
|
|
|
|
the design, the intention is to align this with the broader set of
|
|
|
|
|
tools that exist at this level of abstraction.
|
2020-04-27 06:50:23 +08:00
|
|
|
|
2020-04-30 01:55:35 +08:00
|
|
|
## Design Notes
|
|
|
|
|
|
|
|
|
|
As I work through things, I've been jotting down some design notes:
|
|
|
|
|
|
|
|
|
|
* [Type Extraction - April 15, 2020](https://gist.github.com/stellaraccident/ec1ab0f633cfca0a05866fd77705b4e4)
|
|
|
|
|
* [Ufunc modeling Part 1 - April 29, 2020](https://gist.github.com/stellaraccident/4fcd2a24a66b6588f92b22b2b8ab974f)
|
2020-05-06 04:49:05 +08:00
|
|
|
* [Array funcs and op granularity - May 5, 2020](https://gist.github.com/stellaraccident/2c11652cfdee1457921bc7c98807b462)
|
2020-04-30 01:55:35 +08:00
|
|
|
|
2020-04-27 07:32:10 +08:00
|
|
|
## Quick start
|
2020-04-27 06:50:23 +08:00
|
|
|
|
2020-04-27 07:32:10 +08:00
|
|
|
```
|
2020-04-30 07:33:48 +08:00
|
|
|
LLVM_VERSION=10
|
|
|
|
|
export CC=clang-$LLVM_VERSION
|
|
|
|
|
export CXX=clang++-$LLVM_VERSION
|
|
|
|
|
export LDFLAGS=-fuse-ld=$(which ld.lld-$LLVM_VERSION)
|
2020-04-27 07:32:10 +08:00
|
|
|
export LLVM_SRC_DIR=/path/to/llvm-project
|
2020-04-30 07:33:48 +08:00
|
|
|
|
2020-05-06 05:00:06 +08:00
|
|
|
# Check out last known good commit.
|
2020-05-06 12:00:49 +08:00
|
|
|
(cd $LLVM_SRC_DIR && git checkout 0c4aab27b3da05dd1b0c0c39472525325fda5e23)
|
2020-05-06 05:00:06 +08:00
|
|
|
|
2020-04-27 07:32:10 +08:00
|
|
|
./tools/install_mlir.sh
|
|
|
|
|
./tools/cmake_configure.sh
|
|
|
|
|
|
|
|
|
|
cd build
|
|
|
|
|
ninja
|
2020-04-30 08:05:45 +08:00
|
|
|
ninja check-npcomp-opt
|
|
|
|
|
# Note: currently, python tests run separately
|
2020-04-27 07:32:10 +08:00
|
|
|
./python/run_tests.py
|
2020-04-27 06:50:23 +08:00
|
|
|
```
|
|
|
|
|
|
2020-05-03 10:54:49 +08:00
|
|
|
### Things to look at:
|
|
|
|
|
|
|
|
|
|
* `python/npcomp/tracing/mlir_trace_test.py` : Simple test case of tracing a function to an MLIR module.
|
|
|
|
|
|
2020-04-27 06:50:23 +08:00
|
|
|
### Installing pybind11
|
|
|
|
|
|
|
|
|
|
The native extension relies on pybind11. In a perfect world, this could just
|
|
|
|
|
be installed with your system package manager. However, at least on
|
2020-04-27 11:14:20 +08:00
|
|
|
some tested versions of Ubuntu, the system package installed with broken cmake
|
|
|
|
|
files.
|
2020-04-27 06:50:23 +08:00
|
|
|
|
2020-04-27 11:14:20 +08:00
|
|
|
If this happens, you must install pybind11 from source.
|
2020-04-27 06:50:23 +08:00
|
|
|
|
|
|
|
|
### Building the python native library
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
# From the build directory
|
|
|
|
|
ninja NPCOMPNativePyExt
|
2020-04-27 11:14:20 +08:00
|
|
|
ninja check-npcomp
|
|
|
|
|
python3 ./python/run_tests.py
|
|
|
|
|
|
|
|
|
|
# Setup PYTHONPATH for interactive use.
|
2020-04-27 06:50:23 +08:00
|
|
|
export PYTHONPATH=$(pwd)/tools/npcomp/python
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Notes:
|
|
|
|
|
|
|
|
|
|
* Python sources are symlinked to the output directory at configure time.
|
|
|
|
|
Adding sources will require a reconfigure. Editing should not.
|
|
|
|
|
* It is a very common issue to have both python 2.7 (aka. "python") and python
|
|
|
|
|
3.x (aka. "python3") on a system at a time (and we can only hope that one
|
|
|
|
|
day this ends). Since the native library at development time binds to a
|
|
|
|
|
specific version, if you try to run with a different python, you will get
|
|
|
|
|
an error about the "native" module not being found.
|
2020-04-27 11:14:20 +08:00
|
|
|
|
