1b48d0d80b
The idea was half-baked and after some deep thought felt like a solution looking for a problem. What we had here (and is removed in this patch) just wasn't pulling its weight. I cannot think of anything we would want to do with tcp.island as it is removed here beyond just sinking and merging them within a basic block, such that the witness argument is kind of pointless (only matters for hoisting). TCP compute ops like tcp.add and tcp.broadcast_to have the strong invariant of "pure or undefined behavior", which means they are always safe to sink. The island concept as removed here conferred no benefit. Also, I'll note that "islands" are a trick you can only play once in a system (unless they strictly nest). I have some early-stage thoughs on having an island concept that helps with modeling tensor shapes robustly which seems promising (the island would serve a similar role as tie_shape). |
||
|---|---|---|
| include/npcomp | ||
| lib | ||
| python | ||
| python_native | ||
| test | ||
| tools | ||
| .clang-format | ||
| .gitignore | ||
| CMakeLists.txt | ||
| LICENSE | ||
| README.md | ||
| contributing.md | ||
README.md
npcomp - An aspirational MLIR based numpy compiler
This is a research prototype of MLIR dialects for representing numpy programs, and a set of reference tracing/compiler tools. The primary purpose at this point is to establish a solid modeling of restricted Python programs and Numpy based computations in MLIR. While this project will provide some reference implementations to prove the design, the intention is to align this with the broader set of tools that exist at this level of abstraction.
Design Notes
As I work through things, I've been jotting down some design notes:
- Type Extraction - April 15, 2020
- Ufunc modeling Part 1 - April 29, 2020
- Array funcs and op granularity - May 5, 2020
Quick start
LLVM_VERSION=10
export CC=clang-$LLVM_VERSION
export CXX=clang++-$LLVM_VERSION
export LDFLAGS=-fuse-ld=$(which ld.lld-$LLVM_VERSION)
export LLVM_SRC_DIR=/path/to/llvm-project
# Check out last known good commit.
(cd $LLVM_SRC_DIR && git checkout 310d32cb80a611e6384a921e85607fea05841f26)
./tools/install_mlir.sh
./tools/cmake_configure.sh
# ./tools/test_all.sh runs all of these commands.
cd build
ninja
ninja check-npcomp-opt
# Note: currently, python tests run separately
./python/run_tests.py
Interactive Use
The cmake configuration populates symlinks in the build/python directory
mirroring the source layout. This allows edit-run without rebuilding (unless
if files are added/removed).
Configuring the PYTHONPATH should be sufficient to run any interactive
tooling (python3, Jupyter/Colab, etc).
export PYTHONPATH="$(realpath build/python):$(realpath build/python_native)"
The run_tests.py script is special in that it sets up the PYTHONPATH
correctly when run.
Note that running the cmake_configure.sh script will also output a .env
file in the workspace folder with the correct PYTHONPATH set. This allows
tools like VSCode to work by default for debugging.
Things to look at:
python/npcomp/tracing/mlir_trace_test.py: Simple test case of tracing a function to an MLIR module.
Notes:
- Python sources are symlinked to the output directory at configure time. Adding sources will require a reconfigure. Editing should not.
- It is a very common issue to have both python 2.7 (aka. "python") and python 3.x (aka. "python3") on a system at a time (and we can only hope that one day this ends). Since the native library at development time binds to a specific version, if you try to run with a different python, you will get an error about the "native" module not being found.