llvm
/
torch-mlir
mirror of https://github.com/llvm/torch-mlir
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
torch-mlir/lib/CMakeLists.txt

62 lines
1.4 KiB
CMake
Raw Normal View History

Add public API for constructing a module/function to capture PyTorch ops. * Uses the MLIR-C API since that will save us a lot of grief down the road (i.e. will give PyTorch and libMLIR/libNPCOMP the ability to skew version-wise). * Quite a few TODOs and not yet populating the function in any way.
2020-09-29 09:36:00 +08:00
add_subdirectory(CAPI)
Initial TCF/TCP E2E seed. Very much WIP. This is enough to get tcf.add down to approximately the "linalg.generic on buffers" level of abstraction. (but there are nuances)
2020-05-07 09:41:54 +08:00
add_subdirectory(Conversion)
Add stub numpy dialect.
2020-04-27 08:20:58 +08:00
add_subdirectory(Dialect)
Initial TCF/TCP E2E seed. Very much WIP. This is enough to get tcf.add down to approximately the "linalg.generic on buffers" level of abstraction. (but there are nuances)
2020-05-07 09:41:54 +08:00
add_subdirectory(E2E)
Rework e2e flow to use new "npcomprt" This ~totally reworks the existing "runtime" stuff to be more principled and usable, such as from Python. It's still not fully production-quality, mainly in the department of memory management (e.g. it currently leaks memory; we need to figure out "who frees memrefs" + the analysis and transformation needed to do that (maybe use upstream buffer allocation pass?)). The user API is in include/npcomp/runtime/UserAPI.h, though include/npcomp/JITRuntime/JITModule.h is a friendlier wrapper. The stuff under {include,lib}/runtime is totally firewalled from the compiler and tiny (<6kB, though no attention has gone into optimizing that size). For example, we don't link in libSupport into the runtime, instead having our own bare bones replacements for basics like ArrayRef (the JITRuntime helps with bridging that gap, since it *can* depend on all common LLVM utilities). The overall features of npcomprt is that it exposes a module that with multiple function entry points. Each function has arguments and results that are tensor-valued, and npcomprt::Tensor is the runtime type that is used to interact with that (and a npcomprt::Ref<T> reference-counting wrapper is provided to wrap npcomprt::Tensor in the common case). From an implementation perspective, an npcomprt module at the LLVM/object/binary level exposes a single module descriptor struct that has pointers to other metadata (currently just a list of function metadata descriptors). All interactions with the npcomp runtime are keyed off of that module descriptor, including function lookups and dispatching. This is done to dodge platform ABI issues and also allow enough reflection to e.g. verify provided arguments. Most of the compiler-side work here was in LowerToNpcomprtABI and LowerToLLVM. Also, - Rename npcomp_rt/NpcompRt to npcomprt/Npcomprt; it was getting annoying to type the underscores/caps. - misc improvements to bash_helpers.sh
2020-07-09 08:15:40 +08:00
add_subdirectory(JITRuntime)
Move most python sources to the include and lib tree.
2020-06-19 09:02:39 +08:00
add_subdirectory(Python)
Add some support classes for implementing a CPA type inference algorithm.
2020-07-01 09:28:39 +08:00
add_subdirectory(Typing)
Rework e2e flow to use new "npcomprt" This ~totally reworks the existing "runtime" stuff to be more principled and usable, such as from Python. It's still not fully production-quality, mainly in the department of memory management (e.g. it currently leaks memory; we need to figure out "who frees memrefs" + the analysis and transformation needed to do that (maybe use upstream buffer allocation pass?)). The user API is in include/npcomp/runtime/UserAPI.h, though include/npcomp/JITRuntime/JITModule.h is a friendlier wrapper. The stuff under {include,lib}/runtime is totally firewalled from the compiler and tiny (<6kB, though no attention has gone into optimizing that size). For example, we don't link in libSupport into the runtime, instead having our own bare bones replacements for basics like ArrayRef (the JITRuntime helps with bridging that gap, since it *can* depend on all common LLVM utilities). The overall features of npcomprt is that it exposes a module that with multiple function entry points. Each function has arguments and results that are tensor-valued, and npcomprt::Tensor is the runtime type that is used to interact with that (and a npcomprt::Ref<T> reference-counting wrapper is provided to wrap npcomprt::Tensor in the common case). From an implementation perspective, an npcomprt module at the LLVM/object/binary level exposes a single module descriptor struct that has pointers to other metadata (currently just a list of function metadata descriptors). All interactions with the npcomp runtime are keyed off of that module descriptor, including function lookups and dispatching. This is done to dodge platform ABI issues and also allow enough reflection to e.g. verify provided arguments. Most of the compiler-side work here was in LowerToNpcomprtABI and LowerToLLVM. Also, - Rename npcomp_rt/NpcompRt to npcomprt/Npcomprt; it was getting annoying to type the underscores/caps. - misc improvements to bash_helpers.sh
2020-07-09 08:15:40 +08:00
add_subdirectory(runtime)
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
Initial python plumbing to interface with the refjit backend.
2020-07-11 08:36:32 +08:00
if(NPCOMP_ENABLE_REFJIT)
add_subdirectory(Backend/RefJIT)
endif()
Add _npcomp.backend.iree module. * Populates with builders for the various path pipelines and translator.
2020-06-19 14:28:30 +08:00
if(NPCOMP_ENABLE_IREE)
add_subdirectory(Backend/IREE)
endif()
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
################################################################################
# Setup the initialization target.
# This includes conditional dependencies based on whether features are enabled.
################################################################################
set(ALL_DEPENDS)
if(NPCOMP_ENABLE_IREE)
list(APPEND ALL_DEPENDS
iree_tools_init_compiler_modules
iree_tools_init_iree_passes_and_dialects
iree_tools_init_mlir_passes_and_dialects
iree_tools_init_targets
iree_tools_init_xla_dialects
)
endif()
[cmake] Updates for basic shared library support (#7) Mostly this is CMake cleanup. Several library dependencies are missing, which is often revealed with shared library builds. Also, it's generally bad to link directly against LLVM libraries because it fails when using LLVM_LINK_LLVM_DYLIB. MLIR will pull in libLLVM.so, and there will be duplicate linkage with the the explicit libraries. There may need to be more refactoring here.
2020-08-06 05:49:18 +08:00
get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
get_property(conversion_libs GLOBAL PROPERTY MLIR_CONVERSION_LIBS)
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
add_mlir_library(NPCOMPInitAll
InitAll.cpp
LINK_LIBS
Bump llvm and IREE version to revisions circa 2020/7/29. * Also fixes a dependency issue that was causing a build race.
2020-07-01 02:22:30 +08:00
PUBLIC
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
# Local depends
NPCOMPE2E
[RefBackend] Split out RefBackend (refback) dialect from TCP. This is the first in a patch series that is refactoring the constellation of things variously called or associated with "E2E", "RefE2E", "npcomprt", and "TCP" into a more cleanly layered result. Concretely, this first patch fixes the fact that TCP was basically acting like a dumping ground needed by the reference backend. This splits it out, which is fairly mechanical, but touches a lot of lines of code (basically replacing `tcp` with `refback` and `TCP` with `RefBackend). Now, the RefBackend dialect is that dumping ground, which is slighly better, as it starts allowing TCP to become a nice clean middle layer that is not related per se to the reference backend. The previous name RefE2E or "reference e2e flow" was super confusing. Now that we are seeing more clearly where the "backend" distinction lies, the [RefBackend] commit tag is born :)
2020-10-07 06:44:18 +08:00
NPCOMPRefBackendDialect
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
NPCOMPTCP
NPCOMPTCF
Add boilerplate for Torch dialect.
2020-09-29 03:02:35 +08:00
NPCOMPTorchDialect
Rework e2e flow to use new "npcomprt" This ~totally reworks the existing "runtime" stuff to be more principled and usable, such as from Python. It's still not fully production-quality, mainly in the department of memory management (e.g. it currently leaks memory; we need to figure out "who frees memrefs" + the analysis and transformation needed to do that (maybe use upstream buffer allocation pass?)). The user API is in include/npcomp/runtime/UserAPI.h, though include/npcomp/JITRuntime/JITModule.h is a friendlier wrapper. The stuff under {include,lib}/runtime is totally firewalled from the compiler and tiny (<6kB, though no attention has gone into optimizing that size). For example, we don't link in libSupport into the runtime, instead having our own bare bones replacements for basics like ArrayRef (the JITRuntime helps with bridging that gap, since it *can* depend on all common LLVM utilities). The overall features of npcomprt is that it exposes a module that with multiple function entry points. Each function has arguments and results that are tensor-valued, and npcomprt::Tensor is the runtime type that is used to interact with that (and a npcomprt::Ref<T> reference-counting wrapper is provided to wrap npcomprt::Tensor in the common case). From an implementation perspective, an npcomprt module at the LLVM/object/binary level exposes a single module descriptor struct that has pointers to other metadata (currently just a list of function metadata descriptors). All interactions with the npcomp runtime are keyed off of that module descriptor, including function lookups and dispatching. This is done to dodge platform ABI issues and also allow enough reflection to e.g. verify provided arguments. Most of the compiler-side work here was in LowerToNpcomprtABI and LowerToLLVM. Also, - Rename npcomp_rt/NpcompRt to npcomprt/Npcomprt; it was getting annoying to type the underscores/caps. - misc improvements to bash_helpers.sh
2020-07-09 08:15:40 +08:00
NPCOMPNpcomprt
Add ATen Dialect (#16) This patch adds a dialect intended to be used as a frontend dialect to facilitate lowering from "A Tensor Library" in torch/pytorch. This patch includes several passes that are useful in conjuction with the dialect: --aten-layer-name: Generates layer names for each operation, which are not present in the original pytorch. --aten-to-std: Lower the ATen dialect into standard dialect function calls. --return-elimination-pass: convert functions (primarily the toplevel function) to pass return values by reference. This simplifies pytorch integration. --aten-op-report: generate a textual report about the model --liveness-report Future patches will implement actual integration with the pytorch jit to intercept and generates MLIR in this dialect, then lower the resulting MLIR into function calls through aten-layer-name -> aten-to-std -> return-elimination -> std-to-llvm. The result would then jitted using the LLVM jit, linked against a runtime library which makes calls back into pytorch to implement all the layers. Co-authored-by: Jeff Fifield <jeff.fifield@xilinx.com> Co-authored-by: Jeff Fifield <jeff.fifield@xilinx.com>
2020-08-13 10:28:04 +08:00
NPCOMPATenDialect
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
NPCOMPBasicpyDialect
NPCOMPBasicpyPasses
Bump submodule versions. * llvm-project: b5924a8e27536d19dd5c4d302db29fb6163d5faa * mhlo: 848ca244d20f045b7921da55a98a04d95ef94f0e * Multiple breakages that need to be fixed. Fixes: * Refactor dialect registration * Remove all kindof methods (Casting functionality has been added upstream and is implicitly available, see https://llvm.discourse.group/t/removing-kinds-from-attributes-and-types/1547.) * Update dialect registration to comply with https://reviews.llvm.org/D85495. * Remove type kinds and update some changed dialect signatures. * Upgrade ATen dialect to match upstream needs. * Move dialect registration to tablegen. * Register the ListType in tablegen. * Change dialect initialization signature. * Use TypeSwitch in MlirIr location printer. * Remove global registry depends from npcomp-opt. * Change LowerToLLVM to pass an MLIRContext vs an LLVMDialect for type creation. * Remove dep on MLIREDSCInterface that is removed upstream. * Thread through the DialectRegistry for opt and python-like tools. * Modernize pass registration (This was forced because the GEN_PASS_REGISTRATION code now generates inline functions vs literal pass registration statements) Co-authored-by: Marius Brehler <marius.brehler@iml.fraunhofer.de>
2020-08-28 06:09:10 +08:00
NPCOMPConversionPasses
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
NPCOMPNumpyDialect
Add NumpyPublicFunctionsToTensor pass. * Rewrites public function signatures to operate on tensors (vs ndarray). * Most of our backends presume immutable tensors at public function boundaries.
2020-07-09 13:51:54 +08:00
NPCOMPNumpyPasses
Allow JITModule passes to be built separately. * Re-introduces frontent/backend split. * Adds a (very) trivial shape refinement pass.
2020-07-11 12:51:03 +08:00
NPCOMPTCFPasses
NFC: Move CPATypeInference to Typing directory.
2020-07-05 07:56:09 +08:00
NPCOMPTypingPasses
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
[cmake] Updates for basic shared library support (#7) Mostly this is CMake cleanup. Several library dependencies are missing, which is often revealed with shared library builds. Also, it's generally bad to link directly against LLVM libraries because it fails when using LLVM_LINK_LLVM_DYLIB. MLIR will pull in libLLVM.so, and there will be duplicate linkage with the the explicit libraries. There may need to be more refactoring here.
2020-08-06 05:49:18 +08:00
${dialect_libs}
${conversion_libs}
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
${ALL_DEPENDS}
)