llvm
/
torch-mlir
mirror of https://github.com/llvm/torch-mlir
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
torch-mlir/lib/InitAll.cpp

56 lines
2.2 KiB
C++
Raw Normal View History

Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
//===----------------------------------------------------------------------===//
//
// This file is licensed under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#include "npcomp/InitAll.h"
Bump llvm-project to c68d2895a1f4019b387c69d1e5eec31b0eb5e7b0 - dialect registration - StringAttr::get: order of context arg - math dialect - LogicalResult nodiscard - error message for invalid broadcast
2021-02-23 04:08:17 +08:00
#include "mlir/IR/Dialect.h"
Add support for compiling through IREE. Recommended review order: - Changes in frontends/pytorch/examples/ - Changes in python/npcomp/compiler/pytorch/backend/ - Boilerplate for the `npcomp-iree-backend-lower-linkage` pass. This change separates out a `npcomp.compiler.pytorch.backend.frontend_lowering` module that does the common lowering for all backends. The individual compiler backends `npcomp.compiler.pytorch.backend.{refjit,iree}` now accept a loosely defined "TCP + scalar code" IR mix that will be formalized in the future as the interface to codegen backends. This also required adding a small pass `npcomp-iree-backend-lower-linkage` which adds `iree.module.export` onto functions, and layering that into the frontend flow. The pass doesn't require a C++-level dependency on IREE, which is nice for now. TBD how we are going to handle lists (we hope we can get away with sneakerneting some td files and relying on loose IR compatibility). Running through IREE requires the ability to import `iree.compiler` and `iree.runtime`, which can be obtained as follows: ``` python3 -m pip install iree-compiler-snapshot iree-runtime-snapshot -f https://github.com/google/iree/releases/tag/snapshot-20210406.200 PYTHONPATH="${PYTHONPATH}:${MY_IREE_BUILD}/bindings/python/" ``` This patch makes it painfully clear that we don't have any e2e testing harness to really plug into, and also don't have a usable Python API to our compiler stack (something usable in a jupyter notebook). That will be addressed in subsequent commits. We've been flying by the seat of our pants with this `examples` directory that isn't subject to any kind of testing or real usability concerns.
2021-04-09 04:05:16 +08:00
#include "npcomp/Backend/IREE/Passes.h"
Bump llvm-project to c68d2895a1f4019b387c69d1e5eec31b0eb5e7b0 - dialect registration - StringAttr::get: order of context arg - math dialect - LogicalResult nodiscard - error message for invalid broadcast
2021-02-23 04:08:17 +08:00
#include "npcomp/Conversion/Passes.h"
NFC: Re-organize ATen directory structure and fix warnings. * Still some more work to do on the Transforms tree to bring it in line with the others (will do that as I add things).
2020-10-23 05:13:26 +08:00
#include "npcomp/Dialect/ATen/IR/ATenDialect.h"
NFC: Transition ATen passes to tablegen registration.
2020-10-23 08:12:01 +08:00
#include "npcomp/Dialect/ATen/Transforms/Passes.h"
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
#include "npcomp/Dialect/Basicpy/IR/BasicpyDialect.h"
#include "npcomp/Dialect/Basicpy/Transforms/Passes.h"
#include "npcomp/Dialect/Numpy/IR/NumpyDialect.h"
Add NumpyPublicFunctionsToTensor pass. * Rewrites public function signatures to operate on tensors (vs ndarray). * Most of our backends presume immutable tensors at public function boundaries.
2020-07-09 13:51:54 +08:00
#include "npcomp/Dialect/Numpy/Transforms/Passes.h"
[RefBackend] Rename RefBackend dialect to Refback I now realize that VerboseCamelCase is not the best choice for dialect directory/file names and C++ identifiers (take e.g. "Linalg", "Basicpy", etc. as prior art here; not LinearAlgebra or BasicPython). If I had to name the convention it seems to be "Shortword" (or of course just acronym dialects like LLVM, SCF, etc.). This rename also has the side benefit of differentiating RefBackend directories, which now refer to the actual backend itself, from Refback/Refbackrt, which are the dialects which happen to be used by that backend.
2020-10-08 08:30:10 +08:00
#include "npcomp/Dialect/Refback/IR/RefbackDialect.h"
#include "npcomp/Dialect/Refbackrt/IR/RefbackrtDialect.h"
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
#include "npcomp/Dialect/TCF/IR/TCFDialect.h"
Allow JITModule passes to be built separately. * Re-introduces frontent/backend split. * Adds a (very) trivial shape refinement pass.
2020-07-11 12:51:03 +08:00
#include "npcomp/Dialect/TCF/Transforms/Passes.h"
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
#include "npcomp/Dialect/TCP/IR/TCPDialect.h"
[RefBackend] Use more idiomatic bufferize pattern for TCP. The time has come for BypassShapes/LowerShapedResultsToMemref to go away :( For the reference backend, being consistent with upstream conventions is the name of the game now. This is a step down in a number of ways, e.g. test clarity and separation of concerns. But it is fewer files and fewer tests, and *does* address the "TODO: This is really fragile". It also eliminates two more ops from the refback dialect (sadly, they are the shaped_results/yield that we were getting kind of fond of, but alas).
2020-10-16 03:26:21 +08:00
#include "npcomp/Dialect/TCP/Transforms/Passes.h"
Add boilerplate for Torch dialect.
2020-09-29 03:02:35 +08:00
#include "npcomp/Dialect/Torch/IR/TorchDialect.h"
Implement GlobalizeObjectGraph transformation. This required restructuring of how we model TorchScript on import. The main difference is that now we split out a `torch.class_type` that holds methods and declarations of the types of each slot. This is more consistent with TorchScript (our previous representation was "denormalized"). Recommended reading order: 1. check out the description of `torch.class_type` in `TorchOps.td` and look at `test/Dialect/Torch/ops.mlir` and `frontends/pytorch/test/module_import/` to familiarize with the new representation. - Just look at the new IR. The diff between the old names and new names is confusing. 2. check out `test/Dialect/Torch/globalize-object-graph*.mlir` and read along with the pass description in `include/npcomp/Dialect/Torch/Transforms/Passes.td` 3. Read the code in `GlobalizeObjectGraph.cpp` and miscellaneous changes in `ivalue_importer.cpp`, `TorchOps.cpp`, etc.
2021-02-18 03:28:51 +08:00
#include "npcomp/Dialect/Torch/Transforms/Passes.h"
[RefBackend] Rename "E2E" to RefBackend.
2020-10-07 07:14:37 +08:00
#include "npcomp/RefBackend/RefBackend.h"
Bump llvm-project to c68d2895a1f4019b387c69d1e5eec31b0eb5e7b0 - dialect registration - StringAttr::get: order of context arg - math dialect - LogicalResult nodiscard - error message for invalid broadcast
2021-02-23 04:08:17 +08:00
#include "npcomp/Typing/Transforms/Passes.h"
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
Bump submodule versions. * llvm-project: b5924a8e27536d19dd5c4d302db29fb6163d5faa * mhlo: 848ca244d20f045b7921da55a98a04d95ef94f0e * Multiple breakages that need to be fixed. Fixes: * Refactor dialect registration * Remove all kindof methods (Casting functionality has been added upstream and is implicitly available, see https://llvm.discourse.group/t/removing-kinds-from-attributes-and-types/1547.) * Update dialect registration to comply with https://reviews.llvm.org/D85495. * Remove type kinds and update some changed dialect signatures. * Upgrade ATen dialect to match upstream needs. * Move dialect registration to tablegen. * Register the ListType in tablegen. * Change dialect initialization signature. * Use TypeSwitch in MlirIr location printer. * Remove global registry depends from npcomp-opt. * Change LowerToLLVM to pass an MLIRContext vs an LLVMDialect for type creation. * Remove dep on MLIREDSCInterface that is removed upstream. * Thread through the DialectRegistry for opt and python-like tools. * Modernize pass registration (This was forced because the GEN_PASS_REGISTRATION code now generates inline functions vs literal pass registration statements) Co-authored-by: Marius Brehler <marius.brehler@iml.fraunhofer.de>
2020-08-28 06:09:10 +08:00
void mlir::NPCOMP::registerAllDialects(mlir::DialectRegistry &registry) {
// clang-format off
registry.insert<mlir::NPCOMP::aten::ATenDialect,
Basicpy::BasicpyDialect,
Numpy::NumpyDialect,
[RefBackend] Rename Npcomprt dialect to Refbackrt.
2020-10-08 08:12:52 +08:00
refbackrt::RefbackrtDialect,
[RefBackend] Rename RefBackend dialect to Refback I now realize that VerboseCamelCase is not the best choice for dialect directory/file names and C++ identifiers (take e.g. "Linalg", "Basicpy", etc. as prior art here; not LinearAlgebra or BasicPython). If I had to name the convention it seems to be "Shortword" (or of course just acronym dialects like LLVM, SCF, etc.). This rename also has the side benefit of differentiating RefBackend directories, which now refer to the actual backend itself, from Refback/Refbackrt, which are the dialects which happen to be used by that backend.
2020-10-08 08:30:10 +08:00
refback::RefbackDialect,
Bump submodule versions. * llvm-project: b5924a8e27536d19dd5c4d302db29fb6163d5faa * mhlo: 848ca244d20f045b7921da55a98a04d95ef94f0e * Multiple breakages that need to be fixed. Fixes: * Refactor dialect registration * Remove all kindof methods (Casting functionality has been added upstream and is implicitly available, see https://llvm.discourse.group/t/removing-kinds-from-attributes-and-types/1547.) * Update dialect registration to comply with https://reviews.llvm.org/D85495. * Remove type kinds and update some changed dialect signatures. * Upgrade ATen dialect to match upstream needs. * Move dialect registration to tablegen. * Register the ListType in tablegen. * Change dialect initialization signature. * Use TypeSwitch in MlirIr location printer. * Remove global registry depends from npcomp-opt. * Change LowerToLLVM to pass an MLIRContext vs an LLVMDialect for type creation. * Remove dep on MLIREDSCInterface that is removed upstream. * Thread through the DialectRegistry for opt and python-like tools. * Modernize pass registration (This was forced because the GEN_PASS_REGISTRATION code now generates inline functions vs literal pass registration statements) Co-authored-by: Marius Brehler <marius.brehler@iml.fraunhofer.de>
2020-08-28 06:09:10 +08:00
tcf::TCFDialect,
Add boilerplate for Torch dialect.
2020-09-29 03:02:35 +08:00
tcp::TCPDialect,
mlir::NPCOMP::Torch::TorchDialect>();
Bump submodule versions. * llvm-project: b5924a8e27536d19dd5c4d302db29fb6163d5faa * mhlo: 848ca244d20f045b7921da55a98a04d95ef94f0e * Multiple breakages that need to be fixed. Fixes: * Refactor dialect registration * Remove all kindof methods (Casting functionality has been added upstream and is implicitly available, see https://llvm.discourse.group/t/removing-kinds-from-attributes-and-types/1547.) * Update dialect registration to comply with https://reviews.llvm.org/D85495. * Remove type kinds and update some changed dialect signatures. * Upgrade ATen dialect to match upstream needs. * Move dialect registration to tablegen. * Register the ListType in tablegen. * Change dialect initialization signature. * Use TypeSwitch in MlirIr location printer. * Remove global registry depends from npcomp-opt. * Change LowerToLLVM to pass an MLIRContext vs an LLVMDialect for type creation. * Remove dep on MLIREDSCInterface that is removed upstream. * Thread through the DialectRegistry for opt and python-like tools. * Modernize pass registration (This was forced because the GEN_PASS_REGISTRATION code now generates inline functions vs literal pass registration statements) Co-authored-by: Marius Brehler <marius.brehler@iml.fraunhofer.de>
2020-08-28 06:09:10 +08:00
// clang-format on
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
}
void mlir::NPCOMP::registerAllPasses() {
NFC: Transition ATen passes to tablegen registration.
2020-10-23 08:12:01 +08:00
mlir::NPCOMP::registerATenPasses();
[RefBackend] Rename "E2E" to RefBackend.
2020-10-07 07:14:37 +08:00
mlir::NPCOMP::registerRefBackendPasses();
Bump submodule versions. * llvm-project: b5924a8e27536d19dd5c4d302db29fb6163d5faa * mhlo: 848ca244d20f045b7921da55a98a04d95ef94f0e * Multiple breakages that need to be fixed. Fixes: * Refactor dialect registration * Remove all kindof methods (Casting functionality has been added upstream and is implicitly available, see https://llvm.discourse.group/t/removing-kinds-from-attributes-and-types/1547.) * Update dialect registration to comply with https://reviews.llvm.org/D85495. * Remove type kinds and update some changed dialect signatures. * Upgrade ATen dialect to match upstream needs. * Move dialect registration to tablegen. * Register the ListType in tablegen. * Change dialect initialization signature. * Use TypeSwitch in MlirIr location printer. * Remove global registry depends from npcomp-opt. * Change LowerToLLVM to pass an MLIRContext vs an LLVMDialect for type creation. * Remove dep on MLIREDSCInterface that is removed upstream. * Thread through the DialectRegistry for opt and python-like tools. * Modernize pass registration (This was forced because the GEN_PASS_REGISTRATION code now generates inline functions vs literal pass registration statements) Co-authored-by: Marius Brehler <marius.brehler@iml.fraunhofer.de>
2020-08-28 06:09:10 +08:00
mlir::NPCOMP::registerConversionPasses();
mlir::NPCOMP::registerBasicpyPasses();
mlir::NPCOMP::registerNumpyPasses();
mlir::NPCOMP::registerTCFPasses();
[RefBackend] Use more idiomatic bufferize pattern for TCP. The time has come for BypassShapes/LowerShapedResultsToMemref to go away :( For the reference backend, being consistent with upstream conventions is the name of the game now. This is a step down in a number of ways, e.g. test clarity and separation of concerns. But it is fewer files and fewer tests, and *does* address the "TODO: This is really fragile". It also eliminates two more ops from the refback dialect (sadly, they are the shaped_results/yield that we were getting kind of fond of, but alas).
2020-10-16 03:26:21 +08:00
mlir::NPCOMP::registerTCPPasses();
Implement GlobalizeObjectGraph transformation. This required restructuring of how we model TorchScript on import. The main difference is that now we split out a `torch.class_type` that holds methods and declarations of the types of each slot. This is more consistent with TorchScript (our previous representation was "denormalized"). Recommended reading order: 1. check out the description of `torch.class_type` in `TorchOps.td` and look at `test/Dialect/Torch/ops.mlir` and `frontends/pytorch/test/module_import/` to familiarize with the new representation. - Just look at the new IR. The diff between the old names and new names is confusing. 2. check out `test/Dialect/Torch/globalize-object-graph*.mlir` and read along with the pass description in `include/npcomp/Dialect/Torch/Transforms/Passes.td` 3. Read the code in `GlobalizeObjectGraph.cpp` and miscellaneous changes in `ivalue_importer.cpp`, `TorchOps.cpp`, etc.
2021-02-18 03:28:51 +08:00
mlir::NPCOMP::registerTorchPasses();
Bump submodule versions. * llvm-project: b5924a8e27536d19dd5c4d302db29fb6163d5faa * mhlo: 848ca244d20f045b7921da55a98a04d95ef94f0e * Multiple breakages that need to be fixed. Fixes: * Refactor dialect registration * Remove all kindof methods (Casting functionality has been added upstream and is implicitly available, see https://llvm.discourse.group/t/removing-kinds-from-attributes-and-types/1547.) * Update dialect registration to comply with https://reviews.llvm.org/D85495. * Remove type kinds and update some changed dialect signatures. * Upgrade ATen dialect to match upstream needs. * Move dialect registration to tablegen. * Register the ListType in tablegen. * Change dialect initialization signature. * Use TypeSwitch in MlirIr location printer. * Remove global registry depends from npcomp-opt. * Change LowerToLLVM to pass an MLIRContext vs an LLVMDialect for type creation. * Remove dep on MLIREDSCInterface that is removed upstream. * Thread through the DialectRegistry for opt and python-like tools. * Modernize pass registration (This was forced because the GEN_PASS_REGISTRATION code now generates inline functions vs literal pass registration statements) Co-authored-by: Marius Brehler <marius.brehler@iml.fraunhofer.de>
2020-08-28 06:09:10 +08:00
mlir::NPCOMP::registerTypingPasses();
Add support for compiling through IREE. Recommended review order: - Changes in frontends/pytorch/examples/ - Changes in python/npcomp/compiler/pytorch/backend/ - Boilerplate for the `npcomp-iree-backend-lower-linkage` pass. This change separates out a `npcomp.compiler.pytorch.backend.frontend_lowering` module that does the common lowering for all backends. The individual compiler backends `npcomp.compiler.pytorch.backend.{refjit,iree}` now accept a loosely defined "TCP + scalar code" IR mix that will be formalized in the future as the interface to codegen backends. This also required adding a small pass `npcomp-iree-backend-lower-linkage` which adds `iree.module.export` onto functions, and layering that into the frontend flow. The pass doesn't require a C++-level dependency on IREE, which is nice for now. TBD how we are going to handle lists (we hope we can get away with sneakerneting some td files and relying on loose IR compatibility). Running through IREE requires the ability to import `iree.compiler` and `iree.runtime`, which can be obtained as follows: ``` python3 -m pip install iree-compiler-snapshot iree-runtime-snapshot -f https://github.com/google/iree/releases/tag/snapshot-20210406.200 PYTHONPATH="${PYTHONPATH}:${MY_IREE_BUILD}/bindings/python/" ``` This patch makes it painfully clear that we don't have any e2e testing harness to really plug into, and also don't have a usable Python API to our compiler stack (something usable in a jupyter notebook). That will be addressed in subsequent commits. We've been flying by the seat of our pants with this `examples` directory that isn't subject to any kind of testing or real usability concerns.
2021-04-09 04:05:16 +08:00
mlir::NPCOMP::IREEBackend::registerIREEBackendPasses();
Make a real library for InitAll and extend it to conditionally initialize dependencies. * Conditioned on the top level CMake option to enable IREE. * There is still some warning flags and such that need triage, but it does build/work.
2020-06-12 08:47:14 +08:00
}