Software:

Collaborations:

Iterative compilation

• for large applications
• for small kernels
• using global flags
• using fine grain (function, loop or instruction level) optimizations
• using compiler optimization phases (passes)
• for PathScale/Open64/ORC compilers
• for GCC
• for Intel, IBM, SUN compilers
• for different architectures including Intel, AMD, Alpha, SPARC, PowerPC, CELL, embedded systems, etc
• with multiple search strategies
• with statistical selection of optimizations
• with machine learning to reuse optimization knowledge

I have been working on iterative compilation with Michael O'Boyle et. al. since 1999 and have developed various search techniques for different compilers (PathScale, Intel, Open64, gcc) and on different optimization levels (global/function/loop/instruction). Lately, I have been working with Olivier Temam to make iterative optimizations practical (run-time adaptation for different datasets, speeding up interative search, program characterization using reactions to transformations, etc).

I have been developing FCO framework for PathScale/Open64 compilers at INRIA Saclay for practical iterative compilation and adaptive run-time continuous optimizations.

[ FCO developments ]

I had been developing EOS framework for my Ph.D. at the University of Edinburgh(UK) during 1999-2005. This framework allows experiments on iterative compilation (program optimizations) and performance prediction for various platforms and languages. I had reasonably stable versions for Intel Itanium2 (Linux), IBM PowerPC (AIX), Intel x86 (Windows & Linux), Sun Sparc (Solaris), Compaq Alpha (Unix). This software had been used in several projects or for teaching purposes. Now, I am working on the new Continuous Collective Compilation framework in collaboration with several universities and companies.

[ older EOS developments ]

IBM, STMicroelectronics, NXP (Philips), ARC, CAPS Enterprise, INRIA Saclay, University of Edinburgh, IRISA, UPC, UIUC, others

Used in MILEPOST project, SARC project, HiPEAC network of excellence

Interactive Compilation Interface (ICI)

• for GCC [ GCC ICI development website ]
• for Open64 compiler [ Open64 ICI website ]
• for QLogic PathScale EKOPath compilers [ PathScale ICI website ]
• for Microsoft Phoenix compiler
• for IBM J7 / Jikes JIT

Based on my previous research on iterative compilation, I started developing anInteractive Compilation Interface for PathScale EKOPath compiler in 2003/2004 to improve its optimization heuristic, enable fine-grain program optimizations and unify optimization knowledge reuse among different programs and architectures using statistical search and machine learning techniques. Recently, this project got support from several universities and companies, and I currently lead ICI R&D for GCC with the help of a HiPEAC engineer. We expect that using ICI will help turning GCC (and other compilers) into component-based pluggable compiler with easily extendable and automatically tunable optimization heuristic, particularly in the presence of rapidly evolving multi-core heterogeneous architectures.

[ More information ]

IBM, STMicroelectronics, NXP (Philips), ARC, CAPS Enterprise, INRIA Saclay, University of Edinburgh, IRISA, UPC, UIUC, others

Used in MILEPOST project, SARC project, HiPEAC network of excellence

Multiple datasets

• MiDatasets for MiBench benchmark [ MiDatasets development website ]
• for SPEC 2006

Iterative optimization is now a popular technique to obtain performance or code size improvements over the default settings in a compiler. However, in most of the research projects, the best configuration is found for one arbitrary dataset and it is assumed that this configuration will work well with any other dataset that a program uses. Here we evaluate this assumption and analyze the behavior of various programs with multiple datasets. MiDataSets should enable research on realistic program optimization, run-time adaptation and benchmarking.

[ More information ]

Statistical and Machine Learning techniques to speed up iterative compilation

• for GCC [ development website coming soon ...]

Soon after starting research on iterative optimizations in 1999, I realized that current search methods are too slow in large non-linear optimization space and new techniques are needed to prune it. Since I had some experience with neural networks and machine learning during my undergraduate and M.S. studies, I wanted to apply some of these techniques to compiler optimizations to quickly capture main features and behavior of optimization spaces, speed up the search process and enable optimization knowledge reuse. Lacking manpower ;) we could only proceed with this project in 2003 after teaming up with our colleagues from Machine Learning Group from the Institute for Adaptive and Neural Computation (University of Edinburgh). We currently evaluate different statistical search and machine learning techniques to analyze and narrow down large optimizations spaces considerably and automatically learn best optimization settings among different programs and architectures using static and dynamic features.

I believe that using statistical and machine learning techniques is one of the only ways to tackle the growing complexity of architectures, applications and optimizing compilers in the future.

[ More information ]

INRIA Saclay, University of Edinburgh

Used in MILEPOST project, COLO project

Run-time adaptation

• for statically compiled programs
  • on homogeneous systems
    • for GCC (and source-to-source) [ UNIDAPT development website ]
    • for PathScale/Open64 Compiler [ within previous FCO developments ]
    • for Microsoft Phoenix
  • on heterogeneous systems
    • for GCC (and source-to-source) [ UNIDAPT development website ]
    • for IBM XL compilers [ UNIDAPT development website ]
• for VMs with JIT
  • for IBM J7/Jikes
  • for .NET

Since 2004 I have been working with Prof. Olivier Temam to make iterative optimizations practical and developed a technique that speeded up the iterative search by several orders of magnitude and enabled low-overhead run-time adaptation for varying program behaviour using multiversioning and phase detection for statically compiled programs (different from SimPoints as we detect phases at run-time at function or loop-level to target complex high-level optimizations). I currently extend this technique to speed up iterative optimizations and enable run-time adaptability for statically compiled programs with any behavior (regular or irregular) for different constraints (performance, power consumption) without need for complex run-time recompilation frameworks. I am implementing this technique in GCC (or as a stand-alone source-to-source tool).

[ More information ]

INRIA Saclay, University of Edinburgh

Used in MILEPOST project

Planned to be used in different projects

Continuous Collective Compilation Framework with reuse of optimization knowledge among different programs and architectures using machine learning and statistical techniques

• for GCC [ CCC development website ]
• for PathScale [ within previous FCO developments ]

Based on my previous research on iterative compilation (global and fine-grain levels), run-time adaptation, statistical search and machine learning, I currently lead a research and development for a continuous optimization framework for GCC, Open64 and other compilers.

[ More information ]

INRIA Saclay

Miscellaneous

• Automating experiments for iterative compilation

  Iterative compilation and machine learning requires executing thousands of experiments using different programs, multiple datasets and different architectures. We had to automate the process and to slightly modify benchmarks (SPEC95, SPEC2000, SPEC2006, MiBench) to perform experiments using PathScale compiler, GCC, Intel compilers, IBM compilers, SUN compilers, etc and all our tools.

  [ previous FCO developments ]

  [ previous EOS developments ]

  However, our recent developments on iterative optimizations, ICI and machine learning for GCC do not require any benchmark changes to enable transparent continuous collective optimizations.

• Memory upper-bound performance prediction tool (it was not officially released due to a lack of manpower during my PhD;) )
  • used to guide iterative optimizations
  • used to quickly calculate prefetch distances at run-time
  • used to dynamically instrument all program memory instructions and enable self-modifcation/self-removal of these instructions to quickly collect and analyze memory access traces.

    More information can be found in my PhD thesis, CPC'01 and CPC'04 papers.

    [ previous EOS developments ]

• Static and dynamic program instrumentation tool (it was not officially released due to a lack of manpower during my PhD;) )

When I started working on iterative compilation in 1999, I worked a lot with Alpha platform and used the ATOM tool for my research on performance prediction to dynamically instrument programs. Gradually I moved to Intel and AMD platforms, and in 2001 started developing a new tool for dynamic program instrumentation similar to ATOM that could work both on Linux and Windows. I used it for my research on memory upper-bound performance prediction and for calculating prefetching distances. I also used this tool to dynamically instrument all program memory instructions and enable self-modification/self-removal of these instructions to quickly collect and analyze memory access traces. I didn't have enough time during PhD to finish developments and recently found a new tool called Pin that performs dynamic instrumentation of programs on Intel platforms. It is well supported, so I stopped my developments and suggested my colleagues who used my dynamic instrumentation tool, to gradually move to Pin tool.

[ Pin website ]

[ previous EOS developments ]

Used for teaching and research