CACHE

CRITICAL ASSESSMENT OF COMPUTATIONAL HIT-FINDING EXPERIMENTS

DONATE

  • About
    • WHAT IS CACHE
    • Conferences
  • CACHE News
  • CHALLENGES
    • Challenge #1
      • Announcement
      • Computation methods
      • Preliminary results
      • Final results
    • Challenge #2
      • Announcement
      • Computation methods
      • Preliminary results
      • Final Results
    • Challenge #3
      • Announcement
      • Computation methods
      • Preliminary results
      • Final Results
    • Challenge #4
      • Announcement
      • Computation methods
      • Preliminary results
    • Challenge #5
      • Announcement
      • Computation methods
    • Challenge #6
      • Announcement
    • FAQ
  • PUBLICATIONS
  • CONTACT

Challenge #2

Hit Identification
Method type (check all that applies)
De novo design
Deep learning
Machine learning
Physics-based
Description of your approach (min 200 and max 800 words)

We present an end-to-end lead optimization system for discovery based on an AI-gym environment called ``Reinforcement Learning for Molecular Modeling" (RLMM). RLMM automates running fully customizable molecular dynamic simulations inside of an agent-based molecular design protocol. RLMM is fully autonomous---from a single starting ligand, protein structure, and configuration file, RLMM cycles through designs for lead optimization informed by physics-based simulations. RLMM connects various state-of-the-art molecular dynamics simulations with an agent-based policy environment. We outline the basis of the molecular dynamic’s simulations utilized in RLMM and describe the methods for navigating chemical space in an agent-based model.

 

Computational tools for assessing the binding affinity for a ligand generally rely on molecular dynamics simulations. Given the aim to connect machine learning with physics-based modeling, we focus on employing physics-based simulations of protein-ligand complexes. Various software packages exist with an API for creating, running, and analyzing molecular dynamics simulations Standard molecular dynamics models. Molecular dynamics simulations are widely used to estimate the binding affinity of a proten-ligand complex computationally. Advanced sampling techniques are also used in molecular dynamics simulations such Markov Chain Monte Carlo (MCMC) or replica exchange. Molecular mechanics generalized borne surface area (MMGBSA) is a technique for estimating the binding affinity of a protein-ligand complex. MMGBSA methods are less computationally expensive than free energy estimations. Free energy estimation software packages utilize more complicated achemical techniques. We utilize the MMGBSA.py script from Amber20 to estimate the MMGBSA scores for a series of molecular dynamics snapshots.

 

RLMM is comprised of five general components that make up the backbone of the platform: system building, simulation setup, action space, observation space, and policy. Each of the five components consists of sub-modules with unique properties and behaviors. The connection between modules is provided by RLMM. The general workflow following initial system preparation, simulation, observations drawn from simulation sample, determination of action space (ligand design), AI-policy chooses modification, and the system is rebuilt and re-initialized to continue the simulation. Molecular modifications are small so that systems can be re-initialized without as much warm-up time.

 

System-building

Typically for lead optimization tasks, tautomers and enantiomers are enumerated for the incoming proposed analog or perturbation to the previous ligand. Conformer generation is performed on the ensemble of structures, generating 200-800 3D conformers for every enantiomer and reasonable tautomer generated. The conformer and placement of the ligand is selected based on the best shape overlay to the previous ligand. We utilize this system preparation method for lead optimization to mimic and interrupted simulation, where the start of the new simulation matches the end of the previous simulation as closely as possible.

 

Action-space

The action space abstraction in RLMM defines the space, or domain, of available actions available to the policy module. These actions define the transition from state to state in RLMM. To illustrate the strengths of this abstraction, we provide three implemented action spaces, with more robust formulations for synthetic chemistry restrictions coming. In principle, the action space formulation will allow for a robotic laboratory based action space, calculating possible reactions given a set of known reactions and in-stock reagents. During lead optimization, the goal is to modify a ligand to something similar with more desirable properties such as stronger binding or other properties. In order to transition the ligand, we implemented a similarity search, where the action space returns the $n$ most similar molecules in terms of 3D shape overlay based on a user provided database. The action space for a given state is then defines as the set of molecules that are the top $n$ most similar from a given database, such as PubChem. One benefit of this module is that enumerating the actions is exceptionally fast and all actions are synthetically reasonably, at least up to the quality of the database used. A second action space uses the FastRocs toolkit from OpenEye, which utilizes parallel GPUs to search a local database for known active compounds of similar shape to the given ligand, comparing millions of potential compounds per second. It returns a configurable number of sufficiently similar compounds for further analysis as potential modifications to the ligand.  A second action space is based on the derivation and models trained in this paper for a scaffold-based navigation model.

 

Policy

In each episode of the simulation, the ligand structure will be perturbed to look for better binding and/or new ligand structures. The changed structure will then persist as the base structure in the next episode. RLMM supports various policies to allow for flexible choices in how the ligand will be modified in each episode and which modifications will persist.

 

 

What makes your approach stand out from the community? (<100 words)

To the best of our knowledge, RLMM is the only drug discovery workflow which combines molecular dynamics simulations with AI-based design. Furthermore, while most reinforcement-learning based AI drug discovery techniques utilize unconstrained generative networks (presenting a challenge for experimental validation), RLMM uses a constrained generative network coupled with a database-embedding so that its design space is constrained to purchasable compounds without need for advanced synthesis. This makes it stand out methodologically at large as well as specifically for use in the CACHE challenge.

Method Name
RLMM
Commercial software packages used

OpenEye

Free software packages used

RDKit, OpenMM, AMBER20

Relevant publications of previous uses by your group of this software/method

Scaffold embeddings: Learning the structure spanned by chemical fragments, scaffolds and compounds

Austin Clyde, Bharat Kale, Maoyuan Sun, Michael Papka, Arvind Ramanathan, Rick Stevens

NeurIPS Workshop on Learning Meaningful Representation of Life ‘21.

Virtual screening of merged selections
Method type (check all that applies)
Deep learning
Physics-based
Description of your approach (min 200 and max 800 words)

Our approach for hit identification will be built off the tools used in RLMM (previous section), the BFE toolkits of RLMM will be used. In addition, we will use DeepDriveMD to analyze conformational changes induced by the proposed compounds. DeepDriveMD is a streaming adaptive sampling toolkit that leverages AI to model the states of a system during simulation, allowing for restarts in different regions of the state space to sample more diverse conformations efficiently. We will use this in conjugation with BFE estimation in order to understand how different chemotypes in the proposed compound list interact with and induce changes in the dynamics of the protein. With DeepDriveMD we can sample rapidly the proposed compounds by how they interact with the target (if at all). The resulting dataset which will provide state models of the protein due to the scale of simulations can be used to inform the last step of the competition—optimization. Given the dataset of a few thousand compounds simulated over a large landscape of the protein, due to DeepDriveMD’s adaptive sampling, and the corresponding BFE estimates, we can fine-tune the adaptive sampling strategy so to directly use the adaptive sampling inside of the RLMM design loop during the hit-optimization stage of the challenge.

What makes your approach stand out from the community? (<100 words)

   By leveraging a complete design toolkit, we can continually learn and improve the RL agents based on the data generated. So, by performing large-scale simulations on the proposed compounds, we are not simply just estimating the BFE for each but feeding that data aback into the model’s understanding of how candidate drugs induce changes in the energy landscape.

Method Name
DeepDriveMD
Commercial software packages used

None

Free software packages used

OpenMM, Amber20, PyTorch, RDKit

Relevant publications of previous uses by your group of this software/method

Casalino, Lorenzo, Abigail C. Dommer, Zied Gaieb, Emilia P. Barros, Terra Sztain, Surl-Hee Ahn, Anda Trifan et al. "AI-driven multiscale simulations illuminate mechanisms of SARS-CoV-2 spike dynamics." The International Journal of High Performance Computing Applications 35, no. 5 (2021): 432-451.

Clyde, Austin, Stephanie Galanie, Daniel W. Kneller, Heng Ma, Yadu Babuji, Ben Blaiszik, Alexander Brace et al. "High-throughput virtual screening and validation of a sars-cov-2 main protease noncovalent inhibitor." Journal of chemical information and modeling 62, no. 1 (2021): 116-128.

Brace, Alexander, Michael Salim, Vishal Subbiah, Heng Ma, Murali Emani, Anda Trifa, Austin R. Clyde et al. "Stream-AI-MD: Streaming AI-driven adaptive molecular simulations for heterogeneous computing platforms." In Proceedings of the Platform for Advanced Scientific Computing Conference, pp. 1-13. 2021.

Cache

All rights reserved
v5.47.19.49

Footer first

  • Login
  • Applicant Login
  • Terms of Participation
  • Privacy Policy
  • FAQ
  • Docs
This website is licensed under CC-BY 4.0

Toronto website development by Rebel Trail