Results of CACHE Challenge #1

CACHE#1 participants who asked to be de-anonymized:

Szymon Pach, Marko Breznik, Theresa Noonan, Leon Obendorf, Yu Chen, Katrin Denzinger, Sijie Liu, Julian Kleber, Valerij Talagayev, Amit Pandit, Kristina Puls, Clemens Alexander Wolf, Gerhard Wolber, Freie Universitaet Berlin

Evgeny Gutkin ,Filipp Gusev, Francesco Gentile, Fuqiang Ban, S. Benjamin Koby, Chamali Narangoda, Olexandr Isayev, Artem Cherkasov, Maria G. Kurnikova, Carnegie Mellon University and University of British Columbia 

David Koes and Ian Dunn, University of Pittsburgh & Carnegie Mellon University 

Christina Schindler and Lukas Friedrich, Merck KGaA

Dmitri Kireev, Marta Glavatskikh, Xiaowen Wang, Akhila Mettu, University of Missouri and University of North Carolina, Chapel Hill

Didier Rognan, Merveille Eguida, Francois Sindt, Université Strasbourg

Pavel Polishchuk, Guzel Minibaeva, Aleksandra Ivanova, Alina Kutlushina, Palacky University

Christoph Gorgulla and Dr. Sorin Draga, Harvard University and Virtual Discovery 

Kam Zhang, Ashutosh Kumar, and Francois Berenger, RIKEN, Zurich University of Applied Sciences, The University of Tokyo.

Shuangjia Zheng, Wei Lu, Galixir Jixian Zhang, Galixir Zhenyu Wang, Shanghai Jiao Tong University and Galixir

Fabian Liessmann, Paul Eisenhuth, Alexander Fuerll, Oanh Vu, Rocco Moretti, Jens Meiler, Vanderbilt University & University Leipzig

Carlos Zepeda, Christopher Barden, Treventis/UHN

Rick Stevens’ Lab, Argonne National Laboratory

Jan Jensen and Casper Steinmann, University of Copenhagen and Aalborg University

Emanuele Carosati, University of Trieste

All CACHE#1 compounds were tested experimentally at the Structural Genomics Consortium, University of Toronto, with contributions from Suzanne Ackloo, Cheryl Arrowsmith, Albina Bolotokova, Irene Chau, Aiping Dong, Kristina Edfeldt, Pegah Ghiabi , Elisa Gibson, Levon Halabelian, Rachel Harding, Zarah Hejazi, Scott Houliston, Ashley Hutchinson, Carla Kharadjian, Peter Loppnau, Fengling Li, Helen Li, Sumera Perveen, Matthieu Schapira, Almagul Seitova and Hong Zeng.

For CACHE Challenge #1, 23 participants applied their computational methods to predict which small molecule ligands bind to the central cavity of the WDR domain of LRRK2, a Parkinson’s disease target. An apo structure was available in the PDB and no ligand was previously reported for this site.

CACHE Challenges begin with a hit-finding round (Round 1), where participants can nominate up to 100 commercially available compounds. Round 1 predictions are then experimentally tested (details below) and the resulting data returned to participants. That data enables  the hit-expansion round (Round 2), where participants whose nominated compounds show some sign of activity in experimental binding assays (compounds of interest) can select up to 50 follow-up compounds

Together, these two rounds are designed to avoid both false positives and false negatives, which is critical to evaluate computational methods properly.

Primary screening of nominated compounds was conducted by SPR, an assay that measures direct binding of compounds to the WDR domain of LRRK2 captured on a chip. The output is a binding affinity (K_D) and %binding (i.e. measured versus expected amount of compound bound, based on the amount of protein captured on the SPR chip. Values > 100% indicate non-specific binding).

The most promising compounds were tested with one or both of two additional direct binding assays - isothermal titration calorimetry (ITC) or ¹⁹F NMR for fluorinated compounds - as well as for binding to an unrelated target by SPR (the PWWP domain of NSD2) to flag highly promiscuous compounds.

Aggregating or insoluble compounds as measured by dynamic light scattering (DLS) were also flagged.

After visual inspection to remove experimental artifacts, compounds with K_D < 150 µM and %binding < 150% in the SPR assay were advanced to Round 2.

We were generous in advancing compounds of interests from Round 1 to Round 2 to avoid false negatives. For example, compounds that showed low %binding in the primary screen by SPR and not confirmed by ITC or ¹⁹F NMR were advanced, as were those that bound to the unrelated target NSD2.

The purpose of Round 2 is to avoid false positives. Each participant selected analogs of their compound(s) of interest, within the limits of commercial availability. Here, we expected to see at least some analogs of the original hit showing some activity. The same experimental pipeline was used in Round 2, with the addition of differential scanning fluorimetry (DSF, a thermal stability assay) as an orthogonal binding assay.

To better compare methods directly, we sent participants the structures of all initially nominated Round 1 compounds, without experimental data, and asked them to predict which were active. Enrichment of the most promising compounds were low and not statistically significant and this exercise was not used in the final evaluation.

All data were evaluated by an independent Hit Evaluation Committee composed of industry experts in biophysics (Anders Gunnarsson, Astra Zeneca & Vera Puetter, Nuvisan), medicinal chemistry (Hartmut Beck, Bayer & Lars Wortmann, Boehringer Ingelheim), and computational chemistry (Pat Walters, Relay Therapeutics ) leading to a final score assigned to each participant.

The data are provided as follows:

• An excel file with the annotated ranking of the computational methods based on the experimental data (summarized in the table below)
• A schematic summary of the top performing computational methods
• Slide deck with detailed Round 2 data for each participant
• An excel file with Round 1 hits that were advanced to Round 2
• An excel file with all data for all compounds for both rounds
• Short manuals for non-experts to understand SPR and DLS data 

Summary of experimental results:

• No ligand was previously reported for this target (WDR domain of LRRK2)
• Participants collectively discovered multiple experimentally confirmed chemical series
• All hits are weak
• For all series:
  • multiple compounds show dose response by SPR
  • selectivity versus one unrelated target was tested and confirmed for at least one compound
  • At least one compound shows binding in an orthogonal assay (¹⁹F NMR, DSF, Xray) at concentration where it is 100% soluble and does not aggregate (measured by DLS)
  • At least one validated hit has 18 µM < K_D < 65 µM
• The preliminary crystal structure (refinement in progress) of one compound in complex with LRRK2 shows the compound occupying an unexpected side-pocket, which may be a crystallographic artifact, as binding is still observed in solution when a nearby residue is mutated.

Summary of computational methods (details are provided in the accompanying excel file; participants had the choice to remain anonymous):