Foundational model following in the footsteps of AlphaFold3 attempting prediction of molecular interactions.
Model architecture is closely modeled on this of AF3 - however it was not benchmarked against it (nor ESM3) because of use restrictions.
It takes 30 days on 128 A100s to train the model. Back of the envelope Google Colab pro+ pricing of 128 (which is NOT distributed) puts it at ca. 120k USD :)
Addressing antibodies, they introduce constraints (e.g. known epitope residue) to help the model out. Adding even a single residue makes a big difference with respect to baseline which is tantamount to global docking.
It only takes one residue as constraints to improve ab-ag complex prediction. Success rate for ‘local’ mode, with just one residue is about 50% in getting it with DockQ score >.21, 30% >.49 and less than 10% for high quality hits.
Success rate for ‘global’ mode, so without constraints is about 35% in getting it with DockQ score >.21, 20% >.49 and less than 5% for high quality hits.
So altogether if I want to ‘hit the epitope’, the model has ca. 30% success rate.
If I want a high quality ab-ag complex structure, unfortunately it seems that constraints do not help much currently.
Authors describe how using a structure predictor one can re-design the binding site, to maintain binding.
They use a proprietary GaluxDesign method, the method achieves 1.4 Å Ca RMSD in predicting CDR-H3 loop structures, leveraging a unique scoring metric (G-pass rate) that assesses both confidence and structural consistency for antibody design.
The method outperforms AlphaFold 2.3, ABlooper, and ImmuneBuilder in predicting CDR-H3 loop structures, with significantly lower RMSD values (1.4 Å compared to 2.4-3.7 Å), particularly on a more challenging, time-separated dataset.
The binding propensity to HER2 was evaluated using a large mutant library and calculated via the G-pass rate, outperforming AlphaFold's PAE-based scoring. The model showed strong discrimination with an AUROC of 0.758, compared to 0.529 for AlphaFold. The novel loop is scored using their metric (G-pass rate) in complex with Her2.
Novel antibody sequences were designed by predicting six CDR loops in antibody-protein complexes, using GaluxDesign models. These designs were experimentally tested, achieving high success rates, including a 13.2% success rate for HER2 antibody designs using yeast display methods.
Novel CDR-H3 structure prediction method, ComMat based on ensemble sampling.
Rather than generating a single structure, the method generates several solutions that are then all informing the next iteration.
The method was integrated into the structure module of AlphaFold2.
Crucially, with the introduction of the second prediction into the ‘community’, the predictions become better. However these quickly plateau, showing the limits of the approach.
The method does not produce better results than ABodyBuilder2 and EquiFold.
Update on tfold-AB including modeling of the complex with the antigen, with applications to virtual screening.
They mostly use SABDAB/Covabdab as reference datasets.
The modeling happens by generating antibody & antigen features supplemented by a large language model, followed by flexible docking.
For antigen feature generation, they use AF2
They are training on several tasks simultaneously, ab structure prediction, complex prediction etc. making it a multi-task training.
On docking their method achieves DockQ 0.217 vs AlphaFold-Multimer DockQ score of 0.158 - that is global docking.
Whe local docking information is given, constraining paratope/epitope sites, their algorithm achieves DockQ of 0.416.
They demonstrate that filtering antibodies by their predicted modeling confidence score gives moderate enrichment against PD1 and Sars-cov-2 antigens, showing promise for virtual screening.
An update on SPACE1, employing ABodyBuilder2. Better coverage of the structural method.
They used binders against coronavirus, ebola, lysozyme among others.
Structures are modeled using ABodyBuilder2. Structures are sorted by CDR lengths, frameworks aligned by Cas and RMSD calculated for CDR loops. Finally a clustering algorithm is used.
The clustering algorithms benchmarked were DBSCAN, OPTICS-xi, OPTICS-DBSCAN, K-means, Butina clustering, greedy clustering.
Two more variants were developed, SPACE2-HC, for heavy chains only as well as SPACE2-Paratope, for paratyping.
Two accuracy metrics were used, the fraction of epitope-consistent clusters (number of epitope-consistent multiple-occupancy clusters / number of multiple-occupancy clusters) and the fraction of clustered antibodies in epitope-consistent clusters (number of antibodies in epitope-consistent multiple-occupancy clusters / number of antibodies in multiple-occupancy clusters)
Two coverage metrics were used, the number of multiple-occupancy clusters and the number of antibodies in multiple-occupancy clusters were used. In order to examine accuracy and coverage with one measure they calculated the number of antibodies in consistent multiple-occupancy clusters
They selected agglomerative clustering as best, though it is not better than Optics-XI, but it was providing larger clusters.
Space2 using all loops was better than SPACE2-HC or SPACE2-paratope
Space2 improves the coverage over SPACE1, thanks to the ABodyBuilder2 protocol.
Space2 increases coverage with respect to just clonotyping, but clonotyping remains much more accurate.
Introducing ESM-2 and ESMFold. Scaling transformer model parameter size to 15B allows for more precise predictions of structures.
They make available an atlas of 617 million predicted structures
Learning objective is MLM, masking 15% of protein input.
Perplexity ranges from 1 for a perfect model to 20 for a model that makes predictions at random. Intuitively, perplexity describes the number of amino acids the model is uncertain between when it makes a prediction
After 270k training steps the 8M parameter model has a perplexity of 10.45, and the 15B model reaches a perplexity of 6.37.
The 15B model achieves best perplexity and structural modeling accuracy.
For some structures, accuracy of structure prediction jumps from 7.7A at 8m parameters to 7.0A at 35m parameters and to 3.2A at 15m parameters. The 3b model brings it down to 2.8 and 15B model to 2.6. For other structures, good prediction is only achieved at 15B
Their structure predictor closely follows AlphaFold2, but instead of evoformer, they use the representation from the ESM-2.
100B parameter protein model fine-tuned and a 1B antibody-specific model.
For PLM training they employ data from Uniref90 and ColabFold. After filtering and deduplication they are left with approximately 350m sequences, or 100B tokens.
On proteins, xTrimoPGLM-100B outperforms ESM2-15B on 12 of 15 downstream tasks (e.g. thermostability, structure prediction etc.).
They train a 1B protein model and then fine tune it on antibodies from OAS
Their masking procedure includes span masking not only several residues at a time.
They use 678m OAS sequences.
They benchmarked the antibody model on naturalness and antibody structure prediction and the Xtrimo-pglm-oas outperformed ESMFold, ALphafold2 and IgFold.
Diffusion-based antibody-antigen binding site structural co-design
Sampling of antibody sequence and structure directly conditional on the antigen structure.
Model receives antigen structure and antibody framework in complex. Then CDRs are randomly initialized with AA types, orientations and positions.
The advantage over GANs and VAEs should be that it generates candidates iteratively so filters can be applied on the fly to the sampling process.
Diffusion probabilistic models learn to generate data via denoising samples from a prior distribution
They predict the amino acid type, ca coordinate and orientation in SO(3)
In addition to the joint design of sequences and structures, we can constrain partial states for other design tasks. For example, by fixing the backbone structure (positions and orientations) and sampling only sequences, we can do fix-backbone sequence design.
We cluster antibodies in the database according to CDR-H3 sequences at 50% sequence identity.
RMSD: is the Cα root-mean-square deviation (RMSD) between the generated structure and the original structure with only antibody frameworks aligned - however here higher RMSD means that the generated structure is more diverse.
However they also checked how accurate they are in RMSD when they fix sequences (so structure gets modified). Here for H3 they achieve 3.246 A.
AAR: is the amino acid recovery rate measured by the sequence identity between the reference CDR sequences and the generated sequences
They compared to RosettaAntibodyDesign by IMP (percentage of CDRs with better energy than the original cdr, AAR and Ca RMSD)
They optimize the antibody by perturbing it for several steps (forward diffusion) and then denoise it (going backwards) to find antibodies with better IMP but they also look at RMSD and Seq id.