Prost5 : novel language model using the FoldSeek structural representation to introduce structural dimension to the model.
The foldseek 3Di representation is used to encode 3D protein structures as 1D token sequences, enabling seamless translation between amino acid sequences and structural representations.
The model was fine-tuned on 17 million AlphaFoldDB structures using ProtT5 as a base, with bi-directional translation tasks to map between amino acid (AA) and 3Di sequences.
ProstT5 achieves 3600-fold faster remote homology detection compared to AlphaFold-based methods, while maintaining near-experimental accuracy and improving fold classification tasks like CATH.
ProstT5 embeddings outperform ProtT5, ESM-1b, and Ankh for structure-related tasks and show competitive performance in inverse folding, generating diverse sequences with preserved structural similarity. Though in most cases ProteinMPNN still performs better for inverse folding.
Novel language model for antibodies, blending sequence and structural information.
The model encodes sequence ‘as usual’ and uses GVP-GNN (like esm-if) for structural representation. Only the three backbone atoms (C,N,Ca) are taken per residue to get the structural representation.
The data is a mix of sequence data and X-ray structures. The sequence datasets were modeled using ImmuneBuilder to increase structural coverage.
The model has an MLM objective on sequence & structure with three losses - sequence only, sequence + structure and structure only.
On sequence infilling IgBLEND performs better than other methods (e.g. AbLang, Nanobert), though arguably CDR-H3 predictions look very ‘close’ across the board.
On inverse folding the method performs quite a stretch better with large gaps in CDR-H3 with notable improvements for nanobodies - that other methods like ESM-IF or AntiFold did not handle natively.
The method projects the epitope and paratope onto 2D images and then uses a ResNET to predict the interacting vs non-interacting pairs.
Negative set was done by pairing non-cognate antibody-antigen pairs, rotations etc.
The method was not benchmarked against epitope predictions tools, that arguably do not take pairs into account, but against docking tools, scoring 13 out of 18 methods tested.
AlphaBind, a deep learning model designed to optimize antibody sequences, by leveraging large-scale pre-trained affinity datasets and fine-tuning on experimental data.
AlphaBind was pre-trained on a dataset of 7.5 million antibody-antigen affinity measurements, which includes data from yeast display systems and diverse antibody libraries obtained from multiple experimental sources, focusing on quantitative affinity measurements.
The model utilizes transformer-based architecture with protein sequence embeddings generated using ESM-2nv (Evolutionary Scale Model) embeddings. The model consists of 4 attention heads, 7 layers, and about 15 million parameters.
The model fine-tunes on specific antibody-antigen systems using AlphaSeq data, then performs stochastic greedy optimization by generating sequence mutations (using ESM-2nv logits) to explore sequence space and predict binding affinity. This process generates thousands of candidate sequences, which are filtered based on affinity predictions and developability metrics before in vitro validation.
The novel sequences based on three systems were verified in experimentally.
Novel model for sampling the structural space of proteins - applied to nanobodies.
Protein structures are encoded into latent tokens using a discrete variational auto-encoder (dVAE), which captures residue-level local geometric features in a roto-translation invariant manner.
The model combines a dVAE encoder-decoder with a Structure Language Model (SLM) to model sequence-to-structure relationships.
One can sample alternate conformations by providing an amino acid sequence as input and use the SLM to generate latent tokens representing potential conformations. Decode these latent tokens back into 3D structures using the dVAE decoder to generate diverse structural ensembles.
Altogether, though they develop ESMdiff in the context of the paper, it is rather the structural language in the context of the broader framework of language models that forms the core of the message, rather than any single method.
The model was evaluated on tasks like generating equilibrium dynamics, fold-switching, and intrinsically disordered proteins, using metrics like JS-divergence, TM-score, and RMSD. It outperformed existing methods in speed and accuracy, generating structures 20-100× faster.
Pilot single-shot computational antibody design, where known binders were taken and new ones computationally generated on their basis, maintaining binding with a good developability profile.
The pipeline starts with known binders to the SARS-CoV-2 RBD
Novel binders were generated using a combination of computational approaches, including: Observed Antibody Space (OAS): Paired and unpaired sequences from the OAS dataset to identify antibody candidates within a certain edit distance from the starting antibodies. Inverse Folding Model (AbMPNN):generated new antibody sequences maintaining structural features compatible with binding to the SARS-CoV-2 RBD. ESM: guided the mutation of sequences to retain or improve binding affinity while enhancing developability.
The developability properties were assessed using Rosetta scoring to evaluate antibody stability and interface energetics, alongside TAP.
Experimental methods for screening included size-exclusion chromatography (SEC) to assess aggregation propensity and differential scanning fluorimetry (DSF) for thermal stability. Antibodies that passed these criteria were deemed suitable for development.
Success rate of the method: The pipeline demonstrated a success rate of 54% for generating binding antibodies that retained affinity against escape mutations on the SARS-CoV-2 RBD.
Authors tested RFDiffusion for the design task but with poor success rate - albeit not the antibody-fine tuned version it appears, that should work better.
Novel method to predict stability of proteins, based on ProteinMPNN.
The method employs ProteinMPNN embeddings with a stability prediction module to gauge the effect of single point mutations on protein stability.
The Stability prediction module is composed of the light attention module (that figures out which parts of the embeddings should be upvoted) followed by shallow multi layer perceptron.
For training/evaluation they employed Megascale and Fireprot datasets with measured protein stability data. Though after much pre-processing because the original datasets either contained many unreliable data points, or there was a risk that the mutations would change the structure too much.
Ablations show that all the elements of the network are important and bring something to the prediction, with ProteinMPNN having quite some predictive power out-of-the box.
Authors perform humanization of VHHs and generate experimental data confirming their designs.
The protocol involves grafting CDRs1-3 and then systematically modifying Hallmars/Verniers and others to make them more human.
Positions 49 and 50 (e.g., E49G, H50L in VHH1): These were generally well-tolerated, allowing for humanization without major impact on binding affinity or stability.
Position 52 (e.g., S52W in VHH2): In some cases, changing this residue even improved affinity.
Position 42: Humanizing residue F42 to a more human-like amino acid (e.g., F42V) in VHH2 led to a significant reduction in binding affinity. This residue plays a key role in stabilizing the CDR3 loop through interactions with other regions, making it essential for maintaining the bioactive conformation.
Position 52 (in some contexts): In VHH1, the mutation G52W led to a loss of binding due to steric clashes, demonstrating that this position can be critical depending on the structural context.
Measured binding affinities, expression yields, and purities of humanized variants. Crystal structures confirmed effects of humanization on binding; non-canonical disulfides stabilize CDR3
The model first learns the diffusion of a human sequence (with CDRs intact). The framework residues are diffused back. The network is then fine-tuned on mouse sequences.
There are two flavors of the model - one nanobody, another antibody.
They curate a great dataset from patents with over 300 sequences of paired humanized/native seuqences.
They demonstrate in silico and in vitro that their designs make sense.