Computational Antibody Papers
Filter by tags
language models
Filter by published year
2024
TitleKey points
2024-09-04
Antibody Humanization via Protein Language Model and Neighbor Retrieval
- language models
- developability
- Novel humanization protocol employing language models and large-scale repertoire data.
- Human OAS and germline sequences are embedded using ESM2.
- K-nearest neighbors algorithm is then used to introduce mutations into the ESM-2 embedded query sequence coming from closest functional neighbors in the ESM2-embedded OAS+germlinse space.
- Results of humanized abs are validated experimentally via ELISA.
2024-08-28
AntiBARTy Diffusion for Property Guided Antibody Design
- language models
- generative methods
- developability
- Novel language model AntiBARTy with demonstration of how to use it to diffuse novel antibodies with favorable solubility properties.
- The core model is a BART-based transformer, with 16m parameters.
- It was firstly trained on all human heavy and light chains from OAS (254m heavies and 342m lights <- yes, more lights). This was followed by fine tuning on the higher quality paired data from OAS.
- The diffusion model was based on U-net (CNN used for segmentation of medical images), totaling 3m parameters.
- They define low and high solubility classes as predicted by protein-sol on paired OAS, with roughly 20k samples for each class.
- Overall, one can sample from multivariate to get a vector in Antibarty latent space and use it to get an antibody sequence that is either high or low protein-sol predicted solubility.
2024-08-28
Linguistics-based formalization of the antibody language as a basis for antibody language models
- language models
- Proposal for modeling antibodies using language, that is more fit-for purpose than current approaches.
- It is plausible to represent antibodies/proteins as language to draw from existing trove of research on natural language.
- Current approaches of porting the models from natural language to proteins/antibodies verbatim, might not release their full potential because of not focusing on key differences between natural language and proteins.
- Authors propose a more fit for purpose formalization, where quite an important part is better token definition and associating them with function. For instance do not simply use amino acids or k-mers but have something more complex such as C*U and RA*, associated with hydrophobicity, binding zinc fingers or similar.
2024-08-21
ABodyBuilder3: Improved and scalable antibody structure predictions
- structure prediction
- language models
- Updated version of the popular ABodyBuilder2 program to model antibodies, that is more efficient.
- Unlike ABB2 that did not employ language model embeddings, ABB3 does (ProtT5 to be precise).
- Instead of using several models and aggregating the results for error prediction, they train pLDDT within the model.
- ABB3 achieves CDR-H3 RMSD in the ballpark of 2.4Å whereas the previous version in the region of 2.5Å.
2024-08-15
A generative foundation model for antibody sequence understanding
- generative methods
- language models
- Authors introduce a large antibody-specific language model, Fabcon (2.4B) that demonstrably improves on predicting antibody specificity.
- Model is based on the Falcon LLM and is trained no the CLM objective (predict next amino acid going N-to-C terminal)
- The model was trained on paired (2.5m) and unpaired (821m) data from OAS.
- The pre-trained model was tested on its ability to fine tune on binder prediction using three datasets anti-her2, anti-sars-cov2 and anti-il6.
- When comparing against multiple other models on the binders prediction, the largest Fabcon model fares best, showing the benefit of overparametrization.
- Since the model was trained on CLS objective, it can be used for sequence generation, producing sequences that are very human-like as compared to human PBMCs.
- Benchmarking of deep learning methods on a range of antibody design tasks.
- Design tasks included are expression, thermostability, immunogenicity, aggregation, polyreactivity and binding.
- The tested methods include IGLM, AntiBERTy, ProtGPT2, ProGen2, ProteinMPNN, ESM-IF and Rosetta.
- The scoring procedure includes relating the model perplexities with the experimental values.
- The non-antibody specific model ProGen2 had the best correlations with the scores but no method is best in all fitness classes.
- Predicting intrinsic properties of antibodies gives better performance than extrinsic ones.
- Models are better at distinguishing differences within a set of antibodies derived from a single wild type (intra-family) rather than assessing the differences between antibodies of different origins (inter-family).
- Models with larger number of parameters noted sizeable improvements in fitness correlation with thermostability and polyreactivity
- Sequence based methods perform similarly to structure based ones though, their averages are slightly higher.
- Authors propose a new training regime for language models of antibodies, noting that previous approaches drew a lot from natural language.
- They use a masking rate between 50-70%, opposed to the normal pre-training rate of 15% in natural languages.
- Rather than masking individual residues they also mask entire spans of sequence, with focus on masking CDR-H3.
- Infilling using PARA is much more accurate in the CDR-H3 region - they get 48.7% accuracy in CDR-H3 versus 36.4%, closest SOTA, AntiBERTy.
- They applied their model to downstream tasks such as heavy/light calling and prediction of trastuzumab binding, however the gains here were modest with respect to other models (e.g. Her2 binding using simple CNN model got accuracy of 82.8 vs 83.7 using their method).
- The model is available at: https://github.com/xtalpi-xic/PARA/tree/main
2024-03-13
Protein language models enable prediction of polyreactivity of monospecific, bispecific, and heavy-chain-only antibodies
- language models
- developability
- Using language models to predict polyreactivity.
- Polyspecificity and polyreactivity are cognate, however the first is thought to be driven by factors such as overlapping epitopes whereas polyreactivity by excess charge or hydrophobicity.
- Baculovirus particles assay (BVP) is often used to test polyreactivity. mAbs are added at high-concentrations to BVP coated plates.
- They generated a dataset of polyreactive antibodies (~300 antibodies) that was heterogeneous in terms of antibodies/nanobodies, monospecific and formats.
- They tested different concentrations (from 6.67nM to 667 nM) and well coating types (percentage BVP) - this was aimed at reducing noise from experimental conditions.
- They tested two prediction modes, language models and structural descriptors. For language models, PROT5, ESM2 and Antiberty were used. Descriptors were calculated using Alphafold2-multimer. The language model predictions were superior to those calculated from AF2-multimer ones.
2024-03-13
Efficient evolution of human antibodies from general protein language models
- language models
- binding prediction
- They introduced a set of single and double mutations based on most likely variants proposed by an ensemble of language models (ESMs). Most of the mutations not only didn’t remove binding ability, but actually improved it.
- They performed evolution with the ESM-1b language model and the ESM-1v ensemble of five language models (six language models in total)
- In the first round of evolution, they measured the antigen interaction strength by biolayer interferometry (BLI) of variants that contain only a single-residue substitution from wild-type.
- In the second round, they measured variants containing combinations of substitutions, where we selected substitutions that corresponded to preserved or improved binding based on the results of the first round.
- They performed these two rounds for all seven antibodies, measuring 8–14 variants per antibody in round one and 1–11 variants per antibody in round two
- Across all seven antibodies, they found that 71–100% of the first-round Fab variants (containing a single-residue substitution) retained sub-micromolar binding to the antigen, and 14–71% percent of first-round variants led to improved binding affinity (defined as a 1.1-fold or higher improvement in Kd compared to wild-type)
- Thirty-six out of all 76 language-model-recommended, single-residue substitutions (and 18 out of 32 substitutions that lead to improved affinity) occur in framework regions.
- They found that Fabs for 21 out of the 31 language-model-recommended, affinity-enhancing variants that we tested had a higher melting temperature (Tm) than wild-type, and all variants maintained thermostability (Tm > 70 °C).
- They tested for polyspecificity but there were no off the chart changes in the poly profile.
- Five out of 32 affinity-enhancing substitutions (~16%) involve changing the wild-type residue to a rare or uncommon residue
- Approach based on general protein language models consistently outperformed all baseline methods, including the antibody-specific ones (!).
2024-03-13
Protein-Protein Interaction Prediction is Achievable with Large Language Models
- language models
- binding prediction
- They developed a language model to predict protein protein interactions from sequence on the basis of a large language model.
- They Train protBERT to predict PPI.
- They use the BIOGRID dataset, where interactors are mapped if they are confirmed by two independent sources, such as two independent experimental techniques in two separate studies. In total they have 179,018 positive pairs.
- They use negatome 2.0 as a negative dataset. It relies on various sources such as manual curation from the literature or subunits from the PDB that do not interact with each other. Total of 3,958 pairs were used.
- They use ProtBERT-BFD to pretrain the model.
- They mapped each protein pair as [CLS] Protein A [SEP] Protein B [SEP], mapping the final output to binary.
- They achieve 92% accuracy on the test set.
- They also perform well on annotating negative binders as coming from different subcellular compartments. On positive samples in this dataset, the model was 85% accurate. On negative samples, SYNTERACT was only 38% accurate, classifying many negatives from subcellular compartment sampling as interactors.