We are thrilled to announce that M12 has co-led with First Spark Ventures, the Seed round for Synth Labs.ai, a startup developing leading technology for a fully auditable, transparent and robust AI alignment platform. This investment will enable them to expand their team, continue research efforts in AI alignment, evaluations, synthetic pipelines and launch their initial offering to the right partners. We look forward to this significant journey to revolutionize the way AI systems learn and interact with the world. Reinforcement Learning from Human Feedback (RLHF) is at the forefront of AI research, offering a path to more adaptable and intuitive AI systems. Synth Labs is poised to create AI that is not only more efficient but also aligns with human values and preferences by incorporating RLHF into the learning process using RLAIF and other optimization mechanisms.

We chose to invest in Synth Labs as they aim to make inference at the edge of LLMs well-suited for large, intensive models like GPT-4 using RLAIF delivering a more well-aligned and parametrized AI system. Reinforcement Learning is unavoidable, and it is not solved in a way that scales with enterprises. It is also currently a severe bottleneck to continuously align the LLMs, which ends up costing enterprises several thousands of dollars and puts them at a risk of bias and brittleness as prompt engineering is still not robust. Generating high quality training data on a scale and then integrating it with RLHF is the challenge.

RLHF allows AI models to be aligned with human values and intentions by allowing them to generate responses that are not only more accurate but also relevant to the tasks provided to them. However, RLHF faces a primary challenge of collecting high-quality human feedback. Human evaluators can have misaligned or even malicious intent and are prone to errors. The spectrum of human preferences also makes it very difficult to align artificial intelligence and posing the risk of reward hacking and challenging feedback aggregation. Moreover, due to diverse and unknown data, constant fine-tuning at the inference is required for the deployed AI models, and currently this process takes weeks to months along with human capital (annotators) to solve this problem. Also, such open models can be unsafe and prompt engineering is often not enough as there is a risk for models to go stale without the continuous fine tuning. Moreover, accessing and using sensitive data to maximize the value of the LLM is a real enterprise challenge in the production as traditional RLHF methods might have a risk of data exposure to human agents. At the core of the preference tuning challenge is retrieving enough high-quality, diverse and private customer data.

With this investment, we are partnering with a strong team led by Nathan Lile (CEO & Co-Founder) and Louis Castricato (CTO & Co-Founder) with deep roots in open source AI. Their contributions to open-source alignment frameworks have included libraries such as trlX, one of the most performant RLHF libraries. Nathan was previously the Chief of Staff at Stability AI, where he led several enterprise projects. Louis was the co-founder of Carper AI and served as a research scientist at Eleuther AI. He later joined as Head of LLMs at Stability AI, where he was responsible for shipping Stable Beluga2 on Hugging Face, one of the pioneering Llama based 70B parameter model. We are very excited that Francis deSouza, the former president and CEO of Illumina and also member of board of The Walt Disney Company, will be joining the team as a Co-Founder. Stella Biderman, exec head at EleutherAI, will continue in lead advisory role. We believe the excellent and diverse founding team at Synth Labs will achieve significant milestones in the field of RLHF, RLAIF and AI alignment. We look forward to leveraging the platform of Microsoft to help the team as they embark on this exciting journey.