LangSmith SDK v0.2 Enhances Evaluation Experience with New Features
Felix Pinkston Dec 06, 2024 06:02
LangSmith SDK v0.2 introduces simplified evaluation methods, improved performance, and enhanced documentation, significantly improving developer experience across Python and TypeScript.
In a recent update, LangChain has announced the release of version 0.2 of the LangSmith SDKs, aimed at enhancing the developer experience in evaluating applications. This release introduces several key improvements, including simplified usage of evaluation methods, performance enhancements, and expanded documentation, according to LangChain.
Simplified Evaluation Methods
The update simplifies the usage of the evaluate() and aevaluate() methods. Developers can now utilize simpler evaluators, which allow functions to score application outputs directly without needing complex argument structures. This change is applicable to both Python and TypeScript SDKs, streamlining the process for developers.
Moreover, the SDK now supports direct evaluation of langgraph and langchain objects, further simplifying the evaluation process. This feature allows developers to pass these objects directly into evaluation methods, reducing the steps needed for assessing application performance.
Consolidated Evaluation Functions
In the new version, LangSmith has consolidated its evaluation methods, reducing them to a single evaluate() method. This method now encompasses functionalities previously spread across multiple methods, making it easier for developers to run applications on datasets, evaluate existing results, and conduct comparative assessments.
Local Evaluations and Improved Performance
LangSmith SDK v0.2 introduces a beta feature allowing evaluations to be run locally without uploading results. This is particularly beneficial for developers who want to quickly test prompts or validate functions without recording evaluations. However, this feature is currently only supported in the Python SDK.
The update also includes performance improvements for the Python SDK, achieving approximately a 30% speedup in aevaluate() for large examples, enhancing the efficiency of the evaluation process significantly.
Revamped Documentation
The LangSmith team has overhauled its documentation, rewriting most of the evaluation guides and introducing new ones to align with the latest updates. This revamp includes a consolidated Python SDK API Reference, aiming to provide comprehensive and accessible resources for developers.
Breaking Changes in Python SDK
The update introduces some breaking changes in the Python SDK, including modifications to default concurrency settings and dataset identification processes. Additionally, support for Python 3.8 has been officially dropped following its end of life in October 2024.
LangSmith SDK v0.2 offers a more streamlined and efficient experience for developers by simplifying the evaluation process, enhancing performance, and providing improved documentation, making it a valuable tool for software development in both Python and TypeScript environments.
Image source: Shutterstock