Datadog’s Next Gen: ‘Bits’ and LLM Solution
At its annual Dash conference, Datadog announced two new products in beta: “Bits,” an AI tool, and an end-to-end solution for overseeing large language models (LLMs). Datadog provides cloud observability services to both enterprise applications and infrastructure.
Bits AI Assistant
Bits assists its users by generating responses after understanding natural language commands. For instance, if a user notices that its application’s response times have suddenly increased, they can seek assistance from Bits. Bits allows them to ask questions like: “Our app is responding slowly. Can you help us figure out the reason?”
The AI assistant then analyzes data from various sources, such as logs, metrics, user actions, and information from Confluence pages and Slack conversations related to previous performance optimizations. In this scenario, it can quickly identify that a recent code change has led to a slowdown. The identification may be followed by explaining the issue, such as the specific lines of code involved. Hence, users save time they would otherwise have spent on grunt work.
The image below shows another use case of the service.
Source: Datadog
LLM Observability Solution
Datadog also introduced a tool for monitoring large language models (LLMs), which tracks model behavior, usage, costs, and API performance. It detects occurrences like hallucinations, enabling app developers and ML engineers to collaborate for corrections. For instance, if an e-commerce LLM starts generating odd product recommendations, Datadog’s solution may alert the team and provide insights. As a result, joint efforts are made to refine the model and ensure reliable customer responses.
The rise of large language models in enterprises drives the demand for effective monitoring solutions. Datadog’s advancements simplify observability for enterprise teams, enable efficient workflow, and reduce the time required to address problems in applications and infrastructure.
