Despite the rapid rise of Meta's new model Llama 3 as one of the widely used and customizable large language models (LLMs), OpenAI, the competitor leading the generative AI era, has successfully addressed the challenge by adding new enterprise-level features to models like GPT-4 Turbo LLM.

Recently, OpenAI announced the expansion of its enterprise-level features for API customers, further enriching its Assistants API and introducing new tools aimed at enhancing security and management control, as well as more efficient cost management.

Olivier Godement, the product manager of OpenAI's API, stated in a recent video call interview with VentureBeat, "OpenAI continues to be at the forefront when it comes to meaningful work with AI models for developers and businesses. However, we always welcome more competition - that's how we can all progress together."

Private Link and Enhanced Security Features

In terms of security upgrades, OpenAI has introduced Private Link, a secure communication method that enables a direct connection between Microsoft's Azure cloud services and OpenAI. OpenAI states that this helps minimize the risk of customer data and queries sent via the API being "exposed to the open internet."

This new feature complements existing security measures, including SOC 2 Type II certification, single sign-on (SSO), AES-256 static data encryption, TLS 1.2 transport encryption, and role-based access control.

In addition, to meet the growing compliance requirements, OpenAI has introduced native multi-factor authentication (MFA) to strengthen access control.

For healthcare companies that need to comply with HIPAA regulations, OpenAI continues to provide a business associate agreement and a zero data retention policy for eligible API customers.

Assistant API Upgrade to Handle Up to 500x More Documents

One of OpenAI's less publicized but crucial enterprise-level products is the Assistant API, which allows businesses to deploy custom-tuned models in their own applications and provide conversational assistants through retrieval-augmented generation (RAG) for specific documents.

For example, e-commerce company Klarna praised the AI assistant it created using OpenAI's Assistant API, which completed the work of 700 full-time human agents, reduced repetitive queries by 25%, and almost halved resolution time from 11 minutes to 2 minutes.

Now, OpenAI has upgraded the Assistant API with the addition of the "file_search" feature, enhancing its document retrieval capabilities, allowing each assistant to handle up to 10,000 files.

This represents a 500x improvement compared to the previous limit of 20 files and includes additional functionalities such as parallel queries, improved re-ranking, and query rewriting.

Furthermore, the API now supports streaming responses for real-time conversations, meaning AI models like GPT-4 Turbo or GPT-3.5 Turbo can return outputs as fast as generating tokens without waiting for a complete response.

It also integrates a new "vector_store" object to better manage files and provides finer control over token usage for effective cost management.

Projects Feature for Fine-Grained Access Control

The new Projects feature, which allows organizations to manage roles and API keys at the project level, provides improved management oversight.

This feature enables enterprise customers to set permission scopes, control available models, and establish usage-based limits to avoid unexpected costs. These improvements are expected to greatly simplify project management.

Essentially, businesses can isolate specific fine-tuned versions of AI models or even unmodified base models for specific tasks or document collections and grant specific individuals permissions to handle each task or document collection.

For example, if your company has one team responsible for handling public-facing documents and another team handling confidential or internal documents, you can assign separate projects to each team using OpenAI's API. This allows both teams to work with AI models without the risk of information confusion or leakage.

Miqdad Jaffer, a member of OpenAI's product team, stated in a recent video call interview with VentureBeat, "As more organizations and independent developers start deploying AI, they want to work within a confined scope. The Projects feature allows you to isolate resources and members into small personalized projects. You can get usage reports for each project and have control over access permissions, security, latency, throughput, and costs, enabling organizations to build in a very secure way. For independent developers, they can deploy hundreds of projects without worrying about any issues."

This is particularly helpful for development teams serving or handling multiple clients.

In addition, OpenAI has introduced a range of new cost management features to further assist organizations in scaling their AI operations economically.

These features include discounted rates for customers maintaining a consistent token usage level per minute and a 50% cost reduction for asynchronous workloads through the new Batch API. The Batch API also has higher rate limits and promises results within 24 hours.

However, to use the Batch API, customers need to send their batch of tokens (i.e., the inputs they want the AI model to analyze, whether prompts or files) in a single request and be willing to wait for a response from OpenAI's AI models for up to 24 hours.

Although this may seem like a wait time, OpenAI executives told VentureBeat that the results can be returned as quickly as 10-20 minutes.

Furthermore, this feature is designed specifically for customers and businesses that do not require immediate responses from AI models. For example, investigative journalists may want to send a stack of government documents to OpenAI's GPT-4 Turbo for filtering and extracting detailed information for writing in-depth articles.

Alternatively, a company preparing a report analyzing its financial performance over a period of time may have a submission deadline in weeks rather than days or minutes, making this feature highly applicable.

As OpenAI continues to strengthen its focus on enterprise-level security, management control, and cost management, these updates demonstrate the company's aim to provide a more "plug-and-play" experience for businesses, countering the rise of open-source models like Llama 3 and Mistral that may require more configuration work at the organization's end.