When a new AI model is released, benchmark results showing its performance are published. However, the tasks evaluated in these benchmarks are often general, such as elementary school level arithmetic (GSM8K) and graduate level reasoning (GPQA), and are not specialized for a specific industry. To fill this gap, OpenAI has launched the “OpenAI Pioneers Program.” The program aims to promote the development of AI models tailored to specific industries and real-world use cases. Specifically, it is an initiative in which companies work with OpenAI researchers to develop more domain-specific evaluation criteria and fine-tuned models.
In a blog post , OpenAI pointed out that “many industries, including law, finance, insurance, healthcare, and accounting, lack a unified, reliable source of information for model benchmarking.” Therefore, OpenAI plans to work with multiple companies in each industry to develop these evaluation criteria in the future. This initiative is not just aimed at advancing model development, but also at building better trust between society as a whole and AI systems . In fact, research has also pointed out that the lack of such industry-specific benchmarks is a major barrier to the adoption of AI in business applications. For example, Silvio Savarese, head of Salesforce AI Research, published a blog post on “Enterprise General Intelligence (EGI).” EGI is a concept he advocates that refers to more advanced AI solutions tailored to the domain-specific needs of companies. He told ZDNET that one of the key steps to realizing EGI is the need for benchmarks that focus on evaluating domain-specific features.
Also Read: AnyMind Group Inc begins AnyAI Workflow for Automation
In addition to developing evaluation criteria, OpenAI will also work with teams to improve existing models for three industry-specific use cases using a technique known as reinforcement learning fine-tuning (RFT). The OpenAI team will provide participating companies with guidance on how to use RFT, and the companies can then decide how to deploy the models. OpenAI says these models are expected to be ready for large-scale deployment. The first group of participating companies will be made up of a small number of startups . They will work on use cases that “have a real-world impact.” Companies that meet these criteria can apply by filling out a form on the OpenAI Pioneers Program webpage with basic information. This article was edited for Japan by Asahi Interactive from an article published by Ziff Davis overseas.
SOURCE: Yahoo
