avatar

Jacob Morrow

2023-10-11 14:19:46

3243 Views, 5 min read

In the field of artificial intelligence, large language models (LLMs) are "learned" based on pre-trained datasets to obtain parameters that can fit results. Although the capabilities of the models increase with more parameters, both specialized small models and the most popular large models with the best performance today share a common disadvantage: they cannot accurately/correctly answer results outside of the training dataset (different from the validation and test sets, such as real-time news, undisclosed corporate information, etc.), and then fabricate answers to respond, known as the LLM hallucination problem.

To solve the above problem while avoiding the cost of fine-tuning/retraining, LLM plugins have emerged. By combining the powerful content generation capabilities and contextual understanding capabilities of LLMs with data and specific functions provided by plugins, not only are the application areas of LLMs expanded, but the credibility of the results generated by LLMs is also increased, serving users better.

Plugins

A plugin is a software component that can be added to a main application or system to expand its capabilities or provide additional features. Plugins are typically designed to be installed and uninstalled independently and can interact with the main application. The functionality of a plugin depends on the applied field and the specific application. In the rapid development of AI today, LLM plugins have risen abruptly, changing the form of plugin existence, which is also the focus of this article. Combined with the current development of LLM plugins, plugin classifications are as follows:

Traditional Plugins

Traditional plugins (also known as addin, add-on, addon) are programs written in accordance with certain specifications of the application programming interface. They can only run on the system platforms specified by the program (may support multiple platforms at the same time), and cannot run separately from the specified platform. That is, the operation of the plugin depends on the host software and cannot selectively enable or disable plugin functions. Traditional plugins can be divided into browser plugins and client software plugins. The existence of traditional plugins is shown in the figure below.

LLM Plugins

LLM plugins are an entirely new type of plugin born with the development of large language models. The core of LLM plugins is the Web API, independent of large language models. The plugin development process is not constrained by large language models, and there are no restrictions on development languages, making them more universal. As long as the Web API follows RESTful related rules, LLM plugins can be built. When configuring plugins for large language models, configuration rules should be followed, such as native ChatGPT plugin configuration following OpenAPI format and adding relevant descriptions. Large language models and plugins are two relatively independent parts. The relationship between large language models and plugins is shown in the figure below.

Large language models are selectors for plugins, using plugin functions on demand, that is, plugins are only called when the user's question or data meets the plugin call conditions, not indiscriminately using plugin functions. The workflow of LLM plugins is shown in the figure below.

ChatGPT Plugins

Currently the most powerful commercial large language model is ChatGPT-3.5/4.0 from OpenAI. They both support plugin functions (later uniformly referred to as ChatGPT), and provide plugin development entry for developers to develop Web APIs as ChatGPT plugins based on their own needs. However, creating plugins in ChatGPT is quite cumbersome. Below we use the GPTBots plugin creation process for comparison. The plugin creation processes for both are shown in the figure below (left: ChatGPT plugin creation; right: GPTBots plugin creation).

GPTBots Plugins

We're different! The technical approach of GPTBots plugins is different from ChatGPT. GPTBots plugins integrate the ChatGPT plugin creation specifications (common OpenAPI specifications) and function invocation capabilities. This has the following advantages:

  • Developers only need to focus on developing their own functional interfaces without developing additional interfaces.
  • Plugin configuration follows OpenAPI specifications. Developers can directly reuse plugins for ChatGPT Web and publish them to GPTBots with one click.
  • GPTBots plugins automatically support mainstream LLMs that already have plugin capabilities. Developers don't need to adapt to each LLM.

Creating Plugins

Through the above plugin creation process comparison, we know that using GPTBots plugins only requires four steps: develop the plugin interface, create a new plugin, configure plugin authentication, and add interface configuration conforming to OpenAPI specifications. GPTBots handles the rest.

Develop the plugin interface

This is the same as the native ChatGPT way. Developers need to develop the plugin interface themselves. However, in GPTBots, developers do not need to develop a separate plugin manifest interface, plugin interface configuration information interface, and plugin logo interface. Developers only need to focus on developing the plugin function interface.

Create a new plugin

The GPTBots New Plugin entry is shown below.

Configure plugin authentication

GPTBots plugins provide three authentication methods: no authentication (None), Basic authentication and Bearer authentication. The configuration process is shown below.

Add interface configuration conforming to OpenAPI specifications

After completing the above preparation, finally configure the interface rules for the plugin, that is, the developer's functional interface. GPTBots uses interface rules that comply with OpenAPI specifications. The configuration instructions are shown below.

After plugin creation is complete, you can see the successfully created plugin in "My Plugins". At the same time, GPTBots provides a one-click plugin release feature to publish the plugin to the plugin market for use by other developers.

In summary, GPTBots streamlines the process of users creating and using plugins to the greatest extent, greatly reducing the threshold for using plugins.

Plugin Applications

GPTBots not only makes the plugin creation process very simple and developer-friendly, but using it is also very easy. Before using plugins, we need to add plugins to the Bot. Take the Gaode weather plugin as an example. The process of adding plugins to the "Weather Elf" bot is as follows:

After we add the weather plugin and search plugin to the "Weather Elf" bot, when there are weather and real-time information related issues in the problem, the plugins will take the initiative to take over the content production responsibilities. The actual effects are as follows:

Plugin Market

GPTBots officially provides developers and users with many practical plugins, such as PDF generation plugins, weather plugins, search plugins, etc. Some GPTBots plugins are as follows:

Thoughts and Outlook

In actual business applications, due to the token limit per LLM call, and plugins inevitably occupy tokens, multiple plugins cannot be provided in one request call. Or if a plugin protocol is too complex, it may cause a direct call to fail.

To better solve such problems, GPTBots platform launched the Flow feature. Developers can orchestrate multiple LLMs to participate in business processing at different steps. Each LLM can add up to 3 plugins, which solves the plugin usage restriction problem very well. At the same time, by reducing the length of the request context content, let the LLM focus more, thereby improving the plugin call success rate.

LLM plugins are different from traditional plugins. They are independent, flexible, free, and powerful. The core of LLM plugins is the Web API, so LLM plugins fully embrace the Internet, and there is no "discrimination" against development languages. As long as developers can develop HTTP protocol interfaces and follow RESTful rules, LLM plugins can be built, regardless of whether the developer's language stack is Python, Java, Go, PHP, etc. It is believed that in the future, the types and capabilities of LLM plugins will become more and more abundant. Compared to ChatGPT's cumbersome plugin creation process, the GPTBots platform simplifies the plugin creation process, simple, easy to use, and efficient, improves developer efficiency, reduces developer learning costs, and will certainly attract more and more developers to choose the GPTBots platform in the future.