What are Large Language Model Plugins and How to Create and Use Them

In the field of artificial intelligence, large language models (LLMs) are "learned" based on pre-trained datasets to obtain parameters that can fit results. Although the capabilities of the models increase with more parameters, both specialized small models and the most popular large models with the best performance today share a common disadvantage: they cannot accurately/correctly answer results outside of the training dataset (different from the validation and test sets, such as real-time news, undisclosed corporate information, etc.), and then fabricate answers to respond, known as the LLM hallucination problem.

To solve the above problem while avoiding the cost of fine-tuning or retraining, LLM plugins have emerged. By combining the powerful content generation capabilities and contextual understanding capabilities of LLMs with data and specific functions provided by plugins, not only are the application areas of LLMs expanded, but the credibility of the results generated by LLMs is also increased, serving users better.

Part 1. What is Plugins?

A plugin is a software component that can be added to a main application or system to expand its capabilities or provide additional features.

Plugins are typically designed to be installed and uninstalled independently and can interact with the main application. The functionality of a plugin depends on the applied field and the specific application. In the rapid development of AI today, LLM plugins have risen abruptly, changing the form of plugin existence, which is also the focus of this article.

Combined with the current development of LLM plugins, plugin classifications are as follows:

1. Traditional Plugins

Traditional plugins (also known as addin, add-on, addon) are programs written in accordance with certain specifications of the application programming interface. They can only run on the system platforms specified by the program (may support multiple platforms at the same time), and cannot run separately from the specified platform. That is, the operation of the plugin depends on the host software and cannot selectively enable or disable plugin functions.

Traditional plugins can be divided into browser plugins and client software plugins. The existence of traditional plugins is shown in the figure below.

2. LLM Plugins

LLM plugins are an entirely new type of plugin born with the development of large language models.

The core of LLM plugins is the Web API, independent of large language models. The plugin development process is not constrained by large language models, and there are no restrictions on development languages, making them more universal. As long as the Web API follows RESTful-related rules, LLM plugins can be built.

When configuring plugins for large language models, configuration rules should be followed, such as native ChatGPT plugin configuration following OpenAPI format and adding relevant descriptions. Large language models and plugins are two relatively independent parts. The relationship between large language models and plugins is shown in the figure below.

Large language models are selectors for plugins, using plugin functions on demand, that is, plugins are only called when the user's question or data meets the plugin call conditions, not indiscriminately using plugin functions.

The workflow of LLM plugins is shown in the figure below.

Part 2. What are LLM Plugins and How to Create and Use Them?

LLMs Plugin consists of two types of plugins: ChatGPT Plugins and Custom AI Bot Platform Plugins, taking GPTBots as an example.

1. ChatGPT Plugins

Currently, the most powerful commercial large language model is ChatGPT-3.5/4.0 from OpenAI. They both support plugin functions (later uniformly referred to as ChatGPT), and provide plugin development entry for developers to develop Web APIs as ChatGPT plugins based on their own needs.

However, creating plugins in ChatGPT is quite cumbersome.

Below we use the GPTBots plugin creation process for comparison. The plugin creation processes for both are shown in the figure below (left: ChatGPT plugin creation; right: GPTBots plugin creation).

2. Custom AI Bot Platform Plugins - GPTBots Plugins(Tools)

The technical approach of GPTBots plugins is different from ChatGPT. GPTBots plugins integrate the ChatGPT plugin creation specifications (common OpenAPI specifications) and function invocation capabilities. This has the following advantages:

• Developers only need to focus on developing their own functional interfaces without developing additional interfaces.
• Plugin configuration follows OpenAPI specifications. Developers can directly reuse plugins for ChatGPT Web and publish them to GPTBots with one click.
• GPTBots plugins automatically support mainstream LLMs that already have plugin capabilities. Developers don't need to adapt to each LLM.

Through the above plugin creation process comparison, we know that using GPTBots plugins only requires four steps: develop the plugin interface, create a new plugin, configure plugin authentication, and add interface configuration conforming to OpenAPI specifications. GPTBots handles the rest. Next, we will go through how to create a GPTBots Plugin(Tools).

• 1

  Develop the plugin interface

  This is the same as the native ChatGPT way. Developers need to develop the plugin interface themselves. However, in GPTBots, developers do not need to develop a separate plugin manifest interface, plugin interface configuration information interface, and plugin logo interface. Developers only need to focus on developing the plugin function interface.
• 2

  Create a new plugin

  Log in to your GPTBots account and enter the portal backend. Click tools in the left menu bar to jump to the tools list. Next, click Create Tool and fill in the information on the creation form, including the name of the tools, the API interface that the tools connect to, and the Tool Introduction, which describes the functions of Tool to help LLM better understand and assist users in invoking them during conversations.
• 3

  Configure plugin authentication

  GPTBots plugins provide 4 authentication methods: no authentication (None), Basic authentication, Bearer authentication, and Service authentication(Custom). The configuration process is shown below. Especially for Service authentication, you can customize authentication and easily set whether the authentication parameter value comes from the Request Header or Querystr, the corresponding authentication Parameter name and Service token/API key, which is very convenient for you to configure on demand.
• 4

  Create Actions

  Action, which is equivalent to an API. A Tool can contain multiple Actions. You can directly create a Tool by writing a JSON document that conforms to the OpenAPI specification for the API.

  Firstly, fill the name, description, and service address for the Tool. After that Click "Next" for saving.

  

  2. Secondly, fill the input parameters.

  

  3. Thirdly, Fill the output parameters. The output parameters can be left blank, but filling it in can further help the LLM understand the meaning of the output fields to perform better. You can use the 'auto parse' to let the system automatically fill in the output parameters for you.

  

  4. Finally, by Debugging to verify that the API is successfully called, you can complete the creation of an Action. Once the authentication and Action information are configured, the Tool is fully set up and ready to use.

Part 3. GPTBots Open Tools Plugin Market

Log in to your GPTBots portal, and click "Tools Open Tools". There are many types of Tools (i.e. Plugins mentioned above), including free or paid plug-ins for Development, SEO writing or translation, life (weather query), image and video processing, etc.

Each Tool can be reviewed for its specific information, including an introduction, usage, input parameters, and output parameters, etc. The following figure is an example of tools that use GPT4 to convert images into text.

Thoughts and Outlook

In actual business applications, due to the token limit per LLM call, and plugins inevitably occupy tokens, multiple plugins cannot be provided in one request call. Or if a plugin protocol is too complex, it may cause a direct call to fail.

To better solve such problems, GPTBots platform launched the Flow feature. Developers can orchestrate multiple LLMs to participate in business processing at different steps. Each LLM can add up to 3 plugins, which solves the plugin usage restriction problem very well. At the same time, by reducing the length of the request context content, let the LLM focus more, thereby improving the plugin call success rate.

LLM plugins are different from traditional plugins. They are independent, flexible, free, and powerful. The core of LLM plugins is the Web API, so LLM plugins fully embrace the Internet, and there is no "discrimination" against development languages. As long as developers can develop HTTP protocol interfaces and follow RESTful rules, LLM plugins can be built, regardless of whether the developer's language stack is Python, Java, Go, PHP, etc. It is believed that in the future, the types and capabilities of LLM plugins will become more and more abundant. Compared to ChatGPT's cumbersome plugin creation process, the GPTBots platform simplifies the plugin creation process, simple, easy to use, and efficient, improves developer efficiency, reduces developer learning costs, and will certainly attract more and more developers to choose the GPTBots platform in the future.