Batch Chat Completion with LM Studio — lms_chat

Process a vector of inputs sequentially through LM Studio.

Usage

lms_chat_batch(
  model,
  inputs,
  system_prompt = NULL,
  format = c("vector", "list", "data.frame"),
  host = "http://localhost:1234",
  simplify = TRUE,
  quiet = NULL,
  ...,
  token = NULL
)

Arguments

model: Character. The loaded model name. Must be one name, given as a single string. The string must be valid in its declared encoding and not marked "bytes". A class, names, and the S4 bit are removed before the name is sent.
inputs: Character vector. The prompts to process. Must hold at least one value and no missing values.
system_prompt: Character. Optional system prompt.
format: Character. Output format: "vector", "list", or "data.frame".
host: Character. Server URL.
simplify: TRUE or FALSE. If TRUE, parses outputs. Any other value, NULL and NA included, aborts before the check for a running server.
quiet: TRUE, FALSE, or NULL, the default. NULL follows the rlmstudio.quiet option. TRUE starts no progress bar, and FALSE starts one, also when the option is TRUE. cli draws a started bar only after a delay, two seconds by default. Any other value, NA included, aborts before the check for a running server. quiet does not hide the warnings about failed inputs, cut-off replies, a vector format that returns a list, or a logprobs = TRUE that the "native" route ignores.
...: Additional arguments passed to lms_chat, such as api_type, logprobs, schema, ttl, previous_response_id, or store. A schema, a ttl, a previous_response_id, a store, and the api_type that each needs are checked before the first call. A previous_response_id goes to every call, so each input continues the same stored reply. It can be an earlier reply of lms_chat() itself, and the response_id attribute of that reply is sent, by the rules of lms_chat(). A reply that came back with no id goes out as its own text, and each input then fails with status 400, unless that text breaks the id rules of lms_chat() and aborts first. The character vector of format = "vector" and the output column of format = "data.frame" carry no attribute, as described below. A store here goes to lms_chat() for every call, and it is checked there by the rules of lms_chat(). It must be TRUE, FALSE, or NULL, and NULL, the default, sends no store field. Any other value, NA included, aborts before the check for a running server. With api_type = "openai", TRUE or FALSE aborts before the check for a running server. Only this exact name is checked. A shortened name, such as sto, goes into each request body unchecked, under the name you wrote. With store = FALSE, a native reply has no response_id field. An OpenResponses reply still has an id, but the server does not hold that reply, so a later call on the OpenResponses route that passes it as previous_response_id gets status 400 with the code "previous_response_not_found". A logprobs here, or a shortened name that lms_chat() reads as logprobs, must be TRUE or FALSE. Any other value, NULL and NA included, aborts before the check for a running server. With api_type = "native", which has no logprobs, a logprobs of TRUE is ignored. The batch then returns what it returns with logprobs = FALSE, in each format. It gives one warning for the whole batch, after the check for a running server, even with quiet = TRUE. With api_type = "openai" and simplify = TRUE, the batch sends the flag, and the logprobs of each reply is NULL, because the package reads no log probabilities from that route. The batch then gives one warning of class rlmstudio_no_logprobs that counts those replies, after the failed-inputs and cut-off warnings. A batch that aborts on a lost server, a refused token, a model the server cannot find, or a reply from another model gives it before the abort. The rlmstudio.quiet option and quiet = TRUE do not hide it. A batch in which no reply is read, for example because every input fails, gives no such warning. With simplify = TRUE and logprobs = TRUE on the "openresponses" or "openai" route, format = "vector" returns the list and says why in a warning. The rlmstudio.quiet option and quiet = TRUE do not hide it. In a batch that returns, that reason is a line of the failed-inputs warning if an input failed. Otherwise it is its own warning, before any cut-off or no-logprobs warning. Two values here that lms_chat() reads as the same argument abort with a message that names the argument. Examples are logprobs = TRUE, logprobs = FALSE, the shortened names log = TRUE, lo = FALSE, and two previous_response_id values. An exact name and a shortened name, such as logprobs and log, do not abort, because R gives the shortened one to the ... of lms_chat(). An input here also aborts, because the batch passes each element of inputs as input. An input reaches ... only when inputs is given by its full name. Otherwise R reads input as a shortened inputs, so it becomes inputs, and the value given by position fills the next unnamed argument, such as system_prompt. These aborts come before every other check of ... and before the check for a running server. An instructions here on the "openresponses" route, the default, and a messages here on the "openai" route abort before the check for a running server, with the message that lms_chat() gives. A previous_response_id here follows the rules of lms_chat(). It must be valid in its declared encoding and not marked "bytes". A class, names, and the S4 bit are removed before the id is sent. The package checks a stream here. A stream other than FALSE or NULL aborts before the call checks for a running server, because the package reads a whole reply and not a streamed one.
token: Character or NULL. An API token for a server that requires authentication. NULL reads the rlmstudio.token option and then the RLMSTUDIO_API_TOKEN environment variable. See rlmstudio_token.

Value

The return type depends on the format argument:

"vector": A character vector of responses, with NA for an input that failed. This format is only supported if simplify = TRUE, and logprobs = FALSE or api_type = "native". With a schema, it warns and returns the list instead. With logprobs = TRUE on the "openresponses" or "openai" route, it also warns and returns the list.
"list": A list where each element is the response corresponding to the provided input, or the condition for an input that failed. With a schema, simplify = TRUE, and logprobs = FALSE, each element that did not fail is the parsed reply.
"data.frame": A data.frame containing input and output columns, with NA in output for an input that failed. If logprobs = TRUE on the "openresponses" or "openai" route, an additional list-column named logprobs is included, with NULL for an input that failed. The "native" route adds no logprobs column. With a schema and logprobs = FALSE, output is a list-column of parsed replies, with the condition in place of an input that failed. Columns read from each reply follow output, or logprobs when it is there, as described below. With an object schema and logprobs = FALSE, one column per schema property comes after them, as described below.

On the native and OpenResponses routes with simplify = TRUE, lms_chat() returns each reply that carries an id with a response_id attribute, as its help describes. With format = "list", each reply keeps it. So does each element of the list that format = "vector" returns with logprobs = TRUE on the OpenResponses route. The character vector of format = "vector" and the output column of format = "data.frame" carry no response_id attribute. The data frame holds the ids in its response_id column.

With api_type = "native" and format = "data.frame", the data frame ends with seven columns read from each reply: response_id, input_tokens, total_output_tokens, reasoning_output_tokens, tokens_per_second, time_to_first_token_seconds, and model_load_time_seconds. response_id is character, and it identifies the reply on the server. The other six are double, and they come from the stats object of the reply. A cell is NA when its field is absent or is not one value of the column type, a string for response_id and a number for the others. An empty string is a string, so an empty response_id is kept. If stats is absent or is not a JSON object, all six stats cells are NA. The server can leave a field out, such as model_load_time_seconds. Such a cell does not fail the input and gives no warning.

With api_type = "openresponses" or api_type = "openai" and format = "data.frame", the data frame has four columns read from each reply after output, or after logprobs when it is there. They are response_id, input_tokens, total_output_tokens, and reasoning_output_tokens. These are the first four native column names, but the servers send the values under other names:

response_id is the reply's id on both routes.
input_tokens is usage.input_tokens on the OpenResponses route and usage.prompt_tokens on the OpenAI route.
total_output_tokens is usage.output_tokens on the OpenResponses route and usage.completion_tokens on the OpenAI route.
reasoning_output_tokens is usage.output_tokens_details.reasoning_tokens on the OpenResponses route and usage.completion_tokens_details.reasoning_tokens on the OpenAI route.

The columns are there for every setting of logprobs and schema. response_id is character, and the three counts are double. The NA rule is the one for the native columns: a cell is NA when its field is absent or is not one value of the column type, and an empty string is kept. If usage is absent or is not a JSON object, all three count cells are NA. If the details object is absent or is not a JSON object, only reasoning_output_tokens is NA. Such a cell does not fail the input and gives no warning.

A reply with no readable answer text fails its input, whatever its other fields hold. The row of an input that failed holds NA in every column read from the reply. If every input failed, those columns are still there, response_id as character and the others as double. The vector and list formats add no such column.

With api_type = "openai", format = "data.frame", logprobs = FALSE, and an object schema, the data frame ends with one column per top-level property of the schema, after the four reply columns. An object schema has a type of "object", written as a string or as a list that holds that one string. Its properties is a list of one or more named entries. Each column has the property name, in the order of properties. The columns come from the schema and not from the replies, so they are there even when every input failed. The properties of a nested object get no columns of their own. Any other schema adds no column.

The type of a property column follows the type of the property. "string" gives character, "integer" gives integer, "number" gives double, and "boolean" gives logical. Each type can be a string or a list that holds that one string. A pair of one of these types and "null", such as c("integer", "null"), gives the same column type. Any other type, no type, or a property that is not a list gives a list-column.

A cell of a character, integer, double, or logical property column holds the field of the parsed reply when the field is one value of the column type. A one-item JSON array counts as one value, and an empty string is kept. A double column takes any number. An integer column takes an integer, or a whole number from -2147483647 to 2147483647. Otherwise the cell is NA, and the call gives no warning. The output column keeps the parsed reply, with the value that did not fit. A cell of a list-column holds the field as parsed, or NULL when the field is absent or null. A reply that is not a JSON object, such as an array or a number, gives NA and NULL cells. So does the row of an input that failed.

A property name that is empty, NA, repeated, or equal to another column name aborts before the call checks for a running server. The other column names are input, output, response_id, input_tokens, total_output_tokens, and reasoning_output_tokens. A data frame with logprobs = TRUE, and the list and vector formats, do not check the names.

Details

This function calls lms_chat() once for each element of inputs. It raises rlmstudio_no_server itself, before the first call.

An rlmstudio_bad_response that lms_chat() raises for one input fails that input alone, unless it is an rlmstudio_model_mismatch. So does an rlmstudio_api_error with any status other than 401, 403, or 404, unless its status is 400 and its code is "model_not_found" or its server_message starts with "No models loaded". The batch goes on to the next input. Where the result is a list, or the output list-column that a schema gives, the element for that input holds the condition without its backtrace. An rlmstudio_bad_response for reply content that does not parse keeps that content in its content field. Where the result is text, the element holds NA. The result is text with format = "vector" when it returns a vector (simplify = TRUE, no schema, and logprobs = FALSE or the native route), and with a data frame whose replies are not parsed (no schema, or logprobs = TRUE). On the OpenResponses and OpenAI routes, the logprobs column holds NULL for a failed input. A reply with no readable answer text, such as one whose content is null, fails as an rlmstudio_bad_response in the same way. So does a status-200 body that does not parse as JSON, such as an HTML page from a proxy or an empty body. That input fails alone, and the other elements keep their replies. Use format = "list" to keep the conditions. The call gives one warning that names the count and the positions of the failed inputs. That warning shows even with quiet = TRUE.

An rlmstudio_api_error with status 401, 403, or 404 aborts the batch with that condition, and no request goes out after that input. Such a status comes from a fault that does not depend on the prompt, such as a token that the server refuses, so every later input fails in the same way. The condition carries a results field, as described in the "API failure" section below. No warning about failed inputs is given.

The batch aborts in the same way at an rlmstudio_api_error with status 400 and the code "model_not_found", at one with status 400 whose server_message starts with "No models loaded", and at an rlmstudio_model_mismatch. The "openai" and "openresponses" routes give these for a model name that the server cannot find, so every later input fails in the same way. They gave the "No models loaded" status 400 when no model was loaded. Within one batch, a reply id that the model-list lookup accepted is not looked up again. See the "API failure" and "Reply from another model" sections below.

A previous_response_id that the server does not hold gives status 400 with the code "invalid_value" on the native route and "previous_response_not_found" on the OpenResponses route. The batch does not abort at either one. Each such input fails alone, as described above, and the batch goes on to the next input.

An rlmstudio_no_server from lms_chat() also aborts the batch. Its results field holds the results so far, as described in the "Server not running" section below. An error of any other class aborts the batch unchanged.

Server not running

Functions that call the LM Studio REST API open a TCP connection to the hostname and port named in host before they send the request. A function that checks its own arguments does that first, so a bad model, job_id, input, inputs, messages, schema, ttl, previous_response_id, or batch_size, a bad type of list_models() or list_instances(), a bad TRUE or FALSE argument such as simplify, logprobs, quiet, force, or store, a store on the "openai" route of lms_chat() or lms_chat_batch(), a stream in the ... of a chat function, a logprobs in the ... of lms_chat_batch() or lms_chat_native(), a value in the ... of lms_chat_batch() that lms_chat() reads as an argument already given, an instructions or messages in the ... of lms_chat() or lms_chat_batch() on the route where lms_chat() sets it, or a name, id, or type string that is not valid in its encoding or is marked as bytes, aborts with an argument message and no condition class even when the server is down. A condition of class rlmstudio_no_server is raised when that connection cannot be opened. A refused connection raises it. So do an address the package cannot parse and a hostname that does not resolve. An address that neither accepts nor refuses the connection also raises it. That case waits for the operating system to give up, which can take a minute. If the hostname of host is localhost, in any letter case, or 127.0.0.1, start the server with lms_server_start() and the port of host. For the default host, that is lms_server_start(port = 1234). With no port, LM Studio uses the port that it used last. For any other host, start the server on the machine that host names, or give host the address that your server listens on. A host that does not parse gets this second hint too. The condition message gives the same hint for the host of the call.

The check reads the port and nothing else. Any process holding that port accepts the connection, so the condition is not raised even though no LM Studio server is there. The call then does not raise rlmstudio_no_server, and what it does depends on what answers. On a status-200 body that does not parse as JSON, the chat functions, lms_embed(), list_models(), list_instances(), lms_load(), lms_download(), lms_download_status(), and lms_unload_all() raise rlmstudio_bad_response. lms_unload() does not read the body, so it can report success. A body that parses as JSON but has another shape can come back unchanged with simplify = FALSE. With simplify = TRUE, the chat functions and lms_embed() raise rlmstudio_bad_response for it. list_models() and list_instances() raise it for a model list with another shape, and so do lms_unload_all() and lms_load() without force = TRUE, which read that list. lms_chat_openai() and lms_chat_openresponses() raise it for such a model list too, with either setting of simplify, through the model lookup that a reply from another model starts. lms_chat() raises it through them. lms_embed() raises it in the same way for the inputs of that request. lms_load(), lms_download(), and lms_download_status() raise it for a reply of their own with another shape, such as {}. A process that does not answer in HTTP gives an httr2_failure error. Use lms_server_ready() for the stronger test: it asks the host for a model list and reports TRUE only for a model list that list_models() can read.

lms_chat_batch() checks the server once before its first input, and lms_chat() checks it again for each input. If that check finds the server gone during the batch, the batch aborts with rlmstudio_no_server, and no request goes out after that. The condition then carries a results field, a list as long as inputs. Its elements before the lost input hold the values that format = "list" returns for those inputs. The element of the lost input and every element after it are NULL. The check before the first input adds no results field. A connection that fails after the check passes, such as a server that stops during a request, raises an httr2_failure error instead. That error aborts the batch and carries no results field.

lms_embed() checks the server before each request, and a request carries at most batch_size inputs. If a check after the first request, or the check of a model-list lookup, finds the server gone, the call aborts with rlmstudio_no_server. Once a request has succeeded, the condition carries a results field. With simplify = TRUE, results is a matrix with NA in each row whose embedding did not arrive. With simplify = FALSE, it is a list with one element per batch, with NULL in the element of the request that ended the call and in every element after it.

API failure

A condition of class rlmstudio_api_error is raised when a REST call returns a response that the wrapper treats as a failure. The condition carries a status field, which holds the HTTP response status as an integer. It also carries a code field. The field holds the string at error.code of the response body, such as "model_not_found". It is NULL when the body does not parse, when error is not a JSON object, or when its code is not one string. A server_message field holds the error text as the server sent it: the error string of the body, or else the string at error.message. It is NULL when the body does not parse or holds neither string.

On LM Studio 0.4.26+4 with no model loaded, /v1/chat/completions, /v1/responses, and /v1/embeddings answered a model name that the server could not find with status 400, no code, and a server_message that starts with "No models loaded".

lms_chat_batch() aborts on it when its status is 401, 403, or 404, when its status is 400 and its code is "model_not_found", and when its status is 400 and its server_message starts with "No models loaded", in that letter case. No request goes out after that input. The condition then carries a results field that follows the rule for a lost server in the "Server not running" section: its elements before the failed input hold the values that format = "list" returns for those inputs, and the element of the failed input and every element after it are NULL. For any other status, the element of the failed input holds the condition, or NA where the result is text, and the batch warns once and goes on. See the details of lms_chat_batch().

lms_embed() follows the same rule for each request, except for the code "model_not_found". A 401, 403, or 404 aborts the call, and so does a 400 whose server_message starts with "No models loaded". Once a request has succeeded, the condition carries a results field, a matrix or a list as the "Server not running" section describes. Any other status, a 400 with the code "model_not_found" included, fails the inputs of that request alone, and the call warns once and goes on. If every request fails, the call aborts with the first condition. See the details of lms_embed().

Malformed response

A condition of class rlmstudio_bad_response is raised when the server answers with a status the wrapper accepts and a body the wrapper cannot read. It is raised where a wrapper checks the body before it reshapes it, rather than indexing straight into whatever arrived. Eleven functions raise it for a body they cannot read: lms_embed(), lms_chat(), lms_chat_native(), lms_chat_openresponses(), lms_chat_openai(), list_models(), list_instances(), lms_load(), lms_download(), lms_download_status(), and lms_unload_all(). lms_chat() raises it through the chat function it calls. lms_unload_all() raises it through list_models(), and so does lms_load() unless force = TRUE. lms_chat_batch() raises it only as rlmstudio_model_mismatch, which the "Reply from another model" section of lms_chat_openai() describes.

All eleven raise it for a status-200 body that does not parse as JSON, such as an HTML page from a proxy, JSON text that stops part way, or an empty body. In the functions that take simplify, the body is parsed before simplify is read, so the condition is raised whatever simplify is. The body is parsed by its content and not by its Content-Type header, so valid JSON under text/plain is read as JSON. The body is read as JSON text and nothing else. A body whose text is a URL or the path of a file does not parse, and the package does not fetch the URL or read the file. The message says that the body did not parse as JSON and that something other than LM Studio may be answering on the host. It does not hold the body text.

The condition carries a status field, which holds the HTTP response status as an integer. Today the status is always 200: each of these functions reads the body only after a 200, and reports every other status as an rlmstudio_api_error instead.

Malformed model list

list_models() also raises it for a status-200 model list with the wrong shape. lms_unload_all() and lms_load() without force = TRUE raise it through list_models(). lms_chat_openai(), lms_chat_openresponses(), and lms_embed() apply these rules to the model list of their model lookup, as the "Reply from another model" section of lms_chat_openai() describes. A model list must follow four rules. Each field is read by its exact name, so a field named keyX does not stand in for key.

The body is a JSON object whose models field is an array. The array can be empty.
Each entry of models is a JSON object. Its type and key are strings, and its loaded_instances is an array.
The size_bytes of an entry is a number, or absent, or null. So is its max_context_length.
Each entry of loaded_instances is a JSON object whose id is a string with a character that is not whitespace.

The rules are checked before the type and loaded filters, so an entry that the filters drop can still raise the condition. The message names the field or entry that broke a rule. lms_server_ready() applies the same rules and returns FALSE for a body that breaks one.

list_instances() applies the four rules too, and three more. They cover only an entry whose type is in its type argument and that has at least one loaded instance. The display_name of such an entry is a string, or absent, or null. The remaining_ttl_seconds of each of its instances is a number, or absent, or null. The config of each of its instances is a JSON object, or absent, or null. list_models() and lms_server_ready() do not apply these three rules.

Malformed chat reply

lms_chat_native() and lms_chat_openresponses() raise it with simplify = TRUE when the reply holds no readable answer text. Both read the answer from the items of type "message" in the output array. They raise it when output is missing, empty, or not an array, or when an item in it is not a JSON object. They also raise it when no item has the type "message", as in a reply that holds only reasoning or a tool call. The text of a message must be one string. For lms_chat_native() that is the content of the item. For lms_chat_openresponses() it is the text of each part of type "output_text". For lms_chat_openresponses() only, the content of each message must be an array of JSON objects, and the messages together must hold at least one "output_text" part.

Apart from a body that does not parse as JSON, lms_chat_openai() raises it in four cases, all only with simplify = TRUE. The first case is a response whose choices field is missing, empty, or not an array, or whose first element is not a JSON object with a message object in it, so there is no reply to read. This case is raised with or without a schema, and with logprobs = TRUE as well. The second case is a reply that does not parse. A schema was given, logprobs = FALSE, and the reply content is not one string of valid JSON. The third case is reply content that is not one string, such as null, a missing content field, a number, or an array. A reply that holds only a tool call has null content. This case is raised without a schema, and with logprobs = TRUE with or without one. The fourth case is a cut-off reply. A schema was given, logprobs = FALSE, and the server reports the finish reason "length". This case is raised also when the reply content parses, because a reply that stops part way can parse to a wrong value, such as the first digit of a longer number. In the second, third, and fourth cases, if the server reports the finish reason "length", a length limit ended the reply. The limit is max_tokens or the context length of the model. The message then says so and names both.

With simplify = TRUE, lms_chat_native(), lms_chat_openresponses(), and lms_chat_openai() also raise it for a body that is a bare JSON value, such as 5, "s", or true. The message says that the response body is not a JSON object. A body of null gets the message about its missing output or choices field instead. lms_chat() can raise the condition through all three chat functions.

lms_chat_batch() does not abort on it, except on the subclass rlmstudio_model_mismatch, as the "Reply from another model" section of lms_chat_openai() says. The element of the failed input holds the condition, or NA where the result is text, and the batch warns once and goes on. See the details of lms_chat_batch().

A condition from lms_chat_openai() about its reply also carries two more fields. A condition from the model-list lookup does not, as the "Reply from another model" section of lms_chat_openai() says. The content field holds the reply content of the first choice, and the finish_reason field holds the finish reason of the first choice. Both are NULL for a response with no choices. In the third case, content holds the value that was read, which is NULL for null or missing content. For the second, third, and fourth cases, the message names the content field, so you can read what the model wrote without a second request. The other messages of the chat functions about the reply name simplify = FALSE, which returns the body unchanged, with one exception. A body that did not parse as JSON is checked before that argument is read, so its message points at the host instead. For such a body, the content and finish_reason fields of a condition from lms_chat_openai() are NULL. The messages of rlmstudio_model_mismatch and of the model-list lookup do not name simplify = FALSE, because the check runs with either setting of simplify.

Malformed logprobs

With simplify = TRUE and logprobs = TRUE, lms_chat_openresponses() also raises it for a logprobs value that breaks one of these rules. The logprobs value of each "output_text" part is checked. A logprobs value, token, logprob, or top_logprobs that is null or absent passes its rule. A null step or candidate breaks rule 2 or rule 6.

The value is an array.
Each step in the array is a JSON object.
The token of a step is a string.
The logprob of a step is a number.
The top_logprobs of a step is an array.
Each candidate in top_logprobs is a JSON object.
The token and logprob of each candidate follow rules 3 and 4.

The parts are checked in order, then the steps of a part, then the candidates of a step, one at a time. Within a step, rules 3, 4, and 5 come before the candidates. The message names the first broken rule that this order reaches. These checks run only after the text of every "output_text" part is read, so a reply that also has a bad text in any part gets the text message. Parts of other types, such as a refusal, are not checked, and with logprobs = FALSE no part is checked. Fields are read by their exact names, so a field whose name only starts with the one asked for, such as tokenX, reads as absent and gives NA in the data frame.

Cut-off reply

A warning of class rlmstudio_reply_cut_off is given when a length limit ended a reply that lms_chat_openai() returns as text. The server reports this with the finish reason "length". The limit is max_tokens or the context length of the model, and the reply does not say which one. The message names both. The warning is given with simplify = TRUE for reply content that is one string, in three settings: no schema, logprobs = TRUE, and a schema with logprobs = TRUE. The call returns what the same reply returns without the cut-off. A schema reply with logprobs = FALSE raises rlmstudio_bad_response instead, as the "Malformed chat reply" section says. The warning and that abort read the finish reason of the first choice, which is the choice that the call reads. No other element of choices is read. With simplify = FALSE, the call returns the body with every choice and no warning.

lms_chat() gives the warning through lms_chat_openai() with api_type = "openai". lms_chat_batch() gives one warning of this class for the whole batch in place of one for each input. It names the count and the positions of the cut-off inputs, and each of those elements keeps its reply. It comes after the warning about failed inputs. A batch that aborts with a results field gives this warning before the abort. It names the cut-off inputs whose replies the results field of the abort holds. The native and OpenResponses routes give no such warning.

The warning shows whatever quiet and the rlmstudio.quiet option say, because it is the only sign that an answer is not complete.

Reply from another model

lms_chat_openai(), lms_chat_openresponses(), and lms_embed() read the model field of each status-200 reply, with either setting of simplify. On LM Studio 0.4.25+1 with one chat model loaded, both chat routes answered a model name that the server could not find with status 200 and a reply from the loaded model. With two chat models loaded, they answered with status 400 and the code "model_not_found", as the "API failure" section describes. On LM Studio 0.4.26+4 with one embedding model loaded, /v1/embeddings answered a name that it could not find with status 200 and vectors from the loaded embedding model. The model field names the loaded instance that answered. Its id can differ from the key that the call asked for.

If the model field equals the name that the call sent, the reply is accepted. If it differs, the call sends one request for the model list to api/v1/models on the same host, with the token of the call, and prints no message. The reply is accepted if a model whose key equals the asked name in any letter case has a loaded instance whose id is the model field. Otherwise the call aborts with a condition of class rlmstudio_model_mismatch. That condition is also an rlmstudio_bad_response, and its status is 200. Its model field holds the asked name, and its reply_model field holds the model field of the reply. The message names both. A condition from lms_chat_openai() also carries the content and finish_reason fields, both NULL.

Within one call of lms_chat_batch() or lms_embed(), a reply whose model field a lookup of that call already accepted for the asked name and the same host is accepted with no other request. Each such call, and each call of lms_chat_openai() or lms_chat_openresponses() outside a batch, starts with no accepted ids.

The check runs before the reply is read, so it also runs with logprobs = TRUE, with a schema on lms_chat_openai(), and for a reply with no answer text. In lms_embed(), it runs before the vectors are read. A reply is not checked if its body is not a JSON object, if it has no model field, or if its model is not one string or holds only whitespace. A body that is not a JSON object is returned with simplify = FALSE, and with simplify = TRUE it raises the error that the "Malformed chat reply" section describes, or for lms_embed() the "Malformed embeddings" section. If the instance that answered is unloaded before the model-list request, the call aborts, also when that instance belongs to the asked model.

If the model-list request fails, the check raises the condition of that failure. The message opens with the label of the function, followed by "because the model-list lookup failed". A status other than 200 raises rlmstudio_api_error. A body that does not parse as JSON, or that breaks a rule of a model list in the "Malformed model list" section, raises rlmstudio_bad_response. A server that the port check before the request cannot reach raises rlmstudio_no_server. A condition from the lookup does not carry the content and finish_reason fields, also when lms_chat_openai() raises it.

lms_chat() runs the check through lms_chat_openai() and lms_chat_openresponses(). lms_chat_native() does not check the reply. On LM Studio 0.4.25+1, /api/v1/chat answered a name that it could not find with status 404, which raises rlmstudio_api_error.

lms_chat_batch() aborts at the first input that raises rlmstudio_model_mismatch, on the "openai" and "openresponses" routes. It also aborts at an rlmstudio_api_error with status 400 whose code is "model_not_found", and at one with status 400 whose server_message starts with "No models loaded". No request goes out after that input. In each case, the condition carries a results field that follows the rule for a lost server in the "Server not running" section.

lms_embed() aborts at the first request that raises rlmstudio_model_mismatch, and no request goes out after it. A failed model-list lookup that raises rlmstudio_no_server, or an rlmstudio_api_error that would end the call as an embeddings failure, such as one with status 401, 403, or 404, ends the call in the same way. Once a request has succeeded, the condition carries a results field, a matrix or a list as the "Server not running" section describes. Any other rlmstudio_api_error or rlmstudio_bad_response from the lookup fails the inputs of that request alone, as the "API failure" section describes for a failed request. An error of any other class, such as an httr2_failure from a server that stops during the lookup, aborts the call unchanged.