Load a model via REST API

Usage

lms_load(
  model,
  context_length = NULL,
  eval_batch_size = NULL,
  flash_attention = NULL,
  num_experts = NULL,
  offload_kv_cache_to_gpu = NULL,
  echo_load_config = FALSE,
  force = FALSE,
  host = "http://localhost:1234",
  ...,
  token = NULL
)

Arguments

model: Character. Unique identifier for the model to load. Must be one name, given as a single string. The string must be valid in its declared encoding and not marked "bytes". A class, names, and the S4 bit are removed before the name is sent.
context_length: Integer. Maximum number of tokens that the model will consider. With force = FALSE, a value above the maximum in the model list gives a warning. See the "Long prompts" section.
eval_batch_size: Integer. Number of input tokens to process together in a single batch during evaluation.
flash_attention: TRUE, FALSE, or NULL. Whether to optimize attention computation. NULL, the default, leaves the field out of the request. Any other value, NA and "true" included, aborts before the check for a running server.
num_experts: Integer. Number of experts to use during inference for MoE models.
offload_kv_cache_to_gpu: TRUE, FALSE, or NULL. Whether KV cache is offloaded to GPU memory. NULL, the default, leaves the field out of the request. Any other value, NA and "true" included, aborts before the check for a running server.
echo_load_config: TRUE or FALSE. If TRUE, echoes the final load configuration in the response. Any other value, NULL and NA included, aborts before the check for a running server.
force: TRUE or FALSE. If TRUE, bypasses the check for currently loaded models and requests a new instance from the server. Note that this does not overwrite or replace the existing model; it loads a second concurrent instance into VRAM. Defaults to FALSE. Any other value, NULL and NA included, aborts before the check for a running server.
host: Character. The host address of the local server. Defaults to "http://localhost:1234".
...: Additional arguments passed to the API request body (useful for future API parameters).
token: Character or NULL. An API token for a server that requires authentication. NULL reads the rlmstudio.token option and then the RLMSTUDIO_API_TOKEN environment variable. See rlmstudio_token.

Value

Invisibly returns a character string of the loaded model identifier upon success, also when the model was already loaded. It is a plain string, with no class, names, or S4 bit. If echo_load_config = TRUE and this call loads the model, it instead invisibly returns a list containing the model's detailed load configuration.

Long prompts

A prompt longer than the context length of the loaded model fails. On LM Studio 0.4.25+1, with google/gemma-3-1b loaded at a context_length of 512, the /v1/responses and /api/v1/chat routes answered a longer prompt with status 500. The /v1/chat/completions route answered it with status 400. Each message began "The number of tokens to keep from the initial prompt is greater than the context length". The chat functions raise such a reply as an rlmstudio_api_error, and the message holds the server text. lms_chat_batch() fails that input alone and goes on to the next input. With format = "list", the element of that input holds the condition. With format = "vector", it holds NA.

To fit a longer prompt, load the model with a larger context_length in lms_load(). When the server reports it, the max_context_length column of list_models(detailed = TRUE) gives the largest context length that the model list reports for each model. If context_length is larger, the server loads the model with the asked value and gives no message. On LM Studio 0.4.25+1, it loaded google/gemma-3-1b at 65536 tokens, above its maximum of 32768. So lms_load() gives a warning of class rlmstudio_context_above_max that names both numbers, and then sends the load. The rlmstudio.quiet option does not hide it. The warning needs the model list, so it comes only with force = FALSE, and only when the list has a maximum for the model and the model is not loaded yet.

On LM Studio 0.4.25+1, the load endpoint answered a rope_frequency_scale field in ... with status 400 and the code "unrecognized_keys". So lms_load() cannot set RoPE scaling through that field. RoPE scaling stretches the position encoding of a model past its trained length.

To see how many tokens a prompt took, use lms_chat_batch(format = "data.frame"). Its input_tokens column holds the prompt token count that the server reports for each reply, on every route.

Server not running

Functions that call the LM Studio REST API open a TCP connection to the hostname and port named in host before they send the request. A function that checks its own arguments does that first, so a bad model, job_id, input, inputs, messages, schema, ttl, previous_response_id, or batch_size, a bad type of list_models() or list_instances(), a bad TRUE or FALSE argument such as simplify, logprobs, quiet, force, or store, a store on the "openai" route of lms_chat() or lms_chat_batch(), a stream in the ... of a chat function, a logprobs in the ... of lms_chat_batch() or lms_chat_native(), a value in the ... of lms_chat_batch() that lms_chat() reads as an argument already given, an instructions or messages in the ... of lms_chat() or lms_chat_batch() on the route where lms_chat() sets it, or a name, id, or type string that is not valid in its encoding or is marked as bytes, aborts with an argument message and no condition class even when the server is down. A condition of class rlmstudio_no_server is raised when that connection cannot be opened. A refused connection raises it. So do an address the package cannot parse and a hostname that does not resolve. An address that neither accepts nor refuses the connection also raises it. That case waits for the operating system to give up, which can take a minute. Start the server with lms_server_start(), or give host the address that your server listens on.

The check reads the port and nothing else. Any process holding that port accepts the connection, so the condition is not raised even though no LM Studio server is there. The call then does not raise rlmstudio_no_server, and what it does depends on what answers. On a status-200 body that does not parse as JSON, the chat functions, lms_embed(), list_models(), list_instances(), lms_load(), lms_download(), lms_download_status(), and lms_unload_all() raise rlmstudio_bad_response. lms_unload() does not read the body, so it can report success. A body that parses as JSON but has another shape can come back unchanged with simplify = FALSE. With simplify = TRUE, the chat functions and lms_embed() raise rlmstudio_bad_response for it. list_models() and list_instances() raise it for a model list with another shape, and so do lms_unload_all() and lms_load() without force = TRUE, which read that list. lms_chat_openai() and lms_chat_openresponses() raise it for such a model list too, with either setting of simplify, through the model lookup that a reply from another model starts. lms_chat() raises it through them. lms_load(), lms_download(), and lms_download_status() raise it for a reply of their own with another shape, such as {}. A process that does not answer in HTTP gives an httr2_failure error. Use lms_server_ready() for the stronger test: it asks the host for a model list and reports TRUE only for a model list that list_models() can read.

lms_chat_batch() checks the server once before its first input, and lms_chat() checks it again for each input. If that check finds the server gone during the batch, the batch aborts with rlmstudio_no_server, and no request goes out after that. The condition then carries a results field, a list as long as inputs. Its elements before the lost input hold the values that format = "list" returns for those inputs. The element of the lost input and every element after it are NULL. The check before the first input adds no results field. A connection that fails after the check passes, such as a server that stops during a request, raises an httr2_failure error instead. That error aborts the batch and carries no results field.

lms_embed() checks the server before each request, and a request carries at most batch_size inputs. If a check after the first request finds the server gone, the call aborts with rlmstudio_no_server. Once a request has succeeded, the condition carries a results field. With simplify = TRUE, results is a matrix with NA in each row whose embedding did not arrive. With simplify = FALSE, it is a list with one element per batch, with NULL in the element of the request that ended the call and in every element after it.

API failure

A condition of class rlmstudio_api_error is raised when a REST call returns a response that the wrapper treats as a failure. The condition carries a status field, which holds the HTTP response status as an integer. It also carries a code field. The field holds the string at error.code of the response body, such as "model_not_found". It is NULL when the body does not parse, when error is not a JSON object, or when its code is not one string.

lms_chat_batch() aborts on it when its status is 401, 403, or 404, and when its status is 400 and its code is "model_not_found". No request goes out after that input. The condition then carries a results field that follows the rule for a lost server in the "Server not running" section: its elements before the failed input hold the values that format = "list" returns for those inputs, and the element of the failed input and every element after it are NULL. For any other status, the element of the failed input holds the condition, or NA where the result is text, and the batch warns once and goes on. See the details of lms_chat_batch().

lms_embed() follows the same rule for each request. A 401, 403, or 404 aborts the call, and once a request has succeeded the condition carries a results field, a matrix or a list as the "Server not running" section describes. Any other status fails the inputs of that request alone, and the call warns once and goes on. If every request fails, the call aborts with the first condition. See the details of lms_embed().

Malformed response

A condition of class rlmstudio_bad_response is raised when the server answers with a status the wrapper accepts and a body the wrapper cannot read. It is raised where a wrapper checks the body before it reshapes it, rather than indexing straight into whatever arrived. Eleven functions raise it for a body they cannot read: lms_embed(), lms_chat(), lms_chat_native(), lms_chat_openresponses(), lms_chat_openai(), list_models(), list_instances(), lms_load(), lms_download(), lms_download_status(), and lms_unload_all(). lms_chat() raises it through the chat function it calls. lms_unload_all() raises it through list_models(), and so does lms_load() unless force = TRUE. lms_chat_batch() raises it only as rlmstudio_model_mismatch, which the "Reply from another model" section of lms_chat_openai() describes.

All eleven raise it for a status-200 body that does not parse as JSON, such as an HTML page from a proxy, JSON text that stops part way, or an empty body. In the functions that take simplify, the body is parsed before simplify is read, so the condition is raised whatever simplify is. The body is parsed by its content and not by its Content-Type header, so valid JSON under text/plain is read as JSON. The body is read as JSON text and nothing else. A body whose text is a URL or the path of a file does not parse, and the package does not fetch the URL or read the file. The message says that the body did not parse as JSON and that something other than LM Studio may be answering on the host. It does not hold the body text.

The condition carries a status field, which holds the HTTP response status as an integer. Today the status is always 200: each of these functions reads the body only after a 200, and reports every other status as an rlmstudio_api_error instead.

Malformed model list

list_models() also raises it for a status-200 model list with the wrong shape. lms_unload_all() and lms_load() without force = TRUE raise it through list_models(). lms_chat_openai() and lms_chat_openresponses() apply these rules to the model list of their model lookup, as the "Reply from another model" section of lms_chat_openai() describes. A model list must follow four rules. Each field is read by its exact name, so a field named keyX does not stand in for key.

The body is a JSON object whose models field is an array. The array can be empty.
Each entry of models is a JSON object. Its type and key are strings, and its loaded_instances is an array.
The size_bytes of an entry is a number, or absent, or null. So is its max_context_length.
Each entry of loaded_instances is a JSON object whose id is a string with a character that is not whitespace.

The rules are checked before the type and loaded filters, so an entry that the filters drop can still raise the condition. The message names the field or entry that broke a rule. lms_server_ready() applies the same rules and returns FALSE for a body that breaks one.

list_instances() applies the four rules too, and two more. They cover only an entry whose type is in its type argument and that has at least one loaded instance. The display_name of such an entry is a string, or absent, or null. The config of each of its instances is a JSON object, or absent, or null. list_models() and lms_server_ready() do not apply these two rules.

Malformed load or download reply

lms_load(), lms_download(), and lms_download_status() also raise it for a status-200 reply of their own with the wrong shape. Each reply must follow the rule of its function. Each field is read by its exact name. The rules check the type of a field and not its value, with four exceptions. The status of a load reply must be "loaded". A download reply whose status is "already_downloaded" needs no job_id. A download reply whose status is "failed" always aborts. The job_id of any other download reply must hold a character that is not whitespace.

A reply of lms_load() is a JSON object whose status is the string "loaded". With echo_load_config = TRUE, its load_config is also a JSON object.
A reply of lms_download() is a JSON object whose status is a string other than "failed". If the status is not "already_downloaded", its job_id is a string with a character that is not whitespace. For a "failed" status, the message says that LM Studio reports that the download failed. It names the reply's job_id if that is a string with a character that is not whitespace.
A reply of lms_download_status() is a JSON object whose job_id and status are strings. Its total_size_bytes, downloaded_bytes, and bytes_per_second are each a number, or absent, or null.

For the other faults, the message names the field that broke the rule, or it says that the body is not a JSON object. It also says that something other than LM Studio may be answering on the host.

Examples

if (FALSE) { # \dontrun{
lms_server_start()
lms_download("google/gemma-3-1b")

# Load a model with default settings
lms_load("google/gemma-3-1b")

# Load a model with custom context length and flash attention enabled
lms_load("google/gemma-3-1b", context_length = 8192, flash_attention = TRUE)
} # }